Wrong Hardware for the Job . .... storage managementâis king, and virtualization and cloud are its courtiers. Foundati
White Paper Optimizing Data Management Through Efficient Storage Infrastructures By Mark Peters, Senior Analyst
May 2014
This ESG White Paper was commissioned by IBM and is distributed under license from ESG. © 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
2
Contents Embracing Data Management ...................................................................................................................... 3 The Data Conundrum ............................................................................................................................................... 3
Foundational Storage Challenges ................................................................................................................. 3 Wrong Hardware for the Job .................................................................................................................................... 5 Maintenance versus New Projects ........................................................................................................................... 5 Infrastructures Are Too Complex ............................................................................................................................. 5
The Data Infrastructure Opportunity ........................................................................................................... 6 An Efficient Infrastructure Matters .......................................................................................................................... 6 Automated Tools and a Range of Products Is Key .................................................................................................... 6 Improved ROI with the Right Infrastructure............................................................................................................. 6
The Alternate IBM Approach ........................................................................................................................ 8 Less Pain, More Gain ................................................................................................................................................ 8 Broad Product Portfolio ............................................................................................................................................ 8 Recent IBM Storage Announcements Up the Data Management Ante ................................................................... 9
The Bigger Truth ......................................................................................................................................... 11
All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
3
Embracing Data Management While the issue of provisioning ever-increasing volumes of storage is nothing new, the arrival—and nascent domination—of “big data” is forcing IT organizations specifically, and businesses generally, to think differently about data. Organizations are increasingly looking for ways to consider data in terms of the business value of its information, instead of as simple bits and bytes—what it can mean, instead of what it is. Furthermore, keeping in line with the dictum that nature abhors a vacuum, organizations are now shifting their focus from holding back the data deluge to accepting and even encouraging it as they develop ever-more-sophisticated and mission-critical analytics capabilities. This new reality dictates that data management is the key business requisite and, therefore, that managing storage to qualitatively optimize the data infrastructure—rather than managing storage merely as a quantitative and reactive service—is of the utmost importance. This is a dramatic departure from the traditional approach, and it requires organizations to change not only their IT storage infrastructures, but also their hearts and minds. We all know how disruptive “cultural changes” can be to organizational behavior. The reality is, however, that tremendous value is imprisoned in all this burgeoning organizational data, and it’s waiting to be refined to help businesses improve their outcomes significantly by storing and accessing it in a timely, flexible, and affordable manner.
The Data Conundrum Mining for that value has created a pressurized environment in which tremendous technology advancements and user expectations are driving each other in a circular and increasing manner. The more data is collected via new sources and methods, the higher user expectations rise, so additional data is wanted and needed. Plus, it’s fairly inexpensive to collect additional data these days, so more is collected and kept. However, data retention (that is, storage infrastructure) is not getting cheaper at the same rate that more data is being collected, so while data volumes are growing, the budgets required to support these escalating volumes must also grow…and the availability of increased IT budgets is far from uniform or sufficient. The outcome of this vicious-infrastructuralcircle is that organizations categorically have to find ways to be more efficient with the resources they have or change the way they deploy and manage them. That’s really what the whole point is: The IT world is having to adopt a different way to think; it is about data specifically rather than storage generically. Irrespective of this academic, semantic, and management shift, traditional infrastructures can no longer keep up. As a result, un-sharable storage silos, error-prone manual labor processes, and low storage availability are necessarily headed for the proverbial dustbin of IT history. Amidst all the churn, data centers as we know them are moving outside the four walls that have housed them for so long as they become increasingly virtualized, distributed, and inextricably linked to on-demand services. In our rapidly maturing, software-defined world, data management—not storage management—is king, and virtualization and cloud are its courtiers.
Foundational Storage Challenges Savvy IT organizations that are able to secure funding for key projects can concentrate on dealing with their biggest overall storage challenges, which respondents to ESG research indicate are:
Rapid growth and management of unstructured data (40%); this is also the lead response—at 15%—when respondents were asked to identify their primary storage challenge.
Data protection (39%).
Hardware costs (39%).1
The full details are shown in Figure 1, but the point is to emphasize that data volume is indeed the major pragmatic challenge, significantly ahead of other primary storage challenges.
1
Source: ESG Research Report, 2012 Storage Market Survey, November 2012.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
4
Figure 1. Storage Challenges
In general, what would you say are your organization’s biggest challenges in terms of its storage environment? Which would you characterize as the primary storage challenge for your organization? (Percent of respondents, N=418) Rapid growth and management of unstructured data
15%
40%
11%
Data protection (e.g., backup/recovery, etc.)
39%
10%
Hardware costs
39%
7%
Running out of physical space Need to support growing virtual server environments
25%
5%
25%
4%
Data migration
25%
Staff costs
5%
Management, optimization & automation of data placement
5%
20%
6%
Lack of skilled staff resources Discovery, analysis and reporting of storage usage
19%
5%
Device management
3%
File system expansion
3%
Poor performance (throughput)
4%
Poor performance (I/Os)
4%
17% 17% 15% 15% 14%
2%
Lengthy implementation time
12%
3%
Poor storage hardware utilization Difficulty supporting desktop virtualization environment
1% 2%
Lengthy storage provisioning time 0%
All storage challenges
17%
3%
Power and cooling costs
Primary storage challenge
19%
11% 9% 9% 10%
20%
30%
40%
50%
Source: Enterprise Strategy Group, 2014.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
5
Together, these challenges may appear to be “vanilla” storage issues, but they give clear indicators to the need for better data management. The better IT groups understand data needs and can manage to optimally address those needs, then the better they can optimize storage to efficiently manage all this data without runaway hardware costs, physical space limitations, and increased OpEx costs.
Wrong Hardware for the Job Not surprisingly, this enlightened, contemporary view of data and storage does not resonate with everyone in IT— some still believe that throwing more storage at problems will either resolve them, or at least keep them at bay. This is sub-optimal in any case, but using the wrong kind of storage for problems only compounds the problems and results in resource waste. For example, in a high-performance environment, well-meaning IT professionals may end up keeping low priority data on high-performance devices simply because they don’t have the ability to automatically migrate it to appropriate systems as its usage becomes less urgent or frequent. Plenty of empirical and research-based evidence shows some companies are purchasing additional capacity for storage arrays even though they are not using anywhere near the capacity they already have. In fact, according to IBM’s own research, companies are purchasing 24% additional capacity per year, yet are often using less than half of the capacity they already have. ESG’s data is a little more generous, but still finds that the self-attested capacity utilization rates remain stubbornly south of 60%.2
Maintenance versus New Projects An IT group cannot focus on getting value and insights from its organizational data, or meet competitive business demands, if it is spending an excessive proportion of its budget and staff time on managing existing storage infrastructures. Though the promise of cutting-edge projects is alluring, the financial resources required to implement them are often not built into many IT budgets, which remain notoriously parsimonious. For example, ESG’s 2014 IT Spending Intentions Survey found that 62% of 2014 IT budgets will be spent on maintaining existing infrastructures, while only 38% will be devoted to developing new technology projects/purchases. 3 If you look deeper, however, a leavening factor emerges. Specifically, when IT purchase profiles are broken down, the results indicate that “leading edge” IT organizations anticipate spending 45% of their 2014 budgets on new technology projects, while their “average” and “laggard” counterparts expect to apportion 37% and 28% respectively.
Infrastructures Are Too Complex Because many infrastructures become (through the sporadic and unplanned acquisition of disparate, nonintegrated systems) overly complex, it is all too easy to spend too much money on them—it’s like “Ponzi storage” —and, hence, to utilize the storage elements poorly. For example, it can be hard or impossible to complete necessary migrations or to perform the constant tuning that is a requisite to maintaining high performance. In fact, IT organizations sometimes buy the “wrong” products with full cognizance of doing so because they know that if they purchase something else or try to integrate a heterogeneous mix, they will also be buying themselves months of tedious operational problems. Too often, it is easier to continue pouring money into the “least worst” options rather than bite the bullet and make things right for the longer term. That is what has often been done with storage, apropos of Winston Churchill’s well-known quote about democracy: “Democracy is the worst form of government ever invented…except for all the others.” There are simply too many heterogeneous storage systems being thrown together without enough automated features and sophisticated tools to support data management. This gets back to having the right data on the right device at the right time, which has been discussed forever, but in practice, happens rarely because it’s difficult to do if IT is just managing storage.
2 3
Ibid Source: ESG Research Report, 2014 Spending Intentions Survey, February 2014.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
6
The Data Infrastructure Opportunity An Efficient Infrastructure Matters Storage alone provides a plain usefulness to business, but the right storage infrastructure matters, and an efficient storage infrastructure that optimizes management and delivers on the potential of big data analytics matters even more. The best approach to efficient data management is implementing an advanced, data-driven infrastructure that includes such capabilities as automated tiering, automated migration, and heterogeneous virtualization on the high end, and somewhat less sophisticated products featuring compression and deduplication features on the low end. This kind of unified systems approach is key to success in the new data-driven world, and recent joint research conducted by IBM and the University of Oxford verifies that claim.4 The study found that 63% of respondents reported that the use of big data and analytics is creating a competitive advantage for their organizations. This is similar to the finding for 37% of respondents in IBM’s 2010 New Intelligent Enterprise Global Executive Study and Research Collaboration, and represents a 70% increase in two years. Put colloquially, the good use of data is measurably good for business. The kind of data-driven infrastructures cited in these two studies implies a balanced purchasing pattern that calls for the judicious acquisition of new storage hardware along with the optimization of existing, legacy equipment. In that fashion, IT can make sure to buy the right amount and type of new hardware and software products. Using storage resources cost-efficiently is essential. If advanced tools—such as automated tiering and migration—are not used to manage data, expensive storage resources will be wasted because the use of less expensive (and more appropriate) resources is not maximized.
Automated Tools and a Range of Products Is Key Automation is a necessity in this data/storage management mix, for the simple reason that human beings cannot begin to match the speed and volume-of-change requirements that the applications, data, and agile businesses shaping today’s global market place demand. Additionally, automation both increases the speed and reduces the disruption of optimization efforts, such as data migration, which also reduces the staff effort needed; hence, it reduces the overall cost of the effort. The same is true with such things as meeting SLAs, which have become integral elements of many computing environments. Moreover, automation is the only vehicle that can consistently and realistically drive the flexibility, scalability, and availability requirements necessary to ensure a constant, reliable, and high level of performance in mission-critical environments. By definition, to be efficient as well as effective, automation demands that you have an infrastructure that includes the right tools, e.g., tiering (which IBM has with its Easy Tier) and migration (which is a part of IBM’s SVC virtualization product). Despite its importance, however, automation is not a standalone answer to every data management challenge. In order to maximize their IT assets, users require a portfolio of products because no individual offering can be simultaneously cheap, deep, full-functioned, and offer variable performance.
Improved ROI with the Right Infrastructure How can you manage data and unleash its power “at the speed of business” with a storage infrastructure built for a different paradigm? Trying to configure and manage multitudes of different infrastructure components and processes to provide for strict availability, scalability, and performance is complex, and only gets worse as data volumes grow. While there’s assuredly untold value trapped in big data—business’s crown jewels—it can only be released in a business-optimized manner if the data is stored, accessed, and analyzed appropriately in a timely, flexible, and affordable manner. To do this, an efficient storage infrastructure is key, not only because of the necessity to drive down costs, but also to drive up return on investment (ROI).
4
Source: IBM Global Business Services and Said Business School, University of Oxford Report, Analytics: The real-world use of big data—How innovative enterprises extract value from uncertain data, 2012.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
7
As the data in Figure 2 shows, ESG’s 2014 IT Spending Intentions Survey revealed that ROI is the most-cited consideration for justifying IT investments (38%), while reducing OpEx and business process improvement were very close seconds with 37% of responses each.5 If users can control costs and utilize their storage infrastructures efficiently, they can then invest in projects with positive business outcomes rather than just “keep the lights on.” Figure 2. Top Five Most Important Considerations in Justifying IT Investments Over the Next 12 Months Which of the following considerations do you believe will be most important in justifying IT investments to your organization’s business management team over the next 12 months? (Percent of respondents, N=562, three responses accepted) Return on investment
38%
Reduction in operational expenditures
37%
Business process improvement
37%
Improved security/risk management
31%
Reduction in capital expenditures
26% 0%
5%
10%
15%
20%
25%
30%
35%
40%
Source: Enterprise Strategy Group, 2014.
The “right” infrastructure—that is, optimal from both an operational and financial perspective, with automated flexibility to respond to changing data needs—can contribute to successful business results by:
5
Increasing performance, enabling faster analytics, and speeding up “time to business insights.”
Storing more data in less/optimized space (that is, better utilization of data-appropriate media over time), thereby reducing energy costs and freeing up capital expense budgets and saving on operational costs.
Reducing complexity, driving down operational expenses, and enabling workers to focus on strategic priorities.
Source: ESG Research Report, 2014 Spending Intentions Survey, February 2014.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
8
The Alternate IBM Approach Less Pain, More Gain IBM espouses the separation of data management from physical storage via a virtualization layer of (what is increasingly referred to as) software-defined storage. This includes the automated, non-disruptive movement or placement of data to where it provides the most value—whether reducing cost or improving business results— whether that be high-capacity disk, flash, or anything else. This approach essentially equals optimally self-managing storage with less cost and less effort. Adhering to these tenets contributes significantly to fixing the problem of optimizing data management. Although it is important to get rid of older pain, it’s far more compelling, in the process, to introduce new gains, including:
Improvements in the economics of storage with the implementation of consistent efficiency capabilities, which allow data to be stored at the lowest possible cost without sacrificing the required performance over time.
Simplified and standardized management to reduce operational costs.
Reduced hardware costs.
Insights from data to improve business success and an organization’s competitive stance in the market.
Broad Product Portfolio So, the future of bits and bytes is actually all about data, not about storage: IT organizations shouldn’t have to be worried about how they provision, move, or manually tune their systems—that should be automated. That being said, organizations still need fundamental elements: hardware, software, and systems. Indeed, if IT organizations are to start from the data side, they had better have a broad portfolio of products, which will be different depending on the environment in which the business operates and the workloads that are being supported. Some of the key attributes of, and products within, IBM’s data management foundations are: Storage virtualization: This capability is a critical prerequisite component for optimizing data management inasmuch as it is the foundational element that enables the creation of storage pools that can be accessed as needed for changing workloads, budgets, business needs, and so on. The best virtualization will embrace and control a variety of heterogeneous storage devices, even those from a range of vendors. This maximizes the range of tools for each data job. No hammer looking for homogeneous nails here! Uptime can be another major benefit of a good virtualized storage environment, and it is achieved in two main ways: through continuous operations, or via non-disruptive migrations. With continuous operations, as new applications are rolled out, and new storage capacity or a new storage tier is brought online—there need not be any downtime. In the case of non-disruptive migrations, as data is migrated to new storage systems or technologies—such as flash storage—downtime can again be avoided. IBM’s rich portfolio of software-defined storage software includes such capabilities as Real-time Compression and Easy Tier migration, which are compatible with multi-vendor systems; it can also enable common data optimization functions, such as thin provisioning, with both automation and ease of use.
Real-time Compression can allow users to effectively store more active, primary data (up to 5X is not atypical) in a given amount of physical disk space. This provides key benefits, including reduced physical storage requirements; improved efficiency, which leads to reduced storage costs; or a combination of increased capacity and decreased costs.
Easy Tier enables the automatic migration of differently-accessed data elements to the appropriate “tier” of storage: for example, “hot” workloads might be migrated to flash.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
9
Overall storage virtualization benefits can include up to a 50% reduction in the overall cost per terabyte via this tiered storage optimization; perhaps a 100% improvement in capacity utilization as a result of large pool management; and the ability to perform near-instant migrations.
Recent IBM Storage Announcements Up the Data Management Ante IBM has continued to make improvements throughout both its storage and its data management portfolio. This is a brief summary of some of the latest new features: Storwize V7000 Highlights: Up to 2x greater performance (which could be 10x with compression); more than 4 petabytes maximum capacity; additional connectivity options; and an industry-leading hardware compression acceleration that uses Intel’s QuickAssist technology Easy Tier 3: Supports flash plus two disk tiers with Storwize V7000 automatically moving data between tiers; deploys flash and enterprise disk for performance; and provides automatic storage pool balancing Benefits: Greater performance for big data and analytics workloads; reduction in acquisition cost by as much as half; simple management; reuse of existing assets; and seamless scalability SAN Volume Controller (SVC) Highlights: New hardware platform delivers up to 2x the prior performance and the same hardware compression technology as Storwize V7000; and now supports up to 48 flash drives per pair of engines for low-cost, high-performance options Benefits: Can help to transform data management economics by dramatically reducing or eliminating storage acquisitions; virtualizes larger configurations for greater consolidation advantages; uses two HDD tiers for more performance/cost options; and supports larger storage and server configurations for operational cost savings IBM FlashSystem Highlights: FlashSystem 840 includes more flexibility with additional data-protected capacity offerings and uses 2TB, 4TB, 6TB, and 8TB options Benefits: Makes flash and data virtualization more economical for midmarket and growth markets; provides lower entry points; and increases connectivity and tiering options XIV Storage System Highlights: For its Cloud Service Provider users (or on-premises/private clouds), a new pay-per-use flexibility caters to varying tenant requirements yet retains extreme efficiency Benefits: Greater enablement and economics for service providers; enhanced security for public, private, and hybrid clouds IBM DS8870 Highlights: Next-generation hybrid-flash and all-flash DS8870 systems with new ultra-highperformance, self-encrypting flash enclosures Benefits: Boosts performance of database applications by up to 3.2x compared with the prior system; and Easy Tier updates provide new flash options that enable flexibility to meet current and future storage needs
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
10
Large Scale Deployments: In IBM’s view, data management is particularly beneficial in new, large-scale deployments, which are not bound to legacy storage devices. In these environments, grid scale storage is optimal for clouds because it requires almost no effort to manage and maintain high performance and provide enhanced utilization as capacity scales. Large-scale deployment users realize three key benefits: 1. Self-tuning to meet variable quality of service without the attention of system administrators 2. High efficiency based on thin provisioning and incremental snapshots 3. Low latency and high performance created by flash caching IBM’s XIV disk storage with its grid architecture works well in large scale deployments because it integrates highspeed flash storage, and supports multiple data copies without overhead—a combination that is very well suited, for example, to SAP environments. This storage system provides high service levels for dynamic, mixed workloads in addition to efficient deployments that can support rich OpenStack, hypervisor, and other integrations. Business-critical Operations: In these environments, data management is essential, which implies that data access is universally available anywhere, anytime to properly credentialed personnel. It also implies the presence of strong security that ensures data is properly protected at all times, and performance that meets the needs of the most demanding applications. “Business critical” translates into implementing cutting-edge IT systems, such as continuous operations, with highly available and resilient architectures that likely include multisite disaster recovery capabilities. It typically requires service level agreements (SLAs) with high performance across a spectrum of dynamic workloads, and it relies on proven security enhanced by powerful encryption on all drive types. Properly planned and implemented IT infrastructures combining IBM’s DS8870 storage system family with all these capabilities can support cutting-edge systems with “six nines” availability—meaning just 30 seconds or less a year of downtime—and reduced recovery points for unplanned outages down to zero seconds when configured with GDPS, IBM’s multisite or single-site end-to-end application availability solution. The IBM Information Lifecycle Governance (ILG) Group: This organization is based on the principle of information economics, an approach that demands the pragmatic management of data according to its changing value to an organization. Toward that end, ILG was created to help with data management in accordance with its solutions, which provide data archiving and tiering for both structured and unstructured data, along with tools that analyze data utilization and value. ILG tags data, cleans it up, moves it to the right place based on its content, and sets retention periods. IBM is dedicated to helping clients derive value from big data, make good strategic choices about retaining it, respond more efficiently to litigation and regulations, and automatically dispose of information that is no longer worth keeping. It is a key capability in the IBM arsenal as the world moves from storage provisioning to data management.
A Word on Cloud Implementations Some readers may wonder why cloud computing is not being paid more specific attention in this paper. The reason is simply that all the requirements referred to herein, and the portfolio offerings that IBM has to address them, apply in any implementation. Whether you run a traditional data center, or a private or public cloud, the same issues and approaches apply. The economics might be slightly different, as might be the balance between which products should be used, but the essence of everything stated here applies across the board. Cloud is simply a consumption and an implementation model. The issues discussed transcend that, so they would apply to whatever one’s implementation and consumption model is.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
White Paper: Optimizing Data Management Through Efficient Storage Infrastructures
11
The Bigger Truth The first step in introducing changes to the way people work is getting them to think differently—and openly— about new environments and how they can improve the success of their business. With data pouring into businesses from an army of mobile devices, new applications, and virtualized servers, the need to manage primary data, backup data, and other legitimate copies of required data has never been greater. The world is changing dramatically, and data management is a key business requisite to the success of that transition. Helping line of business managers to mine the wealth of their data treasure troves is the new IT reality. Storage can no longer be just a commodity in future IT infrastructures. Data management is now driving storage, and the storage world (infrastructure vendors and users alike) must respond with the right products and tools to efficiently support and exploit that data, to both operational and financial advantage. The goal is more favorable technologies that will not depend so heavily on increasing hardware footprints, but will leverage automation and other sophisticated tools for greater efficiencies. IBM’s tools and products such as those mentioned in this paper provide the means for such an efficient infrastructure, which lowers cost, makes it affordable to store more data, and helps alleviate the data management problem. It’s not just a bandage; it’s steroids! More efficient storage (ahem…data) infrastructures lead to decreased costs, which lead in turn to more affordable storage and more data being better stored and used. The new equation for storage success is about getting the maximum value from organizational data at minimal expense, with the least effort and the greatest speed. Companies that get serious about this equation now will find increased business success and innovation in IT strategies that will help them and their teams prosper in the increasingly competitive marketplace.
© 2014 by The Enterprise Strategy Group, Inc. All Rights Reserved.
20 Asylum Street | Milford, MA 01757 | Tel: 508.482.0188 Fax: 508.482.0218 | www.esg-global.com
