Six Major Goals for Enterprise Data Storage and ... - Huawei Enterprise

TECHNOLOGY SPOTLIGHT

Six Major Goals for Enterprise Data Storage and Management in Times of Cloud Services William Zhang Frank Cai October 2015

TECHNOLOGY SPOTLIGHT

Six Major Goals for Enterprise Data Storage and Management in Times of Cloud Services William Zhang June 2015 Sponsored by Huawei

Opinion of IDC With the rapid deployment of cloud computing and big data, some emerging enterprises, or business that has revived via the third-party platform technology, have quickly launched innovative products and business modes by applying emerging technologies and new delivery methods, and have achieved revenue growth and increase in value under the new cost structure. IDC finds that the drive for enterprises’ IT environment has shifted from conventional technology to business, and the business needs-oriented approach has become a focus in manufacturing, sales and services. The data center serves as a cornerstone for the business of an enterprise. The explosive growth and diversity of data, as core assets of enterprises, pose a huge challenge to in-house data centers. IDC finds that an enterprise’s annual investment in IT infrastrucutre is relatively stable. How to establish efficient, reliable IT infrastructure that accomodates business development and maintain stable business growth with limited financial input has become an inevitable consideration for enterprises in transformation. Therefore, IDC summarizes six requirements on data storage and management to be urgently met by enterprises under the current circumstances:   Meet big data storage, management and analysis requirements   Also meet speed requirements of big data information processing and capacity requirements of information storage   Meet security and reliability requirements without compromising the efficiency of data use   Scale up smoothly as the enterprise grows rapidly   Reduce costs and preserve existing investment   Data management infrastructure oriented to business and enterprise applications

Based on its in-depth research of data storage and management needs of enterprise users, IDC opines that user needs have undergone sweeping changes in the era of clould computing and big data and new-generation storage products have to innovate and develop in terms of reliability, performance and convergence to meet growing user needs. Huawei is able to help its customers embrace the era of cloud computing and big data in storage systems with its storage products which is centered on OceanStor OS.

©2015 IDC

2

About Technology Focus In this technology focus, we will dig into changes in user needs in the era of cloud computing and big data and how these changes affect IT infrastructure requirements. We will analyse new user needs in data management in the new era and look into special user needs in storage architecture, data management and service continuity protection. This technology focus will also discuss how the OceanStor storage system is designed to meet user needs in the era of cloud computing and big era, provides reliable, converged, simple and efficient storage service and maximize the value of users.

Market Overview Rapid technological advances are driving the evolvement of the industry landscape. At the core of the new round of industry reforms are the application of information network technology and the development of Internet, smart terminals and other new-generation information technologies, which will bring about changes and innovations in many sectors. ICT is causing industries to break boundaries and collaborate in innovation and forcing enterprises to reengineering processes of R&D, production, sale and service. As industries are experiencing Internet-based upgrading and the government is promoting the Internet Plus policy, conventional enterprises will depend more on internet in their business in the future. They will launch smarter and more Internet-based products and services with the technical supports from cloud computing and big data and further speed up industrial transformation. Each enterprise will become an IT company with IT embedded in its genes. For example, manufacturers will leverage on Internet products to accelerate their shift to service-oriented manufacturing. Enterprises will, taking hardware as media of service, provide customers with an uninterrupted flow of value added services and tap into the potential business value to the greatest extent. Furthermore, the big data and its prompt response to customers’ need will bring manufacturers and consumers closer, turning C2B into a new normal. Enterprises can optimize product design, enhance product quality, improve marketing strategy and boost after sales service by analysing preferences, habits and frequency of use of customers. The rapidly developing Internet Plus will break industry boundaries, lead to frequent cross-boundary competition and convergence and make overwhelming and being overwhelmed a new normal of industries.

3

©2015 IDC

FIGURE. 1 Enterprises Evolve To Be IT-Focused

The commercial economy

The startup economy

The personal economy

The public economy

The tech economy

The skills economy

The data economy

Source: IDC, 2015

Nowadays when business is increasingly associated with IT, business line management exerts a greater influence in relevant procurements. The transition from supporting system to production system will reengineer the internal value system of customers. For most enterprises, the CIO function will have a big say in business, or the CIO function will become an integral part of business. IDC expects that over 70% of CIOs will be dedicated to provide IT value, other than maintenance service, for enterprises. CIO will shoulder a greater responsibility and face a heavier pressure. Value reengineering is also a process of business transformation, and also a redistribution process to customers, which requires supports by effective, reasonable and smooth ICT development. Changes in economic environment and user needs are transforming the ICT industry into a new technology platform that enables growth and innovation. In the opinion of IDC, currently IT is in a transition from conventional platform technologies to emerging technologies typically including cloud computing, mobility, big data and social media. Given the rapidly expanding demand for emerging products and solutions in the IT market, new products and solutions that build on them will grow explosively and reshape the ICT ecosystem. Such technologies as cloud computing and big data will become the main powerhouse for ICT market transformation and growth in the next 20-25 years. This trend is particularly notable in developing markets, like China. Meanwhile, the six major technologies including third platform-based Internet of Things, next-generation security, natural interface, 3D printing and robots will become accelerators for future innovation of enterprises.

©2015 IDC

4

FIGURE. 2 Cloud Computing and Big Dataarethe Core Technologies Driving Enterprises IT Transformation

Internet of Things

3D Printing Natural Interfaces

Cognitive System

Cloud Robotics

Mobility

Big Data/Analytics

Next Gen Security

Social Business

3rd Platfrom 2nd Platfrom Source: IDC, 2015

1st Platfrom

With the rapid deployment of cloud computing and big data, some emerging enterprises, or business that has revived via the third-party platform technology, have quickly launched innovative products and business modes by applying emerging technologies and new delivery methods, and have achieved growth in income and increase in value under the new cost structure. The business needs-oriented approach has become a focus in manufacturing, sales and services. The market has transited from technology-driven to business-driven. Efficiency and technological capacity are just the foundation. The rapid development of cloud computing and big data provides greater user convinience, lower technical threshold and higher experience requirements. Given its immediate effect on industry transformation and innovation, the control over IT budget will continue to shift to the business line. Only 10% of business departments participated in ICT building in 1970, but that percentage became 53% in 2014, and will further increase to 80% in 2020. In an ICT market survey targeted at CIOs in 2014, 16% of CIOs stated that “adapting ICT architecture to the ever-changing busienss environment” is the core cause for carrying out ICT investment, the No. 1 cause of all, suggesting that busines reform has become the primary driver of ICT demands. The data center serves as a cornerstone of business. The explosive growth and diversity of data, as core assets of enterprises, pose a huge challenge to in-house data centers. IDC finds that an enterprise’s annual investment in IT infrastrucutre is relatively stable. How to establish efficient, reliable IT infrastructure that accomodates business development and maintain stable business growth with limited financial input has become an inevitable consideration for enterprises in transformation. Therefore, IDC summarizes six requirements on data storage and management to be urgently met by enterprises under the current circumtances: 

5

©2015 IDC

  Meet big data storage, management and analysis requirements Big data analysis enables an enterprise to get a better understanding of its customers and market, launch innovative products and business modes and eventually increase its income and profit. The current information environment is much different from conventional IT scenarios composed mainly of structured data. Internal data types of enterprises are diversified along with the development of big data. An enterprise needs to manage documents, images, videos and other multi-media files and objects and then conducts analysis in conjunction with conventional structured data that build principally on block structures. The conventional storage system with decentralized storage blocks, files and objects cannot meet enterprises’ demand for centralized management and use of multi-protocol storage. This poses the first new requirement to IT infrastructure: A system must support management of multiple storage types, including blocks, files and objects.   Data management infrastructure oriented on business and enterprise applications Nowadays when business is increasingly associated with IT, business line management exerts a greater influence in relevant procurements. In the conventional IT mode, business requirements need to go through analysis, redevelopment, deployment, software analysis-based use of IT infrastructure resources prior to final data writing or reading. In the era of cloud computing and big data, however, many business requirements have been closely conjoined with data management and use. Also, time-sensitive business requires such quick response of IT infrastructure that directly points business and application requirements to data and provides feedback to business after processing or analysis. This poses the second requirement of data management to IT infrastructure in the new era: The future data management infrastructure must be oriented to business and applications and integrate data management with application interface, so as to lay a solid foundation for IT integration of enterprises and meet the business-oriented IT infrastructure requirements.   Also meet speed requirements of big data information processing and capacity requirements of information storage In the present era when IT infrastructure bears a close tie to business, the performance of IT equipment relates closely to an enterprise’s time to market. The speed of response to big data analysis needs determines whether an enterprise can make quick decisions or not to gain an edge over competitors. The demand for quick response in information processing warrants the necessity of high-performance storage media. Meanwhile, information buildup in the big data era requires an extremely big capacity to store data. The contradictions among performance, capacity and costs pose the third requirement to IT infrastructure: IT infrastructure must be compatible with various storage types, including large-capacity conventional disks, high-speed disks and flash memory.   Scale up smoothly as the enterprise grows rapidly The rapid growth of China’s economy has given birth to a large number of fast-growing small and mediumsized enterprises, many of which see their business size increasing in folds, and their data volume expanding exponentially, in a couple of years. The transition to Internet Plus also leads to tremendous changes in many traditional big enterprises’ data size of certain type. Given such huge changes in data size, the major question facing these enterprises is how to ensure continued use of data in existing lower-end storage equipment in the big data environment, or to ensure smooth data migration from lower-end storage to higher-end equipment. Such needs will be better met if IT infrastructure builds on the same management platform, provides compatibility for lowend, medium-end and high-end storages and ensures data flows between different equipment. This is the fourth requirement posed by new-era data management to IT infrastructure.

©2015 IDC

6

  Reduce costs and preserve existing investment User expectations for smooth scalability of IT infrastructure are not limited to compatibility with low-end, mediumend and high-end storages, but also include compatibility with more existing equipment with different architectures and brands. Nowadays when enterprises gradually control IT budget overshooting, preserving the existing IT investment is a concern of each CIO. Therefore, enterprises pose the fifth requirement to IT infrastructure: Be able to integrate the equipment of different architectures from different vendors on a single management platform and preserve previous investments while cutting management costs.   Meet security and reliability requirements without compromising the efficiency of data use In the big data era, data and information is the lifeline of an enterprise. Therefore, enterprises have an urgent need for data security and reliability. When it comes to IT infrastructure supporting data management, security and reliability of components, systems and overall solutions are the primary consideration. Also, the rising frequency of data use causes enterprises to seek security and reliability enhancements without affecting efficiency of data use. In the conventional IT architecture, it is a good option for enterprises to create backup systems or backup data centers using data backup solutions. However, the networking solution for backup systems is complicated in architecture, high in price and time-consuming in recovery, which seriously affects business performance. In such circumstances, enterprises pose the sixth requirement to data management: IT infrastructure must provide enhanced security and reliability without comprising the efficiency of data use.

Huawei OceanStor Converged Storage System Solutions Overview of OCEANSTOR OS OceanStor OS is Huawei’s newly designed storage operating system oriented to cloud computing. With a converged design, OceanStor OS incorporates different types of storage media (SSD/SAS/NL-SAS), different types of storage (lowend, medium-end and high-end) and storage products of different vendors into a unified storage pool for sharing by all services, scalable out to 16 controllers and 27.6 PB capacity. The storage pool employs the RAID 2.0+ full virtualization technology, manages resources in a unified disk management mode and provides a variety of data protection service. Storage resources are dynamically allocated according to different service loads and support multiple types of service, including database and video, enabling flexible matching with the needs posed by hybrid load in the cloud environment to storage capacity, performance, QoS and SLA in line with ample software features in the OS. Besides, Huawei innovatively integrates container technology into OceanStor OS to allow direct container access to storage, thereby avoiding consumption in iSCSI protocol stack and other intermediate steps, increase quickness of response and helping enterprises evolve to cloud architecture.

7

©2015 IDC

FIGURE. 3 OceanStor OS Architecture

Data Layout

Container

Clone

NFS/CIFS

Copy

Snapshot Replication

Mirror

Metro

Backup

Block Service

File service

LUN Semantics

File Semantics

Space

Space

Compression

Deduplication

Compression

Deduplication

LUN Migration

SSD Cache

LUN Migration

SSD Cache

POOL Tier Thin Provision Equalization

sanitization Heterogeneous Virtualization RAID 2.0+

Disk Mgmt Logic Disk Mgmt SSD

SAS

Common hardware

Management Layout

IO Control

System Control

Load Balance

Fault Mgmt

OMM Update Alert

Distributed cache Service

iSCSI/FC/FCOE

Control Layout

Multipath

License

QoS

Overload Control

Object Mgmt

perform -ance Authen -tication CLI

Perform -ance data Collect

SNMP Cluster Mgmt

CFG

Foundation capacity model Hardware logic Memory Mgmt

Dispatch

Communicate

Log

Test Architecture

Lib

Time Mgmt

Hardware Drive SSD

SAS

NL-SAS

3rd Party

Source: Huawei, 2015

Based on a unified pool of converged storage resources, OceanStor OS innovatively enables convergence of storage architectures, multiple applications, multiple media, multiple types, multiple vendors and data protection and provides users with reliable, converged, simple and efficient storage services.   Convergence of storage architectures: one system supports blocks and files and carries multiple types of service   Convergence of storage and applications: The storage system has embedded third-party applications to reduce loss caused by protocol conversion and cut costs   Convergence of storage media: One architecture supports HDD and SSD, giving full play to SSD performance and HDD capacity   Convergence of multiple storage types: Low-end, medium-end and high-end storages are based on OceanStor OS and data follow on an as-needed basis   Convergence of multiple vendors: Integrate storages of multiple vendors and preserve previous investments   Convergence of storage and backup: Integrate storage and backup without any need for additional backup software, thereby providing higher efficiency of data protection.

©2015 IDC

8

Reliability Reliability is the lifeline of storage. Enterprises become more demanding on reliability in the cloud service era featuring data explosion. Reliability, however, involves a systematic program other than individual components or several lines of codes. Each vendor is dedicated to boosting reliability of storage system by optimizing design. Huawei's OceanStor OS, based on the design of converged storage pool, enhances reliability and converges failures at architecture, system and solution layers and ensures data integrity and long-term service stability.   Reliability of architecture layer Architecture underpins the survival of products. The quality of architecture design is key to reliability, scalability, performance and other critical factors of storage. Based on Harvard architecture design, the storage system underwent bus architecture, direct connect architecture and matrix architecture, with controllers evolving from two to more, or even to 24 controllers thanks to scale-out technology. However, it is still rare to have multi-controller architecture with full redundancy in one storage engine. The SmartMatrix 2.0 architecture of Huawei’s OceanStor storage provides 4-controller high-reliability products with full redundancy at 6U, coming as a new benchmark for the reliability in the cloud service era. Huawei's OceanStor storage supports up to four engines, each of which includes four controllers. Thus the system supports 16 controllers. With up to four controllers, there is no need to provide external switches and switching cables; instead, the PCIe 3.0 high-speed channel on the backplane of frame provides full mesh with a bandwidth of 128 GB/s, enhancing performance while simplifying deployments and boosting reliability. The back-end SAS interface card of each engine is shared by and connected via high-speed PCIe to the four controllers in the engine. With the back-end SAS interface card shared, controllers in the engine are fully connected to hard drive frame. If one to three controllers fail in the engine, the path from the remaining controller(s) to hard drive frame is still available. As four SAS interface cards are supported inside one storage engine, if one interface card fails, controllers remain connected to hard drive via the remaining interface cards, thereby ensuring high reliability of storage system.

FIGURE. 4 Smart Matrix2.0 Architecture

CPU

IO5 IO0

IO5 IO0

CPU

CPU

CPU

PCLe Switch

PCLe Switch

PCLe Switch

PCLe Switch

CPU

CPU

CPU

IO5 IO0

IO5 IO0

CPU

Source: Huawei, 2015

9

©2015 IDC

OceanStor OS automatically distributes different LUNs evenly to different controllers, with LUN space scattered evenly among all hard drives in the system. If each controller from host to storage has I/O path, Huawei’s proprietary multi-patch software UltraPath will choose the optimal patch of LUN’s owner controller for distribution. If there is no optimal path, after I/O is distributed to storage, the system will automatically match relevant LUN service with the right controller and forward I/O to that controller via SmartMatrix 2.0 for processing. With LUN allocated evenly among controllers and the LUN space distributed evenly on a global basis, different controller services and hardware disk pressures are relatively balanced and I/O is distributed via the optimal patch chosen with Huawei’s proprietary UltraPath, thereby delivering the optimal reliability and performance of system. To enhance system performance, the cache mechanism was invented in the 1990s, with cache data mirrored between controllers to assure reliability. In the traditional design, two controllers are mirrored to each other in the storage system. When a controller fails, the other controller’s cache data will lose mirror. The other controller is changed to write-through to prevent data loss in a failure, leading to sharp decline in performance. OceanStor OS innovatively employs the persistent cache technology to create cache mirror among controllers to ensure reliability and performance of the storage system. The four controllers of each engine are interconnected via the high-reliability passive backplane, each controller having cache. The cache of each controller is mirrored to another controller to prevent cache data loss when controllers fail. By default, Controllers A and B are mirrored to each other, and Controllers C and D are mirrored to each other. If a controller fails, mirror relationship will be recreated among the remaining controllers. If the four controllers fail successively or one to three controllers are removed, cache data will not be lost and customers’ business will not be interrupted.

FIGURE. 5 SmartMatrix 2.0: Continuous Cache Mirroring

A

A1

C

C1

A1

C

C1

A1

C

B1

B

D1

D

B

D1

D

B

D1

A

B1

C1

A

D

B1

Controller Controller Controller Controller A B C D

Controller Controller Controller Controller A B C D

Controller Controller Controller Controller A B C D

Normally

If one controller fails (for example, controller A fails)

If one more controller fails (for example, controller D fails)

Source: Huawei, 2015

©2015 IDC

10

As shown in the figure above, when controller A fails, for example, controller B will take over the cache of controller A first, and then cache data in controller B (including cache data in controller A) will be re-mirrored to controllers C and C respectively (the target controller is automatically chosen with software algorithms) to ensure caches of all controllers are still mirrored. When controller B fails next, its service will be taken over by controller C or D and its cache data mirrored to controller C or D. If a failed controller is restored to normal, OceanStor OS also will redistribute caches among controllers according to algorithms to ensure all controllers are effectively used and all caches mirrored. If all controllers restored to normal, cache distribution in the system will be restored to default status, i.e. controllers and B are mirrored to each other and controllers C and D to each other. Cache redistribution among controllers consumes system resources (e.g. CPU resources and mirror bandwidth between controllers). When a controller fails, therefore, cache redistribution will not start immediately; instead, a silence period of ten minutes has to elapse prior to cache redistribution. The silence period is designed to prevent unnecessary cache redistribution in controller resetting and controller replacement scenarios. Without the silence period design, the system would start cache redistribution when a controller is reset and initiate cache redistribution again when the controller starts up. Both cash redistributions are an unnecessary waste of system resources.   Reliability of system layer RAID is the most fundamental and most important data protection technology for storage. Traditional RAID is based on physical hard disk drive management. As technology evolves, hard drives are becoming ultra-largecapacity ones. 8TB/10TB HDD is now available in the market, and so is 40TB IP. Large-Capacity hard drives have a big shock on traditional RAID technology. In the past, it took 10 hours to rebuild 1TB HDD and nearly 100 hours for 10TB HDD. According to analysis, the factors affecting RAID rebuild are two-faceted. On the one hand, a RAID group consists of a limited number of hard drives and only data disks participate in rebuild, so few hard drives are involved in rebuild. On the other hand, all rebuild data are written to one or two hot spare disks, which will become the bottleneck on rebuild performance, extend rebuild time and increase the probability of RAID group invalidity. Enterprises are in urgent need of a new RAID technology that could enable quick rebuild, enhance system reliability and mitigate impact on business performance under the large-capacity hard drive tendency. OceanStor OS employs the RAID 2.0+ global virtualization technology, integrates different types of Hard drives (SSD/SAS/NL-SAS) into a single virtual storage pool and split all hard drives at a granularity of 63MB into data spaces and hot spare spaces to replace traditional RAID data disks and shot spare disks, with all business data distributed evenly among hard drive spaces. All hard drives participate in rebuild to avoid performance bottleneck resulting in involvement of only one or two hot spare disks and reduce data rebuild time. According to test results, it takes 10 hours to rebuild 1TB data using the traditional RAID technology, while it takes only 30 minutes to rebuild 1TB data with RAID 2.0+ technology, representing a 20-fold increase in data rebuild speed compared with traditional RAID.

11

©2015 IDC

FIGURE. 6 Complete Virtualization Technology Completes 1TB Data Rebuilding in 30 Minutes

Source: Huawei, 2015

As shown in the figure above, in the RAID 2.0+ rebuild time test, 128 1TB NL-SAS drives comprised a storage pool with 90 1TB LUNs. Data were written to formatted LUNs until the storage pool got full. Then we removed one hard drive, OceanStor storage initiated rebuild and data were written to the hot spare space of storage pool. The log shows that 1TB data rebuild took a total of 30 minutes. As the core technology of OceanStor OS, RAID 2.0+ shields differences of different types of storage media, creates a unified virtualized storage pool in which all resources are split into fine-grained units, providing dramatic convenience for unified management of storage resources and storage pool-based development of advanced storage features, e.g. convergence of storage architectures and online duplicated data deletion and compression. Literally the RAID 2.0+ technology is one of Huawei’s milestones in creating reliable, converged, simple and efficient storage.

  Reliability of solution layer The year 2010 marked the world’s entry into a geologically active period, leading to threat on the security of data assets by possible unforeseeable natural disasters such as tsunami and volcanic eruption; on the other hand, fibre optic cable breakdown, fire and other man-made disasters will lead to business interruption. In such a backdrop, disaster recovery solutions thrive in the cloud service era that is highly demanding on business continuity. Local high-availability, intra-city point-to-point, active-active and 3DC solutions for disaster recovery provide enterprises with protection at different levels and in different scopes. In particular, the active-active solution builds on two data centers with equivalent or different configurations to provide services to the same business system simultaneously, enabling mutual backup and seamless switchover under failure scenarios with real-time data synchronization technology. Huawei converges IT, network and transport, providing the unique complete end-to-end active-active data center solutions in the industry, and also providing guidance for standard guidance for previous complicated active-active solutions based on end-to-end disaster recovery system deployments at storage, computing, application, network, transport and security layers. At the storage layer, the storage active-active solution that builds on OceanStor OS supports both active-active gateways and active-active arrays to ensure RPO=0 and RTO≈0 and warrant online stability of core services.

©2015 IDC

12

 Active-active array architecture: Huawei's OceanStor storage employs a gateway-free design in which two sets of OceanStor storage and one arbiter server to establish storage active-active domains. Each set of storage provides a LUN to create an active-active pair, with the two LUNs having the same LUN World Wide Name (WWN) and appearing as a single LUN externally. Normally, arrays at both ends provide read-write permissions for the LUN, and offer identical concurrent access to application server and process I/O requests of the application server. •• Multi-site cluster: Two sets of OceanStor storage comprise a multi-site cluster system that supports up to 32 storage controllers, i.e. two sets of 16-controller OceanStor are active-active. The multi-site cluster system directly employs inter-array FC or IP link as communication link to complete creation of global node view and status monitoring. Based on global site view, the multi-site cluster system provides distributed mutual exclusion capacity and supports A-A architecture. Cluster nodes have concurrent access and, if a controller fails, their services will be switched over to other controllers that run normally and host I/O will automatically switch over to normal controllers, thereby assuring business continuity. •• Distributed locks: Distributed mutual exclusion is one of the critical capabilities for active-active (A-A). OceanStor storage employs consistency algorithms including Paxos and Consistent Hash Table (CHT), provides distributed object locks and distributed scope locks to meet distributed mutual exclusion requirements of A-A. When the host distributes I/O, LBA gap locks are not cached locally and permissions need be requested first from the lock master. Then the lock master recalls the LBA gap lock of the corresponding I/O and grant authorization to the lock applicant; now data is writable to storage. The distributed lock technology employs the smart lock prefetching and caching policy. With write permissions not cached locally, write permissions are requested from the lock permissions cache node via control message and written permissions for selected gaps are prefetched and cached locally. Subsequent write I/ Os can quickly hit write permissions locally without cross-site application for write permissions. The smart lock prefetching and caching policy eliminates unnecessary data transmission and effectively reduces interarray data transmission times. •• Real time synchronization: A-A enables two data centers to provide services for the same business system, which requires full consistency of data of the two data centers. Huawei HyperMetro A-A technology creates a virtual A-A LUN (i.e. A-A pair) using A-A member LUNs of the two sets of storage. Host write operations write to A-A member LUNs of both data centers synchronously via the real-time mirroring technology. Host write I/O operations are not accomplished until write operations are successful on both ends, thereby ensuring real-time consistency of data between storages at the two sites.  Gateway-free: Huawei creates the multi-site cluster with two independent sets of OceanStor storage to provide A-A service without adding any storage gateway. Compared with gateway-based A-A architecture, this architecture is simple and provides the following benefits: •• Free of storage gateway to reduce points of failure and increase reliability of solutions; •• A-A solution based on two sets of storage arrays costs less than gateway-based A-A solution;

13

©2015 IDC

•• No performance loss introduced by storage gateways, shorter I/O path and better performance. In the case of gateway-based A-A solution, as the host needs to run through storage gateway to connect disk arrays, the storage gateway will introduce a 1ms to 1.6ms delay in I/O response. Array-based A-A solution is immune from the I/O response introduced by storage gateway, leading to a significant improvement in performance. •• The host is connected directly to disk arrays, other than via storage gateway, which greatly reduces networking complexity, shortens lead time and accelerates application go-live. Take 4-controller cluster networking for example, four FC switches occupy 16 less FC ports compared with gateway-based solution.  High-reliability: Building on the high-reliability design of OceanStor storage, HyperMetro also employs many special high-reliability technologies to maximize business continuity. •• Cross-site automatic block repair: A disk may have bad blocks due to power failure or other abnormalities. When read I/O encounters a bad block and reruns a failure, manual repair is required. Huawei employs dedicated cross-site automatic block repair technology to execute local repair first for local bad block, or, if local repair is unavailable, redirect to the far end to read data prior to local write repair. The cross-site automatic block repair technology effectively repairs damaged data blocks and enhance system reliability. •• Service-grained arbitration: When inter-site link fails in the ordinary A-A solution, all services will switch over to one site, which then carries all services. Huawei HyperMetro A-A storage solution supports arbitration by service consistency group. An independent arbitration priority policy is developed for each service consistency group, i.e. site 1 is set as the priority site for some consistency groups, while site 2 as the priority site for the other consistency groups. When the link fails, some services run at site 1 and others at site 2, thereby boosting continuity of each service without comprising performance.  High performance: To ensure real-time consistency of data stored at both data centers, write operations will not return “successful write” to the host until writing to the storage of both data centers succeeds, which will lead to longer delay in I/O operations. Huawei OceanStor takes a variety of measures to mitigate impact on write delay. •• FastWrite: In a normal SCSI process, write I/O will experience interactions including “write command”, “write allocated”, “write data” and “write execution status”, with two round-trip times (RTTs) in the 100KM transmission link. HyperMetro provides protocol-class optimization for inter-array data transmission via FastWrite, optimizes the I/O interaction process with the First Burst Enabled feature of the SCSI protocol, combines “write command” and “write data” into one sending, cancels the “write allocated” interaction process and reduces the number of cross-site write I/O interactions by half. There is only one RTT in the 100KM transmission link, improving performance by 30%. •• Zero copy of data: During initial synchronization or post-failure incremental synchronization of A-A mirror data, there will be massive inter-array data copy. Usually any zero page encountered will not be identified and full copy will be executed. Invalid full zero pages occupy transmission bandwidth and prolong the time of data synchronization. HyperMetro employs the zero page identification technology to optimize inter-array data copy. Data copy source ends are quickly identified via hardware chip to find zero data. Full zero data are specially marked during copy without inter-array full copy, thereby reducing transmission bandwidth occupation and time consumption of transmission. This technology effectively reduces the volume of data synchronized between arrays and significantly shortens the time of initial or incremental synchronization of data.

©2015 IDC

14

 Smooth evolvement of disaster recovery solution •• Disaster recovery smoothly evolves from active-active to three-data-center (3DC)/64:1 centralized solution. Huawei OceanStor storage employs unified storage operating system OceanStor OS, allowing free data flows between different types of storage and enabling smooth evolvement of disaster recovery solution from A-A to 3DC or even 64:1 centralized disaster recovery solution without changing existing network. It dramatically increases flexibility of disaster recovery solution and helps users choose suitable data protection methods according to subsequent business development. •• Coupled by ReplicationDirector disaster recovery management software, HyperMetro features are combined with HyperReplication features to enable 3DC disaster recovery. The local disaster recovery center takes over service when the production center suffers a disaster and maintains the disaster recovery relationship with the remote disaster recovery center. If both production center and local disaster recovery center are affected by disaster, the remote disaster recovery center can execute master-slave switchover for remote copy and restore service. •• Compared with conventional “synchronous replication + asynchronous replication” 3DC solution, the solution consisting of A-A and asynchronous replication provides better resource utilization and quicker failover. In the case of A-A solution-based local disaster recovery, the load of a business can be balanced between two data centers to ensure zero business interruption and zero data loss when a data center fails. At the deployment level, the A-A solution supports smooth scale-up to 3DC solution. Local A-A can be implemented before asynchronous replication is added upon completion of the remote data center to provide remote application protection. ReplicationDirector provides topology display with simplified management and end-to-end monitoring features, manifests status and changes of data protection solution clearly and intuitively, monitors relevant components of equipment on a real-time basis, identifies and assists the user to remove problems and failures prior to disaster switchover and avoids disaster recovery switchover that affects business. •• Heterogeneous A-A solution. For third-party storage in use, HyperMetro is combined with heterogeneous virtualization features of SmartVirtualization. Heterogeneous storage is virtualized to local storage resource eDevLUN first. As eDevLUN has substantially the same LUN attributes as local LUN, A-A technology is then employed to provide A-A data protection for heterogeneous LUN. Heterogeneous A-A enables highreliability integration and optimization of storage resources and increase resource utilization. For third-party storage with a warranty period close to expiration, SmartVirtualization is combined with HyperMirror feature to create volume mirror for and improve reliability of LUNs of third-party storage system. Then the HyperMetro feature is used to provide A-A solution for heterogeneous equipment, thereby ensuring cross-site switchover triggered by failure of third-party equipment and providing a higher level of protection.

15

©2015 IDC

Besides, according to different application scenarios and disaster levels, Huawei storage based on OceanStor OS also provides local high-availability solutions for HyperClone LUN technology and HyperSnap virtual snap technology, solves equipment-class failures and enables local automatic failover. The HyperReplication/S synchronous remote replication technology is used to solve business system breakdown or data loss caused by power or air conditioner failure in computer room, thereby enabling real-time data mirroring and automatic failover between two local data centers. The remote disaster recovery solution that employs the HyperReplication/A asynchronous remote replication technology addresses data center failures caused by regional disasters like flood and earthquake or man-made disasters like power failure or fire, allowing cross-regional data protection and quick business recovery. The cloud service era poses higher requirements to reliability of storage. OceanStor OS employs the SmartMatrix 2.0 high-reliability architecture, RAID2.0+ quick rebuild and HyperMetro A-A solution, providing 99.9999% of system reliability on an end-to-end basis, fortifying the cornerstones of IT infrastructure in the cloud service era and facilitating rapid business development.

Convergence Cloud service promotes IT infrastructure transformation, while the storage technology is updated continuously to follow cloud-based trends. OceanStor OS innovatively employs converged storage services to integrate storage architectures, media, backups, multiple types, multiple vendors and multiple applications into one system, with storage resources supplied according to business development.   Convergence of storage architectures IDC believes that the compound annual growth rate (CARG) of SAN storage will be 5.5% and that of NAS storage will be 10.5% from 2013 to 2018. Enterprise-class NAS market is expanding fast to reshape the enterprise data management landscape. Different types of data result in management complexity, capacity insufficiency and decline in efficiency and reliability. Enterprises are seeking efficient unified storage solutions that could reduce costs, increase efficiency and reduce management complexity without hindering the pursuit of reliability. Currently, unified storage solutions available in the industry are mainly as follows:  SAN with NAS gateways. This architecture requires two pieces of storage equipment to provide unified storage. Two pieces of equipment means larger footprint, greater complexity of networking, more energy consumption and higher purchase cost. The administrator has to maintain two sets of different storage at the same time, or even in different styles, adding to management costs. Also, the file system and block service run on different operating platforms, with the file system building on SAN to increase complexity of architecture and software stack and lengthen access path, leading to a lower efficiency of NAS access.  Block interface above file system. This architecture offers unified storage within one piece of equipment to reduce complexity of networking and maintenance, footprint and energy consumption. As the block interface is based on encapsulation of the file system, block semantics is above the file system and software layers are complicated, leading to a decline in the performance of the block.  OceanStor OS is the first to create unified storage using parallel SAN and NAS architectures and provide both block and file features in one piece of equipment.

©2015 IDC

16

FIGURE. 7 Parallel Architecture of OceanStor OS

System Control

SAN

NAS

Block Service (FC/iSCSI)

File Service (CIFS/NFS)

Space

Space

LUN Semantics

File Semantics

Object

Object

Cache

Storage Pool

Source: Huawei, 2015

The virtual storage pool based on RAID 2.0+ is used to store data, with storage capacity resources shared between SAN and NAS services. A space management subsystem is established above the pool to maintain the mapping relationship between logical and physical spaces of business objects (including blocks and files). For SAN and NAS, the space subsystems are fully parallel and independent of each other. There are two protocol adapter modules, i.e. block semantics and file semantics, in space subsystems. For SAN, SCSI protocol interacts with the protocol adapter layer of block semantics of space subsystems; for NAS, the file system protocol interacts with the protocol adapter layer of file semantics of space subsystems. The protocol adaptor module converts data of block applications (e.g. database) and data of file applications (e.g. video) into object semantics, and then the object management module provides object mapping management and space distribution features. For either SAN or NAS, independent space management and independent protocol analysis offer the highest access efficiency via the shortest software stack and the optimal I/O route and provide one piece of equipment with multiple stacks via parallel architecture to enable concurrent access.  Lower costs: Both SAN and NAS features are available in one piece of equipment without the need for NAS gateways. There is a 20% reduction in the initial purchase cost, 66.7% in footprint, 38.1% in energy consumption and overall 36% for TCO.

17

©2015 IDC

 Enhanced performance: The parallel architecture design of OceanStor OS is characterized by simpler software stack and shorter I/O path, compared with the other two architectures. SAN and NAS share caches. Memory resources are adjusted dynamically according to SAN/NAS business pressure. SAN and NAS share system IOD scheduling for coordinated allocation of CPU resources. QoS technology and flow control mechanism are also employed to avoid competition for resources. In terms of system resource use, SAN and NAS are fully converged. The in-depth optimization of architecture provides high performance to both SAN and NAS. In the SPC-1 test in 2014, OceanStor OS-based OceanStor 6800 V3 converged storage products reach 650,000 IOPS. For more details, please visit the website of SPC: http://www.storageperformance.org/home/.  Simplified management: Relative to SAN with NAS gateways, OceanStor OS-based converged storage networking is simpler and more convenient in deployment. Unified administration pages are used to reduce complexity and increase efficiency by 50%.  Abundant features: A wide variety of value-added software are developed based on OceanStor OS, including duplicated data deletion and compression, hierarchical storage, remote copy and active-active, thereby enhancing efficiency and reliability of storage comprehensively and meeting business needs under different scenarios.

  Convergence of storage and application IT presents a history of spiral development. The first computer came into being at the University of Pennsylvania in 1946. It met large-scale computing needs at that time, encompassing functional components including computing, storage and network. As the division of work got increasingly refined, storage, computing and network components were gradually removed from large-scale computers to become stand-alone IT equipment that meet application needs under different scenarios. In the decades that followed, given different service attributes, dedicated IT equipment varied more and more in forms to shape dedicated hardware, independent management systems and different protocols and networks. In the era of bid data and cloud computing, it is found that, though dedicated equipment plays a crucial role in specific fields, the broad variety of forms, independent management systems and complicated networking within data centers cause many inconveniences to the management and maintenance of data centers. Converging dedicated IT equipment under different scenarios into one system or one piece of equipment becomes the trend. Huawei OceanStor OS employs the SmartContainer software to provide application container with embedded storage via OceanStor V3 converge storage. Third-party applications are deployed on a unified enterprise data platform to boost service reliability and performance, reduce deployment and operating costs, thereby providing enterprise IT solutions with hyper-converged infrastructure on an open platform. Each controller in OceanStor V3 converged storage supports up to three application containers, which run on surplus CPU and temporary resources in the storage system, use physical network ports of V3 converged storage as network resources and user LUNs in the storage system as storage resources. Application containers integrated into OceanStor OS converts LUNs in the storage system into visible block equipment on OceanStor OS via embedded front-end drive, the block equipment mapping mechanism in the container maps block equipment to the application container as data disk, the port management sub-module assists in enabling port status maintenance reporting and

©2015 IDC

18

mutually exclusive management operations, and the virtual resource module carries out management configuration and resource adjustments to the container. The virtual machine file system (VMFS) formats user-designated host block equipment into ext3 file system. The block equipment is LUN created by user in the storage system, which is converted from vhost. The file system is mainly to store large system disk files of each container stored on the current node, and also to store container template mirror files, operation ISO system mirror files and Vmtools drive files imported by user or temporarily generated. SmartContainer specially provides a simple file management mechanism to manage such files.  Enhanced efficiency: Traditionally the business deployment process is complicated after each creation of virtual machine. SmartContainer allows the user to produce properly installed container system disk into template and export it for storage and also supports creation of new containers based on existing template, providing great convenience to users and shortening the time needed for processing of the same business volume or cross-controller deployment. Actual test results show that all deployment operations for containers and applications are finished within 20 minutes. SmartContainer integrates container, storage, server and network into OceanStor OS. The container has direct access to raw device mapping to avoid high-consumption ISCSI protocol stack, removes protocol conversion time in intermediate steps and provides low-delay access. Actual test results show a 30% decline in access delay, 5% decline in CPU utilization and 20% enhancement in performance. SmartContainer supports the user to configure the number of CPU cores occupied by container and storage, enabling container and storage to share all CPU cores and help the user to find the best configurations under different stress scenarios. For different applications, SmartContainer supports allocation of different resources (e.g. different memories and CPU cores occupied) to the container of different applications, and also supports the user to flexibly adjust CPU resources of the storage system occupied by the container on each controller, so as to help the user to optimize performance, configure flexibly under different application scenarios and maximize equipment performance.  Enhanced reliability: To ensure service stability and reliability and enhance system security, SmartContainer sets an effective isolation mechanism between services. •• Install various mainstream operating systems and independent applications in different containers. •• Operating systems and applications of different containers run in fully independent environment and with fully independent storage resources. They do not affect one another at all and are securely isolated. To minimize service interruption time, SmartContainer supports controller failover. Given limited resources on standby controller and importance of services running on each container of the current controller, each container has a failover switch configurable by the user, which is intended for the user to decide whether or not to migrate the container to the controller on the opposite end for continued running under failure scenarios. Besides, the failover switch of each container must go through a series of resource check prior to turning on. The failover switch turns on successfully only when the container is equipped with sufficient mirror resources; otherwise the switch is not permitted to turn on. The container automatically switches back after the controller on the current end recovers. Besides, the abundant data protection features of storage, such as local data protection and remote data protection, are available to containers to enhance business system reliability.

19

©2015 IDC

 Lower costs: In the traditional mode, building an IT ecosystem requires such centralized hardware as server and storage. SmartContainer offered by OceanStor OS embeds server and application container into storage to reduce TCO substantially. Taking typical configuration of video surveillance scenario as an example, there is a 26% drop in footprint, 43% in energy consumption and 28% for overall TCO.  Open platform: Strictly speaking, SmartContainer application scenarios are not oriented directly to end users, but to independent software vendors (ISVs). Containers are embedded into storage and applications embedded into container, providing applications with a relatively open, mutually isolated running environment. Also, the high reliability and high performance of Huawei OceanStor V3 converged storage provides a relatively stable and reliable running environment for applications. IDC finds in its industry survey that operating system requirements vary with industry, necessitating purchase of multiple sets of equipment, compatibility with different operating systems and deployment of different applications by ISVs, a very big waste of resources. Based on the strong compatibility of OceanStor OS, SmartContainer currently supports 36 mainstream operating systems including Windows and Linux. ISVs can choose among operating systems as they want to avoid waste of recourses, cut purchase costs and increase deployment efficiency. Usually an ISV will purchase hardware from different manufacturers to develop solutions to the satisfaction of industry customers in conjunction with in-house or purchased software. But different manufacturers’ equipment has different modes of operation, causing great inconvenience to software development and commissioning by ISV. Embedded containers of SmartContainer retain the native development environment and operation habits, so ISV does not have to change their previous application development modes and end user’s habits, bringing down the difficulty level in third-party software integration. Besides, Huawei provides free testing and validation platforms and deployment tools, helping ISVs develop unique solutions and partners gain an edge in competition. SmartContainer converges container and storage seamlessly, applicable to “data-intensive” services that require frequent data writing and reading, such as video surveillance, bill image and online disk. Currently massive SmartContainer sales have taken place in the video surveillance field.

  Convergence of storage media A large number of HDDs stacked cannot provide satisfactory performance enhancement and place pressure on users in terms of energy consumption and footprint, gradually becoming unable to meet business needs in the cloud service era. Solid state drives (SSDs) have a notable advantage in performance. A SSD boasts over 30x IOPS and less than 1/15 time delay compared with HDD. Storage manufacturers began to use SSDs broadly as SSD price declines. SSD is high in performance but limited to reliability and scalability. The SSD/HDD hybrid storage based on conventional storage operating systems cannot give full play to the high-reliability and low-time-delay advantages of SSD. For example, the limited erasure times of SSD will affect its service life if the operating system is not optimized. The time delay caused by system software consumption has exceeded the time delay of SSD itself, which has weakened the low-time-delay characteristic of SSD in performance enhancement. Enterprises are eager for a new storage architecture that taps fully into the potential of SSD and accommodates business needs in the cloud service era.

©2015 IDC

20

OceanStor OS employs RAID 2.0+ global virtualization technology, converges SSD and HDD into one system perfectly and provides optimized design for SSD to improve reliability, performance and capacity comprehensively.  Enhanced reliability •• To address limited erasure times of SSD, ROW and stripe reorganization technologies are employed to effectively reduce write amplification. With ROW, all newly written data form new stripes and are written into hard drive, with write amplification falling from 3 to below 1.1. The stripe reorganization technology allows recycling of selected blocks of a strip on which a few valid data exist and reorganizes the remaining blocks of that strip and remaining blocks of other strips into new strips, thereby effectively reducing write amplification caused by garbage collection. •• The global wear levelling technology ensures hard drives are written evenly to prevent a single drive to reach service life bottleneck. If many hard drives are close to the service life ceiling, the anti-wear levelling will work to write first to hard drives with less remaining life and accelerate its expiration of service life to avoid simultaneous failure of multiple drives. •• Huawei’s proprietary Erasure Code technology is employed to organize data into strips, each of which consists N original data strips D and M redundant check data strips P, thereby forming ND+MP data protection. If X hard drives fail in the system and the remaining hard drives are insufficient to form ND+MP data protection, the number of member drives will diminish automatically to form (N-X)D+MP, so as to avoid reoccurrence of hard drive failure that leads to lower reliability or data loss. If the system adds hard drives, the number of member drives will increase automatically to restore ND+MP to safeguard data reliability in the case of multi-drive failure. Optimized algorithms embedded into OceanStor OS enhance reliability of Huawei OceanStor converged storage system as a whole and reduces probability of single point failures, especially SSD. Actual test results show that Huawei SSD has a service life of 7.5 years and average damage ratio of hard drives is only 0.29% annually (industry average: 0.44%), improving service life by 60%.  In-depth performance optimization •• End-to-end lock-free design is employed. CPUs process fixed LBA or fingerprint I/Os. CPUs do not interfere with one another and are free of lock conflicts, with storage performance proportional to number of CPUs. •• The multi-core scheduling mechanism is optimized. Performance is optimized for NUMA architecture to reduce access consumption among CPUs and increase CPU cache hit ratio. •• Storage arrays and SSDs interact to move most computing functions down to SSD control chip, thereby mitigating the impact of background operations on performance and ensuring stable host I/O time delay. With OceanStor OS performance optimized, Huawei OceanStor storage provides satisfactorily high performance. Actual test results show that OceanStor 6800 V3 reaches 650,000 SPC-1 IOPS and 3ms stable time delay. High-end OceanStor 18800 reaches 1 million SPC-1 IOPS and 5ms stable time delay. The new-generation high-end OceanStor 18800 V3 reaches up to 3 million SPC-1 IOPS and 1ms stable timely delay. In addition to enhanced performance, the large capacity of HDD is fully utilized to expand the storage system capacity. OceanStor converged storage system is scalable up to 27.6 PB, enabling a combination of optimal performance and optimal capacity in one system.

21

©2015 IDC

  Convergence of different types of storage The history of storage development manifests differences between the system architectures of high-end storage and medium- to low-end storage. High-end storage evolved from bus architecture and Hi-Star architecture to direct connect architecture and virtual matrix architecture. The multi-controller pattern is common at present. Medium- to low-end storage is mostly double-controller in spite of ongoing evolvement to multiple controllers. Given different storage architectures, different types of storage follow different standards in data replication. Data flow between storages of the same type via private inter-array replication command and data transmission format, but compatibility and scalability are weak. Data intercommunication cannot be guaranteed between different types of storage due to different replication commands and data transmission formats, adding to management costs and impeding enhancement of resource utilization. Nearly all manufacturers today cannot support data intercommunication among high-end, medium-end and low-end products within the same sales cycle. In the cloud service era, enterprises hope to shield differences in storage types and utilize data more efficiently. Based on analysis of historical problems, Huawei OceanStor storage took the lead in creating a common replication protocol (CRP) across different types of storage and defines standard formats of data transmission at link, transport and application layers respectively. It fully addresses the intercommunication between different storage types and enables data flows across equipment.  Link layer: Private link specification is defined to provide standard channels of data transmission between different storage types. Private device identification, link handshake and logon authentication feature are supported. Compatibility with FC, iSCSI and TCP links are provided. Performance and costs are also well considered in addition to the adaptation of complicated network environment.  Transport layer: It is to carry data transmission. The numerical control separation technology is employed to provide the highest SLA for protocol data unit (PDU) and ensure timely response to user operations under heavy stress scenarios. The load balancing and link compression technologies are used to reduce redundant data transmission, increase efficiency of data transmission and help customers reduce OPEX. QoS, abnormality detection and repair and other management technologies are employed to assure reliability of data transmission.  A pplication layer: The communication protocol is unified at the application layer via version negotiation, capability negotiation and parameter negotiation, enabling intercommunication between different types of storage and data flows across equipment. •• Version negotiation: CRP automatically completes version negotiation when different versions of equipment are interfaced and the two sides communicate via the protocol version determined through negotiation. •• Capability negotiation. Usually a new version supports more features. When different versions of equipment are interfaced, lists of features supported by both sides are generated via negotiation to avoid system collapse due to feature inconsistencies. Huawei OceanStor storage series runs on OceanStor OS to enable intercommunication among different types of storage (low-end, medium-end and high-end), effectively utilizes resources and saves investment. In disaster recovery scenarios, enterprises can chose a disaster recovery solution that combines low-end, medium-end and high-end storages according to budget, cutting purchase cost by 45%. Besides, the entire storage series is based on one management interface to effectively cut management cost. Currently only Huawei provides data intercommunication among all products in a series.

©2015 IDC

22

  Convergence of multiple manufacturers Given purchases in different periods, different business system requirements and vendor balancing considerations, an enterprise may have different types of equipment made by different manufacturers in its data centers. As the rising number of manufacturers and equipment puts a very heavy pressure on management of data centers, how to effectively use storage resources and simplify management is a common concern of enterprises’ CIOs. OceanStor OS employs the SmartVirtualization heterogeneous virtualization technology to map LUN of heterogeneous storage to OceanStor storage and marks it as external LUN with globally unique WWN, thereby managing the path, status and other attributes of each external LUN. The LUN (eDevLUN) of heterogeneous equipment is created via external LUN and eDevLUN is mapped to host access. eDevLUN consists of meta volume and data volume. The storage space of meta volume is provided by OceanStor storage to store management data and other information of eDevLUN. Data volume itself does not occupy space of the OceanStor storage system and only serves as space map from eDevLUN to external LUN. OceanStor will not modify data or information in the data volume, except business read and write data distributed by host, and the data integrity of external LUN will not be damaged. Heterogeneously virtualized third-party storage and Huawei's storage are unified in terms of management and resource management to effectively increase resource utilization. Software-based heterogeneous virtualization eliminates the need for additional gateway, reduces complexity of networking, effectively avoid single points of failure, boost system reliability and lower overall TCO.  Extensive compatibility: Storages from different vendors vary in understanding of and conformity with SCSI protocol, posing a great compatibility challenge to takeover of heterogeneous storage. OceanStor OS employs different compatibility policies for storages from different vendors, provides different array support libraries and identifies and deals with compatibility with heterogeneous equipment. Huawei has established the largest compatibility library in Asia to provide professional compatibility certification for heterogeneous storage. OceanStor storage is compatible with more than 100 heterogeneous storage models now, which will exceed 200 in 2015. It is believed that Huawei will be able to establish more extensive compatibility certification and industry  Continuous data protection: In the cloud service era, enterprises require full utilization of storage equipment from different manufacturers in data centers and the assurance of heterogeneous storage at higher overall rate of resource utilization. Huawei’s heterogeneous storage data protection solution based on SmartVirtualization heterogeneous virtualization and HyperMirror volume mirror software provides continuous data protection for heterogeneous storage and improves service continuity. SmartVirtualization virtualizes heterogeneous storage into eDevLUN of Huawei OceanStor storage. HyperMirror creates multiple physical copies of the LUN, each of which has the same capacity as LUN. When the server writes to the LUN, HyperMirror will write data to each copy simultaneously. When the server reads LUN, the system will read one of these copies. If one copy is temporarily unavailable (e.g. service interruption due to unstable heterogeneous storage or abnormal link), the server still can access the LUN normally. Besides, there is no much difference between eDevLUN attributes built by third-party storage and local LUN of Huawei OceanStor storage. All the existing data protection solutions of Huawei are applicable to eDevLUN; that is, third-party storage can intercommunicate with Huawei storage directly via SmartVirtualization and support Huawei’s data protection solutions in smooth evolvement, e.g. heterogeneous local/remote disaster recovery, heterogeneous A-A and heterogeneous 3DC.

23

©2015 IDC

 Online data migration: Obsolete equipment must be upgraded and replaced due to expiration of service life or warranty period, but key services carried by them must remain online permanently. This will require a data migration solution that does not interrupt service. Huawei’s online migration solution based on SmartVirtualization heterogeneous virtualization and SmartMigration allows data migration from heterogeneous storage to Huawei OceanStor storage without interrupting existing service, thereby improving service performance and reliability. SmartVirtualization virtualizes heterogeneous virtualization into resources of Huawei OceanStor storage. SmartMigration enables complete migration of services from source LUN of heterogonous storage to target LUN of Huawei OceanStor storage without interrupting existing service, so that the target LUN fully replaces LUN to carry service and complete migration without affecting service. During migration, double write and data change log (DCL) technologies are used to ensure data consistency. The double write technology distributes I/O requests from host to source LUN and target LUN simultaneously to ensure consistency of data changes. DCL is the log of data changes in source LUN during service operation (e.g. I/O differences between source LUN and target LUN when the target LUN fails) and synchronizes difference data after recovery from failure. When migration is completed, exchange of configuration information (e.g. ID of storage pool of LUN, cache policy, owner controller and prefetching policy) of source LUN and target LUN. Source LUN becomes target LUN, services migrating to target LUN without feeling any impact. After data migration, the advanced data access algorithms and flash memory acceleration capability of Huawei storage enhance the data access performance of existing storage. Meanwhile, the existing storage can be retained and taken over by Huawei storage as cold data storage or backup, thereby preserving existing investment.

  Convergence of storage and backup Backup is a traditional and important means of data protection. But full-fledged, complicated traditional backup solutions cannot accommodate business development in the cloud service era. The call for quick and convenient data protection is becoming stronger. IDC is in the opinion that the cloud service era poses challenges to traditional backup mainly in the following areas:  Costs. Traditional backup software is costly, taking up to over 50% of the total cost of backup solution.  Complicated management. Traditional backup/recovery is lengthy in process, complicated in networking and difficult in management.  Performance. Traditional backup/recovery has a long time window, which has an ongoing impact on existing service and putting enterprises in a dilemma between reliability and performance. OceanStor OS employs the HyperVault integrated backup technology to provide flexible and convenient backup solutions. Users no longer need purchase separate backup software or server, for one set of equipment provides both storage and backup features to cut purchase costs. A single system is used for management to simplify networking and reduce complexity of management. The backup technology based on snapshot and replication enables second-level backup/recovery. Backup data are stored in original format and are immediately accessible.

©2015 IDC

24

•• Lower costs: The snapshot and replication technologies provided by primary storage are used to enable backup without purchase of separate backup software and backup server, saving tens of thousands of dollars per TB data. •• Unified management: Compared with the complicated networking mode of traditional backup solution (primary storage + backup server + backup media), integrated backup provides storage/backup features in a single set of equipment without deploying separate backup server or media, thereby simplifying networking. One system is used to manage primary storage/backup to reduce complicity of management. Unified management pages provide abundant backup policies (e.g. setting frequency of backup execution, startup time and retention period of backup data) and one-key recovery feature to avoid complicated settings and simplify backup/recovery process. Local backup/recovery and remote backup/recovery are supported, deployed according to business scenarios: Local backup/recovery: The primary storage provides both production data and backup data spaces. Remote backup/recovery: The primary storage system provides production data space, while remote backup storage provides backup data space.  Efficient backup and recovery: During data backup, cache data are flushed to storage via HostAgent deployed on service host to ensure consistency of service and reduce complexity of operation and maintenance. Based on snapshot features, local backup employs the ROW technology to enable second-level real-time backup without affecting performance at all. Data recovery employs the snapshot rollback technology to redirect root nodes of source data to root nodes of snapshot data to complete data rollback. Compared with traditional backup solutions, local backup eliminates data analysis and allows second-level recovery. Based on remote replication features, remote backup employs difference bitmap to track data differences between primary storage and backup storage, executes incremental data backup, saves transmission bandwidth and enables rapid backup. At the time of data recovery, data rolls quickly back to historical time points via the latest historical snapshot of primary storage, and then differences between historical snapshot and the largest backup data are obtained from the backup end. Only different data are transmitted to primary storage for recovery, thereby reducing data recovery bandwidth and enabling quick recovery. OceanStor OS provides integrated backup that meets both primary storage and backup scenarios, allows quick backup/recovery and satisfies enterprises’ needs for low cost, high efficiency and simple management of backup system. However, Huawei is also aware that the current integrated backup capability lags somewhat behind traditional backup software. Fine-grained backup/recovery will be implemented in the future based on Oracle and Exchange applications so that a datasheet in the database or an email of a user can be backed up/recovered, meeting both reliability and efficiency requirements. OceanStor OS innovatively introduces converged storage service based on reliable converged resource pool. OceanStor OS converges storage architecture, media, backup, multiple storage types, multiple manufactures and multiple applications into one system. Based on the innovative parallel architecture, 1ms stable time delay and the industry's unique intercommunication among low-end, medium-end and high-end storages and comparability with over 100 heterogeneous equipment models, OceanStor OS enables supply of service deployment, performance and capacity, data protection, data value, lifecycle and applications on an asneeded basis and meets business development in the cloud service era. Literally, OceanStor OS is the core of Huawei storage. Currently the entire converged storage series of Huawei builds on this operating system. It will be further optimized from time to time to provide enterprises simple, efficient storage experience.

25

©2015 IDC

Simplicity Path visualization is critical to failure location during storage operation and maintenance. Space utilization analysis is an effective way of cost reduction, but most storage management pages are functionally complicated and management style varies greatly with storage type, adding to complexity of management. In the cloud service era, greater storage convenience and lower management cost are very important to CIO. To simplify management, the storage management software of OceanStor OS provides comprehensive optimization.   User-friendly UCD design  B ased on understanding of business, provide scenario-based interface layout, procedure-based configuration wizard and intuitive equipment exhibition to reduce user’s cognitive burden, increase efficiency of learning and facilitate quick configuration.  T he storage space allocation template designed for mainstream applications such as Oracle and VMWare greatly increases the efficiency and reasonableness of space allocation and enhance user experience.  C omplete initialization configurations with just 5 steps to allow quick go-live. Expand equipment capacity with just 2 steps within 1 minute. One-key disaster recovery to eliminate the possibility of man-made errors and assure system reliability.   Centralized management of operation and management  One set of pages is used to manage SAN and NAS. Over 30 Huawei’s existing storage models, over 100 heterogeneous storage models and over 100 IT equipment models including switches, storages and servers in the data center are managed centrally to reduce management cost by 20%.  Based on the end-to-end visualized operation and maintenance philosophy, the storage path E2D topology is used to clearly show resource mapping from host to switch to storage, storage resource allocation, alarm status, performance status and analysis and statistics, improving the efficiency of operation and maintenance by 2 times.  Introduce the business intelligence (BI) analysis technology, provide data analysis and processing capability and increase efficiency of analysis.   Open external interface  The RESTful northbound management interface based on standard http protocol makes the integrated development of the high-level network management system easier.  Support standard protocols and interfaces such as SMI-S, SNMP, WMI and SSH to manage heterogeneous equipment.  Enable interfacing between storage and industry ISV software based on standard and open interface system. For example, provide standard interface in the video surveillance and media & entertainment industries to allow industry software to control storage, and enable interfacing with the user’s network management system in the data center operation and maintenance scenario.

©2015 IDC

26

Efficiency Various applications, explosive data growth and limited IT budget require a higher efficiency of storage system, including quick response to business needs, adaptive resource allocation and higher space utilization. The Smart software series of OceanStor OS employs deduplication and compression, multi-tier storage and cache partition technologies to provide customers with smart, efficient solutions.   Improved space utilization The cloud service era brings massive duplicated data in addition to unexpected data growth. Such duplicated data creates very limited value for enterprises, but occupy massive storage space, increase energy consumption and cooling costs, reduce data access performance of system and consume limited IT budget of enterprises. Enterprises are in urgent need of a method to reduce duplicated data without compromising access performance of existing data. This problem is addressed usually with the deduplication and compression technology across the industry. However, deduplication and compression involves much compression, decompression and fingerprint algorithms and occupy a large part of CPU. Once the feature is activated, it will affect performance significantly. Thus traditional storage employs the post-process deduplication and compression technology. For users, the post-process deduplication and compression technology cannot reduce the reserved storage space or initial purchase cost, and additional data read and write operations increase system burden. Huawei OceanStor storage employs professional reduplication and compression acceleration card to embed compression, decompression and fingerprint algorithms and reduce CPU workload, the first to enable online deduplication and compression among traditional storage and hybrid storage arrays.  Online deduplication and compression: After starting the SmartDedup application, data will flow from cache to the deduplication and compression acceleration card. The fingerprint of each data block is calculated via SHA-1 algorism and compared with historical fingerprint database. If the fingerprint already exists in the historical fingerprint database, delete the data, and modify the pointer of the data block only to point it to existing data in the hard drive. If the fingerprint of each data block is inconsistent with historical fingerprint database, the SmartCompression application should be initiated. The compression module combines and compresses multiple small data blocks that belong to the same compression object and have continuous logical addresses via LZ4 or ZLIB compression algorithm, thereby increasing compression ratio.  Reduce initial purchase cost: By employing the post-process deduplication and compression technology, data is saved to hard drive in original size to start with, and then the deduplication and compression algorithm is invoked to read data from hard drive to cache for reduction. Reduced data are eventually rewritten to hard drive to reduce space occupation. It is obvious that the post-process deduplication and compression technology not only requires the user to reserve a space for saving reduced data, but also stores prereduction data, which drives up initial purchase cost.

27

©2015 IDC

Online deduplication and compression technology finishes data reduction before data reaches hard drive. The user only needs to save reduced data to effectively cut reserved storage space and lower initial purchase cost.  Reduce service load: Professional deduplication and compression acceleration card is used to unload fingerprint computing, compression and decompression that consume computing resources heavily to the acceleration card, thereby effectively reducing service load. Actual test results show that in the sequential large I/O scenario, CPU usage ratio drops by 24.6%, IOPS increases by 342.4% and time delay falls by 77.4%.  Support both SAN and NAS: SmartDedup and SmartCompression support both SAN and NAS, enabling LUN and files to deliver effective data reduction.  Enhance resource utilization: Actual test results show that in typical application scenarios, such as virtual desktop infrastructure (VDI), the user’s storage space is saved by 75% and 4x data are stored in the same space.   Accelerated performance The cloud service era requires high performance and limited purchase budget of IT systems. The use of SSDs alone meets performance requirements but exceeds budget substantially; the use of SASs alone meets budget but does not satisfy performance requirements. Enterprises are in urgent need of a kind of solutions that strike a balance between cost and performance.   OceanStor OS establishes a storage pool consisting of SSD, SAS and NL-SAS, which are organized via SmartTier software into high-performance storage tier, performance storage tier and capacity storage tier. Data corresponding to I/O is stored to relevant areas of the hard drive according to initial policy, with the size, sequence and access frequency of each data block recorded during the monitoring period. Such information is used to calculate and rank activity, determine data blocks to be migrated and then migrate data on that basis. Frequently accessed hotspot data automatically migrate to high-performance storage tier, while less frequently accessed non-hotspot data migrate automatically to large-capacity, low-cost storage tier to increase performance and reduce TCO. According to actual test, in the OLTP scenario of Oracle database, the solution consisting of 8 SASs, 8 SSDs and SmartTier cuts cost by 50%, increase transactions per minute (TPM) by 233.5% and shortens time delay by 85% compared with the solution that consists of 48 SASs without SmartTier. As high-performance storage media, SSD is also used as L2 cache in addition to use in multi-tier storage to further enhance system performance. SmartCache can serve as the expansion of RAM cache in the storage system to cache unmodified hotspot data that cannot be stored in RAM cache, and also can serve as the read cache for internal metadata and store frequently accessed data in SSD by identifying hotspot data, thereby accelerating and enhancing overall system performance. According to actual test, in the OLTP scenario of Oracle database, read/write ratio is 2:1. With 4 200G SSDs for Smart Cache relative to 50 SASs, I/O hit ratio is as high as 78%, time delay shortens by 55% and TPM improves by 100%.

  Assurance of mission-critical performance Multi-service hybrid carrying has becomes the trend in the cloud service era. Load characteristics vary much with service and compete for key resources, such as CPU, concurrency and cache, leading to untimely response of critical services and substantial resources occupied by non-critical services. The storage system needs smart reallocation of resources to ensure service quality at different levels. SmartQoS intelligently allocates and adjusts computing resources, cache resources, concurrency resources and hard drive resources in the storage system, applies end-to-end fine-grained control on the entire storage path and adopts the I/O scheduling priority and flow control technology to ensure more resources are allocated to services with higher priority on the same storage equipment and meet required service quality. Currently Huawei is the only manufacturer that supports priority, upper limit restriction and lower limit restriction QoS technologies. Actual test results show that in the hybrid scenario that combines OLTP, OLAP and VDI, with SmartQoS used, time delay shortens by 58.7% and performance improves two times for higher-priority OLTP, effectively assuring service quality of higher-priority applications and restricting pre-emptive use of bandwidth resources by lower-priority services. The cache partition software SmartPartition configures different cache partitions for different services. The system ensures cache resources in a partition are used exclusively by relevant service and allocate front-end and backend concurrency resources among partitions according to service conditions on a real-time basis. Cache limit is set for critical services to prevent non-critical services from occupying cache resources for critical services. Currently only EMC high-end, Huawei and HDS storages support cache partition. Actual test results show that in the hybrid scenario that combines OLTP, OLAP and VDI, with SmartPartition and SmartQoS used, OLTP timely delay drops by 81.4%, VDI time delay falls by 76.2% and OLSP inquiry time increases by 87.1%. It is clear that the combination of SmartPartition and SmartQoS further improves service quality of critical applications.

Challenges and Opportunities Facing Huawei OceanStor Storage System IDC notices that the rapid development and implementation of cloud computing, big data, mobile and social network technologies in recent years impose increasingly stricter requirements on storage system and require more technological innovations in storage system, such as multi-protocol support, multi-data-type support, stronger performance and greater scalability. The storage system is transiting to hyper-converged system. Backed by its strong R&D capacity, Huawei has launched OceanStor V3 converged storage to meet market needs well. IDC also notices that, however, Huawei will face some challenges in promoting its OceanStor V3 converged storage solutions. A mature storage system must be one that has been tested for a long period in the market. The development of a product with sound architecture and features is just part of the process. Subsequently the product need be updated and improved from time to time in actual application environment to meet complicated IT environment needs. In particular, Huawei storage should be further refined in terms of best practices and user experience of client applications for different industries. Huawei recently announced its establishment of an industry solution and consulting function in a bid to gain a deeper insight into needs of different industries and serve customers better.

29

©2015 IDC

Conclusion The IT environment of enterprises is experiencing cloud-based transformation. The storage system, as a key infrastructure of enterprises’ IT environment, is evolving to diversified, converged, integrated and eventually business-oriented storage system. In this technology focus, IDC outlines the architecture, high lights, value and future trends of Huawei’s OceanStor OS to the IT sector. Based on its in-depth research of data storage and management needs of enterprise users, IDC opines that user needs have undergone sweeping changes in the era of could computing and big data and new-generation storage products have to innovate and develop in terms of reliability, performance and convergence to meet growing user needs. With its storage products focused on OceanStor OS, Huawei helps customers embrace the era of cloud computing and big data in the field of storage systems and meets enterprises’ urgent needs in data storage and management.

©2015 IDC

30

ABOUT THIS PUBLICATION This publication was produced by IDC Go-to-Market Services. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Go-to-Market Services makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. COPYRIGHT AND RESTRICTIONS Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires prior written approval from IDC. For permission requests, contact the GMS information line at [email protected]. Translation and/or localization of this document requires an additional license from IDC. For more information on IDC, visit www.idc.com. For more information on IDC GMS, visit www.idc.com/gms. Asia/Pacific Headquarters 80 Anson Road #38-00 Fuji Xerox Towers Singapore P.65.6226.0330

F.65.6220.6116