security to ensure no performance impact by other users of the object storage. Object. Storage in a private cloud enable
WHITE PAPER
VPSA® OBJECT STORAGE An overview of the VPSA Object Storage technology
Zadara Storage 6 Venture, Suite 140, Irvine, CA 92618 www.zadarastorage.com
WHAT IS OBJECT STORAGE? Object Storage is an alternative way to store, organize and access units of data. It provides a reasonable balance between performance and functionality versus simplicity and scalability. Object Storage enables a minimal set of features: store, retrieve, copy, and delete objects. These basic operations are done via REST APIs that allow programmers to work with the objects. The HTTP interface to Object Storage systems allows fast and easy access to the data for users from anywhere in the world. THE DIFFERENCES BETWEEN OBJECT BLOCK, AND FILE STORAGE Object Storage is much more scalable than file storage because it is vastly simpler. Objects are not organized in hierarchical folders, but in a flat organization of containers or buckets. Each object is assigned a unique ID or key. Their keys, regardless of where the objects are stored, retrieve objects. Access is via APIs at the application level, rather than via OS at the file system level. As a result, Object Storage requires less metadata, and less management overhead than file systems. This means Object Storage can be scaled out with almost no limits. Object Storage is easier to use than block storage and overcomes the limitation of fixed size LUNs. It also removes file system limitations such as the folder size or path name length. Unlike block or file, Object Storage does not use RAID for data protection. It simply keeps a number of copies of each object. WHAT IS PRIVATE OBJECT STORAGE? Private Object Storage is an Object Storage solution built on dedicated compute resources either in a private or public cloud. As such, it guarantees better isolation and security to ensure no performance impact by other users of the object storage. Object Storage in a private cloud enables all of the benefits of the public Object Storage. However, it eliminates the latencies associated with the Internet and the public cloud, and therefore, provides much better performance.
1
OBJECT STORAGE USE CASES
The common use of object storage is for big amounts of relatively static data, like: • Multimedia - music, images and video • Backup files, database dumps and logs • Big Data - Large data sets. Pharmaceutical or financial data, sensor data, scientific research data samples • Archive files in place of local tape drives • Web content
Figure 1: Typical Data Profile
VPSA OBJECT STORAGE CHARACTERISTICS VPSA Object Storage is Zadara’s object storage service. It is provided on Zadara clouds, side by side with the VPSA that provides block and file storage services. These are the key properties of VPSA Object Storage: • Flat namespace (key/value store) • Unlimited scalability (scaling out) in both capacity and total objects simply keep adding drives • User-defined object’s metadata • 2 or 3 way distributed data protection • “Eventually consistent” copies • Supports both S3 and Swift REST API • Comprehensive usage reporting and metering • Multi-tenancy at both the object storage and account levels
2
PROVISIONING VPSA Object Storage is ordered and provisioned just like the VPSA, by a request at the Zadara Provisioning portal. The VPSA administrator selects the number and type of drives to be assigned to the newly created VPSA, and the system automatically assigns the needed number of Virtual Controllers (VC’s). The VPSA administrator receives access to the object storage that contains a default account. At that point, the VPSA administrator can create accounts, assign account administrators, and can begin storing objects. VPSA OBJECT STORAGE USAGE MODEL
Figure 2: VPSA Object Storage Provisioning Portal
VPSA Object Storage may contain any number of accounts that are assigned to different organizations. Users of each account can store objects, and organize them in containers, as illustrated to the right. • VPSA Object Storage is the entire object storage. It may contain any number of accounts as defined by the VPSA administrator. • Accounts are assigned to an organization unit or a department, Figure 3: VPSA Object Storage Structure and contain a list of user-defined containers. • Containers are uniquely named buckets with flat structures that contain objects. • Objects are a piece of data of any size. A key that is assigned to it uniquely identifies them.
3
AUTHENTICATION AND PERMISSIONS There are 3 types of users assigned to VPSA Object Storage: • VPSA Administrator is responsible for the administration of the object storage. The user (registered in Zadara Provisioning Portal) that orders the object storage becomes the Administrator. By default, the object storage is created with one account (VPSA administrator account) and the Administrator is a member of this account. Administrators can add other users with the same role. Administrator is a super-user with privileges to create accounts and users of any role. Users with Administrator role can define policies, add/ remove drives and assign drives to policies. Users with Administrator role can perform containers and objects operations across accounts. The Administrator is also responsible for the VPSA Object Storage settings (like IP addresses, SSL certification, etc.), and has access to the VPSA Object Storage metering and usage information. • Account Administrator can create an account and can manage their own account. They can perform any user management and containers/objects • Member can do object storage operations according to the permission given by the account administrator, within the limits of that account. These operations include create/delete/list containers and create/delete/list objects. User authentication is done against an internal VPSA Object Storage users database. DATA PROTECTION AND HIGH AVAILABILITY VPSA Object Storage data protection and reliability is achieved by replicating objects across multiple storage nodes and disk drives. The user can configure the number of replicas to suit their needs. The minimal Zadara object storage configuration utilizes 3 storage nodes. Zadara object storage with the redundancy of 3 copies can survive any double failure of drives, virtual controllers, or physical storage nodes. Administrators can define a number of protection policies, determine the drive types, and redundancy levels of each policy. Containers are created within a given policy. Default policy is automatically generated at the VPSA Object Storage creation time.
4
ARCHITECTURE VPSA Object Storage is built on 2 main building blocks: Proxy Layer (Node) and Storage Layer (Node). The proxy node is the interface to the users or the application using the data objects. The Proxy node gets the API command, authenticates the requester, decides what Figure 4: Logical VPSA Object Storage Architecture storage node should handle the coming request, and passes it over. Once the appropriate storage node handles the request, the proxy returns the answer to the caller. The storage node is responsible for storing the objects on the drives, and updating the metadata in the databases. The load balancer controls the traffic from all clients to all Proxy Nodes. VPSA Object Storage comes with an internal load balancer, but users can configure the external load balancer outside of the Zadara environment, to get even better performance. VPSA Object Storage lives on the same Zadara Cloud side by side with Zadara VPSA Storage Arrays. The object storage infrastructure is a collection of Virtual Controllers (VC) and drives across the cloud storage nodes. VC’s are dedicated Virtual Machines (VM) running the Proxy and Storage Nodes as described above. Just like VPSA Storage Arrays, VPSA Object Storage has dedicated drives assigned to it to store the data. VPSA Object Storage automatically distributes the load across the storage nodes in a way that guarantees the system survival, even in case of Storage Node failures. Drives assigned to object storage are organized in policies according to the redundancy level required (2-way or 3-way). When drives are added to the configuration, VPSA Object Storage automatically Figure 5: Logical VPSA Object Storage Architecture redistributes the data across all drives.
5
VPSA Object Storage uses large SATA drives (per customer order) to store object contents, and fast SSD drives (automatically assigned) to store the users and container databases. By default, each Zadara object storage Virtual Controller contains one Proxy Node and one Storage Node. Users with heavy traffic can add additional Proxy Nodes in additional VC’s. These can be ordered from the provisioning portal. OPERATIONS VPSA Object Storage fully supports both AWS S3 API and OpenStack Swift API. For API details refer to the API documentation. Both are REST API over HTTPS. The four basic operations are: 1) PUT that writes an object, 2) GET that reads an object, 3) DELETE; and 4) COPY object. The flow of these operations is described in the diagrams below: PUT Operation 1. PUT (Object) request comes from client. 2. Load-balancer selects a Proxy Node and routes request to it. 3. Proxy determines the Storage Nodes to handle the request. 4. Proxy sends the object to 2 or 3 replicas. 5. Proxy queues container DB updates, and completes the PUT operation. GET Operation 1. GET (Object) request comes from client. 2. Load-balancer selects a Proxy Node and routes request to it. 3. Proxy determines the Storage Node to handle the request. 4. Storage Node reads the object. If failed, the proxy will retry to read from another replica. 5. Proxy returns the object to the caller.
6
PERFORMANCE VPSA Object Storage does not guarantee performance, as it depends on the configuration and the workload. As a quick reference, here are some performance measurements taken with Zadara object storage built over 4 VC’s with 20 x 6TB drives, using a 2-way protection policy.
BILLING AND CHARGEBACK VPSA Object Storage provides usage and billing reports both at the storage and the account level. The reports contain details regarding the capacity consumed and upload/download traffic. Usage reports can be exported as CSV files for further processing in external billing tools. CONCLUSION VPSA Object Storage complements Zadara Storage offerings for public and private clouds. In addition to block and file storage services, Zadara now offers Object v to fit the needs of any storage workload. Customers storing static data can enjoy Zadara object storage and its infinite scalability. Cloud providers using Zadara’s technology can now offer S3/Swift compatible object services. VPSA Object Storage is simple, easy to use, and well protected solution for storing multimedia data, sensors sampling big data, or archiving data of any sort. VPSA Object Storage runs side by side with VPSA Storage Arrays on Zadara’s clouds, utilizing the same infrastructure, and providing higher value to our customers.
7
CONTACT US USA Sales
[email protected] +1-949-251-0360
International Sales
[email protected] +44-1285-610045
24/7 Support
[email protected] +1-949-284-0713
2016 Zadara Storage. All rights reserved. Company or Product names may be the property of their respective owners. MAR062017
8
