Distributed Cloud Computing Platform as a ... - Open Grid Forum

Distributed Cloud Computing Platform as a Service (PaaS) Analysis and Recommendations OGF 34, 14 March 2012

www.r2ad.com

Michael V. Behrens David Moolenaar

Eugene Luster

CTO, R2AD, LLC

Cloud Researcher, R2AD, LLC

© 2012 Open Grid Forum

VP West Cost, R2AD LLC

OGF IPR Policies Apply • •

• •

•

“I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.” Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to: • • • • • •

the OGF plenary session, any OGF working group or portion thereof, the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, the ADCOM, or any member thereof on behalf of the ADCOM, any OGF mailing list, including any group list, or any other list functioning under OGF auspices, the OGF Editor or the document authoring and review process

Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions. Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, nondiscriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.” OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.

© 2012 Open Grid Forum

2

Background: Cloud Computing Standards Research Focus We (R2AD, LLC) are sponsored by DISA’s Office of the CTO to pursue cloud computing research with emphasis on the following: 1. 2. 3. 4.

Open Source High Security Interoperability Portability

5. 6. 7.

High Performance Transparency Ease of Management

We are currently directly involved with these standards groups: – Open Grid Forum (OGF)

• OCCI specification (more later) – Storage Network Industry Association (SNIA)

• CDMI specification (more later) – National Institute of Standards and Technology (NIST)

• Document roadmap of cloud standards • Others as time/energy permit © 2012 Open Grid Forum

3

R2AD Android Cloud Management Client

Goal: Cloud Interoperability • Create a specification for PaaS which outlines the key architecture components to enable interoperability • Document standards (de-facto or other) to include along with open-source components • Specify APIs where needed which can be used by cloud brokers or by developers • Facilitate resource utilization between clouds and migration of data and VMs between clouds Cloud Computing IaaS

PaaS Web

© 2012 Open Grid Forum

Cloud Computing

Inter-Cloud SaaS

IaaS

Data

PaaS Web

4

Data

SaaS

Recommend OCCI Platform as a Service (PaaS) Specification •

PaaS next big wave of cloud technology •

•

Operating System really should not matter • • •

•

It’s the next generation for enterprise workloads black cloud computing PaaS layer should be Operating System agnostic Separating allows a set of heterogeneous systems to handle the workflow. Performance/Efficiency/Cost based decisions on back-end instead of technical.

Major new choices available for open on-premise private clouds •

OpenShift •

•

Cloud Foundry •

•

Available for on-premise in near future – not fully open however.

Globus •

•

Backed by VMware. Uses Tomcat, Spring, etc.

Jelastic •

•

Backed by Red-Hat. Uses JBoss, Spring, etc.

Open source private cloud PaaS. Globus World is next April in Chicago

Others: Cumulogic, DISA’s, Services, etc.

© 2012 Open Grid Forum

5

NIST Definition of PaaS: Platform as a Service (PaaS). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment.

OCCI PaaS Standardization Recommendations • Evaluate existing PaaS Products/Standards • • • •

Understand the impact to overall integration and interoperability. Compare requirements/features with existing legacy or proprietary PaaS. Apply use case approach to help evaluation each offering Create a set of requirements based on this evaluation period

• Engage open PaaS offerings to gain support from the get go • •

Contact OpenShift, Cloud-Foundry, others? Involve Cloud Brokers, i.e.: CompatibleOne, Open Nebula, jCloud, Deltacloud

• Focus on a standardized Architecture and APIs (REST Based) • Mark out key architecture components and needed interaction • Database Layer, Web Layer, Control/Management Layer, Load Balancer, provision/config • Interactions with IaaS for scalability • Incorporation of OCCI IaaS and CDMI

• Document key interfaces for developers and create new OCCI PaaS Specification • Consider these areas: Security/Identity, Data access, CRUD, Monitoring, Tenants of distributed computing (inter-cloud scalability or bursting), Load balancer © 2012 Open Grid Forum

6

PaaS Vision (work in progress) •

Replace stove pipe architectures and proprietary use of APIs with one or more standardized PaaS and Cloud APIs • •

Common services (ReSTful APIs) for data storage and access, identity, logging/auditing, messaging, processing, monitoring, deployment, replication, SLA Use pattern based APIs/Tools in order to remain focus on application logic instead •

•

J2EE helped us through the last decade. PaaS binds scalable components together, similar to what Sun did with J2EE (EJB, JMS, JDBC, JSP, etc). PaaS supports multiple languages and web engines (Weblogic, glassfish, JBoss, Tomcat, Jetty, Nginx , node.js, etc)…however ideally using same REST APIs to ensure portability in a heterogeneous cloud environment (i.e. CouchDB). •

•

Automatic load balancing for web and data. Distributed data for speed and redundancy •

Use cloud based repository development, test, and field •

Runs on top of IaaS in production. Agnostic as to which IaaS. Use the cloud to test the cloud, monitor the cloud, etc.

Supporting Mobile end-user •

SaaS Applications, Software

End Users/Customers

PaaS Application Infrastructure Web Server+Data Storage

Developers

IaaS Hardware Infrastructure Servers, Raw Storage, Network

Data Center/Integrators

Forge.mil: GIT, SVN, Maven, DISA CM

Cloud Oriented • •

•

Built in replication/synchronization/caching, based on API which specifies data policy

Provide automated self-service for full life cycle •

•

Engine and Operating System should not matter….black box cloud computing

Provide On-Demand scalability • •

•

Do not keep re-inventing the platform (identity management, logging, database, management, etc)

Data and apps accessible to mobile user (location transparency) © 2012 Open Grid Forum

7

Cloud Technologies Compared (work in progress) Offering

Reference URL

Installable Open On-Premise Source

Cloud Style

OpenShift Cloud Foundry Stackato

https://openshift.redhat.com/app/ http://www.cloudfoundry.com/ http://www.activestate.com/cloud

Yes Yes Yes

Soon Yes Yes

PaaS (backed by RedHat) PaaS (backed by VMware) PaaS (based on cloud foundry)

Cumulogic

http://www.cumulogic.com

Yes

No

PaaS

Jelastic

http://www.jelastic.com

Soon

No

PaaS

Apprenda

http://apprenda.com/

Yes

No

PaaS (primarily .Net focused)

Appistry

http://www.appistry.com/

Yes

No

PaaS –Hadoop Analytics

RightScale

http://www.rightscale.com/

No

Management

Google App Engine

http://code.google.com/appengine

No.

IaaS and PaaS

SalesForce Force.com Microsoft Azure Services Platform Amazon EC2, Amazon S3, Amazon Beanstalk

http://www.salesforce.com/ http://www.force.com/ http://www.microsoft.com/windowsazure

No

SaaS PaaS PaaS + DaaS + IaaS

http://aws.amazon.com/ec2/ http://aws.amazon.com/s3/ http://aws.amazon.com/elasticbeanstalk/

No.

IaaS DaaS PaaS

Rackspace OpenStack CloudStack Eucalyptus

http://www.rackspace.com/ http://www.openstack.org/ http://www.cloud.com/ http://www.eucalyptus.com

No Yes Yes Yes

IaaS IaaS and DaaS IaaS (backed by Citrix) IaaS + DaaS (mimics Amazon)

© 2012 Open Grid Forum

No.

8

Yes

Generic Cloud PaaS Model Big Data & Storage

Source/App/VM Repository Auto Deploy/Configure

PaaS Node

Memory Cache

Distributed & Indexed Synchronization

Node Data Memory Cache

Messages/Queue

Algorithms

Algorithms

REST Web Scalable Platform

REST Web

Management & Security

Node Data

PaaS Node

Inter-Cloud

Proxy Load Balancer Perimeter Security https (RESTful)

Mashups (Distributed Observed State) 9 © 2012 Open Grid Forum PaaS Offerings that include well known/used/documented/supported Consider: Open components Hadoop, HAProxy, Restlet, ProxyCA, Node.js, web server, etc.

Survey: OpenShift PaaS • OpenShift is a PaaS backed by Red Hat since 2011 – Languages: Java, Perl, PHP, Python, and Ruby – Application Services: SQLite, Apache HTTP, JBOSS, Spring, built-in management layer – Open Sourced: Not yet (planned) – On-Premise: Future Offering

• OpenShift Express – Free version, no auto-scaling. Good for developers – Hosting: Amazon initially

• OpenShift Flex – Auto-scaling, performance monitoring, app management – Can autodeploy into OpenShift using tools like http://www.jboss.org/arquillian

• Web Site(s): – https://openshift.redhat.com © 2012 Open Grid Forum

10

Survey: Cloud Foundry PaaS • Cloud Foundry is a PaaS backed by VMware since 2011 – Languages: Java, .NET, Ruby, Scala, Node.js, PHP, Python – Application Services: RabbitMQ, vFabric PostgreSQL, MySQL, MongoDB, Redis, Spring, Chef Configuration Management – Open Sourced: Yes – On-Premise: Yes

• Cloud Foundry Open Source – CloudFondry.org is the open source site. Others, like VMware, and Stackato build on it.

• Micro Cloud Foundry – A VM which is available from VMware to run the PaaS environment on your laptop. Alternatively, VMware is hosting a free cloud for developers to experiment with.

• Web Site(s): – http://www.cloudfoundry.com/ – http://www.cloudfoundry.org/ – http://www.activestate.com/cloud © 2012 Open Grid Forum

11

Jelastic Example…

© 2012 Open Grid Forum

12

Globus Toolkit (Potential PaaS)

Applications High-level Services and Tools

DRM Cactus

MPI

globusrun

PUNCH

Grid Status

Nimrod/G

Condor-G

Core Services GASS GridFTP

Metacomputin g Directory Service

Condor

MPI

LSF

PBS

Globus Security Interface

GRAM Replica Catalog

Local Services NQE

Linux

I/O

TCP

UDP

AIX

Solaris

globusWorld, April 10-12, 2012 at Argonne National Laboratory http://globusworld.org/ © 2012 Open Grid Forum

13

Paas Data Layer - Transparency • Implement Location Transparency • Make data access layer virtual in order to decouple data sources from data consumers. This turns data into an enterprise information service. Secure Restful access and Scalability are important considerations.

• NoSQL is a common solution, implemented by: • • •

•

CouchBase or MongoDB – synchronization of document stores Hadoop/Hbase – based on Google’s file distributed file system/database, used by Facebook. Suitable for real-time and data warehouse. Riak – based on Amazon’s Dynamo , scalable, fault-tolerant, open source key/value database server Hypertable - high performance data storage for applications requiring maximum performance, scalability, and reliability.

• Achieve data virtualization, don’t focus on a specific tool. • Consider Cloud Data Management Interface (CDMI) as the PaaS Layer to data – it provides an access API to data and metadata • Pick appropriate underlying storage products based on requirements © 2012 Open Grid Forum

14

OCCI PaaS…Brainstorm Brainstorming session….White Board

Replication

Monitor

Ambient

Management

Billing

Protocol Mediation

Transport

Logging Synchronization

Repository Standards Instead of Cloud Silos!

Provisioning Configuration Data Store Security Service Privacy Streaming Presentation Load Balance

Distributed Computing Web Platform Availability Encryption © 2012 Open Grid Forum

NoSQL

High Availability 15

Migration