Internet-based Virtual Computing Environment: Beyond the data ...

Future Generation Computer Systems 29 (2013) 309–322

Contents lists available at SciVerse ScienceDirect

Future Generation Computer Systems journal homepage: www.elsevier.com/locate/fgcs

Internet-based Virtual Computing Environment: Beyond the data center as a computer Xicheng Lu a , Huaimin Wang a , Ji Wang a , Jie Xu b , Dongsheng Li a,∗ a

National Laboratory for Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha, PR China

b

School of Computing, University of Leeds, Leeds, UK

article

info

Article history: Received 2 November 2010 Received in revised form 12 May 2011 Accepted 5 August 2011 Available online 17 August 2011 Keywords: Cloud computing Multi-scale resource aggregation Elastic binding Virtual Computing Environment

abstract The two dominating characteristics of new and emerging Internet applications are ultra-large scales and utility. Centralized data centers alone are often inadequate for running such applications. In this paper we introduce the concept of an Internet-based Virtual Computing Environment (iVCE), which aims to provide Cloud services by a dynamic combination of data centers and other multi-scale computing resources on the Internet. We present a model that addresses two critical challenges in iVCE: multiscale resource aggregation and elastic binding. We then describe the design and implementation of our iVCE software platform that embodies the model. Comprehensive experiments show that iVCE provides a novel, promising way to deal with scalability and utility, thereby enabling economical and elastic Cloud Computing. © 2011 Elsevier B.V. All rights reserved.

1. Introduction Cloud Computing [1] is becoming increasingly popular as an Internet-based computing paradigm. It provides users with convenient IaaS (Infrastructure as a service), PaaS (Platform as a service) or SaaS (Software as a Service). However, the scale and utility characteristics of new Internet applications introduce real challenges to Cloud Computing. The scale of Internet applications, including user scale, data scale and task scale, is increasing exponentially in recent years, and their demand for resources often varies dramatically as well. In particular, the number of users of typical Internet applications, such as network search engineering, network shopping, and social networks, often reaches the giga scale, and the data volume of many applications will soon exceed the peta scale. Meanwhile, the number of concurrent tasks in an Internet service has reached the giga scale, and the difference between the peak and the average page views of a website can be several orders of magnitude now. It is also noticeable that Internet service providers become more and more dependent on public platforms to construct and publish services; they in fact encourage users to participate in service improvements via public platforms. Such a utility model of delivering Internet services accelerates further the increase of

∗ Correspondence to: PDL Lab, College of Computer, National University of Defense Technology, Changsha City, Hunan Province, 410073, PR China. E-mail addresses: [email protected], [email protected] (D. Li). 0167-739X/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.future.2011.08.005

the scale of both users and data. Typical Cloud Computing systems, such as Google App Engine [2], IBM BlueCloud [3], Amazon EC2 [4], and Microsoft Azure [5], provide services mainly based on large data centers [6]. While data centers are important to provide Internet services with the possibility of offering better Quality of Service (QoS), they often fail to match the aforementioned largescale and utility characteristics of new Internet applications. There are various reasons for this. Firstly, a data center has its limit on service patterns and capability. For example, a centralized data center is not suited for delivering a Cloud service to large-scale distributed Internet users. Its network capacity also confines its service capability. Furthermore, the burst phenomena (e.g., the flash crowd of users) of popular Internet applications require fast and elastic computing capacity. Data centers attempt to provide support for the elastic scalability of some applications when other applications are at low tide. Unfortunately, many Internet services are highly related and they could reach their peaks at the same time. When the burst requirement of concurrent Internet services is beyond the capacity of the data center, the QoS of Internet services could not be ensured. To deal with the burst phenomena, a data center has to increase the scale of its resources, leading to an increase in cost and a decrease in the average usage ratio. Secondly, it costs much energy and money to build and maintain a large-scale data center. There are actually many idle distributed computing resources (e.g., edge servers, networked PCs) on the Internet. If these resources are utilized together with data centers to deliver Cloud services, the scale of data centers

310

X. Lu et al. / Future Generation Computer Systems 29 (2013) 309–322

could be controlled and reduced, and the effect of cost reduction could be more obvious than that of the ongoing energy saving effort for data centers. Thirdly, each computing pattern has its own suitable scenarios. For example, data center-based schemes could provide better QoS assurance for certain applications, but with limited scalability due to the cost and scale of data centers. The peer-to-peer-based schemes may offer good scalability, but its QoS provision is often unstable. It seems that a combination of different computing patterns would have the potential in tackling the scalability and utility challenges. In fact, cloud services can be composed and provided jointly by several data centers. The federation of distributed multiple data centers is indeed an important emerging method for providing Cloud services in a more cost-effective manner [7]. In this paper we present the concept of an Internet-based Virtual Computing Environment (iVCE) [8], which aims at combining data centers and other kinds of computing resources (e.g., edge servers and client PCs) on the Internet in order to provide efficient and economical Cloud services. These different kinds of Internet resources are called multi-scale resources in iVCE. Beyond data centers, iVCE provides support for multi-scale resource sharing and collaboration on Internet. It provides users (or end systems) with a transparent, all-in-one solution through virtualization and autonomization of multi-scale Internet resources. However, Internet resources with various scales usually have different characteristics and management patterns. This leads to difficulties of combining them into a unified computing environment. For example, the resources in a data center are relatively stable and its resource management pattern is typically based on the master/slave scheme. Internet edge resources have however characteristics of growth, autonomy and diversity. It is impossible to control and manage Internet resources in a global and centralized manner, and the typical management pattern is peer-to-peer based collaboration. Many problems emerge in such a multi-scale complex computing environment, e.g., how to elastically tune multiscale networked resources on demand according to the variations of applications, users and data scales; how to model and manage multi-scale Internet resources in a consistent way; how to aggregate multi-scale resources on-demand to support elastic and economical scalability of Cloud services; how to deal with the failover of multi-scale resources and data consistency; and how to enhance the self-management and dependability of the system, etc. In order to seek for an integrated solution to these problems, we propose a model for multi-scale and elastic resource management in iVCE. We design and implement the iVCE software platform to embody the model, and provide the capability of supporting the elastic and economical Cloud services. Nowadays Internet service providers have become more and more dependent on public platforms to construct and publish services, they also broadly encourage users to participate in service improvement via public platforms, which bring challenges to the service trustworthiness. Specifically, Internet applications provide continuous services to users and users also participate in the services, leading to a vague borderline between the providers and the users. There exist complicated relationships between users and computing environments. The ‘‘utility’’ characteristic results in a series of problems for trustworthiness in Cloud Computing. iVCE also tries to provide trustworthiness enhancement mechanisms for Cloud Computing, and this paper does not discuss the details of the topic. This paper is organized as follows. Section 2 introduces a model that addresses multi-scale resource management and elastic scalability in iVCE. A detailed description of the iVCE software platform is given in Section 3. Section 4 discusses the testbeds and experiments of iVCE. Section 5 gives a brief introduction of the related work. Conclusions from the design, deployment and initial experiences with iVCE appear in Section 6.

2. Model for multi-scale resource management Internet resources in different scales have special characteristics. The characteristics of data centers are much different from that of Internet edge resources, and the interaction model of resources in data centers is different from that of edge resources. The large-scale and utility characteristics of Internet applications as well as multi-scale resource characteristics have brought crucial problems for resource management in iVCE. In this section, we first give some descriptive definitions on multi-scale computing systems, and introduce models of iVCE to support multi-scale resource management and elastic scalability. Definition 1. Scalability limit. The scalability limit of a computing pattern is referred to an uppermost threshold of the system scale, beyond which the speedup of the system running in the computing pattern will decrease dramatically. The speedup could be an integrated evaluation parameter containing many aspects (such as QoS properties, cost, and performance), and it could be evaluated by benchmarks or theoretical analysis. For example, the SMP machine has no more than 16 CPUs generally and its scalability limit of SMP is about 16. Each computing pattern has its suitable scenario, and the scalability limit of different computing pattern is various. Definition 2. Multi-scale computing system. A multi-scale computing system is a distributed computing system in which there are multiple Internet resources with different scale and capability, and the computing pattern might be diverse. When the system running in one computing pattern reaches its scalability limit, it may shift to another computing pattern or use multiple computing patterns concurrently. For example, a system composed of a data center and many edge PC resources is a multi-scale computing system. The organization and interaction model of nodes in the data center is master/slave, and that of edge PC resources is peer-to-peer cooperation pattern. Definition 3. Elastic binding. If a computing system could always use suitable resources to satisfy task requirements, the system has the capability of elastic binding. Such a computing system should be able to sensor the status change of the computing environment or tasks, and dynamically bind resources to tasks accordingly. In this section, we will model the multi-scale resources in a consistent way, and present models and mechanisms to support multi-scale resource management and elastic binding of resources. 2.1. Multi-scale resource modeling The multi-scale Internet resources have characteristics of diversity. The data centers, edge servers and even client PCs might work together to support effective and efficient cloud applications. The broad differences among these Internet resources make it difficult to model and utilize multi-scale resources in a consistent way. We extend the resource virtualization model (i.e., the autonomic element (AE) [8]) in iVCE to model multiscale resources. AE encapsulates and virtualizes Internet resources with abilities of dynamic perception, autonomic decision-making and collaboration. The internal structure of an autonomic element consists of a sensor, a behavior-driven engine and an executor, as shown in Fig. 1(a). The sensor provides the perception ability to obtain the presence of the environment. It can perceive the dynamic change of the environments and the operating status of resources. The behavior-driven engine makes decisions based on the perceived environmental and resource status, and then the executor performs the action. Multi-scale resources could be modeled by the AE model in a consistent way. Resources in different scales, such as the pizzabox server, refrigerator-sized server, black-box server cluster and


(a) AE model.

(b) AE of warehouse-scale computer.

(d) AE of refrigerator-sized server.

311

(c) AE of pizza-box server.

(e) AE of black-box cluster.

Fig. 1. Multi-scale resource modeling.

warehouse-scale computers, and client PCs on the Internet, can all be modeled with the AE model, as shown in Fig. 1(b)–(e). According to the application demands, AEs can be used in different granularity and recursively. For example, there are many 1U server mainboards, racks that contain some server mainboards, and clusters that consist of some racks connected with a switch/router in a data center. And each server mainboard, rack, cluster or the whole data center can be modeled with the AE model, while the embodiment of the AE is different. For example, each server mainboard is equipped with a sensor agent, which polls the runtime status of the mainboard and the applications periodically and the AE model can be easily implemented in the server mainboard. When a cluster is modeled as AE, the sensor of the AE detects states of CPU/Memory/Disk/Network in the cluster, and the executor loads corresponding tasks. The AE provides meta-information required in resource aggregation and presents a consistent interface for accessing and interacting of heterogeneous Internet resources. Once a resource is modeled as an AE, it can be regarded as one unit in iVCE (e.g., it can be operated as one role in the iVCE programming language presented in Section 3.8), no matter whether the resource is a large data center or a small PC. The detail of the embodiment of the AE model and related mechanisms will be introduced in Section 3. 2.2. Multi-scale resource aggregation To support the on-demand aggregation of multi-scale Internet resources, the resource aggregation model, the virtual commonwealth (VC) [8], which specifies the certain scope to publish, discover and organize resources for aggregation, and provides a relatively stable view of resources involved with respect to specific applications, is proposed in iVCE. The VC model describes the closure of the module set of AEs and environments, as well as the set of basic services to support resource aggregation, as shown in

Fig. 2. An example of the virtual commonwealth.

Fig. 2. The VC shares some common points and issues with the Virtual Organization (VO) concept [9], but VC can be programmed by the language facility in iVCE. The organization and interaction model of AEs in VCs might vary from a tightly-coupled master–worker schedule pattern to a loosely-coupled peer-to-peer cooperation pattern. Fig. 3 shows examples of different interaction models in VCs. In the examples, when the resources within a cluster form a VC, AEs in the VC are the virtualization of mainboards in a cluster. In this VC, AEs are tightly-coupled, and the resource management is based on the schedule model (e.g., master–worker model). Meanwhile, when clusters in a large data center form a VC, AEs are a virtualization of clusters in the data center and they are slightly-coupled.

312


(a) A cluster as a VC with a master/worker interaction model.

(b) A data center as a VC with a publish/subscription interaction model.

Fig. 3. Examples of VCs with a variety of interaction models.

The resource interaction in this VC could be performed in the publish/subscription model. Furthermore, when a VC is formed by an alliance of multiple data centers, each AE in the VC is the virtualization of a data center and the resource interaction model in this VC could be the peer-to-peer interaction model. There might be various interaction models in a VC, and these interaction models are supported in the iVCE software platform that is presented in Section 3. The VC model is designed to support the multi-scale resource aggregation in iVCE and it could contain resources of different scales, such as the cluster in a data center, one data center, multiple data centers, or large number of resources at the edge of the Internet. Fig. 4(a) shows an example of VC for video services. In the example, there is a data center to provide video content, edge servers to act as CDN of the video service, and end

PCs which use the video service and assist in the delivery of video services in P2P model. The multi-scale resources could collaborate to provide a large-scale and elastic video service. And there are multiple interaction models (including the C/S model and the p2p model) in the VC. Furthermore, multiple VCs could be combined to form a new VC. Fig. 4(b) shows an example of an alliance of multiple VCs for video services. The resource aggregation in iVCE could be classified into two types: on-demand aggregation by dynamic binding and ondemand aggregation by collaboration. (1) On-demand aggregation by dynamic binding. On-demand aggregation by dynamic binding provides mechanisms to aggregate AEs which are interested in the same type of resources. The Join/Adapt semantics is proposed in iVCE to bind


313

(c) An alliance of data centers as a VC with a peer-to-peer interaction model. Fig. 3. (continued)

(a) A VC for video services with a variety of AEs.

(b) An alliance of multiple VCs. Fig. 4. Examples of multi-scale resource aggregation in iVCE.

AEs and VC dynamically. An AE can join a VC to get the capability with respect to the behavior rules, and it will adapt itself according to its environment and internal states of the meta level. As shown in Fig. 5, in a VC, an AE can take the following four actions to react to the environment: join, quit, suspend and resume. An AE can use the join action to bind to one or several roles to operate on resources. Once bound to a role, it acquires the ability to operate on the corresponding resources. An AE can use the quit action to abandon the ability and release allocated resources. Moreover, an AE can use suspend/resume actions to dynamically adjust it usage

on the resource between active and inactive states. An AE can operate on the resource when it is active, otherwise it can only access the metadata of the resource. (2) On-demand aggregation by collaboration. On-demand aggregation by collaboration provides mechanisms to associate related AEs to accomplish tasks regarding a specific resource. Internet applications usually involve a large number of distributed nodes. Since a task can usually be broken down into a hierarchy of sub-tasks, the interactions among these nodes are usually hierarchical, rather than flat. For instance, in a peer-to-

314


status of the tasks, resources and the computing environment, and then makes a decision as to which resources the VE should be dynamically bound to. Fig. 8(a)–(b) show an example of elastic binding in video applications across multiple VCs. 2.4. Discussions

Fig. 5. On-demand aggregation by dynamic binding.

peer file sharing application, the nodes requesting the same file are associated together, and those working on the same chunk of the file are further associated. On-demand aggregation by collaboration uses conversations to associate related AEs at each level. A conversation has an application provided identifier, and it can contain sub-conversations, thus forming a hierarchy. Events can only propagate through the conversation hierarchy, as shown in Fig. 6. Hierarchical conversations can not only describe complex interactions efficiently, but also facilitate the maintenance of interaction protocols by providing hierarchical state management. Both of two types of resource aggregation are supported in the iVCE programming language, resource aggregation service and resource collaboration service which are presented in Section 3. The multi-scale resource aggregation mechanisms are also presented in Section 3. 2.3. Elastic binding of resources In the design of iVCE applications, there are many applications roles and these roles are dynamically bound to appropriate AEs in the runtime [10]. The Virtual Executor (VE) model [8], which refers to the state space combined with AEs involved in a task, is used in iVCE to support the elastic binding of multi-scale resources. As there might be many tasks running concurrently in iVCE, it needs to select corresponding AEs based on the task requirement and bind AEs to tasks dynamically. One task may require several AEs to provide services, and one AE may serve for several tasks. The VE contains the state space combined with AEs involved in a task (like the process model in the operating system), and thus we can get all the management information of the task, such as AEs involved in the task and their interaction states. Fig. 7 shows an example of the snapshot of VE in a Map/Reduce application in a VC. The dashed line in the figure includes the AEs participating the applications (some AEs act out the role of a Mapper, and some AEs act out the role of a Reducer). Resource elastic binding in iVCE is constructed by three sequential processes: perception and detection, evaluation and decision, and finally, dynamic configuration and binding. When iVCE detects that the resource requirement of a task is changing or new suitable resources becomes available at runtime (with services presented in Sections 3.3 and 3.5, etc.), it evaluates the

The iVCE tries to combine data centers and other distributed resources on the Internet to support cloud computing, which is a promising way to provide efficient and economical cloud services. However, as there are different kinds of resources in the computing environment, it also brings new challenges for the resource management and the trustworthiness of the computing environment. As mentioned in Section 2.2, the iVCE uses a multi-scale resource aggregation model to manage various resources. Tightlycoupled master–worker models and loosely-coupled peer-to-peer cooperation models are both provided by iVCE. Resources in data centers could be organized in a master/slave model, while resources out of data centers could be organized in a selfmanagement P2P overlay [11] by the resource aggregation service (in Section 3.4) in iVCE. The FissionE [11] P2P overlay protocol in the resource aggregation service is used to deal with the dynamic Join/Leave of Internet resources. When resources fail, the FissionE protocol adapts the overlay and invokes fault-tolerant mechanisms accordingly. The resource collaboration service (in Section 3.5) in iVCE is performed in a distributed event pub/sub model and it can also improve the reliability of the system. Replication is a common mechanism used in the basic services in iVCE (in Section 3) to improve system reliability. In a word, iVCE tries to uses aggregation and collaboration mechanisms to provide a relatively stable logic resource view for applications, no matter what kind of Internet resources are used. Trustworthiness is an important problem as there are various kinds of resources in the iVCE system, which may belong to one or multiple organizations. If these resources are from one organization (e.g., one Internet company with data centers and CDN) or multiple organizations with a trust relation (e.g., business companies with contracts), trustworthiness is not a big problem. And iVCE provides some fundamental mechanisms to enhance the trustworthiness. When resources join in iVCE, they should get permission from the iVCE system. Though the current version of the iVCE software platform did not provide strict security management, identification authentication, authorization, account as well as access control mechanisms could be utilized in the future. After a node joins in iVCE, it downloads the basic container of the iVCE software platform, which uses virtualization mechanisms to provide secure isolation of the iVCE runtime software from other applications on the node. The behavior of the nodes participating in iVCE is monitored by the container, and some distinct malicious behaviors could be prohibited. The

S

S

S S

P

S

AE

P

AE (event publisher)

S

S

Conversation S

S Event propagation

Fig. 6. Hierarchical conversations.


315

Fig. 7. The snapshot of VE in a Map/Reduce application.

Fig. 8. Snapshot sequence of VE for video services across VCs.

replication and backup in iVCE could also reduce the influence of malicious behaviors. However, security and dependability assurance is still an open problem in iVCE when many resources are idle individual resources on Internet. We are making efforts to design sandbox security, encryption/decryption, Byzantine faulttolerance and other mechanisms to improve the trustworthiness of the iVCE system. 3. iVCE software platform To support multi-scale resource management and elastic scalability, we have implemented the iVCE software platform v1.0. The basic objective of the software platform is to provide support for the up-rising large-scale and utility Internet services, which is somehow close to the motivation of some grid, peer-to-peer, and especially cloud computing platforms/systems. In contrast with these platforms, the iVCE software platform has provided a unified framework for the aggregation and collaboration of multi-scale Internet resources. It not only supports large scale and highly organized resources such as cluster and data centers, it also emphasizes the support to the massive decentralized resources

across the Internet which could be aggregated under the P2P paradigm. By leveraging the characteristic of different computing models, the iVCE software platform provides many pervasive services, and the implementation of Internet applications built upon this platform will be decoupled with underlying resource management and communication mechanisms. Based on the iVCE platform, developers can focus their work on the business logic and pay less attention to the details of basic interaction of networked resources. In a word, the iVCE software platform establishes a unified framework for efficient aggregation and collaboration of multi-scale Internet resources. The structure of the iVCE software platform can be illustrated by Fig. 9. It is composed by a series of iVCE fundamental services, three tools (including virtual network memory, virtual network storage and virtual cluster), an iVCE programming language as well as the runtime management. The software platform will be described in detail as follows. 3.1. Virtual node service To support multi-scale resource virtualization, we have designed and implemented the virtual node service in the iVCE plat-

316


Virtual Virtual Network Mem Network Storage

Virtual Cluster Runtime Management

Programming Language

Internet Applications

Resource Collaboration Service Resource Aggregation Service Net. Delay Estimation

Virtual Network

Virtual Node

Internet Resources

remote nodes. VirNet is a completely user space implementation and does not require modification to any kernel part of the operating system. Experiments show that VirNet is efficient. The average latency overhead of VirNet for a single hop is less than 0.4 ms and the average throughput is more than 85% of the physical network’s bandwidth. 3.3. Network delay estimation service

form, which is an embodiment of the Autonomic Element (AE) model. The virtual node module is a light-weight virtual machine monitor [12,13] for the iVCE software platform, which can virtualize various resources and provide a secure isolation of the iVCE runtime software. The key points in the virtual node include the volume snapshot based local virtualization, the optimization with the dynamical instruction translation engine, dynamical physical memory allocation, and so forth. Such techniques provide a system-level isolated execution environment with the characteristics of transparency and security. The performance impacted is minimized, as well as the host operating system image is precisely copied for the reappearance of the environment in a host system. Since some of the Internet resources are autonomous, applications can run on the virtual nodes without compromising performance and security. In the implementation of virtual node module, we solve the problem of data conflict between the shared host and the virtual file system using copy-on-write, thus the host operating system image can be accessed by the virtual node without data pollution. The problem of the hardware difference between the host and the virtual node can also be solved by merging the drivers and configuration files into the shared file system. The memory management is optimized by the mapping from the virtual address in a virtual node to the physical address in the host system. The I/O virtualization is also implemented by the emulation of the operations on the I/O ports.

To support elastic binding of resources and provide efficient platform services, there are great requirements for iVCE to be aware of underlying dynamic changes of the Internet environment. According to the latency-sensitive characteristics of many Internet applications, we study and design a network delay estimation service for iVCE, consisting of a delay estimation module and a K nearest neighbor search module. The delay estimation module includes network delay computation, network coordinate update, network coordinate computation, network delay measurement and neighborhood management. The K nearest neighbor search module includes backtracking search, nearest neighbor search, concentric ring management and clustering management. The implementation of the network delay estimation service for iVCE is explained in detail as follows. The delay estimation module provides fast and accurate network delay estimation between target nodes based on novel network coordinate techniques [15]. Specifically, the delay estimation module embeds the Internet network into a geometric space, where each node in the Internet network is mapped to a point in the geometric space, and the point position is computed according to network delays between two corresponding nodes in a decentralized manner. As a result, internode network delays are estimated quickly and accurately based on geometric distances. The K nearest neighbor search module finds K nearest nodes in a global perspective for any target node in ascending order of delays according to pairwise physical network delays. The K nearest neighbor search module assigns a concentric ring to each node which is made up of several close nodes based on pairwise physical network delays, and quickly locates K nearest nodes for any target node by a distributed recursive search process over the concentric rings.

3.2. Virtual network service

3.4. Resource aggregation service

Elastic binding of resources requires efficient communication and cooperation among virtual nodes. The heterogeneity (e.g., NAT and firewall) of the physical network connections of multi-scale resources has brought many challenges (e.g., asymmetric communication, self-configuration problem) to the communication of virtual nodes. Meanwhile, different applications on the same platform require the support of isolated virtual networks. To solve these problems, we proposed scalable virtual network technique called VirNet [14], which can support multiple customized and isolated virtual networks in the same physics network environment. By using the virtual routing and network virtualization techniques, VirNet is able to hide the heterogeneity and asymmetry of the physical network connections, and provide distributed applications their desire transparent network environments, such as a specific virtual IP address space and customized IP network with symmetric connections. Existing LAN applications could be run seamlessly on VirNet without any modification. And such a virtual network can benefit not only from existing applications, but also new applications, because application developers do not need to deal with complex network environments any longer and they can focus on the application logic. The VirNet captures the packets in the virtual network, and implements the NAT and firewall penetration by the initiating connection of NATed nodes and caching the NAT information of

To support the aggregation of multi-scale resources, the iVCE software platform needs to use mechanisms to organize the large-scale dynamic Internet resources into a relatively stable logic resource view, and provide such basic functions as resource publication and discovery. Based on the logic resource view, the development of applications running on the iVCE software platform could focus on the application logic and does not need to care for the status of the dynamic multi-scale resources. Therefore, we design the resource aggregation service, which could organize the large-scale dynamic Internet resources effectively, to deal with the dynamic join/departure of resources and provide reliable message routing capability as well as resource publication and discovery functions. Both master–worker and P2P resource management mechanisms are provided in the resource aggregation service. In the aspect of resource aggregation mechanisms, we have proposed a constant-degree and high performance DHT P2P overlay based on the Kautz graph (called FissionE) [11], a delay-bounded range query scheme for DHT overlay (called Armada) [16], and a universal topology maintenance technique for arbitrary constant-degree DHT overlay (called DLG) [17], etc. The P2P overlay can deal with the dynamic join/leave/fail of the nodes. These mechanisms are used in the resource aggregation service to construct the routing table of nodes in the overlay, deal with

Fig. 9. The iVCE software platform.


the dynamics of resources, and implement the fault-tolerance to satisfy the efficiency and reliability requirements of the resource aggregation. Resource information has two replicas and faulttolerant routing is implemented to improve the reliability of the system. To enhance the scalability of the software platform, we divide the resource aggregation service into the transport layer and the overlay layer. The transport layer is responsible for the implementation of network communication among nodes, and the overlay layer can use any protocol in the transport layer. By dividing two separate layers, the software could not only use FissionE overlay protocol, but also be easily extended to use other centralized or decentralized protocols. 3.5. Resource collaboration service Elastic binding of resources require the support of collaboration of multi-scale resources. In order to support the flexible collaboration of multi-scale resources, we design the iVCE resource collaboration service, which uses distributed event pub/sub mechanisms and hierarchical conversation management in effectively describing the dynamic complex collaboration relationship to support loosely-coupled collaboration. The resource collaboration management service includes the event publication module, subscription registration module, event matching and delivery module, callback function registration and execution module and routing path reconfiguration module. An application registers its interest via subscription, each subscription contains the restrictions of events and callback function. When a matching event is found, it is delivered to the event service, after that it will invoke the callback function. The delivery of subscription is a content-based P2P routing approach [18]. The subscription is routed according to the hash value of some part of the subscription, and it is stored persistently in a destination node and two replication nodes. The delivery of events resembles the delivery of subscriptions, the events will be matched against subscription stored in the nodes and delivered to the subscribed application if matched. When some nodes leave or fail, the subscription is replicated to another suitable node by the underlying P2P overlay, and the event will be forwarded to the new destination node by the P2P routing approach. 3.6. Virtual network memory service Virtual network memory service is an important tool to support the elastic scalability of memory capacity in the iVCE system. The memory resource sharing is important but also special. On one hand, the characteristics of Internet resources as growing, autonomy and diversity baffle the traditional memory sharing approaches. On the other hand, compared with the storage, memory resource also emerges its specialty, which makes the memory sharing much more challenging. The virtual network memory service [19] in iVCE tries to balance the dynamics in the global view and the statics in the local view with certain memory resource management policies. It can leverage the distributed idle memory resources effectively, and improve the system performance. Furthermore, we also designed and implemented several optimization schemes [20] for the virtual network memory service, such as the stripping of data to boost the potential parallel transmission, and the perfecting and pushing with the user’s access pattern. The Internet resources are usually heterogeneous, e.g., the size of the memory page which is supported by the hardware may be different, and each node may have different local management policies. The service encapsulates the heterogeneous memory resources into a uniform unit through the AE and the heterogeneity

317

can be shielded by the memory services provided by the AEs. The implementation of a virtual network memory service will detect the memory usage in each node, and decide the memory amount to be shared automatically. The memory resources organization and management is implemented upon the iVCE resources aggregation service, the memory space distributed among multiple nodes can thus be linked together as a unitary logical space in the user’s view. The network delay estimation service in iVCE is also used to guarantee the network transmission overhead between the memory resource and the user is bounded. The problems of the dynamics and volatility of memory resource is handled with data redundancy, the data replica will be dispatched on multiple nodes, as well as the local disk, thus the system can be recovered from the data lost when necessary. Upon the memory service, we also implement the VMM-based remote virtual memory mechanism, which is transparent to the applications running in the virtual machine. When an application bursts a large memory requirement, our system can automatically aggregate available memory distributed in other nodes to satisfy the application. Such elastic features of virtual network memory service are fit for the Internet applications with a variable number of users. Indeed, some of the running services in AliCloud Inc (a subsidiary of the Alibaba Group) encounter the problem of dynamic and unpredictable memory allocation, which lags the system performance and thus degrades the service quality. We are currently cooperating with AliCloud Inc to deploy a virtual network memory service to solve the problem. 3.7. Virtual network storage service Virtual network storage service is a tool to support largescale and scalable data storage. It attempts to provide users with high-capacity, high data transfer performance, low-cost and more user-friendly storage services, and to improve storage resource utilization through effective aggregation of storage resources on the Internet. The virtual network storage service includes a metadata catalog module, storage server module and client module, which can provide efficient storage services such as upload and download files, as well as delete files. Replicas are used in the service to improve the reliability and data access performance. In the virtual network storage service, an adaptive parallel data access algorithm [21] and a fast adaptive data dissemination algorithm [22] which are based upon delay optimization are designed and implemented. In the first algorithm, different data blocks of the target data are concurrently read from multiple data replicas which are selected according to the network delay, and multiple network links are selected to improve network resource utilization as well as to reduce the access delay. In order to optimize data access performance, the blocks which are read from multi-replicas can be adjusted during the process of data access according to the network dynamics. In the latter algorithm, to achieve the goal of balanced distribution of the file replicas, multiple target nodes are selected from all the available storage nodes according to the cluster information of nodes. Based on the network delay, the data dissemination paths are determined, and the single-source data dissemination are transformed to the parallel multi-source data dissemination, multiple network links are selected to improve the efficiency of data dissemination. In order to optimize the performance of the data dissemination, data dissemination paths are adaptively adjusted during the process of data dissemination according to the network dynamics. 3.8. Programming language and runtime management In order to effectively support the concept model of iVCE, we studied the programming language aspect to implement the AE and VC. An experimental programming language called Owlet

318


Fig. 10. The concept model of Owlet.

[10,23], which is an interactive language based on the peer-to-peer content-based publish/subscribe scheme, is proposed to develop new Internet applications on the iVCE platform. Owlet uses AEs to represent resources on the Internet, and characterizes the AEs into different roles of interactions for resource aggregation. The AEs are organized in a peerto-peer manner, which are capable of adapting to failures and accommodating transient populations of resources, while maintaining acceptable connectivity and performance. Unlike the existing programming languages for building distributed systems, which typically use point-to-point message passing as their communication primitives, Owlet uses a content-based event publish/subscribe interaction scheme, which is claimed to provide the loosely coupled form of interaction required in large-scale settings. An event is asynchronously propagated to all subscribers that registered interest in that given event. This eventbased interaction style brings full decoupling in time, space, and synchronization between publishers and subscribers [24]. Owlet separates the concerns of constructing distributed systems into the resource aggregation logic and the business logic. The aggregation logic is specified by the interactions among roles to collect and organize available resources. Each role contains rules that guard on certain events to take some actions. The actions may be the publishing of another event, or invoking the business logic. By using roles to characterize the underlying resources, programs can aggregate resources by interacting with the corresponding role without the knowledge of individual AEs that represent the resources. Although an individual may be transient due to the autonomic and dynamic nature of the resource it represents, their role as a whole forms a relatively stable view of the resources. The Owlet runtime is responsible for selecting individual resources in a proper way, e.g. with load balancing and fault tolerance. Once the resources are selected, Owlet delegates the business logic to the existing programming languages and component technologies, which are well suited for processing resources in a determined context. Owlet provides language constructs that (1) use distributed event driven rules to describe interaction protocols among different roles to facilitate collecting, organizing, as well as utilizing of resources on the Internet, (2) use conversations to correlate events and rules into a common context to better specify the interactions in a hierarchical way, and (3) use resource pooling to do fault tolerance and load balancing among networked nodes based on the event publish/subscribe interaction scheme. With these constructs, large-scale distributed systems can be elegantly

built with a clear separation of resource aggregation logic and business logic. Owlet specifies the interactions by using roles as the first class entity, because roles can give programmers a relatively stable view of the resources by representing the AEs as a whole, while individual resources may be transient. The relationship between roles and AEs are dynamically bound in Owlet, i.e. an AE can join and quit a role at runtime either under user request, or by its rules. Fig. 10 shows the concept model of an Owlet program, which defines a set of resource metadata schema and involves several roles to describe the interactions. Owlet uses the XML data model to represent states, variables and events. It supports primitive types such as Boolean, integer, floating point number and string. Composite types and list types can be structured hierarchically based on these primitive types. Owlet expressions use a subset of XPath expressions. The resource metadata schema defines data types for describing the properties of the resources. They will be used by the roles to define states, variables and events in the interactions. Roles are composed of three parts: state variable declarations, interaction rules and component references. The rules describe patterns of interested events and corresponding actions to execute when an appropriate event is triggered. Rules can spawn conversations that further correlate events and rules into a common context. The local operations on resources are delegated to the components. We have implemented an Owlet compiler and its runtime environment [10], and built several Owlet applications, including a peer-to-peer file sharing application and distributed web page crawling. 4. Experiments Based on the iVCE software platform, we have carried out a series of experiments to justify our concepts and the software platform. Besides that, some representative existing applications were deployed on the platform to verify the effectiveness. In this section, we will briefly introduce our different experimental environments, the experiments running under the environments and the results of the experiments. 4.1. Experimental environments For better justifying the techniques of iVCE through experiments, we have designed and built the following three experimental environments:


319

4.2. Experimental results We have deployed the iVCE software platform to the preceding three experimental environments for a series of experiments. Firstly, we evaluate the performance of each basic service of the platform, and part of the results are presented as below:

• Experiments of the network delay estimation service. We have

Fig. 11. Emulational experimental environment in LAN.

deployed the network delay estimation service on over 500 nodes of PlanetLab. Extensive measurement experiments show that the service can quickly provide accurate delay estimation and approximately optimal K nearest neighbor search results with a lower communication overhead [15]. • Experiments of virtual network. We have deployed our virtual execution network VirNet on the emulational experimental environment in LAN, and also OpenVPN [26]. The experiment results show that the bandwidth supported by VirNet virtual network is approximate to that of the physical network, and better than that of OpenVPN. In addition, the network delay of using VirNet virtual network is lower than that of using OpenVPN, and the scalability of VirNet is also better under concurrent commutations [14]. Experiments showed that these services are efficient and elastically scalable. The details of the evaluation of individual services can be referred to in papers [12–15,11,16–18,21,22,19,20]. Secondly, we designed and implemented some applications on the platform with the Owlet programming language, including:

• Distributed file sharing: an Owlet program for a peer-to-

Fig. 12. Site map of the WAN testbed.

• An Emulational experimental environment in our local area network (LAN), which is shown in Fig. 11. There are three well-managed network domains in the environment. Each domain contains about 30 nodes with different configurations, and the nodes are connected by a gigabit network. For simulating the real cases in an Internet environment, the three domains are connected by a network simulator for network impairment. In addition, this environment can be connected to outside experimental environments through the Internet.

• A WAN experimental testbed. Fig. 12 shows the site map of the environment. This testbed contains more than 30 sites (about 500 nodes) spanning the main ISPs of China. Four sites have about four hundred rack servers, and each of the other sites spread in thirtyone provinces in China has several rack nodes. To satisfy different application requirements, mainstream operating systems are installed on the nodes in the environment, including RedHat Linux, Windows 2003 Server, etc. At the same time, this environment can be connected to the international experimental environments.

• An experimental environment built on PlanetLab [25]. PlanetLab is a group of computers available as a test bed for computer networking and distributed systems research. PlanetLab contains 1090 nodes at 508 sites now [25]. We have joined PlanetLab and carried out some experiments by using the nodes distributed all over the world.

peer file sharing application in a distributed environment. The program contains about 140 lines in Owlet, and the underlying business logic component is very straightforward to implement. The program can be easily deployed and executed without any central control. Experimental results show that the separation of resource aggregation logic and business logic significantly eases the construction of large-scale distributed systems. Experiments also show that Owlet programs can achieve a high level of fault tolerance and a certain degree of load balancing even when individual resources are fairly unstable. • Distributed prime number checking: a typical scientific application implemented in Owlet. There are two roles in the application, the Checker role and the Client role. Clients submit tasks to the Checker to check whether a number is prime. We deployed the application in our emulational experimental environment in LAN and WAN, and the experimental results show that it can easily achieve fault tolerance and load balancing among distributed nodes. • Distributed web page crawling. We have rewritten a distributed web page crawling system in Owlet. The original system was implemented by Java and is about 11 000 LOC, in which around 3000 LOC are for commutation. By using Owlet, we only wrote 160 LOC for implementing the same communication protocols, and reused the original code for web page crawling. We have deployed the Owlet version system on our WAN experimental testbed, and the amount of work for deployment is significantly reduced. From the applications’ experiments above, it can be referred that Owlet language can largely reduce the amount of work for implementing the communication protocols in a distributed system. Experiments also show that an application running on the iVCE platform is efficient and easy to deploy. For example, in the experiments of the distributed web page crawling application, we deployed the application on 180 nodes in our WAN experimental testbed across three main ISPs of China. Many widely distributed nodes were used to crawl web pages on the Internet, and the

320


Fig. 13. Node map of the distributed web page crawling on PlanetLab.

application can dynamically adjust the crawling nodes according to the status of crawling tasks. In addition, we deployed the application to more than 240 nodes in PlanetLab to test its applicability in a worldwide environment. The deployment of the application is very fast, and we can start the application by using the nodes all over the word in a few minutes. Fig. 13 shows a node map of running the application on PlanetLab. Thirdly, we have tested the performance and scalability of existing applications running on the iVCE platform. A virtual cluster service based on the virtual network has been designed and implemented to create virtual clusters to support existing applications. The scale of the virtual clusters created by the service can be adjusted dynamically according to the task requirements. A variety of experiments and existing scientific programs were run on the virtual cluster in the iVCE software platform. The service was deployed on the emulational experimental environment, and some existing scientific MPI programs (e.g., Fire dynamic simulator (FDS) and, Biological information program) have been successfully run without any modification on the virtual clusters, which are created and managed through the iVCE software platform. For example, FDS is a application developed by NIST [27] to simulate fire and it emphasizes on simulating smoke and heat transport from fires [28]. We deployed the MPI version of FDS to a 5-node virtual cluster, and each virtual node has 512 MB memory and one 2 GHz CPU. The program can be run successfully on the virtual cluster. Fig. 14 shows a snapshot of FDS running on a virtual cluster. The node number of the virtual cluster can be dynamically adapted to the applications requirement. We are deploying a large-scale video service running on the iVCE software platform in our WAN experimental testbed. The video content is stored on one data center site with more than one hundred rack servers in the testbed, and more than 30 sites spread across China in the testbed provide collaboration support for the video service. User behavior is simulated by the software running on the nodes in both the emulational experimental environment and the WAN testbed. This experiment is trying to demonstrate the case that iVCE uses multi-scale resources to provide cloud services. We would like to perform this experiment for a long period (one or two years) and collect a large amount of statistics data to improve the design of iVCE. From these three kinds of experiments above, it can be inferred that the iVCE software platform can provide effective support for developing large-scale new Internet applications. Also, it can

Fig. 14. A snapshot of FDS running on a virtual cluster.

support efficient execution of existing large-scale applications. The capability of the core services (e.g., virtual cluster, virtual network memory) of the iVCE platform can scale up/down flexibly according to the application requirements. 5. Related work Grid computing [9] was proposed in the middle of the 1990’s, and its goal is to integrate various geographically-distributed resources from different organizations to provide integrated services for users. In the past ten years, many projects for grid computing have been launched, such as TeraGrid, DataGrid and e-Science, to explore the methods to build a new computing environment across multiple organizations [9,29–33]. As the virtual machine technology becomes increasingly popular, it is used for grid resource virtualization at the system level [31,32] to accelerate the seamless expansion of current existing application software to grid environments. This further promotes the development of grid computing. The idea of resource sharing on the Internet in the iVCE is similar to that of grid computing or cloud computing [1]. And iVCE tries to extend the capability of network computing (e.g., grid computing) by using techniques developed recently (e.g., virtualization, data center and P2P, etc.). The iVCE attempts to combine data centers and edge resources to provide elastic and economical Internet services. And multi-scale resource sharing and elastic resource binding are the new challenging problems for iVCE. The research of


the iVCE tries to bring a theoretical breakthrough to the above challenging problems, and its results will accelerate the research of grid computing and other network computing technologies. Meanwhile, the iVCE also tends to share resources among multiple grid systems to realize resource aggregation and collaboration in the broad scope. Cloud computing is a popular commercial model for resource sharing [1,34,35]. Since 2007, it has become increasingly popular and has received broad attention from both academy and industry. Compared with traditional technologies, current cloud computing has three new features [1]: The infinite computing resources available on demand and quickly enough to satisfy load surges, the elimination of resource commitment allowing people to add resources as needed, and the ability to fine-grain resource allocation and release as needed. Cloud computing brings great challenges to traditional network computing technologies [1,35,36]. For example, Google has deployed more than 30 data centers containing millions of servers [37–39]. To organize and manage such large scale resources, highly scalable data center architectures in a centralized manner are designed for cloud computing, which can dynamically add new servers to the existing server set or temporarily discard faulty nodes according to a certain policy to achieve a trade-off between scalability and practicality on cheap low-end nodes. Google also proposes the MapReduce programming model [38], aiming at Google application-specific problems such as data accessing and processing, automatic task division and resource allocation. Large-scale data center [40] has become the popular technique to support cloud computing. In a sense, the data center hardware and software is what is called a cloud [1]. In order to achieve elastic and infinite capacity available on demand, cloud computing systems require automatic resource allocation and management. For example, Amazon EC2 [4] provides virtualized machine instances, which look much like physical hardware. Users can control nearly the entire software stack. Google AppEngine [2], which is targeted at traditional Web applications, requires its applications to explicitly distinguish the stateless computation tier and the stateful storage tier. Microsoft’s Azure [5] enforces its applications to be written using the.NET libraries and compiled to CLR. Cloud computing technology is tightly related to grid concepts and it has much to offer to grid infrastructures [33]. Current cloud computing systems are mainly centralized controlled where the design of data centers is based on the view of a totally controlled computer. Inspired by recent successes in CDNs [41] and media streaming [42], iVCE is a promising direction for cloud computing to go beyond the data center and turn to more resources distributed in the Internet, exploiting a multi-scale networkcentric resource sharing model to provide efficient and economical services. 6. Conclusions The large-scale and utility characteristics of new Internet applications have brought many challenges to Cloud Computing. In this paper, we have introduced the Internet-based Virtual Computing Environment (iVCE) that aims to provide Cloud services by a dynamic combination of data centers and other multiscale computing resources on the Internet. Model for multi-scale resource management are proposed to model the multi-scale resources in a consistent way and address multi-scale resource management and elastic scalability. The iVCE software platform that embodies the model has been designed and implemented, and it has been deployed and tested on testbeds. Comprehensive experiments show that iVCE provides a novel, promising way to support economical and elastic Cloud Computing.

321

Acknowledgments This work is supported in part by the National Basic Research Program of China (973) under Grant No. 2011CB302600 and the Foundation for the Author of National Excellent Doctoral Dissertation of PR China (FANEDD) under Grant No. 200953. We are grateful to all members of the 973 project ‘‘Fundamental Research on Highly Efficient/Effective and Trustworthy InternetBased Virtual Computing Environments’’ for discussions on the topics covered here. References [1] Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, Matei Zaharia, A view of cloud computing, Communications of the ACM 53 (4) (2010) 50–58. [2] Google App Engine, 2010. http://code.google.com/intl/zh-CN/appengine/. [3] IBM BlueCloud, 2010. http://www.ibm.com/ibm/cloud/. [4] Amazon EC2, 2010. http://aws.amazon.com/ec2/. [5] Microsoft Azure, 2010. http://www.microsoft.com/windowsazure/. [6] Luiz André Barroso, Urs Holzle, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Morgan & Claypool, 2009. [7] Sharad Agarwal, John Dunagan, Navendu Jain, Stefan Saroiu, Harbinder Bhogan. Alec Wolman, Volley: automated data placement for geo-distributed cloud services, in: Proc. of NSDI’10, 2010. [8] Xicheng Lu, Huaimin Wang, Ji Wang, Internet-based virtual computing environment(iVCE): concepts and architecture, Science in China Series F: Information Sciences 49 (6) (2006) 681–701. [9] I. Foster, C. Kesselman, S. Tuecke, The anatomy of the grid: enabling scalable virtual organizations, International Journal of Supercomputer Applications 15 (3) (2001). [10] The Owlet Programming Language. http://owlet-code.sourceforge.net. [11] Dongsheng Li, Xicheng Lu, Jie Wu, FISSIONE: a scalable constant degree and low congestion DHT scheme based on Kautz graphs, in: Proc. of INFOCOM 2005, 2005, pp. 1677–1688. [12] Y. Wen, H. Wang, A secure virtual execution environment for untrusted code, in: Proc. of 10th International Conference on Information Security and Cryptology, 2007, pp. 156–167. [13] Y. Wen, J. Zhao, H. Wang, A novel approach for untrusted code execution, in: Proc. of 9th International Conference on Information and Communications Security, 2007, pp. 398–411. [14] Dongsheng Li, Yixin Chen, Feng Huang, Creating virtual networks for distributed virtual computing environment, in: Proc. of 2009 IEEE International Workshop on Internet-Based Virtual Computing Environment, iVCE-09, Shenzhen, China, December 8–11, 2009. [15] Yongquan Fu, Yijie Wang, HyperSpring: accurate and stable latency estimation in the hyperbolic space, in: Proc. of 2009 IEEE International Workshop on Internet-based Virtual Computing Environment, iVCE-09, Shenzhen, China, December 8–11, 2009, pp. 864–869. [16] Dongsheng Li, Jiannong Cao, Xicheng Lu, Kaixian Chen, Efficient range query processing in Peer-to-Peer systems, IEEE Transactions on Knowledge and Data Engineering 21 (1) (2009) 78–91. [17] Yiming Zhang, Ling Liu, Dongsheng Li, Xicheng Lu, Distributed line graphs: a universal framework for building dhts based on arbitrary constant-degree graphs, in: Proc. of ICDCS 2008, 2008, pp. 152–159. [18] Shendong Zhang, Ji Wang, Rui Shen, Jie Xu, Towards building efficient contentbased publish/subscribe systems over structured P2P overlays, in: Proc. of the 39th International Conference on Parallel Processing, ICPP’10, San Diego, CA, USA, Sep. 13–16, 2010. [19] R. Chu, N. Xiao, Y. Zhuang, et al. A distributed paging RAM grid system for widearea memory sharing, in: Proc. of 20th International Parallel and Distributed Processing Symposium, Rhodes Island, Greece. 2006. [20] R. Chu, N. Xiao, L. Chen, et al. A push-based prefetching for cooperative caching RAM Grid, in: Proc. of 13th International Conference on Parallel and Distributed Systems, Hsinchu, Taiwan, China, 2007. [21] Xiaoming Zhang, Yijie Wang, Research of routing algorithm in hierarchyadaptive P2P systems, in: Proc. of Fifth International Symposium on Parallel and Distributed Processing and Applications, ISPA2007, Niagara Falls, Ontario, Canada, August 29–31, 2007. [22] Zhong Zheng, Yijie Wang, SemanticCast: content-based data distribution over self-organizing semantic overlay networks, in: Proc. of 11th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT-10, IEEE Press, Wuhan, China, 2010. [23] Rui Shen, Ji Wang, Shengdong Zhang, Siqi Shen, Pei Fan, A framework for constructing peer-to-peer overlay networks in Java, in: Proc. of PPPJ 2009, 2009, pp. 40–48. [24] P. Eugster, P. Felber, R. Guerraoui, A. Kermarrec, The many faces of publish/subscribe, ACM Computing Surveys 35 (2) (2003) 114–131. [25] PlanetLab. 26th July, 2010. http://www.planet-lab.org/.

322


[26] Open Source VPN. http://openvpn.net/. [27] National Institute of Standards and Technology. http://www.nist.gov/. [28] Fire Dynamics Simulator and Smokeview, FDS-SMV. http://www.fire.nist.gov/fds/. [29] Vladimir V. Korkhov, Valeria V. Krzhizhanovskaya, P.M.A. Sloot, A Grid-based virtual reactor: parallel performance and adaptive load balancing, Journal of Parallel and Distributed Computing 68 (5) (2008). [30] M. Birkin, A. Turner, B. Wu, P. Townend, J. Arshad, J. Xu, MoSeS: a grid-enabled spatial decision support system, Social Science Computer Review 27 (2009) 493–508. [31] Sumalatha Adabala, Vineet Chadha, Puneet Chawla, From virtualized resources to virtual computing grids: the In-VIGO system, Future Generation Computer Systems 21 (6) (2005). [32] Stephen L. Scott, Geoffroy Vallée, Thomas Naughton, Anand Tikotekar, Christian Engelmann, Hong Ong, System-level virtualization research at Oak Ridge National laboratory, Future Generation Computer Systems 26 (3) (2010) 304–307. [33] Constantino Vázquez, Eduardo Huedoa, Rubén S. Monteroa, Ignacio M. Llorente, On the use of clouds for grid resource provisioning, Future Generation Computer Systems 27 (5) (2011) 600–605. [34] R. Buyya, C.S. Yeo, S. Venugopal, Market-oriented cloud computing: vision, hype, and reality for delivering IT services as computing utilities, in: Proc. of 10th IEEE International Conference on High Performance Computing and Communications, 2008. [35] B. Hayes, Cloud computing, Communications of the ACM 51 (7) (2008) 9–11. [36] M.D. Assuncao, A. Costanzo, R. Buyya, Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters, in: Proc. of 18th ACM International Symposium on High Performance Distributed Computing, 2009. [37] L. Andr, J.Dean. Barroso, Web search for a planet: the google cluster architecture, IEEE Micro 23 (2) (2003) 22–28. [38] J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, Communications of the ACM 51 (1) (2008) 107–113. [39] F. Chang, J. Dean, S. Ghemawat, W.C. Hsieh, D.A. Wallach, M. Burrows, T. Chandra, A. Fikes, R.E. Gruber, Bigtable: a distributed storage system for structured data, in: Proc. of the Seventh Symposium on Operating System Design and Implementation, 2006. [40] D. Patterson, Technical perspective: the data center is the computer, Communications of the ACM 51 (1) (2008) 105–105. [41] Wenjie Jiang, Rui Zhang-Shen, Jennifer Rexford, Mung Chiang, Cooperative content distribution and traffic engineering, in: Proc. of NetEcon’08, 2008. [42] Yan Huang, Tom Z.J. Fu, Dah-Ming Chiu, John C.S. Lui, Cheng Huang, Challenges, design and analysis of a large-scale P2P-VoD system, in: Proc. of Sigcomm 2008, 2008.

Xicheng Lu received his B.Sc. degree in computer science from Harbin Military Engineering Institute, China, in 1970. He was a visiting scholar at the University of Massachusetts between 1982 and 1984. He is now a professor at the College of Computer, National University of Defense Technology (NUDT), China. He has published more than 120 papers in international journals and conferences. He has served as member of editorial boards of several journals and co-chaired many professional conferences. He is an academician of the Chinese Academy of Engineering. His research interests include distributed computing, computer networks, parallel computing, etc.

Huaimin Wang received his Ph.D. in Computer Science from National University of Defense Technology (NUDT) in 1992. He is now a professor and vice dean in the College of Computer, NUDT. He has been awarded the ‘‘Chang Jiang Scholars Program’’ professor by Ministry of Education of China, and the Distinct Young Scholar by the National Natural Science Foundation of China (NSFC), etc. He has worked as the director of several grand research projects and has published more than 100 research papers in international conferences and journals. His current research interests include middleware, software Agent, trustworthy computing. Ji Wang received his B.S., M.S. and Ph.D. in Computer Science from National University of Defense Technology in 1987, 1990 and 1995 respectively. Since 2002, he is a full professor in the School of Computer of National University of Defense Technology. He is deputy director of the National Laboratory for Parallel and Distributed Processing of China since 2007. In 2007, he won the National Science Fund for Distinguished Young Scholars of China. His research interests include formal analysis and verification of software systems, programming methodology for distributed computing. Jie Xu received his Ph.D. from the University of Newcastle upon Tyne. He was a full professor at Durham before he joined the Multidisciplinary Informatics Institute at Leeds, UK. He is now Chair of Computing and Head of the Institute for Computational and Systems Science at the University of Leeds, UK, and Director of the EPSRC WRG e-Science Centre involving the three White Rose Universities of Leeds, York and Sheffield. He has worked in the field of Distributed Computing for over twentyseven years and has industrial experience in building large-scale networked systems. Professor Xu now leads a collaborative research team at Leeds studying Cloud and Internet technologies with a focus on complex system engineering, system security and dependability, and evolving system architectures. He is the recipient of the BCS/IEE Brendan Murphy Prize 2001 for the best work in the area of distributed systems and networks. He has led or co-led many key research projects worth a total of over £20 M. He has published more than 300 academic papers, and given invited speeches at, and served as General Chair/Program Chair/PC member of, many international conferences. Dongsheng Li received the B.Sc. degree (with honors) and Ph.D. degree (with honors) in computer science from the College of Computer, National University of Defense Technology, Changsha, China, in 1999 and 2005, respectively. He was awarded the prize of National Excellent Doctoral Dissertation of PR China by Ministry of Education of China in 2008. He is now an associate Professor at National Lab for Parallel and Distributed Processing, National University of Defense Technology, China. His research interests include peer-to-peer computing, Cloud computing and computer networks.