Dynamic configuration support for power-aware virtualized server clusters Vinicius Petrucci, Orlando Loques Institute of Computing Fluminense Federal University Niteroi, Brazil Email: {vpetrucci,loques}@ic.uff.br
Abstract—Currently, an increasing number of large-scale server clusters (data centers) are being designed for supporting many different web-based application services in a seamless, transparent fashion. In this scenario, the rising energy costs for keeping up those server clusters is becoming an important concern. We present in this work a dynamic configuration support for specifying and deploying power management policies in a platform running multiple application services. In particular, it assumes a virtualized server environment and includes mechanisms that allow a designer to control power consumption, while meeting performance requirements, e.g., soft real-time. Keywords-dynamic configuration, power management, virtualization, soft real-time
I. I NTRODUCTION Currently, an increasing number of large-scale server clusters (data centers) have been designed for supporting many different web-based application services in a seamless, transparent fashion. For example, utility/cloud computing platforms [1], such as Amazon EC2 and Google Apps, are engineered to host and manage multiple application services (with workloads varying over time) on several shared physical servers. In this context, the rising energy costs for keeping up those server clusters is becoming an increasingly important concern for many businesses. It is recognized that consolidation of services workloads and dynamic service migration based on virtualization techniques show a great opportunity for increased server utilization and power optimization [2], [3]. The goal in the system is to determine the most power efficient server cluster configuration that can handle multiple varying services workloads in a virtualized environment. Towards that goal, we developed an application programming interface (API) to support the building of different algorithms for dynamic configuration in a system. This API relies on a set of operators that are invoked to faciilitate the implementation of the aforementioned algorithms, so that dynamic changing of the system is done in an easier fashion. We outline an example of configuration script based on a dynamic configuration model for power and performance optimization [4]. To achieve dynamic cluster
Daniel Moss´e Department of Computer Science University of Pittsburgh Pittsburgh, USA Email:
[email protected]
management, we rely on an adaptation framework which acts as a controller to monitor run-time properties of an executing system (e.g., request response time) and to manage the cluster configuration [5]. II. R ELATED WORK A similar configuration approach, based on the bin packing problem, for virtualized servers is defined in [6]. However, their model is not designed for power optimization and has some limitations. For example, it does not consider dynamic voltage/frequency scaling (DVFS) for the CPU frequency of the servers. A heuristic method for the power-aware consolidation problem of virtualized clusters is presented in [3]. Their model is based on a modified bin packing problem, but it may not guarantee to find solutions that are near to the optimal solution. In contrast, we adopt an exact approach based on mixed integer programming. In addition, their algorithm also does not include dynamic configurations for the DVFS. Another related approach is described in [7], wherein a dynamic resource provisioning framework is developed based on lookahead control. Their approach also does not consider DVFS. In [2], the authors present a two-layer control architecture aimed at providing power-efficient realtime guarantees for virtualized computing environments. Although their approach shows a rigorous basis on control theory, they do not address dynamic migration and on/off mechanisms in multiple servers. Another approach based on DVFS is presented in [8] for power optimization and end-to-end delay control in multitier web servers. The works presented in [9] and [10], [11] also rely on DVFS techniques and include server on/off mechanisms for power optimization. In contrast to our work, their approaches are not designed (and not applicable) for virtualized server clusters. That is, they do not consider multiple service workloads in a shared infrastructure. III. A PPROACH The main goal of our approach is to reduce power consumption while meeting performance requirements in virtualized computing environments. To achieve power and performance management in virtualized server clusters, our
proposal follows a well-defined closed control loop: (a) monitor system properties, (b) process and evaluate them, (c) execute a procedure to select a new configuration and (d) impose the chosen configuration. Following this control loop paradigm, we have proposed and developed a framework [5], which works on top of an abstract configuration model of the system and provides general configuration mechanisms: (1) to specify and monitor run-time properties of an executing system, (2) evaluate the model for system’s requirements violation and (3) perform adaptations on the system configuration to maintain the system behavior within acceptable bounds. The dynamic configurations are performed under guidance of scripts written in a high-level configuration language, designed separately from the target system. To apply the framework to the virtualized computing environment, we need to take some basic steps. First, a metric for controlling the system quality (e.g., system load or response time) has to be defined. Second, we need to specify some primitive configuration operators. For example, to start (or stop) a new virtual machine. Finally, we need to write a configuration script using the specified operators. In our approach, we write the configuration scripts using the Python programming language, which combines high-level dynamic data types with very clear syntax. Figure 1 depicts the main elements of our proposal. The dynamic configuration knowledge captures aspects, such as system quality requirements, optimization models and configuration scripts. Thus, it makes it accessible for the adaptation framework. The framework is responsible for system monitoring and control. It assumes that the target system (or platform) layer provides well-defined hooks for system measurements (in terms of probes modules) and supports dynamic configuration (through effectors modules). In many cases, these mechanisms can be implemented using an application programming interface (API) given by the system support level. For example, the Apache web server [12] supports an API to enable developers to extend a server with their own extension modules. The Xen hypervisor [13] also provides capabilities and mechanisms to monitor and manage virtual machines in a server cluster by means of an API. IV. DYNAMIC CONFIGURATION API In this section we introduce the concept of an application programming interface (API) to support developing different algorithms for dynamic configuration in a system. The key idea of an API is that it specifies an abstract and welldefined interface to control the behavior of the system, which builds on lower-level mechanisms at the system level. A desired feature for an API is that it can be called from several programming languages and is available as a remote procedure call, such as the XML-RPC protocol.
Dynamic configuration knowledge
- quality requirements - optim. models - config. scripts Adaptation
Framework control
monitoring
API effectors - start - stop - migrate - set sched Xen (hypervisor)
probes - turn on - turn off - set freq
- cluster load - response time
Apache (load balancer)
virtualized cluster
Figure 1.
Dynamic configuration support.
Our goal is to encapsulate most of the required functionality for dynamic configuration in terms of an API and provide generic configuration operators. This, in turn, enables the dynamic configuration logic to be described in a more appropriate way by using a number of high-level constructs (e.g., high level scripts). Examples of our API and operators include a call to a configuration operator, termed bestConfig, which encapsulates an optimization algorithm for solving the cluster configuration problem. In the next section, we present an example of a configuration script that makes use of this concept of API. V. E XAMPLE We consider a virtualized server cluster which needs to meet power and performance objectives. First, the aim is to provide soft real-time guarantees in the server cluster, in which the requests processing time have a specified deadline. Second, the set of currently active servers and their respective processors’ speeds should be dynamically configured to minimize the power consumption. We measure the cluster quality in terms of the average request response time, which refers to the time interval measured between the instant of that the request arrives at the cluster and the time that the associated response leaves the cluster. In order to keep this quality attribute within acceptable levels, we define two profiles, using the RT_LOW and RT_HIGH as the response time thresholds: profile { webcluster.responseTime < RT LOW } lowRT; profile { webcluster.responseTime > RT HIGH } highRT; These profiles are used to identify the system conditions for triggering a dynamic configuration [5]. We denote the
high threshold (RT_HIGH) as the request deadline, which is the maximum allowed response time. In order to guarantee the system’s quality requirements, we specify a configuration algorithm to be performed in response to changes in its execution environment, such as variation in the request response time. The configuration script is outlined below. when highRT or lowRT do // 1. Input variables d = webcluster.demands curconf = webcluster.currentConfig curalloc = webcluster.currentAlloc // 2. Run optimization newconf, newalloc = bestConfig(d) // 3. Generate config. and alloc. sets chgconf = sort(diff(newconf, curconf)) chgalloc = sort(diff(newalloc, curalloc)) // 4. Power management operations for i, j in chgconf: if j == 0: turnOff(j) else: if curconf[i] == 0: turnOn(i) setFreq(i, j) // 5. Virtualization management operations for k, i in chgalloc: if i == 0: stopVm(k, i) else: if curalloc[k] == 0: startVm(k, i) else: migrateVm(k, curalloc[k], i)
The input variables for the algorithm, described in the step 1, are: the current service demand (load) vector, the current server configuration and service allocation. The bestConfig operator, at algorithm step 2, returns a cluster configuration and allocation solution, where newconf represents a configuration of servers and their respective status (i.e., its operating frequency or inactive), and newalloc determines which service has to be associated with each server. For more details on the cluster optimization problem, we refer to our previous work [4]. In fact, the configuration to be imposed is a difference between two sets: the new configuration and the current configuration solution. For example, suppose the current configuration is curconf = {(1, 0), (2, 2), (3, 0)} and the new configuration is newconf = {(1, 0), (2, 0), (3, 4)}. Thus, we need to impose a configuration change given by newconf − curconf = chgconf = {(2, 0), (3, 4)}. That is, we need to turn off server 2, and turn on server 3 at the frequency 4. To handle this, we apply a diff operator
in the configuration and allocation solution provided by the optimization operator (see the algorithm step 3). Note that the order in which the operations are executed may lead to a problematic behavior. Specifically, in the example above, if the new configuration shutdowns the current active server before the new server is ready to respond the requests, as the server booting time is not instantaneous, the cluster will be in an unavailable state. To solve this issue, we sort the new configuration representation so that the operation to shutdown servers is always performed at last, and the operations to increase frequency and turn on servers are performed at first. This scheme can be similarly adopted to the new allocation representation, in which the operations to start and migrate virtual machines are performed at first. To achieve this, we make use of a sort operator in the configuration algorithm, as shown in algorithm step 3. In the step 4, we employ dynamic configurations for power optimization which consists of (a) turning servers off in periods of low activity and turning them back on when the demand increases, and (b) exploiting the dynamic voltage/frequency scaling capability of current processor architectures to further reduce power consumption. Finally, to manage the application services (which are associated to virtual machines), we rely on configuration operators to start, stop, and migrate the virtual machines in the server cluster, such as those described in the algorithm step 5. VI. C ONCLUSION In this paper, we have presented a dynamic configuration approach for power optimization in virtualized server clusters. Leveraging the dynamic configuration model described in [4], we outlined an algorithm to dynamically manage the virtualized server cluster. In future work, we aim to apply our approach to a real virtualized computing environment using Apache web servers [12] and virtual machine technology, such as the Xen hypervisor, which provides capabilities and mechanisms to manage virtualized server clusters. [13]. The increasing number of processing cores has become a promising way of improving the performance of servers. This would allow for many interesting configuration possibilities for power optimization, such as dynamic voltage/frequency scaling and on/off mechanisms at the core-level. We also intend to incorporate prediction techniques in our approach to improve dynamic configuration decisions, such as those described in [14]. R EFERENCES [1] B. Hayes, “Cloud computing,” Commun. ACM, vol. 51, no. 7, pp. 9–11, 2008. [2] Y. Wang, X. Wang, M. Chen, and X. Zhu, “Power-efficient response time guarantees for virtualized enterprise servers,” in RTSS ’08: Proceedings of the 2008 Real-Time Systems Symposium. Washington, DC, USA: IEEE Computer Society, 2008, pp. 303–312.
[3] S. Srikantaiah, A. kansal, and F. Zhao, “Energy aware consolidation for cloud computing,” in USENIX Workshop on Power Aware Computing and Systems, 2008. [4] V. Petrucci, O. Loques, and D. Moss´e, “A dynamic configuration model for power-efficient virtualized server clusters,” in 11th Brazillian Workshop on Real-Time and Embedded Systems (WTR), 2009. [5] ——, “A framework for dynamic adaptation of power-aware server clusters,” in SAC ’09: Proceedings of the 24th ACM Symposium on Applied Computing. ACM, 2009. [6] M. Bichler, T. Setzer, and B. Speitkamp, “Capacity Planning for Virtualized Servers,” Workshop on Information Technologies and Systems (WITS), Milwaukee, Wisconsin, USA, 2006. [7] D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasamy, and G. Jiang, “Power and performance management of virtualized computing environments via lookahead control,” IEEE International Conference on Autonomic Computing, 2008. [8] T. Horvath, T. Abdelzaher, K. Skadron, and X. Liu, “Dynamic voltage scaling in multitier web servers with end-to-end delay control,” IEEE Transactions on Computers, vol. 56, no. 4, pp. 444–458, 2007. [9] C. Rusu, A. Ferreira, C. Scordino, A. Watson, R. Melhem, and D. Moss´e, “Energy-efficient real-time heterogeneous server clusters,” in IEEE Real Time Technology and Applications Symposium. Washington, DC, USA: IEEE Computer Society, 2006, pp. 418–428. [10] L. Bertini, J. Leite, and D. Moss´e, “Statistical QoS guarantee and energy-efficiency in web server clusters,” in 19th Euromicro Conference on Real-Time Systems, 2007, pp. 83–92. [11] ——, “Dynamic configuration of web server clusters with QoS control,” in WIP Session of the 20th Euromicro Conference on Real-Time Systems, 2008. [12] The Apache Software Foundation, “Apache HTTP server version 2.2,” http://httpd.apache.org/docs/2.2/, 2008. [13] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield, “Xen and the art of virtualization,” in SOSP ’03: Proceedings of the nineteenth ACM symposium on Operating systems principles. New York, NY, USA: ACM, 2003, pp. 164–177. [14] D. Kusic, “Combined power and performance management of virtualized computing environments using limited lookahead control,” Ph.D. dissertation, Drexel University, 2008.
