Energy-Efficient Scheduling Algorithm of Task Dependent Graph on DVS-Unable Cluster System Yan Ma #1, Bin Gong #2, Lida Zou *3 #
School of Computer Science and Technology, Shandong University Jinan Shandong, China 1
[email protected] [email protected]
2
*
Information Technology School,Women’s Academy at Shandong Jinan Shandong, China 3
[email protected]
Abstract-At present, energy-efficient scheduling algorithm in High-Performance Computing (HPC) environment is becoming a research hotspot owning to its high operation cost and low reliability. In this paper, we investigate energy-efficient scheduling algorithm of data dependent tasks in DVS-Unable cluster system. The proposed ESTD algorithm efficiently integrates task clustering with task duplication technologies to reduce data transmission time and communication energy consumption. In order to decrease the static power of processing elements, it also uses one of the power-saving techniques in system level—Dynamic Power Management on the premise that application execution is non-preemptive and predictive. ESTD algorithm not only optimizes the makespan of task dependent graph, but also confines its energy consumption into a certain extent. Compared with EAD algorithm and PEBD algorithm, experimental results and case studies demonstrate that ESTD algorithm can reduce more energy consumption while not affecting scheduling performance of applications.
I.
INTRODUCTION
Traditionally, research interest of High-Performance Computing (HPC) system is to improve performance, such as throughput, resource utilization rate, load balancing and so on. In recent years, researchers start to focus on power management of HPC environment. We attribute the change to two reasons. 1) High operation cost. For IBM Roadrunner ranking first in the 32nd Top500 [3], its Linpack value is 1105.00Tflops, and its power is 2483.47KW. For Cray Jaguar ranking the second, its power is as high as 6950.60KW. For Dawn 5000A in Shanghai supercomputing center ranking the tenth, its power isn’t given, but we can see its high power from ￥10 millions of electricity fee each year. In addition, IDC [1] indicates that energy cost will account for one quarter of hardware cost in IT organization. 2) Low reliability. Experiments test and verify that Pflops system with 12-thousand nodes will appear once hardware failure every 24 hours. Arrhenius life-stress model [2] shows that failure rate of electron device will double as temperature rises by every 10 degrees. In the meantime, large amounts of energy consumption will make temperature ascend, while cooling equipments will in turn consume much electric energy.
978-1-4244-5149-4/09/$26.00 © 2009 IEEE
In summary, no matter for economical, technical or environmental reason, efficient power management is an urgent unsolved problem for HPC environment. Power consumption of HPC system is a multilevel and integrated management. At present, many HPC (supercomputing) centers adopt some power-saving measures, such as choosing products whose performance match their power, installing low-power cooling devices and heat sinks, enhancing equipment maintenance, application optimization and so on. However, these measures either are too expensive and architecture-based or belong to coarse-grained management. They cannot solve technical problems in real-time execution process. At present, compared with low-power architectural power saving method, power-aware software based method is more practical and widespread used [4]. In general, task scheduling is the kernel of system performance and efficiency management. Therefore, power-aware (energy-aware) task scheduling algorithm is an efficient method to ensure HPC system running applications in low energy consumption. Power-aware scheduling algorithm considers the energy consumption of application execution, and regards energy consumption as one of the scheduling indicators. In the past, power-aware scheduling algorithm is a matter of concern in wireless sensor network, embedded system and mobile system etc. The power source of these systems is battery and they have no continuous power supply, so energy consumption will be their primary scheduling objective[29]. However, due to the peculiarity of HPC system, power-aware scheduling algorithm in HPC environment should save energy consumption with no or little performance loss. At present, system-level power saving methods are DVS (Dynamic Voltage/ Frequency / Speed Scaling) [5] and DPM (Dynamic Power Management) [6]. DVS is an efficient way of managing dynamic power of processing elements (PE), because the power will generate square-reduction when supply voltage decreases in CMOS circuit[7]. DPM encompasses a set of techniques that achieves energy-efficient computation by selectively turning off system components when they are idle. This paper focuses on the power saving of processing elements and network resources. We mainly execute DPM for processing element. DPM can reduce static power of processing element through turning it off or switching it into the other low-power states.
217
10th IEEE/ACM International Conference on Grid Computing
Power-aware scheduling algorithms in HPC environment are starting to interest more scholars. We next enumerate the main related work. Kim et al. [8] investigate energy optimization algorithm of bag of tasks with specified deadline on cluster system, and their method of energy saving is EDF (Earliest Deadline First) based DVS technique. Freeh et al. [9, 10, 11] focus on energy optimization algorithm of MPI program in HPC environment, and their method of energy saving is performing DVS technique for non-critical node. Qin et al. [12, 13, 14] propose energy-efficient scheduling algorithm of parallel applications without performance loss on cluster system, and their method of energy saving is DVS or task duplication technique. Cameron et al. [15, 16, 17, 18] present phase based power-aware scheduling algorithm of scientific application on cluster system, their method is executing DVS technique using the delay of memory access and communication.
to a low–power state. Of course, the premise is that task execution is non-preemptive, and its execution time and idle period can be predicted in advance. After introducing DPM technique, task duplication also considers energy variability before DPM and after DPM. Our simulation results demonstrate that compared with EAD and PEBD algorithm, ESTD algorithm using task clustering, task duplication, DPM and new conditions of duplication can reduce more energy consumption while not affecting the optimization of execution time.
However, power-aware scheduling algorithms at present in HPC environment have the following limitations.
Dependent task set is abstracted into directed graph, node represents computation task, and directed edge represents data dependency between tasks. We focus on directed acyclic graph, because directed cyclic graph in the scheduling algorithm can change to directed acyclic graph through conditional value prediction, parallel loop unrolling and linear loop elimination etc. Dependent task set is formulated by G = (T , E ) , where T is task set, and E is data dependence set. For each task, it can start executing only when all its parent tasks are completed or all the needed data are arrived. In a given task dependent graph, a task not having any parent task is called an entry task, and a task not having any child task is called an exit task. If a scheduling algorithm requires a single entry task or a single exit task, and a given graph contains more than one entry tasks or exit tasks in the graph, we can produce a new graph by connecting entry nodes to a zero-cost pseudo entry Tin or exit nodes to a zero-cost pseudo exit Tout , without affecting the schedule [19]. Each task Ti , i = 1, 2,..., n has a parameter: the number of instructions cci (Million Instructions). Each edge (Ti , T j ) has a parameter: volume of transferred data ddij (Mb). The concrete model is shown in Figure 1.
1) Unavailability of DVS technique. Many HPC systems do not support DVS technique, that is, they are DVS-Unable. 2) Inadaptability of data-intensive application. Modern scientific research based on high-performance computation is data-oriented, computation-intensive, analysis-intensive and visualization-intensive, such as bioinformatics, environmental science and astronomy. Therefore, HPC environment needs power-aware scheduling algorithms especially propitious to the characteristics of this field. It should lay stress on the importance of data dependency and communication energy consumption. 3) Neglect of static power in PE. Traditionally, dynamic power in PE accounts for most of the whole power. Static power is the part consumed just by leakage current. However, along with the development of chip miniaturization and multi-core technology, static power increases exponentially as the number of the electric components in unit size rises. Obviously, scheduling algorithm should not only consider dynamic power of processor, but also reduce static power using DPM technique of processing elements. Based on the above considerations, we research Energy-efficient Scheduling algorithm of data-intensive application abstracted by Task Dependent graph in DVS-Unable cluster system (ESTD for short). The objective of our algorithm is to minimize the execution time (makespan) of task dependent graph while maintaining its low energy consumption. There exists large data dependency between tasks in data-intensive application. In general, task clustering and task duplication are the most efficient means of reducing communication time and communication energy consumption. Therefore, this paper employs and combines the above two methods, and systematically analyze the conditions of executing task duplication in power-aware situation. Task duplication considers both not increasing execution time and constraints of energy consumption. DVS-Unable cluster system cannot scale voltage and frequency, so we cannot use DVS technique. But we can make use of DPM technique in order to reduce static energy consumption. If processor stays idle for a long time, we can turn off the processor or switch it
II.
SYSTEM MODEL
In this section, we first give system model of ESTD algorithm in HPC environment, including dependent task model, DVS-Unable cluster system and power model. A. Dependent Task Model
Tin
Ti
cci
ddi j
Tj
Tout
Fig.1. Dependent task model
B. DVS-Unable Cluster System We assume a cluster consists of m homogeneous processing elements PE1 , PE2 ,..., PEm , and it is formulated by L{PEi | i = 1, 2,..., m} . All the PEs are fully connected through network resource, and they have the same data transfer bw (Mbps), the same communication speed power Pcomm (Joule/Mb) and processing speed s (MIPS). Each
218
PE has three states: active, idle and shutdown. Of course, some PEs may have more low-power states, but we focus on the simple case in this paper. During executing task, PE is in active state, and dynamic power Pdynamic and static power Pstatic of processor are both consumed. During data transmission, communication power of network resources is consumed. In this case, if some PE has no computation to do but waits for input data arrival, PE is in idle state and static power of processor is consumed. When a PE neither does computation nor waits for data, PE is idle and static power of processor is also consumed. There exists no energy consumption when PE is in shutdown state and no data transfer. However, it takes some time and energy cost to turn off and turn on the processing element. We formulate the shutdown time and energy cost as t DPM and eDPM . Our scheduling algorithm is conscious of energy consumption of computation resource and network resource, ignoring energy of storage resource.
duplicated to which cluster. Our rule of task duplication is that it should reduce makespan of application within an energy-increase threshold. After task duplication, it should check if DPM is necessary to perform. 4) Task Assignment. It assigns each cluster to a processor.
C. Power Model Power in CMOS circuit consists of dynamic power and static power. Dynamic power is incurred by charging and discharging capacitor, while static power is caused by leakage current. We describe energy consumption of processor Ti using executing task i Ecomp = ( Pstatic + Pdynamic ) ⋅ ti = ( Pstatic + Pdynamic ) ⋅ cci / s . When processor PEi transfers data to processor PE j and they do not execute any task and just wait for data arrival, they consume static = Pstatic * tcomm , where tcomm is pure static energy Ecomm communication time. Correspondingly, communication energy consumption is represented as Ecomm = Pcomm * ddij . In real system, turning off and turning on processor need some time and energy, tDPM and eDPM . Therefore, if we perform DPM technique, we must check if the time and energy saving after DPM can compensate t DPM and eDPM of executing DPM. III.
ESTD ALGORITHM
In this section, we first give energy-aware scheduling framework of task dependent graph. It introduces a novel energy-aware scheduling framework of large data-intensive application in HPC environment. In the scheduling process, the framework puts data transmission time and energy consumption in the most important position of optimization. Moreover, it takes static energy of processor into accounts, which adapts to the development trends of chip miniaturization and multi-core technology. The framework of ESTD algorithm is depicted in Figure 2, and it comprises four steps. 1) Task Clustering. First it implements task clustering algorithm for directed acyclic graph after abstraction, gains the initial task assignment plan and the number of processors, and then applies for the given number of processors to cluster system. 2) Dynamic Power Management. According to the relation between idle time and cost of executing DPM technique, it determines whether there exist idle periods that can compensate cost of executing DPM. 3) Task Duplication. According to our proposed mathematical conditions of task duplication, it determines which task is
Our ESTD algorithm reduces energy consumption on the basis of makespan optimization: it reduces communication cost through task clustering and task duplication; it reduces static power through DPM technique. Next we investigate each step of ESTD algorithm in detail. A.
Task Clustering(TC)
In parallel and distributed system, task clustering is an effective means to reduce communication cost of DAG graph. Task clustering is to assign all the tasks in a cluster to the same resource, and aims to reduce data transfer cost between tasks. Data-intensive application abstracted into data dependent graph in HPC environment has large amounts of data to generate and transfer. Therefore, task clustering is our best choice to reduce communication time and energy consumption. Classical task clustering algorithms with no duplication are MCP [20], Sarkar [21], DSC (Dominant Sequence Clustering) [22] and CASS-II (Clustering And Scheduling System-II) [23]. Experiments proves DSC algorithm and CASS-II algorithm are the two best algorithms in performance, and CASS-II algorithm has lower time complexity. In this paper, ESTD algorithm uses CASS-II algorithm to proceed task clustering. Its main idea is as follows. Firstly, it computes parameter topi for each task according to the following formula. ⎧⎪0.........Ti is.entry.node topi = ⎨ ⎪⎩max{top j + t j + t ji ,( j , i) ∈ E}......otherwise
(1)
Secondly, it computes another parameter bottomj for each task from bottom to up according to formula (2). ⎧⎪t j .........T j is.exit.task (2) bottom j = ⎨ ⎪⎩max{bottomi + t ji + t j ,( j , i ) ∈ E}......otherwise
Lastly, in the bottom computation process, it selects the task
219
whose priority=top+bottom value is the largest to try to merge with its dominant successor into a cluster. If the bottom values of current task and all the tasks in the cluster do not increase, the actual merger can perform; else, the current task becomes a separate cluster. You can refer to paper [23] about its concrete algorithm. After clustering, we can get the initial task grouping and task ordering of applications. B.
Dynamic Power Management(DPM)
Dynamic power management is turning off processor or switching into low-power state when they are idle. Its precondition is that application on PE is exclusive and non-preemptive. To turn off and turn on processor consume some time and energy. Therefore, only when idle period exceeds a certain threshold can performing DPM method reduce time and energy. In this paper, we give the following formula about the time threshold. tthreshold = max{tDPM , eDPM / Pstatic } eDPM / Pstatic represents the eDPM when processor is idle.
least
scheduling algorithm based on TDS algorithm. Its conditions of duplication are last(i)-lact(j)
