Jan 19, 2012 ... Green Computing refer to 2 different things ... This presentation : Reducing
energy consumption of ICT ..... In: (Ed.), Workshop on Micro.
Green Computing Johan Lilius
January 19, 2012
Johan Lilius
Green Computing
1/67
Introduction
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
2/67
Introduction
What is Green-Computing (Green-ICT)?
Green Computing refer to 2 different things 1 2
Reducing energy consumption of ICT Using ICT to reduce energy consumption
Goal: reduce carbon footprint This presentation : Reducing energy consumption of ICT
Johan Lilius
Green Computing
3/67
Introduction
Green-Computing aspects [11]
Start 1992 EPA Energy star rating
5 issues (2009) 1 2 3 4 5
Johan Lilius
E-Waste Data-centers and Servers PCs, Monitors and Workstations Software Telecommuting
Green Computing
4/67
Introduction
E-Waste
Recycling issue estimates that over 25 billion computers, televisions, cell phones, printers, gaming systems, and other devices have been sold since 1980, 2 million tons of unwanted electronic devices in 2005 alone, with only 15 to 20 percent being recycled
Material is transfered to developing countries Clear environmental hazards
Johan Lilius
Green Computing
5/67
Introduction
Data centers & Servers Energy efficiency •
Power draw of computing clusters is becoming an increasing fraction of their cost1
Power consumption is an issue
•
•
•
9/3/10&
Johan Lilius
The density of the datacenters that house them is in turn limited by
Driver money, not and environmental aspects their ability to supply cool 10–20 kW of power:-( per rack and up to
10–20 MW per datacenter Future datacenters may require as much as 200 MW, and datacenters are being constructed today with dedicated electrical substations to feed them. 1Kenneth
G. Brill: “The Invisible Crisis in the Data Center: The Economic Meltdown of Moore's Law” Uptime Institute, 2009
Green Computing
2&
6/67
Introduction
PCs, Monitors, Workstations
5th upgarde Energy Star requirements for monitors (1 April 2009) Requires a 20 percent increase in electrical efficiency. Estimate if all monitors comply, saving of roughly $1 billion per year in energy expenses and avoid GHG emissions equivalent to 1.5 million cars
Johan Lilius
Green Computing
7/67
Introduction
Software
Cost of Spam (McAfee, 2008) 62 trillion spam messages in 2008 0.3 grams of carbon dioxide (CO2) per message annual spam energy use 33 terawatt hours (tWh) equivalent to the electricity used in 2.4 million homes every year, with the same GHG emissions as 3.1 million automobiles using two billion US gallons of gasoline
How we use the computing systems influences power consumption
Johan Lilius
Green Computing
8/67
Introduction
Telecommuting
An ITIF report found that, if only 14 percent of existing American office jobs were converted to work-from-home jobs, the savings would be dramatic: estimated at 136 billion vehicle travel miles annually in the US by 2020 and 171 billion miles by 2030
Johan Lilius
Green Computing
9/67
Introduction
This presentation
How to reduce energy consumption in datacenters Central concept: Energy Proportionality Use only as much computation power that is needed for the task at hand
Still a lot of R&D to do!
Johan Lilius
Green Computing
10/67
The economics of Data-Centers: Why Power Matters
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
11/67
The economics of Data-Centers: Why Power Matters
Energy cost in data-centers I Energy efficiency
•
•
•
•
9/3/10&
Johan Lilius
Power draw computing clusters is becoming an increasing Power drawofof computing clusters is becoming an fraction of their cost1 increasing fraction of their cost
The density of the datacenters that house them is in turn limited by
The the and datacenters isand in up to their density ability to of supply cool 10–20that kW ofhouse power them per rack 10–20 MW perby datacenter turn limited their ability to supply and cool 10–20 kW Future datacenters may require as much as 200 MW, and datacenters of per rack and upwith to dedicated 10–20 MW per substations datacenterto arepower being constructed today electrical feed them. Future datacenters require as much as >100 MW, and Kenneth G. Brill: “The Crisisconstructed in the Data Center: The Economic Meltdown of Moore's Law” datacenters areInvisible being today with dedicated Uptime Institute, 2009 electrical substations to feed them. 1
Facebook, Luleå, budget 120MW Green Computing
2&
12/67
The economics of Data-Centers: Why Power Matters
Energy cost in data-centers II
Exascale computing roadmap, budgets of >100MW for supercomputing facilities
Nuclear powerplants: Loviisa reactors 488MW nominal power Olkiluoto new reactor 1600 MW nominal power
Johan Lilius
Green Computing
13/67
The economics of Data-Centers: Why Power Matters
Cost breakdown for data-centers I
A $200M facility capable of delivering 15MW of critical load (server power) Facility: ˜$200M for 15MW DC (15 yr Amortization) Servers: ˜$2k/each, roughly 50,000 (3 yr Amortization) Commercial Power: ˜$0.07/kWh 5% cost of money
Johan Lilius
Green Computing
14/67
period of the equipment. This converts the capital expenses to effective cost per month. And, by considering amortization periods, we normalize long lived and short lived capital and The economics of Data-Centers: Why Power Matters recognize each appropriately. In this model, land, taxes, security and administration are not included due to their relatively small Cost breakdown data-centers II contribution tofor overall costs.
defined as IT tells us what actually gets d
These terms 1.7 states tha servers), we d systems (air c same as a DC to the facility also tells us t lost in power
PUEs vary g as low as 2 facilities are reports, howe
In this explo generation fa understood te available equ 1.7, putting i But it is not innovations. A best and form where the pow
Figure 1: Monthly Server, Power, and Infrastructure Costs Figure 1 shows thatarepower costs are related much lower than Infrastructure costs mostly power
infrastructure costs, and also much less than the servers 50% of costs are power related themselves. Servers are the dominant cost, but, before we conclude that power is only 23% of the total, it’s worth looking Looking mor more closely. Infrastructure includes the building, power Johan Lilius Green Computing 15/67 definition of distribution, and cooling. Power distribution and cooling make up power to the 82% of the costs of infrastructure [2] with the building itself down remaining 41 in the 12-15% range. Power distribution is functionally related to The economics of Data-Centers: Why Power Matters To understan the power consumed in that sufficient power distribution going, we loo equipment is required to distribute the maximum amount of power both easier to Measures (PUE, DCiE) I consumed. Cooling is also functionally related to power in that track. Lookin
PUE: Power Usage Effectivenes PUE =
2
TotalPower ITEquip.Power
DCie: Data Center Infrastructure Effectiveness DCiE =
ITEquip.Power ⇤ 100% TotalPower
Google average Q3 2011: PUE 1.16 CSC Kajaani data-center goal: PUE 1.15 PUE is highly dependent on outside temperature
Johan Lilius
Green Computing
16/67
The economics of Data-Centers: Why Power Matters
Measures (PUE, DCiE) II Nordic countries are interesting Google: Summa Facebook: Luleå
Added advantage: green power Facebook datacenter will be powered by hydropower Diesel only as backup
Other ideas: Use feed excess heat into city-wide heating system (Gaudeamus datacenter, Helsinki)
NOTE PUE is a measure on the effectiveness of the cooling, not of the energy-efficiency of the computing!!! Quite a lot remains to be done to cut down the total power consumption of the system
efficiency of each conversation from the power delivered by the utility at 115,000V through to deliver to the servers at 208V.
Johan Lilius Starting at
pricing out at more than $2M. Most facilities will have at least 1 extra generator (N+1) and many facilities will have 2 spares
Green Computing (N+2) allowing one to be in maintenance, one to fail on startup 2, we see the utility
the upper left corner of Figure delivers us 115kV and we first step it down to 13.2kv. The 13.2kv feed is delivered to the Uninterruptable Power Supply (UPS). In this case we use a battery-based UPS system, but rotary systems are also common. This particular battery-based UPS is 94% efficient, taking all current through rectifiers to direct current and inverting it all back to AC. Rotary designs are usuallyPower more Thethen economics of Data-Centers: Why efficient than the example shown here and bypass designs can exceed 97% efficiency. In this example, a non-bypass UPS installation, all power flowing to UPS protected equipment (the servers and most of the mechanical systems) is first rectified to DC and then inverted back to AC. All the power destined to the servers flows through these two conversions steps whether or not there is a power failure, and these two conversion steps contribute the bulk of the losses, bringing down the UPS efficiency to 94%. More efficient bypass UPSs avoid these losses by routing most power “around” the UPS in the common, non-power failure case.
and still to be able to run the facility at full load during a power failure. A 2.5MW generator will burn just under 180 gallons/hour of diesel so environmentally conscious operators work hard to minimize their generator time. And the storage of well over 100,000 gallons of diesel at the facility brings additional cost, storage space, insurance risk, and maintenance issues.
17/67
Matters
After the UPS, we step down the 13.2kV voltage to 480V and then that is further stepped down to 208V for distribution to the critical load, the servers. In this facility, we are using very high quality transformers, so we experience losses of only 0.3% at each transformer. We estimate that we lose a further 1% in switch gear and conductor losses throughout the facility.
Energy losses in Power Distribution
We know we deliver 59% of the facility power to the critical load and, from the electrical distribution system analysis above, we know we lose 8% of total power to power distribution losses. By subtraction, we have 33% lost to mechanical systems responsible for data center cooling.
Figure 2: Power Distribution For longer term power outages, there are usually generators to keep the facility operational. The generation system introduces essentially no additional losses when not being used but they greatly increase the capital expense with a 2.5MW generator
Johan Lilius
In summary, we have three 99.7% efficient transformers, a 94% efficient UPS and 1% losses in distribution for an overall power distribution loss of 8% (0.997^3*0.94*0.99 => 0.922).
Green Computing 3
18/67
The economics of Data-Centers: Why Power Matters
AC vs DC? Ishikari Datacenter in Ishikari City, Hokkaido Japan Use High-Voltage (400V) DC power as much as possible
Johan Lilius
Green Computing
19/67
The economics of Data-Centers: Why Power Matters
Energy losses in Mechanical Systems
Figure 3: Mechanical Systems Several observations emerge from this summary. The first is that power distribution is already fairly efficient. Taking the 8% efficiency number down to 4% to 5% by using a 97% efficient UPS and eliminating 1 layer of power conversion is an easy improvement. Further reductions in power distribution losses are possible but the positive impact can’t exceed 8% so we’re better rewarded looking to improvements in the mechanical systems, where we are spending 33% of the power, and in the servers, where we are dissipating 59% of the power.
Johan
4.1 System Balance
Looking back 25 years, we have experienced steady improvement in CPU performance and, for a given algorithm, increased performance generally requires increased data rates. In the high performance computing world, this is reported in bytes/FLOP but it’s just as relevant in the commercial processing world. More CPU performance requires more memory bandwidth to get value from that increase in performance. Otherwise, the faster processor just spends more time in memory stalls and doesn’t actually get more work done. For the bulk of the last 25 years, CPU performance improvements have been driven by design The CEMS project focused on the latter, increasing the efficiency improvements and clock frequency increases. Having hit the Lilius Green Computing of the servers themselves. power wall, we’re now less reliant on clock frequency improvements than in the past and more dependent upon increases 4. CEMS Introduction in core counts. But the net is that processor performance From the previous section, we understand that 59% of the power continues to grow unabated and this is expected to continue. dissipated in a high-scale data center is delivered to the critical load. Generally that is a good thing in that power delivered to the Looking at the first row of Table 1, from Dave Patterson’s
20/67
The economics of Data-Centers: Why Power Matters
Power losses in Servers
100%
45%
90%
40%
80%
Power per com ponent
50%
35%
70%
30%
60%
25%
50%
20%
40%
15%
30%
10%
20%
5%
10%
0%
C u m u l a t iv e p o w e r
Electricity consumption in a typical data center
0% Load
PSU
Chiller
UPS
VRs
Server CRAC fan PDU CW pump Total fans baseline
Source: Intel Systems Technology Lab 2008
Johan Lilius
Green Computing
21/67
The economics of Data-Centers: Why Power Matters
Heat density I Power dissipated / per unit area Chip-level issue Building-level issue
Chip level 40W/cm2
Shuttle re-entry 100W/cm2
Building level Blade server > 5kVA Finnish sauna ˜6kVA Most dense datacom products > 9kVA
One cannot pack servers into a small space Cooling & Air Handling Gains
Intel
Verari
• Tighter control of air-flow increased delta-T • Container takes one step further with very little air in motion, variable speed fans, & tight feedback between CRAC and load • Sealed enclosure allows elimination of small, inefficient (6 to 9W each) server fans Intel
2009/4/1
Johan Lilius
http://perspectives.mvdirona.com
14
Green Computing
22/67
The economics of Data-Centers: Why Power Matters
Processor
Issues: 1 2
Johan Lilius
Where does processor power go? What can we do about it?
Green Computing
23/67
The economics of Data-Centers: Why Power Matters
Processor Power I
22% yearly increase
Johan Lilius
Green Computing
24/67
The economics of Data-Centers: Why Power Matters
Processor Power II Concrete values for AMD and Intel
Johan Lilius
Green Computing
25/67
The economics of Data-Centers: Why Power Matters
Processor Power III Cooler in 1993 and in 2005
Johan Lilius
Green Computing
26/67
The economics of Data-Centers: Why Power Matters
Processor Power IV
How far can you go? AMD FX Processor Takes Guiness World Record http://bit.ly/ux5ORA
Johan Lilius
Green Computing
27/67
The economics of Data-Centers: Why Power Matters
Power budget of processor
Exact numbers difficult if impossible to obtain Intel Penryn (2007) uses 50% of silicon for L2/L3 caches Measurement on ARM shows that memory subsystem uses 50% of total power
Johan Lilius
Green Computing
28/67
The economics of Data-Centers: Why Power Matters
Processor power Power as function of voltage and frequency P = c ⇤ V2 ⇤ f c is activity factor What can you do? 1
Use techniques to shutdown unused parts of the chip Adds complexity to chip, shutdown circuits add to energy consumption
2
Run the chip at lower frequencies and/or lower voltage Does not scale linearly downwards: running at 20% of capacity may still use 50% of energy
Dead-end?
Johan Lilius
Green Computing
29/67
Energy-proportional computing
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
30/67
Energy-proportional computing
Introduction Energy-proportional computing introdcued by Barroso and Hölzle at Google˜[2] ments because minor traffic fluctuations or any internal disruption, such as hardware or software faults, could tip it over the edge. Moreover, the lack of a 0.025 reasonable amount of slack makes regular operations exceedingly complex because 0.02 any maintenance task has the potential to cause serious service disruptions. Similarly, well-pro0.015 visioned services are unlikely to spend significant amounts of time completely idle because 0.01 doing so would represent a substantial waste of capital. Even during periods of low ser0.005 vice demand, servers are unlikely to be fully idle. Large-scale services usually require hundreds of 0 servers and distribute the load 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 over these machines. In some CPU utilization cases, it might be possible to completely idle a subset of servers Figure 1. Average CPU utilization of more than 5,000 servers during a six-month period. during low-activity periods by, Servers are rarely completely idle and seldom operate near their maximum utilization, for example, shrinking the numinstead operating most of the time at between 10 and 50 percent of their maximum ber of active front ends. Often, utilization levels. though, this is hard to accomplish because data, not just comvoltage-frequency scaling. Mobile devices require high putation, is distributed among machines. For example, performance for short periods while the user awaits a common practice calls for spreading user data across response, followed by relatively long idle intervals of many databases to eliminate the bottleneck that a censeconds or minutes. Many embedded computers, such tral database holding all users poses. as sensor network agents, present a similar bimodal Spreading data across multiple machines improves usage model.4 data availability as well because it reduces the likeliThis kind of activity pattern steers designers to empha- hood that a crash will cause data loss. It can also help size high energy efficiency at peak performance levels hasten recovery from crashes by spreading the recovand in idle mode, supporting inactive low-energy states, ery load across a greater number of nodes, as is done such as sleep or standby, that consume near-zero energy. in the Google File System.6 As a result, all servers must However, the usage model for servers, especially those be available, even during low-load periods. In addition, used in large-scale Internet services, has very different networked servers frequently perform many small backcharacteristics. ground tasks that make it impossible for them to enter Figure 1 shows the distribution of CPU utilization lev- a sleep state. els for thousands of servers during a six-month interWith few windows of complete idleness, servers canval.5 Although the actual shape of the distribution varies not take advantage of the existing inactive energysignificantly across services, two key observations from savings modes that mobile devices otherwise find so Figure 1 can be generalized: Servers are rarely com- effective. Although developers can sometimes restrucpletely idle and seldom operate near their maximum uti- ture applications to create useful idle intervals during lization. Instead, servers operate most of the time at periods of reduced load, in practice this is often difficult between 10 and 50 percent of their maximum utiliza- and even harder to maintain. The Tickless kernel7 exemtion levels. Such behavior is not accidental, but results plifies some of the challenges involved in creating and from observing sound service provisioning and distrib- maintaining idleness. Moreover, the most attractive inacuted systems design principles. tive energy-savings modes tend to be those with the highAn Internet service provisioned such that the average est wake-up penalties, such as disk spin-up time, and load approaches 100 percent will likely have difficulty thus their use complicates application deployment and meeting throughput and latency service-level agree- greatly reduces their practicality. Fraction of time
0.03
Observation: Servers are idling a lot of the time Where does idling come from: There is not enough computation Waiting for I/O Waiting for memory Johan Lilius
Green Computing
34
31/67
Computer
Energy-proportional computing
Energy-proportional computing I Barroso and Hölzle suggest dynamic power control of nodes in the datacenter The datacenter would only keep a needed number of nodes online to match the computational requirements Goal: given a load curve match it with computational power as closely as possible Energy Waste Energy Waste
Computation Requirement Energy Waste Energy Waste Computation Requirement
Johan Lilius
Green Computing
32/67
Energy-proportional computing
Energy-proportional computing II
Big servers, large granularity => low energy-proportionality Small servers, small granularity => better energy-proportionality Small servers will have less computational power, is this a problem?
Johan Lilius
Green Computing
33/67
Energy-proportional computing
Memory Wall
big discrepancy between processor speed and memory access speeds processor speed has been growing with 50-100% annually memory speed has been growing with 7% annually
speed gap that is growing all the time to avoid wait times, and keep deep pipelines working, large caches are required, which use a lot of power complex memory subsystems DDR1-3 are developed which also consume power
Johan Lilius
Green Computing
34/67
Energy-proportional computing
Amdahl blades˜[12] I
A balanced computer system needs: sequential I/O per sec - Amdahl number 2 memory with Mbyte/MIPS ratio of close to 1 - Amdahl memory ratio 3 performs on I/O operation per 50k instructions Amdahl IOPS ratio 1
Graywulf system (2008, state of the art architecture) Amdahl number 0.56, Amdahl ratio 1.12, Amdahl IOPS ratio 0.014 Data cannot be fed fast enough into the system
Johan Lilius
Green Computing
35/67
Energy-proportional computing
Amdahl blades˜[12] II
Compare Graywulf with COTS systems Table 2: Performance, power, and cost characteristics of various data-intensive architectures.
GrayWulf ASUS Intel Zotac AxiomTek Alix 3C2
CPU [GHz]
Mem [GB]
SeqIO [GB/s]
RandIO [kIOPS]
Disk [TB]
Power [W]
Cost [$]
21.3 1.6 3.2 3.2 1.6 0.5
24 2 2 4 2 0.5
1.500 0.124 0.500 0.500 0.120 0.025
6.0 4.6 10.4 10.4 4.0 N/A
22.5 0.25 0.50 0.50 0.25 0.008
1,150 19 28 30 15 4
19,253 820 1,177 1,189 995 225
Relative Power 1.000 0.017 0.024 0.026 0.013 0.003
Amdahl numbers Seq Mem Rand 0.56 0.62 1.25 1.25 0.60 0.40
1.13 1.25 0.63 1.25 1.25 1.00
0.014 0.144 0.156 0.163 0.125
(essentially the same today). We note that for the three times more power than the equivalent Intel It2008 would seem that running atmately lower frequencies gives SSD-based systems the cost and disk size columns in system (240 W vs. 84 W). Table 2 represent projections for a 250 GB drive with better powerand efficiency? the same performance a projected cost of $400 at
Johan Lilius
the end of 2009, in line with historic SSD price trends. Power consumption varies between 15W-30W depending on the chipset used (945GSE, USW15, ION) and generally agrees with the values reported in the motherboards’ specifications. The current university rate for electric power at JHU is $0.15/kWh. The total cost of power should include the cost for cold water and air conditioning, thus we multiply the electricity cost by 1.6 [7]. Table 2 presents these cumulative costs. Lastly, we present the different Amdahl numbers and ratios for the various node types. It is clear that, compared to the GrayWulf and Alix, the Atom systems, esGreen Computing pecially with dual cores, are better balanced across all three dimensions. Scaling Properties. Table 3 illustrates what happens when we scale the other systems to match the Gray-
5. DISCUSSION
The nature of scientific computing is changing – it is becoming more and more data-centric while at the same time datasets continue to double every year, surpassing petabytes. As a result, the computer architectures currently used in scientific applications are becoming increasingly energy inefficient as they try to maintain sequential read I/O performance with growing dataset sizes. The scientific community therefore faces the following dilemma: find a low-power alternative to existing systems or stop growing computations on par with the size of the data. We thus argue that it is unavoidable 36/67 to build scaled-down and scaled-out systems comprising large numbers of compute nodes each with a much lower relative power consumption at a given sequential read I/O throughput.
Servers built on mobile processors
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
37/67
Servers built on mobile processors
Introduction
Mobile processors are very energy efficient Could we used mobile processors (e.g ARM) to implement energy proportional data centers? Research problems: Are mobile processors more energy efficient compared to IA processors? How would one implement energy proportionality?
Our goal: build an energy proportional cluster based on ARM processors
Johan Lilius
Green Computing
38/67
Servers built on mobile processors
Commercial drivers
Money Mobile processors are cheap
Optimized for low energyconsumption Architecture Low-power states
Heat-density It will be possible to place servers into spaces where it currently is not possible
Johan Lilius
Green Computing
39/67
Servers built on mobile processors
Energy efficiency of ARM processors Comparison of Energy per Instruction (i.e. what could Challenges be ideally achieved?) Energy efficiency: ARM vs Pentium Processor
EPI(nJ)
NominalCPI
Power consumption
ARM720T
0.22
2.2
65mW
ARM926EJ-S
0.46
1.6
95mW
ARM1136J-S
0.63
1.4
115mW
Processor
EPI (nJ)
Nominal CPI
Power consumption
Pentium 4
48
2.59
5060mW
Pentium M
15
2320mW
Core Duo
11
1190mW
ARM720T is 220 times more energy efficient than a Pentium 4
Do 220 ARM720T’s have more computational power than a Pentium 4? For which workloads? 9/3/10&
Johan Lilius
5&
Green Computing
40/67
Servers built on mobile processors
Hardware Versatile Express Quad-core Cortex A9, 1GB DDR2,
400Mhz Tegra 250 Dual-core Cortex-A9, 1GB DDR2, 1G
Johan Lilius
Green Computing
41/67
Servers built on mobile processors
Benchmarks
Autobench and Apache 2 HTTP server static web pages
SPECweb2005 more demanding web services
Erlang micro benchmarks real world SIP proxy
Johan Lilius
Green Computing
42/67
Servers built on mobile processors
Autobench and Apache
Johan Lilius
Green Computing
43/67
Servers built on mobile processors
SPECWeb2005
Johan Lilius
Green Computing
44/67
Servers built on mobile processors
Erlang
Johan Lilius
Green Computing
45/67
Servers built on mobile processors
Results
The performance of 2 ARMv7 based ARM cortex-A9 was measured and evaluated and compared to Xeon processors Measurements show that the Cortex A9 can be up to 11 times more efficient with the Apache server 3.6 times more efficient with Erlang base SIP proxy 2.9 times more efficient with the SPECweb2005
Johan Lilius
Green Computing
46/67
Servers built on mobile processors
Implementation of energy proportianality
Since the computational capacatiy of an ARM processor is lower that an IA processer we need more of them This enables a more fine-grained control of the computational power
Johan Lilius
Green Computing
47/67
Servers built on mobile processors
Evaluation Platform
Few fast CPUs dissipate much power and have rough power granularity Instead use slow but many mobile CPUs to increase the power granularity Evaluation done on a cluster using ARM Cortex-A8 (Beagleboard) Low energy consumption and low price tag Running Ubuntu 11.04 with Linux 2.6.34
Johan Lilius
Green Computing
48/67
Servers built on mobile processors
The problem with DVFS The CPU is the main energy consumer in the server Power managers have been used to scale the performance of the CPU according to the demand DVFS scales the voltage and frequency DVFS does not scale the power dissipation linearly !
Johan Lilius
Green Computing
49/67
Servers built on mobile processors
Power Manager using Sleep States The power manager is used to dynamically adjust the system capacity to the workload The manager wakes up cores when the capacity is too low and shuts down cores when capacity is unnecessary high Goals: Good power-to-workload proportionality resulting in little energy waste Show a minimal performance degradation to users Scale in large clusters
Johan Lilius
Green Computing
50/67
Servers built on mobile processors
System level power manager Switches on/off CPU cores in the cluster Operates on systems level, meaning that it controls the whole cluster as one entity Master core controlling the other workers Workers taking orders: Sleep/Wake and workload
Johan Lilius
Green Computing
51/67
Servers built on mobile processors
Simulation framework
Johan Lilius
Green Computing
52/67
Servers built on mobile processors
Simulation framework Quality of Service (QoS) is our current measurement of performance QoS shows how many % of the incoming requests are handled in a time frame The unhandled requests are moved to the next time frame and result in a QoS drop QoS drop is a result of latency that triggers a deadline miss Simulation framework can select static cores that are not altered by the power manager, i.e. always running
Johan Lilius
Green Computing
53/67
Servers built on mobile processors
CPU characteristics
Data based on ARM Cortex-A8 benchmarks The power dissipation of the CPU is 1.4W at full speed Waking up a core takes about 650 ms The load capacity of a CPU was benchmarked with the tool Autobench resulting in 5 requests/second for a 248 KB file An arbitrary amount of CPUs can be simulated with the framework assumed that the CPU specs are given
Johan Lilius
Green Computing
54/67
Servers built on mobile processors
Comparison with DVFS
Johan Lilius
Green Computing
55/67
Servers built on mobile processors
Power savings vs QoS
Johan Lilius
DVFS has the highest QoS but wastes much energy Up to 60% in energy can be saved with only 4% degradation in QoS 20% energy can still be saved if less than 1% QoS degradation isGreen requested Computing
56/67
Servers built on mobile processors
Other results and future work Implementation of a real-life demo based on cluster consisting of 8 XM BeagleBoards This is done and we have validated the results
Implementation of PID power manager into Linux Scheduler Validated the approach using LinSched Currently implementing the framework into Linux kernel
Video transcoding “over the cloud” Evaluation of different control mechanisms State based control theory vs. PID
Evaluation of more complex settings: Is the approach applicable with VMs running on the cluster?
Johan Lilius
Green Computing
57/67
Summary
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
58/67
Summary
Summary 1
PUE measures the overall power loss in the datacenter, it does not measure the total power consumption
2
ARM based processors are more energy efficient that IA based processors for typical datacenter loads
3
IA based processors are more energy efficient in computations with large computational kernels
4
Energy Proportionality is the key to lower total power consumption in datacenters
5
It is feasible to implement Energy Proportional Power Managers into modern OS kernels
6
Many interesting research challenges remain!
Johan Lilius
Green Computing
59/67
Summary
Acknowledgements The Lean Server Team Prof. Johan Lilius Dr. Sebastien Lafond M.Sc. Simon Holmbacka M.Sc. Fareed Johkio M.Sc. Tewodros Deneke M.Sc Fredric Hällis Alumni M.Sc. M.Sc. M.Sc. M.Sc.
Johan Lilius
Jens Smeds Olle Swanfeldt-Winter Joachim Sjölund Joakim Nylund
Green Computing
60/67
Summary
Our Papers I Simon Holmbacka, Sébastien Lafond, Johan Lilius, A PID-Controlled Power Manager for Energy Efficient Web Clusters. In: International Conference on Cloud and Green Computing (CGC2011), 721-728, IEEE Computer Society, 2011. Simon Holmbacka, Jens Smeds, Sébastien Lafond, Johan Lilius, System Level Power Management for Many-Core Systems. In: (Ed.), Workshop on Micro Power Management for Macro Systems on Chip, 2011. Sébastien Lafond, Simon Holmbacka, Johan Lilius, A System Level Power Management for Web Clusters. In: COST Action IC0804 on Energy Efficiency in Large Scale Distributed Systems, 2nd Year, 127-131, IRIT, 2011. Johan Lilius
Green Computing
61/67
Summary
Our Papers II Sébastien Lafond Johan Lilius Simon Holmbacka, Power Proportional Characteristics of an Energy Manager for Web Clusters. In: Embedded Computer Systems: Architecture, Modeling and Simulation (SAMOS) 2011, 8, IEEE pres, 2011. Olle Svanfeldt-winter, Sébastien Lafond, Johan Lilius, Cost and Energy Reduction Evaluation for ARM Based Web Servers. In: International Conference on Cloud and Green Computing (CGC2011), 480-487, IEEE Computer Society, 2011.
Johan Lilius
Green Computing
62/67
Bibliography
Contents 1 Introduction 2 The economics of Data-Centers: Why Power Matters 3 Energy-proportional computing 4 Servers built on mobile processors 5 Summary 6 Bibliography
Johan Lilius
Green Computing
63/67
Bibliography
References I David Andersen, Jason Franklin, Michael Kaminsky, Amar Phanishayee, Lawrence Tan, and Vijay Vasudevan. FAWN: a fast array of wimpy nodes. SOSP ’09: Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, October 2009. LA Barroso and U Holzle. The case for energy-proportional computing. Computer, 40(12):33–37, 2007. BG Chun, G Iannaccone, G Iannaccone, R Katz, G Lee, and L Niccolini. An energy case for hybrid datacenters. ACM SIGOPS Operating Systems Review, 44(1):76–80, 2010. Johan Lilius
Green Computing
64/67
Bibliography
References II James Hamilton. Cooperative Expendable Micro-Slice Servers (CEMS): Low Cost, Low Power Servers for Internet-Scale Services. pages 1–8, December 2008. James Hamilton. Data Center Efficiency Best Practices. pages 1–29, April 2009. Urs Holzle. Brawny cores still beat wimpy cores, most of the time. IEEE Micro, pages 1–2, June 2010.
Johan Lilius
Green Computing
65/67
Bibliography
References III W Lang, JM Patel, and S Shankar. Wimpy Node Clusters: What About Non-Wimpy Workloads? 2010. Jacob Leverich and Christos Kozyrakis. On the Energy (In)efficiency of Hadoop Clusters. pages 1–5, July 2009. Reijo Maihaniemi. Energy Efficient ICT. Presentation, pages 1–27, September 2009. J Mankoff, R Kravets, and E Blevis. Some computer science issues in creating a sustainable world. Computer, 41(8):102–105, 2008. Johan Lilius
Green Computing
66/67
Bibliography
References IV
S Ruth. Green IT More Than a Three Percent Solution? IEEE Internet Computing, 2009. Alexander Szalay, Gordon Bell, H Huang, Andreas Terzis, and Alainna White. Low-power amdahl-balanced blades for data intensive computing. SIGOPS Operating Systems Review, 44(1), March 2010.
Johan Lilius
Green Computing
67/67