Energy-Proportional Networked Systems - EPFL

Energy-Proportional Networked Systems Dejan Kostić EPFL, Switzerland Networked Systems Laboratory

Networked Systems Laboratory

2

Our Mission • Make distributed systems more reliable and easier to develop and manage

• Build networked systems that mimic the energyproportionality of biological systems

3

Networking Energy inEnergy ICT 20% of total server energy consumption (3 TWh in US in 2006) Tens of TWh/year by 2015 for broadband equipment

ACCESS NETWORK

DATACENTER NETWORK

CORE NETWORK

Datacenter

Several TWh/year for major telcos (Telefonica 4.5 TWh, Verizon 9.9 TWh) 4

Causes of networking energy consumption • Network redundancy – Achieving high availability

• Bandwidth overprovisioning – Tolerate traffic variations (address lack of QoS)

[SWITCH] 5

Energy-(un)proportionality 100

Core Router

Power (% of peak)

80

Home gateway 60

Typical utilization levels

40

Existing networking hardware

20

Ideal energy-proportionality 0

0

20

40

60

Utilization (%)

80

100 6

Networking energy outlook • More demands will result in further increases – Video streaming, Cloud computing

• CMOS reaching a plateau in power-efficiency – Cooling costs of new equipment will increase • 1 MW for latest Cisco platform, CRS-1

• Power is not limitless – 60 Amps per rack Rate of traffic increase > rate in which underlying technologies improve their energy efficiency 7

Datacenter network (yesterday’s tree)

8

Datacenter network (today’s fat tree)

9

Reduced or no cooling

10

Threats to Internet’s growth Energy deliver/ Power consumption, Coolingdelivery/ Power problems Cooling problems

DATACENTER NETWORK

CORE NETWORK ACCESS NETWORK

Datacenter

Excessive energy consumption 11

Goal: Energy-Proportional Networked Systems 100

Power (% of peak)

80

60

40

20

Goal Ideal energy-proportionality

0

0

20

40

60

Utilization (%)

80

100 12

Make all devices energy-proportional? 100

… but it is hard: • CMOS energy-efficiency limits • Performance penalties • Always-on components

Power (% of peak)

80

60

CPU

DRAM

Disk

Other

100.00

40

Power (% of peak)

90.00 80.00 70.00 60.00 50.00

20

40.00 30.00 20.00

Ideal energy-proportionality

10.00 0.00

0 Idle 0

7

14

21

20

29

36

43

50

57

64

71

40Compute load (%) 60

“Highly efficient” Google Utilization (%)server:

79

86

80

93

100

100 13

Network-wide energy-proportionality Sleeping saves energy Dynamically match resources to the demand  make the network energy-proportional

14

A simple optimization problem? 100

Power (% of peak)

80

Computational intensity 60

Maintaining SLOs

Avoiding oscillations Responsiveness to traffic variations

40

Ease of deployment 20

Goal Ideal energy-proportionality

0

0

20

40

60

Utilization (%)

80

100 15

Overview REsPoNse

Energy deliver/ Power consumption, Coolingdelivery/ Power problems Cooling problems

DATACENTER NETWORK


Datacenter

BH2 Excessive energy consumption 16

Access dominates energy consumption

Backbone/ Metro/ Transport

20-30%

70-80%

ACCESS

17

A typical DSL access network ACCESS ISP PART USER PART Gateway CORE

METRO

18

A typical DSL access network ACCESS ISP PART USER PART Gateway CORE

METRO

Cable bundle Central Office

DSL Access Multiplexers (DSLAMs) 19

WHY DOES THE ACCESS CONSUME SO MUCH?

20

#1: Huge number of devices Individually, they do not consume a lot

But collectively …

2 orders of magnitude more gateways than DSLAMs 1 order of magnitude more DSLAMs than metro devices PhotoBlackburn 2 orders of magnitude more DSLAMs than backbone devices 21

2#: High per bit energy consumption Backbone/ Metro

ACCESS

At full load, access devices 2-3 orders of magnitude higher than metro/backbone 22

3#: Utilization < 10%

Average utilization [%]

Daily utilization of 10K access links in a commercial ADSL provider 10%

uplink

8%

downlink

6% 4% 2%

0% 0

5

10

Time [h]

15

20 23

Sleeping saves energy

Sleep-on-Idle (SoI) Devices enter sleep mode upon periods of inactivity 24

SoI fails in access networks ACCESS ISP PART USER PART

O

An ADSL line needs 1 minute to wake up … but cannot enjoy a minute’s sleep 25

What if we can put 80% of gateways to sleep? 100 W

15 W

1 W per modem Save big fraction at the user side

ISPs … not so much 26

Line cards very unlikely to sleep by SoI Line cards

DSLAM

Modem on Modem off

Static assignment of lines to DSLAM ports is a problem

27

OUR APPROACH [SIGCOMM ‘11] ⟹Greening the user part: aggregation ⟹Greening the ISP part: line switching

28

Greening the user part – Aggregation

Broadband Hitch-Hiking (BH2) Threshold-based heuristic algorithm: direct traffic to neighbor gateways during light traffic conditions

On average 5-6 WiFi networks overlap in typical urban settings 29

Broadband Hitch-Hiking 2 (BH )

Load on neighbor home gateway gateway is low is high low  direct  look golight back for traffic to to a neighbor another home gateway neighbor gateway or goand back lettohome homegateway gatewaysleep 30

Greening the ISP part – Line switching

DSLAM

Line cards

40-way switch Full switching maximizes savings … but cost quickly grows with the number of ways 31

Small 4-way switches are enough 4-way switches

DSLAM

Line cards

Put line cards to sleep Each micro-electro-mechanical k-switch packs active linesswitches to the top Simple with near-zero power consumption 32

How much energy can we save? Energy savings vs no-sleep [%]

[trace-based simulation] 100

Optimal

80

BH2 + k-switch

60 40

SoI

20 0 0

5

10

Time [h]

15

20

24

BH2 + k-switch saves 66% Optimal savings are 80% 33

A performance bonus Bonus: reduced crosstalk 14

12

4

24 11

1

23 10

3

22

9

15

6 17

2 8

16

5

7

21 20 19

18

50

Avg. speedup [%]

13

62 Mbps; loop lengths 50-600 m

40 30 20 10 0 0

2

4

6

8

10

12

16

Number of inactive lines

20

Powering off lines makes the remaining … go faster due to reduced crosstalk! 34

Overview REsPoNse

Energy deliver/ Power consumption, Coolingdelivery/ Power problems Cooling problems

DATACENTER NETWORK


Datacenter

BH2 Excessive energy consumption 35

Routing table computation • Goal: match network resources to traffic • Routing that minimizes energy consumption – Multi-commodity flow problem, but with additional constraints for energy objective: Links + routers (switches) on/off

– Problem is computationally intensive – Heuristics take 5-15 minutes for small topologies

When traffic demand changes, optimal routing changes! 36

r ]

6

[Geant2 - European academic network, 15-day trace]

o

5

e

r

4

[ p

3

t e

t r a c e g r a n u la r it y ( u p p e r b o u n d )

M a y -2 8

J u n -1

J u n -5

J u n -9

t i o o c

2

e

Routing table recomputed 3-4 times per hour! (state-of-the-art)

3

m

p

T im e

4

t a

t r a f f ic d e m a n d s

u

1

0

5

n

r a

2

R

D

e

m

a

n

d

h

[ G

u

b

p

s

]

How often is recomputation needed?

1

0

M a y -2 8

J u n -1

J u n -5 T im e

J u n -9

37

Issues with recomputation Traffic Volume

1) Recomputation wastes energy or causes congestion 2) Oscillations 3) Complexity

Time

Recomputation causing congestion

Recomputation causing energy waste 38

Can we precompute routing tables? One routing table used 60% of the time

Fraction of time an energy-optimal routing configuration is used (Geant2 trace) Too many routing configurations 39

Insight CDF of Optimal paths included

120 100 80

Geant FatTree

60 40

20 0 1

2

3

4

5

Number of alternative paths

Just a few precomputed paths offer near-optimal energy savings 40

REsPoNse (Responsive Energy-Proportional Networks)

Always-on paths provide a routing that can carry low to medium amounts of traffic at the lowest energy consumption On-demand paths start carrying traffic when the load is beyond the capacity offered by the always-on paths Failover paths are designed to minimize the impact of single failures

41

Service-Level Objectives

REsPoNse Overview Energy-aware Traffic Engineering (EATe) [e-Energy ‘10]

Runtime Traffic Measurement [COMSNETS ‘09] Online components Offline components

Energy-proportional Routing Table Computation [CoNEXT ‘11]

Traffic Estimation

42

REsPoNse routing example G

D A

K

B

C

E

F

H

J

43

Traffic Volume

REsPoNse in action

Time

Online adaptation Recomputation causing congestion

Recomputation causing energy waste 44

REsPoNse benefits • • • •

Energy savings match state-of-the-art Quick, stable adaptation to traffic changes Deployable Power/cooling provisioning for common case

45

r k

120

• Replayed a 15-day trace • 2 power models

100

o

r i g

i n

a

l

n

e

t w

o

Responsiveness/Energy-Proportionality (Geant)

– Today – Future (static power is significantly reduced)

60

40

P

o

w

e

r

[ %

80

20

0

ospf R E sP oN se R E s P o N s e ( A lt e r n a t iv e H W m o d e l) M a y -2 8

J u n -1

J u n -9

s ]

T im e 6

REsPoNse saves 30% - 45% with adding only 1 carefully precomputed routing table

5

4

3

D

e

m

a

n

d

[ G

b

p

J u n -5

2

1

0

tr a ffic d e m a n d s M a y -2 8

J u n -1

J u n -5 T im e

J u n -9

46

Responsiveness/stability 10 Click open-source routers in a diamond topology (16 ms per-hop latency ) 8

middle lower upper

7 6

Rate (Mbps)

5 4 3 2 1 0 4

4.5

5

5.5

6

6.5

Time elapsed (s)

EATe starts running

Link failure

EATe quickly and in a stable manner shifts traffic as needed (either to save energy or to avoid failed links) 47

resources1

Workload 1 volume/type

Map cloud workloads to minimal resources for power and cooling

Time resources 2


State-of-the-art performance tuning takes several minutes, every time the workload changes Time

resources3


DejaVu [ASPLOS ’12]: 10-15 seconds to adapt

Time

Time

Conclusions • Energy can substantially limit growth of networked systems • BH2 (Aggregation) + switching saves 66% of access energy • Turning DSL modems off increases performance • REsPoNse: hybrid approach in backbone and data centers • Enables provisioning power/cooling for the common case

49

Thanks! BH2:

Marco Canini, Eduard Goma, Alberto Lopez, Nikolaos Laoutaris, Pablo Rodriguez, Rade Stanojević, Pablo Yague

REsPoNse:

Nedeljko Vasić, Dejan Novaković, Satyam Shekhar, Prateek Bhurat, Marco Canini

DejaVu:

Nedeljko Vasić, Dejan Novaković, Svetozar Miucin, Ricardo Bianchini

50