Power Capping Linux

Power Capping Linux

Len Brown, Jacob Pan, Srinivas Pandruvada

Agenda • Context • System Power Management Issues • Power Capping Overview • Power capping participants • Recommendation • Linux Power Capping Framework RFC

2

Context: Power Planning Issues • Over budgeting • Unpredictability of load

3

Power over budgeting • Worldwide, the digital warehouses use about 30 billion watts of electricity •

Equivalent to the output of 30 nuclear power plants

• On average, using only 6 percent to 12 percent of the electricity powering their servers to perform computations. • The rest was essentially used to keep servers idling and ready in case of a surge in activity that could slow or crash their operations. Source: http://www.nytimes.com/2012/09/23/technology/data-centers-waste-vast-amounts-of-energy-belying-industryimage.html?pagewanted=all&_r=0

4

Unpredictability of battery life

"Many report unpredictable spikes in battery use, and batteries becoming unnervingly hot, even when used outdoors in the shade."

Read more: http://www.dailymail.co.uk/sciencetech/article-2055562/Apple-iPhone-4S-battery-dies-12hours--users-forced-ways-patch-mend.html#ixzz2dJ5L1Uqz

5

System Power Management Issues • Limited power capacity • Limited cooling capacity • Unexpected peaks in system utilization • Downtime caused by unexpected power surges

• Mobile devices: Unpredictable battery life • Too many consumers of power

6

Power Capping Overview • Limit power consumed by devices • Dynamically adjust to meet power budget • Redistribution of power among server systems • Maximize performance

7

Power Capping Participants • CPUs • GPUs • DRAM • Others • •

8

Multimedia sub system Wireless sub system

Power Capping CPU/GPUs • Dynamic P-states adjustment • Dynamic T-states adjustment • Processor offline • Idle injection • ACPI power meter • ACPI processor aggregator

9

Power Capping Measurements • Test Setup • • •

10

Intel Ivy Bridge Dual Core Laptop Power meter: Yokogawa WT210 Test load: openssl speed sha256

P-States P0

• P-state: a voltage/frequency pair

Turbo H/W Control

• P1 : Guaranteed frequency • Pn-P1 range in OS control

P1

OS controlled States

• P0: Max possible frequency under HW control

Pn

11

Controlling P-states • Intel P State Driver •

sysfs: max_perf_pct, min_perf_pct, no_turbo

• CPUFREQ •

Sysfs: scaling_max_freq, scaling_min_freq, scaling_setspeed

• Thermal Cooling device •

sysfs: /sys/class/thermal/cooling_device# /type = “Processor”

• RAPL (Running Average Power Limit) • 12

sysfs: via power capping framework

P States Performance (Using Intel P State Driver)

13

RAPL • Power monitoring capability • Power Limiting • Performance feedback mechanism • Implemented in processor • Interface via MSR, PCIe config space

14

RAPL Domains and Interfaces

15

RAPL Performance

16

T-States • Allow Software Controlled Clock modulation • Controls stop clock duty cycle •

Time period for clock signal to drive processor

• Controlled via thermal cooling device interface for processor Processor clock

Stop clock duty cycle

Example: 25% 17

T-States Performance

18

Idle Injection • Implemented by Intel Power Clamp Cooling Driver •

sysfs: /sys/class/thermal/cooling_device# /type = “intel_powerclamp”

• Monitors and enforces idle time for each online CPU • User selectable idle ratio from 0 to 50%

19

CPU 0 Kidle_inject/0

inactive

CPU 1 Kidle_inject/1

inactive

Force idle

Force idle

inactive

inactive

Idle Injection Performance

20

CPU Offline • Migrate activity on current CPU to new CPU •

Processes, interrupts, timers

• Logical online/offline CPUs using sysfs interface •

sysfs: /sys/devices/system/cpu/cpu#/online

• Conditional physical offline/online •

depends on BIOS and kernel build flags

• Limited CPU 0 offline

21

CPU Offline Performance

22

ACPI Power Meter • Expose power meter support defined in ACPI 4.0 • Depends on BIOS support • Interface to read power over a configurable interval • Trip point configuration for notification • Configuration for power capping parameters •

23

power#_cap_min/power#_cap_max

ACPI Processor Aggregator(ACPI_PAD) • Triggered by ACPI notification only • Used to resolve short term thermal emergencies • Not a CPU Offline/online but has similar affect • Doesn’t affect cupset • Puts affected CPUs in deep C states

24

ACPI_PAD Vs. CPU Offline 350 CPU Offline

90

250

acpi_pad

80 70

200

Perf %

Watts

100 300

150 100

60 50 40

CPU Offline

30

acpi_pad

20 10

50

0 0

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

# cpu Offlined

# cpu Offlined

Test platform: Intel Romley 2 socket system 25

Recommendation • In order

26

•

RAPL

•

P State

•

Idle Injection

•

CPU Offline

•

T States

Linux Power Capping Framework • Interface to set limits on maximum power a system/sub system can consume, i.e. Power Capping • Interface to read current power for a system or a sub-system • Some API which can be used by power capping driver for easy implementation • Avoid code duplication if multiple power capping drivers are present

• RFC: http://lwn.net/Articles/562015/

27

Linux Power Capping Framework Class • Define a Power Capping Class driver interface • •

Define API and callbacks to for power capping client drivers Present a uniform interface to user space via sysfs

Registration sysfs I/F

Power cap class driver

Power cap client drivers Callbacks

28

Power Capping Class driver • Allow multiple power zones •

Power zone: Independent unit which has capability to measure and enforce power

• Allow parent child relationship among zones • Exports Sys-FS Interface •

Get Current energy consumption per power zone

•

Set Power limit per zone

• Power capping driver interface

29

Power Capping Sys-FS hierarchy example /sys/class/powercap

intel-rapl

enabled

enabled

powerclamp

intel-rap:0

name

energy_uj

intel_rapl:0:1

30

control_type n

intel-rapl:1

max_energy_range_uj

intel-rapl:n

constraint_X_name

constraint_X_power_limit_uw

constraint_X_time_window_us

Power Capping Sys-FS • Arranged as a tree, with root as a control type

31

•

Control type : Method to implement power capping. E.g. intel-rapl

•

A control type contains multiple power zones

•

Power zone node names are qualified with the control type, E.g. intelrapl:0, intel-rapl:1

•

Each power zone can have children as power zones

•

Parent child relationship should be based on the relationship of power. E.g. When child power limit is applied, parent power is also affected

Q&A

32

Thank You