Apr 13, 2011 - Virtual-CPU Scheduling in the Quest Operating System ... Fixed Priority, Rate-Monotonic Sporadic Servers ... Bandwidth-preserving Server.
Virtual-CPU Scheduling in the Quest Operating System Matthew Danish, Ye Li and Richard West Boston University
April 13, 2011
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
1 / 19
Focus of this Work
Predictability and temporal isolation Guaranteed resource allocation over specific windows of time
Integrated management of tasks and I/O Programs sleeping and waking Periodic tasks Interrupts from I/O devices
Supporting temporal isolation using Virtual CPUs Consolidate threads on Virtual CPUs Divide up physical resources Statically scheduled
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
2 / 19
Background
Building a new OS to focus on safety, predictability, and efficiency. Memory safety, program correctness Temporal isolation Optimized use of hardware resources
Why not Linux? Start from a clean slate Design freedom Linux was never meant to be real-time Linux is a constantly moving target
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
3 / 19
Quest
Small x86 SMP research operating system Under 11,000 lines of core kernel code Support for ACPI, PCI bus, USB, ATA drives, TCP/IP (lwIP)
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
4 / 19
VCPU Scheduling Subsystem Scheduling Class
Scheduling Thread
Core 1
...
Main VCPU Core m I/O VCPU
Shared Cache
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
5 / 19
Main VCPUs
VCPU 0 (C=1, T=3) VCPU 1 (C=1, T=4) VCPU 2 (C=2, T=10)
Timeline
Budgets
Fixed Priority, Rate-Monotonic Sporadic Servers Budgets replenish on a timer Replenishments can be split, or merged Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
6 / 19
I/O VCPUs I/O VCPU (Utilization=20%)
1/20%
2/20%
1/20%
For tasks that react to hardware Inherits priority from Main VCPU Bandwidth-preserving Server I/O VCPU arrangement is part of policy Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
7 / 19
Evaluation
Using single core of an Intel Core2 Extreme QX6700 2.6GHz 4GB DDR SDRAM available Intel 8254x-series “e1000” network adapter UHCI-based USB connected to Mass Storage device Parallel ATA CD-ROM drive
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
8 / 19
Sporadic vs Bandwidth-Preserving Servers Experiment: Packet handling scheduling overhead Ping-flood at t = 50 Scenario 1
Scenario 2
Net
Net
Bandwidth Preserving Server
Sporadic Server Bandwidth Preserving Server
Sporadic Server
I/O VCPU
Network Card Danish, Li and West (BU)
Network Card
VCPU Scheduling in the Quest OS
April 13, 2011
9 / 19
Sporadic vs Bandwidth-Preserving Servers Experiment: Packet handling scheduling overhead Ping-flood at t = 50 Sporadic server overhead much higher 0.7
PIBS SS
0.6
CPU%
0.5 0.4 0.3 0.2 0.1 40
Danish, Li and West (BU)
60
80
100 120 Time (seconds)
VCPU Scheduling in the Quest OS
140
160
180
April 13, 2011
9 / 19
I/O VCPUs Inherit Priority Experiment: Reading from CD-ROM on behalf of VCPU 1
Name VCPU2 VCPU0 VCPU1 VCPU3 IOVCPU
Capacity 1 2 2 1 10%
Danish, Li and West (BU)
Period 4 5 8 10
VCPU Scheduling in the Quest OS
April 13, 2011
10 / 19
I/O VCPUs Inherit Priority Experiment: Reading from CD-ROM on behalf of VCPU 1 Only lower-priority VCPU 3 loses CPU time Name VCPU2 VCPU0 VCPU1 VCPU3 IOVCPU
Capacity 1 2 2 1 10%
Period 4 5 8 10
100 90 80 70
CPU%
60
IOVCPU VCPU3 VCPU2 VCPU1 VCPU0
50 40 30 20 10 0 Without CD-ROM I/O
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
With CD-ROM I/O
April 13, 2011
10 / 19
Shared vs Separate I/O VCPUs Two I/O tasks: USB storage reading, network packet handling Experiment: single shared I/O VCPU vs two separate I/O VCPUs Scenario 1
Scenario 2
USB Net
USB
Net
USB Disk
Network Card
I/O VCPU
USB Disk Danish, Li and West (BU)
Network Card
VCPU Scheduling in the Quest OS
April 13, 2011
11 / 19
Shared vs Separate I/O VCPUs Two I/O tasks: USB storage reading, network packet handling Experiment: single shared I/O VCPU vs two separate I/O VCPUs Compare USB bandwidth during a network ping-flood Scenario 1
Scenario 2
USB Net
USB
Net
USB Disk
Network Card
I/O VCPU
USB Disk Danish, Li and West (BU)
Network Card
VCPU Scheduling in the Quest OS
April 13, 2011
11 / 19
Shared vs Separate I/O VCPUs Two I/O tasks: USB storage reading, network packet handling Experiment: single shared I/O VCPU vs two separate I/O VCPUs Compare USB bandwidth during a network ping-flood 200
180
USB Network USB (pingflood)
kB/s
160
140
120
100
80 Shared
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
Separate
April 13, 2011
11 / 19
Scheduler Overhead Experiment: scaling as the number of Main VCPUs increases
Run 1
Danish, Li and West (BU)
Run 2
Run 3
Run 4
Run 5
VCPU Scheduling in the Quest OS
Run 6
April 13, 2011
12 / 19
Scheduler Overhead Experiment: scaling as the number of Main VCPUs increases Scheduler overhead increases linearly 2
100μs 1 ms
CPU%
1.5
1
0.5
0 4
8
12
16
20
24
# VCPUs
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
12 / 19
Conclusions
Virtual CPUs for predictability and temporal isolation Main VCPUs partition physical CPU resources I/O VCPUs regulate servicing of hardware events Two-tiered scheduling hierarchy simplifies design Basis for performance isolation Real-time and embedded systems Virtual machine scheduling Partitioning of distributed cloud computing resources
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
13 / 19
Future Directions
Real-time VCPU scheduling on multiple processors and cores Minimize cache contention and communication costs Maximize instructions per cycle
Better safety with hardware sandboxing or software techniques Virtualization Programming language support
Componentization with predictable communication Static verification of useful properties
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
14 / 19
Development
Acknowledgements Richard West and Gary Wong (original developers) Matthew Danish and Ye Li (SMP and VCPU implementation)
Source: http://QuestOS.github.com/ http://www.cs.bu.edu/fac/richwest/quest.html
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
15 / 19
Thank You
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
16 / 19
Extra Slides
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
17 / 19
Premature Replenishment
VCPU 0 (Capacity=10, Period=40, Start=1) VCPU 1 (Capacity=20, Period=50, Start=0)
+18
0
2
4
6
8 10 2
4
6
8 20 2
4
6
8 30 2
4
6
8 40 2
4
+18
6
8 50 2
4
6
8 60 2
4
6
8 70 2
4
6
8 80 2
4
6
8 90 2
4
6
8 100 2
4
6
8 110
VCPU 1 effective capacity=27 over period=50
Over-capacity exploit without proper splitting of replenishments Defects of the POSIX Sporadic Server and How to Correct Them
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
18 / 19
Fin
Danish, Li and West (BU)
VCPU Scheduling in the Quest OS
April 13, 2011
19 / 19
