Code generation for real-time embedded convex optimization. Spencer .... custom code solve time: 50µs (SDPT3: 100ms) ..... Sigma (n,n) psd # covariance.
Real-Time Embedded Convex Optimization
Stephen Boyd joint work with Michael Grant, Jacob Mattingley, Yang Wang Electrical Engineering Department, Stanford University
Spencer Schantz Lecture, Lehigh University, 12/6/10
Outline
• Real-time embedded convex optimization • Examples • Parser/solvers for convex optimization • Code generation for real-time embedded convex optimization
Spencer Schantz Lecture, Lehigh University, 12/6/10
1
Embedded optimization
• embed solvers in real-time applications • i.e., solve an optimization problem at each time step • used now for applications with hour/minute time-scales – process control – supply chain and revenue management – trading
Spencer Schantz Lecture, Lehigh University, 12/6/10
2
What’s new embedded optimization at millisecond/microsecond time-scales millions problem size −→
traditional problems →
thousands
← our focus tens microseconds
seconds
hours days
solve time −→ Spencer Schantz Lecture, Lehigh University, 12/6/10
3
Applications • real-time resource allocation – update allocation as objective, resource availabilities change • signal processing – estimate signal by solving optimization problem over sliding window – replace least-squares estimates with robust (Huber, ℓ1) versions – re-design (adapt) coefficients as signal/system model changes • control – closed-loop control via rolling horizon optimization – real-time trajectory planning • all of these done now, on long (minutes or more) time scales but could be done on millisecond/microsecond time scales Spencer Schantz Lecture, Lehigh University, 12/6/10
4
Outline
• Real-time embedded convex optimization • Examples • Parser/solvers for convex optimization • Code generation for real-time embedded convex optimization
Spencer Schantz Lecture, Lehigh University, 12/6/10
5
Multi-period processor speed scheduling • processor adjusts its speed st ∈ [smin, smax] in each of T time periods • energy consumed in period t is φ(st); total energy is E =
PT
t=1 φ(st )
• n jobs – job i available at t = Ai; must finish by deadline t = Di – job i requires total work Wi ≥ 0 • Sti ≥ 0 is effective processor speed allocated to job i in period t st =
n X i=1
Spencer Schantz Lecture, Lehigh University, 12/6/10
Sti,
Di X
Sti ≥ Wi
t=Ai
6
Minimum energy processor speed scheduling • choose speeds st and allocations Sti to minimize total energy E PT
minimize E = t=1 φ(st) max subject to smin ≤ s ≤ s , t = 1, . . . , T t Pn st = i=1 Sti, t = 1, . . . , T PDi i = 1, . . . , n t=Ai Sti ≥ Wi , • a convex problem when φ is convex • more sophisticated versions include – multiple processors – other constraints (thermal, speed slew rate, . . . ) – stochastic models for (future) data Spencer Schantz Lecture, Lehigh University, 12/6/10
7
Example • T = 16 periods, n = 12 jobs • smin = 1, smax = 6, φ(st) = s2t • jobs shown as bars over [Ai, Di] with area ∝ Wi 40 12
35
10
25
8
job i
φt(st)
30
20
6
15 4
10 2
5
0
0
1
2
3
4
5
st Spencer Schantz Lecture, Lehigh University, 12/6/10
6
7
0
0
2
4
6
8
10
12
14
16
18
t 8
Optimal and uniform schedules • uniform schedule: Sti = Wi/(Di − Ai); gives E unif = 374.1 ⋆ gives E ⋆ = 167.1 • optimal schedule Sti
uniform
6
6
5
5
4
4
st
st
optimal
3
3
2
2
1
1
0
0
2
4
6
8
10
12
14
t Spencer Schantz Lecture, Lehigh University, 12/6/10
16
18
0
0
2
4
6
8
10
12
14
16
18
t 9
Minimum time control with active vibration supression
F T
• force F (t) moves object modeled as 3 masses (2 vibration modes) • tension T (t) used for active vibration supression • goal: move object to commanded position as quickly as possible, with |F (t)| ≤ 1,
Spencer Schantz Lecture, Lehigh University, 12/6/10
|T (t)| ≤ 0.1
10
Ignoring vibration modes • treat object as single mass; apply only F • analytical (‘bang-bang’) solution 3
F (t)
1
2.5
y3(t)
2
0.5 0 −0.5 −1 −2
0
2
4
6
8
10
12
14
16
18
20
10
12
14
16
18
20
t
1.5
0.1
T (t)
1
0.5
0.05 0 −0.05 −0.1
0 −2
0
2
4
6
8
10
12
14
16
18
t Spencer Schantz Lecture, Lehigh University, 12/6/10
20
−2
0
2
4
6
8
t 11
With vibration modes • no analytical solution, but reduces to a quasiconvex problem • can be solved by solving a small number of convex problems 3
F (t)
1
2.5
y3(t)
2
0.5 0 −0.5 −1 −2
0
2
4
6
8
10
12
14
16
18
20
10
12
14
16
18
20
t
1.5
0.1
T (t)
1
0.5
0.05 0 −0.05 −0.1
0 −2
0
2
4
6
8
10
12
14
16
18
t Spencer Schantz Lecture, Lehigh University, 12/6/10
20
−2
0
2
4
6
8
t 12
Grasp force optimization • choose K grasping forces on object to – resist external wrench (force and torque) – respect friction cone constraints – minimize maximum grasp force • convex problem (second-order cone program or SOCP): maxi kf (i)k2 P (i) (i) ext subject to Q f = f i P (i) (i) (i) ext p × (Q f ) = τ i 1/2 (i)2 (i)2 (i) µi f z ≥ f x + f y minimize
max contact force force equillibrium torque equillibrium friction cone constraints
variables f (i) ∈ R3, i = 1, . . . , K (contact forces) Spencer Schantz Lecture, Lehigh University, 12/6/10
13
Example
Spencer Schantz Lecture, Lehigh University, 12/6/10
14
Grasp force optimization solve times
• example with K = 5 fingers (grasp points) • reduces to SOCP with 15 vars, 6 eqs, 5 3-dim SOCs • custom code solve time: 50µs (SDPT3: 100ms)
Spencer Schantz Lecture, Lehigh University, 12/6/10
15
Robust Kalman filtering • estimate state of a linear dynamical system driven by IID noise • sensor measurements have occasional outliers (failures, jamming, . . . ) • model:
xt+1 = Axt + wt,
yt = Cxt + vt + zt
– wt ∼ N (0, W ), vt ∼ N (0, V ) – zt is sparse; represents outliers, failures, . . . • (steady-state) Kalman filter (for case zt = 0): – time update: x ˆt+1|t = Aˆ xt|t – measurement update: x ˆt|t = x ˆt|t−1 + L(yt − C x ˆt|t−1) • we’ll replace measurement update with robust version to handle outliers Spencer Schantz Lecture, Lehigh University, 12/6/10
16
Measurement update via optimization • standard KF: x ˆt|t is solution of quadratic problem minimize v T V −1v + (x − x ˆt|t−1)T Σ−1(x − x ˆt|t−1) subject to yt = Cx + v with variables x, v (simple analytic solution) • robust KF: choose x ˆt|t as solution of convex problem minimize v T V −1v + (x − x ˆt|t−1)T Σ−1(x − x ˆt|t−1) + λkzk1 subject to yt = Cx + v + z with variables x, v, z (requires solving a QP) Spencer Schantz Lecture, Lehigh University, 12/6/10
17
Example
• 50 states, 15 measurements • with prob. 5%, measurement components replaced with (yt)i = (vt)i • so, get a flawed measurement (i.e., zt 6= 0) every other step (or so)
Spencer Schantz Lecture, Lehigh University, 12/6/10
18
State estimation error kx − x ˆt|tk2 for KF (red); robust KF (blue); KF with z = 0 (gray)
0.5
Spencer Schantz Lecture, Lehigh University, 12/6/10
19
Robust Kalman filter solve time
• robust KF requires solution of QP with 95 vars, 15 eqs, 30 ineqs • automatically generated code solves QP in 120 µs (SDPT3: 120 ms) • standard Kalman filter update requires 10 µs
Spencer Schantz Lecture, Lehigh University, 12/6/10
20
Linearizing pre-equalizer • linear dynamical system with input saturation
∗h
y
• we’ll design pre-equalizer to compensate for saturation effects u
equalizer
v
Spencer Schantz Lecture, Lehigh University, 12/6/10
∗h
y
21
Linearizing pre-equalizer
∗h u
e equalizer v
∗h
• goal: minimize error e (say, in mean-square sense) • pre-equalizer has T sample look-ahead capability Spencer Schantz Lecture, Lehigh University, 12/6/10
22
• system:
xt+1 = Axt + B sat(vt),
• (linear) reference system:
yt = Cxt
ref xref = Ax t + But , t+1
ytref = Cxref t
• et = Cxref t − Cxt • state error x ˜t = xref t − xt satisfies x ˜t+1 = A˜ xt + B(ut − vt),
et = C x ˜t
• to choose vt, solve QP Pt+T
2 minimize ˜Tt+T +1P x ˜t+T +1 τ =t eτ + x subject to x ˜τ +1 = A˜ xτ + B(uτ − vτ ), eτ = C x ˜τ , |vτ | ≤ 1, τ = t, . . . , t + T
τ = t, . . . , t + T
P gives final cost; obvious choice is output Grammian Spencer Schantz Lecture, Lehigh University, 12/6/10
23
Example
• state dimension n = 3; h decays in around 35 samples • pre-equalizer look-ahead T = 15 samples • input u random, saturates (|ut| > 1) 20% of time
Spencer Schantz Lecture, Lehigh University, 12/6/10
24
Outputs desired (black), no compensation (red), equalized (blue) 2.5
0
−2.5
Spencer Schantz Lecture, Lehigh University, 12/6/10
25
Errors no compensation (red), with equalization (blue) 0.75
0
−0.75
Spencer Schantz Lecture, Lehigh University, 12/6/10
26
Inputs no compensation (red), with equalization (blue) 2.5
0
−2.5
Spencer Schantz Lecture, Lehigh University, 12/6/10
27
Linearizing pre-equalizer solve time
• pre-equalizer problem reduces to QP with 96 vars, 63 eqs, 48 ineqs • automatically generated code solves QP in 600µs (SDPT3: 310ms)
Spencer Schantz Lecture, Lehigh University, 12/6/10
28
Constrained linear quadratic stochastic control • linear dynamical system:
xt+1 = Axt + But + wt
– xt ∈ Rn is state; ut ∈ U ⊂ Rm is control input – wt is IID zero mean disturbance • ut = φ(xt), where φ : Rn → U is (state feedback) policy • objective: minimize average expected stage cost (Q ≥ 0, R > 0) T −1 1 X T T E xt Qxt + ut Rut J = lim T →∞ T t=0
• constrained LQ stochastic control problem: choose φ to minimize J Spencer Schantz Lecture, Lehigh University, 12/6/10
29
Constrained linear quadratic stochastic control • optimal policy has form φ(z) = argmin{v T Rv + E V (Az + Bv + wt)} v∈U
where V is Bellman function – but V is hard to find/describe except when U = Rm (in which case V is quadratic) • many heuristic methods give suboptimal policies, e.g. – projected linear control – control-Lyapunov policy – model predictive control, certainty-equivalent planning Spencer Schantz Lecture, Lehigh University, 12/6/10
30
Control-Lyapunov policy • also called approximate dynamic programming, horizon-1 model predictive control • CLF policy is φclf (z) = argmin{v T Rv + E Vclf (Az + Bv + wt)} v∈U
where Vclf : Rn → R is the control-Lyapunov function • evaluating ut = φclf (xt) requires solving an optimization problem at each step • many tractable methods can be used to find a good Vclf • often works really well Spencer Schantz Lecture, Lehigh University, 12/6/10
31
Quadratic control-Lyapunov policy • assume – polyhedral constraint set: U = {v | F v ≤ g}, g ∈ Rk – quadratic control-Lyapunov function: Vclf (z) = z T P z • evaluating ut = φclf (xt) reduces to solving QP minimize v T Rv + (Az + Bv)T P (Az + Bv) subject to F v ≤ g
Spencer Schantz Lecture, Lehigh University, 12/6/10
32
Control-Lyapunov policy evaluation times • tclf : time to evaluate φclf (z) • tlin: linear policy φlin(z) = Kz • tkf : Kalman filter update • (SDPT3 times around 1000× larger) n
m
k
tclf (µs)
tlin (µs)
tkf (µs)
15
5
10
35
1
1
50
15
30
85
3
9
100
10
20
67
4
40
1000
30
60
298
130
8300
Spencer Schantz Lecture, Lehigh University, 12/6/10
33
Outline
• Real-time embedded convex optimization • Examples • Parser/solvers for convex optimization • Code generation for real-time embedded convex optimization
Spencer Schantz Lecture, Lehigh University, 12/6/10
34
Parser/solvers for convex optimization • specify convex problem in natural form – declare optimization variables – form convex objective and constraints using a specific set of atoms and calculus rules (disciplined convex programming) • problem is convex-by-construction • easy to parse, automatically transform to standard form, solve, and transform back • implemented using object-oriented methods and/or compiler-compilers • huge gain in productivity (rapid prototyping, teaching, research ideas) Spencer Schantz Lecture, Lehigh University, 12/6/10
35
Example: cvx • parser/solver written in Matlab • convex problem, with variable x ∈ Rn; A, b, λ, F , g constants minimize
kAx − bk2 + λkxk1
subject to F x ≤ g • cvx specification: cvx begin % declare vector variable variable x(n) minimize (norm(A*x-b,2) + lambda*norm(x,1)) subject to F*x
