f(x; t), for t ⥠0 . â» Objective f(·, t) â time-varying cost. â» For fixed time t â convex program. â» Examples problems: â finding control gains. â target tracking.
Target Tracking with Dynamic Convex Optimization Alec Koppel? , Andrea Simonetto† , Aryan Mokhtari? , Geert Leus† , Alejandro Ribeiro? ? †
University of Pennsylvania, Philadelphia, PA
Delft University of Technology, Delft, The Netherlands
Global Conference on Signal & Information Processing Orlando, FL, Dec. 16, 2015 Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
1
Continuous-time Optimization
x∗ (t) := argmin f (x; t),
for t ≥ 0 .
x∈Rn
I
Objective f (·, t) ⇒ time-varying cost
I
For fixed time t ⇒ convex program
I
Examples problems: ⇒ finding control gains ⇒ target tracking ⇒ statistical model parameters
I
Focus: iterative algorithms
Photo credits: Libelium and iRobot Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
2
Iterative Methods in Time-varying Settings x∗ (t) := argmin f (x; t),
for t ≥ 0 .
x∈Rn I
One approach: sample at discrete times tk x∗ (tk ) := argmin f (x; tk ). x∈Rn
x∗ (t) I
∗
Find x (tk ) via Newton or gradient? ⇒ only viable if x∗ (t) ≈ x∗ ⇒ otherwise, minimum drifts: x∗ (tk ) 6= x∗ (tk+1 )
I
Solution is an entire trajectory.
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
3
Prediction-Correction Methods
I
Goal: xk = x(tk ) via f (x, tk ) s.t. xk → x∗ (tk ) ⇒ Main challenge: distinct x∗ (t) for each t ⇒ Stationary convex opt. methods not viable
I
Instead, adapt iteration to time variation ˜(tk+1 ) from f (·, tk ) ⇒ Predict x
x(t)
x∗ (t)
˜(tk+1 ) based on f (·, tk +1 ) ⇒ Correct x I
Main idea: track how x∗ (tk ) varies in time
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
4
Accurate Solution Tracking is Possible
I
Sample the problem at time instances tk , where h = tk − tk−1
I
We develop a method which converges to O(h2 ) nbhd. of x∗ (t) ⇒ based on prediction-correction scheme ⇒ even tighter O(h4 ) for Newton-based correction
I
Track optimal trajectory ≈ error-free
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
5
Background
I
Non-stationary opt. [Gupal, Kheisin, Nurminskii, etc., 1970s] ⇒ prediction only, require knowledge of initial optimizer x∗ (t0 )
I
Parametric programming [Robinson, Rockafellar et.al., 1980s- ] ⇒ prediction only; also require initial optimizer x∗ (t0 )
I
Distributed signal processing & control (2010-on) ⇒ Dual decomp. [Jakubeic], ADMM [Boyd], DGD [Nedich] ⇒ Correction only, arbitrary initialization x0
I
All track x∗ (t) to O(h) neighborhood; sensitive to x∗ (t0 ),
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
6
Prediction Step via Euler Integration
I
Denote xk+1|k as prediction of x∗ (tk+1 ) using objective f (x, tk )
I
Prediction via Newton step xk +1|k = xk − h (∇xx f (xk ; tk ))
−1
∇tx f (xk ; tk )
⇒ ∇xx f (xk ; tk ) ⇒ Hessian of objective at time tk ⇒ Step-size h given as sampling rate ⇒ derived via Euler integration of optimality condition residual
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
7
Prediction via Residual Tracking
I
Prediction step ⇒ from optimality condition ∇x f (x∗ (t); t) = 0
I
For any other vector x, we have ∇x f (x; t) = r(t) ⇒ r(t) is residual vector
I
Track dynamics of residual via first-order difference eqn. ∇x f (x; t) + ∇xx f (x; t)δx + ∇tx f (x; t)δt = r(t),
I
δt is variation of t. Divide above eqn. by δt, take limit δt → 0. . . −1 x˙ = − (∇xx f (x; t)) ∇tx f (x; t)
I
Using first-order Forward Euler integration, obtain prediction step
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
8
Correcting the prediction
I
Prediction xk+1|k of x∗ (tk+1 ) obtained from Euler integration xk +1|k = xk − h (∇xx f (xk ; tk ))
−1
∇tx f (xk ; tk )
⇒ uses f (x; tk ) objective at time tk
I
Correction ⇒ either gradient or Newton step using f (x; tk+1 ) ( γ∇x f (xk+1|k ; tk+1 ) −1 xk+1 = xk+1|k − ∇xx f (xk+1|k ; tk+1 ) ∇x f (xk+1|k ; tk+1 )
⇒ Multiple correction steps τ possible for large enough h
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
9
GTT: Gradient Trajectory Tracking
I
Initialize x0 arbitrarily, set h = tk+1 − tk , step-size γ > 0.
I
For k = 1, 2, . . . 1. Predict via Euler-integration of sub-optimality residual xk +1|k = xk − h (∇xx f (xk ; tk ))−1 ∇tx f (xk ; tk ) ⇒ uses f (x; tk ) objective at time tk 2. Correct predicted trajectory using gradient step xk+1 = xk +1|k − γ∇x f (xk +1|k ; tk +1 ) ⇒ uses objective f (x; tk +1 ) at time tk+1
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
10
NTT: Newton Trajectory Tracking
I
Initialize x0 arbitrarily, set h = tk+1 − tk , step-size γ > 0.
I
For k = 1, 2, . . . 1. Predict via Euler-integration of sub-optimality residual xk +1|k = xk − h (∇xx f (xk ; tk ))−1 ∇tx f (xk ; tk ) ⇒ uses f (x; tk ) objective at time tk 2. Correct predicted trajectory using Newton step xk +1 = xk+1|k − ∇xx f (xk +1|k ; tk +1 )
−1
∇x f (xk +1|k ; tk +1 )
⇒ uses objective f (x; tk +1 ) at time tk+1
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
11
Technical Conditions
I
The function f is smooth in x and uniformly in t: ⇒ Twice differentiable ⇒ m-strongly convex ⇒ eigenvalues of Hessian lower bounded ⇒ Gradients of f are L-Lipschitz continuous
I
Bounded higher-order derivatives with respect to both x and t ⇒ the mapping t 7→ x∗ (t) is one-to-one and locally Lipschitz
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
12
Convergence of GTT
Theorem i) The GTT sequence {xk } converges to nbhd. of optimal x∗ (t) lim kxk − x∗ (tk )k = O(h)
k→∞
ii) If sampling rate h and step-size γ are sufficiently small, lim kxk − x∗ (tk )k = O(h2 )
k →∞
⇒ Convergence rate is Q-linear if step-size satisfies γ < 2/L. h i−1 ⇒ Sampling rate threshold: h < Cm0 C2 1 + Cm2 ρ−1 − 1 . ⇒ ρ = max{|1−γm|, |1−γL|} ⇒ step-size, cvx, Lipschitz constant
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
13
Convergence of NTT
Theorem With 0 < K depending on grad. and cvx. constants, if rate h satisfies h ≤ min {1, K } , and the algorithm is initialized such that x0 satisfies kx0 − x∗ (t0 )k ≤ ch2 Then NTT sequence {xk } converges to nbhd. of optimal x∗ (t) lim kxk − x∗ (tk )k = O(h4 )
k→∞
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
14
Time-Derivative Approximation
I
In many application domains, ∇tx f (xk ; tk ) is not known ⇒ can be approximated by a first-order backward derivative ˜ tx f (xk ; tk ) ≈ ∇x f (xk ; tk ) − ∇x f (xk −1 ; tk −1 ) ∇ h ⇒ situations in which exact motion of target unknown
I
Can establish comparable convergence guarentees to exact algs.
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
15
Approximate Gradient Tracking
Theorem GTT method with approx. time-derivative in prediction step yields i) If step-size satisfies γ < 2/L, {xk } → x∗ (tk ) Q-linearly rate to O(h) bounded error kxk − x∗ (tk )k ≤ ρk kx0 − x∗ (t0 )k+ O(h) + O(h2 )
ii) For small enough h and γ > 0, {xk } → x∗ (tk ) Q-linearly up to a bounded O(h2 ) error kxk − x∗ (tk )k ≤ (ρσ)k kx0 − x∗ (t0 )k + O(h2 )
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
16
Approximate Newton Tracking
Theorem Suppose the NTT sampling rate satisfies h ≤ min {1, K 0 } , where K 0 depends on grad. & cvx. constants, and is initialized so that kx0 − x∗ (t0 )k ≤ ch2 Then the alg. achieves an O(h4 ) asymptotic tracking error kxk − x∗ (tk )k ≤ O(h4 )
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
17
Target Tracking Example 0
Scalar numerical example f (x; t) :=
1 κ (x − cos(ωt))2 + sin2 (ωt) exp(µx 2 ) 2 2
I Predictor-corrector methods track x∗ (t)
⇒ magnitudes less in tracking error
Tracking error }xk ´ x˚ ptk q}
10
I
−5
10
RG GTT, τ “ 1 AGT, τ “ 1 GTT, τ “ 3 GTT, τ “ 5 NTT, τ “ 1
−10
10
Bounds Th. 1 and 3
−15
10
0
1
10
10
2
3
10
10
4
5
10
10
Sampling time tk
(a) Tracked sample planar trajectory I Worst-case error:
0
10
Ophq
max{kxk − x ∗ (tk )k} where k¯ = 104 . ⇒ performance improvement clearer
Oph2 q
−5
10
Worst-case error
k>k¯
−10
10
I Newton more accurate but higher cost
RG GTT, τ AGT, τ GTT, τ GTT, τ NTT, τ
Oph4 q −15
10
−2
10
−1
10
“1 “1 “3 “5 “1 0
10
Sampling time interval h
(b) Worst-case error Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
18
Target Tracking Example Target tracking example f (x; t) :=
1 kx − y(t)k2 + µ1 exp(µ2 kx − bk2 ) , 2
I Track a moving planar object y(t)
⇒ stay close to base station b I Predictor-corrector methods track optimal
RG AGT, τ “ 1 ANT, τ “ 1 Optimal trajectory Reference path
100
Vertical Position Coordinate y [m]
I
50
0
−50
−100 −100
−50
0
50
100
Horizontal Position Coordinate x [m]
(a) Tracked sample planar trajectory
I Worst-case error: I Fix comp. cost for given sampling rate
2
10
0
10 −1 10
Worst-case error [m]
maxk >k¯ {kxk − x ∗ (tk )k}, with k¯ = 104
−3
10
I Newton computationally heavier
⇒ higher control latency I Main takeaway: use NTT (ANT) if
sampling rate large enough
−5
10
RG AGT ANT
−7
10
−10
10
−14
10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Sampling time interval h [s]
(b) Worst-case error Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
19
Concluding Remarks I
Focused on continuously-varying convex opt. problems
I
Brute force approach intractable ⇒ discrete-time sampling
I
Developed prediction-correction methods ⇒ Predict where optimal trajectory will be at next time ⇒ Once we observe info. at next sample time, make correction
I
Correction step ⇒ Gradient or Newton steps (GTT or NTT)
I
Theorem: GTT & NTT track x∗ (t) up to O(h2 ) or better
I
No problem if we don’t know exact objective time variation
I
Numerical examples illustrate utility in control domain
⇒ Correction reduces tracking error relative to existing methods
⇒ When sampling rate is large enough, better to use NTT Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
20
References
I
A. Simonetto, A. Koppel, A. Mokhtari, G. Leeus, and A. Ribeiro, “A Class of Prediction-Correction Methods for Time-Varying Convex Optimization,” IEEE Trans. Signal Process (submitted)., Sept. 2015.
I
A. Koppel, A. Simonetto, A. Mokhtari, G. Leus, and A. Ribeiro, “Target Tracking with Dynamic Convex Optimization”, IEEE Global Conference on Signal and Information Processing, Orlando, FL, Dec. 14-16, 2015.
I
A. Simonetto, A. Koppel, A. Mokhtari, G. Leeus, and A. Ribeiro “Prediction-Correction Methods for Time-Varying Convex Optimization.” in Proc. Asilomar Conf. on Signals Systems Computers, Pacific Grove, CA, November 8-11 2015.
http://seas.upenn.edu/˜akoppel/
Koppel, Simonetto, Mokhtari, Leus, Ribeiro
Target Tracking with Dynamic Convex Optimization
21