Target Tracking with Dynamic Convex Optimization

Target Tracking with Dynamic Convex Optimization Alec Koppel? , Andrea Simonetto† , Aryan Mokhtari? , Geert Leus† , Alejandro Ribeiro? ? †

University of Pennsylvania, Philadelphia, PA

Delft University of Technology, Delft, The Netherlands

Global Conference on Signal & Information Processing Orlando, FL, Dec. 16, 2015 Koppel, Simonetto, Mokhtari, Leus, Ribeiro

Target Tracking with Dynamic Convex Optimization

1

Continuous-time Optimization

x∗ (t) := argmin f (x; t),

for t ≥ 0 .

x∈Rn

I

Objective f (·, t) ⇒ time-varying cost

I

For fixed time t ⇒ convex program

I

Examples problems: ⇒ finding control gains ⇒ target tracking ⇒ statistical model parameters

I

Focus: iterative algorithms

Photo credits: Libelium and iRobot Koppel, Simonetto, Mokhtari, Leus, Ribeiro


2

Iterative Methods in Time-varying Settings x∗ (t) := argmin f (x; t),

for t ≥ 0 .

x∈Rn I

One approach: sample at discrete times tk x∗ (tk ) := argmin f (x; tk ). x∈Rn

x∗ (t) I

∗

Find x (tk ) via Newton or gradient? ⇒ only viable if x∗ (t) ≈ x∗ ⇒ otherwise, minimum drifts: x∗ (tk ) 6= x∗ (tk+1 )

I

Solution is an entire trajectory.

Koppel, Simonetto, Mokhtari, Leus, Ribeiro


3

Prediction-Correction Methods

I

Goal: xk = x(tk ) via f (x, tk ) s.t. xk → x∗ (tk ) ⇒ Main challenge: distinct x∗ (t) for each t ⇒ Stationary convex opt. methods not viable

I

Instead, adapt iteration to time variation ˜(tk+1 ) from f (·, tk ) ⇒ Predict x

x(t)

x∗ (t)

˜(tk+1 ) based on f (·, tk +1 ) ⇒ Correct x I

Main idea: track how x∗ (tk ) varies in time



4

Accurate Solution Tracking is Possible

I

Sample the problem at time instances tk , where h = tk − tk−1

I

We develop a method which converges to O(h2 ) nbhd. of x∗ (t) ⇒ based on prediction-correction scheme ⇒ even tighter O(h4 ) for Newton-based correction

I

Track optimal trajectory ≈ error-free



5

Background

I

Non-stationary opt. [Gupal, Kheisin, Nurminskii, etc., 1970s] ⇒ prediction only, require knowledge of initial optimizer x∗ (t0 )

I

Parametric programming [Robinson, Rockafellar et.al., 1980s- ] ⇒ prediction only; also require initial optimizer x∗ (t0 )

I

Distributed signal processing & control (2010-on) ⇒ Dual decomp. [Jakubeic], ADMM [Boyd], DGD [Nedich] ⇒ Correction only, arbitrary initialization x0

I

All track x∗ (t) to O(h) neighborhood; sensitive to x∗ (t0 ),



6

Prediction Step via Euler Integration

I

Denote xk+1|k as prediction of x∗ (tk+1 ) using objective f (x, tk )

I

Prediction via Newton step xk +1|k = xk − h (∇xx f (xk ; tk ))

−1

∇tx f (xk ; tk )

⇒ ∇xx f (xk ; tk ) ⇒ Hessian of objective at time tk ⇒ Step-size h given as sampling rate ⇒ derived via Euler integration of optimality condition residual



7

Prediction via Residual Tracking

I

Prediction step ⇒ from optimality condition ∇x f (x∗ (t); t) = 0

I

For any other vector x, we have ∇x f (x; t) = r(t) ⇒ r(t) is residual vector

I

Track dynamics of residual via first-order difference eqn. ∇x f (x; t) + ∇xx f (x; t)δx + ∇tx f (x; t)δt = r(t),

I

δt is variation of t. Divide above eqn. by δt, take limit δt → 0. . . −1 x˙ = − (∇xx f (x; t)) ∇tx f (x; t)

I

Using first-order Forward Euler integration, obtain prediction step



8

Correcting the prediction

I

Prediction xk+1|k of x∗ (tk+1 ) obtained from Euler integration xk +1|k = xk − h (∇xx f (xk ; tk ))

−1

∇tx f (xk ; tk )

⇒ uses f (x; tk ) objective at time tk

I

Correction ⇒ either gradient or Newton step using f (x; tk+1 ) ( γ∇x f (xk+1|k ; tk+1 ) −1 xk+1 = xk+1|k − ∇xx f (xk+1|k ; tk+1 ) ∇x f (xk+1|k ; tk+1 )

⇒ Multiple correction steps τ possible for large enough h



9

GTT: Gradient Trajectory Tracking

I

Initialize x0 arbitrarily, set h = tk+1 − tk , step-size γ > 0.

I

For k = 1, 2, . . . 1. Predict via Euler-integration of sub-optimality residual xk +1|k = xk − h (∇xx f (xk ; tk ))−1 ∇tx f (xk ; tk ) ⇒ uses f (x; tk ) objective at time tk 2. Correct predicted trajectory using gradient step xk+1 = xk +1|k − γ∇x f (xk +1|k ; tk +1 ) ⇒ uses objective f (x; tk +1 ) at time tk+1



10

NTT: Newton Trajectory Tracking

I

Initialize x0 arbitrarily, set h = tk+1 − tk , step-size γ > 0.

I

For k = 1, 2, . . . 1. Predict via Euler-integration of sub-optimality residual xk +1|k = xk − h (∇xx f (xk ; tk ))−1 ∇tx f (xk ; tk ) ⇒ uses f (x; tk ) objective at time tk 2. Correct predicted trajectory using Newton step xk +1 = xk+1|k − ∇xx f (xk +1|k ; tk +1 )

−1

∇x f (xk +1|k ; tk +1 )

⇒ uses objective f (x; tk +1 ) at time tk+1



11

Technical Conditions

I

The function f is smooth in x and uniformly in t: ⇒ Twice differentiable ⇒ m-strongly convex ⇒ eigenvalues of Hessian lower bounded ⇒ Gradients of f are L-Lipschitz continuous

I

Bounded higher-order derivatives with respect to both x and t ⇒ the mapping t 7→ x∗ (t) is one-to-one and locally Lipschitz



12

Convergence of GTT

Theorem i) The GTT sequence {xk } converges to nbhd. of optimal x∗ (t) lim kxk − x∗ (tk )k = O(h)

k→∞

ii) If sampling rate h and step-size γ are sufficiently small, lim kxk − x∗ (tk )k = O(h2 )

k →∞

⇒ Convergence rate is Q-linear if step-size satisfies γ < 2/L. h i−1 ⇒ Sampling rate threshold: h < Cm0 C2 1 + Cm2 ρ−1 − 1 . ⇒ ρ = max{|1−γm|, |1−γL|} ⇒ step-size, cvx, Lipschitz constant



13

Convergence of NTT

Theorem With 0 < K depending on grad. and cvx. constants, if rate h satisfies h ≤ min {1, K } , and the algorithm is initialized such that x0 satisfies kx0 − x∗ (t0 )k ≤ ch2 Then NTT sequence {xk } converges to nbhd. of optimal x∗ (t) lim kxk − x∗ (tk )k = O(h4 )

k→∞



14

Time-Derivative Approximation

I

In many application domains, ∇tx f (xk ; tk ) is not known ⇒ can be approximated by a first-order backward derivative ˜ tx f (xk ; tk ) ≈ ∇x f (xk ; tk ) − ∇x f (xk −1 ; tk −1 ) ∇ h ⇒ situations in which exact motion of target unknown

I

Can establish comparable convergence guarentees to exact algs.



15

Approximate Gradient Tracking

Theorem GTT method with approx. time-derivative in prediction step yields i) If step-size satisfies γ < 2/L, {xk } → x∗ (tk ) Q-linearly rate to O(h) bounded error kxk − x∗ (tk )k ≤ ρk kx0 − x∗ (t0 )k+ O(h) + O(h2 )

ii) For small enough h and γ > 0, {xk } → x∗ (tk ) Q-linearly up to a bounded O(h2 ) error kxk − x∗ (tk )k ≤ (ρσ)k kx0 − x∗ (t0 )k + O(h2 )



16

Approximate Newton Tracking

Theorem Suppose the NTT sampling rate satisfies h ≤ min {1, K 0 } , where K 0 depends on grad. & cvx. constants, and is initialized so that kx0 − x∗ (t0 )k ≤ ch2 Then the alg. achieves an O(h4 ) asymptotic tracking error kxk − x∗ (tk )k ≤ O(h4 )



17

Target Tracking Example 0

Scalar numerical example f (x; t) :=

1 κ (x − cos(ωt))2 + sin2 (ωt) exp(µx 2 ) 2 2

I Predictor-corrector methods track x∗ (t)

⇒ magnitudes less in tracking error

Tracking error }xk ´ x˚ ptk q}

10

I

−5

10

RG GTT, τ “ 1 AGT, τ “ 1 GTT, τ “ 3 GTT, τ “ 5 NTT, τ “ 1

−10

10

Bounds Th. 1 and 3

−15

10

0

1

10

10

2

3

10

10

4

5

10

10

Sampling time tk

(a) Tracked sample planar trajectory I Worst-case error:

0

10

Ophq

max{kxk − x ∗ (tk )k} where k¯ = 104 . ⇒ performance improvement clearer

Oph2 q

−5

10

Worst-case error

k>k¯

−10

10

I Newton more accurate but higher cost

RG GTT, τ AGT, τ GTT, τ GTT, τ NTT, τ

Oph4 q −15

10

−2

10

−1

10

“1 “1 “3 “5 “1 0

10

Sampling time interval h

(b) Worst-case error Koppel, Simonetto, Mokhtari, Leus, Ribeiro


18

Target Tracking Example Target tracking example f (x; t) :=

1 kx − y(t)k2 + µ1 exp(µ2 kx − bk2 ) , 2

I Track a moving planar object y(t)

⇒ stay close to base station b I Predictor-corrector methods track optimal

RG AGT, τ “ 1 ANT, τ “ 1 Optimal trajectory Reference path

100

Vertical Position Coordinate y [m]

I

50

0

−50

−100 −100

−50

0

50

100

Horizontal Position Coordinate x [m]

(a) Tracked sample planar trajectory

I Worst-case error: I Fix comp. cost for given sampling rate

2

10

0

10 −1 10

Worst-case error [m]

maxk >k¯ {kxk − x ∗ (tk )k}, with k¯ = 104

−3

10

I Newton computationally heavier

⇒ higher control latency I Main takeaway: use NTT (ANT) if

sampling rate large enough

−5

10

RG AGT ANT

−7

10

−10

10

−14

10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sampling time interval h [s]

(b) Worst-case error Koppel, Simonetto, Mokhtari, Leus, Ribeiro


19

Concluding Remarks I

Focused on continuously-varying convex opt. problems

I

Brute force approach intractable ⇒ discrete-time sampling

I

Developed prediction-correction methods ⇒ Predict where optimal trajectory will be at next time ⇒ Once we observe info. at next sample time, make correction

I

Correction step ⇒ Gradient or Newton steps (GTT or NTT)

I

Theorem: GTT & NTT track x∗ (t) up to O(h2 ) or better

I

No problem if we don’t know exact objective time variation

I

Numerical examples illustrate utility in control domain

⇒ Correction reduces tracking error relative to existing methods

⇒ When sampling rate is large enough, better to use NTT Koppel, Simonetto, Mokhtari, Leus, Ribeiro


20

References

I

A. Simonetto, A. Koppel, A. Mokhtari, G. Leeus, and A. Ribeiro, “A Class of Prediction-Correction Methods for Time-Varying Convex Optimization,” IEEE Trans. Signal Process (submitted)., Sept. 2015.

I

A. Koppel, A. Simonetto, A. Mokhtari, G. Leus, and A. Ribeiro, “Target Tracking with Dynamic Convex Optimization”, IEEE Global Conference on Signal and Information Processing, Orlando, FL, Dec. 14-16, 2015.

I

A. Simonetto, A. Koppel, A. Mokhtari, G. Leeus, and A. Ribeiro “Prediction-Correction Methods for Time-Varying Convex Optimization.” in Proc. Asilomar Conf. on Signals Systems Computers, Pacific Grove, CA, November 8-11 2015.

http://seas.upenn.edu/˜akoppel/



21