Model Predictive Control - FER

Model Predictive Control Manfred Morari Institut f¨ ur Automatik ETH Z¨ urich

Spring Semester 2014

Table of Contents 1. Linear Systems 1.1 Models of Dynamic Systems 1.2 Analysis of LTI Discrete-Time Systems 2. Linear Quadratic Optimal Control 2.1 Optimal Control 2.2 Batch Approach 2.3 Recursive Approach 2.4 Infinite Horizon Optimal Control 3. Uncertainty Modeling 3.1 Objective Statement, Stochastic Processes 3.2 Modeling using State Space Descriptions 3.3 Obtaining Models from First Principles 3.4 Obtaining Models from System Identification 4. State Estimation 4.1 Linear State Estimation 4.2 State Observer 4.3 Kalman Filter Model Predictive Control


i

1 1. Linear Systems



1 1. Linear Systems

i

1.1 Models of Dynamic Systems

Table of Contents

1. Linear Systems 1.1 Models of Dynamic Systems 1.2 Analysis of LTI Discrete-Time Systems

Model Predictive Control


Zusatzmaterial

1 1. Linear Systems


Models of Dynamic Systems

Goal: Introduce mathematical models to be used in Model Predictive Control (MPC) describing the behavior of dynamic systems Model classification: state space/transfer function, linear/nonlinear, time-varying/time-invariant, continuous-time/discrete-time, deterministic/stochastic If not stated differently, we use deterministic models Models of physical systems derived from first principles are mainly: nonlinear, time-invariant, continuous-time, state space models (*) Target models for standard MPC are mainly: linear, time-invariant, discrete-time, state space models (†) Focus of this section is on how to ’transform’ (*) to (†)



1 1. Linear Systems

1-2


Nonlinear, Time-Invariant, Continuous-Time, State Space Models (1/3) x˙ = g(x, u) y = h(x, u) x ∈ Rn u ∈ Rm y ∈ Rp

state vector input vector output vector

g(x, u) : Rn × Rm → Rn h(x, u) : Rn × Rm → Rp

system dynamics output function

Very general class of models Higher order ODEs can be easily brought to this form (next slide) Analysis and control synthesis generally hard → linearization to bring it to linear, time-invariant (LTI), continuous-time, state space form



1-3

1 1. Linear Systems


Nonlinear, Time-Invariant, Continuous-Time, State Space Models (2/3) Equivalence of one n-th order ODE and n 1-st order ODEs x (n) + g n (x, x, ˙ x¨, . . . , x (n−1) ) = 0 Define xi+1 = x (i) ,

i = 0, . . . , n − 1

Transformed system x˙ 1

= x2

x˙ 2 = x3 .. .. . . x˙ n−1 x˙ n Model Predictive Control

= xn = −g n (x1 , x2 , . . . , xn ) Spring Semester 2014

1-4

1 1. Linear Systems


Nonlinear, Time-Invariant, Continuous-Time, State Space Models (3/3) Example: Pendulum Moment of inertia wrt. rotational axis m l 2 Torque caused by external force Tc Torque caused by gravity m g l sin(θ)

System equation

Tc l u

m l 2 θ¨ = Tc − m g l sin(θ) mg

Using x1 , θ, x2 , θ˙ = x˙ 1 and u , Tc /m l the system can be brought to standard form x˙ 1 x2 x˙ = = = g(x, u) x˙ 2 − gl sin(x1 ) + u 2

Output equation depends on the measurement configuration, i.e. if θ is measured then y = h(x, u) = x1 . Model Predictive Control


1-5

1 1. Linear Systems


LTI Continuous-Time State Space Models (1/6)

x˙ = Ac x + B c u y = Cx + Du x ∈ Rn u ∈ Rm y ∈ Rp

state vector input vector output vector

Ac ∈ Rn×n B c ∈ Rn×m C ∈ Rp×n D ∈ Rp×m

system matrix input matrix output matrix throughput matrix

Vast theory exists for the analysis and control synthesis of linear systems Exact solution (next slide)



1-6

1 1. Linear Systems



Solution to linear ODEs Consider the ODE (written with explicit time dependence) x(t) ˙ = Ac x(t) + B c u(t) with initial condition x0 , x(t0 ), then its solution is given by Z x(t) = e

Ac (t−t0 )

t

eA

x0 +

c

(t−τ )

Bu(τ )dτ

t0

where e

Ac t

,

∞ P n=0


(Ac t)n n!


1-7

1 1. Linear Systems


LTI Continuous-Time State Space Models (3/6) Problem: Most physical systems are nonlinear but linear systems are much better understood Nonlinear systems can be well approximated by a linear system in a ’small’ neighborhood around a point in state space Idea: Control keeps the system around some operating point → replace nonlinear by a linearized system around operating point First order Taylor expansion of f (·) around x¯  ∂f ∂f  f (x) ≈ f (¯ x) + (x − x ¯ ), with = 0 0 ∂x x=¯x ∂x


∂f1 ∂x1

.. .

∂fn ∂x1

∂f1 ∂x2 ∂fn ∂x2

...

∂f1 ∂xn

.. .

...

∂fn ∂xn

  


1-8

1 1. Linear Systems


LTI Continuous-Time State Space Models (4/6) Linearization us keeps the system around stationary operating point xs → x˙ s = g(xs , us ) = 0, ys = h(xs , us )

x˙

⇒ x˙ − x˙s |{z}

∂g ∂g = g(xs , us ) + (x − xs ) + (u − us ) | {z } ∂x 0 x=xs | {z } ∂u 0 x=xs | {z } u=u u=u =0 | {z }s =∆x | {z }s =∆u =

=Ac c

c

=B c

∆x˙ = A ∆x + B ∆u

=0

y

∂h ∂h = h(xs , us ) + (x − xs ) + (u − us ) | {z } ∂x 0 x=xs ∂u 0 x=xs u=u u=u ys | {z }s | {z }s =C

⇒ ∆y |{z}

=D

= C ∆x + D∆u

y−ys



1-9

1 1. Linear Systems



Linearization The linearized system is written in terms of deviation variables ∆x, ∆u, ∆y Linearized system is only a good approximation for ’small’ ∆x, ∆u Subsequently, instead of ∆x, ∆u and ∆y, x, u and y are used for brevity



1-10

1 1. Linear Systems


LTI Continuous-Time State Space Models (6/6) Example: Linearization of pendulum equations x˙ =

x˙ 1 x˙ 2

=

x2 g − l sin(x1 ) + u

= g(x, u)

y = x1 = h(x, u) Want to keep the pendulum around xs = (π/4, 0)0 → us =

g l

sin(π/4)

∂g ∂g 0 1 0 Ac = = , Bc = g x=xs = 0 0 − 1 cos(π/4) 0 ∂x x=xs ∂u l u=us u=us ∂h ∂h C = = (1 0) , D = =0 ∂x 0 x=xs ∂u 0 x=xs u=us


u=us


1-11

1 1. Linear Systems


LTI Discrete-Time State Space Models (1/4) Linear discrete-time systems are described by linear difference equations

x(k + 1) = Ax(k) + Bu(k) y(k) = Cx(k) + Du(k) Inputs and outputs of a discrete-time system are defined only at discrete time points, i.e. its inputs and outputs are sequences defined for k ∈ Z+ Discrete time systems describe either 1

Inherently discrete systems, eg. bank savings account balance at the k-th month x(k + 1) = (1 + α)x(k) + u(k)

2

’Transformed’ continuous-time system



1 1. Linear Systems

1-12


LTI Discrete-Time State Space Models (2/4)

Vast majority of controlled systems not inherently discrete-time systems Controllers almost always implemented using microprocessors Finite computation time must be considered in the control system design → discretize the continuous-time system Discretization is the procedure of obtaining an ’equivalent’ discrete-time system from a continuous-time system The discrete-time model describes the state of the continuous-time system only at particular instances tk , k ∈ Z+ in time, where tk+1 = tk + Ts and Ts is called the sampling time Usually u(t) = u(tk ) ∀t ∈ [tk , tk+1 ) is assumed (and implemented)



1-13

1 1. Linear Systems


LTI Discrete-Time State Space Models (3/4) Discretization of LTI continuous-time state space models Rt c c Recall the solution of the ODE x(t) = e A (t−t0 ) x0 + t0 e A (t−τ ) Bu(τ )dτ Choose t0 = tk (hence x0 = x(t0 ) = x(tk )), t = tk+1 and use tk+1 − tk = Ts and u(t) = u(tk ) ∀t ∈ [tk , tk+1 ) x(tk+1 ) = e

A c Ts

Z

tk+1

eA

x(tk ) +

c

(tk+1 −τ )

B c dτ u(tk )

tk A c Ts

Z

= |e {z } x(tk ) + ,A

Ts

|0

eA

c

(Ts −τ 0 )

{z

B c dτ 0 u(tk ) }

,B

= Ax(tk ) + Bu(tk ) We found the exact discrete-time model predicting the state of the continuous-time system at time tk+1 given x(tk ), k ∈ Z+ under the assumption of a constant u(t) during a sampling interval B = (Ac )−1 (A − I )B c , if Ac invertible Model Predictive Control


1 1. Linear Systems

1-14


LTI Discrete-Time State Space Models (4/4) Example: Discretization of the linearized pendulum equations Using g/l = 10[s −2 ] the pendulum equations linearized about xs = (π/4, 0) are given by 0√ 1 0 x(t) ˙ = x(t) + u(t) 1 −10/ 2 0 Discretizing the continuous-time system using the definitions of A and B, and Ts = 0.1 s, we get the following discrete-time system 0.965 0.099 0.005 x(k + 1) = x(k) + u(k) −0.699 0.965 0.100



1-15

1 1. Linear Systems

1.2 Analysis of LTI Discrete-Time Systems

Table of Contents

1. Linear Systems 1.1 Models of Dynamic Systems 1.2 Analysis of LTI Discrete-Time Systems



1 1. Linear Systems

Zusatzmaterial


Analysis of LTI Discrete-Time Systems

Goal: Introduce the concepts of stability, controllability and observability From this point on we consider only discrete-time LTI systems for the rest of the lecture



1-16

1 1. Linear Systems


Coordinate Transformations (1/2)

Consider again the system

x(k + 1) = Ax(k) + Bu(k) y(k) = Cx(k) + Du(k) Input-output behavior, i.e. the sequence {y(k)}k=0,1,2... entirely defined by x(0) and {u(k)}k=0,1,2... Infinitely many choices of the state that yield the same input-output behavior Certain choices facilitate system analysis



1-17

1 1. Linear Systems


Coordinate Transformations (2/2) Consider the linear transformation x˜ = Tx with det(T ) 6= 0 (invertible) T −1 x˜(k + 1) = AT −1 x˜(k) + Bu(k) y(k) = CT −1 x˜(k) + Du(k) or −1 x˜(k + 1) = TAT TB u(k) | {z } x˜(k) + |{z} ˜ A −1

˜ B

y(k) = CT D u(k) | {z } x˜(k) + |{z} ˜ C

˜ D

Note: u(k) and y(k) are unchanged



1-18

1 1. Linear Systems


Stability of Linear Systems (1/3)

Theorem: Asymptotic Stability of Linear Systems The LTI system x(k + 1) = Ax(k) is globally asymptotically stable lim x(k) = 0, ∀x(0) ∈ Rn

k→∞

if and only if |λi | < 1, ∀i = 1, · · · , n where λi are the eigenvalues of A.



1 1. Linear Systems

1-19


Stability of Linear Systems (2/3) “Proof” of asymptotic stability condition Assume that A has n linearly independent eigenvectors e1 , · · · , en then the coordinate transformation x˜ = [e1 , · · · , en ]−1 x = Tx transforms an LTI discrete-time system to   λ1 0 ··· 0  ..   0 λ 0 .  2 −1  x˜(k) = Λ˜ x˜(k + 1) = TAT x˜(k) =  x (k)  ..  . .  . . 0  0 0 ··· 0 λn The state x˜(k) can be explicitly formulated as a function of x˜(0) = Tx(0)  k  λ1 0 ··· 0  ..  k  0 λ 0 .  2  x˜(0) x˜(k) = Λk x˜(0) =   ..  . .  . . 0  0 0 ··· 0 λkn Model Predictive Control


1-20

1 1. Linear Systems


Stability of Linear Systems (3/3) “Proof” of asymptotic stability condition

We thus have that x˜(k) = Λk x˜(0)

⇒ |˜ x (k)| = |Λk x˜(0)| (component-wise) ⇒ |˜ x (k)| = |Λk | · |˜ x (0)| k xi (0)| = |λi |k · |˜ xi (0)| ⇒ |˜ xi (k)| = |λi | · |˜

If any |λi | ≥ 1 then limk→∞ x˜(k) 6= 0 for x˜(0) 6= 0. On the other hand if |λi | < 1 ∀i ∈ 1, · · · , n then limk→∞ x˜(k) = 0 and we have asymptotic stability If the system does not have n linearly independent eigenvectors it can not be brought into diagonal form and Jordan matrices have to be used for the proof but the assertions still hold



1 1. Linear Systems

1-21


Stability of Nonlinear Systems (1/5) For nonlinear systems there are many definitions of stability. Informally, we define a system to be stable in the sense of Lyapunov, if it stays in any arbitrarily small neighborhood of the origin when it is disturbed slightly. In the following we always mean “stability” in the sense of Lyapunov. We consider first the stability of a nonlinear, time-invariant, discrete-time system xk+1 = g(xk ) (1) with an equilibrium point at 0, i.e. g(0) = 0. Note that system (1) encompasses any open- or closed-loop autonomous system. We will then derive simpler stability conditions for the specific case of LTI systems. Note that always stability is a property of an equilibrium point of a system.



1-22

1 1. Linear Systems


Stability of Nonlinear Systems (2/5) Definitions Formally, the equilibrium point x = 0 of a system (1) is stable if for every > 0 there exists a δ() such that ||x0 || < δ() → ||xk || < , ∀k ≥ 0 unstable otherwise.

A stable equilibrium point x = 0 of system (1) is asymptotically stable in Ω ⊆ Rn if lim xk = 0, ∀x0 ∈ Ω

k→∞

globally asymptotically stable if it is asymptotically stable and Ω = Rn Model Predictive Control


1 1. Linear Systems

1-23


Stability of Nonlinear Systems (3/5) Lyapunov functions We can show stability by constructing a Lyapunov function Idea: A mechanical system is asymptotically stable when the total mechanical energy is decreasing over time (friction losses). A Lyapunov function is a system theoretic generalization of energy

Definition: Lyapunov function Consider the equilibrium point x = 0 of system (1). Let Ω ⊂ Rn be a closed and bounded set containing the origin. A function V : Rn → R, continuous at the origin, finite for every x ∈ Ω, and such that V (0) = 0 and V (x) > 0, ∀x ∈ Ω \ {0} V (g(xk )) − V (xk ) ≤ −α(xk ) ∀xk ∈ Ω \ {0}

where α : Rn → R is continuous positive definite, is called a Lyapunov function. Model Predictive Control


1-24

1 1. Linear Systems


Stability of Nonlinear Systems (4/5) Lyapunov theorem

Theorem: Lyapunov stability (asymptotic stability) If a system (1) admits a Lyapunov function V (x), then x = 0 is asymptotically stable in Ω.

Theorem: Lyapunov stability (global asymptotic stability) If a system (1) admits a Lyapunov function V (x) that additionally satisfies kxk → ∞ ⇒ V (x) → ∞, then x = 0 is globally asymptotically stable.



1 1. Linear Systems

1-25


Stability of Nonlinear Systems (5/5) Remarks Note that the Lyapunov theorems only provide sufficient conditions Lyapunov theory is a powerful concept for proving stability of a control system, but for general nonlinear systems it is usually difficult to find a Lyapunov function Lyapunov functions can sometimes be derived from physical considerations One common approach: Decide on form of Lyapunov function (e.g., quadratic) Search for parameter values e.g. via optimization so that the required properties hold

For linear systems there exist constructive theoretical results on the existence of a quadratic Lyapunov function



1-26

1 1. Linear Systems


Global Lyapunov Stability of Linear Systems (1/3) Consider the linear system (2)

x(k + 1) = Ax(k)

Take V (x) = x 0 Px with P > 0 (positive definite) as a candidate Lyapunov function. It satisfies V (0) = 0, V (x) > 0 and kxk → ∞ ⇒ V (x) → ∞. Check ’energy decrease’ condition V (Ax(k)) − V (x(k)) = x 0 (k)A0 PAx(k) − x 0 (k)Px(k) = x 0 (k)(A0 PA − P)x(k) ≤ −α(x(k)) We can choose α(x(k)) = x 0 (k)Qx(k), Q > 0. Hence, the condition can be satisfied if a P > 0 can be found that solves the discrete-time Lyapunov equation A0 PA − P = −Q, Q > 0.


(3)


1 1. Linear Systems

1-27


Global Lyapunov Stability of Linear Systems (2/3)

Theorem: Existence of solution to the DT Lyapunov equation The discrete-time Lyapunov equation (3) has a unique solution P > 0 if and only if A has all eigenvalues inside the unit circle, i.e. if the system x(k + 1) = Ax(k) is stable. Therefore, for LTI systems global asymptotic Lyapunov stability is not only sufficient but also necessary, and it agrees with the notion of stability based on eigenvalue location. Note that stability is always “global” for linear systems.



1-28

1 1. Linear Systems


Global Lyapunov Stability of Linear Systems (3/3) Property of P The matrix P can also be used to determine the infinite horizon cost-to-go for an asymptotically stable autonomous system x(k + 1) = Ax(k) with a quadratic cost function determined by Q. More precisely, defining Ψ(x(0)) as Ψ(x(0)) =

∞ X k=0

0

x(k) Qx(k) =

∞ X

x(0)0 (Ak )0 QAk x(0)

(4)

k=0

we have that Ψ(x(0)) = x(0)0 Px(0).

(5)

“Proof” P∞ Define Hk , (Ak )0 QAk and P , k=0 Hk (limit of the sum exists because the system is assumed asymptotically stable). We have that A0 Hk A = (Ak+1 )0 QAk+1 = Hk+1 . P∞ P∞ P∞ Thus A0 PA = k=0 A0 Hk A = k=0 Hk+1 = k=1 Hk = P − H0 = P − Q. Model Predictive Control


1 1. Linear Systems

1-29


Controllability (1/3) Definition: A system x(k + 1) = Ax(k) + Bu(k) is controllable if for any pair of states x(0), x ∗ there exists a finite time N and a control sequence {u(0), · · · , u(N − 1)} such that x(N ) = x ∗ ,i.e.   u(N − 1)    u(N − 2)  ∗ N N −1 x = x(N ) = A x(0) + B AB · · · A B   ..   . u(0) It follows from the Cayley-Hamilton theorem that Ak can be expressed as linear combinations of Ai , i ∈ 0, 1, · · · , n for k ≥ n. Hence for all N ≥ n range B AB · · · AN −1 B = range B AB · · · An−1 B



1-30

1 1. Linear Systems


Controllability (2/3) If the system cannot be controlled to x ∗ in n steps, then it cannot in an arbitrary number of steps Define the controllability matrix C = B AB · · · An−1 B The system is controllable if  u(N − 1)  u(N − 2)  C ..  . u(0)

    = x ∗ − An x(0) 

has a solution for all right-hand sides (RHS) From linear algebra: solution exists for all RHS iff n columns of C are linearly independent ⇒ Necessary and and sufficient condition for controllability is rank(C) = n Model Predictive Control


1 1. Linear Systems

1-31


Controllability (3/3)

Remarks Another related concept is stabilizability A system is called stabilizable if it there exists an input sequence that returns the state to the origin asymptotically, starting from an arbitrary initial state A system is stabilizable iff all of its uncontrollable modes are stable Stabilizability can be checked using the following condition if rank ([λi I − A | B]) = n

∀λi ∈ Λ+ A ⇒ (A, B) is stabilizable

where Λ+ A is the set of all eigenvalues of A lying on or outside the unit circle. Controllability implies stabilizability



1-32

1 1. Linear Systems


Observability (1/3)

Consider the following system with zero input

x(k + 1) = Ax(k) y(k) = Cx(k) Definition: A system is said to be observable if there exists a finite N such that for every x(0) the measurements y(0), y(1), · · · y(N − 1) uniquely distinguish the initial state x(0)



1-33

1 1. Linear Systems


Observability (2/3) Question of uniqueness of the linear equations    C y(0)    CA y(1)     = .. ..    . . y(N − 1)

CAN −1

    x(0) 

As previously we can replace N by n wlog. (Cayley-Hamilton) 0 Define O = C 0 (CA)0 · · · (CAn−1 )0 From linear algebra: solution is unique iff the n columns of O are linearly independent ⇒ Necessary and sufficient condition for observability of system (A, C ) is rank(O) = n



1-34

1 1. Linear Systems


Observability (3/3) Remarks Another related concept is detectability A system is called detectable if it possible to construct from the measurement sequence a sequence of state estimates that converges to the true state asymptotically, starting from an arbitrary initial estimate A system is detectable iff all of its unobservable modes are stable Detectability can be checked using the following condition if rank ([A0 − λi I | C 0 ]) = n

∀λi ∈ Λ+ A ⇒ (A, C ) is detectable

where Λ+ A is the set of all eigenvalues of A lying on or outside the unit circle. Observability implies detectability



1-35

2 2. Linear Quadratic Optimal Control



xxxv


2.1 Optimal Control

Table of Contents

2. Linear Quadratic Optimal Control 2.1 Optimal Control 2.2 Batch Approach 2.3 Recursive Approach 2.4 Infinite Horizon Optimal Control



Zusatzmaterial


2.1 Optimal Control

Optimal Control Introduction (1/2) Discrete-time optimal control is concerned with choosing an optimal input sequence U0→N , [u00 , u10 , . . .]0 (as measured by some objective function), over a finite or infinite time horizon, in order to apply it to a system with a given initial state x(0). The objective, or cost, function is often defined as a sum of stage costs q(xk , uk ) and, when the horizon has finite length N , a terminal cost p(xN ): J0→N (x0 , U0→N ) , p(xN ) +

N −1 X

q(xk , uk )

k=0

The states {xk }N k=0 must satisfy the system dynamics xk+1 = g(xk , uk ), k = 0, . . . , N − 1 x0 = x(0) and there may be state and/or input constraints h(xk , uk ) ≤ 0, k = 0, . . . , N − 1. Model Predictive Control


2-36


2.1 Optimal Control

Optimal Control Introduction (2/2) In the finite horizon case, there may also be a constraint that the final state xN lies in a set Xf xN ∈ Xf A general finite horizon optimal control formulation for discrete-time systems is therefore ∗ J0→N (x(0)) , min J0→N (x(0), U0→N ) U0→N

subject to

xk+1 = g(xk , uk ), k = 0, . . . , N − 1 h(xk , uk ) ≤ 0, k = 0, . . . , N − 1 xN ∈ Xf x0 = x(0)



2-37


2.1 Optimal Control

Linear Quadratic Optimal Control In this section, only linear discrete-time time-invariant systems x(k + 1) = Ax(k) + Bu(k) and quadratic cost functions J0 (x0 , U0 ) ,

0 xN PxN

+

N −1 X

[xk0 Qxk + uk0 Ruk ]

(6)

k=0

are considered, and we consider only the problem of regulating the state to the origin, without state or input constraints. The two most common solution approaches will be described here 1

Batch Approach, which yields a series of numerical values for the input

2

Recursive Approach, which uses Dynamic Programming to compute control policies or laws, i.e. functions that describe how the control decisions depend on the system states.



2-38


2.1 Optimal Control

Unconstrained Finite Horizon Control Problem 0 0 Goal: Find a sequence of inputs U0 , [u00 , . . . , uN −1 ] that minimizes the objective function

J0∗ (x(0))

, min U0

subject to

0 xN PxN

+

N −1 X


k=0

xk+1 = Axk + Buk , k = 0, . . . , N − 1 x0 = x(0)

P 0, with P = P 0 , is the terminal weight Q 0, with Q = Q 0 , is the state weight R 0, with R = R0 , is the input weight N is the horizon length Note that x(0) is the current state, whereas x0 , . . . , xN and u0 , . . . , uN −1 are optimization variables that are constrained to obey the system dynamics and the initial condition. Model Predictive Control



2-39

2.2 Batch Approach

Table of Contents




Zusatzmaterial


2.2 Batch Approach

Solution approach 1: Batch Approach (1/4) The batch solution explicitly represents all future states xk in terms of initial condition x0 and inputs u0 , . . . , uN −1 . Starting with x0 = x(0), we have x1 = Ax(0) + Bu0 , and x2 = Ax1 + Bu1 = A2 x(0) + ABu0 + Bu1 , by substitution for x1 , and so on. Continuing up to xN we obtain:       u  I x0 0 0 ··· ··· 0    x1   A  u  1 B 0 ··· 0             ..   ..  .   .. B ··· 0    .  =  .  x(0) +  AB         .. . .    ..   ..  . . .  . . 0  .. .   .   .  N −1 A B · · · AB B uN −1 xN AN The equation above can be represented as X , S x x(0) + S u U0 .


(7)


2-40


2.2 Batch Approach

Solution approach 1: Batch Approach (2/4) Define Q , blockdiag(Q, . . . , Q, P) and R , blockdiag(R, . . . , R) Then the finite horizon cost function (6) can be written as J0 (x(0), U0 ) = X 0 QX + U0 0 RU0 .

(8)

Eliminating X by substituting from (7), equation (8) can be expressed as: J0 (x(0), U0 ) = (S x x(0) + S u U0 )0 Q(S x x(0) + S u U0 ) + U0 0 RU0 0

= U0 0 HU0 + 2x(0)0 FU0 + x(0)0 S x QS x x(0) 0

0

where H , S u QS u + R and F , S x QS u . 0

Note that H 0, since R 0 and S u QS u 0.



2-41


2.2 Batch Approach

Solution approach 1: Batch Approach (3/4) Since the problem is unconstrained and J0 (x(0), U0 ) is a positive definite quadratic function of U0 we can solve for the optimal input U0∗ by setting the gradient with respect to U0 to zero: ∇U0 J0 (x(0), U0 ) = 2HU0 + 2F 0 x(0) = 0 ⇒ U0∗ (x(0)) = −H −1 F 0 x(0) 0

0

= −(S u QS u + R)−1 S u QS x x(0) , which is a linear function of the initial state x(0). Note H −1 always exists, since H 0 and therefore has full rank. The optimal cost can be shown (by back-substitution) to be 0

J0∗ (x(0)) = −x(0)0 FHF 0 x(0) + x(0)0 S x QS x x(0) 0

0

0

0

= x(0)0 (S x QS x − S x QS u (S u QS u + R)−1 S u QS x )x(0) ,



2-42


2.2 Batch Approach

Solution approach 1: Batch Approach (4/4) Summary The Batch Approach expresses the cost function in terms of the initial state x(0) and input sequence U0 by eliminating the states xk . Because the cost J0 (x(0), U0 ) is a strictly convex quadratic function of U0 , its minimizer U0∗ is unique and can be found by setting ∇U0 J0 (x(0), U0 ) = 0. This gives the optimal input sequence U0∗ as a linear function of the intial state x(0): 0

0

U0∗ (x(0)) = −(S u QS u + R)−1 S u QS x x(0) The optimal cost is a quadratic function of the initial state x(0) 0

0

0

0

J0∗ (x(0)) = x(0)0 (S x QS x − S x QS u (S u QS u + R)−1 S u QS x )x(0) If there are state or input constraints, solving this problem by matrix inversion is not guaranteed to result in a feasible input sequence



2-43


2.3 Recursive Approach

Table of Contents




Zusatzmaterial



Solution approach 2: Recursive Approach (1/8) Alternatively, we can use dynamic programming to solve the same problem in a recursive manner. Define the “j-step optimal cost-to-go” as the optimal cost attainable for the step j problem: Jj∗ (x(j))

,

min

uj ,...,uN −1

subject to

0 xN PxN

+

N −1 X


k=j

xk+1 = Axk + Buk , k = j, . . . , N − 1 xj = x(j)

This is the minimum cost attainable for the remainder of the horizon after step j



2-44



Solution approach 2: Recursive Approach (2/8) Consider the 1-step problem (solved at time N − 1) 0 0 0 JN∗ −1 (xN −1 ) = min {xN −1 QxN −1 + uN −1 RuN −1 + xN PN xN } uN −1

subject to

(9) (10)

xN = AxN −1 + BuN −1 PN = P

where we introduced the notation Pj to express the optimal cost-to-go xj0 Pj xj . In particular, PN = P. Substituting (10) into (9) 0 0 JN∗ −1 (xN −1 ) = min {xN −1 (A PN A + Q)xN −1 uN −1

0 0 + uN −1 (B PN B + R)uN −1 0 0 + 2xN −1 A PN BuN −1 }




2-45


Solution approach 2: Recursive Approach (3/8) Solving again by setting the gradient to zero leads to the following optimality condition for uN −1 2(B 0 PN B + R)uN −1 + 2B 0 PN AxN −1 = 0 Optimal 1-step input: ∗ 0 −1 0 uN B PN AxN −1 −1 = −(B PN B + R)

, FN −1 xN −1 1-step cost-to-go: 0 JN∗ −1 (xN −1 ) = xN −1 PN −1 xN −1 ,

where PN −1 = A0 PN A + Q − A0 PN B(B 0 PN B + R)−1 B 0 PN A .



2-46



Solution approach 2: Recursive Approach (4/8) The recursive solution method used from here relies on Bellman’s Principle of Optimality For any solution for steps 0 to N to be optimal, any solution for steps j to N with j ≥ 0, taken from the 0 to N solution, must itself be optimal for the j-to-N problem Therefore we have, for any j = 0, . . . , N ∗ Jj∗ (xj ) = min{Jj+1 (xj+1 ) + xj0 Qxj + uj0 Ruj } uj

s.t.

xj+1 = Axj + Buj

Suppose that the fastest route from Los Angeles to Boston passes through Chicago. Then the principle of optimality formalizes the obvious fact that the Chicago to Boston portion of the route is also the fastest route for a trip that starts from Chicago and ends in Boston.



2-47



Solution approach 2: Recursive Approach (5/8) Now consider the 2-step problem, posed at time N − 2 ( N −1 ) X 0 JN∗ −2 (xN −2 ) = min xk0 Qxk + uk0 Ruk + xN PxN uN −1 ,uN −2

s.t.

k=N −2

xk+1 = Axk + Buk ,

k = N − 2, N − 1

From the Principle of Optimality, the cost function is equivalent to JN∗ −2 (xN −2 ) = min {JN∗ −1 (xN −1 ) uN −2

0 0 + xN −2 QxN −2 + uN −2 RuN −2 } 0 0 = min {xN −1 PN −1 xN −1 uN −2

0 + xN −2 QxN −2 + uN −2 RuN −2 }



2-48



Solution approach 2: Recursive Approach (6/8) As with 1-step solution, solve by setting the gradient with respect to uN −2 to zero Optimal 2-step input ∗ 0 −1 0 uN B PN −1 AxN −2 −2 = −(B PN −1 B + R)

, FN −2 xN −2 2-step cost-to-go 0 JN∗ −2 (xN −2 ) = xN −2 PN −2 xN −2 ,

where PN −2 = A0 PN −1 A + Q − A0 PN −1 B(B 0 PN −1 B + R)−1 B 0 PN −1 A We now recognize the recursion for Pj and uj∗ , j = N − 1, · · · , 0.




2-49


Solution approach 2: Recursive Approach (7/8) We can obtain the solution for any given time step k in the horizon u ∗ (k) = −(B 0 Pk+1 B + R)−1 B 0 Pk+1 Ax(k) , Fk x(k)

for k = 1, . . . , N

where we can find any Pk by recursive evaluation from PN = P, using Pk = A0 Pk+1 A + Q − A0 Pk+1 B(B 0 Pk+1 B + R)−1 B 0 Pk+1 A

(11)

This is called the Discrete Time Riccati Equation or Riccati Difference Equation (RDE). Evaluating down to P0 , we obtain the N -step cost-to-go J0∗ (x(0)) = x(0)0 P0 x(0)



2-50



Solution approach 2: Recursive Approach (8/8) Summary From the Principle of Optimality, the optimal control policy for any step j is then given by u ∗ (k) = −(B 0 Pk+1 B + R)−1 B 0 Pk+1 Ax(k) = Fk x(k) and the optimal cost-to-go is Jk∗ (x(k)) = xk0 Pk x(k) Each Pk is related to Pk+1 by the Riccati Difference Equation Pk = A0 Pk+1 A + Q − A0 Pk+1 B(B 0 Pk+1 B + R)−1 B 0 Pk+1 A , which can be initialized with PN = P, the given terminal weight




2-51


Comparison of Batch and Recursive Approaches (1/2) Fundamental difference: Batch optimization returns a sequence U0∗ (x(0)) of numeric values depending only on the initial state x(0), while dynamic programming yields feedback policies u ∗ (k) = Fk x(k), k = 0, . . . , N − 1 depending on each x(k). If the state evolves exactly as modelled, then the sequences of control actions obtained from the two approaches are identical. The recursive solution should be more robust to disturbances and model errors, because if the future states later deviate from their predicted values, the exact optimal input can still be computed. The Recursive Approach is computationally more attractive because it breaks the problem down into single-step problems. For large horizon length, the Hessian H in the Batch Approach, which must be inverted, becomes very large.



2-52



Comparison of Batch and Recursive Approaches (2/2)

Without any modification, both solution methods will break down when inequality constraints on xk or uk are added. The Batch Approach is far easier to adapt than the Recursive Approach when constraints are present: just perform a constrained minimization for the current state. Doing this at every time step within the time available, and then using only the first input from the resulting sequence, amounts to receding horizon control.




2-53

2.4 Infinite Horizon Optimal Control

Table of Contents




Zusatzmaterial



Infinite Horizon Control Problem: Optimal Sol’n (1/2) In some cases we may want to solve the same problem with an infinite horizon: (∞ ) X J∞ (x(0)) = min [xk0 Qxk + uk0 Ruk ] u(·)

subject to

k=0

xk+1 = Axk + Buk ,

k = 0, 1, 2, . . . , ∞ ,

x0 = x(0) As with the Dynamic Programming approach, the optimal input is of the form u ∗ (k) = −(B 0 P∞ B + R)−1 B 0 P∞ Ax(k) , F∞ x(k) and the infinite-horizon cost-to-go is J∞ (x(k)) = x(k)0 P∞ x(k) .




2-54


Infinite Horizon Control Problem: Optimal Sol’n (2/2) The matrix P∞ comes from an infinite recursion of the RDE, from a notional point infinitely far into the future. Assuming the RDE does converge to some constant matrix P∞ , it must satisfy the following (from (11), with Pk = Pk+1 = P∞ ) P∞ = A0 P∞ A + Q − A0 P∞ B(B 0 P∞ B + R)−1 B 0 P∞ A , which is called the Algebraic Riccati Equation (ARE). The constant feedback matrix F∞ is referred to as the asymptotic form of the Linear Quadratic Regulator (LQR). In fact, if (A, B) is stabilizable and (Q 1/2 , A) is detectable, then the RDE (initialized with Q at k = ∞ and solved for k & 0) converges to the unique positive definite solution P∞ of the ARE.



2-55



Stability of Infinite-Horizon LQR In addition, the closed-loop system with u(k) = F∞ x(k) is guaranteed to be asymptotically stable, under the stabilizability and detectability assumptions of the previous slide. The latter statement can be proven by substituting the control law u(k) = F∞ x(k) into x(k + 1) = Ax(k) + Bu(k), and then examining the properties of the system x(k + 1) = (A + BF∞ )x(k) .

(12)

The asymptotic stability of (12) can be proven by showing that the infinite ∗ horizon cost J∞ (x(k)) = x(k)0 P∞ x(k) is actually a Lyapunov function for ∗ ∗ the system, i.e. J∞ (x(k)) > 0, ∀k 6= 0, J∞ (0) = 0, and ∗ ∗ J∞ (x(k + 1)) < J∞ (x(k)), for any x(k). This implies that lim x(k) = 0 .

k→∞




2-56


Choices of Terminal Weight P in Finite Horizon Control (1/2)

1

The terminal cost P of the finite horizon problem can in fact trivially be chosen so that its solution matches the infinite horizon solution To do this, make P equal to the optimal cost from N to ∞ (i.e. the cost with the optimal controller choice). This can be computed from the ARE: P = A0 PA + Q − A0 PB(B 0 PB + R)−1 B 0 PA This approach rests on the assumption that no constraints will be active after the end of the horizon.



2-57



Choices of Terminal Weight P in Finite Horizon Control (2/2) 2

Choose P assuming no control action after the end of the horizon, so that x(k + 1) = Ax(k), k = N , . . . , ∞

This P can be determined from solving the Lyapunov equation APA0 + Q = P . This approach only makes sense if the system is asymptotically stable (or no positive definite solution P will exist). 3

Assume we want the state and input both to be zero after the end of the finite horizon. In this case no P but an extra constraint is needed xk+N = 0



2-58

3 3. Uncertainty Modeling



lviii


3.1 Objective Statement, Stochastic Processes

Table of Contents

3. Uncertainty Modeling 3.1 Objective Statement, Stochastic Processes 3.2 Modeling using State Space Descriptions 3.3 Obtaining Models from First Principles 3.4 Obtaining Models from System Identification




Zusatzmaterial


Objective Statement One of the main reasons for control is to suppress the effect of disturbances on key process outputs. A model is needed to predict the disturbances’ influence on the outputs on the basis of measured signals. For unmeasured disturbances, stochastic models are used.

Objective In this part we introduce stochastic models for disturbances and show how to integrate them into deterministic system models for estimation and control. We discuss how to construct models of the form: x(k + 1) = y(k) =

Ax(k) + Bu(k) + Fw(k) Cx(k) + Gw(k)

(13)

where w(k) is a disturbance signal.



3-59



Stochastic processes (1/2)

Stochastic processes are the mathematical tool used to model uncertain signals. A discrete-time stochastic process is a sequence of random variables {w(0), w(1), w(2), ...} The realization of the process is uncertain. We can model a stochastic process via its probability distribution In general, one must specify the joint probability distribution function (pdf) for the entire time sequence P(w(0), w(1), ...)




3-60


Stochastic processes (2/2)

Stochastic processes are modeled using data: Estimating the joint pdf usually is intractable Thus the normal distribution assumption is often made: only models of the mean and the covariance are needed Further distinction of stochastic processes: Stationary stochastic processes Nonstationary stochastic processes Informally, a stationary process with normal distribution has mean and variance that do not vary over a shifting time window.



3-61



Normal stochastic process Joint pdf is a normal distribution Completely defined by its mean and covariance function µw (k) , E{w(k)} Rw (k, τ ) , E{w(k + τ )w 0 (k)} − µw (k + τ )µw (k) Stationary if µw (k) = µw and Rw (k, τ ) = Rw (τ ) Typically data are used to estimate µw (k) and Rw (k, τ ) Special case: Normal white noise stochastic process ε(k) µε = 0 ( Rε Rε (k, τ ) = 0

if τ = 0 otherwise

Since jointly normally distributed and uncorrelated over time, ε(k) is independent of time Model Predictive Control



3-62

3.2 Modeling using State Space Descriptions

Table of Contents




Zusatzmaterial



Stationary Case (1/2) We model the stochastic process as the output w(k) of a linear system driven by normal white noise ε(k). It will turn out that w(k) is a normal stochastic process and µw (k), Rw (k, τ ) can be chosen through Aw ,Bw ,Cw . In the stationary case and using a state space description we have: xw (k + 1) = Aw xw (k) + Bw ε(k) w(k) = Cw xw (k) + ε(k)

(14)

where: xw is an additional state introduced to model the linear system’s response to white noise all the eigenvalues of Aw lie strictly inside the unit circle. (14) is the standard form for many filter and control design tools. We will show how to determine Aw ,Bw ,Cw in practical situations.




3-63


Stationary Case (2/2) The output w(k) of system (14) with white noise ε(k) and stable Aw has the following properties: E{xw (k)} E{xw (k)xw0 (k)} E{xw (k + τ )xw0 (k)}

= Aw E{xw (k − 1)} = Akw E{x(0)} = Aw E{xw (k − 1)xw0 (k − 1)}A0w + Bw Rε Bw0 = Aτw E{xw (k)xw0 (k)}

(15)

From this one can deduce that w ¯ = limk→∞ E{w(k)} Rw (τ ) = limk→∞ E{w(k + τ )w 0 (k)}

= 0 ¯ w C 0 + Cw Aτ −1 Bw Rε = Cw Aτw P w w (16)

where ¯ w = Aw P ¯ w A0 + Bw Rε B 0 , P w w

(17)

¯ w is a positive semi-definite solution to a Lyapunov equation. i.e. P These relations can be used in order to determine Aw , Bw , Cw matching a certain covariance Rw (τ ). Model Predictive Control


3-64



Nonstationary Case (1/2) If a disturbance signal has “persistent” characteristics (exhibiting shifts in the mean), it is not appropriate to model it with a stationary stochastic process. For example, controller design based on stationary stochastic processes will generally lead to offset. In this case one can superimpose the output of a linear system driven by integrated white noise εint (k) to the stationary signal: εint (k + 1) = εint (k) + ε(k)

(18)

The state space description is then: xw (k + 1) = Aw xw (k) + Bw εint (k) w(k) = Cw xw (k) + εint (k)


(19)



3-65


Nonstationary Case (2/2)

The state space description (19) can be rewritten, using differenced variables, as: ∆xw (k + 1) = Aw ∆xw (k) + Bw ε(k − 1) (20) ∆w(k) = Cw ∆xw (k) + ε(k − 1) ∆

where ∆w(k) = w(k) − w(k − 1) and ε(k) is a zero-mean stationary process. Since ε(k) is a white noise signal, (20) is equivalent to: ∆xw (k + 1) = ∆w(k) =


Aw ∆xw (k) + Bw ε(k) Cw ∆xw (k) + ε(k)


(21)

3-66


3.3 Obtaining Models from First Principles

Table of Contents





Zusatzmaterial


Obtaining Models from First Principles From first principles, after linearization, one obtains an ODE of the form: x˙ p y

= Acp xp + Bpc u + Fpc w = Cp x + Gp w

(22)

which can be discretized, leading to: xp (k + 1) = Ap x(k) + Bp u(k) + Fp w(k) y(k) = Cp xp (k) + Gp w(k)

(23)

Remarks: Subscript p is used here to distinguish the process model matrices from the disturbance model matrices introduced before. If the physical disturbance variables cannot be identified, one can express the overall effect of the disturbances as a signal directly added to the output → output disturbance, i.e. Gp = I and Fp = 0.



3-67



Stationary Case We can combine the model (14) with (23) to get:

xp (k + 1) xw (k + 1)

=

y(k) =

Ap Fp Cw xp (k) Bp Fp + u(k) + ε(k) 0 Aw xw (k) 0 Bw xp (k) Cp Gp Cw + Gp ε(k) xw (k)

(24)

With some appropriate re-definition of system matrices, the above is in the standard state-space form of: x(k + 1)

Ax(k) + Bu(k) + F ε(k) | {z }

=

ε1 (k)

y(k)

=

(25)

Cx(k) + Gε(k) | {z } ε2 (k)

Notice that the state is now expanded to include both the original system state xp and the disturbance state xw . Model Predictive Control



3-68


Nonstationary Case (1/2) We can combine (21) with a differenced version of (23) to obtain: ∆xp (k + 1) Ap Fp Cw ∆xp (k) Bp Fp = + ∆u(k) + ε(k) ∆xw (k + 1) 0 Aw ∆xw (k) 0 Bw | {z } | {z } | {z } A B F ∆xp (k) + Gp ε(k) ∆y(k) = Cp Gp Cw |{z} | {z } ∆xw (k) C

(26)

(27)

G

where ∆xp (k) , xp (k) − xp (k − 1), and ∆xw (k), ∆u(k) are defined similarly. For estimation and control, it is further desired that the model output be y rather than ∆y. This requires yet another augmentation of the state...



3-69



Nonstationary Case (2/2) The augmented system is: ∆x(k + 1) A 0 ∆x(k) B F = + ∆u(k) + ε(k) y(k + 1) C I y(k) 0 G ∆x(k) y(k) = 0 I y(k)

(28) (29)

It can be brought into the standard state-space form after re-definition of the system matrices: x¯(k + 1) = y(k) =

¯ x (k) + B∆u(k) ¯ ¯ ε(k) A¯ +F ¯ x¯(k) C

(30)

except that now the system input is ∆u rather than u. System (30) has ny integrators to express the effects of the white noise disturbances and the system input ∆u on the output.




3-70

3.4 Obtaining Models from System Identification

Table of Contents




Zusatzmaterial



Stationary Case Input output models obtained from an identification have the typical structure of: (31)

y(z) = H1 (z)u(z) + H2 (z)ε(z) where H1 (z) and H2 (z) are stable transfer matrices.

This can be brought into the form of (25) by finding a state-space realization of H1 (z) H2 (z) : x(k + 1) = A(k) + Bu(k) + F ε(k) y(k) = Cx(k) + Du(k) + Gε(k)

(32)

Remarks: H1 (z) has relative degree of at least one We may assume – without loss of generality – that H2 (0) = I , D = 0 and G = I.



3-71



Nonstationary Case In this case, the driving noise should be integrated white noise. y(z) = H1 (z)u(z) + H2 (z)

1 ε(z) 1 − z −1 | {z }

(33)

εint (z)

Using the fact (1 − z −1 )y(z) = (∆y)(z), we can rewrite the above as: (∆y)(z) = H1 (z)∆u(z) + H2 (z)ε(z) Denoting the realization of H1 (z) H2 (z) as x(k + 1) = ∆y(k) =

Ax(k) + B∆u(k) + F ε(k) Cx(k) + ε(k)

(34)

(35)

The state can be augmented with y as before to bring it into the form of (30).



3-72

4 4. State Estimation



lxxii


State Estimation

Motivation: In many control applications, information about the previous, current or future states is required but it is usually not possible to measure all state variables. Hence the previous/current/future states need to be estimated from past input and measurement sequences



4-73


General Estimation Problem (1/2)

Consider the following nonlinear time-varying discrete-time system subject to disturbances w1 (·) and w2 (·) x(k + 1) = g(x(k), u(k), w1 (k), k) y(k) = h(x(k), u(k), w2 (k), k) The goal of state estimation at timestep k is to estimate x(k + i) given {y(j), u(j)}j=0,1,··· ,k Estimating x(k + i) with i > 0 is called prediction while the case with i = 0 is referred to as filtering Some applications require the estimation of x(k + i) with i < 0 which is called smoothing



4-74


General Estimation Problem (2/2)

State estimation is an integral part of MPC, which is why good knowledge of it is crucial for designing an effective model predictive control Disturbance modeling is very important in estimation, simply adding white noise to the equations of a deterministic model can give very poor estimation results We will denote by xî|j an estimate of x(i) using all information available up to and at timestep j



4-75


State Estimation of Linear Systems From here on we focus on the estimation for linear systems Consider the following model with white process and measurement noise sequences {ε1 (i)}i=0,1,··· and {ε2 (i)}i=0,1,··· x(k + 1) y(k)

= Ax(k) + Bu(k) + ε1 (k) = Cx(k) + ε2 (k)

(36)

The (white) noise sequences are assumed to have zero mean, i.e. E{ε1 (k)} = 0 and E{ε2 (k)} = 0 ∀k ≥ 0 and to have covariance  ( 0 )  R1 R1,2 ε1 (i) ε1 (j) if i=j 0 R1,2 R2 E = ε2 (i) ε2 (j)  0 otherwise Additionally, it is assumed that the noise sequences are independent of the initial state estimate xˆ0|0




4-76

4.1 Linear State Estimation

Table of Contents

4. State Estimation 4.1 Linear State Estimation 4.2 State Observer 4.3 Kalman Filter



Zusatzmaterial



Linear Estimator Structure

We assume here that we want to design a linear estimator with the following structure The prediction step propagates the last estimate xˆk−1|k−1 using the nominal (assume ε1 = 0) model to generate the a priori estimate xˆk|k−1 xˆk|k−1 = Aˆ xk−1|k−1 + Bu(k − 1) The update step corrects the a priori estimate based on the prediction error y(k) − C xˆk|k−1 multiplied with the filter gain Kk xˆk|k = xˆk|k−1 + Kk (y(k) − C xˆk|k−1 )




4-77


Estimation Error (1/3)

e Define the state estimation error xi|j as the difference between the true value x(i) and the estimate xî|j , i.e. e , x(i) − xî|j xi|j

We focus on estimating x(k) using information up to time k: {y(i)}i=0,1,··· ,k , {u(j)}j=0,1,··· ,k , i.e. on calculating xˆk|k e We want to choose the filter gain Kk in order to minimize xk|k in some meaningful way



4-78



Estimation Error (2/3) Recall the model and the postulated filter equations x(k) = Ax(k − 1) + Bu(k − 1) + ε1 (k − 1)

(37)

y(k) = Cx(k) + ε2 (k)

(38)

xˆk|k−1 = Aˆ xk−1|k−1 + Bu(k − 1)

(39)

xˆk|k = xˆk|k−1 + Kk (y(k) − C xˆk|k−1 )

(40)

Substituting y(k) in (40) using (38) xˆk|k = xˆk|k−1 + Kk (C [x(k) − xˆk|k−1 ] + ε2 (k)) In a next step we obtain e xk|k = x(k) − xˆk|k

= x(k) − xˆk|k−1 − Kk (C [x(k) − xˆk|k−1 ] + ε2 (k)) e = (I − Kk C )xk|k−1 − Kk ε2 (k) Model Predictive Control

(41)



4-79


Estimation Error (3/3)

Subtracting (39) from (37) we get e e xk|k−1 = Axk−1|k−1 + ε1 (k − 1)

(42)

e Substituting finally xk|k−1 from (42) in (41) yields the dynamics of the e estimation error xk|k which we will analyze next e e xk|k = (A − Kk CA)xk−1|k−1 + (I − Kk C )ε1 (k − 1) − Kk ε2 (k)



(43)

4-80


4.2 State Observer

Table of Contents





Zusatzmaterial

4.2 State Observer

State Observer (1/2)

The theory of state observers is based on the behavior of the estimation error in a deterministic setting (ε1 ≡ 0, ε2 ≡ 0) with a fixed Kk = K The error dynamics in the deterministic setting with fixed K are obtained by setting ε1 and ε2 to zero and fixing Kk in (43) e e xk|k = (A − KCA)xk−1|k−1

(44)

e An estimator for system (36) is said to be observer-stable if xk|k → 0 as e k → ∞ for any x0|0 when ε1 ≡ 0 and ε2 ≡ 0

From (44) it is clear that for observer stability the eigenvalues of A − KCA must lie strictly inside the unit circle



4-81


4.2 State Observer

State Observer (2/2) Iff (CA, A) is observable, the eigenvalues of A − KCA can be placed arbitrarily by choosing an appropriate K If A has full rank then observability of (CA, A) is equivalent to observability of (C , A) A slightly different (ˆ xk+1|k = Aˆ xk|k−1 + K (y(k) − C xˆk|k−1 ) + Bu(k)) observer structure yields an error dynamics matrix A − KC where observability of (C , A) is necessary and sufficient to arbitrarily place the eigenvalues through K Determining K such that the eigenvalues of A − KCA are at desired locations is called observer pole placement Observer pole placement is not frequently used since 1 2

In general, pole placement does not determine the observer gain K uniquely and it is unclear how to best use the remaining degrees of freedom Although the eigenvalues can be freely chosen, the eigenvectors cannot and the observer may recover only very slowly from certain types of errors




4-82

4.3 Kalman Filter

Table of Contents




Zusatzmaterial


4.3 Kalman Filter

Kalman Filter Derivation (1/9) It is intuitively clear that incorporating statistical information on how the disturbances affect the state can improve the quality of the estimation The Kalman filter accounts for this information in an optimal (to be defined) way e Denote the covariance of the estimation error xi|j by e Pi|j ,Cov{xi|j }

The optimality criterion of the Kalman filter is the error covariance Pk|k , i.e. we choose Kk such that α0 Pk|k α is minimized for any α of appropriate dimension For simplicity it is assumed that, i.e. E{ε1 (i)ε2 (i)0 } = R1,2 = 0 and R2 0.



4-83


4.3 Kalman Filter

Kalman Filter Derivation (2/9) e Unbiasedness of xk|k

Recall the error update equations e e xk|k = (A − Kk CA)xk−1|k−1 + (I − Kk C )ε1 (k − 1) − Kk ε2 (k)

e xk|k can be rewritten in explicit form

" e xk|k =

#

k Y

e (A − Ki CA) x0|0

i=1

+

k X

"

i=1

−

k X i=1


k Y

#

!

(A − Kj CA) (I − Ki C )ε1 (i − 1)

j=i+1

"

k Y

# (A − Kj CA) Ki ε2 (i)

! (45)

j=i+1


4-84


4.3 Kalman Filter

Kalman Filter Derivation (3/9) e Unbiasedness of xk|k

Since by assumption E{ε1 (i)} = 0 and E{ε2 (i)} = 0, we have from the previous equation " k # Y e e E{xk|k }= (A − Ki CA) E{x0|0 } (46) i=1

If Kk → K∞ as k → ∞ and if the eigenvalues of (A − K∞ CA) are stable, the filter is asymptotically unbiased, i.e. e lim E{xk|k }=0

k→∞


(47)



4-85

4.3 Kalman Filter

Kalman Filter Derivation (4/9)

In the following we will use the fact that e xk−1|k−1 and ε1 (k − 1) are uncorrelated e This is obvious since ε1 (k − 1) only influences x(k), which itself is not part of xk−1|k−1

e xk|k−1 and ε2 (k) are uncorrelated e This is obvious since ε2 (k) influences y(k), which itself is not part of xk|k−1

0 Pi|j = Pi|j and Pi|j 0

Any covariance matrix is symmetric positive-semidefinite by definition

Sk , CPk|k−1 C 0 + R2 0 This follows from Pk|k−1 0 and R2 0



4-86


4.3 Kalman Filter

Kalman Filter Derivation (5/9) In the next slides, we will compute the expressions for Pk|k−1 and Pk|k Recall the estimation error equations e e + ε1 (k − 1) = Axk−1|k−1 xk|k−1 e e − Kk ε2 (k) = (I − Kk C )xk|k−1 xk|k e Using that xk−1|k−1 and ε1 (k − 1) are uncorrelated we can compute Pk|k−1 as

e Pk|k−1 =Cov{Axk−1|k−1 + ε1 (k − 1)} e =Cov{Axk−1|k−1 } + Cov{ε1 (k − 1)} e =ACov{xk−1|k−1 }A0 + R1

=APk−1|k−1 A0 + R1




(48)

4-87

4.3 Kalman Filter


e Similarly for Pk|k using that xk|k−1 and ε2 (k) are uncorrelated we get e Pk|k =Cov{(I − Kk C )xk|k−1 − Kk ε2 (k)} e =Cov{(I − Kk C )xk|k−1 } + Cov{−Kk ε2 (k)} e =(I − Kk C )Cov{xk|k−1 }(I − Kk C )0

+ Kk Cov{ε2 (k)}Kk0 =(I − Kk C )Pk|k−1 (I − Kk C )0 + Kk R2 Kk0



4-88


4.3 Kalman Filter

Kalman Filter Derivation (7/9) Rewrite Pk|k Pk|k =(I − Kk C )Pk|k−1 (I − Kk C )0 + Kk R2 Kk0 =Kk (CPk|k−1 C 0 + R2 ) Kk0

{z

|

}

,Sk

− Kk CPk|k−1 − (Kk CPk|k−1 )0 + Pk|k−1 =(Kk − Pk|k−1 C 0 Sk−1 )Sk (Kk − Pk|k−1 C 0 Sk−1 )0 + Pk|k−1 − Pk|k−1 C 0 Sk−1 CPk|k−1

(49)

Since Sk 0 it always can be inverted and α0 Pk|k α is minimized for any α if we choose Kk = Pk|k−1 C 0 Sk−1 . (49) can be rewritten as Pk|k = (I − Kk C )Pk|k−1




4-89

4.3 Kalman Filter

Kalman Filter Derivation (8/9) Kalman Filter Algorithm Initialize xˆ0|0 and P0|0 The filter gain Kk and the error covariance matrix Pk|k can be computed online or in advance Pk|k−1 = APk−1|k−1 A0 + R1 Kk = Pk|k−1 C 0 (CPk|k−1 C 0 + R2 )−1 Pk|k = (I − Kk C )Pk|k−1 At time k 1

Compute the a priori estimate xˆk|k−1 = Aˆ xk−1|k−1 + Bu(k − 1)

2 3

(50)

Get measurement y(k) Compute new estimate xˆk|k = xˆk|k−1 + Kk (y(k) − C xˆk|k−1 )

(51)

k →k +1 Model Predictive Control


4-90


4.3 Kalman Filter


Remarks The derivation of the Kalman filter with R1,2 6= 0 follows the same lines but requires a bit more algebra Generalization of the Kalman filter equations for a system where A, B, C are time-varying is straightforward. It is possible to replace A by A(k), B by B(k) and C by C (k) in all equations (except the steady-state equations in the next slides) without changing the arguments used for the derivation




4-91

4.3 Kalman Filter

Kalman Filter Analysis (1/3)

Stability of the Kalman filter If Pk|k−1 converges to a steady-state solution P∞ and the corresponding steady-state Kalman gain K∞ satisfies the condition that A − K∞ CA has all eigenvalues strictly inside the unit circle, then the expected estimation error goes asymptotically to zero, see (47). Together with the bounded covariance, it is said that the Kalman filter is stable In the following slides, we investigate under which conditions the Kalman gain converges to K∞ and the asymptotic filter gain is stabilizing, i.e. the eigenvalues of A − K∞ CA lie inside the unit circle 0 Note that if Pk|k−1 → P∞ as k → ∞ then also Pk|k → P∞



4-92


4.3 Kalman Filter

Kalman Filter Analysis (2/3) Substituting the Kalman gain into (49) yields Pk|k = Pk|k−1 − Pk|k−1 C 0 Sk−1 CPk|k−1

(52)

Substitute Pk−1|k−1 using the above in (48) we can rewrite Pk|k−1 as Pk|k−1 =APk−1|k−1 A0 + R1 −1 CPk−1|k−2 A0 + R1 =APk−1|k−2 A0 − APk−1|k−2 C 0 Sk−1

From the last equation together with Sk−1 = CPk−1|k−2 C 0 + R2 it can be seen that if Pk|k−1 converges it must satisfy the Algebraic Riccati Equation P∞ =AP∞ A0 − AP∞ C 0 (CP∞ C 0 + R2 )−1 CP∞ A0 + R1




4-93

4.3 Kalman Filter

Kalman Filter Analysis (3/3)

P∞ is called a stabilizing solution if all eigenvalues of A − K∞ CA are strictly inside the unit circle We have the following properties: If (C , A) is detectable, then Pk|k−1 converges regardless of P0|0 1/2 If (C , A) is detectable, (A, R1 ) is stabilizable and P0|0 0, then Pk|k−1 converges to P∞ , where P∞ is a stabilizing solution to the ARE



4-94