Identification for control: can the optimal restricted complexity model

Identi cation for control: can the optimal restricted complexity model always be identi ed? F. De Bruyne, M. Gevers

CESAME, Universite Catholique de Louvain, B^atiment Euler, Avenue Georges Lema^tre 4-6, B1348 Louvain-La-Neuve, Belgium.

Abstract We consider model based certainty equivalence optimal control design for a high order plant on the basis of a model that is identi ed within a restricted complexity model set. Each model in this model set yields a \restricted complexity" controller. This de nes a controller set which contains a \best" controller for the high order plant, i.e. the controller that achieves the lowest cost on the high order plant. The following question is addressed: can the restricted complexity model (or models) that produces this \best" restricted complexity controller always be obtained by identi cation from plant data, given adequate experimental conditions? The answer is `not always'. Keywords: experiment design, input spectrum, open loop identi cation, closed loop identi cation, identi ability.

1 Problem formulation Let P be some \true" linear time invariant but unknown system, which is typically of high order, and consider that the task is to design a feedback controller for P using a certainty equivalence optimal control design on the basis of a model P^ of P identi ed from input-output data. It is assumed that after some initial analysis, the designer has selected a certain (typically low order) model set M:

M =4 fP^ ();  2 D  Rdg:

(1.1)

It is further assumed that the true plant is not contained in the (reduced order) model set, i.e. there exists no value of  for which P^ () = P at almost all frequencies. The situation described above is typical of complex process control applications, in which the model used for control design is typically much simpler than the plant and is necessarily data-based.  This

paper presents research results of the Belgian Programme on Interuniversity Poles of Attraction, initiated by the Belgian State, Prime Minister's Oce for Science, Technology and Culture. The scienti c responsibility rests with its authors.

It is assumed that the control design is based on the minimization of some control performance criterion, J (P; C ), that has been selected once and for all by the designer: an H1 design criterion, an LQG control criterion, etc. If the (high order) plant were known exactly, then the minimization over the class of all admissable full order controllers would result in the ideal optimal controller, ho Copt = arg min J (P; C ); (1.2) C to which corresponds an optimal cost, denoted1 ho ho Jopt = J (P; Copt ). This \optimal high order controller" depends on the true unknown plant P and is therefore not computable. The design of the controller is performed using the certainty equivalence principle on the basis of a nominal loop, in which the true plant is replaced by a model, P^ (), identi ed in the reduced order model set M. This allows us to de ne the \reduced order controller set"; it is the set of controllers that are optimal for all models in the reduced order model set: C = fC (P^ ()) : C (P^ ()) = ^ (); C ); 8 2 D g: (1.3) J (P arg min C When any one of the controllers C (P^ ()) 2 C is applied to the true system, this produces an \achieved cost" J (P; C (P^ ())). This cost can be in nite if the certainty equivalence controller C (P^ ()) does not stabilize the true plant P , but we shall assume that there are at least some controllers in the set C that produce a nite achieved cost. We can therefore de ne the best \low order controller" as the controller in the set C that minimizes the achieved cost on the real plant2: lo Copt = arg min J (P; C ): (1.4) C 2C lo The cost achieved by Copt on the real plant is denoted lo lo ho Jopt = J (P; Copt )  Jopt . The optimal low order con-

The superscript ho stands for high order. The set C might not be closed w.r.t. the operation minC 2C ( ), or there might be more than one optimum, but for the sake of simplicity we shall assume that there is a lo for which lo = argmin unique opt ). C 2C ( opt 1 2

J P; C

C

C

J P; C

troller is again not computable since it also depends on the unknown true plant P , but the point is that such optimal low order controller can be de ned. In addition, it is clear that the performance achieved by dierent controllers in C can be compared experimentally by applying these controllers to the real plant and computing the corresponding costs. lo The optimal low order controller Copt is the one that one would like to select through our procedure of identi cation in a reduced order model set M followed by model based controller design. A question that arises immediatelyis the following: is there an identi cation setup, i.e. experimental conditions (open loop identi cation, closed loop identi cation, etc) and choices of the design parameters (input spectra, data lters, etc), such that lo Copt = arg min J (P (^); C ) with ^ = arg min V (); C 2C  2D (1.5) where V () is some identi cation criterion. In other words, is it possible to identify, in the reduced order model set, the restricted complexity model lo which yields Copt as corresponding optimal controller. Stated even more brie y, is it possible to \identify", in the reduced order controller set, the optimal low lo order controller Copt , from data collected on P ? The answer, perhaps somewhat surprisingly, is \not always". This is shown by an example.

The outline of the paper is as follows. In Section 2, we show what is actually meant by identi cation setup and we describe the \identi cation design tools" that are available to cope with the previously de ned problem. In Section 3, we illustrate the problem with a Linear Quadratic Gaussian (LQG) control design for a third order (ARX) plant in which the control design is based on a rst order ARX model to be identi ed using plant input output data. We conclude in the last section.

2 What is meant by identi cation setup ?

We rst brie y recall the procedure and properties of prediction error identi cation. The true plant is assumed to be representable as follows, yt = P (z )ut + vt (2.1) where P (z ) is a scalar strictly proper rational transfer function, ut is the input, vt is an unmeasurable disturbance acting on the output yt . We assume that vt has a spectral density function v that can be factored as v = 2 jH (ej! )j2 with H (z ) monic, stable and inversely stable. We consider that the model set is described by ^ (z; )ut + H^ (z; )et : yt () = P (2.2)

Here P (z; ) and H (z; ) are proper rational transfer functions parametrized by some real vector  2 D , ^ (z; ) is monic, stable and inversely stable and et H is a zero mean white noise sequence. In addition, as stated in the introduction, we assume that there is no value of  2 D for which P^ (ej! ; ) = P (ej! ) and ^ (ej! ; ) = H (ej! ), 8!. From the model set (2.2) it H is easy to write the one step ahead prediction for yt [2]: y ^tjt?1() = H^ ?1(z; )P (z; )ut + [1 ? H^ ?1(z; )]yt : (2.3) From (2.1) and (2.3), we nd that the prediction error yt ? y ^tjt?1() is given by t() = yt ? y ^tjt?1() i h  = H^ ?1 (z; ) P (z ) ? P^ (z; ) ut + vt (: 2.4) In Least-Squares prediction error identi cation, the estimation of the parameter vector  on the basis of N input-output data is obtained by minimizing the sum of the squares of the prediction errors ft(); t = 1; : : : ; N g. In most cases, the prediction error sequence is ltered through some stable transfer function D(z ). We denote by ft () the ltered errors: f t () = D(z )t (): (2.5) Least-squares prediction error identi cation then amounts to estimate the optimal ^N that minimizes the identi cation criterion VN () N ^ = arg min VN () where VN () = 1 X[ft ()]2 :  2D N t=1 (2.6) Under reasonable conditions on the data and the model structure [2], ^N converges as N ?! 1 to

N



?

V () (2.7) = arg min 2D h i2 f where V () = N lim E VN () = E t () ; ?!1

this last equality being valid only if the data sequence is a realization of a stationary stochastic process. Using Parseval's relation Z  h i2 f f (!)d!; (2.8) E t () = 21 ? it is possible, by expressing the ltered prediction errors as a function of the \true system" and the model transfer functions, to obtain an expression for the frequency distribution of the asymptotic model error. This exercise makes it possible to express the variance of ft () in the frequency domain both for the case of open loop identi cation and closed loop identi cation.

If we assume that the data have been collected while the process was operating in open loop, in which case ut and es are uncorrelated for all t and s, then it follows from (2.4) that, ( )=

Z

V 



?

 P (z )



2 ? P^ (z; ) u + v



jD(z )j2 d!: jH^ (z; )j2

(2.9) This integral expression gives an implicit characterization of the model P^ (z; ? ) to which P^ (z; ^N ) converges if the number of data go to in nity: see (2.7). In other words, it gives an implicit characterization of the asymptotic bias error. Expression (2.9) is useful because it shows that, in the situation where some restricted complexity model set has been chosen for ^ (z; ) and H^ (z; ), one can still manipulate the freP quency distribution of the plant/model error to a certain extent by playing with the design variables D(z ) and u , namely the data lter and the input spectrum.

We now consider that the data have been collected on the true processs when some one degree of freedom controller, ut = ?C (z )yt + rt, was operating, and we derive a similar expression for the frequency distribution of the plant/model error in this case of closed loopf identi cation. First, we compute the expression of t (). Substituting the \closed loop" expression of ut in (2.4) yields h  D(z ) f ^ (z; ) rt P (z ) ? P t = ^ (z; ) (1 + C (z )P (z )) H   i + 1 + C (z )P^ (z; ) vt : (2.10) When identi cation is performed on closed loop data collected on the process operating under a one degree of freedom controller, the estimate ^N converges to the minimumof the following cost function (dropping the z dependence for simplicity of notation): () =

V 

Z



?

 P









2 2 ? P^ () r + 1 + C P^ () v 

jDj2 d!: (2.11) jH^()j2 j1 + CP j2

Expression (2.11) describes in an implicit way the asymptotic distribution of the error between the true system P (ej! ) and the estimated model P^ (ej! ; ? ) when the identi cation is performed on data collected in closed loop using a reduced order model set. The following remarks can be made [1].

 The model t is de nitely in uenced by the

controller: the controller C (z ) exerts its in uence through the noise driven second term and

through the sensitivity function 1+C (z)P (z) of the actual loop. The weighting on both terms of the integrand will be large where the sensitivity is large, namely around the crossover frequency of the closed loop system.

 External excitation is clearly needed for closed

loop identi cation. The model P^ (z; ) will approximate P (z ) only if the reference signal spectrum r dominates the noise signal spectrum v within the closed loop bandwith. Without external reference signal, the model will try to approximate the inverse of the controller, C ?1(z ).

 The data lter D(z ) can again be used to shape the t globally.

Thus, by identi cation setup, we mean the choice of the experimental conditions: one can either do the identi cation in open loop or in closed loop. Also, tools are available to orient the frequency t of the model to certain frequency ranges. By choosing a low pass input spectrum (u or r ), for instance, the t is enhanced at low frequencies. If a sinusoidal input signal is used (discrete input spectrum) which dominates the noise signal, the identi ed low order model is exact at the corresponding frequency. The data lter D(z ) has essentially the same role as the input spectrum, the dierence being that it weighs both terms of the integrand in the same way. In the simpli ed case study that follows, the in uence of the data lter is therefore not considered (D(z ) = 1).

3 Analysis for a simpli ed case In this section, we search a solution to the problem described in Section 1 in the case where the \true plant" is a third order continuous time ARX system, and where the controller is obtained by LQG design performed on a rst order continuous time ARX model to be identi ed from data obtained on this third order plant. We will consider the case of prediction error identi cation and examine dierent experimental conditions. We will also consider that an in nite amount of data can be used so that the identi ed models are characterized by the minimizing arguments of the integral expressions (2.9) and (2.11), depending on whether open or closed loop identi cation is used. The \true" third order plant is a stable continuous time ARX process, with relative degree two and positive gain ab00 , described by: y

= P (s)u + H (s)e

(3.1)

where

b1s + b0 + a 2 s2 + a 1 s + a 0 (3.2) 1 : H (s) = 3 s + a 2 s2 + a 1 s + a 0 and e is white noise of unit variance. We consider a rst order ARX model set3 described as follows: y ^() = P^ (s; )u + H^ (s; )e (3.3) where ^ (s; ) = b and H^ (s; ) = 1 : (3.4) P s+a s+a The control performance index is chosen to be the following LQG criterion: ) ( Z T 1 2 2 fy + u gdt : (3.5) J (P; C ) = lim E

() =

P s

T

?!1

s3

T

0

t

t

ho The optimal high order controller Copt de ned in Section 1 is the third order controller that results from the minimization of (3.5) over all admissable full order controllers.

The reduced order controller set corresponding (by minimization of (3.5)) to the reduced order model set of all rst order ARX models consists of all proportional output controllers, ut = ?k yt . The optimal lo low order controller Copt is then the solution of the following minimization problem: lo kopt = Copt = arg min J (P; k): (3.6) If we consider that the true system (3.1) is described by the following state-space equations,  x_ = F x + G u + G0 e (3.7) y = H x; it is easily seen that the control design problem (3.5)(3.6) can be solved by the minimization of the following criterion over all k: (

J P; k

) =

Z

1

?

tr (1 + k2 )H [sI ? (F ? GHk)]?1

?1
G0 G0T [sI ? (F ? GHk)]?? H T d!:

This minimization is equivalent to the following minimization problem: ? 
2 T kopt = arg min J (P; k) = tr (1 + k )HQH (3.8) 3 This is an academic example. The reason for taking rst order models is that they can be pictorially represented by their parameters ( ) in a two-dimensional parameter space: see later. a; b

under the constraint of the Lyapunov equation 0 = (F ? GHk)Q + Q(F ? GHk)T + G G T : (3.9) Inserting the parameters of the true plant (3.2), this yields:  kopt = arg min (1 + k2) 0



0



a2

2(a0 + b0k)(a1 a2 ? a0 ? (b0 ? a2b1 )k) (3.10) This last minimization problem is easily solved once numerical values are given to the transfer function parameters. Since we now have a procedure to calculate kopt , we can restate the question addressed in Section 1 as follows. Assuming that we perform an identi cation from data generated by the \plant" (3.1) using the model set (3.4), do there exist choices of the identi cation design parameters such that the identi ed rst order model has a corresponding optimal controller which is equal to kopt ? In other words, can the optimal reduced order controller be obtained by identi cation using an indirect scheme? In the following, we characterize all rst order models for which kopt is optimal by their parameters (a; b) in the parameter space. By imposing that kopt must be the stabilizing solution of the Riccati equation related to the low order control problem, we obtain a set of equations that de ne a half line L1 which characterizes, in the parameter space (a; b), all rst order models for which kopt is optimal [3]:

L1 :



(

2 b kopt a

? 1 ) + 2akopt = 0

(3.11) + bkopt > 0 (Stability of the nominal closed loop system).

We note that the half lines L1 depend on the design parameter . Several such half lines, for dierent values of , are represented in Figure 3.1 to which we return later. We now use a prediction error method to identify rst order ARX models from data obtained on the third order plant, using dierent input designs. We will consider open loop identi cation in which the input is generated with a rst order low pass input spectrum of increasing bandwidth; the bandwidth parameter c is used as a design parameter: (

2

) = c2 +c !2 :

u !; c

(3.12)

We will also consider closed loop identi cation with a regulator ut = ?kid yt + rt where kid is the control gain applied during identi cation and where r has

the same form as u above. Substituting this u (!; c) or r (!; c), as well as the noise spectrum v (!) corresponding to the \true" noise model H of (3.1) in (2.9) or (2.11), and minimizing with respect to a and b, we obtain, in the parameter space (a; b), a curve L2 of all ARX models that can be identi ed for all possible c. Our identi ability question can then be rephrased as follows:

 Is there an intersection between the half line L1

of all rst order models for which kopt is optimal, and the curve L2 of all models that are identi able ?  Alternatively, does there exist a bandwidth c (that is a spectrum  = c2 c+2!2 ) for which the identi ed model is a point on the curve L1 of all rst order models for which kopt is optimal ? We can make the following observations:

 When the bandwidth c of the low pass input

spectrum is very small, the identi ed model will t the static gain of the system. Thus, for c ! 0, the identi ed model will converge to a point on the line b = ab00 a both in the case of open loop and closed loop identi cation.  When the bandwidth c of the input spectrum increases, the gain of the identi ed model will increase since for large c one attempts to t a rst order model of slope -20dB/decade to a third order model with relative degree two of slope 40dB/decade.  When the bandwidth c of the input spectrum goes to in nity,  ?! 1, i.e. the input signal (ut or rt ) tends to white noise and the model tries to t the system over the whole frequency band. An analysis of the expressions (2.9) and (2.11) shows that a and b go to zero when c goes to in nity. In the case of open loop identi cation, the curve of all identi ed models is tangent to the b axis for very large c since the gain of the model becomes in nite. In the case of closed loop identi cation, the second term in (2.11) imposes that the curve of all identi ed models is tangent to b = ? k1id a, i.e. for very large c, the gain of the identi ed model is ? k1id where kid is the controller used in the identi cation step.

A typical curve L2 of open loop (top picture) and closed loop (bottom picture) identi ed models can be found in Figure 3.1. The models are characterized by there parameters (^a; ^b). The dash-dotted line in

both gures represents all systems (a; b) that have the same static gain as the true system. By the observation made above, the curve L2 necessarily lies above that line both in the case of open loop and closed loop identi cation. The curve of identi ed models lies to right of a = 0 in the open loop case and above the dashed line b = ? k1id a in the case of closed loop identi cation. In the open loop case, the curves L1 and L2 will therefore intersect if and only if the slope of the half line L1 is positive and larger than the system gain ab00 . In the case of closed loop identi cation, there is an extra degree of freedom: the curves L1 and L2 will intersect if and only if the slope of the half line L1 is larger than the system gain ab00 or smaller than ? k1id . Using (3.11), we arrive at the following conditions for the intersection of L1 and L2. Open loop identi ed models: b0 ?2kopt () < 1: (3.13) < a0 (kopt ())2 ? 1 Closed loop identi ed models: 8 ?2kopt() < 1 b0 > > < > > (kopt())2 ? 1 < a0 or (3.14) > > 1 ? 2 kopt() > > < ? : : ?1 < kid (kopt())2 ? 1 By computing kopt as a function of the system parameters and of , the slope of L1 can be rewritten as: 2a0b0 ? a1 a2b0 ? a0 a2 b1 : ?2kopt () = (kopt ())2 ? 1 a0 (a0 ? a1 a2) ? b0?1 (b0 ? a2 b1) (3.15) The two previous conditions imply that the existence of identi cation design conditions that allow one to obtain the optimal low order controller depends on the weighting  of the input signal in the quadratic control criterion (3.5) both in the case of open loop and closed loop identi cation. Note than in the case of closed loop identi cation there is an extra design parameter: the controller kid used during the identi cation phase. It is clear from conditions (3.13) and (3.14) that the range of 's for which it is possible to identify the optimal low order controller is smaller in the case of open loop identi cation compared with closed loop identi cation. The insight gained by this simple analysis allows us to construct an example in which the curves L1 and L2 do not intersect, whatever the input design that is used during identi cation. Consider the same example as before with the following numerical values: a0 = 3000; a1 = 650; a2 = 45; b0 = 3000; b1 = 3000; kid = 0:5:

For these numerical values, we have represented in Figure 3.1 the half lines L1 of (a; b) values yielding kopt for several values of . 40

lambda = 5.03

lambda = 10

35 c=10 30

25

lambda = 20

b

c=100 20

lambda = 2

4 Some concluding remarks

15

10

c=500 c=1.5

5 c=0.5 0 -6

-4

-2

0 a

2

4

6

40 lambda=5.03 35

lambda=10 lambda=20

30 c=10

b

25

20

c=100

15

lambda=2

10

5

c= 1 c=0.05

0 -6

the rst order model, then an unstable model cannot be obtained, whatever the choice of the input signal spectrum: thus, the curve L2 of identi ed output error models cannot be in the upper left hand quadrants of Figure 3.1. This is true both in the case of open loop and closed loop identi cation. Now, the half lines L1 of optimal (a; b) values for dierent values of  are the same for ARX models and for output error models. Thus, we conclude that if an output error model structure is used, then there are values of the weighting factor  in the LQG criterion for which the curves L1 and L2 will not intersect whatever the method used: open or closed loop identi cation.

-4

-2

0 a

2

4

6

Figure 3.1: Open and closed loop identi cation of a third order ARX plant by a rst order ARX model using a low pass input spectrum (resp. reference spectrum) of increasing bandwidth c. The top picture represents open loop identi cation, while the bottom picture represents closed loop identi cation with an operating controller ut = rt ? 0:5 yt. The pictures also portray the set of ARX models L2 that can be identi ed by applying an input signal (respectively a reference signal) with spectrum 2  = c2 c+!2 . It can easily be seen from the top picture that when  < 5:03, it is impossible to identify in open loop the model that produces the optimal low order controller. When the identi cation is done in closed loop (bottom picture), it is always possible (in this particular example) to identify a model that yields the optimal low order controller. It follows from the gure that for  < 5:03, the half line L1 de ned by (3.11) corresponds to negative values of a, that is unstable models. If an output error model structure is used for the identi cation of

It is very often the case in practice that a certainty equivalence control design is based on a low order identi ed model, even though the plant to which this controller is applied has a more complex structure. To a given choice of restricted complexity model structure, there corresponds (by certainty equivalence design) a restricted complexity controller set, and hence a best controller within this set for the actual system. The question addressed in this paper has been whether the model (or set of models) that yields this best reduced order controller can always be obtained by identi cation, given that the choice of model structure is xed but that the other identi cation design parameters are free. We have shown, using a suitably designed example and a zest of analysis, that this is not always the case. We have also shown that there are cases where this \best model for control" can be obtained by closed loop identi cation but not by open loop identi cation.

Acknowledgements We would like to thank J.W. Polderman and I.M.Y. Mareels for stimulating discussions on this problem. In particular, we thank J.W. Polderman for urging us to submit this modest contribution to a dicult problem.

References

[1] Gevers M. (1993). \Towards a joint design of identi cation and control ?", (Plenary talk at the 2nd European Control Conference, Groningen, Holland), Birkhauser, Boston, Basel, Berlin, pp. 111-151. [2] Ljung L. (1987). System Identi cation: Theory for the User. Prentice-Hall, Englewood Clis, New Jersey. [3] Polderman J. W. (1985). \A note on the structure of two subsets of the parameter space in adaptive control problems", Systems & Control letters, Vol. 7, pp. 25-34.