A Framework for Closed-loop Subspace Identification

TWMCC ? Texas-Wisconsin Modeling and Control Consortium

2

1

Technical report number 2004-07

A Framework for Closed-loop Subspace Identification with Innovation Estimation Weilu Lina , S. Joe Qina∗, Lennart Ljungb a

Department of Chemical Engineering The University of Texas at Austin Austin, TX 78712, USA

b

Department of Electrical Engineering Linköping University SE-581 83 Linköping, Sweden Feb 09, 2004

Abstract Subspace identification methods (SIMs) for estimating state-space models have been proven to be very useful and efficient. They have been successfully applied to many industrial cases but have one main drawback - most of them cannot provide consistent estimates for closed-loop data, even though the data satisfy identifiability condition for traditional methods such as prediction error methods (PEMs). In this paper, we show that the closed-loop consistency with SIMs can be achieved through innovation estimation. Based on this analysis, a new SIM with parsimonious formulation is proposed to handle data collected under feedback. Order determination through Akaike information criterion (AIC) and a consistent estimate of the Kalman gain under closed-loop conditions are also provided. Simulation studies are provided to show that it is consistent under closed-loop condition, when the traditional SIMs fail to provide consistent estimates. Keywords: Subspace identification, Innovation estimation, Closed-loop identification, Consistency analysis, Akaike information criterion ∗

Corresponding author. E-mail: [email protected]. Tel.: +1-512-471-4417. Fax: +1-512-471-7060

1

TWMCC Technical Report 2004-07

1

2

Introduction

The closed-loop identification is of special interest for a large number of engineering applications. For safety reasons or quality restrictions, it is desirable that identification experiments are carried out under the closed-loop or partial closed-loop condition. As pointed out by many researchers (e.g., Söderstr¨ om and Soica, 1989; Ljung, 1999), the fundamental problem with closed-loop data is the correlation between the unmeasurable noise and the input. Traditional closed-loop identification approaches fall into the prediction error methods (PEMs) framework. A comprehensive study in this area is provided by Forssell and Ljung (1999). Based on their analysis, the closed-loop identification methods can be categorized into three main groups: the direct approach, the indirect approach, and the joint input-output approach. The advantage of PEMs is that the convergence and asymptotic variance results are available (Ljung, 1978; Ljung, 1985), which are important for ”identification for control” applications (Hjalmarsson, 2003). The disadvantage of PEMs is that they involve in a complicated parametrization step, which makes them difficult to apply to multi-input-multi-output (MIMO) identification problems. The motivation of circumventing the complicated parametrization of PEMs, especially for the MIMO identification, gave birth to subspace identification methods (SIMs). Most SIMs fall into the unifying theorem proposed by Van Overschee and De Moor (1995), among which are canonical variate analysis (CVA) (Larimore, 1990), N4SID (Overschee and Moor, 1994), subspace fitting (Jansson and Wahlberg, 1996), and MOESP (Verhaegen and Dewilde, 1992). Based on the unifying theorem, all these algorithms can be interpreted as a singular value decomposition of a weighted matrix. The statistical properties such as consistency and efficiency have been investigated recently (Jansson and Wahlberg, 1998; Knudsen, 2001; Bauer and Ljung, 2002; Gustafsson, 2002; Bauer, 2003). Although SIM algorithms are attractive because of the state space form that is very convenient for estimation, filtering, prediction, and control, several drawbacks have been experienced. In general, the estimates from SIMs are not as accurate as those from prediction error methods (PEMs). Further, very few SIMs are applicable to closed-loop identification, even though the data satisfy identifiability conditions for traditional methods such as PEMs. In a companion paper (Qin et al., 2003), we give the reasons why subspace identification approaches exhibit the first drawback and propose parsimonious SIMs (PARSIMs) for open-loop applications. Contrary to the open loop SIMs, the traditional SIMs (e.g., CVA, N4SID and MOESP) are biased under closed-loop condition. Verhaegen (1993) proposed a closed-loop SIM via the identification of an overall open-loop state space model followed by a model reduction step to obtain state space representations of the plant and controller. The disadvantages of the approach is that a high order overall system has to be identified, which introduces extra computational burden. Ljung and McKelvey (1996) investigated the SIM through the classical realization path and proposed a recursive approach based on ARX model as a feasible closed-loop SIM. The drawback of the approach is that the ARX parametrization is not applicable for the generic system. Recently, Chiuso and Picci (2003) analyzed SIMs with feedback through stochastic realization theory and provided a theoretical analysis to construct the geometric state based on an oblique predictor space. Nevertheless, they did


3

not provide any algorithm in detail. Formulated in an errors-in-variables (EIV) framework, Chou and Verhaegen (1997) proposed a new SIM that can be applied to closed-loop data. The algorithm is nevertheless very complex which has to treat the case of white input from non-white input differently. Wang and Qin (2002, 2003) proposed the use of parity space and principal component analysis (PCA) for EIV and closed-loop identification which is applicable to correlated input excitation. Huang, Ding and Qin (2003) analyzed the reason why these methods cannot be applied to white input directly and proposed a revised instrumental variables approach. To the best of our knowledge, the possibility of closed-loop identification with SIMs has not been thoroughly analyzed. The main purpose of this paper is to reveal the feasibility of the consistent estimation with SIMs under the closed-loop operation. It is shown that the consistency of closed-loop SIMs can be achieved through innovation estimation as a by-product of a sub-task in the PARSIM formulation (Qin et al., 2003). The rest of the paper is organized as follows. In Section 2, we analyze feasibility of closed-loop SIMs through innovation estimation. The consistency of closed-loop SIMs is also presented in this section. Based on this analysis, a feasible closed-loop SIM is presented in detail in Section 3. In Section 4, numerical simulations are given to show the efficiency of the proposed algorithm. Section 5 concludes the paper.

2 2.1

Analysis of subspace identification under closed-loop condition Problem formulation and assumptions

In the paper, we assume that the system to be identified can be written in innovations form as xk+1 = Axk + Buk + Kek

(1a)

yk = Cxk + Duk + ek

(1b)

where yk ∈ Rny , xk ∈ Rn , uk ∈ Rnu , and ek ∈ Rny are the system output, state, input, and innovation, respectively. A, B, C and D are system matrices with appropriate dimensions. K is the Kalman filter gain. The system described by Eq. 1 can also be presented as yk = G(q)uk + H(q)ek

(2a)

where G(q) = C(qI − A)−1 B + D, and H(q) = C(qI − A)−1 K + I. We shall assume that the input is determined through feedback as uk = −F (q)yk + rk

(3)

where rk is the reference signal, and F (q) is the filter standing for the feedback mechanism. To establish the statistical consistency of the SIM under closed-loop condition, we introduce following assumptions:


4

A1 : The eigenvalues of A − KC are strictly inside the unit circle. A2 : The system is minimal in the sense that (C, A) is observable and (A, [B, K]) is controllable. A3 : The innovation sequences ek is a stationary, zero mean, white noise process with the second order moments E(ei eTj ) = Rδij where δij is the Kronecker delta. A4 : The input u(k) and innovation sequence e(j) are uncorrelated for ∀j ≥ k, which implies that either the system or the controller contains a delay. A5 : The reference signal and innovation sequence are uncorrelated to each other, and the reference signal is persistently exciting of a sufficient order. A6 : The closed-loop subsystem from r and e to y are asymptotically stable. The closed-loop identification problem is: given a set of input/output measurements and reference measurements, estimate the system matrices (A, B, C, D), Kalman filter gain K up to within a similarity transformation, and the innovation covariance matrix R. The exact knowledge of the controller is not required for the closed-loop identification approach proposed in this work. Based on state space description in Eq. 1, an extended state space model can be formulated as Yf

= Γf Xk + Hf Uf + Gf Ef

(4a)

Yp = Γp Xk−p + Hp Up + Gp Ep

(4b)

where the subscripts f and p denote future and past horizons, respectively. The extended observability matrix is   C  CA    Γf =  (5)  ..   . CAf −1 and Hf and Gf are Toeplitz matrices:  Hf

  =   

Gf

  =  

D CB .. .

··· ··· .. .

0 0 .. .

CAf −2 B CAf −3 B · · ·

D

I CK .. .

0 D .. . 0 I .. .

··· ··· .. .

0 0 .. .

CAf −2 K CAf −3 K · · ·

I

    

(6a)

    

(6b)


5

The input and output data are arranged in the following Hankel form:   uk uk+1 · · · uk+N −1  uk+1  uk+2 · · · uk+N   Uf =   .. .. .. ..   . . . . uk+f −1 uk+f · · · uk+f +N −2   uk−p uk−p+1 · · · uk−p+N −1  uk−p+1 uk−p+2 · · · uk−p+N    Up =   .. .. .. ..   . . . . uk−1

uk

···

(7a)

(7b)

uk+N −2

The state sequences are defined as: Xk = [xk , xk+1 , · · · , xk+N −1 ]

(8a)

Xk−p = [xk−p , xk−p+1 , · · · , xk−p+N −1 ]

(8b)

Similar formulations are made for Yf , Yp , Ef , and Ep . Subspace identification consists of estimating the extended observability matrix first and then the model parameters.

2.2

Analysis of the closed-loop SIM

The main purpose of the subsection is to explore the feasibility of closed-loop SIMs with innovation estimation. We can partition the extended state space model in Eq. 4 row-wise as follows:     Yf 1 Yf 1  Yf 2    ∆  Yf 2    Yf =  .  ; Yi =  .  ; i = 1, 2, . . . , f (9)  ..   ..  Yf f

Yf i

Partition Uf and Ef in a similar way to i = 1, 2, . . . , f . Denote further  Γf 1  Γf 2  Γf =  .  .. ∆

£

∆

£

∆

£

∆

£

Hf i = = Gf i = =

define Uf i , Ui , Ef i , and Ei , respectively, for     

(10a)

Γf f CAi−2 B · · · Hi−1 · · ·

CB D ¤ H1 H0

CAi−2 K · · · Gi−1 · · ·

CK I ¤ G1 G0

¤

(10b) (10c)

¤

(10d) (10e)

where Hi and Gi are the Markov parameters for the deterministic input and innovation sequence, respectively. We have the following partitioned equations:


6

Yf i = Γf i Xk + Hf i Ui + Gf i Ei

(11)

for i = 1, 2, · · · , f . Denote further, ∆

£

∆

£

Hf−i = G− fi =

Hi−1 · · · Gi−1 · · ·

H1 G1

¤

(12a)

¤

(12b)

The partitioned Yf i in Eq. 11 is equal to Yf i = Γf i Xk + Hf−i Ui−1 + Hf 1 U1 + G− f i Ei−1 + Ef i

(13)

By eliminating ek in the innovation model (Eq. 1) through iteration, it is straightforward to derive the following relation (Knudsen, 2001), Xk = Lz Zp + ApK Xk−p

(14)

where ∆

£

∆p (AK , K) ∆p (AK , BK ) ¤ ∆ £ Ap−1 B · · · AB B ∆p (A, B) = Lz =

¤

(15a) (15b)

AK

= A − KC

∆

(15c)

BK

∆

(15d)

Zp

= B − KD ¤T ∆ £ YpT UpT =

(15e)

Substituting this equation into Eq. 13, we obtain Yf i = Γf i Lz Zp + Γf i ApK Xk−p + Hf−i Ui−1 + Hf 1 U1 + G− f i Ei−1 + Ef i

(16)

for i = 1, 2, · · · , f . Note that the second term in the right hand side (RHS) of Eq. 16 tends to zero as p tends to infinity under assumption A1. To facilitate the derivation of the main results, we assume that, in this subsection, the process described by Eq. 1 does not contain the direct term, i.e., Hf 1 = 0. Therefore, Eq. 16 reduced to Yf i = Γf i Lz Zp + Hf−i Ui−1 + G− f i Ei−1 + Ef i

(17)

Since the future innovation, Ef i , is uncorrelated with Zp , Ui−1 and Ei−1 in Eq. 17 under closed-loop condition. If the Ef is already known, the coefficient matrices can be estimated through a straightforward linear regression as: †  Zp h i ˆ f i Lz H ˆ− G ˆ − = Yf i  Ui−1  (18) Γ fi fi Ei−1 where † stands for pseudo-inverse operation. A remaining question is whether this approach gives us a consistent estimate of Γf i Lz , Hf−i and G− f i . Here we formulate the results as follows.


7

Lemma · 1 ¸Under the assumptions stated in Subsection 2.1, the joint input-output signal, yk χk = , is persistently exciting of any order. uk [Proof ] See Appendix A10.1 in (Söderstr¨ om and Stoica, 1989). Theorem 1 The estimates of Γf i Lz , Hf−i and G− f i in Eq.· 18 ¸are consistent for ∀i = yk 1, 2, · · · , f if and only if the joint input-output signal, χk = , is persistently exciting uk to the order of p+f −1, where p and f denote the past and the future horizons, respectively. [Proof ] See Appendix A. [Remark 1] The analysis of consistency for the case of D 6= 0 is similar to that presented in this subsection with the help of the assumption A4, while it requires χk to be consistently exciting to the order of f + p. The proof is similar to the one provided in Appendix A through a minor modification. [Remark 2] From the derivation in Appendix A, we can see that the key is to maintain £ ¤T T T Ei−1 the full row rank of ZpT Ui−1 , which is the assumption for the existence of the oblique predict space in Chiuso and Picci’s (2003) analysis. One of the main contributions of this work is to provide the explicit assumptions under which the rank condition is valid.

3

Closed-loop subspace identification methods with innovation estimation

From the analysis in Section 2, we can conclude that, under certain assumptions, the consistency estimation with SIMs can be achieved if the innovation sequence is already known. The only challenge left now is how to estimate the innovation signal. Qin and Ljung (2003b) provided an algorithm using a parsimonious model formulation with innovation estimation (PARSIM-E) for closed-loop identification. In this section, we introduce the algorithm in more detail. We would like to stress that there should be some other ways to identify the closed-loop system with innovation estimation using the parsimonious formulation. The algorithm reported should be seen as an illustration of the feasible method.

3.1

Parsimonious SIM with innovation estimation

In this subsection, we present that parsimonious SIM algorithm with innovation estimation. Similar to Subsection 2.2, in this subsection, we assume that D = 0 in Eq. 1. By ignoring the second term on the RHS of Eq. 16 and set i = 1, we have Yf 1 = Γf 1 Lz Zp + E1

(19)

Therefore, a least squares estimate of the innovation process is: ˆ 1 = Yf 1 − Γ ˆ f 1 Lz Zp E

(20)

TWMCC Technical Report 2004-07 where

8

ˆ f 1 Lz = Yf 1 Zp† Γ

(21)

Now return to Eq. 17 for a general i = 2, 3, . . . , f . Noting that   Ef 1 ¸  Ef 2  · Ei−1   Ei =  .  = Ef i  ..  Ef i

(22)

î−1 , Eq. 17 becomes and replacing Ei−1 with E h Yf i =

Γf i Lz Hf−i G− fi

i



 Zp  Ui−1  + Ef i î−1 E

(23)

The least squares estimate h

ˆ f i Lz H ˆ− G ˆ− Γ fi fi

i



† Zp = Yf i  Ui−1  î−1 E

(24)

With the least squares estimates of Γf i Lz from Eq. 21 and Eq. 24, we obtain  ˆ Γf 1 L z  Γ ˆ f 2 Lz ˆ f Lz =  Γ  ..  . ˆ f f Lz Γ

    

The observability matrix, Γf , can be estimated similarly to the PARSIM-E procedures given in (Qin and Ljung, 2003a). ˆ f = W −1 U S 1/2 Γ 1 where

(25)

ˆ f Lz )W2 = U SV T W1 (Γ

and the weighting matrices W1 = I W2 =

T 1/2 (Zp Π⊥ Uf Z p )

(26a) (26b)

ˆ f from PARSIM-E is consistent under the assumptions A1 Theorem 2 The estimate of Γ to A6 stated in Section 2.1.


9

ˆ f from PARSIM-E, it is sufficient to show that as [Proof ] To prove the consistency of Γ N →∞ h i h i ˆ f i Lz H ˆ− G ˆ − → Γf i Lz H − G− Γ fi fi fi fi Note that if the innovation sequence is already known it has been proven in Theorem 1. Therefore, Theorem 2 is valid if î−1 → Ei−1 E as N → ∞, which can be proven recursively with the help of Theorem 1. [Remark 1] After obtaining the consistent estimate of the extended observability matrix, the A and C matrices can be estimated as in (Verhaegen, 1994). [Remark 2] For D 6= 0 case, the PARSIM-E is consistent as well if there is a delay in the controller. [Remark 3] As stated at the beginning of this section, there are other ways to estimate the innovation with the parsimonious formulation. For example, one can estimate E1 ˆf . directly from Eq. 20, and takes advantage of the Hankel structure of Ef to construct E In this case, the proof of Theorem 2 needs not to be recursive and the possibility of error propagation through recursion is eliminated.

3.2

Determination of the system order

Another problem associated with the subspace identification is the lack of well established rule for determining the order of the system. Larimore (1990) adopted Akaike information criterion (AIC) to determining the order of the system, but the detail derivation was omitted in the paper. AIC was original developed by Akaike (1974) from maximum likelihood estimation. For a given set of samples and a sequence of system order, the order of system is the one which makes the AIC index: AIC(n) = N (ny (1 + ln 2π) + ln |Σn |) + 2fn Mn

(27)

minimum, where n is the system order, N is the number of samples, Σn =

N 1 X eˆ(k)ˆ eT (k) N k=1

and eˆ(k) = y(k) − yˆ(k) is the estimated innovation sequence. Mn and fn are the total number of independently adjustable parameters in the model and the small sample factor (Larimore, 1990), respectively. To successfully apply AIC for the order determination for PARSIM-E, one needs to estimate the innovation sequence. Here we propose an algorithm to estimate the sequence through the best rank-n approximation of the following weighted matrix ˆ f Lz )W2 = Un Sn VnT + ε W1 (Γ

(28)


10

where W1 and W2 are represented in Eq. 26, Un is the first n left singular vectors, Sn is the corresponding singular values in decreasing order, and Vn is the first n right singular vectors. The residual term ε stands for the product of the remaining singular vectors and ˆ f Lz is singular values. Therefore, a rank-n approximation of Γ ˆ f Lz = W −1 Un Sn VnT W −1 Γ 1 2

(29)

ˆ f 1 Lz . Substituting it into Eq. 16, we obtained and the first ny rows of it correspond to Γ the estimated innovation sequence as ˆ f 1 Lz Zp − Γ ˆ f 1 Ap Xk−p − Hf 1 U1 Eˆ1 = Yf 1 − Γ K

(30)

The third term on the RHS tends to zero as p tends to infinity, and fourth term can be eliminated by a projection ⊥ ˆ Eˆ1 = Yf 1 Π⊥ U1 − Γf 1 Lz Zp ΠU1

Here we use the fact that E1 Π⊥ U1 = E1 as N → ∞, which is valid for closed-loop data under the assumption A4. After calculating Σn as Σn =

1 ˆ ˆT E1 E1 N

(31)

the AIC index for different system order n can be obtained through Eq. 27. Note that with this approach the estimated innovation sequence is white given the correct order of the system.

3.3

K estimation under closed-loop condition

Ruscio (1996) proposed a way to identify the Kalman filter gain with QR implementation for open loop data. It requires that Ef is uncorrelated to Zp and Uf , which is invalid under closed-loop condition. In this subsection, we provide a new way to calculate K with closed-loop data. After the determination of the system order through the Akaike information criterion, we can obtain the estimate of the innovation sequence ek , which can be used to construct ˆf . Substituting the Xk in the extended state space model (Eq. 4) with Eq. 14, we obtain E Yf = Γf Lz Zp + Γf ApK Xk−p + Hf Uf + Gf Ef

(32)

ˆ f Lz (Eq. 29), Omitting the second term for a sufficient large p, and replacing Γf Lz with Γ Eq. 32 becomes Y˜f = Hf Uf + Gf Ef (33) where

ˆ f Lz Zp Y˜f = Yf − Γ

We can partition Y˜f row-wise as follows: Y˜f i = Hf i Ui + G− f i Ei−1 + Ef i ; i = 1, 2, . . . , f

(34)


11

î−1 , the least squares estimation Again, replacing Ei−1 with E · ¸† h i Ui − ˆ ˆ ˜ Hf i Gf i = Yf i ˆ ; i = 2, 3, . . . , f Ei−1 With the definition of G− f i in Eq. 12b, h i ˆ − I ; i = 2, 3, . . . , f ˆf i = G G fi

(35)

(36)

ˆ f i is the estimation of Gf i in Eq. 10d. where G Therefore, the estimate of Gf can be obtained based on Eq. 6b, which is lower triangular but is not exactly Toeplitz due to estimation error. After taking the average of the diagonal ˆ f , the Toeplitz structure of G ˆ f can be preserve as, block components of G   I 0 ··· 0  G ˆ1 I ··· 0   ˆf =  G (37)  .. .. ..  . .  . . .  . ˆ f −1 G ˆ f −2 · · · I G Furthermore, notice that

    

G1 G2 .. . Gf −1







C CA .. .

    =  

   K = Γf −1 K 

CAf −2

The Kalman gain, K, can be calculated as   ˆ =Γ ˆ†  K f −1  

ˆ1 G ˆ2 G .. . ˆ Gf −1

    

(38)

ˆ † can be obtained as discussed in Subsection 3.1. To make the eigenvalue of the where Γ f −1 ˆ can be further refined by solving predictor A − KC lie strictly inside the unit circle, the K the steady state algebraic Riccati equation (Arnold and Laud, 1984). After calculating the estimates of K, the B and D matrices can be estimated optimally using the estimates of A, C, K and F (Qin and Ljung, 2003a), where F is the Cholesky decomposition of Eq. 31.

4

Simulation studies

In this section, two simulated case studies are reported under closed-loop condition. For comparison we use the N4SID routine in the System Identification Toolbox (Version 5.0) of Matlab, which actually implemented the canonical variate analysis (CVA) weighting, as the standard SIM algorithm.


4.1

12

Simulation example: a SISO process

We simulate the following single input and single output (SISO) process yk + ayk−1 = buk−1 + ek + cek−1

(39)

where a = −0.9, b = 1, and c = 0.5. The feedback controller is uk = −Kyk + rk

(40)

The reference signal, rk , and innovation sequence, ek , are white noise with cov(rk ) = 2, and cov(ek ) = 1, respectively. Open-loop experiments are simulated with K = 0 and closedloop experiments with K = 0.6. In both cases, we choose p = 9, f = 3 for PARSIM-E, and run 20 independent Monte Carlo simulations. The sampling size for each experiment is 4000. The identification results from PEM implemented using the ARMAX routine in Matlab’s System Identification Toolbox are used here as as a benchmark. The pole estimation results for both open-loop and closed-loop experiments are shown in Fig. 1. From the results, we can conclude that the performances of all three methods are excellent in the open-loop case. For the close-loop identification, the estimate from PARSIM-E is comparable with that from PEM, while the traditional SIM with the CVA weighting fail to provide consistent estimates. The results of frequency response estimations for the closed-loop simulation are shown in Fig. 2, Fig. 3, and Fig. 4. From the results we can see that the estimate of frequency response from N4SID is biased. The identification results from PARSIM-E is very close to those from PEM. The estimates of Kalman predictor’s (A − KC) pole for both open-loop and closed-loop simulations are shown in Fig. 5. As expected the the estimates are excellent for the openloop case. For the close-loop case, the estimate result from PARSIM-E is unbiased as well as that from PEM, while the traditional SIM with the CVA weighting results in a biased estimate.


4.2

13

Simulation example: a MIMO process

In this subsection, we simulate the following 2 × 2 linear dynamic system   0.67 0.67 0 0  −0.67 0.67 0 0  x xk+1 =   0 0 −0.67 −0.67  k 0 0 0.67 −0.67   0.6598 −0.5256  1.9698 0.4845   +   4.3171 −0.4879  uk −2.6436 −0.3416   −0.6968 −0.1474  0.1722 0.5646   +   0.6484 −0.4660  ek −0.9400 0.1032 · ¸ −0.3749 0.0751 −0.5225 0.5830 yk = xk + ek −0.8977 0.7543 0.1159 0.0982

(41a)

(41b)

The output feedback controller is uk = rk + Fb yk

(42)

where rk is the reference signal and Fb is the feedback gain matrix. In the experiment, we use the pseudo-random binary signals (PRBS) with clock period of 5 samples as the reference sequences. 4000 samples of the input and output data are generated to identify the model with · ¸ 1 0 cov(ek ) = 0 1 and

· Fb =

−0.25 0 0 −0.25

¸

We choose p = 9, f = 5 for PARSIM-E, and run 10 independent Monte Carlo simulations. The pole estimation results for the closed-loop experiments are shown in Fig. 6 and Fig. 7. From the results we can see that the PARSIM-E provide consistent estimates, while the N4SID subroutine with CVA weighting results in biased estimates. The estimates of the frequency response for the closed-loop simulations are shown in Fig. 8 and Fig. 9. We can see that the estimated frequency responses from PARSIM-E match well with that of the real system. The traditional SIM fails to provide the consistent frequency responses. The Kalman predictor’s pole estimation from N4SID and PARSIM-E with the closedloop data are shown in Fig. 10 and Fig. 11, respectively. From the figures, we can conclude that PARSIM-E has much better performance for the Kalman gain estimation than the N4SID routine in Matlab System Identification Toolbox.


5

14

Conclusions

In this paper, the feasibility of the closed-loop subspace identification is established. It is shown that SIMs are feasible and consistent for closed-loop data with roughly the same identifiability requirements as more traditional methods such as PEMs. The key idea is that the consistent identification can be achieved through innovation estimation. The new algorithm is shown to be consistent under certain assumptions, which are also required by traditional joint input-output identification approach in PEM framework. The simulation studies show that the proposed algorithm is consistent under closed-loop condition, while the traditional SIMs with CVA weighting fail to provide consistent estimates.

Acknowledgement Financial support from Natural Science Foundation under CTS-9985074, National Science Foundation of China under an Overseas Young Investigator Award (60228001), a Faculty Research Assignment grant from University of Texas, and Weyerhaeuser Company through sponsorship of the Texas-Wisconsin Modeling and Control Consortium is gratefully acknowledged.

References [1] H. Akaike. A new look at the statistical model identificatin. IEEE Trans. Auto. Cont., 19:716–723, 1974. [2] W. F. Arnold and A. J. Laub. Generalized eigenproblem algorithms and software for algebraic riccati equations. Proc. IEEE, 72:1746–1754, 1984. [3] D. Bauer and L. Ljung. Some facts about the choice of the weighting matrices in larimore type of subspace algorithms. Automatica, 38:763–773, 2002. [4] Dietmar. Bauer. Subspace algorithms. In Proceedings of the 13th IFAC SYSID Symposium, pages 1030–1041, Rotterdam, NL, Aug 2003. [5] Alessandro Chiuso and Giorgio Picci. Constructing the state of random processes with feedback. In Proceedings of the 13th IFAC SYSID Symposium, pages 881–886, Rotterdam, NL, Aug 2003. [6] C. T. Chou and M. Verhaegen. Subspace algorithms for the identification of multivariable dynamic errors-in-variables models. Automatica, 33:1857–1869, 1997. [7] U. Forssell and L. Ljung. Closed-loop identification revisited. Automatica, 35:1215– 1241, 1999. [8] T. Gustafsson. Subspace-based system identification: weighting and pre-filtering of instruments. Automatica, 38:433–443, 2002.


15

[9] H. Hjalmarsson. From experiments to closed loop control. In Proceedings of the 13th IFAC SYSID Symposium, pages 1–14, Rotterdam, NL, Aug 2003. [10] M. Jansson and Bo Wahlberg. A linear regression approach to state-space subspace system identification. Signal Processing, 52:103–129, 1996. [11] M. Jansson and Bo Wahlberg. On consistency of subspace methods for system identification. Automatica, 34:1507–1519, 1998. [12] T. Knudsen. Consistency analysis of subspace identification methods based on linear regression approach. Automatica, 37:81–89, 2001. [13] W. E. Larimore. Canonical variate analysis in identification, filtering and adaptive control. In IEEE Conference on Decision and Control, pages 596–604, Dec 1990. [14] L. Ljung. Convergence analysis of parametric identification methods. IEEE Trans. Auto. Cont., 23:770–783, 1978. [15] L. Ljung. Asymptotic variance expression for identified black-box tranfer function models. IEEE Trans. Auto. Cont., 30:834–844, 1985. [16] L. Ljung. System Identification - Theory for the User. Prentice Hall PTR, second edition, 1999. [17] L. Ljung and T. McKelvey. Subspace identification from closed loop data. Signal Processing, 52:209–215, 1996. [18] P. Van Overschee and B. De Moor. N4sid: subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica, 30:75–93, 1994. [19] P. Van Overschee and B. De Moor. A unifying theroem for three subspace system identification algorithms. Automatica, 31:1853–1864, 1995. [20] S. J. Qin, W. Lin, and L. Ljung. A novel subspace identication approach with parsimonious parametrization. Submitted for publication, 2003. [21] S.J. Qin and Lennart Ljung. Closed-loop subspace identification with innovation estimation. In Proceedings of the 13th IFAC SYSID Symposium, pages 887–892, Rotterdam, NL, Aug 2003. [22] S.J. Qin and Lennart Ljung. Parallel qr implementation of subspace identification with parsimonious models. In Proceedings of the 13th IFAC SYSID Symposium, pages 1631–1636, Rotterdam, NL, Aug 2003. [23] D. Di Ruscio. Combined deterministic and stochastic system identification and realization: Dsr-a subspace approach based on observations. Modeling, Identification and Control, 17:193–230, 1996. [24] T. Soderstrom and P. Soica. System Identification. Prentice Hall, 1989.


16

[25] M. Verhaegen. Application of a subspace model identification technique to identify lti systems operating in closed-loop. Automatica, 29:1027–1040, 1993. [26] M. Verhaegen. Identification of the deterministic part of mimo state space models given in innovations form from input-output data. Automatica, 30:61–74, 1994. [27] M. Verhaegen and P. Dewilde. Subspace model identification. part i: the output-error state-space model identification class of algorithms. International Journal of Control, 56:1187–1210, 1992. [28] J. Wang and S. J. Qin. A new subspace identification approach based on principle component analysis. J. Process Control, 12:841–855, 2002.

Appendix A From Eq. 17 we can conclude that, †  T   T −1 Zp Zp Zp Zp   Ui−1  =  Ui−1    Ui−1   Ui−1   Ei−1 Ei−1 Ei−1 Ei−1 

in Eq. 18. Therefore, the estimate of Γf i Lz , Hf−i and G− f i is consistent if and only if 1. Ef i is uncorrelated with Zp , Ui−1 and Ei−1 in Eq. 16.   Zp 2.  Ui−1  has full row rank (f.r.r.). Ei−1 The first condition is satisfied as mentioned in Subsection 2.2. Here we provide a proof for the second condition by induction. For i = 2, we can obtain Ef 1 from From Eq. 16 as Ef 1 = Yf 1 − Γf 1 Lz Zp Therefore,

    Zp I 0 0 Zp  U1  =  0 I 0   U1  Y1 M1 0 I E1 

where M1 = −Γf 1 Lz . From the results of Lemma 1, we know that χk is persistently ¤T £ has f.r.r. Now exciting of any order. Therefore, we can conclude that ZpT U1T E1T we assume that, for i = k + 1,      Zp I 0 0 Zp  Uk  =  0 I 0   Uk  Yk Mk Nk I Ek


17

where Mk and Nk are matrices with appropriate dimensions. Then for i = k + 2, from Eq. 17, we obtain Ef (k+1) = Yf (k+1) − Γf (k+1) Lz Zp − Hf−(k+1) Uk − G− f (k+1) Ek − − − = Yf (k+1) − (Γf (k+1) Lz + G− f (k+1) Mk )Zp − (Hf (k+1) + Gf (k+1) Nk )Uk − Gf (k+1) Yk

Therefore,      

Zp Uk Uf (k+1) Ek Ef (k+1)





    =    

I 0 0 I 0 0 Mk Nk Mk+1 N1(k+1)

0 0 0 0 I 0 0 I 0 N2(k+1)

0 0 0 0 I

     

Zp Uk Uf (k+1) Yk Yf (k+1)

     

where Mk+1 = −(Γf (k+1) Lz + G− f (k+1) Mk ) N1(k+1) = −(Hf−(k+1) + G− f (k+1) Nk ) N2(k+1) = −G− f (k+1) Then from Lemma 1, we conclude that the RHS of the above equation is f.r.r.


18

0

−0.5

Closed−loop Pole Estimation 0.5 PARSIM−E

PARSIM−E

Open Loop Pole Estimation 0.5

0

0.5

−0.5

1

0

0

0.5

1

0

0.5

1

0

0.5

1

0.5

PEM

PEM

0.5

0

−0.5

1

0.5

0

−0.5

0

0.5

N4SID

N4SID

0.5

−0.5

0

0

0.5

1

0

−0.5

Figure 1: Pole estimates for the SISO simulation example


19 Bode Diagram

30

true PEM

25

Magnitude (dB)

20 15 10 5 0 −5 −10 0

Phase (deg)

−45

−90

−135

−180 −2

10

−1

0

10

10 Frequency (rad/sec)

Figure 2: The estimates of the frequency response from PEM for SISO closed-loop simulations


20 Bode Diagram

30

true PARSIM−E

25

Magnitude (dB)

20 15 10 5 0 −5 −10 0

Phase (deg)

−45

−90

−135

−180 −2

10

−1

0

10


Figure 3: The estimates of the frequency response from PARSIM-E SISO closed-loop simulations


21 Bode Diagram

20 true N4SID

Magnitude (dB)

15 10 5 0 −5 −10 0

Phase (deg)

−45

−90

−135

−180 −2

10

−1

0

10


Figure 4: The estimates of the frequency response from N4SID for SISO closed-loop simulations


0

−0.5

Closed−Loop Predictor Pole Estimation 0.5 PARSIM−E

PARSIM−E

Open Loop Predictor Pole Estimation 0.5

22

−1

−0.5

−0.5

0

0

−1

−0.5

0

−1

−0.5

0

−1

−0.5

0

0.5

PEM

PEM

−0.5

0

−0.5

0

0.5

0

−0.5

−1

0.5

N4SID

N4SID

0.5

−0.5

0

−1

−0.5

0

0

−0.5

Figure 5: Predictor’s pole estimates for the SISO simulation example


23

1

0.8

0.6

0.4

Imaginary

0.2

0

−0.2

−0.4

−0.6

−0.8

−1 −1

−0.8

−0.6

−0.4

−0.2

0 Real

0.2

0.4

0.6

0.8

1

Figure 6: The N4SID pole estimation for 10 Monte-Carlo closed-loop simulations: × estimated pole, + system pole


24

1

0.8

0.6

0.4

Imaginary

0.2

0

−0.2

−0.4

−0.6

−0.8

−1 −1

−0.8

−0.6

−0.4

−0.2

0 Real

0.2

0.4

0.6

0.8

1

Figure 7: The PARSIM-E pole estimation for 10 Monte-Carlo closed-loop simulations: × estimated pole, + system pole


25 Bode Diagram

From: U(1)

From: U(2)

To: Y(1)

50

0 true N4SID −50

Phase (deg); Magnitude (dB) To: Y(1) To: Y(2)

360 0

−360 −720 50

0

−50

To: Y(2)

360 0 −360 −720 −2 10

−1

10

0

10

−2

10

−1

10

0

10

Frequency (rad/sec)

Figure 8: The estimates of the frequency response from N4SID for MIMO closed-loop simulations


26 Bode Diagram

From: U(1)

From: U(2)

To: Y(1)

50

0 true PARSIM−E −50

Phase (deg); Magnitude (dB) To: Y(1) To: Y(2)

360 0

−360 −720 40 20 0 −20

To: Y(2)

360 0 −360 −720 −2 10

−1

10

0

10

−2

10

−1

10

0

10

Frequency (rad/sec)

Figure 9: The estimates of the frequency response from PARSIM-E for MIMO closed-loop simulations


27

1

0.8

0.6

0.4

Imaginary

0.2

0

−0.2

−0.4

−0.6

−0.8

−1 −1

−0.8

−0.6

−0.4

−0.2

0 Real

0.2

0.4

0.6

0.8

1

Figure 10: The estimates of the predictor’s pole from N4SID for 10 Monte-Carlo MIMO closed-loop simulations: × estimated predictor’s pole, + true predictor’s pole


28

1

0.8

0.6

0.4

Imaginary

0.2

0

−0.2

−0.4

−0.6

−0.8

−1 −1

−0.8

−0.6

−0.4

−0.2

0 Real

0.2

0.4

0.6

0.8

1

Figure 11: The estimates of the predictor’s pole from PARSIM-E for 10 Monte-Carlo closed-loop simulations: × estimated predictor’s pole, + true predictor’s pole