Two-stage parameter estimation algorithms for Box ... - IEEE Xplore

www.ietdl.org Published in IET Signal Processing Received on 25th June 2012 Revised on 22nd December 2012 Accepted on 22nd April 2013 doi: 10.1049/iet-spr.2012.0183

ISSN 1751-9675

Two-stage parameter estimation algorithms for Box–Jenkins systems Feng Ding1,2, Honghong Duan1 1

Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), Jiangnan University, Wuxi 214122, People’s Republic of China 2 Control Science and Engineering Research Center, Jiangnan University, Wuxi 214122, People’s Republic of China E-mail: [email protected]

Abstract: A two-stage recursive least-squares identification method and a two-stage multi-innovation stochastic gradient method are derived for Box–Jenkins (BJ) systems. The key is to decompose a BJ system into two subsystems, one containing the parameters of the system model and the other containing the parameters of the noise model, and then to estimate the parameters of the system model and the noise model, respectively. The simulation examples indicate that the proposed algorithms can generate highly accurate parameter estimates and require small computational burden.

1

Introduction

Parameter estimation methods have been used widely in system identification [1–3], fault detection [4], signal processing [5–7], state estimation and filtering [8] and adaptive control [9–11]. For example, Petrović and Stevanović [5] studied the estimation algorithm for Fourier coefficients; Fattah, Zhu and Ahmad [6] discussed identification problems of autoregressive moving average systems based on the noise compensation in the correlation domain; Abderrahim, Mathlouthi and Msahli [7] presented an approach to identify the parameters of the finite impulse response systems using the higher-order statistics information. The Box–Jenkins (BJ) models can describe general stochastic systems, which include the output error model [12], the output error moving average (OEMA) model [13, 14], the output error autoregressive (OEAR) model [15] as special cases. Recently, Wang et al. presented the least-squares-based recursive and iterative estimation methods for OEMA systems using the data filtering technique [13], the auxiliary model-based recursive extended least squares (RELS) and multi-innovation extended least squares (MI-ELS) algorithms for Hammerstein OEMA systems [14] and the auxiliary model-based recursive generalised least-squares parameter estimation for Hammerstein OEAR systems [15]. For BJ models, Wang et al. derived a gradient-based iterative identification algorithm with finite measurement input / output data [16]; Liu et al. developed a least-squares-based iterative identification algorithm [17]. Also, Ding et al. proposed a gradient based and a least squares-based iterative identification algorithms [12], the auxiliary model based multi-innovation extended stochastic gradient (SG) algorithms [18] and the auxiliary model based multi-innovation extended least-squares (AM-RGELS) algorithms [19] for OEMA systems. 646 & The Institution of Engineering and Technology 2013

It is well known that the least-squares identification algorithm requires large computational burden because of computing the covariance matrix with large sizes and the SG identification algorithm [20] has poor parameter estimation accuracies. In order to improve parameter estimation accuracies and computational efficiencies, a two-stage least-squares-based iterative identification algorithm was proposed for controlled autoregressive moving average (CARMA) systems [21]; a two-stage recursive least-squares (TS-RLS) parameter estimation algorithm was proposed for output error models [22]; a two-stage least-squares-based iterative estimation algorithm was presented for CARMA systems using the data filtering [23, 24]. On the basis of the work in [24–26], this paper uses the auxiliary model identification idea [27] and the decomposition technique [28–31] to study identification problems of the BJ systems. The basic idea is to decompose a BJ system to two subsystems and then identify the parameters of each subsystem by using the auxiliary model identification idea, and furthermore to use the multi-innovation identification theory [32] and gives a two-stage multi-innovation SG identification algorithm. The rest of this paper is organised as follows. Section 2 describes the identification problems of the BJ systems. Section 3 derives a TS-RLS algorithm. Section 4 proposes a two-stage multi-innovation SG algorithm. Section 5 introduces the AM-RGELS identification algorithm for comparisons. Section 6 provides an illustrative example. Finally, concluding remarks are given in Section 7.

2

Problem formulation

Let us define some notations. ‘A = :X’ or ‘X: = A’ stands for ‘A is defined as X’; the symbol I (In) stands for an identity matrix of appropriate sizes (n × n); the superscript T denotes the IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

www.ietdl.org matrix / vector transpose; 1n represents an n-dimensional column vector whose elements are 1; the norm of a matrix X is defined by ||X||2: = tr[XX T]. Consider the BJ system depicted in Fig. 1 B(z) D(z) u(t) + v(t) A(z) C(z)

y(t) =

where the parameter vectors Θ, θ and ϑ are defined as 

 u [ Rn , n := na + nb + nc + nd q  T u: = a1 , a2 , . . . , ana , b1 , b2 , . . . , bnb [ Rna +nb

Q: =

(1)

 T q: = c1 , c2 , . . . , cnc , d1 , d2 , . . . , dnd [ Rnc +nd

A(z): = 1 + a1 z−1 + a2 z−2 + . . . + ana z−na B(z): = b1 z−1 + b2 z−2 + . . . + bnb z−nb C(z): = 1 + c1 z

−1

D(z): = 1 + d1 z

−1

+ c2 z

−2

+ d2 z

−2

+ . . . + cnc z

and the information vectors φ(t), φ(t) and ψ(t) are defined as −nc

+ . . . + dnd z



where [u(t)] is the system input sequence, [y(t)] is the system output sequence, [v(t)] is a stochastic white noise with zero mean and variance σ 2, A(z), B(z), C(z) and D(z) are polynomials in the unit backward shift operator z − 1 [i.e. z − 1y(t) = y(t − 1)] with the known orders na, nb, nc and nd. The objective of this paper is to develop a two-stage least-squares identification algorithm and an auxiliary model based multi-innovation SG identification algorithm to estimate the unknown parameters and test the effectiveness of the proposed algorithms by simulation. It is shown in Fig. 1 that x(t) is the output of the system model and w(t) is the output of the noise model. Define two intermediate variables B(z) u(t), A(z)

x(t) :=

w(t) :=

 f(t) [ Rn w(t): = c(t)   f(t): = −x(t − 1), − x(t − 2), . . . , − x t − na ,   T u(t − 1), u(t − 2), . . . , u t − nb [ Rna +nb   c(t): = −w(t − 1), − w(t − 2), . . . , − w t − nc ,   T v(t − 1), v(t − 2), . . . , v t − nd [ Rnc +nd

−nd

D(z) v(t) C(z)

From (1), we have y(t) = x(t) + w(t) = fT (t)u + cT (t)q + v(t) = wT (t)Q + v(t)

(4)

This equation is the identification model of the BJ systems in (1).

or

3 x(t) = [1 − A(z)]x(t) + B(z)u(t) =−

na 

ai x(t − i) +

i=1

nb 

The two-stage identification is based on the decomposition technique. In order to derive a TS-RLS algorithm for parameter estimation, we need to define an intermediate variable

bi u(t − i)

i=1

w(t) = [1 − C(z)]w(t) + D(z)v(t) =−

nc  i=1

ci w(t − i) +

nd 

The TS-RLS algorithm

y1 (t) := y(t) − cT (t)q

di v(t − i) + v(t)

(5)

Then the system in (4) can be decomposed into the following two fictitious subsystems

i=1

In vector forms, the above equations can be written as

y1 (t) = fT (t)u + v(t)

x(t) = fT (t)u

(2)

w(t) = cT (t)q + v(t)

w(t) = cT (t)q + v(t)

(3)

one containing parameter vector θ of the system model and the other containing parameter vector ϑ of the noise model. Define the cost function

J1 (u, q): =

t 

y1 (j) − fT (j)u

2

j=1

= Fig. 1 The BJ system IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

t 

w(j) − cT (j)q

2

j=1

647

& The Institution of Engineering and Technology 2013

www.ietdl.org then we have

Define the stacked information matrices ⎡

⎤ fT (1) ⎢ fT (2) ⎥ ⎢ ⎥ t×(na +nb ) ⎥ , F(t) := ⎢ ⎢ .. ⎥ [ R ⎣ . ⎦ ⎡



−1 uˆ (t) = FT (t)F(t) FT (t)[Y (t) − C(t)q]

(8)



−1 qˆ (t) = CT (t)C(t) CT (t)[Y (t) − F(t)u]

(9)

From the above expressions, uˆ (t) and qˆ (t) depend on the inverses of the matrices [Φ T(t)Φ(t)] and [Ψ T(t)Ψ(t)], respectively. In order to avoid computing the matrix inversion and to reduce the computational load, we define two covariance matrices

fT (t)

⎤ cT (1) ⎢ cT (2) ⎥ ⎢ ⎥ t×(nc +nd ) ⎥ C(t) := ⎢ ⎢ .. ⎥ [ R ⎣ . ⎦

T P −1 1 (t) := F (t)F(t)

(10)

T P −1 2 (t) := C (t)C(t)

(11)

−1 T P −1 1 (t) = P 1 (t − 1) + f (t)f(t)

(12)

−1 T P −1 2 (t) = P 2 (t − 1) + c (t)c(t)

(13)

cT (t) and the stacked vectors Y(t), Y1(t) and W(t) as ⎡

y(1)

It follows that

⎤

⎢ y(2) ⎥ ⎢ ⎥ t ⎥ Y (t) : = ⎢ ⎢ .. ⎥ [ R , ⎣ . ⎦ ⎡

y(t) y1 (1)

In terms of (8), (10), (12) and the definitions of Y(t) and Φ(t), we have

⎤

⎢ y (2) ⎥ ⎢ 1 ⎥ t ⎥ Y 1 (t) : = ⎢ ⎢ .. ⎥ = Y (t) − C(t)q [ R ⎣ . ⎦ y1 (t) ⎡ ⎤ w(1) ⎢ w(2) ⎥ ⎢ ⎥ t ⎥ W (t) : = ⎢ ⎢ .. ⎥ = Y (t) − F(t)u [ R ⎣ . ⎦ w(t) Then J1(θ, ϑ) can be written as  2 J1 (u, q) = Y 1 (t) − F(t)u

(6)

 2 = W (t) − c(t)q

(7)

uˆ (t) = P 1 (t)FT (t)[Y (t) − C(t)q]  = P 1 (t) FT (t − 1)[Y (t − 1) − C(t − 1)q]

 + f(t) y(t) − cT (t)q  T = P 1 (t) P −1 1 (t − 1)P 1 (t − 1)F (t − 1)[Y (t − 1)

 −C(t − 1)q] + f(t) y(t) − cT (t)q

T ˆ = P 1 (t) P −1 1 (t) − f(t)f (t) u (t − 1)

+ P 1 (t)f(t) y(t) − cT (t)q

= uˆ (t − 1) + P 1 (t)f(t) y(t) − cT (t)q − fT (t)uˆ (t − 1) In the same way, from (9), (11) and (13), we have

Let uˆ (t) and qˆ (t) be the estimates of θ and ϑ at time t. Letting the partial derivatives of J1(θ, ϑ) with respect to θ and ϑ be zero, respectively, gives ∂J1 (u, q) = −2FT (t)[Y 1 (t) − F(t)u] ∂u = −2FT (t)[Y (t) − C(t)q − F(t)u] = 0 ∂J1 (u, q) = −2CT (t)[W (t) − C(t)q] ∂q

qˆ (t) = P 2 (t)CT (t)[Y (t) − F(t)u]

= qˆ (t − 1)+ P 2 (t)c(t) y(t) − fT (t)u − cT (t)qˆ (t − 1) Applying the matrix inversion formula  −1 (A + BC)−1 = A−1 − A−1 B I + CA−1 B CA−1 to (10) and (11) gives P 1 (t) = P 1 (t − 1) −

P 1 (t − 1)f(t)fT (t)P 1 (t − 1) 1 + fT (t)P 1 (t − 1)f(t)

P 2 (t) = P 2 (t − 1) −

P 2 (t − 1)c(t)cT (t)P 2 (t − 1) 1 + cT (t)P 2 (t − 1)c(t)

= −2CT (t)[Y (t) − F(t)u − C(t)q] = 0 Thus, we have

FT (t)F(t) uˆ (t) = FT (t)[Y (t) − C(t)q] T

C (t)C(t) qˆ (t) = CT (t)[Y (t) − F(t)u] If the matrices [Φ T(t)Φ(t)] and [Ψ T(t)Ψ(t)] are non-singular, 648 & The Institution of Engineering and Technology 2013

Define two gain vectors

−1 L1 (t): = P 1 (t)f(t) = P 1 (t − 1)f(t) 1 + fT (t)P 1 (t − 1)f(t)

−1 L2 (t): = P 2 (t)c(t) = P 2 (t − 1)c(t) 1 + cT (t)P 2 (t − 1)c(t) IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

www.ietdl.org Then, we can obtain the following least-squares algorithm

uˆ (t) = uˆ (t − 1)

+ L1 (t) y(t) − cT (t)q − fT (t)uˆ (t − 1)

L1 (t) = P 1 (t − 1)f(t) 1 + f (t)P 1 (t − 1)f(t) T

−1



P 1 (t) = I − L1 (t)fT (t) P 1 (t − 1)

L2 (t) = P 2 (t − 1)c(t) 1 + cT (t)P 2 (t − 1)c(t)

(15) (16)

qˆ (t) = qˆ (t − 1)

+ L2 (t) y(t) − fT (t)u − cT (t)qˆ (t − 1)

(14)

−1



P 2 (t) = I − L2 (t)cT (t) P 2 (t − 1)

(17) (18)

the BJ models as

uˆ (t) = uˆ (t − 1)

+ L1 (t) y(t) − fˆ T (t)uˆ (t − 1) − cˆ T (t)qˆ (t − 1) (20)

−1 L1 (t) = P 1 (t − 1)fˆ (t) 1 + fˆ T (t)P 1 (t − 1)fˆ (t)

P 1 (t) = I − L1 (t)fˆ T (t) P 1 (t − 1), P(0) = p0 I   fˆ (t) = −ˆx(t − 1), − xˆ (t − 2), . . . , − xˆ t − na ,   T u(t − 1), u(t − 2), . . . , u t − nb

(21) (22) (23)

qˆ (t) = qˆ (t − 1)

+ L2 (t) y(t) − fˆ T (t)uˆ (t − 1) − cˆ T (t)qˆ (t − 1) (24)

(19)

Here, we can see that the right-hand sides of (14) and (17) contain the unknown parameter vectors ϑ and θ, respectively. The solution is to replace the unknown ϑ in (14) and θ in (17) with their corresponding estimates ϑ(t) and θ(t) at time t – 1. Thus, we have

uˆ (t) = uˆ (t − 1) + L1 (t)

y(t) − cT (t)qˆ (t − 1) − fT (t)uˆ (t − 1) qˆ (t) = qˆ (t − 1) + L2 (t)

y(t) − fT (t)uˆ (t − 1) − cT (t)qˆ (t − 1) A difficulty arises in that the information vectors φ(t) and ψ(t) contain the unknown inner variables x(t – i), w(t – i) and noise terms v(t – i), so it is impossible to obtain the estimates uˆ (t) and qˆ (t). The method here is based on the auxiliary model ˆ and vˆ (t) be the identification idea [27, 33, 34]. Let xˆ (t), w(t) estimates of x(t), w(t) and v(t), respectively, and define



−1 L2 (t) = P 2 (t − 1)cˆ (t) 1 + cˆ T (t)P 2 (t − 1)cˆ (t)

P 2 (t) = I − L2 (t)cˆ T (t) P 2 (t − 1), P(0) = p0 I   ˆ − 1), − w(t ˆ − 2), . . . , − w ˆ t − nc , cˆ (t) = −w(t   T vˆ (t − 1), vˆ (t − 2), . . . , vˆ t − nd

(25) (26) (27)

xˆ (t) = fˆ T (t)uˆ (t)

(28)

ˆ = y(t) − xˆ (t) w(t)

(29)

ˆ − cˆ T (t)qˆ (t) vˆ (t) = w(t)

(30)

The steps of implementing the TS-RLS algorithm in (20)–(30) to compute uˆ (t) and qˆ (t) are listed in the following: 1. Let t = 1, set the initial values uˆ (0) = 1na +nb /p0 , ˆ = 0, qˆ (0) = 1nc +nd /p0 , u(i) = 0, y(i) = 0, xˆ (i) = 0, w(i) vˆ (i) = 0 (i ≤ 0), p0 being a large number, for example, p0 = 106. 2. Collect the input / output data u(t) and y(t), from fˆ (t) by (23), and cˆ (t) by (27).



 ˆ (t) f [ Rn wˆ (t) := ˆ c (t)   fˆ (t): = −ˆx(t − 1), − xˆ (t − 2), . . . , − xˆ t − na ,   T u(t − 1), u(t − 2), . . . , u t − nb [ Rna +nb   ˆ − 1), − w(t ˆ − 2), . . . , − wˆ t − nc , cˆ (t): = −w(t   T vˆ (t − 1), vˆ (t − 2), . . . , vˆ t − nd [ Rnc +nd ˆ and vˆ (t) can be According to (2)–(4), the estimates xˆ (t), w(t) computed by the following equations xˆ (t) = fˆ T (t)uˆ (t) ˆ = y(t) − xˆ (t) w(t) ˆ − cˆ T (t)q(t) vˆ (t) = w(t) Thus, replacing φ(t) and ψ(t) in (14)–(19) with their estimates fˆ (t), cˆ (t), we can summarise the TS-RLS identification algorithm for estimating the parameter vectors θ and ϑ of IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

Fig. 2 Flowchart of computing the TS-RLS parameter estimates uˆ ( t) and qˆ ( t) 649

& The Institution of Engineering and Technology 2013

www.ietdl.org 3. Compute L1(t) by (21), P1(t) by (22), L2(t) by (25) and P2(t) by (26). 4. Update the parameter estimates uˆ (t) by (20) and qˆ (t) by (24). ˆ by (29) and vˆ (t) by (30). 5. Compute xˆ (t) by (28), w(t) 6. Increase t by 1 and go to Step 2, continue the recursive calculation.

Define the innovation vector ˆ T (p, t)uˆ (t − 1) − C ˆ T (p, t)qˆ (t − 1) E(p, t) = Y (p, t) − F Then, (31) and (32) can be written as ˆ F(p, t) uˆ (t) = uˆ (t − 1) + E(p, t) r(t)

The flowchart of computing the parameter estimates uˆ (t) and qˆ (t) in the TS-RLS algorithm is shown in Fig. 2 [1, 12].

4 The two-stage multi-innovation SG algorithm It is well known that the SG algorithm has poor parameter accuracy and slow convergence rate [18, 32]. In order to improve the parameter accuracy, we consider p data from t ˆ = t − p + 1 to t = t and define the information matrix F(p, t), ˆ C(p, t) and the stacked output vector Y( p, t) as

ˆ F(p, t) = fˆ (t), fˆ (t − 1), . . . , fˆ (t − p + 1) [ R(na +nb )×p

ˆ C(p, t) = cˆ (t), cˆ (t − 1), .. . , cˆ (t − p + 1) [ R(nc +nd )×p Y (p, t) = [y(t), y(t − 1), . . . , y(t − p + 1)]T [ Rp where the positive integer p denotes the innovation length. Define the cost function J2 (u, q): =

p−1 



y1 (t − i) − f (t − i)u T

2

i=0

=

p−1 

w(t − i) − cˆ T (t − i)q

2

i=0

Using the SG search [32], we have

uˆ (t) = uˆ (t − 1) + m(t)

p−1 

fˆ (t − i)

i=0

y(t − i) − cˆ T (t − i)qˆ (t − 1) − fT (t − i)uˆ (t − 1)



ˆ t) = uˆ (t − 1) + m(t)F(p,   ˆ T (p, t)uˆ (t − 1) ˆ T (p, t)qˆ (t − 1) − F Y (p, t) − C (31)

qˆ (t) = qˆ (t − 1) + m(t)

p−1 

cˆ (t − i)

i=0

y(t − i) − fˆ T (t − i)uˆ (t − 1) − cˆ T (t − i)qˆ (t − 1)



ˆ = qˆ (t − 1) + m(t)C(p, t)   ˆ T (p, t)qˆ (t − 1) ˆ T (p, t)uˆ (t − 1) − C Y (p, t) − F (32) Here, μ(t) = 1 / r(t) is the step-size and we let [35] r(t) = r(t − 1)+  f(t) 2 +  c(t) 2 ,

r(0) = 1

650 & The Institution of Engineering and Technology 2013

ˆ C(p, t) qˆ (t) = qˆ (t − 1) + E(p, t) r(t) From here, we can obtain the two-stage multi-innovation stochastic gradient (TS-MI-GESG) algorithm with the innovation length p 

     ˆ 1 F(p, uˆ (t) uˆ (t − 1) t) = + E(p, t) ˆ qˆ (t) qˆ (t − 1) r(t) C(p, t)

(33)

ˆ T (p, t)qˆ (t − 1) ˆ T (p, t)uˆ (t − 1) − C E(p, t) = Y (p, t) − F (34) r(t) = r(t − 1)+  fˆ (t) 2 +  cˆ (t) 2 , r(0) = 1 (35)

ˆ (36) F(p, t) = fˆ (t), fˆ (t − 1), . . . , fˆ (t − p + 1)

ˆ C(p, t) = cˆ (t), cˆ (t − 1), . . . , cˆ (t − p + 1)

T Y (p, t) = y(t), y(t − 1), . . . , y(t − p + 1)   fˆ (t) = −ˆx(t − 1), − xˆ (t − 2), . . . , − xˆ t − na ,   T u(t − 1), u(t − 2), . . . , u t − nb   ˆ − 1), − w(t ˆ − 2), . . . , − w ˆ t − nc , cˆ (t) = −w(t   T vˆ (t − 1), vˆ (t − 2), . . . , vˆ t − nd

(37) (38) (39)

(40)

xˆ (t) = fˆ T (t)uˆ (t)

(41)

ˆ = y(t) − xˆ (t) w(t)

(42)

ˆ − cˆ T (t)qˆ (t) vˆ (t) = w(t)

(43)

As p = 1, the TS-MI-GESG algorithm reduces to the two-stage generalised extended SG (TS-GESG) algorithm, that is, the GESG algorithm is a special case of the TS-MI-GESG algorithm. The steps of implementing the TS-MI-GESG algorithm in (33)–(43) to compute uˆ (t) and qˆ (t) are listed in the following: 1. Let t = 1, set the initial values uˆ (0) = 1na +nb /p0 , ˆ = 0, qˆ (0) = 1nc +nd /p0 , u(i) = 0, y(i) = 0, xˆ (i) = 0, w(i) vˆ (i) = 0 (i ≤ 0), p0 = 106. 2. Collect the input / output data u(t) and y(t), from fˆ (t), ˆ ˆ cˆ (t), Y( p, t), F(p, t) and C(p, t) by (36)–(40). 3. Compute E( p, t) by (34) and r(t) by (35). 4. Update the parameter estimates uˆ (t) and qˆ (t) by (33). ˆ by (42) and vˆ (t) by (43). 5. Compute xˆ (t) by (41), w(t) 6. Increase t by 1 and go to step 2. The flowchart of computing the parameter estimates uˆ (t) and qˆ (t) in the TS-MI-GESG algorithm is shown in Fig. 3 [1, 12]. IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

www.ietdl.org xˆ (t) = fˆ T (t)uˆ (t)

(50)

ˆ = y(t) − xˆ (t) w(t)

(51)

ˆ − cˆ T (t)qˆ (t) vˆ (t) = w(t)

(52)

Here, P(t) is a covariance matrix and L(t) is a gain vector. To initialise this algorithm, we take uˆ (0) to be a small real vector ˆ = 1/p0 , for example, uˆ (0) = 1n /p0 , xˆ (0) = 1/p0 , w(0) vˆ (0) = 1/p0 and p0 = 106.

6

Examples

Example 1: consider the following BJ system y(t) =

B(z) D(z) u(t) + v(t) A(z) C(z)

A(z) = 1 + 0.58z−1 + 0.72z−2 B(z) = −0.25z−1 + 0.46z−2 C(z) = 1 + 0.75z−1 D(z) = 1 − 0.20z−1 Fig. 3 Flowchart of computing the TS-MI-GESG parameter estimates uˆ ( t) and qˆ ( t)

Q = [a1 , a2 , b1 , b2 , c1 , d1 ]T = [0.58, 0.72, − 0.28, 0.46, 0.75, − 0.20]T

5

The AM-RGELS algorithm

To illustrate the advantage of less computational load, we introduce the AM-RGELS algorithm for comparison with the proposed algorithm in this section. From Fig. 1, x(t – i), w(t – i) and v(t – i) in the information vector are the unmeasurable variables φ(t). If these unknown variables x(t – i), w(t – i) and v(t – i) are replaced with their ˆ − i) and vˆ (t − i), then minimising the estimates xˆ (t − i), w(t cost function J3 (Q) =

t 

y(j) − wˆ T (j)Q

2

j=1

leads to the following AM-RGELS algorithm for estimating parameter vector Θ [17, 27]   ˆ = Q(t ˆ − 1) + L(t) y(t) − wˆ T (t)Q(t ˆ − 1) Q(t)

−1 L(t) = P(t − 1)wˆ (t) 1 + wˆ T (t)P(t − 1)wˆ (t)



P(t) = I − L(t)wˆ (t) P(t − 1), P(0) = p0 I     uˆ (t) fˆ (t) ˆ , Q(t) = ˆ wˆ (t) = ˆ q (t) c (t)   fˆ (t) = −ˆx(t − 1), − xˆ (t − 2), . . . , − xˆ t − na ,   T u(t − 1), u(t − 2), . . . , u t − nb T

  ˆ − 1), − w(t ˆ − 2), . . . , − w ˆ t − nc , cˆ (t) = −w(t   T vˆ (t − 1), vˆ (t − 2), . . . , vˆ t − nd

Example 2: consider the following BJ system

(44) y(t) = (45) (46) (47)

B(z) D(z) u(t) + v(t), A(z) C(z)

A(z) = 1 + 0.80z−1 + 0.84z−2 , B(z) = −0.25z−1 + 0.45z−2 , C(z) = 1 + 0.63z−1 , D(z) = 1 − 0.72z−1 ,

(48)

Q = [a1 , a2 , b1 , b2 , c1 , d1 ]T = [0.80, 0.84, − 0.25, 0.45, 0.63, − 0.72]T .



IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

In simulation, [u(t)] is taken as a persistent excitation signal sequence with zero mean and unit variance, and [v(t)] as a white noise sequence with zero mean and variances σ 2. Applying the proposed TS-RLS algorithm and the AM-RGELS algorithm to estimate the parameters of this system, the parameter estimates and estimation errors ˆ − Q  /  Q  against t are shown in Tables 1 d := Q(t) and 2, the parameter estimation errors of the TS-RLS and AM-RGELS algorithms are shown in Figs. 4 and 5 with different variances, and the parameter estimation errors of the two algorithms are compared in Fig. 6. The parameter estimates and estimation errors given by the BJ method in matlab are shown in Table 3. When σ 2 = 0.102 and σ 2 = 0.502, the corresponding noise-to-signal ratios are δns = 18.63 and 93.13%, respectively.

(49)

The simulation conditions are as in Example 1 with noise variance σ 2 = 0.502. Applying the proposed 651

& The Institution of Engineering and Technology 2013

www.ietdl.org Table 1 The TS-RLS estimates and errors with σ 2 = 0.102 and 0.502 σ2 0.102

0.502

T

a1

a2

b1

b2

c1

d1

δ, %

100 200 500 1 000 2 000 3 000 100 200 500 1 000 2 000 3 000

0.57686 0.58546 0.58391 0.58256 0.58059 0.58207 0.62922 0.62042 0.58099 0.58164 0.57860 0.58832 0.58000

0.75408 0.73883 0.72913 0.72162 0.72012 0.71903 0.74914 0.76238 0.73427 0.71332 0.71471 0.71142 0.72000

−0.27063 −0.28359 −0.27868 −0.27812 −0.27940 −0.27993 −0.23028 −0.30619 −0.27689 −0.27443 −0.27910 −0.28071 −0.28000

0.42775 0.44265 0.45233 0.45931 0.46311 0.46179 0.32224 0.40993 0.44172 0.46800 0.48125 0.47253 0.46000

0.22581 0.45792 0.57197 0.65911 0.72865 0.74155 0.40541 0.54883 0.63574 0.69811 0.74787 0.75516 0.75000

−0.80507 −0.50975 −0.41844 −0.29597 −0.22978 −0.20338 −0.66468 −0.46809 −0.39148 −0.27809 −0.22417 −0.19912 −0.20000

60.67129 32.27018 21.33967 10.00338 2.78266 0.72298 45.35201 26.09319 16.96184 7.15042 2.47607 1.36987

true values

Table 2 The AM-RGELS estimates and errors with σ 2 = 0.102 and 0.502 σ2 0.102

0.502

true values

t

a1

a2

b1

b2

c1

d1

δ, %

100 200 500 1 000 2 000 3 000 100 200 500 1 000 2 000 3 000

0.58630 0.58801 0.58418 0.58242 0.58051 0.58199 0.59482 0.62228 0.58639 0.58506 0.57995 0.58867 0.58000

0.75432 0.73779 0.72739 0.72027 0.71930 0.71836 0.81389 0.77848 0.73975 0.71618 0.71510 0.71109 0.72000

−0.27647 −0.28744 −0.27962 −0.27882 −0.27985 −0.28017 −0.23715 −0.30307 −0.27385 −0.27358 −0.27892 −0.28031 −0.28000

0.42853 0.44422 0.45298 0.45975 0.46353 0.46216 0.32999 0.40985 0.43766 0.46600 0.48107 0.47275 0.46000

0.43337 0.53387 0.60731 0.67407 0.73426 0.74541 0.54600 0.62456 0.66901 0.71376 0.75493 0.76010 0.75000

−0.63920 −0.44938 −0.39950 −0.28757 −0.23037 −0.20368 −0.52444 −0.36083 −0.34644 −0.25337 −0.21312 −0.19134 −0.20000

41.11511 25.04355 18.57462 8.77077 2.60275 0.51287 31.61582 16.89193 12.87668 4.94852 1.95126 1.68165

Fig. 4 TS-RLS parameter estimation errors δ against t (σ2 = 0.102 and 0.502)

Fig. 5 AM-RGELS parameter estimation errors δ against t (σ2 = 0.102 and 0.502)

TS-MI-GESG algorithm to estimate the parameters of this system, the parameter estimates and estimation errors with different innovation length are shown in Table 4 and Fig. 7. To illustrate the advantages of the proposed algorithms, the numbers of multiplications and additions for each recursive computation of the TS-RLS algorithm and the AM-RGELS algorithm are listed in Table 5, where the numbers in the brackets denote the computation loads for a system with na = nb = 2, nc = nd = 1 and n = na + nb + nc + nd at each step [1, 36, 37]. From the simulation results in Tables 1–5 and Figs. 4–7, we can draw the following conclusions: 652 & The Institution of Engineering and Technology 2013

Fig. 6 Parameter estimation errors against t for the two algorithms (σ2 = 0.502) IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

www.ietdl.org Table 3 The BJ estimates and errors (σ 2 = 0.102 and 0.502) σ2 0.102

0.502

t

a1

a2

b1

b2

c1

d1

δ, %

100 200 500 1 000 2 000 3 000 100 200 500 1 000 2 000 3 000

0.58016 0.58338 0.57931 0.57886 0.57788 0.57889 0.58395 0.59820 0.57856 0.57503 0.57003 0.57501 0.58000

0.71885 0.71987 0.71789 0.71675 0.71668 0.71689 0.71581 0.72003 0.70941 0.70294 0.70306 0.70420 0.72000

−0.28155 −0.28553 −0.28230 −0.28077 −0.27972 −0.27979 −0.28817 −0.30907 −0.29314 −0.28468 −0.27862 −0.27904 −0.28000

0.45709 0.46227 0.46296 0.46381 0.46580 0.46421 0.44502 0.46950 0.47696 0.48110 0.49014 0.48174 0.46000

0.78652 0.78412 0.69165 0.72852 0.75976 0.76172 0.77179 0.77647 0.75738 0.76317 0.77555 0.77296 0.75000

−0.21139 −0.19142 −0.34554 −0.25298 −0.21670 −0.19818 −0.21841 −0.19343 −0.25104 −0.20299 −0.19309 −0.18132 −0.20000

2.90573 2.71153 11.86690 4.34260 1.55664 0.98435 2.55207 3.39182 4.30238 2.35006 3.38037 3.04843

true values

Table 4 The TS-MI-GESG estimates and errors (σ 2 = 0.502) p

t

a1

A2

b1

B2

c1

d1

δ, %

1

100 200 500 1 000 2 000 3 000 100 200 500 1 000 2 000 3 000 100 200 500 1 000 2 000 3 000

0.23803 0.24438 0.24556 0.24862 0.25186 0.25302 0.26801 0.26613 0.27836 0.30898 0.37203 0.41411 0.45689 0.49160 0.63366 0.74167 0.80518 0.80311 0.80000

0.11049 0.12692 0.14304 0.15710 0.17494 0.18546 0.41372 0.45149 0.49993 0.55040 0.61856 0.65317 0.47102 0.61639 0.77041 0.81504 0.85490 0.85548 0.84000

−0.15072 −0.16138 −0.16543 −0.16709 −0.17241 −0.17996 0.02246 −0.08369 −0.13877 −0.16424 −0.19220 −0.22317 −0.10851 −0.21175 −0.23567 −0.22944 −0.23277 −0.25468 −0.25000

0.42862 0.44280 0.44462 0.45359 0.46134 0.46315 0.19131 0.26772 0.29888 0.36521 0.41549 0.43145 0.46424 0.49202 0.46264 0.47283 0.46955 0.45831 0.45000

0.42777 0.45748 0.49664 0.51793 0.50914 0.50253 0.47334 0.51320 0.52663 0.56786 0.48310 0.46123 0.51307 0.55575 0.54210 0.67211 0.61372 0.60619 0.63000

−0.60194 −0.63546 −0.67859 −0.70575 −0.72670 −0.73472 −0.83561 −0.88347 −0.85466 −0.90421 −0.89147 −0.89173 −0.83322 −0.81582 −0.79878 −0.83673 −0.77151 −0.75480 −0.72000

60.12582 58.41802 56.97135 55.92571 54.98148 54.46002 50.48517 46.11752 42.29182 38.64913 33.74944 31.00741 34.49616 25.40604 13.61301 8.97985 3.90218 2.89573

5

10

true values

Fig. 7 TS-MI-GESG estimation errors against t (σ2 = 0.502)

Table 5

† As the data length t increases, the parameter estimation errors become smaller – see Tables 1, 2 and 4. † The parameter estimation accuracies of the TS-RLS and AM-RGELS algorithms become higher with the noise variances decreasing – see Figs. 4 and 5. † The parameter estimation errors given by the TS-RLS and AM-RGELS algorithm are close – see Fig. 6. † Increasing the innovation length p leads to smaller parameter estimation errors for the same data lengths – see Fig. 7. † The variances of the parameter estimation errors given the BJ method in matlab are larger than those of the TS-RLS algorithm. † The TS-RLS algorithm has less computational load than the AM-RGELS algorithm – see Table 5. † The TS-RLS and AM-RGELS algorithms can give more accurate parameter estimates than the BJ algorithm in matlab – see Table 3.

Comparison of the computational efficiency of the AM-RGELS, TS-RLS and TS-MI-GESG algorithms

7

Algorithms

The main contribution of this paper is to derive a TS-RLS algorithm and a two-stage multi-innovation SG algorithm for the BJ systems. The analysis indicates that the proposed TS-RLS and AM-RGELS algorithms can generate highly accurate parameter estimates and requires less computational load compared with the BJ algorithm in matlab. The

TS-RLS AM-RGELS

Number of multiplications

Number of additions

2(na + nb)2 + 2(nc + nd)2 + 5n[70] 2n 2 + 5n[102]

2(na + nb)2 + 2(nc + nd)2 + 3n[58] 2n 2 + 3n[90]

IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183

Conclusions

653

& The Institution of Engineering and Technology 2013

www.ietdl.org proposed methods in this paper can be extended to other linear or non-linear, scalar or multivariable systems with white noises or coloured noises [38–43].

8

Acknowledgments

This work was supported by the National Natural Science Foundation of China (grant no. 61273194), the Natural Science Foundation of Jiangsu Province (China, BK2012549), the PAPD of Jiangsu Higher Education Instituttions and by the 111 Project (B12018).

9

References

1 Ding, F.: ‘System identification – new theory and methods’ (Science Press, Beijing, 2013) 2 Wang, W., Ding, F., Dai, J.Y.: ‘Maximum likelihood least squares identification for systems with autoregressive moving average noise’, Appl. Math. Model., 2013, 36, (5), pp. 1842–1853 3 Li, J.H., Ding, F., Yang, G.W.: ‘Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems’, Math. Comput. Model., 2012, 55, (3–4), pp. 442–450 4 Dai, X., Breikin, T., Gao, Z., Wang, H.: ‘Dynamic modelling and robust fault detection of a gas turbine engine’. 2008 American Control Conf., Seattle, USA, June 11–13, 2008, pp. 2160–2165 5 Petrović, P.B., Stevanović, M.R.: ‘Algorithm for Fourier coefficient estimation’, IET Signal Process., 2011, 5, (2), pp. 138–149 6 Fattah, S.A., Zhu, W.P., Ahmad, M.O.: ‘Identification of autoregressive moving average systems based on noise compensation in the correlation domain’, IET Signal Process., 2011, 5, (3), pp. 292–305 7 Abderrahim, K., Mathlouthi, H., Msahli, F.: ‘New approaches to finite impulse response systems identification using higher-order statistics’, IET Signal Process., 2010, 4, (5), pp. 488–501 8 Shi, Y., Fang, H.: ‘Kalman filter based identification for systems with randomly missing measurements in a network environment’, Int. J. Control, 2010, 83, (3), pp. 538–551 9 Shi, Y., Yu, B.: ‘Output feedback stabilization of networked control systems with random delays modeled by Markov chains’, IEEE Trans. Autom. Control, 2009, 54, (7), pp. 1668–1674 10 Yan, M., Shi, Y.: ‘Robust discrete-time sliding mode control for uncertain systems with time-varying state delay’, IET Control Theory Appl., 2008, 2, (8), pp. 662–674 11 Fang, H., Wu, J., Shi, Y.: ‘Genetic adaptive state estimation with missing input/output data’, Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., 2010, 224, (5), pp. 611–617 12 Ding, F., Liu, X.P., Liu, G.: ‘Gradient based and least-squares based iterative identification methods for OE and OEMA systems’, Digit. Signal Process., 2010, 20, (3), pp. 664–677 13 Wang, D.Q.: ‘Least squares-based recursive and iterative estimation for output error moving average (OEMA) systems using data filtering’, IET Control Theory Appl., 2011, 5, (14), pp. 1648–1657 14 Wang, D.Q., Chu, Y.Y., Ding, F.: ‘Auxiliary model-based RELS and MI-ELS algorithms for Hammerstein OEMA systems’, Comput. Math. Appl., 2010, 59, (9), pp. 3092–3098 15 Wang, D.Q., Chu, Y.Y., Yang, G.W., Ding, F.: ‘Auxiliary model-based recursive generalized least squares parameter estimation for Hammerstein OEAR systems’, Math. Comput. Model., 2010, 52, (1– 2), pp. 309–317 16 Wang, D.Q., Yang, G.W., Ding, R.F.: ‘Gradient-based iterative parameter estimation for Box–Jenkins systems’, Comput. Math. Appl., 2010, 60, (5), pp. 1200–1208 17 Liu, Y.J., Wang, D.Q., Ding, F.: ‘Least-squares based iterative algorithms for identifying Box–Jenkins models with finite measurement data’, Digit. Signal Process., 2010, 20, (5), pp. 1458–1467 18 Ding, F., Liu, X.P., Liu, G.: ‘Auxiliary model based multi-innovation extended stochastic gradient parameter estimation with colored measurement noises’, Signal Process., 2009, 89, (10), pp. 1883–1890

654 & The Institution of Engineering and Technology 2013

19 Ding, F., Liu, X.P., Liu, G.: ‘Multi-innovation least squares identification for system modelling’, IEEE Trans. Syst. Man Cybern.B, Cybern., 2010, 40, (3), pp. 767–778 20 Liu, Y.J., Xiao, Y.S., Zhao, X.L.: ‘Mlti-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model’, Appl. Math. Comput., 2009, 215, (4), pp. 1477–1483 21 Yao, G.Y., Ding, R.F.: ‘Two-stage least squares based iterative identification algorithm for controlled autoregressive moving average systems’, Comput. Math. Appl., 2012, 63, (5), pp. 975–984 22 Duan, H.H., Jia, J., Ding, R.F.: ‘Two-stage recursive least squares parameter estimation algorithm for output error models’, Math. Comput. Model., 2012, 55, (3–4), pp. 1151–1159 23 Ding, F.: ‘Two-stage iterative estimation algorithm for systems with colored noises using the data filtering’. The 24th Chinese Control and Decision Conf. (2012 CCDC), Taiyuan, China, May 2012, pp. 23–25 24 Ding, F.: ‘Two-stage least squares based iterative estimation algorithm for CARMA system modeling’, Appl. Math. Model., 2013, 37, (7), pp. 4798–4808 25 Ding, R., Duan, H.H.: ‘TS-RLS algorithm for pseudo-linear regressive models’. The 2012 American Control Conf. (ACC), Montréal, Canada, June 27–29, 2012, pp. 2683–2688 26 Ding, F.: ‘Hierarchical multi-innovation stochastic gradient algorithm for Hammerstein nonlinear system modeling’, Appl. Math. Model., 2013, 37, (4), pp. 1694–1704 27 Ding, F., Chen, T.: ‘Combined parameter and output estimation of dual-rate systems using an auxiliary model’, Automatica, 2004, 40, (10), pp. 1739–1748 28 Bai, E.W.: ‘An optimal two-stage identification algorithm for Hammerstein–Wiener nonlinear systems’, Automatica, 1998, 34, (3), pp. 333–338 29 Li, K., Peng, J.X., Bai, E.W.: ‘A two-stage algorithm for identification of nonlinear dynamic systems’, Automatica, 2006, 42, (7), pp. 1189–1197 30 Ding, F.: ‘Decomposition based fast least squares algorithm for output error systems’, Signal Process., 2013, 93, (5), pp. 1235–1242 31 Cao, Z.H.: ‘Rounding error analysis of two-stage iterative methods for large linear systems’, Appl. Math. Comput., 2003, 139, (2–3), pp. 371–381 32 Ding, F., Chen, T.: ‘Performance analysis of multi-innovation gradient type identification methods’, Automatica, 2007, 43, (1), pp. 1–14 33 Ding, F., Ding, J.: ‘Least squares parameter estimation with irregularly missing data’, Int. J. Adapt. Control Signal Process., 2010, 24, (7), pp. 540–553 34 Ding, F., Liu, G., Liu, X.P.: ‘Parameter estimation with scarce measurements’, Automatica, 2011, 47, (8), pp. 1646–1655 35 Goodwin, G.C., Sin, K.S.: ‘Adaptive filtering prediction and control’ (Prentice-Hall, Englewood Cliffs, NJ, 1984) 36 Golub, G.H., Loan, C.F.V.: ‘Matrix computations’ (Johns Hopkins University Press, Baltimore, MD, 1996, 3rd ed.) 37 Ding, F.: ‘Computational efficiency of the identification methods – part A: recursive algorithms’, J. Nanjing Univ. Inf. Sci. Technol. (Nat. Sci. Ed.), 2012, 4, (4), pp. 289–300 38 Ding, J., Ding, F.: ‘Bias compensation based parameter estimation for output error moving average systems’, Int. J. Adapt. Control Signal Process., 2011, 25, (12), pp. 1100–1111 39 Liu, Y.J., Sheng, J., Ding, R.F.: ‘Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems’, Comput. Math. Appl., 2010, 59, (8), pp. 2615–2627 40 Ding, F., Gu, Y.: ‘Performance analysis of the auxiliary model based least squares identification algorithm for one-step state delay systems’, Int. J. Comput. Math., 2012, 89, (15), pp. 2019–2028 41 Ding, F., Liu, Y.J., Bao, B.: ‘Gradient based and least squares based iterative estimation algorithms for multi-input multi-output systems’, Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., 2012, 226, (1), pp. 43–55 42 Ding, F.: ‘Coupled-least-squares identification for multivariable systems’, IET Control Theory Appl., 2013, 7, (1), pp. 68–79 43 Ding, F., Liu, X.G., Chu, J.: ‘Gradient-based and least-squares-based iterative algorithms for Hammerstein systems using the hierarchical identification principle’, IET Control Theory Appl., 2013, doi: 10.1049/iet-cta.2012.0313

IET Signal Process., 2013, Vol. 7, Iss. 8, pp. 646–654 doi: 10.1049/iet-spr.2012.0183