Engineering Computations

Engineering Computations Emerald Article: Least squares and stochastic gradient parameter estimation for multivariable nonlinear Box-Jenkins models based on the auxiliary model and the multi-innovation identification theory Jing Chen, Feng Ding

Article information: To cite this document: Jing Chen, Feng Ding, (2012),"Least squares and stochastic gradient parameter estimation for multivariable nonlinear Box-Jenkins models based on the auxiliary model and the multi-innovation identification theory", Engineering Computations, Vol. 29 Iss: 8 pp. 907 - 921 Permanent link to this document: http://dx.doi.org/10.1108/02644401211271654 Downloaded on: 01-11-2012 References: This document contains references to 58 other documents To copy this document: [email protected]

Access to this document was granted through an Emerald subscription provided by CARLETON UNIVERSITY For Authors: If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service. Information about how to choose which publication to write for and submission guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information. About Emerald www.emeraldinsight.com With over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation. *Related content and download information correct at time of download.

The current issue and full text archive of this journal is available at www.emeraldinsight.com/0264-4401.htm

Least squares and stochastic gradient parameter estimation for multivariable nonlinear Box-Jenkins models based on the auxiliary model and the multi-innovation identification theory

MIMO nonlinear Box-Jenkins models 907 Received 22 February 2011 Revised 5 August 2011 Accepted 19 June 2012

Jing Chen Key Laboratory of Advanced Process Control for Light Industry, Jiangnan University, Wuxi, China and Wuxi Professional College of Science and Technology, Wuxi, China, and

Feng Ding Key Laboratory of Advanced Process Control for Light Industry, Jiangnan University, Wuxi, China and School of Internet of Things Engineering, Jiangnan University, Wuxi, China Abstract Purpose – The purpose of this paper is to study the identification methods for multivariable nonlinear Box-Jenkins systems with autoregressive moving average (ARMA) noises, based on the auxiliary model and the multi-innovation identification theory. Design/methodology/approach – A multi-innovation generalized extended least squares (MI-GELS) and a multi-innovation generalized ex-tended stochastic gradient (MI-GESG) algorithms are developed for multivariable nonlinear Box-Jenkins systems based on the auxiliary model. The basic idea is to construct an auxiliary model from the measured data and to replace the unknown terms in the information vector with their estimates (i.e. the outputs of the auxiliary model). Findings – It is found that the proposed algorithms can give high accurate parameter estimation compared with existing stochastic gradient algorithm and recursive extended least squares algorithm. Originality/value – In this paper, the AM-MI-GESG and AM-MI-GELS algorithms for MIMO Box-Jenkins systems with nonlinear input are presented using the multi-innovation identification theory and the proposed algorithms can improve the parameter estimation accuracy. The paper provides a simulation example. Keywords Programming and algorithm theory, Time series analysis, Recursive identification, Parameter estimation, Stochastic gradient, Auxiliary model, Box-Jenkins systems, Multivariable system Paper type Research paper

This work was supported by the National Natural Science Foundation of China.

Engineering Computations: International Journal for Computer-Aided Engineering and Software Vol. 29 No. 8, 2012 pp. 907-921 q Emerald Group Publishing Limited 0264-4401 DOI 10.1108/02644401211271654

EC 29,8

908

1. Introduction Much work focuses on identification of single-input single-output systems (Ding and Chen, 2005, d; El-Sherief and Sinha, 2002; Prasad and Sinha, 1977; Shi and Fang, 2010; Fang et al., 2010; Shi et al., 2009; Yu et al., 2008; Shi and Chen, 2006). Some earlier system identification methods are based on the state-space models by using the two-stage bootstrap identification technique (Goodwin and Sin, 1984; Overshee and De Moor, 1994; Verhaegen, 1994); others are based on the difference equation models obtained from state-space models (Gauthier and Landau, 1978; El-Sherief and Sinha, 1979; Sinha and Kwong, 1979; El-Sherief, 1981; Ding and Chen, 2005a; Liu et al., 2009b, 2010a; Ding et al., 2009, 2010a; Yin et al., 2011; Xie et al., 2011). The auxiliary model identification idea is an effective tool for identifying systems with unknown variables in the information vector. The key is to replace the unknown variables in the information vector with the outputs of an auxiliary model (Ding and Chen, 2004a, b, 2005c; Wang et al., 2010a, c; Ding and Ding, 2010; Ding et al., 2011a). For example, Ding and Chen presented the auxiliary model based least squares algorithm (Ding and Chen, 2004a), the auxiliary model based stochastic gradient algorithm (Ding and Chen, 2005c), and the auxiliary finite impulse response model based identification algorithm for dual-rate systems (Ding and Chen, 2004b); Wang et al. (2010a, c) proposed an auxiliary model based RELS and multi-innovation extended least squares algorithms for Hammerstein OEMA systems and an auxiliary model based recursive generalized least squares parameter estimation for Hammerstein OEAR systems. Recently, Chen et al. (2010) presented an auxiliary model based multi-innovation extended stochastic gradient (AM-MI-ESG) algorithm for multivariable output error moving average (OEMA) nonlinear systems with moving average (MA) noises, a multi-innovation generalized extended stochastic gradient (MI-GESG) algorithm for multi-input multi-output nonlinear Box-Jenkins systems based on the auxiliary model (Chen and Wang, 2010), and a gradient based iterative estimation algorithm for Hammerstein systems with saturation and dead-zone nonlinearities (Chen et al., 2012). On the basis of the work in Chen et al. (2010) and Chen and Wang (2010), this paper studies the identification methods for multivariable nonlinear Box-Jenkins systems with autoregressive moving average (ARMA) noises, based on the auxiliary model and the multi-innovation identification theory (Ding and Chen, 2007; Han and Ding, 2009a, b; Ding, 2010; Wang and Ding, 2010a; Zhang et al., 2009; Ding et al., 2007a, 2010b; Liu et al., 2009a, 2010b; Xie et al., 2010). This paper is organized as follows. Section 2 discusses the system model and identification model. Sections 3 and 4 derive identification algorithms for multivariable nonlinear Box-Jenkins models based on the auxiliary model and multi-innovation theory. Section 5 provides an illustrative example. Finally, concluding remarks are given in Section 6. 2. System description and identification model Chen et al. (2010) studied multi-innovation gradient identification algorithms for multivariable OEMA nonlinear systems and this paper studies the auxiliary model based multi-innovation least squares identification algorithm for a multivariable nonlinear Box-Jenkins model in Chen and Wang (2010): yðtÞ ¼ A 21 ðzÞBðzÞf ðuðtÞÞ þ C 21 ðzÞDðzÞvðtÞ;

ð1Þ

f ðuðtÞÞ ¼ ½f 1 ðu1 ðtÞÞ; f 2 ðu2 ðtÞÞ; . . . ; f r ður ðtÞÞT [ Rr ; AðzÞ ¼ I þ A 1 z 21 þ A 2 z 22 þ · · · þ A na z 2na ; BðzÞ ¼ B 1 z 21 þ B 2 z 22 þ · · · þ B nb z 2nb ;

MIMO nonlinear Box-Jenkins models

A i [ Rm£m ; B i [ Rm£r ;

CðzÞ ¼ I þ C 1 z 21 þ C 2 z 22 þ · · · þ C nc z 2nc ;

C i [ Rm£m ;

DðzÞ ¼ I þ D 1 z 21 þ D 2 z 22 þ · · · þ D nd z 2nd ;

D i [ Rm£m ;

909

where uðtÞ ¼ ½u1 ðtÞ; u2 ðtÞ; . . . ; ur ðtÞT [ Rr is the system input vector, y (t) [ Rm is the system output vector, v(t) [ Rm a stochastic white noise vector with zero mean, A(z), B(z), C(z) and D(z) are polynomial matrices in the unit backward shift operator ½z 21 yðtÞ ¼ yðt 2 1Þ and the symbol I stands for an identity matrix of appropriate sizes. For convenience, we assume that fi(ui(t)) is a nonlinear function of a known basis (g1, g2, . . . , gl): f i ðui ðtÞÞ ¼ c1 g1 ðui ðtÞÞ þ c2 g2 ðui ðtÞÞ þ · · · þ cl gl ðui ðtÞÞ with the same unknown parameters ci. This can be extended to a general case with: f i ðui ðtÞÞ ¼ ci1 g1 ðui ðtÞÞ þ ci2 g2 ðui ðtÞÞ þ · · · þ cil gl ðui ðtÞÞ: One of the gains of f (u (t)) and B (z) has to be fixed, otherwise none of the identification schemes can distinguish between (f (u (t)), B(z)) and (af (u(t)), a 2 1B (z)). There are several ways to normalize the gains. Here, we adopt the assumption used in Chen et al. (2010), Ding et al. (2006, 2011b) and Wang and Ding (2008, 2011) and let the first coefficient of the function f (· ) equal 1, i.e. c1 ¼ 1. Define the intermediate variables: xðtÞ U A 21 ðzÞBðzÞf ðuðtÞÞ;

ð2Þ

wðtÞ U C 21 ðzÞDðzÞvðtÞ:

ð3Þ

Define the parameter matrix u and the information vector w (t) as: u T U uTs ; uTn [ Rm£n ;

n U mna þ lrnb þ mnd þ mnc ;

uTs U½A 1 ;A 2 ; ...;A na ;B 1 ;B 2 ; ...;B nb ;B 1 c2 ;B 2 c2 ; ...;B nb c2 ; ...;B 1 cl ;B 2 cl ; ...;B nb cl [ Rm£ðmna þlrnb Þ ;

uTn U ½C 1 ; C 2 ; . . . ; C nc ; D 1 ; D 2 ; . . . ; D nd [ Rm£ðmnd þmnc Þ ; T w ðtÞ U w Ts ðtÞ; w Tn ðtÞ [ Rn ;

EC 29,8

910

w s ðtÞ U 2x T ðt 2 1Þ; 2x T ðt 2 2Þ; . . . ; 2x T ðt 2 na Þ; gT1 ðuðt 2 1ÞÞ; gT1 ðuðt 2 2ÞÞ; . . . ; gT1 ðuðt 2 nb ÞÞ;

gT2 ðuðt 2 1ÞÞ; gT2 ðuðt 2 2ÞÞ; . . . ; gT2 ðuðt 2 nb ÞÞ; T [ Rmna þlrnb ; . . . ; gTl ðuðt 2 1ÞÞ; gTl ðuðt 2 2ÞÞ; . . . ; gTl ðuðt 2 nb ÞÞ w n ðtÞ U ½2w T ðt 2 1Þ; 2w T ðt 2 2Þ; . . . ; 2w T ðt 2 nc Þ; [ Rmnd þmnc ;

v T ðt 2 1Þ; v T ðt 2 2Þ; . . . ; v T ðt 2 nd ÞT

g i ðuðtÞÞ U ½gi ðu1 ðtÞÞ; gi ðu2 ðtÞÞ; . . . ; gi ður ðtÞÞT [ Rr : Once the estimate u^s ðtÞ of the parameter matrix u s is obtained, the estimates of B 1, B 2, . . . , Bnb, B 1c2, B2c2, . . . , Bnbc2, . . . , B 1cl, B 2cl,. . . and Bnbcl can be read from the entries of u^s ðtÞ. Then we compute the parameter estimates c^i ðtÞ of the nonlinear block simply by using the average method like in Ding and Chen (2005b) and Ding et al. (2007b). From equations (1) to (3), we have: xðtÞ ¼ uTs w s ðtÞ;

ð4Þ

uTn w n ðtÞ

ð5Þ

wðtÞ ¼ yðtÞ ¼ xðtÞ þ wðtÞ

þ vðtÞ;

¼ uTs w s ðtÞ þ uTn w n ðtÞ þ vðtÞ

¼ u T w ðtÞ þ vðtÞ:

ð6Þ

Equation (6) is the identification model of multivariable nonlinear Box-Jenkins systems. 3. The auxiliary model based identification algorithm The difficulty of identification is that the information vector w (t) in equation (6) contain the unknown terms x(t 2 i ), w (t 2 i ) and v (t 2 i ). The solution is that the unknown x (t 2 i ) in the information vector w (t) are replaced with the outputs ^ 2 i Þ of an auxiliary model, i.e. use the estimate of the transfer function of the xðt linear part: ^ ¼ xðtÞ

^ BðzÞ f ðuðtÞÞ ^ AðzÞ

or

T

^ ¼ u^s ðtÞw^ s ðtÞ; xðtÞ

as the auxiliary model. Of course, there are other ways to choose auxiliary models, e.g. using the finite impulse response model (Ding and Chen, 2004b, 2005c). According to the auxiliary model identification principle: the unknown ^ 2 i Þ of the auxiliary variables x (t 2 i ) in ws(t) are replaced by the output xðt model, v (t 2 i ) and w (t 2 i ) are replaced by the estimated residuals v^ ðt 2 i Þ and ^ ðt 2 i Þ. w 2 Let kX k U tr½XX T . Defining and minimizing the cost function: J ðuÞ U

t X i¼1

2

kyði Þ 2 u T w^ ði Þk ;

it is easy to obtain an auxiliary model based generalized extended least squares (AM-GELS) algorithm and an auxiliary model based generalized extended stochastic gradient (AM-GESG) algorithm. The AM-GELS algorithm is as follows:

u^ðtÞ ¼ u^ðt 2 1Þ þ P ðtÞw^ ðtÞe T ðtÞ;

ð7Þ

eðtÞ ¼ yðtÞ 2 u^T ðt 2 1Þw^ ðtÞ;

ð8Þ

P 21 ðtÞ ¼ P 21 ðt 2 1Þ þ w^ ðtÞw^ T ðtÞ; P ð0Þ ¼ p0 I ;

ð9Þ

h iT w^ ðtÞ ¼ w^ Ts ðtÞ; w^ Tn ðtÞ ;

ð10Þ

w^ s ðtÞ ¼ 2x^ T ðt 2 1Þ; 2x^ T ðt 2 2Þ; . . . ; 2x^ T ðt 2 na Þ; gT1 ðuðt 2 1ÞÞ; gT1 ðuðt 2 2ÞÞ; . . . ; gT1 ðuðt 2 nb ÞÞ;

gT2 ðuðt 2 1ÞÞ; gT2 ðuðt 2 2ÞÞ;

. . . ; gT2 ðuðt 2 nb ÞÞ; . . . ; gTl ðuðt 2 1ÞÞ; gTl ðuðt 2 2ÞÞ; T . . . ; gTl ðuðt 2 nb ÞÞ ; ð11Þ ^ T ðt 2 1Þ; 2w ^ T ðt 2 2Þ; . . . ; 2w ^ T ðt 2 nc Þ; v^ T ðt 2 1Þ; w^ n ðtÞ ¼ ½2w v^ T ðt 2 2Þ; . . . ; v^ T ðt 2 nd ÞT ;

ð12Þ

T ^ ¼ u^s ðtÞw^ s ðtÞ; xðtÞ

ð13Þ

^ ¼ yðtÞ 2 xðtÞ; ^ wðtÞ

ð14Þ

T ^ ¼ wðtÞ ^ 2 u^n ðtÞw^ n ðtÞ: vðtÞ

ð15Þ

The AM-GESG algorithm is as follows (Chen and Wang, 2010):

w^ ðtÞ T e ðtÞ; u^ðtÞ ¼ u^ðt 2 1Þ þ rðtÞ

ð16Þ

eðtÞ ¼ yðtÞ 2 u^T ðt 2 1Þw^ ðtÞ;

ð17Þ

2

rðtÞ ¼ rðt 2 1Þ þ kw^ ðtÞk ; rð0Þ ¼ 1;

ð18Þ

h iT w^ ðtÞ ¼ w^ Ts ðtÞ; w^ Tn ðtÞ ;

ð19Þ

MIMO nonlinear Box-Jenkins models 911

EC 29,8


gT2 ðuðt 2 1ÞÞ; gT2 ðuðt 2 2ÞÞ; . . . ; gT2 ðuðt 2 nb ÞÞ;

T . . . ; gTl ðuðt 2 1ÞÞ; gTl ðuðt 2 2ÞÞ; . . . ; gTl ðuðt 2 nb ÞÞ ; ð20Þ

912 ^ T ðt 2 1Þ; 2w ^ T ðt 2 2Þ; . . . ; 2w ^ T ðt 2 nc Þ; v^ T ðt 2 1Þ; w^ n ðtÞ ¼ ½2w v^ T ðt 2 2Þ; . . . ; v^ T ðt 2 nd ÞT ; T

ð21Þ

^ ¼ u^s ðtÞw^ s ðtÞ; xðtÞ

ð22Þ

^ ^ wðtÞ ¼ yðtÞ 2 xðtÞ;

ð23Þ

T ^ ¼ wðtÞ ^ 2 u^n ðtÞw^ n ðtÞ: vðtÞ

ð24Þ

4. The auxiliary model based multi-innovation algorithms ^ p; tÞ and the stacked Define the innovation vector E( p, t), the information matrix Fð output vector Y ( p, t) as: Eð p; tÞ U ½yðtÞ 2 u^T ðt 2 1Þw^ ðtÞ; yðt 2 1Þ 2 u^T ðt 2 1Þw^ ðt 2 1Þ; . . . ; yðt 2 p þ 1Þ 2 u^T ðt 2 1Þw^ ðt 2 p þ 1Þ [ Rm£p ; ^ p; tÞ U ½w^ ðtÞ; w^ ðt 2 1Þ; . . . ; w^ ðt 2 p þ 1Þ [ Rn£p ; Fð Y ð p; tÞ U ½yðtÞ; yðt 2 1Þ; . . . ; yðt 2 p þ 1Þ [ Rm£p : By expanding the innovation vector e(t) to the innovation matrix E ( p, t), y (t) to ^ p; tÞ like in Ding and Chen (2007), Han and Ding (2009b), Y ( p, t) and w^ ðtÞ to Fð Ding (2010) and Ding et al. (2010b), we can obtain an auxiliary model based multi-innovation generalized extended least squares (AM-MI-GELS) algorithm with the innovation length p for multivariable nonlinear Box-Jenkins systems as follows: ^ p; tÞE T ð p; tÞ; u^ ðtÞ ¼ u^ ðt 2 1Þ þ P ðtÞFð

ð25Þ

^ p; tÞ; Eð p; tÞ ¼ Y ð p; tÞ 2 u^T ðt 2 1ÞFð

ð26Þ

^ FðtÞ ^ T ; P ð0Þ ¼ p0 I ; P 21 ðtÞ ¼ P 21 ðt 2 1Þ þ FðtÞ

ð27Þ

Y ð p; tÞ ¼ ½yðtÞ; yðt 2 1Þ; . . . ; yðt 2 p þ 1Þ; 2 3 ^ s ð p; tÞ F ^ p; tÞ ¼ 4 5 Fð ^ n ð p; tÞ ; F

ð28Þ

^ s ð p; tÞ ¼ ½w^ s ðtÞ; w^ s ðt 2 1Þ; . . . ; w^ s ðt 2 p þ 1Þ; F

ð29Þ ð30Þ

^ n ð p; tÞ ¼ ½w^ n ðtÞ; w^ n ðt 2 1Þ; . . . ; w^ n ðt 2 p þ 1Þ; F h iT w^ ðtÞ ¼ w^ Ts ðtÞ; w^ Tn ðtÞ ;

ð31Þ ð32Þ

MIMO nonlinear Box-Jenkins models


gT2 ðuðt 2 1ÞÞ; gT2 ðuðt 2 2ÞÞ; . . . ; gT2 ðuðt 2 nb ÞÞ; T . . . ; gTl ðuðt 2 1ÞÞ; gTl ðuðt 2 2ÞÞ; . . . ; gTl ðuðt 2 nb ÞÞ ;

913 ð33Þ

^ T ðt 2 1Þ; 2w ^ T ðt 2 2Þ; . . . ; 2w ^ T ðt 2 nc Þ; v^ T ðt 2 1Þ; w^ n ðtÞ ¼ ½2w v^ T ðt 2 2Þ; . . . ; v^ T ðt 2 nd ÞT ;

ð34Þ

T x^ ðtÞ ¼ u^s ðtÞw^ s ðtÞ;

ð35Þ

^ ðtÞ ¼ yðtÞ 2 xðtÞ; ^ w

ð36Þ

^ ðtÞ 2 v^ ðtÞ ¼ w

T u^n ðtÞw^ n ðtÞ:

ð37Þ

Referring to the methods in Ding and Chen (2007) and Chen et al. (2010), E ( p, t) [ Rp£ m is an innovation matrix, namely, multi-innovation, the algorithm in equations (25)-(37) is known as the multi-innovation identification one. As p ¼ 1, the AM-MI-GELS algorithm reduces to the AM-GELS algorithm in equations (7)-(15). The following lists the steps of computing the parameter estimation matrix u^ ðtÞ by the AM-MI-GELS algorithm: ^ ði Þ ¼ 1 m = (1) To initialize, let t ¼ 1, u^ ð0Þ ¼ I =p0 , x a(i ) ¼ 1m/p0, v^ ði Þ ¼ 1 m =p0 , w p0 for i # 0, p0 ¼ 106, and set the innovation length p, where 1m represents an m-dimensional column vector whose entries are all 1. (2) Collect the input-output data u(t) and y (t), and compute gi (u (t)). ^ s ð p; tÞ by equation (30) (3) Form w^ s ðtÞ by equation (33), w^ n ðtÞ . by equation (34), F ^ n ð p; tÞ by equation (31). and F ^ p; tÞ by equation (29) and Y ( p,t) by equation (28). (4) Form Fð (5) Compute P 2 1(t) by equation (27) and E ( p,t) by equation (26). (6) Update u^ ðtÞ by equation (25). ^ ðtÞ by equations (36) and vðtÞ ^ by equation (37). (7) Compute x^ ðtÞ by equation (35), w (8) Increase t by 1 and go to step 2. Similarly, we have an auxiliary model based multi-innovation generalized extended stochastic gradient (AM-MI-GESG) algorithm with the innovation length p for multivariable nonlinear Box-Jenkins systems as follows (Chen and Wang, 2010): ^ p; tÞ Fð E T ð p; tÞ; u^ðtÞ ¼ u^ðt 2 1Þ þ rðtÞ

ð38Þ

EC 29,8

^ p; tÞ; Eð p; tÞ ¼ Y ð p; tÞ 2 u^T ðt 2 1ÞFð

ð39Þ

2

ð40Þ

2

rðtÞ ¼ rðt 2 1Þ þ kw^ s ðtÞk þ kw^ n ðtÞk ;

rð0Þ ¼ 1;

Y ð p; tÞ ¼ ½yðtÞ; yðt 2 1Þ; . . . ; yðt 2 p þ 1Þ; 2 3 ^ s ð p; tÞ F ^ p; tÞ ¼ 4 5 Fð ^ n ð p; tÞ ; F

914

ð41Þ ð42Þ

^ s ð p; tÞ ¼ ½w^ s ðtÞ; w^ s ðt 2 1Þ; . . . ; w^ s ðt 2 p þ 1Þ; F

ð43Þ

^ n ð p; tÞ ¼ ½w^ n ðtÞ; w^ n ðt 2 1Þ; . . . ; w^ n ðt 2 p þ 1Þ; F

ð44Þ T w^ s ðtÞ ¼ 2x^ ðt 2 1Þ; 2x^ T ðt 2 2Þ; . . . ; 2x^ T ðt 2 na Þ; gT1 ðuðt 2 1ÞÞ; gT1 ðuðt 2 2ÞÞ; . . . ; gT1 ðuðt 2 nb ÞÞ;

gT2 ðuðt 2 1ÞÞ; gT2 ðuðt 2 2ÞÞ; . . . ; gT2 ðuðt 2 nb ÞÞ; T . . . ; gTl ðuðt 2 1ÞÞ; gTl ðuðt 2 2ÞÞ; . . . ; gTl ðuðt 2 nb ÞÞ ; ð45Þ T

T

T

T

^ ðt 2 1Þ; 2w ^ ðt 2 2Þ; . . . ; 2w ^ ðt 2 nc Þ; v^ ðt 2 1Þ; w^ n ðtÞ ¼ ½2w

ð46Þ

v^ T ðt 2 2Þ; . . . ; v^ T ðt 2 nd ÞT ; T x^ ðtÞ ¼ u^s ðtÞw^ s ðtÞ;

ð47Þ

^ ðtÞ ¼ yðtÞ 2 x^ ðtÞ; w

ð48Þ

^ ¼w ^ ðtÞ 2 vðtÞ

T u^n ðtÞw^ n ðtÞ:

ð49Þ

As p ¼ 1, AM-MI-GESG ¼ AM-GESG. 5. Example Consider the following two-input two-output nonlinear system: #21 " # " # " y1 ðtÞ 1 þ 0:10z 21 0 1:58z 21 20:06z 21 f ðuðtÞÞ ¼ y2 ðtÞ 20:12z 21 1 2 0:040z 21 0:20z 21 0:18z 21 " þ

1 þ 0:07z 21

0:030z 21

0:01z 21

1 þ 0:018z 21

2 f ðuðtÞ ¼ 4

#21 "

u1 ðt 2 1Þ þ 0:50u21 ðt 2 1Þ u2 ðt 2 1Þ þ 0:40u22 ðt 2 1Þ

1 2 0:010z 21

20:048z 21

0:108z 21

1 þ 0:0423z 21

#"

v1 ðtÞ v2 ðtÞ

# ;

3 5:

In simulation, the inputs {u1(t)} and {u2(t)} are taken as two uncorrelated persistent excitation signal sequences with a zero mean and unit variances, {v1(t)} and {v2(t)} as

two white noise sequences with a zero mean and variances s12 ¼ 0:302 for v1(t) and s22 ¼ 0:302 for v2(t). Applying the AM-MI-GELS algorithm and the AM-MI-GESG algorithms to estimate the parameters of this example system, the parameter estimation errors d U ku^ðtÞ 2 uk=kuk versus t are shown in Figures 1 and 2. From Tables I to IV and Figures 1 to 2, it is clear that the AM-MI-GELS algorithm has faster convergence rates than the AM-MI-GESG algorithm and the large innovation length can generate more accurate parameter estimates.


6. Conclusions An AM-MI-GELS algorithm is developed for multivariable nonlinear Box-Jenkins systems. Many other identification methods can be extended to identify the nonlinear systems with colored noises in this paper (Ding et al., 2008, 2010c, d; Liu et al., 2010; Wang and Ding, 2010; Wang et al., 2010; Xiao et al., 2009; Zhang and Cui, 2011; Chen and Ding, 2011). 0.6 0.5

δ

0.4 0.3 0.2

AM–GELS (AM–MI–GELS, p = 1)

0.1 0

AM–MI–GELS, p = 10 0

500

1,000

1,500 t

2,000

2,500

3,000

Figure 1. The parameter estimation errors d versus t (AM-MI-GELS)

0.6 0.5 0.4 δ

AM–GESG (AM–MI–GESG, p = 1) 0.3 0.2 0.1 AM–MI–GESG, p = 10 0

0

500

1,000

1,500 t

2,000

2,500

3,000

Figure 2. The parameter estimation errors d versus t (AM-MI-GESG)

EC 29,8

916

Table I. The AM-GELS estimates and errors

t a11 a12 b11 b12 c11 c12 f11 f12 d11 d12 a21 a22 b21 b22 c21 c22 f21 f22 d21 d22 d (%)

t

Table II. The AM-MI-GELS estimates and errors with p ¼ 10

a11 a12 b11 b12 c11 c12 f11 f12 d11 d12 a21 a22 b21 b22 c21 c22 f21 f22 d21 d22 d (%)

100

200

500

1,000

2,000

3,000

True values

0.06697 0.09832 1.56954 2 0.11813 0.78648 0.00265 0.34378 0.35945 2 0.08524 2 0.25238 2 0.15185 2 0.13277 0.19123 0.12872 0.04646 0.07675 0.10173 2 0.01608 2 0.10320 2 0.07094 28.08867

0.09164 0.06168 1.59382 20.10499 0.80514 20.02517 0.13887 0.20119 20.24162 20.21884 20.14256 20.12493 0.19874 0.16958 0.03670 0.06957 0.02774 20.08119 20.06431 0.00690 17.10062

0.09465 0.03224 1.57798 20.06647 0.80356 20.03665 0.12373 0.12584 20.27523 20.19901 20.13345 20.07843 0.19126 0.16929 0.07250 0.05994 20.12368 20.16370 0.00267 0.01028 14.87862

0.09176 0.02713 1.58132 2 0.06132 0.79579 2 0.03356 0.02447 0.13142 2 0.19326 2 0.16935 2 0.12531 2 0.07934 0.18793 0.16943 0.08486 0.06208 2 0.04794 2 0.15688 0.07094 0.00697 12.03052

0.09301 0.01208 1.57680 20.06365 0.78954 20.03115 0.10949 0.07943 20.19532 20.09799 20.11856 20.06731 0.20147 0.16878 0.08516 0.07064 20.02049 20.22237 0.10239 0.02650 14.07385

0.09774 2 0.00624 1.57810 2 0.06346 0.79104 2 0.02869 0.07369 0.08447 2 0.18020 2 0.09236 2 0.11533 2 0.05554 0.20075 0.17105 0.08551 0.07598 2 0.04364 2 0.14761 0.08900 0.02330 10.29265

0.10000 0.00000 1.58000 20.06000 0.79000 20.03000 0.07000 0.03000 20.10000 20.04800 20.12000 20.04000 0.20000 0.18000 0.08000 0.07200 0.01000 0.01800 0.10800 0.04230

100

200

500

1,000

2,000

3,000

True values

0.07208 0.08385 1.54659 2 0.12192 0.77694 0.00688 0.08285 0.07705 2 0.06639 2 0.14435 2 0.14457 2 0.05658 0.21040 0.12350 0.05100 0.07994 2 0.04567 0.04943 0.15033 0.04550 9.46216

0.09729 0.02623 1.59172 20.10219 0.80639 20.02487 0.05173 0.10407 20.08416 20.13581 20.13987 20.09792 0.20825 0.16032 0.03781 0.07119 20.01006 20.02391 0.13927 0.10647 8.72406

0.09952 0.04984 1.57551 20.06658 0.80963 20.03678 0.05617 0.11676 20.08804 20.11842 20.12997 20.08310 0.19541 0.16947 0.07394 0.05893 0.03203 0.04058 0.13396 0.04400 6.99839

0.09423 0.04932 1.57868 2 0.06356 0.79989 2 0.03170 0.08840 0.12577 2 0.08426 2 0.11147 2 0.12443 2 0.06332 0.19056 0.16840 0.08667 0.06216 0.03353 0.05559 0.12703 0.04388 7.44355

0.09559 0.02137 1.57569 20.06391 0.79168 20.03043 0.08975 0.06371 20.08969 20.05893 20.11849 20.05907 0.20221 0.16867 0.08573 0.07119 0.01922 0.04060 0.11960 0.05026 3.08935

0.09835 0.00981 1.57688 2 0.06416 0.79340 2 0.02801 0.06745 0.04221 2 0.09295 2 0.05065 2 0.11397 2 0.04916 0.20127 0.17088 0.08643 0.07651 2 0.00298 0.03520 0.11584 0.04227 1.61658

0.10000 0.00000 1.58000 20.06000 0.79000 20.03000 0.07000 0.03000 20.10000 20.04800 20.12000 20.04000 0.20000 0.18000 0.08000 0.07200 0.01000 0.01800 0.10800 0.04230



100

200

500

1,000

2,000

3,000

True values

0.27152 0.06248 0.95509 20.17643 0.52184 0.18201 20.03874 0.02304 0.01718 20.01935 20.01333 20.00079 0.12638 0.05379 0.11813 0.07365 20.00540 0.00318 0.03849 0.00693 42.86557

0.23767 0.05625 1.00716 2 0.17178 0.54892 0.17452 2 0.03857 0.02322 0.03035 2 0.01808 2 0.02784 2 0.00428 0.13422 0.06124 0.12475 0.07643 2 0.00551 0.00296 0.04864 0.00881 39.41236

0.21976 0.05371 1.06858 20.16022 0.57652 0.15865 20.03859 0.02327 0.03265 20.02140 20.04589 20.00603 0.14065 0.07188 0.11780 0.07150 20.00550 0.00305 0.06010 0.00913 35.49619

0.20598 0.05014 1.10855 2 0.15678 0.59835 0.15365 2 0.03861 0.02319 0.03494 2 0.02179 2 0.05828 2 0.00793 0.14380 0.08140 0.11691 0.07209 2 0.00550 0.00303 0.06549 0.00923 33.01418

0.18870 0.04779 1.14733 20.15009 0.61626 0.14480 20.03854 0.02331 0.03768 20.02227 20.07017 20.00986 0.14803 0.09105 0.11497 0.07264 20.00551 0.00299 0.07017 0.00904 30.56428

0.18027 0.04619 1.16846 20.14724 0.62722 0.14222 20.03855 0.02330 0.03782 20.02252 20.07673 20.01077 0.15085 0.09408 0.11506 0.07339 20.00550 0.00300 0.07217 0.00928 29.26294

0.10000 0.00000 1.58000 20.06000 0.79000 20.03000 0.07000 0.03000 20.10000 20.04800 20.12000 20.04000 0.20000 0.18000 0.08000 0.07200 0.01000 0.01800 0.10800 0.04230

100

200

500

1,000

2,000

3,000

True values

0.11506 0.00413 1.42233 20.04846 0.67842 0.04859 0.04258 0.04354 20.12283 20.04559 20.07581 20.01630 0.20466 0.20896 0.14386 0.08090 0.02059 0.00723 0.07577 0.03677 11.97561

0.08074 2 0.00179 1.48293 2 0.05301 0.72736 0.02108 0.04673 0.04025 2 0.07574 2 0.03651 2 0.08689 2 0.02362 0.21267 0.19756 0.13697 0.07573 0.01677 0.01132 0.09597 0.04710 7.82815

0.08566 0.00208 1.53411 20.05471 0.76520 20.00600 0.04791 0.03885 20.08861 20.04069 20.09830 20.02128 0.20407 0.17584 0.10524 0.06151 0.01656 0.01163 0.10936 0.04680 3.81874

0.09445 0.00351 1.54679 2 0.06543 0.77441 2 0.01844 0.04728 0.03956 2 0.09520 2 0.04015 2 0.10547 2 0.02368 0.19720 0.17895 0.10480 0.06261 0.01586 0.01257 0.11268 0.04446 2.61222

0.09173 0.00538 1.56063 20.07123 0.77877 20.02167 0.04902 0.03754 20.08734 20.04049 20.11117 20.02632 0.19494 0.18433 0.09522 0.06524 0.01496 0.01362 0.11626 0.04268 1.96700

0.09330 0.00644 1.56185 20.07233 0.77513 20.02512 0.04934 0.03711 20.08997 20.04046 20.11468 20.02658 0.19861 0.17831 0.09471 0.06637 0.01462 0.01409 0.11564 0.04275 1.91442

0.10000 0.00000 1.58000 20.06000 0.79000 20.03000 0.07000 0.03000 20.10000 20.04800 20.12000 20.04000 0.20000 0.18000 0.08000 0.07200 0.01000 0.01800 0.10800 0.04230


Table III. The AM-GESG estimates and errors

Table IV. The AM-MI-GESG estimates and errors with p ¼ 10

EC 29,8

918

References Chen, J. and Ding, F. (2011), “Modified stochastic gradient algorithms with fast convergence rates”, Journal of Vibration and Control, Vol. 17 No. 9, pp. 1281-6. Chen, J. and Wang, X.P. (2010), “Multi-innovation generalized extended stochastic gradient algorithm for multi-input multi-output nonlinear Box-Jenkins systems based on the auxiliary model”, Lecture Notes in Computer Science, Vol. 6238, pp. 136-46 (also see Proceeding of the International Conference on Life System Modeling and Simulation & 2010 International Conference on Intelligent Computing for Sustainable Energy and Environment, Wuxi, China, September 17-20, 2010, pp. 136-46.). Chen, J., Wang, X.P. and Ding, R.F. (2012), “Gradient based estimation algorithm for Hammerstein systems with saturation and dead-zone nonlinearities”, Applied Mathematical Modelling, Vol. 36 No. 1, pp. 238-43. Chen, J., Zhang, Y. and Ding, R.F. (2010), “Auxiliary model based multi-innovation algorithms for multivariable nonlinear systems”, Mathematical and Computer Modelling, Vol. 52 Nos 9/10, pp. 1428-34. Ding, F. (2010), “Several multi-innovation identification methods”, Digital Signal Processing, Vol. 20 No. 4, pp. 1027-39. Ding, F. and Chen, T. (2004a), “Combined parameter and output estimation of dual-rate systems using an auxiliary model”, Automatica, Vol. 40 No. 10, pp. 1739-48. Ding, F. and Chen, T. (2004b), “Identification of dual-rate systems based on finite impulse response models”, International Journal of Adaptive Control and Signal Processing, Vol. 18 No. 7, pp. 589-98. Ding, F. and Chen, T. (2005a), “Hierarchical identification of lifted state-space models for general dual-rate systems”, IEEE Transactions on Circuits and Systems – I: Regular Papers, Vol. 52 No. 6, pp. 1179-87. Ding, F. and Chen, T. (2005b), “Identification of Hammerstein nonlinear ARMAX systems”, Automatica, Vol. 41 No. 9, pp. 1479-89. Ding, F. and Chen, T. (2005c), “Parameter estimation of dual-rate stochastic systems by using an output error method”, IEEE Transactions on Automatic Control, Vol. 50 No. 9, pp. 1436-41. Ding, F. and Chen, T. (2005d), “Performance bounds of the forgetting factor least squares algorithm for time-varying systems with finite measurement data”, IEEE Transactions on Circuits and Systems – I: Regular Papers, Vol. 52 No. 3, pp. 555-66. Ding, F. and Chen, T. (2007), “Performance analysis of multi-innovation gradient type identification methods”, Automatica, Vol. 43 No. 1, pp. 1-14. Ding, F. and Ding, J. (2010), “Least squares parameter estimation with irregularly missing data”, International Journal of Adaptive Control and Signal Processing, Vol. 24 No. 7, pp. 540-53. Ding, F., Chen, H.B. and Li, M. (2007a), “Multi-innovation least squares identification methods based on the auxiliary model for MISO systems”, Applied Mathematics and Computation, Vol. 187 No. 2, pp. 658-68. Ding, F., Liu, G. and Liu, X.P. (2010a), “Partially coupled stochastic gradient identification methods for non-uniformly sampled systems”, IEEE Transactions on Automatic Control, Vol. 55 No. 8, pp. 1976-81. Ding, F., Liu, G. and Liu, X.P. (2011a), “Parameter estimation with scarce measurements”, Automatica, Vol. 47 No. 8, pp. 1646-55. Ding, F., Liu, P.X. and Liu, G. (2010b), “Gradient based and least-squares based iterative identification methods for OE and OEMA systems”, Digital Signal Processing, Vol. 20 No. 3, pp. 664-77.

Ding, F., Liu, P.X. and Liu, G. (2010c), “Multi-innovation least squares identification for system modeling”, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 40 No. 3, pp. 767-78. Ding, F., Liu, P.X. and Liu, G. (2011b), “Identification methods for Hammerstein nonlinear systems”, Digital Signal Processing, Vol. 21 No. 2, pp. 215-38. Ding, F., Liu, P.X. and Yang, H.Z. (2008), “Parameter identification and intersample output estimation for dual-rate systems”, IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, Vol. 38 No. 4, pp. 966-75. Ding, F., Qiu, L. and Chen, T. (2009), “Reconstruction of continuous-time systems from their non-uniformly sampled discrete-time systems”, Automatica, Vol. 45 No. 2, pp. 324-32. Ding, F., Shi, Y. and Chen, T. (2006), “Gradient based identification algorithms for nonlinear Hammerstein ARMAX models”, Nonlinear Dynamics, Vol. 45 Nos 1/2, pp. 31-43. Ding, F., Shi, Y. and Chen, T. (2007b), “Auxiliary model based least-squares identification methods for Hammerstein output-error systems”, Systems & Control Letters, Vol. 56 No. 5, pp. 373-80. Ding, J., Shi, Y., Wang, H. and Ding, F. (2010d), “A modified stochastic gradient based parameter estimation algorithm for dual-rate sampled-data systems”, Digital Signal Processing, Vol. 20 No. 4, pp. 1238-47. El-Sherief, H. (1981), “Parameters identification of a state-space model of multivariable systems using the extended leastsquares method”, IEEE Transactions on Systems, Man and Cybernetics, Vol. 11 No. 3, pp. 223-7. El-Sherief, H. and Sinha, N.K. (2002), “Bootstrap estimation of parameters and states of linear multivariable systems”, IEEE Transactions on Automatic Control, Vol. 24 No. 2, pp. 340-3. El-Sherief, H. and Sinha, N.K. (1979), “Choices of models for the identification of linear multivariable discrete-time systems”, IEEE Proceedings, Part D, Control Theory and Applications, Vol. 126 No. 12, pp. 1326-30. Fang, H., Wu, J. and Shi, Y. (2010), “Genetic adaptive state estimation with missing input/output data”, Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, Vol. 224 No. 5, pp. 611-17. Gauthier, A. and Landau, I.D. (1978), “On the recursive identification of multi-input multi-output systems”, Automatica, Vol. 14 No. 6, pp. 609-14. Goodwin, G.C. and Sin, K.S. (1984), Adaptive Filtering Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ. Han, L.L. and Ding, F. (2009a), “Identification for multirate multi-input systems using the multi-innovation identification theory”, Computers & Mathematics with Applications, Vol. 57 No. 9, pp. 1438-49. Han, L.L. and Ding, F. (2009b), “Multi-innovation stochastic gradient algorithms for multi-input multi-output systems”, Digital Signal Processing, Vol. 19 No. 4, pp. 545-54. Liu, Y.J., Sheng, J. and Ding, R.F. (2010a), “Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems”, Computers & Mathematics with Applications, Vol. 59 No. 8, pp. 2615-27. Liu, Y.J., Xiao, Y.S. and Zhao, X.L. (2009a), “Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model”, Applied Mathematics and Computation, Vol. 215 No. 4, pp. 1477-83.


EC 29,8

920

Liu, Y.J., Xie, L. and Ding, F. (2009b), “An auxiliary model based recursive least squares parameter estimation algorithm for non-uniformly sampled multirate systems”, Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, Vol. 223 No. 4, pp. 445-54. Liu, Y.J., Yu, L. and Ding, F. (2010b), “Multi-innovation extended stochastic gradient algorithm and its performance analysis”, Circuits, Systems and Signal Processing, Vol. 29 No. 4, pp. 649-67. Liu, Y.J., Wang, D.Q. and Ding, F. (2010c), “Least-squares based iterative algorithms for identifying Box-Jenkins models with finite measurement data”, Digital Signal Processing, Vol. 20 No. 5, pp. 1458-67. Overshee, P.V. and De Moor, B. (1994), “N4SID: subspace algorithm for the identification of combined deterministic-stochastic systems”, Automatica, Vol. 30 No. 1, pp. 75-93. Prasad, R.M. and Sinha, A.K. (1977), “On bootstrap identification using stochastic approximation”, IEEE Transactions on Automatic Control, Vol. 22 No. 4, pp. 671-2. Shi, Y. and Chen, T. (2006), “2-norm based recursive design of transmultiplexers with designable filter length”, Circuits, Systems, and Signal Processing, Vol. 25 No. 4, pp. 447-62. Shi, Y. and Fang, H. (2010), “Kalman filter based identification for systems with randomly missing measurements in a network environment”, International Journal of Control, Vol. 83 No. 3, pp. 538-51. Shi, Y., Fang, H. and Yan, M. (2009), “Kalman filter based adaptive control for networked systems with unknown parameters and randomly missing outputs”, International Journal of Robust and Nonlinear Control, Special Issue on Control with Limited Information (Part II ), Vol. 19 No. 18, pp. 1976-92. Sinha, N.K. and Kwong, Y.H. (1979), “Recursive estimation of the parameters of linear multivariable systems”, Automatica, Vol. 15 No. 4, pp. 471-5. Verhaegen, M. (1994), “Identification of the deterministic part of MIMO state space models given in innovations form from input-output data”, Automatica, Vol. 30 No. 1, pp. 61-71. Wang, D.Q. and Ding, F. (2008), “Extended stochastic gradient identification algorithms for Hammerstein-Wiener ARMAX systems”, Computers & Mathematics with Applications, Vol. 56 No. 12, pp. 3157-64. Wang, D.Q. and Ding, F. (2010a), “Input-output data filtering based recursive least squares identification for CARARMA systems”, Digital Signal Processing, Vol. 20 No. 4, pp. 991-9. Wang, D.Q. and Ding, F. (2010b), “Performance analysis of the auxiliary models based multi-innovation stochastic gradient estimation algorithm for output error systems”, Digital Signal Processing, Vol. 20 No. 3, pp. 750-62. Wang, D.Q. and Ding, F. (2011), “Least squares based and gradient based iterative identification for Wiener nonlinear systems”, Signal Processing, Vol. 91 No. 5, pp. 1182-9. Wang, D.Q., Chu, Y.Y. and Ding, F. (2010a), “Auxiliary model-based RELS and MI-ELS algorithms for Hammerstein OEMA systems”, Computers & Mathematics with Applications, Vol. 59 No. 9, pp. 3092-8. Wang, D.Q., Yang, G.W. and Ding, R.F. (2010b), “Gradient-based iterative parameter estimation for Box-Jenkins systems”, Computers & Mathematics with Applications, Vol. 60 No. 5, pp. 1200-8. Wang, D.Q., Chu, Y.Y., Yang, G. and Ding, F. (2010c), “Auxiliary model-based recursive generalized least squares parameter estimation for Hammerstein OEAR systems”, Mathematical and Computer Modelling, Vol. 52 Nos 1/2, pp. 309-17.

Xiao, Y.S., Zhang, Y., Ding, J. and Dai, J.Y. (2009), “The residual based interactive least squares algorithms and simulation studies”, Computers & Mathematics with Applications, Vol. 58 No. 6, pp. 1190-7. Xie, L., Yang, H.Z. and Ding, F. (2011), “Recursive least squares parameter estimation for non-uniformly sampled systems based on the data filtering”, Mathematical and Computer Modelling, Vol. 54 Nos 1/2, pp. 315-24. Xie, L., Liu, Y.J., Yang, H.Z. and Ding, F. (2010), “Modeling and identification for non-uniformly periodically sampled-data systems”, IET Control Theory & Applications, Vol. 4 No. 5, pp. 784-94. Yin, H.H., Zhu, Z.F. and Ding, F. (2011), “Model order determination using the Hankel matrix of impulse responses”, Applied Mathematics Letters, Vol. 24 No. 5, pp. 797-802. Yu, B., Shi, Y. and Huang, H. (2008), “l2 2 l1 filtering for multirate systems using lifted models”, Circuits, Systems, and Signal Processing, Vol. 27 No. 5, pp. 699-711. Zhang, J.B., Ding, F. and Shi, Y. (2009), “Self-tuning control based on multi-innovation stochastic gradient parameter estimation”, Systems & Control Letters, Vol. 58 No. 1, pp. 69-75. Zhang, Y. and Cui, G.M. (2011), “Bias compensation methods for stochastic systems with colored noise”, Applied Mathematical Modelling, Vol. 35 No. 4, pp. 1709-16. Corresponding author Feng Ding can be contacted at: [email protected]

To purchase reprints of this article please e-mail: [email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints
