robust adaptive identification of nonlinear system ... - Semantic Scholar

156

Asian Journal of Control, Vol. 3, No. 2, pp. 156-162, June 2001

ROBUST ADAPTIVE IDENTIFICATION OF NONLINEAR SYSTEM USING NEURAL NETWORK Q. Song, W.J.Hu and Y.C. Soh ABSTRACT It is well known that disturbance can cause divergence of neural networks in the identification of nonlinear systems. Sufficient conditions using so-called modified algorithms are available to provide guaranteed convergence for adaptive system. They are dead zone scheme, adaptive law modification, and σ-modification. These schemes normally require knowledge of the upper bound of the disturbance. In this paper, a robust weighttuning algorithm is used to train the multi-layered neural network with an adaptive dead zone scheme. The proposed robust adaptive algorithm does not require knowledge of either the upper bound of the disturbance or the bound on the norm of the estimate parameter. A complete convergence proof is provided based on Lyapunov theorem to deal with the nonlinear system. Simulation results are presented to show good perfor-mance of the algorithm. KeyWord: Neural networks, adaptive deadzone, nonlinear function identification.

I. INTRODUCTION Recently, there has seen increasing interesting in the field of using neural networks for the identification and control of nonlinear systems. Adaptive and robust identification algorithms are particularly interesting being considered in the area of system identification and signal processing to cope with time variation of disturbance and system parameters [1-4]. Some theoretical results have been applied to convergence and stability analysis of neural networks for identification of nonlinear systems [57]. A major problem in the identification of nonlinear system is to deal with disturbance. It has been shown that unmodelled dynamics or even a small disturbance can cause most of the identification algorithms to become divergent [8]. Therefore, robustness of the identification algorithm has been an important topic in adaptive filter and control community for the past decade [4-8]. Most of the results focus on the modification of adaptive algorithms by eliminating the pure integral action of the adaptive laws to guarantee the boundedness of all signals in the adaptive loop [8]. One important method is called dead zone scheme, which simply turns off the adaptive algorithm when the identification error is smaller than a certain threshold [4]. This method has been extended to Manuscript received September 13, 2000; revised February 6, 2001; accepted March 29, 2000. The authors are with School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 639798.

design robust adaptive algorithm of neural networks recently to overcome the disturbance problem because the neural network itself is sensitive to disturbance [9-10]. However, a fixed dead zone scheme needs the selection of an appropriate size of the dead zone, which requires the bound of the disturbance must be known a priori [11]. An over-sized dead zone would lead to a large identification error, while, the adaptive system may become divergent if the range of the dead zone is selected too small [9] (also refer to Remark 2 in section 2 and simulation results in section 4 of this paper). In this paper, a robust adaptive dead zone scheme is proposed for the identification of nonlinear system using the multi-layered neural network. There are two major advantages of the new adaptive algorithm over the fixed dead zone scheme. Firstly, the selection of the dead zone does not require knowledge of the upper bound of the disturbance, which can include the reconstruction error of the neural network. Furthermore, the identification algorithm is particularly suitable for the adaptive system in term that the size of the dead zone is estimated on line. Any change of the system parameters or disturbance will not degrade the system performance because the size of the dead zone is updated by the adaptive algorithm accordingly.

II. MODELING OF THE NONLINEAR SYSTEM AND ROBUST ADAPTIVE ALGORITHM OF THE OUTPUT LAYER Consider a class of nonlinear discrete-time system, the output y(k) ∈ Rm is given by:

Q. Song et al.: Robust Adaptive Identification of Nonlinear System Using Neural Network

y(k) = f(xm(k)) + d(k)

(1)

157

where wm* ∈ Rm1xm2 is also a bounded desired weight matrix of the hidden layer as

with the input vector m* m* w m* = [w m* 1 , w 2 , …, w m 1] , T

x m(k) = [x m1 (k) x m2 (k) …… x mm 2(k)]

T

with w *j (k) ∈ Rm2

= [y(k – 1) y(k – 2) … y(k – p)

u(k – d) u(k – d – 10 … u(k – d – q)] . T

m

y m(k) = v mT(k)H (w m(k), x m(k)),

(3)

where vm(k) = [v m1 (k), v m2 (k), …, v mm 1 (k)]T ∈ Rm1xm is the weight matrix of the output layer of the neural network, m1 is the number of neurons in the hidden layer of the network, and H(wm(k), xm(k)) ∈ Rm1 is the nonlinear activation function vector m

m

m

w *j (k) = [w *j1(k) w *j2(k) … w *j m 2(k)] (j = 1, 2, …, m 1). (9) T

(2)

where m2 = p + q + 1 is the number of input neurons in the network, f(xm(k)) ∈ Rm is a nonlinear function, d(k) ∈ Rm denotes a disturbance vector, p, q, and d are known integers that determine the system order and the time delay. Note that we use the notation . for the Euclidean norm of vector and the Frobenius norm of matrix throughout this paper [10]. To simplify our analysis, a three-layered neural network is used for identification of the nonlinear system in this paper, which could also be extended to other multilayered cases. The output of the neural network may be presented as

H(w m(k), x m(k)) = [h 1 (k) h 2 (m) … h m 1(k)]

(8)

Such that f(xm(k)) can be presented by the neural network as

f(x m(k)) = y *(k) + ε(k) = v m* T H(w m*, x m(k)) + ε(k), (10) where ε(k) ∈ Rm is a reconstruction and measurement error vector. Remark 1. The reconstruction error ε(k) is usually assumed to be a disturbance with a known bound according to the well-known universally approximate property of neural networks. However, the bonds of the reconstruction error, and even the disturbance d(k), may be difficult to decide in the adaptive system design. Therefore, in this paper we do not use knowledge of the upper bound in the robust adaptive algorithm. The neural network estimate error e(k) ∈ Rm is e(k) = y(k) – y m(k) = y *(k) – y m(k) + ε(k) + d(k)

T

mT m m mT m = [h(w mT 1 (k)x (k)) h(w 2 (k)x (k)) … h(w m 1(k)x (k))] , (4)

= v mT(k)H(w m(k), x m(k)) + v m* TH (k) + ε(k) + d(k) (11)

T

The adjustable weight matrix wm(k) ∈ Rm1xm2 of the hidden layer

m

with H (k) ∈ Rm1 m

m

w m(k) = [w m1 (k), w m2 (k), …, w mm 1(k)] , T

(5)

where wj(k) = [wj1(k) wj2(k) … wjm2(k)]T ∈ Rm2 with j = 1, 2, … m1, wji(k) is the adjustable weight from the ith neuron of the input layer to the jth neuron of the hidden layer, and h(.) is the nonlinear activation function

h(x) =

1 1 + e – 4λx

(6)

where λ > 0 is the so called gain parameter of the threshold function. Define a bounded ideal weight vm* ∈ R m1xm in the output layer of the neural network, we have the desired output of the network

y *(k) = v m*T(k)H(w m*, x m(k)),

(7)

H (k) = H(w m*, x m(k))– H(w m(k),x m(k)),

(12)

and v m (k) = vm* – vm(k) ∈ Rm1xm is the weight error matrix of the output layer. To achieve global convergence we can define an adaptive dead zone function as

g(e(k), ∆mv (k)) =

e(k) – ∆mv (k)

if

e(k) > ∆mv (k)

0

if

e(k) < ∆mv (k), (13)

where ∆mv is the adaptive dead zone parameter to be estimated. We define the cost function as E(k) = eT(k)e(k)/2.

(14)

Asian Journal of Control, Vol. 3, No. 2, June 2001

158

To update the weight a normalized gradient descent algorithm can be written in the matrix form

v m(k + 1) = v m(k) –

∂E(k) α(k) m ∂v (k) 1 + H(w m(k), x m(k))

= v m(k) +

α(k)H(w m(k), x m(k)) e T(k) 1 + H(w m(k), x m(k))

α(k) e(t)

∆mv (k + 1) = ∆mv (k) +

m

2

mT m = tr{v mT(k)v m(k)}+ ∆v (k)∆v (k).

Then combine equations (15), (16) and (23) we have (15)

,

( – 2α(k)( e(k) – e(k) τ mv (k) )) 2

V (k + 1) – V (k) ≤ m v

m v

1 + H(w m(k),x m(k))

H(w n (k), x m(k)) e T(k)α(k)

(16)

+

1 + H(w (k), x (k))

+

2 2

α(k)e(k)

+

2 2 2

≤

0

if

e(k) > ∆mv (k)

if

e(k) ≤ ∆ (k). m v

Theorem 1. The weight-tuning algorithm in equation (15) and (16) using the dead zone scheme has the following properties

=–

∆mv (k + 1) ≤ ∆mv (k)

(19)

Lim sup e(k) ≤ ∆mv .

(20)

k →∞

where ∆mv is a positive constant, and ∆v (k) = ∆mv – ∆mv (k).

2

2 2

≤

(18)

2

(1 + H(w m(k), x m(k)) )

Note the initial value of ∆mv (k) is selected as a positive number in this paper to make sure that ∆mv (k) is a positive variable.

v m(k + 1) ≤ v m(k) for k = 0, 1, …,

1 + H(w m(k), x m(k))

2

+

2

2α(k) e(k) (∆mv (k) – e(k) )

α(k)e(k) (1 + H(w m(k), x m(k))

(17)

for k = 0, 1, …,

( – 2α(k) e(k) (∆mv – ∆mv (k))) 1 + H(w m(k), x m(k))

(1 + H(w m(k), x m(k)) )

g(e(k), ∆mv (k)) / e(k)

2

2

(1 + H(w m(k), x m(k)) )

where α(k) is a variable learning rate based on the dead zone scheme defined as

α(k) =

(23)

2

, 2

m

2

2

m m V v (k) = v m(k) + ∆v (k)

– 2g mv (e(k), ∆mv (k))( e(k) – ∆mv (k)) 1 + H(w m(k), x m(k))

2

∆m g m2 v (e(k), v (k))

+

1 + H(w m(k), x m(k))

∆m g m2 v (e(k), v (k)) 2

(24)

(1 + H(w m(k), x m(k)) ) Note that the first inequality above is from equation (21), the second inequality is from equation (22), and the last inequality is from the definition in equation (13). m Then V v (k) defined in equation (23) is bounded m blow by zero and is not increasing. The limit of V v (k) exists and the last item in equation (24) will go to zero. Since H(wm(k), xm(k)) is a bounded signal, we have

m

Lim sup e(k) = Lim sup ∆mv (k) ≤ ∆mv . k →∞

k →∞

(25)

Proof. From Equation (11) we have This completes the proof.

e(k) = e (k)v (k)H(w (k), x (k)) + e (k)τ (k) 2

T

mT

m

m

T

m v

≤ tr{e T(k)v mT(k)H(w m(k), x m(k))}+ e(k) τ mv (k) , (21) with

τ mv (k) = v m* TH m(k) + d(k) + ε(k) ≤ ∆mv . Define a Lyapunov function

(22)

Remark 2. From the second inequality, the Lyapunov convergence condition can be re-written as:

α(k)( e(k) – ∆mv (k) ) ≥ α(k) e(k) This is not generally true for a fixed learning rate α(k). Therefore, a variable learning rate like the one defined in equation (17) is used in this paper to provide the guaranteed convergence. Please refer to [9] for further details.

2

Q. Song et al.: Robust Adaptive Identification of Nonlinear System Using Neural Network

Proof. Note that the activation function h(.) in Equation (6) is non-decreasing function. There exists a unique positive number µ(.) with λ ≥ µ(.) ≥ 0 so that the system estimation error e(k) in equation (11) can be rewritten as the following

Remark 3. A major advantage of the robust adaptive dead zone algorithm, compared to the fixed dead zone scheme in [9-10], is that knowledge of the upper bound ∆mv is not used in the design of the adaptive system. Furthermore, since the adaptive dead zone parameter ∆mv (k) is converged to ∆mv , the estimate error e(k) is approximating to a minimum value in the presence of disturbance.

e(k) = f(x m(k)) – y m(k) + ε(k) + d(k)

III. ROBUST TRAINING ALGORITHM FOR THE HIDDEN LAYER

= v mT(k)(H(w m*, x m(k))) – H(w m(k), x m(k))) + e(k) m mT m = v mT(k)[h(w m*T 1 x (k)) – h(w 1 (k)x (k))

To update wm(k) a robust back-propagation algorithm can be written in the matrix form

m mT m … h(w m*T m 1 x (k)) – h(w m 1(k)x (k))] + e(k) T

∂E(k) β(k) m m′ ∂w (k) 1 + ( H (k)v m(k) x m(k) )2

w m(k + 1) = w m(k) –

159

m m m*T m = v mT(k)[µ(w m* – w mT 1 , w 1 (k), x (k))(w 1 1 (k))x (k)

m m m*T mT m … µ(w m* m 1, w m 1(k), x (k))(w m 1 – w m 1(k))x (k)] + e(k) T

β(k)H (k)v (k)e(k)x (k) m′

= w m(k) +

m

mT

m′

1 + ( H (k)v m(k) x m(k) )

∆mw (k + 1) = ∆mw (k) +

2

β(k) e(t) m′

1 + ( H (k)v m(k) x m(k) )

2

,

(26)

= v mT(k)µ(k)w m(k)x m(k) + e(k)

where w m (k) = wm* – wm(k) ∈ Rm1xm2 is the weight error matrix, and µ(k) ∈ Rm1xm1 is a unique positive matrix

(27)

µ(k) = diag [µm1 (k), …, µmm 1(k)]

where β(k) is a variable learning rate based on the dead zone scheme defined as

β(k) =

m m m µ m* m = diag [µ(w m* 1 , w 1 (k)x (k), …, (w m 1, w m 1(k), x (k))]. (31)

Then from equation (30) we have

m′

g(e(k), ∆mw (k)) / e(k)

if

0

if

h min ∆m λ e(k) > w (k) m′ h min ≤ ∆mw (k) λ e(k) (28)

e T(k)e(k) = e T(k)v mT(k)µ(k)w m(k)x m(k) + e T(k)e(k) = [µ1(k)e T(k)v m1 (k)x mT(k)w m1 (k) + …

+ µm 1(k)e T(k)v mm 1(k)x mT(k)w mm 1(k)] + e T(k)e(k)

with

H (k) = diag [h 1 (k), …, h m 1(k)] m′

m′

′

m′

=[ ′

= diag [h (w m1 (k), x m(k)), …, h (w mm 1(k), x m(k))] ∈ R

m 1xm 1

m′ i

where h (k) is the first derivative of the threshold function m′ m′ m′ in equation (6), and h min = min[h i (k), …, h m 1(k)] > 0 is a positive constant. Lemma 1. We have

e(k)

(30)

2

≤ λ′ tr{H m′(k)v m(k)e(k)x mT(k)w mT(k)}+ e T(k)e(k), h min (29)

where e(k) = v mT (k)H(wm*, xm(k)) + ε(k) + d (k) ≤ ∆mw λ′ is an equivalent disturbance signal. h min

µ1(k) m′ T h 1 (k)e (k)v m1 (k)x mT(k)w m1 (k) + … m′ h 1 (k)

,

+

µm 1(k) m′

h m 1(k)

m′

h m 1(k)e T(k)v mm 1(k)x mT(k)w mm 1(k)]

+ e T(k)e(k)

(32)

where vmT(k) = [v m1 (k), …, v mm 1 (k)] ∈ Rmxm1 with v mi (k) ∈ Rm. Rewrite the definition in (3) as y m(k) = [y m1(k), …, y mm(k)]T ∈ Rm, and use the differential rule for vector, we have

∂y m (k) ∂y m(k) ∂e(k) – ∂E(k) o = – e T(k) mT = e T(k) mT = e T(k) mT ∂w mij (k) ∂w j (k) ∂w j (k) ∂w j (k)

mxm 2

Asian Journal of Control, Vol. 3, No. 2, June 2001

160

v m1j(k)h j (k)x m1 (k)

m′

v m1j(k)h j (k)x mm 2(k)

m′

m′

v mmj(k)h j (k)x mm 2(k)

= e T(k) m′

v mmj(k)h j (k)x m1 (k) m′

= h j (k)e T(k)v mj (k)x mT(k)

where

∂y m (k) o ∂w ji(k)

is Jacobian matrix with 1 ≤ i ≤ m2, 1 mxm 2

≤ j ≤ m1, and 1 ≤ o ≤ m. Replace h j (k)eT(k)v mj (k)xmT(k) in m′

the last item of (32) with

– ∂E(k) , we have ∂w mT j (k)

µ1(k) – ∂E(k) m ( )w 1 (k) + … m′ h 1 (k) ∂w mT 1 (k)

w m(k) ≤ w m(k + 1) for k = 0, 1, …,

(33)

µm (k) – ∂E(k) m + ′1 ( T )w m 1(k)]+ e T(k)e(k) h m 1(k) ∂w m 1(k)

∆mw (k) ≤ ∆mw (k + 1) for k = 0, 1, …,

(34)

e T(k)e(k) = [

∂E(k) m ≤ λm′ [( – mT )w 1 (k) + … h min ∂w 1 (k)

+(

– ∂E(k) m )w m 1(k)] + e T(k)e(k) ∂w Tm (k) 1

Lim sup e(k) ≤ ∆mw .

m* where w m (k) = wm* – wm(k), and ∆w (k) = ∆m* w – ∆w (k). m

Proof. Define a Lyapunov function 2

m m V w (k) = w m(k) + ∆w (k)

m′ = λm′ tr{H (k)v m(k)e(k)x mT(k)w mT(k)}+ e T(k)e(k) h min

– ∂E(k) ) ∂w mT i (k) w mi (k)) ≥ 0 with i = 1, 2, …, m1 around a local minimum point. This completes the proof. ■ Note that the inequality above is due to the fact (

2

mT m = tr{w mT(k)w m(k)}+ ∆w (k)∆w (k)

m

m

According to Lemma 1 we have the similar proof procedure as in Theorem 1, i.e. m

V w (k + 1) – V w (k)

≤

– 2β(k)h min λ

2

m′

+(

( e(k) – e(k) e(k) ) m′

1 + ( H (k)v m(k) x m(k) )

β(k)e(k) H m′(k)v m(k) x m(k) m′

m

m

1 + ( H (k)v (k) x (k) ) – ∂E(k) ( mT )w mi (k)) ≥ 0 respect to a local minimum ∂w i (k) m* point w m* i (without loss of generality w i is assumed to be a scalar). Theorem 2. The robust training algorithm in equation (26) and (27) using the adaptive dead zone scheme has the following properties

(36)

where ∆w (k) = ∆mw – ∆w (k).

m

Remark 4. The nonlinear gradient descent algorithm in (26) will search for the edge of an attractor basin randomly as shown in the figure below [12]. The proposed adaptive dead zone scheme can provide guaranteed convergence as long as a local minimum point is concerned. For further improvement, the higher order gradient algorithm can be used as shown in [13], which may provide a global convergence point and is an interesting topic for future research.

(35)

k →∞

+

– 2β(k) e(k) (∆mw – ∆mw (k)) m′

1 + ( H (k)v m(k) x m(k) )

+

2

β(k)e(k) m′

2

2

2 2

(1 + ( H (k)v m(k) x m(k) ) )

)

2

2

Q. Song et al.: Robust Adaptive Identification of Nonlinear System Using Neural Network

2

m m m′ h min g w (k)( e(k) – ∆w (k)λ / h min) ) λ m′ 2 1 + ( H (k)v m(k) x m(k) )

y(k)

m′

≤ ( – 2)(

161

0

m2 w

g (k) m′

1 + ( H (k)v m(k) x m(k) )

=–

2

-2

g m2 w (k) m′

m

m

1 + ( H (k)v (k) x (k) )

2

(37)

Furthermore, if the input vector xm(k) is bounded, then equation (35) follows similar to the proof in Theorem 1. This completes the proof.

The simulation is based on a single-input singleoutput nonlinear time-series model with the delayed output signal as the input as proposed in [14]:

10

20 30 time response

40

50

250

300

20 0

■

VI. SIMULATION RESULTS

0

40 Cost Function

+

0

50

100

150 200 Epochs

Fig. 1. Time response and cost function of the neural network using the mismatched fixed dead zone training algorithm: Dashed line output of neural network; Solid line - output of the plant.

2 y(k)

y(k) = (0.8 – 0.5exp(–y2(k – 1)))y(k – 1) – (0.3 + 0.9exp( – y2(k – 1)))y(k – 2) (38)

where η(k) is a normally distributed random number with the bound η(k) ≤ ∆ = 0.2. A three-layered feed-forward neural network with two linear input neurons, five hidden nonlinear neurons and one linear output neuron was set up for the simulation. A neural network trained with the fixed dead zone scheme [10] is used for the identification of the nonlinear plant. Assume that knowledge of the disturbance is unknown and the size of the fixed dead zone is set to be very small. This makes the fixed dead zone scheme is effectively equivalent to a standard back-propagation algorithm [10]. After 300 epoch training, the time response of the neural network is shown in Fig. 1, which does not match the output of the plant closely. This is explained by the plot of cost function of the neural network, also shown in Figure 1, that the network does not converge to an optimal point due to the disturbance. In contrast to the fixed dead zone algorithm, the adaptive robust training algorithm proposed in sections 2 and 3 can be used to train the neural network objectively to reject disturbance. Figure 2 shows the time response of the neural network, using the adaptive robust training algorithm in the presence of the disturbance with the amplitude ∆ = 2. The performance is improved in terms of both the time response and the quadratic error cost function using the adaptive robust training algorithm as shown in Fig. 2, compared to the results of the fixed dead zone

-2

0

10

20 30 time response

40 Cost Function

+ 0.1sin(3.1415926y(k – 1)) + η(k)

0

40

50

250

300

20 0

0

50

100

150 200 Epochs

Fig. 2. Time response and cost function of the neural network using the adaptive robust training algorithm: Dashed line - output of neural network; Solid line - output of the plant.

algorithm in Fig. 1.

V. CONCLUSION The robust adaptive training algorithm provides guaranteed convergence of the neural network for the identification of nonlinear system without knowledge of the upper bound of disturbance. The gradient descent algorithm has been modified with a variable learning rate, which is referred as the adaptive dead zone scheme. Unlike a conventional dead zone scheme, the range of dead zone is determined by adaptive iteration algorithm on line. A complete convergence proof of the robust adaptive

Asian Journal of Control, Vol. 3, No. 2, June 2001

162

algorithm for the three-layered neural network is provided based on the Lyapunov stability theory. Simulation results shown that the robust adaptive algorithm guarantees the convergence property of the proposed algorithm for identification of the nonlinear system in the presence of disturbance.

REFERENCES 1. Solo, V. and X. Kong, Adaptive Signal Processing Algorithms: Stability and Performance, Prentice-Hall, Englewood Cliffs, NJ (1995). 2. Rupp, M., “The Behavior of LMS and NLMS Algorithms in the Presence of Spherically Invariant Processes,” IEEE Trans. Signal Process., Vol. 41, pp. 1149-1160 (1993). 3. Dasgupta, S., J.S. Garnett and C.R. Johnson, Jr., “Convergence of an Adaptive Filter with Signed Filter Error,” IEEE Trans. Signal Process., Vol. 42, pp.946950 (1994). 4. Goodwin, G. and K.S. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ (1984). 5. Suykens, J.A.K., J. Vandewalle and B.L.R. De Moor, “NLq Theory: Checking and Imposing Stability of Recurrent Neural Networks for Nonlinear Modeling,” IEEE Trans. Signal Process., Vol. 45, No. 11., pp. 2682-2691 (1997). 6. Rupp, M. and A.H. Sayed, “A Time-domain Feedback Analysis of Filtered-error Adaptive Gradient Algorithm,” IEEE Trans. Signal Process., Vol. 44, No. 6., pp. 1428-1439 (1996). 7. Chen, F.C. and H.K. Khalil, “Adaptive Control of A Class of Nonlinear Discrete-time Systems Using Neural Networks,” IEEE Trans. Automat. Contr., Vol. 40, No. 5, pp. 791-801 (1995). 8. Ortega, R. and Y. Tang, “Robustness of Adaptive Controllers - A Survey,” Automatica, Vol. 25, pp. 651-677 (1989). 9. Heiss, M., “Error-Minimizing Dead Zone for Basis Function Networks,” IEEE Trans Neural Networks, Vo. 7, No. 6, pp. 39-50 (1996). 10. Song, Q., “Robust Training Algorithm of Multilayered Neural Networks for Identification of Nonlinear Dynamic Systems,” IEE Proc Part -D, Vol. 145., No. 1, pp. 41-46 (1998). 11. Polycarpou, M.M., “Stable Adaptive Neural Control Scheme for Nonlinear Systems,” IEEE Trans. Automat. Contr., Vol. 41, No. 3, pp. 447-451 (1996). 12. Kosko, B., Neural Networks and Fuzzy Systems, Prentice-Hall, Englewood Cliffs, NJ (1992).

13. Ljung, L. and T. Soderstrom, Theory and Practice of Recursive Identification, MIT Press, Cambridge, Mass (1983). 14. Chen, S. and S.A. Billings, “Neural Networks for Nonlinear Dynamic System Modelling and Identification,” Int. J. Contr., Vol. 56, No. 2, pp. 319346 (1992). 15. Ling Yin, “Neural Network Based Fault Monitoring Scheme for Nonlinear Systems and its Application on Robot Systems,” M.Eng Thesis, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore (1999).

Qing Song received the B.S. and the M.S. degrees from Harbin Shipbuilding Engineering Institute and Dalian Maritime University, China in 1982 and 1986, respectively. He obtained the Ph.D. degree from the Industrial Control Centre at Strathclyde University, U.K. in 1992. He was an assistant professor at the Dalian maritime University from 1986 to 1987. He worked as a research engineer at the System and Control pte Ltd, U.K., before joined the School of Electrical and Electronic Engineering, Nanyang Technological University as a lecturer in 1992. He is now a senior lecturer. Dr. Qing Song is the first inventor of a neural network patent and was the recipient of the Tan Chin Tuan Academic Exchange Fellowship in 1997. He is currently and active industrial consultant. He has been the principal investigator of a few neural network related research programs targeted for industry, such as nonlinear control and pattern recorgnition systems.

Wenjie Hu was born in P.R. China. He received the B.S. degree from Zhe Jiang University, P.R. China. He is currently supported by a research scholarship to pursue Ph.D. degree in the school of EEE, Nanyang Technological University (NTU), Singapore. His research interests include support vector machine, neural networks and their applications.

Yeng Chai Soh received the B. Eng. (Hons. I) degree in electrical and electronic engineering from the University of Canterbury, new Zealand, in 1983, and the Ph.D. degree in electrical engineering from the University of Newcastle, Australia, in 1987.