Proceedings of the 2001 IEEE Congress on Evolutionary Computation Seoul, Korea • May 27-30, 2001
Hammerstein Model Identification Method Based on Genetic Programming Toshiharu Hatanaka and Katsuji Uosaki Department of Information and Knowledge Engineering, Tottori University, Tottori 680, Japan Phone: +81-857-31-5227, Facsimile: +81-857-31-0879 E-mail:
[email protected]
Abstract- In this paper, we address a novel approach to identify a nonlinear dynamic system for a Hammerstein model. The Hammerstein model is composed of a nonlinear static block in series with a linear dynamic system block. The aim of system identification here is to provide the optimal mathematical model of both nonlinear static and linear dynamic system blocks in some appropriate sense. In this paper, we use genetic programming to determine the functional structure for nonlinear static block. Each individual in genetic programming represents a nonlinear function structure. The unknown parameters of linear dynamic block and the nonlinear static block given by each individual are estimated with a least square method. The fitness is evaluated by AIC(Akaike information criterion) as representing the balance of model complexity and accuracy. It is calculated with the number of nodes in the genetic programming tree, the order of linear dynamic model and the accuracy of model for the training data. The results of numerical studies indicate the usefulness of proposed approach to the Hammerstein model identification. Keywords: Genetic programming, Hammerstein model, Nonlinear system identification.
0-7803-6657-3/01/$10.00 © 2001 IEEE
1430
1.
Introduction
The black box modeling for the unknown dynamic system based on the observed input and output data is called system identification. The identification method for the linear dynamic system has been well studied [1],[2]. Since nonlinear characteristics such as saturation, dead-zone, etc., are inherent in many real systems, identification of nonlinear dynamic systems has received much attentions in the last decade [3]. One of the difficulties in nonlinear system identification is that both the structure and the parameters of model should be determined. Most approaches to nonlinear system identification are based on the parametric optimization method including evolutionary algorithms such as genetic algorithms and evolution strategies [4],[5],[6],[7]. Though model structure determination is the primary problem in system identification (see Fig.1), it has been carried out by trial-and-error method to examine the candidate model structures. If the optimal model structure is determined automatically, we can identify the nonlinear systems more efficiently. From this point, we consider here the nonlinear system identification for the Hammerstein model, and propose a novel approach to identify nonlinear model structure automatically based on genetic programming(GP) [8]. The Hammerstein model consists of a nonlinear static block and a linear dynamic block in series and it is able to represent
In section 2, the genetic programming is introduced. Section 3 describes our approach to the Hammerstein model identification and results of some numerical examples are shown in Section 4. Finally, a conclusion is given in Section 5.
Input/Output Data
2.
Model structure determination
Genetic Programming
Parameter estimation
Genetic programming(GP) was developed by Koza to evolve an executable computer programs [8]. While a standard genetic algorithm(GA) deals with data represented by fixed length strings over some finite characters, the individuals in GP are the expression trees and it can be used for structural optimization rather than numerical parameter optimization. For example, the polynomial
is model stisatisfied? (Model validation)
y(t − 1) + y(t − 2)u(t − 1) can be written as an expression tree in Fig.2.
y sin(x) + 2 sin(x) *
Final model Fig. 1. Identification process.
: Nonterminal node
sin This figure
shows that model structure determination is the primary problem in system identification.
x
+
y
: Terminal node
2
Fig. 2. Structure of genetic programming tree for nonlinear function y = sin(x) + 2 sin(x)
a wide class of actual nonlinear systems. Genetic programming is used to determine a mathematical function form for the static nonlinear block. Then, the unknown parameters of this nonlinear function and linear dynamic block are determined by a least squares method. The fitness for each individual is evaluated by AIC(Akaike Information Criterion) to balance the accuracy and the complexity of model structure. The results of numerical simulation studies indicate the usefulness of the proposed approach to the Hammerstein model identification. Remainder of this paper is organized as follows. 1431
The elements of trees are called nodes. The nodes are classified into nonterminal nodes and terminal nodes. • Nonterminal nodes (function nodes) consume one or more inputs and provide one output(e.g. + and ∗ in the Fig.2). • Terminal nodes represent external inputs or constants. These are corresponding to the leaves of the expression trees. The evolution in GP proceeds in similar way to the standard GA, i.e., an initial population is generated at random and their fitness values are eval-
Individual A
Individual A’
f1
f1
f2
f3
f4
t1
f1 Function node mutation
t1
f2
f3 t1
(1)
t2
t4
ion
tat
Mu
t1
t3
g1
t2 f1
g4
t2
f2
s1
s2
f2
s4
t2
Mu
f1
tat
t3
ion
(2)
t1
Crossover
Terminal node mutation
f2
Terminal node
Indivdiual B g1 g2 s1
g3 s2
t2
Indivdiual B’ g1
s3
g2 g4
s1
s4
Terminal node mutation
g3 s2
s3
s1
f1
Function node
f1
f4 t3
(3)
t1
t4
n
t1
s1
tio
ta Mu
f2 t2
Mu
(4)
Fig. 3. Crossover in genetic programming
Function node mutation
f1
tat
t3
ion
g1
t1
g2
s1
uated for each individuals, then it is evolved by means of genetic operators as follows: • Selection: Pairs of parent trees are selected based on their fitness values for reproduction. • Crossover: This process is performed by selecting a node at random then exchanging the associated subtrees to produce a pair of offspring trees. • Mutation: This is performed by either replacing a node selected at random with its associated subtrees generated randomly or changing its type. • Inversion: Inversion is performed by exchanging two nodes which are located in the same level in the genetic programming tree.
3.
Hammerstein Model Identification
Let consider the model(see Fig.6): y(t) =
following
Hammerstein
b1 q −1 + . . . + bmq −m x(t) 1 + a1 q −1 + . . . + an q −n +e(t),
x(t) = F (u(t))
s2
Fig. 4. Mutation in genetic programming f1
f1
f2
f3
f2
f3
Inversion
f4 t3
t1 t4
t2
f5
f4
t5
t3
f5 t4
t2
t1
t5
Fig. 5. Inversion in genetic programming
linear dynamic block m and n are unknown and it should be determined from observed input and output data. We assume F (·) is nonlinear and unknown. The Hammerstein models are applicable to many real processes such as chemical plants, heat exchanger, and so on. Here, we assume that nonlinear function F (·) is expressed by F (u(t)) =
θi Fi (u(t))
(3)
i=1
(1) (2)
where q −1 is delay operator, y(t), u(t) and e(t) are the system output, input and measurement noise at time instant t respectively. The orders of 1432
where θi are unknown parameters and Fi (·) are the suitably selected nonlinear functions. The orthogonal basis functions and an artificial neural network approximations are used for Fi (u(t)) in some applications of the Hammerstein modeling [9].
Consider the following Hammerstein model with saturation–type nonlinearity:
e(t) u(t) Nonlinearity
x(t)
+ Linear Dynamics
+
y (t)
y(t) = 0.9y(t − 1) − 0.7y(t − 2) + x(t − 1) + 0.5x(t − 2) − 0.4x(t − 3) + e(t)
Fig. 6. The Hammerstein model. This model is composed of the static nonlinear block in series with linear dynamic
x(t) =
system block
Now, we can rewrite (1) and (2) as y(t) = =
n
ai y(t − i) +
i=1 n
m
bj x(t − j)
j=1
ai y(t − i) i=1 m
bj θk Fk (u(t − j)).
+
(4)
j=1 k=1
Based on this representation, we can estimate unknown parameters, ai , bj and θk using a least squares algorithm. The orders of this model, m, n and should be determined by AIC, which is defined by, AIC(n, m, ) = N log σ ˆ 2 + 2(n + m + )
u(t) 1.5
(9)
Table 1. Functions for genetic programming
1 {y(t) − yˆ(t)}2 (6) N We apply the genetic programming to automatically generate the candidate set for the nonlinear functions of equation(3) and determine the suitably one by evolution in the genetic programming. Each individual represents a set of nonlinear functions. Then, unknown parameters are estimated for a possible set of (m, n) by a least squares algorithm. The fitness value f for each individual is evaluated with the minimum AIC as
4.
(u(t) < −1.5) (−1.5 ≤ u(t) < 1.5) (1.5 ≤ u(t))
where e(t) is a white noise normally distributed with mean zero and unknown variance and the input signal u(t) is assumed random input uniformly distributed over [-5,5]. Hundred sets of input and output data were used for model construction using the proposed approach and a further 100 sets were used for model validation. The terminal nodes are corresponding to the system input and the constant value. The genetic programming function are listed in Table 1. The population size is 100 and the crossover rate, the mutation rate and inversion rate are 0.85, 0.05 and 0.05, respectively. After the 20th generation, an estimated model is
(5)
σ ˆ2 =
ˆ f = AIC(ˆ n, m, ˆ )
−1.5
(8)
structure estimation
symbol
inputs
u(t) + − ∗ exp sinh cosh tanh
0 0 2 2 2 1 1 1 1
function input(terminal node) constant(terminal node) increment decrement multiplier exponential hyperbolic sine hyperbolic cosine hyperbolic tangent
(7) yˆ(t) = 0.899y(t − 1) − 0.686y(t − 2)
Numerical Example
To show the validity of proposed approach to modeling nonlinear dynamic system, numerical simulation study is carried out. 1433
+ 1.00ˆ x(t − 1) + 0.562ˆ x(t − 2) − 0.454ˆ x(t − 3) x ˆ(t) = 1.483 tanh(u(t))
8
1.5
real value estimated value
6
Fˆ F
1
4
0.5 x(t)
2
0
0 -0.5 -2 -1 -4 -1.5 -4
-6 0
20
40
60
80
-2
0 u(t)
100
Fig. 7. Observed output data and estimated
2
4
Fig. 9. Estimated nonlinearity by the pro-
output by proposed method
posed method. The saturation type nonlinearity is well estimated.
3
error 10
2
real value estimated value
8 6
1
4 0
2 0
-1
-2 -2
-4 -6
-3 0
20
40
60
80
100 t
120
140
160
180
-8 -10 100
Fig. 8. The residual errors of estimated out-
120
140
160
180
200
Fig. 10. Simulation of estimated model. The
put
actual output and estimated one are plotted at time interval [1,200].
σ ˆ 2 = 0.985. Figure 7 shows the observed output data and estimated output by the proposed method. The residual errors are plotted in Fig.8. The estimated nonlinear function of the nonlinear static block is given in Fig.9. This indicates that the proposed method identifies the nonlinearity properly. Figure 10 shows the actual and estimated output for validation data set. The RMS(residual mean square) error is 0.985 for training data set and 1.025 for validation data set. Similar but not same results can be obtained by repeted experiments, since this approach includes probabilistic optimization. The parameter values controlling the genetic programming such as crossover, mutation and inversion rates are heuris1434
tically chosen here, but they are commonly used values. The identification results are not affected so much by the choice of these parameters. This indicates the robustness of the proposed approach. In our simulation, we terminate the evolution after the 20th generation since there are no change in fitness values. Of course, it depends on cases, but we observe that some appropriate (but not optimal) models can be obtained in the earlier generation.
5.
Conclusion
In this paper, we have proposed a nonlinear system identification method for the Hammerstein model based on the genetic programming. Genetic
programming is applied to the determine nonlinearity and unknown parameters in the nonlinear function and linear dynamic system model are estimated by a least square algorithm. The results of the numerical simulations indicate the usefulness of the proposed approach to the Hammerstein model identification. We can easily modify this approach for the other kind of nonlinear system models such as NARMAX models and Wiener models by selecting suitable nodes for genetic programming and they are now under investigation.
Acknowledgments This work was supported in part by the Grantin-Aid for Scientific Research from the Japan Society for the Promotion of Science (10045043 and 10650432).
References [1] Ljung, L., “System Identification: Theory for User,” Prentice-Hall, New Jersey, 1988. [2] S¨ odestr”om, T. and P. Stoica, “System Identification,” Prentice-Hall, London, 1989. [3] Sj¨ oberg, J., Q. Zhang, L. Ljung and A. Benevite et al. “Nonlinear black–box modeling in system identification: a unified overview,” Automatica, Vol.31, No.12, pp.1691–1724,1995. [4] Yao, L. and W.A.Sethares, “Nonlinear parameter estimation via the genetic algorithm,” IEEE Transactions on Signal Processing, Vol.42, No.4, pp.927–935, 1994. [5] Sheta, A.F. and K.A. De Jong, “Parameter estimation of nonlinear system in noisy enviroment using genetic algorithms,” Proceedings of the 1996 IEEE International Symposium on Intelligent Control, pp.360–365, 1996. [6] Chen, S. and S.A. Billings, “Representation of Non-linear Systems: The NARMAX Model,” Int. Journal of Control, Vol.49, No.3, pp.1013– 1032, 1989. [7] Hatanaka, T. et al., “Evolutionary approach to system identification,” Preprints of 11th IFAC 1435
Symposium on System Identification, Vol.3, pp.1383–1388, 1997. [8] Koza, J. R., “Genetic Programming: On the Programming of Computers by Means of Natural Selection ,” MIT Press, 1992. [9] AlDuwaish,H., M.N.Karim and V. Chandrasekar, “Hammerstein model identification by multilayer feedforward neural networks,” International Journal of Control, Vol.28, No.1, pp.49–54,1997.
