BLACK-BOX TOOL FOR NONLINEAR SYSTEM ... - Semantic Scholar

International Journal of Computational Intelligence and Applications  World Scientific Publishing Company

BLACK-BOX TOOL FOR NONLINEAR SYSTEM IDENTIFICATION BASED UPON FUZZY SYSTEM O. Hassanein * Scholl of Engineering and Information Technology, University of New South Wales@ Australian Defence Force Academy, Canberra, ACT 2600, Australia [email protected] Sreenatha G. Anavatti Scholl of Engineering and Information Technology, University of New South Wales@ Australian Defence Force Academy, Canberra, ACT 2600, Australia [email protected] Tapabrata Ray Scholl of Engineering and Information Technology, University of New South Wales@ Australian Defence Force Academy, Canberra, ACT 2600, Australia [email protected] Received 28 August 2012 Revised Day Month Day This paper introduces a novel identifier scheme for identification of non-linear systems with disturbances. The identification process is carried out in two steps; an off-line procedure and an online procedure. The method comprises of an automatic structure generating phase using entropy based technique. The accuracy of the model is suitably controlled using the entropy measure. The parameter learning phase uses the back-propagation technique. To improve the accuracy and also for generalization of the model to handle different data sets, Differential Evolution technique is employed whereby the parameters of the model are suitably tuned using evolutionary technique. A semi serial-parallel model is introduced to improve the on-line identification process in the presence of noisy data. The proposed mechanism is utilized and compared against the classical Sugeno, adaptive network based fuzzy inference system modeling and Laguerre network based fuzzy system for the identification of a nonlinear benchmark problem. In addition, the proposed technique is also used to model a rotary wing unmanned aerial vehicle from real test input-output data. The modelling performance and generalisation capability are seen to be superior with our method. Keywords: Entropy measurement; Black-box identification.

1. Introduction System identification and modelling is an essential tool for controlling the systems without human intervention. A mathematical or artificial intelligent model of the system is a crucial part in the design, analysis and control of the system. Generally, the system *

Corresponding Author. 1

2

O. Hassanein et al

model is classified into two types: physical model and input-output data model1. The physical model uses the physical properties in describing the behaviour of the system. However, the input-output model uses the input-output data of the system to develop the model of that system. The first method uses the mathematical equations and physical laws to describe the behaviour of the system and derive the model. The second method uses the input–output data to describe the relationship between these data and this provides superiority for system identification2. Moreover, as in Ref. 3, the prior knowledge and physical insight about the system helps in the identification being classified into three colour-coded levels, white-box model, gray-box model and blackbox model. Various approaches have been used for identifying the nonlinear dynamic systems4,5. In general, the nonlinear system identification consists of model structure selection and parameter estimation. Some works on system identification are based on known structure and parameters’ estimation. Ref. 4 introduced two identifiers of nonlinear systems based on the fuzzy system models with initially known model structure. The two fuzzy identifiers were used to simulate the chaotic glycolytic oscillator and the results show that by incorporating some linguistic description, the accuracy of the fuzzy identifier was greatly improved. A chaotic optimization method, called CAS (chaotic ant swarm), is introduced to solve the problem of designing a fuzzy system to identify dynamical systems. At each learning time step, the CAS algorithm is iterated to give the optimal parameters of fuzzy systems based on the fitness theory6. The parameter identification problem has been defined and solved by real-coded GA7. Alternative methods are used to generate the system model and are based upon adaptive techniques to find the optimal values of model parameters. Ref. 8 presented a fuzzy modelling technique using ortho-normal basis functions (OBF-fuzzy) in the representation of the model input signals. A Laguerre Network Based Fuzzy (LNBF) system model is proposed in Ref. 4. The model is a combination of orthonormal Laguerre bases and a static nonlinear fuzzy system. The performance of the proposed modeling approach is compared against the classical Sugeno type fuzzy modeling; ANFIS modeling and Oliveira’s OBF-fuzzy approach. Ref. 9 presented a genetic-based fuzzy modelling approach for generating TSK models. The model building process is divided into two phases, the structure learning and the parameter learning phase. Structure learning is performed using genetic-based structure learning (GBSL). In the second phase, the parameters of the initial model are fine-tuned using the suggested GBSL algorithm. Ref. 10 proposed an approach that integrates a fuzzy rule induction algorithm with a rough set-assisted feature reduction method. Learning algorithms for the automatic generation of fuzzy rules are presented in Refs 11-12. These learning algorithms generate fuzzy rules based on learning vectors and learn/optimize the I/O behaviour of the system. Ref. 13 introduced an alternative for the synthesis of Takagi–Sugeno fuzzy logic controller with reduced rule base. A Genetic Approach to Fuzzy Supervised Learning algorithm called GAFSL based on the Multi-objective Genetic Algorithms (MGAs) is used.

Black-Box Tool for Nonlinear System Identification Based Upon Fuzzy System 3

An approach to the refinement of parametric fuzzy if-then rules based on fuzzy entropy maximization, rather than based only on the training error reduction is presented in Ref. 14. Different learning methods are used to adapt the fuzzy system parameters16 based on learning algorithm such as clustering, back propagation, table-lookup and least square methods17. The advantages and the disadvantages of each are presented in Ref 15. In this context, the fuzzy system can be represented as a three-layer feed forward network in order to use the advantages of the back propagation algorithm15. The first written article on Differential evolution (DE) appeared as a technical report by R. Storn and K. V. Price in 199534. Since then, the DE has drawn the attention of many researchers all over the world resulting in many variants of the basic algorithm with improved performance. It is arguably one of the most powerful stochastic real-parameter optimization algorithms in current use. DE operates through similar computational steps as employed by a standard evolutionary algorithm (EA). However, unlike traditional EAs, the DE-variants perturb the current generation population members with the scaled differences of randomly selected and distinct population members18. The most popular EA is genetic algorithm (GA). Although many GA versions have been developed, they are still time consuming and can be trapped by local minima. These disadvantages can be eliminated by using DE which runs several vectors simultaneously. The main difference between the GA and DE is the mutation scheme that makes DE self adaptive and the selection process39. Ref. 19 proposed a generalized hybrid generation scheme to enhance the exploitation and accelerate the convergence of the original DE algorithm. DE- is employed to automatically fine tune the fuzzy membership functions used in the automatic train operation (ATO) of the fuzzy controller38. The tuning minimises multi-objective performance indices, takes into account different factors like interstation distance, rapidly changing gradient profiles, and schedules. Through simulations, the proposed tuning method is shown to improve greatly the performance of the fuzzy ATO controller. In addition, DE is used in the optimization problem of fuzzy neural technique. In Ref 37, a pruning algorithm based on DE with division of work which can determine the structure of the fuzzy neural network based on effective rules obtained through fuzzy rules searching is proposed. Moreover, DE applied in several significant applications to the optimization problems arising from diverse domains of science and engineering and satisfactory results are achived35,36. Most of the techniques discussed above are not suitable for real-time systems where the computational time and capacity constraints restrict elaborate training and online tuning. In addition, there is a necessity for model identification techniques capable of handling variations in the operating conditions and disturbances. In this paper, an autogenerating mechanism with entropy based differential evolution fuzzy system modelling is proposed to generate an adaptive fuzzy model for any physical system without any prior knowledge of the physical relationship inside the system and/or the system behaviour. Moreover, a new scheme for representing the system identification structure is presented that is considered as a combination between the well-known schemes, parallel model and serial-parallel model, semi serial-parallel model (SSPM). The information

4

O. Hassanein et al

required to model the system is only the input-output data, available from the measurements. The process of identification is carried out in two steps. The first step is the off-line procedure and the second step is online procedure. Off-line procedure is performed by generating the model structure automatically using a batch of input-output data based upon the entropy measure that controls the model accuracy. Then this generated fuzzy model is trained for another data batch using differential evolution to optimize the structure parameters that lead to enhance the generality and accuracy of the model. However, in online procedure, data is simultaneously provided from real test to get current output and to optimize the model parameters at the same time. Thus, by combining the offline procedure with the online procedure, the burden for online computation is reduced and hence is implementable in real-time. In addition, the method provides a robust identification technique in the presence of changing operating conditions (parameter variations) and in the presence of disturbances, thus filling a gap in the real-time identification of non-linear systems. This paper is organized as follows. Section 2 describes the problem formulation of the system identification. Section 3 shows the structure of the fuzzy model. Next, Section 4 presents EFSM Modelling Mechanism. Section 5 presents the proposed model identification scheme. The results of simulations of various problems are described in section 6. Finally, Section 7 concludes the paper. 2. System Identification Problem Formulation The problem of determining the mathematical model that represents the behavior of the system based upon the input-output data is generally referred to as system identification. Various approaches can be used in identification of nonlinear dynamic system20. In the black-box identification process, there is no need to use previous knowledge or/and physical structure21. The main two purposes of system identification are to predict the system’s behavior and to design and simulate the controller based on the model of the system7. In general, a nonlinear mapping on past inputs and outputs gives a discrete nonlinear system4. y (k + 1) = f (u (k ),.........., u (k − m + 1), y (k − 1),.........., y (k − n + 1)).

(1)

where f is the unknown operator such as fuzzy system, m and n are the number of past terms of input (u) and output(y), respectively. As shown in Figure 1, there are two schemes15 for representing (1): • Parallel model: yˆ (k + 1) = f (u (k ),.........., u (k − m + 1), yˆ (k − 1),.........., yˆ (k − n + 1)).

(2)

where yˆ is the output of identification model, such as fuzzy system or neural network. • Serial-parallel model yˆ (k + 1) = f (u (k ),.........., u (k − m + 1), y (k − 1),.........., y (k − n + 1)).

(3)


where y is the output of system, mathematical model or real-world plant. In parallel model, the output of the identification model is fed back to the model, while in serial-parallel model the output of the system is fed back to the model. A u

B u

Process or Plant

Process or Plant y

y

e

e

ŷ System Identification Model

System Identification Model

Fig 1. Basic schemes of system identification: A) parallel model, B) serial- parallel model

3. Structure of fuzzy system model Classical control theory is based on mathematical models that describe the behavior of the plant or system under consideration. Fuzzy systems are known for their capabilities to approximate any nonlinear dynamic system22. The main idea of fuzzy control is to build a model of a human control expert who is capable of controlling the plant without thinking in mathematical model terms23. From a conceptual point of view, the design of fuzzy systems from input-output pairs is classified into two types of approaches24. In the first approach, fuzzy IF-THEN rules are first generated from input-output pairs, and the fuzzy system is then constructed from these rules according to certain choices of fuzzy inference engine, fuzzifier, and defuzzifier. In the second approach, the structure of the fuzzy system is specified first and some parameters in the structure are free to change, then these free parameters are determined according to the input-output pairs. In this study, the second approach is considered. Fuzzy Modeling is a method of describing the characteristics of a system using fuzzy rules, and it can express complex non-linear dynamic systems by linguistic if-then rules25 as given by; 𝑅(𝑙) ∶ 𝐼𝐹 �𝑥1 𝑖𝑠 𝐹1𝑙 𝑎𝑛𝑑 … … … . 𝑎𝑛𝑑 𝑥𝑛 𝑖𝑠 𝐹𝑛𝑙 � 𝑇𝐻𝐸𝑁 𝑦 𝑖𝑠 𝐺 𝑙 .

(4)

where 𝑥 = (𝑥1 , … … . . , 𝑥𝑛 )𝑇 ∈ 𝑈 𝑎𝑛𝑑 𝑦 ∈ 𝑅 are the inputs and outputs the fuzzy system, respectively, 𝐹𝑖𝑙 𝑎𝑛𝑑 𝐺𝑖 are labels of fuzzy sets in U, and R, respectively, and l = 1, 2,…, M. As mentioned in Ref. 15, the fuzzy logic system can be represented as a feed forward network with five-layers as shown in Figure 4. Layer one is just passing the crisp input values that come from the controller to the next layer, L(i1) = xi . Then, the calculated membership value in layer two is;

6

O. Hassanein et al

( ) ( )

2  x j − c −j l  L(ij2) = exp − 0.5 2  σ lj + ε 

  . 

(5)

with 𝑐𝑖𝑙 and 𝜎𝑖𝑙 representing the centre and width of Gaussian memberships for input variable 𝑥𝑗 . 𝜀 > 0 is a small constant, the purpose of adding the small constant to the fuzzy membership functions is that even if the σ′𝑠 = 0, the fuzzy membership functions are still well defined. This modification will make the adaptive law simpler because we do not require σ′𝑠 ≠ 0. Network nodes in layer three receive the membership degree of the associated rule in layer two. As a result, the output function of each node in layer three is; N

L(j3) =

∏L . 2

(6)

j =1

where L(j3) of each rule represents the firing strength of its corresponding rule. Layer four is called a consequent layer. The output of that layer is calculated as;

∑ b [L ] (σ ) + ε  . M

L(j4) =

l

l 2

3 j

(7)

l =1

𝜎 𝑙 is a parameter characterizing the shape of the output membership function 𝜇𝐺 𝑙 (𝑦), such that the narrower the shape of 𝜇𝐺𝑙 (𝑦), the smaller is 𝜎 𝑙 . For more details about the concept of the modified center average defuzzifier, please see Refs 5 and 25. Finally, the output comes from layer five that integrates all of the action calculated by layer three and layer four and acts as a defuzzifier. The fuzzy system with product inference engine, singleton fuzzifier, modified center average defuzzifier, and Gaussian membership function is chosen in this study. That is, the fuzzy system that we are going to design is of the following form:   b   l =1   y ( x) =  M    l =1   M

∑ ∏ l

∑∏

    σ l 2 + ε       .     σ l 2 + ε      

2   x − cl   j  j  exp − 0.5   l 2 j =1 σ j +ε  

N

( )

2   x − cl   j  j N  exp − 0.5   l 2 j =1 σ j +ε  

( )

( )

(8)

( )

The final output of the fuzzy model is given by (8) with 𝑐𝑖𝑙 and 𝜎𝑖𝑙 representing the centre and width of Gaussian memberships for input variable 𝑥𝑗 for the rule 𝑖, 𝑙 and 𝑁 being the number of rules of fuzzy model and number of inputs respectively. 𝑐𝑖𝑙 , 𝜎𝑖𝑙 , 𝑏 𝑙 and 𝜎 𝑙 are free parameters which will be updated in the adaptation procedure.


y F=A/B

A

B Output Layer

Consequent Layer

bM/σM

b1/σ1

Z1 µ1

ZM µ j1

1

µ1

µ jN

1

Rule Layer

Linguistic Layer

X1

Xn

Input Layer

Fig 4. Network structure for fuzzy system identifier

4. EFSM Modeling Mechanism In this study, a universal black box modelling tool is offered for modelling automatically any physical system without any prior knowledge of the physical relationship or the behaviour of the system. Sets of input-output data with different operating conditions and disturbances are the only information required for generating the model of the system. The proposed EFSM mechanism is a combination of two stages. The first stage is the off-line procedure and the second one is the on-line procedure. The off-line stage comprises of a structure-generating phase and parameter-learning phase. The structuregenerating phase is based on the entropy measure used to control the model accuracy. Parameter learning phase is executed in two steps, the first step during the structuregenerating phase and it is based on supervised learning algorithms using the back propagation algorithm. After the fuzzy model structure is generated, the fuzzy model enters to the second step, which is based on Differential Evolution (DE) algorithm to adjust the parameters of the model optimally based on a different input-output training data set. The on-line procedure aims to identify the model of the process online by using real-time measurements of the process. A back propagation method is used to adapt the generated model parameters online using the error between the fuzzy model and the actual output of the process. The detailed flowchart of the proposed EFSM mechanism is presented in Figure 5. 4.1. Off-Line Procedure This section presents the off-line stage as a first step in generating the model. The off-line stage comprises of a structure-generating phase and parameter-learning phase. Structure

8

O. Hassanein et al

generating phase is based on the entropy measure26 to determine whether a new rule for fuzzy system should be added to satisfy the fuzzy partitioning of input variables that leads to increase in the model accuracy. The entropy measure is used in EFSM to control the accuracy of the model according to the computational time required. Parameter learning phase is based on the minimization of a given cost function by adjusting the parameter of the memberships in the generated fuzzy model. 4.1.1. Structure generating phase Initially, no rules or memberships exist in the model. The membership functions and rules are generated automatically based upon the reception of incoming input-output data. The first step in the structure generating is to determine the criteria that should be used to extract and generate new fuzzy rules of the fuzzy system from the input-output data. In addition, determine the number of fuzzy sets in the universe of discourse of each input variable, since one cluster in the input space corresponds to one potential fuzzy logic rule. The entropy measure is used as this criterion. The entropy values between data points and current membership functions are calculated to determine whether a new rule should be added or not. For each incoming pattern, the rule firing strength is considered as the degree to which the incoming pattern belongs to the corresponding cluster. The entropy measure between each data point and each membership function is calculated. A data point of closed mean will have lower entropy27. The entropy measure calculation is based upon the firing strength of each rule as given by Ref. 14; N

ENT j =

∑L

( 2) ij

(1 −L(ij2) ).

(9)

i =1

where L(ij2 ) is the firing strength for each rule and the maximum entropy measure is determined as;

ENTMAX = max ENT j . 1≤ X ≤ R ( t )

(10)

where R(t) is the number of existing rules at time t. If ENTMax ≤ ENT , then a new rule is generated and added to the model. ENT is a pre-specified threshold. In the structuregenerating phase, the threshold parameter ENT is an important parameter. A low threshold leads to fewer rules, whereas a high threshold leads to the learning of fine clusters (i.e., more rules are generated). Therefore, the selection of the threshold value ENT will critically affect modelling accuracy. Once a new rule has been generated, the next step is to assign the initial mean and variance to the new membership function and the corresponding output and variance for the consequent part. After the model is generated, the error between the data used in generating phase and the output of generated model are calculated. If the error and its RMSE value meet the design demand in terms of model accuracy and computational time, then the generated model will enter to the next step, otherwise, the ENT threshold should be changed as previously described.


Since the goal is to minimize an objective function, the mean, variance, and output are all adjustable later in the parameter-learning phase.

Fig 5. The flowchart of the EFSM mechanism

4.1.2. Parameter learning phase using BP During the generating process of the model structure according to the current inputoutput data set, the parameter-learning procedure is involved to adjust the parameters of the model based on the same data. The learning process involves determining the

10

O. Hassanein et al

minimum of a given cost function. The gradient of the cost function is computed and the parameters are adjusted with the negative gradient. The back-propagation algorithm (BP) is used for this supervised learning method. The final output of each fuzzy model is given by (8). The gradient method is used to adapt the generated fuzzy model parameters based on the following objective function;

E (k ) =

1 ( y m (k + 1) − y f (k + 1)) 2 . 2

(11)

where E(k) is error between the fuzzy model and the actual plant outputs. If Z(k) represents the parameter to be adapted at iteration k in the fuzzy model, the training algorithm seeks to minimize the value of the objective function15,24;

z (k + 1) = z (k ) − α

∂E . ∂Z

(12)

To train b l ;

zl . Bi ((σ il ) 2 + ε )

(13)

σ l (k + 1) = σ l (k ) − α

bi− l − y zl (−2σ il ). Bi ((σ il ) 2 + ε ) 2

(14)

cil (k + 1) = cil (k ) − α

x j − cil bi− l − y zl . Bi (σ il ) 2 + ε (σ l ) 2 + ε

(15)

2( x j − cil ) 2 σ l bi− l − y zl . Bi (σ il ) 2 + ε ((σ l ) 2 + ε ) 2

(16)

b l (k + 1) = b l (k ) − α To train σ l ;

To train cil ;

To train σil ;

σ il (k +) = σ il (k ) − α where

zl =

∏

2   x − c lj   j N  exp − 0.5  2 j =1  σ lj + ε  

( )

  ., B =  i  

N

∑z

l

((σ l ) 2 + ε ).

(17)

l =1

The learning rate, σ, in (13-16) has a significant effect on the stability and convergence of the system28. A higher learning rate may enhance the convergence rate but can reduce the stability of the system. A smaller value of the learning rate guarantees the stability of the system but slows the convergence. The proper choice of the learning rate is therefore very important.


4.1.3. Parameter learning phase using DE The above two steps ensure that fuzzy model structure is established. In order to improve the generalization capabilities of the model that leads to enhanced accuracy, the generated model enters the parameter-optimization phase based on Differential Evolution (DE) to adjust the parameters of the network optimally using a different input-output training data set. DE is known as a powerful algorithm for real parameter optimization. In DE, an initial population is generated and, for each parent vector from the current population (target vector), a mutant vector (donor vector) is obtained. Finally, an offspring is formed by combining the donor with the target vector. A tournament is then held between each parent and its offspring with the better being copied to the next generation29,30. DE prematurely converges when dealing with a multimodal fitness function because it loses its diversity31,32. The DE learning algorithm consists of four major steps— the initialization step, the evaluation step, the mutation step, and the reproduction step. Initialization step: The first step in DE is the coding of the fuzzy network model parameters into an individual. Equation (18) shows the way of the individual coding of fuzzy parameters, where i and j represent the 𝑖 𝑡ℎ input variable and the 𝑗𝑡ℎ rule, respectively. Individual = c1j , c 2j ,..., c lj , σ 1j , σ 2j ,..., σ lj , b1 , b 2 ,...., b l , σ 1 , σ 2 ,...., σ l .

(18)

Before the optimization algorithm is started, the individual must be created randomly in the range between a pre specified minimum and maximum value of the each parameter. The number of individuals is determined according to the generation and population size. Evaluation step: The objective function is used to provide a measure of how individuals have performed in the problem domain. In the minimization problem, the fit individuals will have the lowest numerical value of the associated problem objective function. This raw measure of fitness is only used as an intermediate stage in determining the relative performance of individuals in a DE. Another Function, the fitness function, is normally used to transform the objective function value into a measure of relative fitness. A “fitness function”, which is in effect a performance index, is used to select the best solution in the population to be parents to the offsprings that will comprise the next generation. The fitter the parent is, the greater the probability of selection. This emulates the evolutionary process of “survival of the fittest”; t2

∫

ITAE = e 2 dt. t1

J E1 = Max( ITAE ). FE1 = 1 J E1 .

(19)

12

O. Hassanein et al

where J E1 is the objective function of each degree of freedom separately for all population. FE1 is the overall fitness function of that degree of freedom. The objective function using fitness produces distribution is in the range (0, 1). Mutation and crossover step: This operation enables DE to explore the search space and maintain diversity. The simplest form of this operation is that a mutant vector is generated by multiplying an amplification factor, 𝐹, by the difference between two random vectors and the result is added to a third random vector (DE/rand/1)33as; V z ,t = X r1 ,t + F × ( X r2 ,t − X r3 ,t ).

(20)

where 𝑟1 , 𝑟2 , 𝑟3 are random numbers (1,2, ..., PS), 𝑟1 ≠ 𝑟2 ≠ 𝑟3 ≠ 𝑧, 𝑋 is a decision vector, PS is the population size, F is a positive control parameter for scaling the DE and t the current generation. For more details, readers are referred to Ref. 18. In this paper, the binomial crossover is performed on each of the 𝑗𝑡ℎ variables whenever a randomly picked number (between 0 and 1) is less than or equal to a crossover rate, Cr. In this case, the number of parameters inherited from the donor has a (nearly) binomial distribution as;

 Vzj ,t , u zj ,t =   X zj ,t 

if ( rand ≤ C r or j = J rand ) . otherwise.

(21)

where 𝑟𝑎𝑛𝑑 ∈ [0,1], and 𝑗𝑟𝑎𝑛𝑑 ∈ [1,2, … , 𝐷] is a randomly chosen index which ensures �⃗𝑧,𝑡 receives at least one component from �V⃗z,t. 𝑈 Reproduction and selection step: To keep the population size constant over subsequent generations, the next step of the algorithm calls for selection to determine whether the target or the trial vector survives to the next generation, i.e., at G = G + 1. The selection operation is described as

X i ,G +1 = U i ,G if f (U i ,G ) ≤ f ( X i ,G ). = X i ,G if f (U i ,G )  f ( X i ,G ).

(22)

where f (X) is the objective function to be minimized. Therefore, if the new trial vector yields an equal or lower value of the objective function, it replaces the corresponding target vector in the next generation; otherwise the target is retained in the population. Hence, the population either gets better (with respect to the minimization of the objective function) or retains the same fitness status, but never deteriorates. 4.2. ON-LINE procedure The proposed fuzzy model is intended to work in situations where there are large uncertainties and unknown variations in plant parameters and structure. Generally, the basic objective of the proposed fuzzy model is identifying the behavior of the original plant online with consistent performance and high accuracy in the presence of these uncertainties. Therefore, the generated fuzzy model is equipped with online adaptation


algorithm to enhance the generality and accuracy of the model during the online operations conditions. The learning algorithm is based upon the BP technique. The BP algorithm minimizes a given cost function, (11), by adjusting the parameter of the membership in the fuzzy system as mentioned in (13-16). The adaptive fuzzy model is a fuzzy system model with a training algorithm where the model is synthesized from a bundle of fuzzy If-Then rules. Each fuzzy membership is characterized by certain parameters. The training algorithm adjusts these parameters based on numerical inputs and outputs data. As shown in Figure 3, the fuzzy model is placed in parallel with the process to be identified. It aims to identify the model of the process online by using the input-output measurements of the process. A back propagation method is used to adapt the model parameters online based on the error between the identified model and the actual output of the process. 5. The proposed model identification scheme As shown in Figure 2, it must be noted that most real-time applications may present limitations regarding the computational time and limited memory. In order to meet the requirements for real-time applications in terms of these limitations, the proposed scheme uses m =0 and n=1 in (1), indicating that no past input terms are fed back to model and just one past output term is fed back. The proposed model is considered as a modified nonlinear moving average model (NMA)3,8. Then, the general equation that describes the new model scheme is:

yˆ (k ) = f (u (k ), yˆ r (k − 1)).

(23)

where yˆ r is a combination between the system output and identification model output. As shown in Figure 2, the output of the system, y, and the output of the identification model, yˆ , are fed back into selector switch based on sampling time of the system identification. This selector switch will pass the system output to the identifier for one sample time, then will pass the identification model output to the identifier for next consecutive fivesamples. In the real time applications, the system outputs are the measurement data from different sensors. Generally, the sampling time required for acquiring and manipulating data from sensors is higher than the sampling time for identifier. In on-line process, it is important to highlight that the process of adapting the identification model parameters is performed in parallel with the normal operation of the system, which does not stop at any point. The advantages of the proposed SSPM are: On the one hand, in the time difference between the sampling time of the identifier and the sensors, the identifier is subjected to unavailability of the sensors data and to noisy data as well. Therefore, in the time difference between two sampling time, the SSPM will feed back one past term of itself to keep the accuracy of the identifier. On the other hand, predication the system behavior in the real-time applications

14

O. Hassanein et al u

Process or Plant y

e

Selector Switch

Z-1

ŷ

System Identification Model Fuzzy system f(.)

Fig 2. the proposed semi serial-parallel model (SSPM) identification model

6. Results This study evaluated the performance of the proposed EFSM mechanism to model nonlinear systems. This section presents two examples, the first example introduces simulation results for the system identification of nonlinear benchmark problem and compares the performance with that of other methods. In the second example, the proposed mechanism is used to model the helicopter dynamics as an application for real time implementation. 6.1. Example 1 The performance of the proposed EFSM mechanism has been tested and compared with Lagurre network based fuzzy, the classical Sugeno fuzzy modeling, and ANFIS approaches on data obtained using the linear-nonlinear Wiener cascade4 shown in Fig. 6. The relationship between input x and output of the LTI block is given by; y1 (k ) = x(k − D) + 0.35 y1 (k − 1) . As the noise w at the input of the static nonlinear block is zero, the output becomes y (k ) = y1 (k ) + y12 (k ) . As mentioned in Ref. 4 this system is driven with the following 300 points long input sequence,

x = 0.4 sin(0.9u ) + cos(2.3u ) sin(1.2u ) − cos(2u ).

(24)

where u is varying from 0 to 2π with increment of 2π / 300 . The input is selected to perturb the system as much as possible to picture its dynamics. w

1 1 − 0.35z −1

y1

y12

y

x

Fig 6. The linear-nonlinear Wiener cascade model

First of all, a suitable value for ENT , entropy threshold value, should be assigned. This selection will critically affect the simulation results, as according to (9), when the membership function of a fuzzy set is equal to 0.5 for all j, the fuzzy entropy of the fuzzy set attains the maximum. It means that the threshold value belongs to the range [0,0.5],


the ENT value used for this system is 0.434. The EFSM requires, as described in Figure 5, an input-output data to build the relationship between them, which is considered the model of the system. The input data of the system is x and the output data is y, as shown in Figure 6. Those data are captured from the mathematical model of system to illustrate the idea of how the EFSM works. Secondly, EFSM mechanism is used to identify the model of nonlinear dynamical system of Figure 6 using the input in (24). After that, the following input is used to test the generated model;

x = cos(1.3u ) sin(2u ) − cos(0.6u ) + 0.9 sin(2.5u ).

(25)

The total number of rules generated by EFSM is four rules with two inputs and one output. The performance of the model generated by EFSM has been tested and compared with LNBF, Sugeno, and ANFIS approaches with the same training and testing inputs4. Figure 7 shows the training performance of the EFSM approach. Figure 8 and Figure 9 show the response of the proposed EFSM mechanism to the test input and resulted error between the response of the EFSM and the actual output, respectively. In addition, Table 1 summarizes the results for different fuzzy approaches that has used for the same nonlinear dynamic system. It is clearly seen that the estimated output of the proposed EFSM approach is almost identical to the original output. 6

Output, y

4

2

0

-2 0

Actual Output 100

50

150

EFSM Output

200

250

300

200

250

300

Time, sec

Fig 7. The identifying model by EFSM 10

Actual Output

EFSM Output

Output, y

8 6 4 2 0 -2 0

50

100

150

Time, sec

Fig 8. Test response of the trained EFSM model

16

O. Hassanein et al 0.05

Error

0.025

0

-0.025

-0.05 0

50

100

150

200

250

300

Time, sec

Fig 9. The error between the actual output and response of the trained EFSM model Table 1. Comparison of different fuzzy system identification approaches (Musa, 2009). Model OBF-fuzzy Sugeno fuzzy ANFIS LNBF EFSN

Training Performance

Testing Performance

MSE 0.098 7.70 × 10-4 6.35 × 10-4 0.12 × 10-4 4.367 × 10-6

MSE 0.30 61.76 × 10-3 1.95 × 10-3 0.22 × 10-3 0.112 × 10-3

6.2. Example 2 This example demonstrates the ability of the proposed EFSM mechanism for modeling real time applications. The real time application is Unmanned Aerial Vehicles (UAVs). Six degree of freedom flight data was collected, flight tests were carried out at UNSW@ADFA to collect a range of data for different flight conditions. A higher payload capacity platform, Eagle UAV, as shown in Figure 10, is used to collect the flight data34. The Eagle helicopter instrumented to measure pitch rate, q , pitch attitude, θ , roll rate,

p , roll attitude, ϕ , yaw rate, r , longitudinal, a x , lateral, a y , and normal accelerations, a z , was test flown and various flight data was logged. The inputs to five servo actuators i.e. collective, δ col , throttle, δ th , aileron, δ ail , elevator, δ ele , and tail rotor pitch, δ ped were also logged. A sampling frequency of 50Hz was used for data collection34,35. The processing unit used is a PC104 computer, where its real time environment is the xPC target. The xPC target is a Matlab toolbox that supports different boards and hence saves considerable development time and debugging process. In addition to the IMU unit, three more sensors have been mounted onboard and interfaced with the PC104. These additional sensors measure forward speed, angle of attack and side slip angle.


Fig 10. Eagle UAV platform at UNSW@ADFA34

Hardware in loop (HIL) results for the six degree of freedom (DOF) dynamic identification are presented in this example. The dynamics of the helicopter need to be modeled as a coupled six-DOF system, it may not be possible to decouple the six-degreeof-freedom system into two three-DOF systems34, 35. The coupled six-DOF of a UAV can be considered as MIMO fuzzy system. EFSM mechanism, as described in Figure 2, is applied to identify an online model of nonlinear dynamical system of Eagle UAV using real fight data. The input vector to EFSM model is u = [δ col , δ ail , δ ail , δ ped ] and one past term of the identifier and the system outputs. The output vector of the model presents the state of the AUV in terms of orientation and linear velocities, y = [ϕ ,θ ,ψ ,V x ,V y ,V z ] . After that, the generated model is trained using a different input-output data set, then uploaded to PC104 and validated using different flight conditions. The total number of rules generated by EFSM is nine rules. The value of the entropy threshold selected for this example is 0.213. As previously described, the selection of the threshold value ENT will critically affect modelling accuracy. For instance, if ENT is selected to be 0.109, the total number of generated rules is six rules, which means less modeling accuracy and computational time. On the contrary, if ENT is increased to be 0.412, 13 rules are generated, that leads to increase the model accuracy at the cost of increasing the computational time. Figure 11 and Figure 12 show the flight data. It is clearly seen that the collected data is affected by noise which makes the identification process a challenging task since noise has a significant effect on the identification and the control performance. The predicted and actual response of Eagle helicopter based upon the proposed EFSM mechanism for the 6-DOF with online tuning capability in the presence of noise is shown in Figure 12. Table 1 shows the RMSE values of the modeling errors of the predicted response of proposed modeling mechanism. It shows the comparison between the response of the EFSM mechanism and Neural Network Identification (NNID) proposed in Ref 33. The results indicate that the proposed technique is capable of imitating the dynamics of the helicopter accurately. Table 2. Comparison of EFSM and NNID approaches.

18

O. Hassanein et al

DOF φ ϴ ψ Vx Vy Vz

Online EFSM

Offline NNID33,34

RMSE 0.0075 0.0273 6.76×10-5 0.0038 0.00099 0.0048

0.0210 0.1237 0.0002 0.0196 0.0019 0.0056

7. Concluding Remarks A semi serial-parallel model (SSPM) identifier based upon an entropy based differential evolution fuzzy system modelling (EFSM) to identify the model for any nonlinear system without any prior knowledge of the physical relationship inside the system or/and the system behaviour is proposed in this study. The performance of the proposed modeling approach is compared against the classical Sugeno type fuzzy modeling; ANFIS modeling and Oliveira’s OBF-fuzzy approach for a bench-mark problem (see example 1). The results indicate that the proposed fuzzy model yields more accurate models than the other approaches and the generalization capability of the proposed modeling scheme is also better. Since, identification plays a critical part in the model based adaptive control technique, in-flight identification (see example 2) is carried out for a UAV. The identification results show that the method has the potential to mimic the dynamics of the UAV in flight. This example shows the numerical efficacy of the proposed technique for on-line identification of a non-linear real-time system. The generalization capability of the technique is well demonstrated with results without on-line adaption. System identification is the essential step towards the design of adaptive control system. In addition, it can also be used for many related applications such as design of simulators, path planning, and simulator data base development. Currently the research is directed towards the design of suitable controller based upon EFSM.


Actual Output

Roll, φ (deg)

10 5 0

Pitch, ϴ (deg)

0 10

5

10

15

20

25

30

35

40

45

50

5

10

15

20

25

30

35

40

45

50

5

10

15

20

25

30

35

40

45

50

0

-10 0 150 Yaw, ψ (deg)

EFSM Output

100

50 0

Time (Sec) Fig 11. The online identification of the orientation by EFSM

20

O. Hassanein et al

Vx, m/sec

2 0

Vy, m/sec

-2 0

Actual Output

EFSM Output

5

10

15

20

25

30

35

40

45

50

5

10

15

20

25

30

35

40

45

50

5

10

15

20

25

30

35

40

45

50

1 0 -1 0

Vz, m/sec

2 0 -2 0

Time (Sec)

Fig 12. The online identification of the linear velocities by EFSM

8. Reference 1. Venkatesh S R, M A Dahleh On System Identification of Complex Systems From Finit Data. IEEE Trans. Automatic Control, (2001),46: 235-257. 2. Pin-Cheng Chen, Using immune network in nonlinear system identification for 3D oarallel Robot. Info Tech J, (2009),8(6): 895-902. 3. Jonas Sjoberg, et al, Nonlinear black box modeling in system identification : a unified overview. Automatica, (1995), 31(12): 1691-1724. 4. Musa Alci, Musa H. Asyali, Nonlinear system identification via laguerre network based fuzzy system. Fuzzy sets and systems, (2009), 160: 3518-3529. 5. Li-Xin Wang, Design and analysis of fuzzy identifier of nonlinear dynamic systems, IEEE transction on autmatic control, (1995), 40(1): 11-23. 6. Lixiang Li, et al, fuzzy system identification via chaotic ant swarm. Chaos solutions and fractals, (1995), 41: 401-409. 7. Mostafa A. Khalik, et al, parameter identification problem : Real-coded GA approach. Applied mathematics and computation, (2007), 187: 1495-1501. 8. G. H. C. Oliveira, et al, fuzzy models within orthonormal basis function framwork. IEEE international fuzzy systems conference proceeding, (1999), 187: 1495-1501. 9. S E Papadakis, J B Theocharis, A GA-Based Fuzzy Modeling Approach For Generating TSK Models. Fuzzy Sets and Systems, (2001), 131: 121 – 152. 10. Qiang Shen, Alexios Chouchoulas, A rough-fuzzy approach for generating classi$cation rules. Pattern Recognition, (2002), 35: 2425 – 2438.


11. Herrera F, J L Verdegay, Studies in Fuzziness and Soft Computing. Genetic Algorithm and Soft Computing Journal, (1996), 8. 12. Salehfar H, N Bengiamin, J Huang, A Systematic Approach To Linguistic Fuzzy Modeling Based on Input-Output Data. Proceedings of the 2000 Winter Simulation Conference. 13. A. Soukkou, A. Khellaf, And S. Leulmi, Systematic Design Procedure Of Ts-Type Fuzzy Controllers. Int. J. Comp. Intel. Appl., (2006), 06, 531. 14. Xi-Zhao Wang, improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans. On fuzzy systems, (2009), 17(3) : 556-567. 15. Li-Xin Wang, Adaptive Fuzzy System and Control, Design and Stability Analysis. Prentic Hall, New Jersey, (1994). 16. Mengali G. “Fuzzy Modelling for Aircraft Dynamics Identification,” Aeronautical Journal, vol. 105, no. 1051, pp. 551-555, 2001. 17. Phan, P. A. and T. J. Gale, “Direct Adaptive Fuzzy Control with Less Restrictions on The Control Gain,” International Conference on Computational Intelligence for Modelling, Control and Automation, pp.168-172, 2006. 18. S. Das, P.N. Suganthan, Differential Evolution: A Survey of the State-of-the-Art, IEEE Transactions on Evolutionary Computation, 15 (2011) 4-31. 19. Wenyin Gong, et al, A Generalized Hybrid Generation Scheme Of Differential Evolution For Global Numerical Optimization. Int. J. Comp. Intel. Appl., (2011), 10, 35. 20. Wang L X, Mendel J M, Back-Propagation Fuzzy System As Nonlinear Dynamic System Identifires. Fuzzy Systems IEEE International Conference,(1992) 1409-1418. 21. Selami Beyhan, Musa Alci, fuzzy function based ARX model and new fuzzy basis function models for nonlinear system identification. Applied soft computing, (2010) 10: 439-444. 22. T. Asokan and M. Singaperumal “Autonomous Underwater Robotic Vehicle (AUV) project” Indian Institute of technology, Madras, January 2008. 23. Wang, L. X., “Fuzzy System are Universal Approximators,” IEEE Proceeding of International Conference on Fuzzy Systems, pp. 1163-1170, 1992. 24. Wang, L. X., (1997)“A course in fuzzy systems and control,” Prentic Hall Inc, USA. 25. Takagi, T., & Sugeno, M,. Fuzzy Ident of Sys & its Applic to Modelling & Control. IEEE Trans. on Systs, Man and Cyb, 1985,15, pp. 116-1 32. 26. Yu Chunyu, Zhang Kaili, Ma Dezhong, Zhou Zhen, Reliability Evaluation of Electromagnetic Flowmeter Based on Entropy Weight and Fuzzy Theory. Advanced Materials Research, (2012) (383-390) : 5321-5326. 27. Miin-Tsair Su, et al, A Rule-Based Symbiotic MOdified Differential Evolution for SelfOrganizing Neuro-Fuzzy Systems. Applied Soft Computing, (2011) 11: 4847–4858. 28. Shaaban A S, Sreenatha G A, Jin Y C, Indirect Adaptive Fuzzy Control of Unmanned Aerial Vehicle. In: Proceedings of the 17th World Congress, The International Federation of Automatic Control Seoul, Korea, (2008) 13229-13243. 29. S.M. Elsayed, R.A. Sarker, D.L. Essam, A three-strategy based differential evolution algorithm for constrained optimization, in: the 17th international conference on Neural information processing: theory and algorithms - Volume Part I, Springer-Verlag, Sydney, Australia, 2010, pp. 585-592. 30. S.M. Elsayed, R.A. Sarker, D.L. Essam, Differential evolution with multiple strategies for solving CEC2011 real-world numerical optimization problems, in: IEEE Congress on Evolutionary Computation, 2011, pp. 1041-1048. 31. J. Lampinen, I. Zelinka, On stagnation of the differential evolution algorithm, in: 6th Int. Mendel Conference on Soft Computing, Brno, Czech Republic, 2000, pp. 76–83. 32. J. Vesterstrom, R. Thomsen, A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, in: IEEE Congress on Evolutionary Computation, 2004, pp. 980-1987.

22

O. Hassanein et al

33. R. Storn, On the usage of differential evolution for function optimization, in: Biennial Conference of the North American Fuzzy Information Processing Society (NAFIPS), 1996, pp. 519–523. 34. Mahendra K. S., Sreenatha A., Matthew G., Neural Network Based System Identification for Autonomous Flight of an Eagle Helicopter, in: the 17th World Congress of the International Federation of Automatic Control, Seoul, Korea, 2008, pp.7421-7426. 35. Mahendra K. S., Sreenatha A., Matthew G., Neural Network based Identification and Control of an Unmanned Helicopter, in: PhD thesis, UNSW@ADFA, Australia, 2009.