Constructing optimized interval type-2 TSK neuro-fuzzy systems with

Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA Sharina Huang a,b, Minghao Chen a a b

School of Science, Harbin Institute of Technology, Harbin 150001, PR China School of Science, Heilongjiang University of Science and Technology, Harbin 150022, PR China

art ic l e i nf o

a b s t r a c t

Article history: Received 12 March 2015 Received in revised form 31 August 2015 Accepted 19 September 2015 Communicated by Manoj Kumar Tiwari

In this paper, a modified rule generation approach with self-constructing property for neuro-fuzzy system modelling is proposed. In structure identification stage, input–output patterns are divided into the clusters and interval type-2 membership functions are generated roughly. Interval type-2 Takagi– Sugeno–Kang (TSK) neuro-fuzzy structure is fine tuned by quantum inspired bacterial foraging algorithm (QBFA) in parameter identification stage to achieve higher precision, a recursive least squares (RLS) estimator is used to update consequent parameters. Comparisons with two type-1 neuro-fuzzy systems on three nonlinear functions and chaotic Mackey-Glass time series show that the proposed systems can approximate the target with little error. Experiments are also executed involving the proposed systems for modelling flue gas denitrification efficiency of a thermal power plant. It is verified by the results that interval type-2 neuro-fuzzy structure can learn knowledge from input–output data set with the aid of QBFA and hybrid training progresses are able to improve its performance. & 2015 Elsevier B.V. All rights reserved.

Keywords: Neuro-fuzzy modelling Structure identification Quantum inspired bacterial foraging algorithm Recursive least squares estimator Interval type-2 fuzzy sets

1. Introduction To cope with noisy input–output data pairs, type-2 fuzzy theory is proposed and used by some researchers to enhance the performance of neuro-fuzzy model [1]. Several type-2 fuzzy membership functions are proposed in the literature, such as, Gaussian, trapezoidal, triangular, pi-shaped, sigmoidal, etc. [2]. Gaussian-type membership functions are widely used in fuzzy system modelling, the uncertainties of the system can be associated to the membership function with two parameters, the mean and standard deviation. Footprint of uncertainty (FOU) is the union of all primary memberships. It is specified by lower and upper membership functions, which are often notated as LMF and UMF, respectively. The inclusion of FOU make the type-2 fuzzy sets have the ability to tackle higher order uncertainties [3]. Interval type-2 (IT2) fuzzy set lets its secondary membership grades are all equal to one, which is preferred by many modelling methods due to its easy calculation and simple implementation. Higher order uncertainties are usually modelled by IT2 fuzzy sets, and analytic or heuristic training algorithm is the most often used method to optimize fuzzy parameters during the learning process [4–7]. The architecture of type-1 fuzzy logic system (T1FLS) and type2 fuzzy logic system (T2FLS) is similar, the major difference between the two systems is the defuzzifier block. To the T2FLS, E-mail address: [email protected] (M. Chen).

type-reduction is applied before defuzzification. Then, a T2FLS can be divided into five parts: fuzzifier, rule base, fuzzy inference engine, type-reducer, and defuzzifier. To carry out a T2FLS, one needs to resolve the following problems, such as operations on type-2 fuzzy sets, inferencing with T2FLS and how to obtain the system output from the inference engine, it is more complex than that of T1FLS for user to adopt T2FLS as a fuzzy inference system. Interval T2FLS (IT2FLS) is a mature technology, has found many applications in literature based on IT2 fuzzy set theory [2,8–10]. To name a few, clustering systems [11,12], intelligent controllers [13– 15], vision and pattern recognition systems [16–18], autonomous mobile robots [19–21], database and information systems [22], medical systems [23], and plant monitoring and diagnostics [24,25]. Many of the above mentioned applications are characterized by input–output relation with non-linearity, high uncertainty, and time-varying property [7,26]. IT2FLS has been verified by many researchers that its a more robust system when the disturbances and uncertainties are involved [20,27,28]. A simpler IT2 TSK FLS is constructed by ellipsoidal membership function, experiments are performed and results show that the IT2 TSK FLS is more suitable for noisy environment than the coincide T1FLSs [1]. To achieve the expected accuracy of input–output non-linear mappings, fuzzy parameters are updated to meet the predesigned criterions so that the non-linear dynamic systems can be identified by IT2 fuzzy systems. There exist many learning algorithms for tuning antecedent and consequent parameters for neuro-fuzzy

http://dx.doi.org/10.1016/j.neucom.2015.09.060 0925-2312/& 2015 Elsevier B.V. All rights reserved.

Please cite this article as: S. Huang, M. Chen, Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.09.060i

2

S. Huang, M. Chen / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

systems. Back-propagation (BP) algorithm used both in TSK type and Mamdani type FLSs is the mostly used steepest descent algorithm in the parameters tuning process. Hybrid algorithm has been proposed for IT2 TSK singleton FLS [29] and for IT2 TSK nonsingleton FLS [30]. To the consequent parameter learning, recursive least-squares (RLS) is the classical recursive parameterestimation methods appeared in literature [2], orthogonal leastsquares [31] and Kalman filters [32] are presented as consequent learning mechanisms. RLS-based estimator with forgetting factors is studied in [33], this strategy tracks the time-varying parameters very well. Many hybrid algorithms have been presented for IT2 Mamdani type FLS [34,35]. In [36], a hybrid learning methodology is proposed for constructing an IT2 A2-C1 TSK FLS, and the method is also used for hot strip mill's entry temperature prediction. The term A2-C1 denotes different combinations of antecedent and consequent of a T2FLS. Where A stands for antecedent and C for consequent. Numbers 0, 1 and 2 mean its consequents are crisp numbers, type-1 and type-2 fuzzy sets, respectively. IT2 A2-C1 TSK FLS represents an IT2 TSK FLS whose antecedents and consequents are type-2 and type-1 fuzzy sets, respectively. Recently, complex problems are solved with the help of bioinspired algorithms [37]. In the case of T2FLS designing for complex problems, the structure and best parameters of the fuzzy systems can be obtained via bio-inspired optimization methods, such as genetic algorithm (GA) [38,39], particle swarm optimization (PSO) and ant colony optimization (ACO) [40,41]. In [12], T2FLS is optimized by differential evolution optimization, and an efficient rule base is obtained in T2FLS structure formation. For TSK-type neuro-fuzzy systems, an RLS-PSO learning algorithm has been presented for IT2 TSK neuro-fuzzy systems [42]. Emission control of NOx for thermal power plant will be the emphasis and difficulty of environmental pollution control in China, it is important to predict and reduce the emissions of NOx from thermal power plant. Selective catalytic reduction (SCR) denitrification is a prevalent technique of flue gas denitrification at present. The NOx sensor ammonia-cross-sensitivity factor observer is designed to measure the diesel engine SCR system's NOx density, simulations and comparisons are performed to validate the observer [43]. The fuel impaction on the characteristic of combustion, volume of gaseous emissions and practical emissions in a gasoline direct injection research engine is investigated in [44]. For two-catalyst SCR system, a control-oriented model is presented in [45]. The scheme used to optimize system identification proposed in [45] also used to optimize a linearized input–output dynamic analytical model for the clutch pressure control system [46]. Quantum inspired bacterial foraging algorithm (QBFA) [47] is a newly proposed bio-inspired optimization method which exhibits good performance when compared to GA and PSO in numerical experiments, has not been presented for IT2 TSK FLS yet. In this paper, we intend to present an RLS-QBFA-based evolutionary learning mechanism for IT2 A2-C0 singleton TSK FLS's parameters tuning. The main contributions of this work are three-fold: (1) A rule generation scheme with self-constructing property is presented to classify the input–output data into several fuzzy clusters, and the initial fuzzy rules are extracted to form an IT2 TSK FLS in the structure identification stage. (2) An IT2 TSK neuro-fuzzy model is trained by a new hybrid evolutionary learning algorithm which includes a QBFA and an RLS-based algorithm. As far as we known, there exists no work on QBFA based IT2 TSK FLS modelling. (3) Simulation results show that the proposed IT2 TSK neuro-fuzzy systems have better noise reduction property and are more robust in noisy environment compared with two type-1 fuzzy systems. The proposed methods are also used for modelling flue gas SCR denitrification efficiency of a thermal power plant, which is a new application field of IT2FLS.

The rest of the paper is organized as follows. Section 2 describes hybrid RLS-QBFA-based IT2FLS structure. Section 3 introduces self-constructing rule generation process. Section 4 presents the hybrid evolutionary learning algorithm, QBFA and RLS-based algorithms are thoroughly described. Section 5 gives simulation results with three nonlinear functions, chaotic MackeyGlass time series and modelling denitrification efficiency of a thermal power plant. The conclusions are provided in Section 6.

2. Overview of hybrid RLS-QBFA-based IT2FLS The whole structure of our approach is given in Fig. 1. There are two stages in the hybrid RLS-QBFA-based IT2FLS constructing. The first stage is structure identification, the second stage is parameter identification. In the first stage, a fuzzy rule generation scheme with self-constructing property is developed to classify the input– output data into several clusters while the cluster number is predefined, then initialized fuzzy rules are extracted to create an IT2 TSK FLS. Successively, an IT2 TSK neuro-fuzzy network can be constructed in first stage and an RLS-QBFA-based hybrid evolutionary learning algorithm is presented in second stage to optimize the fuzzy parameters for higher precision. Then, optimized fuzzy rule base is obtained. For any input pattern presented to the system, output can be calculated by the fuzzy system, also named neuro-fuzzy network. For convenience of description, we only discuss the multiple-input-single-output system in this paper. The method is not difficult generalized to multiple-input-multipleoutput system. Suppose we have a data set with K inputs x1 ; …; xK , and one output. In the first stage, fuzzy clusters ðc1 ; c2 ; …; cJ Þ can be generated by self-constructing rule generation method, and each cluster corresponds a rule. The jth fuzzy rule presents the following IT2 TSK form: Rj : IF x1 IS A~ 1j ðx1 ÞAND ⋯ AND xK IS A~ Kj ðxK Þ; THEN y IS T f~ j ðxÞ ¼ x0 bj ¼

K X

bij xi ;

ð1Þ

i¼0

where 1 r j rJ, J is the predefined rule number we wish to have, x0 ¼ 1; x0 ¼ ½x0 ; x ¼ ½x0 ; x1 ; …; xK , bj ¼ ½b0j ; b1j ; …; bKj , and A~ ij ðxi Þð1 r ir KÞ are IT2 fuzzy sets. Gaussian IT2 fuzzy sets are adopted to formulate the system's uncertainties. There are two manners to define a Gaussian IT2 fuzzy sets, uncertain mean or standard deviation (Fig. 2). We adopt the membership functions which use uncertain standard deviation to construct the fuzzy IF–THEN rules. Membership functions with UMF and LMF (Fig. 2, left) can be defined as follows: ! ðx  m Þ2 μ ij ðxi Þ ¼ exp  i 2 ij ; ð2Þ

σ ij

Fig. 1. Architecture of the IT2 TSK neuro-fuzzy structure.

Please cite this article as: S. Huang, M. Chen, Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.09.060i

S. Huang, M. Chen / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

3

Considering a system with N training patterns, let Dt ¼ ðpt ; qt Þ be an input–output training pattern, where pt ¼ ½x1 ; x2 ; …; xK  and qt is a single value. A fuzzy cluster cj can be described by ðf j ðpt Þ; f j ðpt Þ; yj Þ, where f j ðpt Þ and f j ðpt Þ are the product of K

Fig. 2. Gaussian IT2 fuzzy set with uncertain standard deviation (left) and uncertain mean (right).

μ ij ðxi Þ ¼ exp 

ðxi  mij Þ2

σ 2ij

! ð3Þ

ij

by Eqs. (2) and (3), respectively. The matching degree interval ½μ

ij

ðxi Þ; μ ij ðxi Þ (1 r i r K, 1 r j r J) is constructed. Layer 2 has J nodes. In this layer, the firing strength interval for each rule is calculated, it can be denoted as " # t

t

K

K

∏ μ ðxi Þ; ∏ μ ij ðxi Þ ;

σ ij

i¼1

;

where mij, σ ij , and σ ij ð1 r i rK; 1 r j r JÞ are the mean value, standard deviation of UMF and LMF of the jth fuzzy set for the ith input variable, respectively. Note that mij, σ ij and σ ij are antecedent parameters, b0j ; b1j ; …; bKj are consequent parameters. As a result, the J fuzzy rules generated from the corresponding clusters ðc1 ; c2 ; …; cj Þ comprise a rule base. Furthermore, an IT2 TSK neurofuzzy model is constructed from the obtained rule base. In the parameter identification stage, an RLS-QBFA-based hybrid evolutionary learning algorithm will be proposed to improve the approximation precision. From Fig. 1 we can see that the neuro-fuzzy structure has four layers, input variables are x1 ; …; xK , and output is yt. Layer 1 has J groups of nodes, each group corresponds a rule and consists of K nodes. In this layer, the UMF μ ij ðxi Þ and LMF μ ðxi Þ are calculated

½f j ; f j  ¼

Gaussian type-2 LMFs and UMFs, respectively, and yj is an onedimensional function. In the stage of self-constructing rule generation, we would need to consider the product of K Gaussian type-2 LMFs, i.e., 2 !2 3 K xi  mij 5 ; ð8Þ f j ðpt Þ ¼ ∏ exp4 

pattern Dt passes the input-similarity test on existing cluster cj, here ρ is a predefined parameter for rule generation and 0 r ρ r 1. Furthermore, if training pattern Dt passes the input-similarity test on cluster cj, we will calculate output-similarity for cluster cj and say that training pattern Dt passes the output-similarity test if j qt b0j j 9et r τðqmax  qmin Þ;

1. When j¼0 or j o J and training pattern Dt fails the inputsimilarity test or output similarity test on existing fuzzy clusters. It means that training pattern Dt is far away from any existing cluster. In this case, let j ¼ j þ 1, a new fuzzy cluster cj will be created with the following parameters: mj ¼ pt ;

σ j ¼ σ 0;

Layer 3 has two nodes which correspond the normalized lower and upper outputs, respectively. The normalized lower output f and upper output f can be presented as follows: PJ t~ j ¼ 1 f jf j f ¼ PJ ; ð5Þ t j¼1fj

b0j ¼ qt ;

b1j ¼ ⋯ ¼ bKj ¼ 0;

PJ

i¼1

f j f~ j

i¼1

t

j¼1

f ¼ P J

j¼1

t

fj

:

ð6Þ

Layer 4 has one node. The output of the neuro-fuzzy model is provided in this layer. Using Begian–Melek–Mendel (BMM) method [48], the output yt can be presented as follows: PJ PJ t t~ ~ j ¼ 1 f jf j j ¼ 1 f jf j yt ¼ mf þ nf ¼ m PJ þn ; ð7Þ PJ t t j¼1fj j¼1fj where m þ n ¼ 1.

3. Self-constructing rule generation In this section, the rule generator with self-constructing property will be provided. The given input–output patterns are classified into several clusters by similarity measure in the method. Similar to the traditional fuzzy clustering method, the association among data within a cluster is strong and data in different clusters is weak. In doing so, an IT2 TSK fuzzy rule specified by Eq. (1) is incorporated into rule base. Then, the unknown system's rough model is established by the initial fuzzy rules.

ð9Þ

where 0 r τ r 1, qmax is the maximum output and qmin is the minimum output. The two criteria result four cases, the step by step outline can be described as follows:

ð4Þ

ij

j ¼ 1; 2; …; J:

where mij denotes the input mean, and σ ij the input deviation. The number of maximum fuzzy clusters is denoted by J, Sj is the existing cluster size of cj at the moment, j will be 0 initially. For the training pattern Dt ¼ ðpt ; qt Þ, if f j ðpt Þ Z ρ, we say that training

σ j ¼ σ 0; ð10Þ

where mj ¼ ½m1j ; m2j ; …; mKj , σ j ¼ ½σ 1j ; σ 2j ; …; σ Kj , σ j ¼ ½σ 1j ; σ 2j ; …; σ Kj , σ 0 ¼ ½σ 0 ; …; σ 0  and σ 0 ¼ ½σ 0 ; …; σ 0  are two user

predefined constant vectors. The size of cluster cj should be defined as Sj ¼1. 2. When j Z1, and there exist some existing clusters such that training pattern Dt passes both the input-similarity test and the output-similarity test. Let clusters cj1 ; cj2 ; …; cjf be such existing clusters. For the product of Gaussian type-2 LMFs, let cv be the cluster with the largest membership degree, i.e., f v ðpt Þ ¼ maxff j ðpt Þ; f j ðpt Þ; …; f j ðpt Þg: 1

2

ð11Þ

f

It means that training pattern Dt is closest to cluster cv. In this situation, the mean and deviations of vth membership functions should be modified, and training pattern Dt should be included in cluster cv. The modification of cluster cv can be formulated as mv ¼

Sv mv þ pt ; Sv þ 1

σ v ¼ kσ σ 0 ; b0v ¼

ð12Þ

σ v ¼ kσ σ 0 ;

Sv b0v þ qt ; Sv þ 1

Sv ¼ Sv þ1;

b1v ¼ ⋯ ¼ bKv ¼ 0;

ð13Þ ð14Þ

where mv ¼ ½m1v ; m2v ; …; mKv , kσ ðkσ 4 1Þ and kσ ðkσ 4 1Þ are two predefined constants. 3. When jf  J, f V ðpt Þ Z ρ; et 4 τðqmax  qmin Þ, it means training pattern Dt passes the input-similarity test but fails the outputsimilarity test. In this case, we need to modify the consequent

Please cite this article as: S. Huang, M. Chen, Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.09.060i

S. Huang, M. Chen / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

4

parameter as Sv b0v þqt ; b0v ¼ Sv þ 1

ð15Þ

and Sv ¼ Sv þ1, since J clusters have been established and no more cluster is needed. 4. When jf  J, f v ðpt Þ o ρ, it means training pattern Dt has not passed the input-similarity test, but there is no room to increase the cluster number. In this case, the mean and deviations of vth membership functions should be updated as follows: mv ¼ pt ; and

σ v ¼ kσ σ 0 ;

σ v ¼ kσ σ 0 ;

ð16Þ

Sv ¼ Sv þ1.

The above learning process should be iterated till all the input– output training patterns have been trained. Finally, prescribed J fuzzy clusters are generated and each cluster cj can be characterized by ðf j ðpt Þ; f j ðpt Þ; yj Þ, where f j ðpt Þ is a function of mean vector mj and deviation vector σ j , f j ðpt Þ is a function of mean vector mj and deviation vector σ j , and yj is a function of constant coefficient b0j.

4. Hybrid evolutionary learning algorithm In section 3, the J initial fuzzy rules have been generated, and a four-layer neuro-fuzzy evolutionary modelling has been established (Fig. 1). In this section, parameter identification stage employs an RLS-QBFA-based hybrid evolutionary learning algorithm to optimize the parameters mij ; σ ij ; σ ij , and bkj ð1 r i rK; 1 r j r J; 0 rk rKÞ. In particular, the antecedent parameters mij ; σ ij and σ ij are optimized by QBFA. To obtain consequent parameters bkj, singular value decomposition (SVD) and recursive SVD (RSVD)-based least squares estimators are adopted. Fig. 3 shows the structure of the hybrid evolutionary learning algorithm. Suppose total number of training patterns is N. In each iteration of the algorithm, all of the training patterns are involved. We employ the QBFA to search optimal or suboptimal parameters (in the term of bacterial foraging, the optimal bacterium represents optimal mean and deviations while each bacterium represents a candidate solution of antecedent parameters). For each bacterium, optimized consequent parameters can be obtained by SVD or RSVD-based least squares method. The QBFA and SVD (RSVD) will be carried out alternately until the desired approximation precision or the predefined maximum iteration is achieved. In the next, the antecedent parameters tuning algorithm QBFA and consequence training algorithms are introduced. 4.1. Antecedent parameter tuning Bacterial foraging algorithm (BFA) is a relative newly proposed algorithm which simulates the E. coli bacterial's behaviour of searching for nutrient in animal's intestines. QBFA [47] was firstly proposed for high-dimensional optimization problems. Several quantum computing principles are used in QBFA. The algorithm exhibits good performance when compared to GA and PSO in

Fig. 3. The structure of the hybrid evolutionary learning algorithm.

numerical experiments [47]. In the antecedent parameter identification process, mean vector and deviation vectors are concatenated which represents an individual in the QBFA. The algorithm designed for antecedent parameters optimal value searching can be depicted by the following steps: [Step 0]: Initialization parameters L, Nb, θi , PQ(t), Nc, Ns, Nre, Ned, Ped, and IterMax. Where L denotes the search space dimension, Nb is the number of population, θi is the rotation angle during the life span when bacterium moves to optimal solution, PQ(t) is a Q-bit string and contains Nb elements, each element can be denoted as a 2  L matrix which is determined by θi , Nc, Ns, and Nre are the maximum iteration steps of chemotaxis, swimming, and reproduction, respectively, Ned and Ped are the maximum iteration step and the probability of elimination-dispersal, respectively, and IterMax represents the maximum iteration number. [Step 1]: Convert the Q-bit string PQ(t) to initial position PP(t). Check the condition number of AðtÞ which is given in (22) or (30), if condition number is too small then make a new observation of PQ(t) and generate a new position PP(t), save the best position Pbest and the best fitness value Fbest. [Step 2]: Elimination-dispersal loop: n ¼1,…, N ed : [Step 3]: Reproduction loop: m ¼ 1; …; Nre : [Step 4]: Chemotaxis loop: l ¼ 1; …; N c : (a) For bacterium k ðk ¼ 1; …; Nb Þ, Q-bit individuals of PQ(t) are updated clockwise or anti-clockwise according to the current situation and check the condition number of AðtÞ which is given in (22) or (30). (b) Compute fitness value Fðk; l; m; nÞ and save the fitness value as Flast, i.e. F last ¼ Fðk; l; m; nÞ. (c) Compute chemotactic step size C(k) and generate a new position as follows:

θk ðl þ 1; m; nÞ ¼ θk ðl; m; nÞ þCðkÞðθb ðl; m; nÞ  θk ðl; m; nÞÞ; where θ ðl; m; nÞ is the position of kth bacterium at lth chemotaxis mth reproduction nth elimination-dispersal step, and θb ðl; m; nÞ is the global best of the population found so far. (d) Swim: (i) Set S ¼1. (ii) While S o N s . (iii) If Fðk; l þ1; m; nÞ o F last , let F last ¼ Fðk; l þ 1; m; nÞ, generate a new position along the previous direction as follows: k

θk ðl þ 1; m; nÞ ¼ θk ðl þ 1; m; nÞ þ CðkÞðθb ðl; m; nÞ  θk ðl; m; nÞÞ; and let S ¼ S þ 1. (iv) Else, let S ¼ N s , if F last o F best , update the Fbest and Pbest, end the while loop. (e) If k o N b , let k ¼ k þ 1, and go to (a). [Step 5]: If l oN c , let l ¼ l þ 1, and go to step 4. [Step 6]: For the given m and n, compute the health status for P each bacterium. Let F khealth ¼ lN¼c þ11 Fðk; l; m; nÞ be the health status of kth bacterium. Sort bacteria according to health status, lower cost means higher health. The bacteria which have the better health status will be saved and have the chance to generate the next generation. [Step 7]: If m o N re , let m ¼ m þ 1 and go to step 3. [Step 8]: For k ¼ 1; 2; …; N b , eliminate and disperse the bacterium with probability Ped. If a bacterium is eliminated, simply disperse one to a location on the domain with the quantum operation. If n oN ed , let n ¼ n þ 1 and go to step 2, otherwise the algorithm will be terminated.

Please cite this article as: S. Huang, M. Chen, Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.09.060i

S. Huang, M. Chen / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

The flowchart of QBFA is depicted in Fig. 4 based on the above descriptions. The algorithm is used to reduce the error between the network output and desired output as small as possible. Two error criteria adopted in our work are defined as follows: MSE ¼

N 1X ðq  yt Þ2 ; Nt¼1 t

ð17Þ

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi uPN u ðq  yt Þ2 NRMSE ¼ t PtN¼ 1 t ; 2 t ¼ 1 ðqt  qÞ

ð18Þ

where qt is the desired output, q is the average of q1 ; …; qN , and yt is the network output. 4.2. Consequent parameter tuning For the given training pattern, network output has the following form with BMM type reduction method: 0 1 PJ PJ t t~ t t ~ J fj X fj j ¼ 1 f jf j j ¼ 1 f jf j @m P A yt ¼ m PJ þn P þ nP t ¼ t t t J J J j¼1 j¼1fj j¼1fj j¼1fj j¼1fj f~ j ¼

J X j¼1

f j f~ j ¼

J X j¼1

fj

K X k¼0

xk bkj ¼

J X

f j p0t bj 9 at X; t

j¼1

ð19Þ

5

where m þn ¼ 1, p0t ¼ ½x0 ; pt  ¼ ½1; x1 ; …; xK , bj ¼ ½b0j ; b1j ; …; bKj , X ¼ ½b1 ; …; bj T , at ¼ ½f 1 ; f 2 ; …; f j   p0t and  denotes the Kronecker product. For all the N training patterns, we have the following compact form criterion: JðXÞ ¼ J B  AX J ;

ð20Þ

where B; A and X are matrices with proper dimension, and B ¼ ½q1 ; q2 ; …; qN T A RN1 ;

ð21Þ

A ¼ ½aT1 ; …; aTN T A RNJðK þ 1Þ ;

ð22Þ

X ¼ ½b1 ; …; bJ T A RJðK þ 1Þ1 :

ð23Þ

We would like JðXÞ to be as small as possible, in the following we introduce the SVD-based least square estimator based on BMM type reduction method. IT2FLS1(SVD-QBFA): This method IT2FLS1(SVD-QBFA) employs the SVD least square estimator to estimate the weight of the last layer of neuro-fuzzy model. For a matrix A specified by (22), which can be decomposed as A ¼ U ΣV T ;

ð24Þ JðK þ 1ÞJðK þ 1Þ

NN

where V A R and U A R are orthonormal matrices, Σ A RNJðK þ 1Þ , and the component Σij with entry (i,j) of Σ satisfies ( 0; if i a j; ð25Þ Σij ¼ ei ; otherwise; where ei2 are eigenvalues of At A or AAt , and e1 Z e2 Z ⋯ Z er with r ¼ minðN; JðK þ 1ÞÞ. Furthermore, Σ can be rewritten as " 0#

Σ¼

Σ

;

0

ð26Þ

0

where Σ A RhJðK þ 1Þ and h denotes the rank of A. Let " # B0 UT B ¼ ; B″

ð27Þ

where B0 A Rh1 and B″ A RðN  hÞ1 . By (24), (20) can be rewritten as JðXÞ ¼ J B  U ΣV T X J ¼ J U T B  ΣV T X J " # " 0# " # 0 B0 Σ B0  Σ Y ; ¼  Y ¼ B″ 0 B″

J

JJ

J

ð28Þ

where Y ¼ V T X, and the second equation is induced by U is an orthonormal matrix. Apparently, (28) is minimized by Y n when 0 B0 ¼ Σ Y n . Thus, the optimal solution X n which minimizes (20) is X n ¼ VY n :

ð29Þ

IT2FLS2(RSVD-QBFA): If a large number of training patterns are involved, SVD least squared method will be memory demanding and time consuming. Thus, RSVD in [49] is used as the least square estimator by IT2FLS2(RSVD-QBFA). Based on BMM method, we proposed an RSVD-based estimator in this paper. In each iteration, the estimator only need to decompose a small matrix, and it saves runtime and memory space compared with SVD-based estimator. Let 2 T3 2 3 q1 a1 6 T7 6 7 6 a2 7 6 q2 7 7 6 7 AðtÞ ¼ 6 ð30Þ 6 ⋮ 7; BðtÞ ¼ 6 ⋮ 7; 4 5 4 5 T qt at

Fig. 4. Flowchart of the QBFA.

for t ¼ 1; 2; …; N. In the method, training patterns are considered one by one, start with the first training pattern t¼1 and end with the last training pattern t ¼N. When the tth training pattern

Please cite this article as: S. Huang, M. Chen, Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.09.060i

S. Huang, M. Chen / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

6

arrived ðt ¼ 1; 2; …; NÞ, the following optimization problem min JðXðtÞÞ ¼ J BðtÞ  AðtÞXðtÞ J

ð31Þ

is solved to obtain the optimal value XðtÞ. Apparently, the optimal X n ðNÞ can be obtained at Nth iteration. According to the theorem in [49], optimizing (31) is equivalent to minimizing 0 J^ ðXðtÞÞ ¼ J B0 ðtÞ  Σ ðtÞV T ðtÞXðtÞ J ; 0

ð32Þ

where XðtÞ, B ðtÞ; Σ ðtÞ, and V ðtÞ satisfy 8 Að1Þ ¼ Uð1ÞΣð1ÞV T ð1Þ; > >