Download PDF - Science Direct

Proceedings of the 20th World The International Federation of Congress Automatic Control The International Federation of Congress Automatic Control Proceedings of the 20th9-14, World Toulouse, France, July 2017 Available online at www.sciencedirect.com Toulouse, France,Federation July 9-14, 2017 The International of Automatic Control Toulouse, France, July 9-14, 2017

ScienceDirect

IFAC PapersOnLine 50-1 (2017) 452–457 An Unstructured Flexible Nonlinear Model An Unstructured Flexible Nonlinear Model for Cascaded Water-tanks Benchmark An the Unstructured Flexible Nonlinear Model for the Cascaded Water-tanks Benchmark for theRishi Cascaded Water-tanks Benchmark Relan ∗ Koen Tiels ∗ Anna Marconato ∗

∗ ∗ Anna ∗ ∗ Marconato ∗ Rishi Relan ∗∗ Johan Koen Tiels Schoukens ∗ ∗ ∗ ∗ Rishi Relan ∗ Johan Koen Tiels Anna Marconato Schoukens ∗ Johan Schoukens ∗ Department ELEC, Vrije Universiteit Brussel (VUB) ∗ ∗ Department ELEC, Vrije Universiteit Brussel (VUB) 1050∗ Brussels, Belgium (e-mail: [email protected]) Department ELEC,(e-mail: Vrije Universiteit Brussel (VUB) 1050 Brussels, Belgium [email protected]) 1050 Brussels, Belgium (e-mail: [email protected]) Abstract: Many real world systems exhibit a quasi linear or weakly nonlinear behavior during Abstract: Many real systems exhibit a quasi or weakly nonlinear behavior during normal operation, andworld a hard saturation effect for linear high peaks of the input signal. A typical Abstract: real exhibit a quasi or weakly nonlinear behavior during normal operation, andworld a hard effect for linear high peaks of the input signal. A typical example of Many such systems is systems thesaturation cascaded water-tanks benchmark. This benchmark combines normal operation, and a hard saturation effect for high peaks of the input signal. A typical example of such systems is the cascaded water-tanks benchmark. This benchmark combines soft and hard nonlinearities to be identified based on relatively short data records. In this example of such systems is the cascaded water-tanks This benchmark soft and hard nonlinearities to be based onbenchmark. relatively In this paper, a methodology to identify anidentified unstructured flexible nonlinearshort statedata spacerecords. modelcombines (NLSS) soft and hard nonlinearities to be based on relatively short data In this paper, a methodology to identify anidentified unstructured flexible nonlinearof state spacerecords. modelstructure (NLSS) for the cascaded water-tanks benchmark is proposed. The flexibility the NLSS model paper, a methodology to identify an different unstructured flexible nonlinear state spacemodel model (NLSS) fordemonstrated the cascaded water-tanks benchmark is proposed. The flexibility the NLSS structure is by introducing two initialisation schemes.ofFurthermore the strengths for the cascaded water-tanks benchmark is proposed. The flexibility ofFurthermore thetoNLSS model structure is demonstrated by of introducing two different schemes. the strengths and short-comings the model structure areinitialisation discussed with respect the cascaded wateris demonstrated by introducing two different initialisation schemes. Furthermore the strengths and short-comings of the model structure are discussed with respect to the cascaded waterbenchmark identification problem. and short-comings of theproblem. model structure are discussed with respect to the cascaded waterbenchmark identification © 2017, IFACidentification (International problem. Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. benchmark Keywords: System identification; Nonlinear state space model; Short-data records; Soft and Keywords: System identification; Nonlinear state space model; Short-data records; Soft and hard nonlinearities; Nonlinear systems. Keywords: System identification; Nonlinear state space model; Short-data records; Soft and hard nonlinearities; Nonlinear systems. hard nonlinearities; Nonlinear systems. 1. INTRODUCTION et al. (2002); Rasmussen and Williams (2006); Hastie et al. et al. (2002); Rasmussen and Williams (2006); Hastiemethet al. 1. INTRODUCTION (2009); Suykens et al. (2012)), but most of these 1. INTRODUCTION et al. (2002); Rasmussen and Williams (2006); Hastie et al. (2009); Suykens et al. (2012)), but most of these methods are typically not specifically developed to deal with There is an evident need of good system modelling techetnot al. (2012)), mostfor of to these ods are Suykens typically specifically developed dealmethdynamics and often have limitedbut means dealing with There evident need ofofgood system modelling tech- (2009); niques isinanmany branches engineering. Mathematical areWithin typically specifically developed to deal with There isin evident need ofofgood system dynamics and often have limited means for dealing noise. thenot system identification community two niques branches engineering. (linear oranmany nonlinear) models are needed modelling inMathematical varioustechap- ods dynamics and often have limited means for dealing niques in many branches of engineering. Mathematical noise. Within the system identification community two (linear or nonlinear) models are needed in various apcan plications, for example, to understand and analyse the major approaches to nonlinear system identificationwith noise. Within the system identification community two (linear or nonlinear) models are needed in various apmajor approaches to nonlinear system identification can be distinguished: black-box nonlinear system identification plications, for example, to understand and analyse the system under test, to simulate or predict the behavior approaches to nonlinear system identification can plications, for during example, todesign understand andtothe analyse the major be distinguished: black-box nonlinear system identification (Sjöberg et al. (1995), Billings (2013)) and block-oriented system under test, tothe simulate or predict behavior of the system phase or design and be distinguished: black-box nonlinear system identification system under test, to simulate or predict the behavior (Sjöberg et al. (1995), Billings (2013)) and block-oriented of the system during the designidentification phase or to provides design and implement a controller. System us system identification (Giri and Bai (2010), Mzyk (2013)). et al. (1995),(Giri Billings and Mzyk block-oriented of thea variety system during the design phase or to design and system identification and (2013)) Bai (2010), (2013)). implement a controller. System identification provides us (Sjöberg with of methods to derive accurate mathematical State-space models are general representations that alidentification (Giri and Bai (2010), Mzyk (2013)). implement a of controller. System identification provides with a variety ofthe methods to derive accurate mathematical descriptions underlying system, based on a set us of system State-space models are general representations that allow one to describe a variety of systems. In particular, with a varietyofmeasurements. ofthe methods to derive accurate mathematical descriptions underlying system, based on a set of State-space input/output models are general representations that allow one to describe a variety of systems. In particular, descriptions ofmeasurements. the underlying system, based on a set of nonlinear state-space modeling represents a promising, input/output low one tostate-space describe a modeling variety ofrepresents systems. In particular, nonlinear a promising, and at the same time challenging, class of techniques. input/output measurements. 1.1 Nonlinear System Identification nonlinear modeling promising, and at paper, thestate-space same challenging, class ofa techniques. In this we time focus mainly represents on black-box identifica1.1 Nonlinear System Identification and at the same time challenging, class of techniques. In this paper, we focus mainly on black-box identification of nonlinear state space model (NLSS) structures 1.1 Nonlinear Identification In this paper, we focus mainly on black-box identificaThe recent yearsSystem have witnessed the shift from linear sys- tion of nonlinear state space model (NLSS) structures (Paduart et al. (2010); Schön et al. (2011)). The focus of The identification recent years have witnessed shift from linear sys- tion of nonlinear stateSchön spaceofetmodel (NLSS) structures (Paduart etisal. al. initialization (2011)). The focus of tem (Ljung (1998);the Pintelon and Schoukens this paper the(2010); application two schemes The recent have witnessed the shift from linear sys- (Paduart tem identification (Ljung (1998); Pintelon and Schoukens etisal. al.state-space (2011)). The focusfor of this paper the(2010); application ofettwo initialization schemes (2012); Vanyears Overschee and De Moor (2012)) to nonlinfor the identification ofSchön nonlinear models tem system identification (Ljungand (1998); Pintelon andthe Schoukens paper is the application of two initialization schemes (2012); Vanidentification Overschee De Moor (2012)) to need nonlinfor the identification of nonlinear state-space models for ear methods, driven by to this the cascaded water-tanks benchmark problem (Schoukens (2012); Van Overschee and Deeffects Moor tosystems. nonlinthe identification nonlinear state-space models for ear system methods, driven by the need to for the cascaded water-tanks benchmark problem (Schoukens capture the identification inherent nonlinear of(2012)) real-life et al. (2016)) and testof their suitability and performance ear system identification methods, driven by thethe need to the capture thesystem inherent nonlinear effects of real-life systems. cascaded benchmark (Schoukens et al. (2016)) and test their suitability performance Nonlinear identification constantly faces chalto capture thewater-tanks dynamical behavior of problem theand cascaded watercapture thesystem inherent nonlinear effects ofofreal-life systems. al. benchmark (2016)) test their suitability performance Nonlinear identification constantly faces the chal- et to capture the and dynamical behavior of theand cascaded waterlenge of deciding between the flexibility the fitted model tanks problem. Nonlinear systembetween identification constantly faces themodel capture the dynamical lenge of parsimony. deciding the flexibility the fitted tanks benchmark problem.behavior of the cascaded waterand its Flexibility refers to ofthe ability ofchalthe to is organized as follows: Section 2 introduces lengeits oftoparsimony. deciding the flexibility theability fitted model tanks paper benchmark problem. and Flexibility refers to ofthe of the This model capturebetween complex nonlinearities, while parsimony This paper iswater-tanks organized benchmark as follows: Section 2 introduces the cascaded the identification and its parsimony. Flexibility refers to the ability of the model to capture complex nonlinearities, while parsimony is its ability to possess a low number of parameters. This paper is organized as follows: and Section 2 introduces the cascaded water-tanks benchmark and the identification briefly. model to capture while parsimony challenges associated with this benchmark problem is its ability to complex possess anonlinearities, low number of identification parameters. A general framework for nonlinear system the cascaded water-tanks benchmark and the identification challenges associated with this benchmark problem briefly. Section 3 describes the nonlinear modelling approach usis general itsnot ability to possess a low number of identification parameters. A framework for nonlinear system does exist (Giannakis and Serpedin (2001)), however, challenges associated with this benchmark problem briefly. Section 3 describes the nonlinear modelling approach using the NLSS model structures used in this paper. The A general framework for nonlinear system identification does not exist (Giannakis andis Serpedin (2001)), however, modeling nonlinear systems covered in different fields Section 3 describes the nonlinear modelling approach using the NLSS of model structures used in thistwo paper. The identification NLSS model along with different does not exist (Giannakis and Serpedin (2001)), however, modeling nonlinear systems is covered in different fields like statistical learning and machine learning (Suykens ing the NLSS model structures used in this paper. The of NLSS model along with 4two initialisation schemes is described in Section anddifferent Section modeling nonlinear systems covered learning in different fields identification like statistical learning and ismachine (Suykens The corresponding author can be contacted at the identification of NLSS model along with two initialisation schemes is described in Section 4 anddifferent Section 5 respectively. Section 6 gives provides an overview of like statistical learning and machine learning (Suykens initialisation schemes is described in Section and Section The corresponding author can wasbe supported contactedin part at the ([email protected]). This work by 5therespectively. Section 6 gives provides an4 overview of final objective functions, which are minimised using ([email protected]). ThisFund work in part by corresponding author can be supported contacted at the theThe IWT-SBO BATTLE 639, forwas Scientific Research (FWO5the respectively. Section 6 gives provides an overview of final objective functions, which are minimised using two different initialisation schemes. Section 7 gives an the IWT-SBObyBATTLE 639, Fund Scientific Research (FWO([email protected]). This workforwas supported in by Vlaanderen), the Flemish Government (Methusalem), thepart Belgian the final objective functions, which are minimised using two different initialisation schemes. Section 7 gives an introduction to the experimental set-up as well as the Vlaanderen), byBATTLE the Flemish Government (Methusalem), the Belgian the IWT-SBO Fund for Scientific (FWOGovernment through the 639, Inter university Poles of Research Attraction (IAP two different to initialisation schemes. Section 7 gives an introduction the experimental as well as measurement methodology used forset-up the acquisition of the Government through thethe Inter university Poles of Attraction (IAP Vlaanderen), by and the Flemish Government (Methusalem), the Belgian VII) Program, by ERC advanced grant SNLSID, under introduction to the experimental set-up as well as the measurement methodology used for the acquisition of the VII) Program, and by ERC advanced grant SNLSID, under Government through thethe Inter university Poles of Attraction (IAP contract 320378. measurement methodology used for the acquisition of the contract 320378.and by the ERC advanced grant SNLSID, under VII) Program,

contract 320378. contract 320378. Copyright © 2017 IFAC 454 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2017, 2017 IFAC 454Hosting by Elsevier Ltd. All rights reserved. Peer review©under of International Federation of Automatic Copyright 2017 responsibility IFAC 454Control. 10.1016/j.ifacol.2017.08.074

Proceedings of the 20th IFAC World Congress Rishi Relan et al. / IFAC PapersOnLine 50-1 (2017) 452–457 Toulouse, France, July 9-14, 2017

signals. Results are presented in Section 8, and finally, the conclusions are given in Section 9. 2. CASCADED WATER TANKS SYSTEM In this section, we introduce very briefly the cascaded water-tanks system benchmark problem and state the nonlinear identification challenges associated with it. 2.1 System

453

following input-output model can be constructed based on Bernoulli’s principle and conservation of mass: x˙1 (t) = −k1 x1 (t) + k4 u(t) + w1 (t), (1) (2) x˙2 (t) = k2 x1 (t) − k3 x2 (t) + w2 (t), (3) y(t) = x2 (t) + e(t) where u(t) is the input signal, x1 (t) and x2 (t) are the states of the system, w1 (t), w2 (t) and e(t) are the additive noise sources and k1 , k2 , k3 , and k4 are the constants depending on the system properties. The stochastic nature of the overflow can be captured by the process noise w1 (t), to some extent depending upon the input flowrate. 2.2 Identification Challenges The major nonlinear system identification challenges associated with the water-tanks benchmark are listed below: (1) the hard saturation nonlinearity combined with the weakly nonlinear behavior of the system in normal operation, (2) the overflow from the upper to the lower tank, this effect also introduces input dependent process noise, (3) the relatively short estimation data record, (4) the unknown initial values of the states.

Fig. 1. The water is pumped from a reservoir in the upper tank, flows to the lower tank and finally flows back into the reservoir. The input is the pump voltage, the output is the water level of the lower tank. The cascaded tanks system is a liquid level control system consisting of two tanks with free outlets fed by a pump. The input signal controls a water pump that pumps the water from a reservoir (considered here as an ideal reservoir which is able to provide enough water) into the upper water tank. The water of the upper water tank flows through a small opening into the lower water tank, and finally through a small opening from the lower water tank back into the reservoir. This process is shown in Figure 1. The relation between (1) the water flowing from the upper tank to the lower tank and (2) the water flowing from the lower tank into the reservoir are weakly nonlinear functions. However, when the amplitude of the input signal is too large, an overflow can happen in the upper tank, and with a delay also in the lower tank. When the upper tank overflows, a part of the water goes into the lower tank, the rest flows directly into the reservoir. This effect is partly stochastic, hence it acts as an inputdependent process noise source. Fig.2 shows the input-

In the next section, we introduce the nonlinear state space model structure and discuss two different ways to represent it. Later in the paper, the procedure to identify these two different nonlinear state space model structures from input-output measurements of cascaded water-tanks benchmark will be discussed. 3. NONLINEAR STATE SPACE Physical interpretation of the system under test is not always required, for instance in control or prediction problems. In that case, the user prefers a flexible and an easyto-initialize black-box model. Moreover, the model should preferably be able to describe Multiple-Input MultipleOutput (MIMO) systems in a compact way. A good base for such a model is a state space representation of the system under consideration. A general nth x order discrete-time state space model is described by the following equations: x(t + 1) = f (x(t), u(t)) y(t) = g(x(t), u(t)) (4) nu with u(t) ∈ R the vector containing the nu inputs at time t, and y(t) ∈ Rny the vector containing the ny outputs. The state vector x(t) ∈ Rnx represents the memory of the dynamical system. The theoretical analysis for studying the equivalence between the physical continuous time system and this model structure is out of the scope of this paper. 3.1 Polynomial Nonlinear State-Space Models

Fig. 2. Block diagram with respective input (pump actuator) and output (height of the second tank) respectively output block diagram for the water-tanks system shown in Fig.1. Without considering the overflow effect, the 455

A nonlinear state space model (where f (·), g(·) are approximated by polynomial basis functions) is termed as a Polynomial Nonlinear State-Space (PNLSS). The PNLSS model structure (Paduart et al. (2010)) is described as: x(t + 1) = Ax(t) + Bu(t) + Eζ(t) y(t) = Cx(t) + Du(t) + F η(t) + e(t) (5) The coefficients of the linear terms in x(t) ∈ Rnx and u(t) ∈ Rnu are given by the matrices A ∈ Rnx ×nx and

Proceedings of the 20th IFAC World Congress 454 Rishi Relan et al. / IFAC PapersOnLine 50-1 (2017) 452–457 Toulouse, France, July 9-14, 2017

B ∈ Rnx ×nu in the state equation, C ∈ Rny ×nx and D ∈ Rny ×nu in the output equation. The vectors ζ(t) ∈ Rnζ and η(t) ∈ Rnη contain nonlinear monomials in x(t) and u(t) of degree two up to a chosen degree P and e(t) is the measurement noise. The coefficients associated with these nonlinear terms are given by the matrices E ∈ Rnx ×nζ and F ∈ Rny ×nη . 3.2 Nonlinear State-Space Model-2 Another way (hereinafter termed as NLSS2) of describing the nonlinear state space model structure in (4) is by describing it in the form as described in (Marconato et al. (2014)): x(t + 1) = Ax(t) + Bu(t) + fN L (x(t), u(t)) y(t) = Cx(t) + Du(t) + gN L (x(t), u(t)) + e(t) (6) where A ∈ Rnx ×nx and B ∈ Rnx ×nu in the state equation, C ∈ Rny ×nx and D ∈ Rny ×nu in the output equation. The nonlinear functions fN L (·) and gN L (·) have nx and ny outputs respectively. The next section discusses the steps involved in the identification of PNLSS and NLSS2 models from input-output measurements 4. IDENTIFICATION OF THE NLSS The identification procedure for the NLSS model in Section 3 consists of three major steps. The structure of the black-box state space model given in (5) or (6) lends itself to an efficient, three steps identification procedure. (1) First, initial estimates of the A, B, C and D matrices are obtained. In order to do so, first, a nonparametric estimate of the system’s frequency response function (FRF) is determined in mean square sense. Then, a parametric linear model (linear subspace A, B, C, and D matrices) is estimated from this nonparameteric Best Linear Approximation (BLA). (2) Second, the subspace estimates are optimised in maximum likelihood sense by applying a nonlinear minimisation routine. Thereafter dependending on the model structure used, the model is initialised either with an estimate of only the linear subspace A, B, C and D matrices (PNLSS model structure) or A, B, C, and D along with an estimate of nonlinear functions fN L and gN L (NLSS2 model structure). (3) The last step consists in estimating the full nonlinear model by using again a nonlinear search routine namely the Levenberg-Marquardt method (Levenberg (1944); Nocedal and Wright (2006)). Bounded input-bounded output (BIBO) stability is required for this optimization procedure. Note that the proposed approach targets systems for which the dynamics can be captured by the BLA and systems that are assumed to have only one equilibrium point. In the subsections below, these steps as well as the framework involved in these steps are described briefly.

Within the above described set up, the nonparameteric BLA can be calculated using either the Fast or the Robust method explained in (Pintelon and Schoukens (2012)), or the Local Polynomial Method (LPM) (Pintelon and Schoukens (2012); Schoukens et al. (2009)). 4.2 Parametric BLA A parametric model is more convenient for simulation and control purposes. It offers extra opportunities to better understand the system behavior using the polezero representation. Hence, our next goal is to obtain a parametric model of the system under consideration. Thus, using the nonparametric FRF estimate (GBLA ) and its 2 variance (σG ), which is found in the previous step BLA using the LPM approach (Pintelon and Schoukens (2012); Schoukens et al. (2009)), we estimate a parametric model of our system by solving a nonlinear weighted least squares (NLWLS) problem. This model (discrete-time) describes the system as a rational transfer function. The model considered here is a rational function in the backward shift operator q −1 : −1 −2 −nb ˆ BLA (q, θtf ) = b0 + b1 q + b2 q + ...... + bnb q , G a0 + a1 q −1 + a2 q −2 + ...... + ana q −na (8) The parameter vector θtf ∈ R(nb +na +2)×1 contains the parameters [a0 , a1 , . . . , ana , b0 , b1 , . . . , bnb ]T . Since one parameter can be chosen freely because of the scaling invariance of the transfer function, only nb + na + 1 independent parameters need to be estimated by minimizing the following NLWLS cost function:

Vtf (θtf ) =

F ˆ BLA (q, θtf )|2 |GBLA (jωk ) − G

k=1

2 σG (jωk ) BLA

,

(9)

2 where σG (jωk ) includes both noise and nonlinear disBLA tortion. The order of the parametric model in (8) can for example be determined using a signal theoretic measure such as the minimum description length (MDL) criterion (see page no. 439 of Pintelon and Schoukens (2012)). This NLWLS framework also guarantees the lowest possible uncertainty on the model parameters, i.e. the efficiency of the estimates (Pintelon and Schoukens (2012)). Thereafter, a balanced state space realization Gss for the stable portion ˆ BLA (q, θtf ) is calculated, where the of the linear system G subscript ss stands for the state space. For stable systems, this is an equivalent realization for which the controllability and observability Gramians are equal and diagonal (Moore (1981); Laub et al. (1987)). The resulting estimate ˆBLA , CˆBLA and of the state space matrices are AˆBLA , B ˆ DBLA , which is then used further in the NLSS model structure for the initialisation of the linear part.

5. INITIALISATION OF THE NLSS MODEL

4.1 Best Linear Approximation Definition 1. The Best Linear Approximation (BLA) of a nonlinear system is defined as the model G belonging to the set of linear models G, such that GBLA = arg min Eu |y(t) − Gu(t)|2 (7) G∈G

456

As explained earlier, the main difference between the two approaches discussed in Section 3 lies in the way, in which the full model structure in equations (4) is initialised. In this section, we give an overview of the two different approaches followed in this paper.


5.1 Initialisation approach I In the first approach, the linear part of the PNLSS model structure (5) is initialised using the BLA i.e the ˆBLA , CˆBLA , D ˆ BLA matrices obtained in the preAˆBLA , B vious step. The coefficients of matrices E and F are initialised as numerical value 0, but it is possible to select which of these coefficients are free during the final optimisation step by selecting e.g. either all, none or no crossterms. x(t + 1) = Ax(t) + B u(t) + Eζ(t) y(t) = C x(t) + Du(t) + F η(t) + e(t) (10) 5.1.1. PNLSS-I Model (Estimation of Initial State x0 ) For both periodic and non-periodic excitations, there are two ways to estimate explicitly the initial state x0 either to include it as an extra parameter in Levenberg-Marquardt optimization or to estimate the initial conditions x0 , an extra column in the state space matrix B and an extra value 1 in the input vector is included (as the ordinary model parameters, Vanbeylen and Van Mulders (2014)). Hereinafter whenever the initial condition is explicitly estimated, then the model is termed as PNLSS-I. For both PNLSS and PNLSS-I, we choose a 3rd order model and the polynomial degrees upto 3.

If the state sequence x(t) would be exactly known, the problem of obtaining a nonlinear model could be solved much more easily by estimating f and g individually and as static mappings (Marconato et al. (2014)). As the state sequence is not available in practice, one would like to obtain an approximation of x(t) to be able to obtain initial estimates of f and g. Hence the first step in this approach is to obtain an initial estimate of the state sequence x(t). 5.2.1. Estimation of xLS Based on the obtained BLA, N and on the set of available data {u(t), y(t)t=1 }, an approximation of the (unknown) states x(t) is obtained. If this can be done reliably, it is possible to solve an approximate version of (4), eliminating the recursion in the state equation. An estimate of the states x ˆLS (t) is obtained by solving the following regularised least squares problem for t = 1, 2....., N : (y(t) − CˆBLA x(t))2 (11) x ˆLS (t) = argmin x(t)

+λ

t

output and the measured output (for the validation data set) is selected. 5.2.2. Estimation of functions fN L and gN L Once the N estimate x ˆLS (t)t=1 of the unknown state sequence is available, one obtains the following approximate static problem: x ˆLS (t + 1) = f (ˆ xLS (t), u(t)) + rLS (t) ˆ ˆ = Aˆ xLS (t) + Bu(t) + fN L (ˆ xLS (t), u(t)) + rLS (t) y(t) = g(ˆ xLS (t), u(t)) + eLS (t) ˆ = Cˆ x ˆLS (t) + Du(t)

(12)

+ gN L (ˆ xLS (t), u(t)) + eLS (t) (13) where rLS (t) and eLS (t) are error terms resulting from the fact that here the approximated state sequence is introduced in the problem. Equations (12) and (13) represent two static regression problems that can be solved independently employing simple regression methods. Note that at this stage, the recursion in the state equation is not present anymore, since the state sequence is now assumed to be "known". Therefore, both functions fN L and gN L can be estimated as basis function expansions. Here we use the neural-networks with 2 hidden layer neurons and sigmoid nonlinear function. 6. IDENTIFICATION OF FULL NLSS

5.2 Initialisation approach II

455

In the last step, the coefficients of both the linear and the nonlinear terms in (5) are identified. This problem can either be solved in frequency domain or time domain. In order to keep the estimates of the model parameters unbiased it is assumed that the input u(t) of the model in Section 3 is noiseless, i.e., it is observed without any errors and it is independent of the output noise. It is worth noting that this assumption is voilated in the cascaded water-tanks benchmark problem. 6.1 PNLSS model A weighted least squares approach (in frequency domain) will be employed. The Weighted Least Squares (WLS) cost function that needs to be minimized with respect to the parameter θN L = [vecT (A) , vecT (B), vecT (C), vecT (D), vecT (E), vecT (F )]T is given by: VW LS (θN L ) =

t

ˆBLA u(t))2 (x(t + 1) − AˆBLA x(t) − B

= argmin Ey + λEx x(t)

The first term Ey of the cost function represents the data fit, while the second term Ex represents the linear model fit; λ is the tradeoff parameter that needs to be tuned to change the emphasis given on the two criteria. By tuning λ, a deviation from the linear state (resulting from the BLA estimates) is allowed. In practice, the optimal value of λ is chosen such that a given performance criterion is optimized. In this work, several initialized models resulting from different choices of λ are compared, and the value of λ minimizing the RMSE between the initialized model 457

Nf |Ymod (jωk , θN L ) − Y (jωk )|2

k=1

W (jωk )

(14)

where Nf is the total number of selected frequencies. Ymod and Y are the DFTs of the modelled output and the measured output, respectively. Because in nonlinear systems, model errors often dominate the disturbing noise, we put the weighting factor W (jωk ) = 1. Only if the model errors are below the noise level, W (jωk ) can be put equal to the noise variance σn2 (jωk ). Furthermore, model error (jωk , θN L ) ∈ Cny is defined as (jωk , θN L ) = Ymod (jωk , θN L ) − Y (jωk ), (15) 6.2 NLSS2 model For the final optimisation of the model structure in (6), the following cost function is solved, where the θN L2 now

Proceedings of the 20th IFAC World Congress 456 Rishi Relan et al. / IFAC PapersOnLine 50-1 (2017) 452–457 Toulouse, France, July 9-14, 2017

also contains the parameters of the basis functions fN L and gN L along with other andparameters.

VLS (θN L2 ) =

N 1 (y(t) − yˆ(t, θN L2 ))2 N t=1

(16)

Table 2. Comparison of nonlinear models PNLSS-I NLSS2 Benchmark Est 0.032393 0.1165 Benchmark Val 0.44984 0.3433

7. EXPERIMENTAL DEMONSTRATION In this section, we describe the measurement set-up and data acquisition procedure which was employed to acquire the input-output signals from the benchmark system. 7.1 Measurement Set-up The input multisine signals which are defined as: k max fs A(k) cos(k 2 π t + ϕk ) ums (t) = N

(17)

k=1 kmax fNs

where fmax = = 0.0144 Hz. Here, the period length of the excitation signal is 1024 points. Only 1 period of input and output data were acquired both for the estimation (i.e. Nest = 1024 data points) and the validation (i.e. Nval = 1024 data points) datasets respectively. In the provided dataset, the lowest frequencies have a higher amplitude then the higher frequencies. The sample period Ts is equal to 4s. The input signals are zero-order hold input signals. The process is controlled from a Matlab interface to the A/D and D/A converters attached to the water level sensor and the pump actuator. The water level is measured using capacitive water level sensors, the measured output signals have a signal-to-noise ratio that is close to 40 dB. The water level sensors are considered to be part of the system, they are not calibrated and can introduce an extra source of nonlinear behavior. The system states have an unknown initial value at the start of the measurements for both the estimation and the validation datasets. 8. RESULTS The results of the identification procedures described in Section 4 are presented herein using the benchmark data. 8.1 Figure of Merit The performance of the different model structures is judged based on the following figure of merit. Nt 1 eRM St = (ymod (t) − y(t))2 (18) Nt t=1 where ymod is the modeled output, yt is the output in the test dataset, Nt is the total number of points in yt .

8.2 Comparison Table 1. Comparison of BLA and PNLSS BLA PNLSS Model Est 0.54708 0.1133 Model Val 0.75743 0.75063 Model Test 0.75331 0.69737

Fig. 3. Comparison of outputs of PNLSS-I and NLSS2 models on the validation dataset in time domain 8.3 Discussion Two different investigations were performed to study the effect of various factors on performance of the proposed model. For the comparison between BLA and PNLSS model, the benchmark estimation dataset was further divided into model estimation (Model Est = 70% of Nest = 1024 data points) and model validation (Model Val = 30% of Nest = 1024 data points) datasets, whereas the benchmark validation dataset was used as the model test (Model Test = Nval = 1024 data points) dataset. Table 1 shows the comparison between BLA and PNLSS model on the model estimation, model validation and model test datasets respectively. Table 2 shows the comparison between two different initialisations namely PNLSS-I and NLSS2 on the original benchmark dataset. Figure 3 shows the comparison of the outputs of the PNLSS-I and the NLSS2 models with respect to the output provided in the benchmark validation dataset in time domain. From Table 1 it is evident that the PNLSS model performs better than the BLA on the model estimation dataset whereas its performance on the model validation and model test datasets is quite similar to the BLA, which point towards the case of overfitting. From Table 2 It can be easily observed that the PNLSS-I model performance is better than the BLA. This is to be expected because PNLSS-I model structure has larger number of data points (full benchmark estimation dataset i.e. N = 1024 data points) for the model identification and it estimates the initial state x0 explicitly. But its performance degrades on the benchmark validation dataset too. From Tables 1 and 2 it is also clear that, along with larger estimation dataset, once the effect of the initial conditions is taken into account during the identification procedure, the performance of PNLSS-I is also better than the PNLSS model. In the PNLSS model, it is assumed that we start with zero initial conditions and this assumption is violated in the cascaded water-tanks benchmark. In the context of this benchmark, NLSS2 model performs the best both on the benchmark estimation and benchmark validation datasets respectively (see table 2). One reason could be due to its better generalisation perfor-

458


mance (i.e. less chance of over-fitting because it has less number of parameters ≈ 71 as compared to PNLSS or PNLSS-I ≈ 131 parameters. NLSS2 model structure also offers the possibility to use different basis functions to estimate the nonlinearities in the system as it can be decoupled from the estimation of the dynamics part. Hence the problem of nonlinearity (fN L (·) and gN L (·)) estimation can be treated as a static regression problem, which opens up the possibility to use any static function estimation or regression framework to estimate the nonlinear functions fN L and gN L . 9. CONCLUSION In this paper, a powerful nonlinear state space model structure is introduced to model the cascaded watertanks benchmark. The NLSS model is able to capture the dynamic behavior of the system and is flexible enough to accommodate any nonlinear effect. It has been shown that this model structure can be initialised very easily. Specifically, in this case study, two different initialisation schemes were introduced, namely PNLSS and NLSS2. The NLSS2 performs relatively well on the available dataset despite having structural limitations such influence of process noise. It has been shown that, it is relatively easy to change the basis functions for estimation of the nonlinearities (nonlinear functions). Explicit estimation of initial conditions can easily be done in the PNLSS model structure. One restriction of the PNLSS model structure is that with the increase in the model order, the number of parameters to be estimated also grows rapidly. Therefore it can suffer from curse of dimensionality, which is the case in this benchmark as a very short data record was available. Nevertheless, the promising results were obtained despite having all these limitations and found to be very close to the best results (RMS ≈ 0.29) obtained by the University of Sheffield group in the nonlinear system identification workshop Schoukens et al. (2016) held in Brussels, Belgium. ACKNOWLEDGEMENTS This work was supported in part by the IWT-SBO BATTLE 639, Fund for Scientific Research (FWO-Vlaanderen), by the Flemish Government (Methusalem), the Belgian Government through the Inter university Poles of Attraction (IAP VII) Program, and by the ERC advanced grant SNLSID, under contract 320378.

REFERENCES Billings, S. (2013). Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and SpatioTemporal Domains. Wiley. Giannakis, G.B. and Serpedin, E. (2001). A bibliography on nonlinear system identification. Signal Processing, 81(3), 533 – 580. Special section on Digital Signal Processing for Multimedia. Giri, F. and Bai, E. (2010). Block-oriented Nonlinear System Identification. LNCIS. Springer London. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. Springer Series in Statistics. 459

457

Laub, A., Heath, M., Paige, C., and Ward, R. (1987). Computation of system balancing transformations and other applications of simultaneous diagonalization algorithms. Automatic Control, IEEE Transactions on, 32(2), 115– 122. Levenberg, K. (1944). A method for the solution of certain problems in least squares. Quart. Applied Math., 2, 164– 168. Ljung, L. (1998). System Identification: Theory for the User. Pearson Education. Marconato, A., Sjöberg, J., et al. (2014). Improved initialization for nonlinear state-space modeling. Instrumentation and Measurement, IEEE Transactions on, 63(4), 972–980. Moore, B.C. (1981). Principal component analysis in linear systems: Controllability, observability, and model reduction. Automatic Control, IEEE Transactions on, 26(1), 17–32. Mzyk, G. (2013). Combined Parametric-Nonparametric Identification of Block-Oriented Systems. LNCIS. Springer International Publishing. Nocedal, J. and Wright, S.J. (2006). Numerical Optimization. Springer, New York, 2nd edition. Paduart, J., Lauwers, L., Swevers, J., Smolders, K., Schoukens, J., and Pintelon, R. (2010). Identification of nonlinear systems using polynomial nonlinear state space models. Automatica, 46(4), 647 – 656. Pintelon, R. and Schoukens, J. (2012). System Identification: A Frequency Domain Approach, 2nd Edition. Wiley-IEEE Press. Rasmussen, C. and Williams, C. (2006). Gaussian Processes for Machine Learning. University Press Group Limited. Schön, T.B., Wills, A., and Ninness, B. (2011). System identification of nonlinear state-space models. Automatica, 47(1), 39–49. Schoukens, J., Vandersteen, G., Barbé, K., and Pintelon, R. (2009). Nonparametric preprocessing in system identification: A powerful tool. In 2009 European Control Conference (ECC), 1–14. doi:10.3166/ejc.15.260-274. Schoukens, M., Mattson, P., Wigren, T., and Noël, J.P. (2016). Cascaded tanks benchmark combining soft and hard nonlinearities. In Workshop on Nonlinear System Identification Benchmarks, 20–23. Brussels, Belgium. URL http://homepages.vub.ac.be/ ~mschouke/benchmarkCascadedTanks.html. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.Y., Hjalmarsson, H., and Juditsky, A. (1995). Nonlinear black-box modeling in system identification: a unified overview. Automatica, 31(12), 1691 – 1724. Suykens, J., Van Gestel, T., and De Brabanter, J. (2002). Least Squares Support Vector Machines. World Scientific. Suykens, J., Vandewalle, J., and de Moor, B. (2012). Artificial Neural Networks for Modelling and Control of Non-Linear Systems. Springer US. Van Overschee, P. and De Moor, B. (2012). Subspace identification for linear systems: Theory - Implementation Applications. Springer Science & Business Media. Vanbeylen, L. and Van Mulders, A. (2014). Nonlinear black-box identification of a mechanical benchmark system, 215–222. Springer.