IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
431
A Least-Squares Parameter Estimation Algorithm for Switched Hammerstein Systems With Applications to the VOR Sunil L. Kukreja, Robert E. Kearney, Fellow, IEEE, and Henrietta L. Galiana*, Fellow, IEEE
Abstract—A “multimode” or “switched” system is one that switches between various modes of operation. When a switch occurs from one mode to another, a discontinuity may result followed by a smooth evolution under the new regime. Characterizing the switching behavior of these systems is not well understood and, therefore, identification of multimode systems typically requires a preprocessing step to classify the observed data according to a mode of operation. A further consequence of the switched nature of these systems is that data available for parameter estimation of any subsystem may be inadequate. As such, identification and parameter estimation of multimode systems remains an unresolved problem. In this paper, we 1) show that the NARMAX model structure can be used to describe the impulsive-smooth behavior of switched systems, 2) propose a modified extended least squares (MELS) algorithm to estimate the coefficients of such models, and 3) demonstrate its applicability to simulated and real data from the Vestibulo-Ocular Reflex (VOR). The approach will also allow the identification of other nonlinear bio-systems, suspected of containing “hard” nonlinearities. Index Terms—Hammerstein systems, least-squares, mathematical modeling, NARMAX, parametric models, switched systems, system identification.
I. INTRODUCTION
A
SWITCHED or multimode system (Fig. 1) is one that switches due to external or internal causes between a finite number of different modes of operation. Consequently its response may have discontinuities at each mode switch [1]. We consider multimode systems where the modes of operation are characterized as finite-dimensional, nonlinear, time-invariant, difference equations since they may include nonzero initial conditions. The assumptions we make for this system description are 1) the switch times are known for each subsystem, is Gaussian, white, 2) the output additive noise sequence, zero-mean, 3) the noise model is described by a linear map, and 4) the system is nonzeno, i.e., cannot switch an infinite number of times in a finite time span [2]–[6]. Manuscript received September 16, 2003; revised August 8, 2004. This work was supported by the Natural Sciences Engineering Research Council of Canada, the Canadian Institutes for Health Research, and the Max Stern Fellowship of McGill University. Asterisk indicates corresponding author. S. L. Kukreja is with NASA Dryden Flight Research Center, MailStop 4840 D, Edwards, CA 93523-0273, USA (e-mail:
[email protected]). R. E. Kearney is with the Department of Biomedical Engineering, McGill University, Montréal, QC H3A 2B4, Canada (e-mail:
[email protected]). *H. L. Galiana is with the Department of Biomedical Engineering, McGill University, Montréal, QC H3A 2B4, Canada (e-mail:
[email protected]). Digital Object Identifier 10.1109/TBME.2004.843286
The NARMAX (Nonlinear AutoRegressive, Moving Average eXogenous) model is a general parametric form for describing nonlinear systems [7]. It describes the current output in terms of past inputs, outputs and errors and may include a variety of nonlinear terms. In addition, Chen and Billings [8] suggest extensions of NARMAX to a nonzero-initial-state case. As such, NARMAX models may be used to describe the input-output relationship of nonlinear switched systems. Parameter estimation involves determining values for unknown system coefficients. Many parameter estimation techniques for nonlinear systems depend critically on the choice of model structure, the source of noise within the system, and the input excitation [9]. Most parameter estimation algorithms for linear systems cannot be applied directly to NARMAX models because they assume that the noise terms in the model are independent. Parameter estimation or identification may also be difficult or impossible if the recorded data is not sufficiently long. This is a particular problem in multimode systems where, due to the switching behavior, no single data segment may be long enough for parameter estimation. The extended least-squares (ELS) algorithm [10]–[12] yields unbiased estimates for NARMAX models. Its convergence when applied to linear systems is well documented [13]. Simulation results show that it is also well suited to nonlinear polynomial identification [9]. However, ELS cannot estimate the impulsive behavior of multimode systems or use more than a single data segment. To address these shortcomings we propose a modified ELS (MELS) algorithm to determine parameter values of nonlinear multimode systems, using multiple short data segments. The development in this paper is specific to the Hammerstein model structure, a special case of the general NARMAX polynomial class. Identification of piecewise affine systems is closely related to the problem considered here. The estimation of such systems has received little attention in the hybrid systems community and only a few methods have been proposed for this class of systems [14]–[16]. Identification of piecewise affine systems assumes that the system within each region is linear or affine [14], [15]. Recently, the estimation of such systems has been extended to nonlinear systems using mixed-integer linear or quadratic programming techniques which guarantee an optimal model is found within the given model class [16]. However, these methods do not take advantage of the entire data record for a given mode of operation and estimate separate
0018-9294/$20.00 © 2005 IEEE
432
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
M
M
Fig. 1. General Hammerstein model structure for a mode switched system with output additive noise where U (s) is the input, Y (0) is the initial condition, Y (s) the true (Y (s) is the selected Y (s) driven by X (s); m = 1; 2; . . . ; ) system output, E (s) a Gaussian, white, zero-mean, noise sequence, and Z (s) the measured output.
models for all “piecewise affine” regions of the system. Piecewise affine techniques were not designed to estimate a global model description since they do not assume that there is some global information in the data that makes them all relevant when estimating the model for every part of the regressor space. The vestibulo-ocular reflex (VOR) is well known to exhibit nonlinear “switched” behavior [17]–[19]. Presently, descriptions of the VOR rely on linear a priori modeling methods [18], [20]. These methods provide convenient means of characterizing slow and fast-phase dynamics. However, such models do not account for the rich dynamic behavior due to nonlinearities, and are, therefore, of limited use in diseased cases because of mode interactions through initial conditions [20]. The organization of this paper is as follows. Section II introduces the NARMAX model structure and shows that Hammerstein models are a special class of this general structure. Section III shows how a simple nonlinear switched system can be expressed as a Hammerstein structure NARMAX model. The modified ELS algorithm is presented in Section IV. Section V introduces the VOR system, which is well known to exhibit multimode behavior. Section VI provides results of the proposed algorithm on a simulated VOR system and Section VII illustrates the results of this algorithm on experimental VOR data. Section VIII provides a discussion of our findings and Section IX summarizes the conclusions of our study.
of the NARMAX model is a nonlinear difference equation of the form (1) where is a nonlinear mapping, is the nonlinearity order, is the “controlled” (i.e., exogenous) input, is a linear mapping, is the output, and is the innovation, or uncontrolled input. The nonlinear mapping, , can be described by a wide variety of nonlinear functions such as a half-wave rectifier (i.e., hard nonlinearity) [8], [22]. For simplicity, we only consider nonlinearities that can be described by a polynomial expansion. This class of nonlinear difference equations describes the dynamic behavior of a system as a linear expansion of the output and or ) and a linear and nonerror terms (e.g., or ). linear expansion of the inputs (e.g., The derivation of the general NARMAX model was based on the zero-initial-state response [7], [21], but Chen and Billings [8] suggest a way to extend it to the nonzero-initial-state case. We used this approach to extend the NARMAX Hammerstein formulation to model switched systems. This involved including lagged inputs, outputs and impulse values to account for the input-output model and nonzero-initial-states or discontinuities (response function) as
II. NARMAX MODEL NARMAX models describe both the stochastic and deterministic components of a system in discrete-time. The NARMAX description encompasses most forms of nonlinear difference equations that are linear-in-the-parameters [7], [21]. One special form of NARMAX models are Hammerstein systems. Hammerstein systems are described as blocked structured N-L models (static nonlinearity followed by a causal, linear, time-invariant, dynamic system, see Fig. 1). This special case
(2) where represents the input-output model, the reare as defined sponse function, the modes of operation, previously, and is the Kronecker impulse function applied at the beginning of each data segment. This allows the NARMAX model structure to model switched Hammerstein structure or multimode systems [23].
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
433
III. MULTIMODE MODEL FORMULATION Many systems are described in continuous-time. However, for most identification applications it is necessary to convert the continuous-time system description to a discrete-time representation. This is crucial for parametric identification of nonlinear system since there are few methods available and existing techniques have limitations when noise is introduced [24]–[28]. In this section, we illustrate how to represent a switched continuous-time system in discrete-time. Let us begin by considering a dual mode system structure, a simple case of the general form described in Fig. 1. Let and be defined as
(3) for or represents the initial condition where in continuous-time and are the number of switches, i.e., data segments, of subsystem one and two, respectively. Notice that we select both a high-pass and low-pass system description ( and , respectively) to illustrate that both can be formulated as switched systems. The two pathways may be decoupled, analyzed separately, and then recombined to yield the overall input-output relationship, provided that initial conditions are modeled. The static nonlinearity can be converted to discrete-time by assuming an appropriate sampling rate, , to give (4)
(6) This is a Hammerstein structure NARMAX model since it 1) includes linear output terms, 2) nonlinear input terms, and 3) is linear-in-the-parameters. Note that the global inputs and outputs are used in both models. This provides the coupling of the initial condition after a switch to the state of the model before the switch. If one assumes that the system switches from mode 1 to mode 2 at time instant then the (scaled) ’s are equivalent to and which are the initial conditions given by determined by model 1 since that model was active before the switch occurred. The system representation in (6) is a combination of both the input-output model and response function. Notice that although the response function depends on the initial states, the inputoutput model for the system will always be the same [8]. This mode simple model can easily be extended to the general Hammerstein system given in (2) since, in general, the number . of subsystems can be any finite number, IV. PARAMETER ESTIMATION Many parameter estimation techniques are based on leastsquares theory. However, ordinary least-squares algorithms for linear systems cannot be applied here because they assume that the noise terms in the model are independent and the regressor matrix is deterministic [29]. Both of these conditions are violated in (6), when one considers that only a noisy measure of into the output is available. Substituting (6), we obtain
where is the sampled data point index. The linear system dynamics can be converted to the discrete domain via the bilinear transform to give
(5) , and , , dewhere the coefficients pend on the parameters of the continuous-time linear system are organized and sampling rate. The input-output data according to mode of operation and denoted by subscript 1 for subsystem one and 2 for subsystem two. The Kronecker impulse function, , is used to represent the onset of an initial condition in discrete-time. and are discrete-time initial conditions (coefficients), used as impulse weights to scale the Kronecker impulse, accounting for the discontinuity at each switch time. represent the onset of the th and th impulse. The indices Substituting for and for into (5) and combining terms gives the overall nonlinear model as
(7) Clearly, the regressors in (7) contain correlated error terms, and noisy data. To obtain unbiased parameters other estimation techniques based on least-squares must be used. Two such methods which are applicable to NARMAX model estimation are ELS and prediction error methods (PEM). Both provide unbiased parameter estimates and are iterative [8], [30]. PEM requires a gradient of the parameter vector to be computed which may not be possible for all nonlinear functions. ELS does not require the computation of a gradient and relies on successive improvements of the residuals. Other estimation techniques such as maximum-likelihood (ML), instrumental variables (IV), weighted least-squares, etc., are difficult to implement and have convergence problems [30]–[32]. The convergence of ELS, when applied to linear systems, is well documented [13]. Simulation results show that it is also well suited
434
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
to nonlinear polynomial identification [9]. For these reasons we selected ELS as the basis for our identification algorithm. However, neither standard ELS or PEM procedures can estimate the impulsive behavior of multimode systems or use more than a single data segment. A. Modified Extended Least-Squares The steady-state response of a system can be estimated using the ELS algorithm. However, ELS cannot be applied to switched systems since it does not account for the transients which occur each switch time and as a result will produce biased estimates. Consider a system which does not switch but has segmented input-output data. For this system, define the concatenation of all input-output segments as (8) where denotes the total number of segments, each of equal length for simplicity. Next, consider a switched system with two modes of operation. Define the concatenation of all input-output segments for each mode as
Fig. 2. Example of input-output data segmentation for experimentally recorded human VOR (i.e., a dual mode switched system) during 1=6 Hz passive head rotation in the dark. Light Segments: Mode 1 (slow-phase). Bold and Z input-output segment of a Segments: Mode 2 (fast-phase). U system mode where m = system mode and i = segment number of mth mode.
To model the lagged noise terms an extended regressor matrix is defined as
(9) vecwhere is the mode of operation. Let and be tors of measured input and output, respectively. See Fig. 2 for an example of data segmentation for a dual mode switched system. This can be easily generalized for an arbitrary number of segments and modes of operation. In the subsequent text we derive a modified ELS algorithm (MELS) based on one mode for ease of notation and clarity. To estimate an initial parameter vector for, say, the first sub, is formed. This regressor masystem a regressor matrix, trix is a concatenated matrix of subregressor matrices formed from individual data segments as
.. .
(10)
where is a matrix of regressors with full column rank and based on input-output only. represents all linear output terms, , and linear and nonlinear terms involving . be a vector of initial estimates for the process Let parameters computed via ordinary least-squares (OLS). Due to the recursive nature of the model description (see (6)) and assuming simple output additive noise this will lead to a biased estimate [30]. To obtain unbiased parameters the lagged errors need to be modeled. In general, since the noise sequence is a stochastic process, it is not possible to solve for the noise source , and it will not be equal to the prediction errors [9]. The prediction errors (residuals) are defined as (11) where and denote a concatenated vector of predicted system outputs and errors, respectively.
.. .
.. .
(12)
is a partitioned regressor matrix where is the same is a polynomial function of as defined previously (10) and the prediction errors only [9]. However with switched systems, modeling the lagged error terms is not sufficient to obtain an unbiased parameter estimate since the residuals will contain dynamics due to the transients at each switch time, as shown in (7), [33]. To address the bias due to transients, a modified extended regressor matrix is formed to estimate the switching transients and is defined as (13) where is defined as in (12). The extension, , represents the effects of initial conditions when a switch occurs (see (7)). is equal to the number of data The number of columns in segments and is simply a matrix with a one in each column at the start of a segment and zeros elsewhere. The number of transients or initial conditions in each segment is equal to the dynamic order of the linear system. For a linear subsystem of order , the number of columns needed to model transient effects is equal to the order of the linear dynamic system times the number of segments available for estimation. For example, if the number will be of dimension of segments is , the partition of where is the total data length of a subsystem. For the , case presented in Section III there is simple first order, one initial condition per segment since the dynamics are of order partition of size . one resulting in a To model transients separately, the effects of the forcing function must be removed. For a general order dynamic system the current and lagged inputs up to the order of the linear subsystem,
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
, minus one and up to the order of nonlinearity, , are suboutput points of each segment. tracted from the first is formed for a single data To illustrate this we show how segment using the simple first-order example presented in Section 3. The first input point is subtracted up to the order of the nonlinearity, 3, from the first output of a given segment. Hence, is formed as for a single data segment a column of
.. .
(14)
A modified and extended least-squares formulation for this system is (15) where is a matrix of regressors defined in (13) with full column rank, based on input-output, prediction errors and transients. and are the same as defined previously and is vector of unknown system parameters. a The extended parameter set based on this model formulation is defined as
7) 8) 9)
435
replaced Re-compute the residuals as in (11) with replaced by . by and Go to 5) until convergence. Estimate parameters of the next subsystem, . Go to 2) until . V. VOR MODEL
Ocular responses to head perturbations consist of intermingled segments classified as “slow” or “fast,” according to their average speed characteristics. This describes the vestibulo-ocular reflex (VOR) and a time record of the response has a sawtooth-like pattern called ocular nystagmus (see bottom panel of Fig. 2) [18], [34]. This sawtooth-like pattern is a consequence of the VOR switching between two different modes of operation: the slow-phase which stabilizes the eye in space and the fast-phase which re-orients the eye in the direction of head rotation [18], [19], [35]. Consequently, the VOR is a dual mode switched system, a simple case of the general form in Fig. 1. , and be redefined In Fig. 1 let as
(16) The partition of the parameter vector denoted contains process coefficients and the partition denoted as contains coefficients associated with transients. Although the ELS algorithm is sometimes used for on-line estimation in a recursive form (RELS), it is impossible to use RELS when the switch times are unknown. This holds with our first assumption where we stated that the switch times are known. If the switching instances are known a priori, it is straight forward to establish a relation for a recursive version of the proposed MELS algorithm for on-line estimation, uniting the development of MELS with the traditional RELS algorithm [10]–[13]. The extended parameter vector given in (16) can be shown to be an unbiased estimate since the residuals are zero-mean, in the limit, when all transients and errors are estimated [33]. This modified extended least-squares algorithm has the same asymptotic properties as OLS and ELS since it models all dynamics due to initial conditions or discontinuities as well as system dynamics. This leads to the following algorithm to estimate parameters of Hammerstein structure switched systems. MELS Algorithm: 1) Segment the input-output data record according to mode of operation. for the th subsystem, for 2) Form as in (10). 3) Compute an initial estimate of the unknown parameter vector using OLS as in (11). 4) Estimate the residuals as in (11). 5) Form for the th subsystem as in (13). 6) Compute an estimate of the extended parameter vector as given in (16).
(17) where and are first-order approximations for the slow and fast modes (phases) of the vestibulo-ocular reflex. and for or are the slow and fast-phase gain and pole, respectively. Clinically, evaluation of vestibular patients relies on the characteristics of only VOR slow-phases [18], [20], [36], [37]. However, our method can provide both the slow and fast-phase dynamics in discrete-time. A NARMAX description of VOR slow, and fast-phases, of the model in (17) is phases,
(18) Table I shows the relationship of these discrete-time parameters to the underlying continuous-time parameters. This model of the VOR was used to test the performance of our algorithm.
436
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
TABLE I DISCRETE-TIME RELATIONSHIP OF NARMAX MODEL PARAMETERS TO UNDERLYING CONTINUOUS-TIME PARAMETERS
TABLE III THEORETICAL AND ESTIMATED DISCRETE-TIME COEFFICIENTS OF VOR SUB-SYSTEM 1 AND 2 (SLOW AND FAST-PHASE). COLUMN: THEORETICAL PARAMETER VALUES. ^ COLUMN: ESTIMATED PARAMETERS USING ELS. COLUMN: ESTIMATED PARAMETERS USING MELS. VALUES ^ CORRESPOND TO CONTINUOUS-TIME SLOW-PHASE PARAMETER VALUES IN TABLE II. SEE FIG. 6 (20 dB) FOR PARAMETER STANDARD-DEVIATIONS
TABLE II LEFT: CONTINUOUS-TIME COEFFICIENT VALUES. : SLOW-PHASE TIME-CONSTANT, : FAST-PHASE TIME-CONSTANT, K : SLOW-PHASE GAIN (G = K = = 0:628); K : FAST-PHASE GAIN (G = K = = 4:44), AND T: SAMPLING INTERVAL. RIGHT: COEFFICIENT VALUES OF STATIC NONLINEARITY. a : DC TERM, b : LINEAR TERM, c : SQUARED TERM, CUBIC TERM AND d
0
In the clinical setting, experimental outputs are typically in the range of deg. The input-output is recorded with a 16-bit A/D and a 20-V dynamic range. The convention used in the V. This yields an output SNR (assuming clinic is 9–18 no other output noise source) of between 95–89 dB (see [38, p. 756]). The SNR used in this simulated example is 4.45 times smaller than what we expect in an experimental setting. A. Discrete-Time Parameter Estimation
Fig. 3. Simulated input-output data used for identification. Note that only 6 s of data are plotted to give better resolution of both slow and fast-phase segments.
VI. SIMULATION The accuracy of the MELS parameter estimation algorithm was validated by simulating a VOR model in continuous-time using Simulink. The parameters used in the simulation were typical values found in experiments and are shown in Table II, [18]. Hz freThe system was perturbed using a sinusoid input ( quency and 188 deg/s amplitude) while a Gaussian, zero-mean, noise sequence with 20 dB SNR was added to the output (Fig. 3). A sinusoid was used as input because it is the type of input used in the clinic. The system input-output was sampled at 600 Hz. Notice, although our model formulation allows each mode of operation to have a unique nonlinearity, this simulated system contains the same nonlinearity in each path for simplicity.
Fig. 3 shows 6 s of the input-output data used for this identification. The system was identified from head velocity to eye position. Forty-one slow and forty fast-phase segments were used for identification ( of the simulated slow and fast-phase and ). This cordata; respectively, responds to slow-phase having approximately 14 times more of the data for identificadata than fast-phase. We selected tion so the identified parameters would be as close as possible for cross-validation. to their asymptotic values, leaving Extended least-squares and the modified extended least-squares algorithm were both used to estimate system parameters from simulated data. ELS ignores the effect of switching and associated initial conditions. Therefore, it is equivalent to the traditional clinical analysis of slow-phases, which use a continuous-smooth envelope of the eye velocity response by interpolating the gaps due to fast-phases. MELS instead treats each slow-phase segment as a transient response accounting for both the forced input and switching effects. Table III compares the results of the ELS and MELS algorithms, after estimating the coefficients of both slow and fast modes in this simulated VOR. The first column contains the
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
theoretically computed parameter values, the second contains the estimated parameter values using the ELS algorithm and the third contains the estimates given by the MELS algorithm. As expected, the ELS estimates are biased in both slow and fast-phases. However, the MELS algorithm yields accurate estimates of system parameters in both modes, even in the presence of disof output additive noise. Although ELS estimates crete-time slow-phase coefficients have correct sign, and are biased by estimates corresponding to parameters a factor of 100 and 10, respectively. MELS estimates of all slow-phase process parameters have correct sign and are less than a factor of 10 away from the true value. ELS estimates also all have of discrete-time fast-phase coefficients correct sign. The parameters corresponding to and are biased by a factor of 1000 and 10, respectively. MELS estimates of fast-phase process parameters have correct sign but the paramis biased by a factor of 10 away from eter corresponding to and the true value. With both algorithms the error terms are significantly biased from the true value. These error terms are difficult to estimate since they represent a stochastic noise process. B. Cross-Validation Next, we cross-validated the discrete-time parameter estimates of VOR dynamics using a -step-ahead predictor. Twenty slow and fast-phase segments were used for cross-validation of the simulated slow and fast-phase data, respectively; ( ). The quality of fit was assessed by computing the %QF as
%
437
Fig. 4. Cross-validated eye position output for simulated data. (a): Slow-phase eye position ELS algorithm. (b): Fast-phase eye position ELS algorithm. (c): Slow-phase eye position MELS algorithm. (d): Fast-phase eye position MELS algorithm.
TABLE IV DISCRETE TO CONTINUOUS-TIME RELATIONSHIPS FOR PARAMETERS ; G a ; G b ; G c , AND G d OF THE VOR MODEL, FOR = 1 OR 2 CORRESPONDING TO SLOW OR FAST-PHASE MODE, RESPECTIVELY
(19)
Specifically, we predicted the system output using a -stepahead predictor as (20) A -step-ahead predictor uses past values of the “predicted” ( or ) true system outputs ( or ) to compute the current output. and denote a segmented output according to mode of operation as defined in (9). Fig. 4, shows results of cross-validation for the parameters in Table III. Fig. 4 show the slow and fast-phase cross-validated fit due to the ELS algorithm, respectively. The %QFs obtained for slow and fast-phase dynamics using the ELS method are 27.8% and 62.8%. Fig. 4 (c) and (d) shows the slow and fastphase cross-validated fit obtained using the MELS algorithm, respectively. Using the MELS algorithm the slow and fast-phase %QFs are 98.5% and 99.2%. Fig. 4 shows that the MELS algorithm provides a better fit to the data and that the %QF obtained using our switched approach is superior compared to the ELS method. The improvement is evident in a visual comparison of the plots. Thus, MELS eye prediction fits the simulated data better, as well as giving more accurate estimates of the true discrete-time parameters.
C. Continuous-Time Parameter Estimation We estimated continuous-time parameters of slow and fast-phase dynamics using the discrete-time parameter estimates in Table III and the theoretical relationships in Table IV. Since it is impossible to measure the signal at the output of the static nonlinearity, we consider the linear system to have unity gain and translate the overall gain onto the static nonlinearity. For this reason, the estimated static nonlinearity coefficients for or ) are a product of ( the linear system and static nonlinearity gain. Note that it is possible to compute the continuous-time parameters for this nonlinear switched system only because the system structure is fully known. The resulting parameters of the static nonlinearity are different from the theoretical for each mode since it is difficult to accurately separate the coefficients of the static nonlinearity and the linear subsystem gain. The segregated estimation of the static nonlinearity coefficients and linear system gain may be possible if the internal signal can be “reliably” reconstructed. However, this is problematic for multimode systems and could
438
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
Fig. 5. Hammerstein structure block diagrams of true and estimated continuous-time parameter estimates from simulated VOR data. (a): Slow-phase estimate ELS algorithm. (b): Fast-phase estimate ELS algorithm. (c): Slow-phase estimate MELS algorithm. (d): Fast-phase estimate MELS algorithm. Dash-dot line (“.”) in first block theoretically computed nonlinearity and solid line (“—”) estimated nonlinearity. (a) = 15:0 s, ^ = 4:84 s, (b) = 0:0500 s, ^ = 0:0648 s, (c) = 15:0 s, ^ = 15:5 s, (d) = 0:0500 s, ^ = 0:0515 s.
lead to issues with convergence. Moreover, for VOR identification the predominant parameter of interest is the pole of the linear subsystem. Additional computation of the individual static nonlinearity coefficients and linear subsystem gain are not of prominent interest and only serve to increase computation time and possibly lead to problems with convergence. Fig. 5 shows Hammerstein structure block diagrams of the true and continuous-time estimates of slow and fast-phase modes describing VOR dynamics based on the ELS and MELS algorithms. The first block shows the estimated static nonlinearity (solid line) superimposed on the theoretically computed nonlinearity (dash-dot line) and the second shows the estimated first-order linear subsystem. Note that the actual system is switched and shares states but is drawn as shown in Fig. 5 for clarity. The results in Fig. 5 show that although some discrete-time and ) computed via the ELS algorithm parameters (e.g., “appear” close to their theoretical values (see Table III), when they were used to estimate the continuous-time parameters the bias due to ELS became noticeably large [see nonlinearity plots and time constant estimates and in Fig. 5(a), (b)]. However, continuous-time parameters computed based on the MELS algorithm are close to their true values [see nonlinearity plots and time constant estimates and in Fig. 5(c), (d)]. Hence, for this model of VOR the MELS algorithm performance was superior to the ELS algorithm and gave good estimates of the underlying continuous-time parameters. D. Noise Sensitivity A study of NARMAX parameters describing VOR slow and fast-phase dynamics was performed to assess their estimation
accuracy and variability using the MELS algorithm. One thousand Monte Carlo simulations were generated in which the input-output realization was the same but had a unique Gaussian white, zero-mean, noise sequence added to the output. The output additive noise amplitude was increased in increments of 5 dB, from 20 to 0 dB SNR. Parameter mean and standard deviation was computed from the one thousand estimates. Fig. 6 shows the results of this study for slow and fast-phase, respectively. For comparison purposes, each figure shows the normalized standard deviation (STD) about the normalized mean, for the discrete-time parameters given in Table I from the MELS algorithm. The parameter estimates and STD were normalized by the true parameter value. These values are plotted against SNR, and the normalized value is given as a dashed line in each plot. Fig. 6(a)–(e) shows that the slow-phase parameter estimates corresponded closely to those derived theoretically dB and their STD encompassed the true parameter for SNRs value. For fast-phase, Fig. 6(f)–(j) shows that the identified parameter values corresponded closely to those derived theodB and their STD encompassed the true retically for SNRs parameter value. For both slow and fast-phase parameters, at low SNR ( dB) the parameter STD included the true value but the parameter mean was significantly biased away from the true value. This bias occurs because of the increased noise variance. It is possible to reduce this bias if more data is available to estimate the coefficients [30]. Parameters for the error terms are not provided. They are significantly biased from their true value, as expected since the mean of these parameters corresponds to lagged error terms. Lagged error terms are difficult to identify accurately since they model output additive noise which is a stochastic process and cannot be measured. This stochastic process
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
439
Fig. 6. Normalized (by true value) discrete-time parameter estimates from MELS for simulated slow-phase (top row) and fast-phase (bottom row) dynamics: Sinusoidal input, Gaussian, white, zero-mean output additive noise. Ordinate: Normalized STD about normalized mean. Abscissa: Output SNR = 20; 15; 10; 5; 0 dB. (Note that the abscissa is shown in decreasing SNR which corresponds to increasing noise intensity.)
is modeled (approximated) as a deterministic signal of prediction errors which is only a poor estimate of the noise, but still allows for unbiased estimation of process coefficients. VII. EXPERIMENTAL DATA Lastly, we assessed the MELS algorithm on experimental human VOR data collected in the Dizziness Clinic at the Royal Victoria Hospital in Montréal, Canada as part of a study given ethics approval by the McGill Faculty of Medicine Institutional Review Board. The data analyzed in this study was from a single patient with a history of peripheral vestibular disease, with no function in the right inner ear following surgery. This is often associated clinically with a large nonlinearity in the VOR response and an abnormally small slow-phase time constant [36]. A. Experimental Procedures Silver-silver chloride electrodes were used to record conjugate eye position in the horizontal plane, in the dark. The subject remained in dim red light for 20 min to adapt to the dark condition which minimized electrode drift during recordings. The subject was then seated on a servo-controlled rotating chair, restrained by seat belts and a head holder. The head and body were fixed en-bloc to the chair during rotations, while the subject was instructed to perform mental arithmetic during rotations in the dark. The experimental protocol used a sinusoidal rotation at Hz, with a peak head velocity deg/s, since this is typical of clinical tests. The test lasted 52 s, of which the last 32 s were recorded to measure VOR properties during sensory steady state. Full electro-oculogram (EOG) calibrations were performed before and after the rotation, to correct for any drift. The experimental procedure is fully described in [39] and is
Fig. 7. Experimental VOR data. Top: Head velocity input. Bottom: Eye position output.
automated by Matlab based real-time control (Mathworks, Natwick, MA). Signals were sampled at 500 Hz (analog 40-Hz bandwidth after sixth-order Bessel filtering) and then digitally low-pass filtered to 15 Hz to reduce high frequency content. This 15-Hz bandwidth was sufficient to examine the slow-phase characteristics, but there are mild distortions on the fast-phase trajectories. Fig. 7 shows a typical input-output trial used for this analysis. The characteristics of this trial are consistent with those reported in previous work done in our laboratory [40]. The position traces were digitally differentiated to obtain eye and head velocity trajectories, and scanned by our classification algorithm to demark slow-phase segments automatically [37].
440
Fig. 8.
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
LNL model structure for a dual mode VOR system preceded by a first-order high-pass system representing vestibular dynamics.
B. Data Analysis It is known that head velocity is preprocessed by canal dynamics which serve as a sensor for head movement and processes in the brain-stem that modify apparent sensory function [34]–[36], [39]–[41]. This (global) sensory vestibular process is well described by a first-order, high-pass system [18], [34], [40], which transmits signals to the central circuits acting as a switched system. To model vestibular dynamics, we assumed a first-order highpass system, with time constant to be determined. The structure assumed for this identification is shown in Fig. 8, where and are defined as
(21) are first-order approximations of vestibular dynamics and are the poles of the vestibular high-pass system, . The nonlinearity and linear dynamics for and are the same as defined in (17). This notation allows for possible functional differences in the sensory signal as viewed by the brainstem during slow or fast phases. For identification of this structure, switched dynamics were identified from the reconstructed sensory signal to eye posias the input in the ELS and tion. Specifically, we used MELS algorithm to estimate model parameters (see Fig. 8). To estimate the time constant associated with vestibular dynamics, both identification algorithms fitted parameters assuming a vestibular time constant ranging from 1 to 15 s in increments of 1 s. The combination of vestibular time constant and switched system time constant which yielded the highest cross-validation %QF was deemed the best-fit model. For parameter estimation 48 slow and 45 fast-phase segments of the data; ) and were used ( for cross-validation 22 slow and 26 fast-phase segments were of the data; ). This used ( corresponds to the slow-phase mode having approximately 2.4 times more data than the fast-phase mode. The quality of fit was assessed by computing the %QF as given in (19) with a -step-ahead predictor as in (20).
Fig. 9. Cross-validated eye position output for experimental data. (a) Slow-phase eye position ELS algorithm. (b) Fast-phase eye position ELS algorithm. (c) Slow-phase eye position MELS algorithm. (d) Fast-phase eye position MELS algorithm.
C. Results The results presented in this section are a comparison of the quality of fit for the ELS algorithm and MELS algorithm with this linear-nonlinear-linear (LNL) structure. In addition, we compare the estimated continuous-time parameters and nonlinearity estimate obtained using each approach. Fig. 9 compares the %QF for the predicted trajectories with the cross-validation data set. The panels show the cross-validated eye position output implementing the ELS and MELS algorithm superimposed on top of measured slow and fast-phase eye position. Panels (a) and (b) show that ELS provides a poor fit (%QF 27.2 for slow-phase and 16.7 for fast-phase). Panels (c) and (d) demonstrate the clear improvement in fit with the MELS algorithm (%QF 98.5 for slow-phase and 86.3 for fast-phase). Fig. 10 shows the equivalent LNL structure block diagrams of the continuous-time estimates in slow and fast-phase modes obtained by the ELS (top) and MELS (bottom) algorithms. In
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
441
Fig. 10. LNL structure block diagrams of estimated continuous-time vestibular and switched system parameters for experimental data. (a): Slow-phase estimate ELS algorithm. (b): Fast-phase estimate ELS algorithm. (c): Slow-phase estimate MELS algorithm. (d): Fast-phase estimate MELS algorithm.
each case, the first block shows the estimated vestibular dynamics, the second shows the estimated static nonlinearity and the third shows the estimated first-order linear subsystem dynamics. The panel legends show slow and fast-phase vestibular , and slow/fast-phase switched system time constants, , respectively. Again, note that the actual time constants system is switched and shares states but is drawn as shown in Fig. 10 for clarity. For ELS the best-fit model gave a vestibular time constant s for slow-phase and s for fastof phase. The slow and fast-phase switched system time constants s and s, respectively. For MELS are the best-fit model gave a vestibular time constant of s for slow-phase and s for fast-phase. The slow and fast-phase switched system time constants in this case are s and s, respectively. The results in Figs. 9 and 10 show that the time constant estimates are at least qualitatively consistent with each other despite the poor %QF of ELS. However, the estimated nonlinearities with the ELS method are not physiologically compatible, since the same nonlinear sensory signal is expected to drive both slow and fast-phases. This is not surprising since we expect ELS to be a biased approach. The continuous-time parameters estimated using the MELS algorithm are consistent with other physiological measures of slow and fast-phase dynamics [34], [36], [40], [41]. Not only are the %QF in Fig. 9 excellent, but the nonlinearity in the two modes are mirror images of each other. This is consistent with a shared (nonlinear) vestibular process producing reversed responses in slow and fast-phase modes. VIII. DISCUSSION In more general approaches the identification scheme proceeds sequentially through the modes of a switched system, passing the estimated states directly to the next mode, e.g., [16]. In our approach we propose to estimate the initial states for pooled segments from a single mode for the following reasons.
1) In our problem of nystagmus analysis, the system does not stay long enough in one mode to allow good convergence to a parametric model that would produce a robust state estimate for the first sample of the next mode. 2) Some segment lengths in either mode are so short as to be completely unusable in the presence of noise and, hence, there are many “gaps” in the data record that must be handled. This means that in any case initial states after gaps would have to be estimated anyway. Gaps are caused by intermittent fast phases that cannot be avoided and, therefore, to estimate the initial conditions on a slow phase segment would require simultaneous modeling of the fast phase too in an continuous data stream. In a classified data stream (i.e., single un-concatenated data segment), the lagged input-outputs provide very noisy estimates of initial conditions and poor convergence on true dynamic parameters of the forcing transfer function. The central issue here is estimation of an accurate parameter vector. If the initial conditions are not taken into account the process parameters will be biased, even if the trajectory %QF may be reasonable, when appropriate initial conditions are used from the data set. Our problem is akin to identification with Matlab tools, selecting the optimal simulation performance, but in a stack of disconnected data. One can identify parameters based on the accuracy of one-step prediction alone, and so rely on delayed values in the regressor (ELS). However, we are not looking for a good predictor, but rather for an accurate model representation of the biological system. To faithfully model (simulate) a switched system, one must include in the equations the full state space representation for each segment requiring both the continuous forcing function and the intermittent initial conditions for each data interval. A. MELS Algorithm The success of parameter estimation methods for nonlinear systems depends upon the choice of model structure and the
442
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
development of estimation algorithms to yield unbiased estimates of system parameters. In the case of switched systems, it is crucial to also include the effects of initial conditions due to switching (transients). The limitation of our proposed algorithm for switched systems depends largely on the number of data segments and total data length available for estimation. If the number of measurements per subsystem, say , approaches infinity the statistics of the parameters will approach their asymptotic values. The limiting factors will depend not only on system dynamics but also on the feasibility of measuring the system for long periods of time. (embedding the initial conditions) will in The partition general be sparse and so it may be difficult to invert during estimation. If the dynamic order of the linear subsystem is large and/or the number of data segments is large the partition can become ill-conditioned and lead to numerical instability for many orthogonilazition algorithms. In this case it is possible to use orthogonilazition algorithms such as Givens-rotations to [42]. Another give a numerically stable orthogonalization of alternative would be to use an estimate of initial conditions directly from the data to build the extended regressor. This would not be as robust, but would certainly allow a much reduced regressor dimension for initial estimates.
The continuous-time parameters in Fig. 5 clearly demonstrate that even when some discrete-time parameters “appear” close to the theoretical values, associated continuous-time estimates may be far from the true values. This is due to the nonlinear mapping between the discrete and continuous-time domains (see Table IV), where errors may be amplified when the time constant is large. This study suggests that given equal data lengths: 1) when the time constant of a mode is large relative to segment length ( – s in VOR), the percent improvement from ELS to MELS fit may be small; 2) in the opposite case when the time constant is small, the bias in estimates from ELS can be quite severe (under-estimating the time constant in fast-phase and over estimating it seriously in slow-phase). Thus, without a priori knowledge, MELS is the preferred tool for identifying switched systems. The study of noise sensitivity shows that even with a poor forcing function and significant output additive noise, the proposed algorithm was able to estimate the parameters of this switched system. We expect the performance of this algorithm to be more robust when a richer input is used since the noise tolerance is a function of both input richness and noise amplitude [13], [30], [43]. Previous simulation results of VOR dynamics with a bandlimited white input supports this [23].
B. Simulation Study
C. Experimental Data
We explored the performance of MELS estimation, using a switched model intended to mimic the dual mode behavior of VOR nystagmus (VOR fast and slow-phases). An examination of discrete-time VOR fast-phase parameters (Table III) shows that the estimates are not as “accurate” as those of VOR slow-phase because the segment lengths are shorter. As a result, 14 times fewer data points were available to estimate the coefficients of subsystem 2 (VOR fast-phase). In the simulation (and experimental data) the input was a sinusoid. In general, a single sinusoid is a “terrible” input for identification. Therefore, it is tempting to conclude that the ELS results may simply be due to poor input design. However, though this certainly plays a role, the poor results of the ELS algorithm are mainly due to unmodeled dynamics in the residuals, e.g., ELS gives worse results for simulated fast-phase (small time constant). As the time constant of a mode decreases, switching effects themselves can bias ELS because decaying transients are more significant over the short intervals of data segments. MELS takes advantage of switching despite a poor forcing function, by explicitly correcting for decaying impulses: it effectively uses a high frequency input. In addition, the use of very short data segments will be problematic for the initial OLS estimate and, hence, may affect the overall convergence of the MELS algorithm. This problem is related to the number of switches per time span and the length of each data segment. Although we have assumed a nonzeno system, it only restricts the extreme case. In practice, the user will need to approximate what the system pole(s) is/are then make an a hoc decision as to what minimum segment length to use for estimation. In general, it may not be possible to determine some critical segmentation rate, even via simulations, since it depends on a combined effect of the true underlying system dynamics and noise.
A fundamental step of any multimode identification technique is to classify the signal into modes of operation. Classification is typically performed using ad hoc methods. With our application to experimental VOR data we classify the signal using an algorithm specifically designed for VOR [37]. This method, albeit specific to the VOR, is capable of accurately classifying VOR without resorting to ad hoc techniques. The need to classify the signal a priori is a limitation of any multimode identification technique. Analysis of a typical VOR record from a patient showed there is a large discrepancy between ELS and MELS, in the predicted outputs for both slow and fast-phase dynamics. The ELS estimates are likely biased since the transient effects are not modeled. Simulations showed that biases due to ignoring transient effects can be quite severe and variable depending on mode dynamics (see Table III and Fig. 5). The MELS algorithm provides a better fit to the data (see Fig. 9). Moreover, MELS estimates are consistent with a variety of physiological measures; see, e.g., [17], [34], [36], [39], [41], and [44]. However, there is an indication that the proposed VOR model structure may not be optimal. The MELS fit was less than optimal for some slow and fast-phase segments which may indicate further complexity in the system (see Fig. 9(b), e.g., 26.9–27.1 and 28.5–28.9 s or Fig. 9(d), e.g., 27.1–27.25 and 28.38–28.46 s). The fits for these segments may indicate a directionally sensitive nonlinearity, the existence of additional modes that should be further subdivided and analyzed separately, or that the nonlinear dynamics are not well represented by a LNL cascade, or even that the data may be time-varying, or simply noise effects. These factors remain to be evaluated. Overall, experimental analysis of VOR data indicates that switched identification may be appropriate for this type of system since it provides a much improved data fit, compared to
KUKREJA et al.: LEAST-SQUARES PARAMETER ESTIMATION ALGORITHM FOR SWITCHED HAMMERSTEIN SYSTEMS
classical ELS techniques and provides parameter estimates that are consistent with previous physiological studies. The concept of concatenating normally separated segments to produce an input-output record for system identification has much wider applicability than the present application. An example of this approach has been used to splice a number of exemplar protein sequences from each of several classes together to train a classifier for distinguishing between the classes [45]. D. Implementation While the MELS algorithm presented in Section IV.A.1 can be implemented as outlined, it is advantageous to use orthogonal functions to speed up computations. Between iterations only the prediction errors change but the , and terms do not. Therefore, it is more computationally efficient to order the prediction error terms last. Orthogonal functions can be created of the , and terms at the start of the estimation procedure which will not change between iterations. Ordering the prediction error terms last only requires these terms be re-orthogonalized. Their updates will not necessitate any changes to the earlier orthogonal terms. Details of how an orthogonal approach can be used with NARMAX models in an ELS framework can be found in Korenberg et al. [46] and a faster implicit orthogonalization in Korenberg [47]. Moreover, it is possible to estimate all subsystems simultaneously by creating a block-diagonal matrix which contains the regressor matrix for each subsystem on the diagonal. From a numerical point of view it may not be significantly faster to estimate all subsystems at the same time since a larger regressor matrix will contain more data and parameters to be solved for. E. Future Work Although this algorithm yielded good results for estimating the parameters of a switched system, it is unclear how to determine the model order and structure for these types of systems. Two techniques have been proposed for structure computation of unimodal nonlinear systems which may be generalized to compute an optimal structure in switched cases [47], [48]. Given the model structure limitations mentioned above for the VOR, it is clear that such tools are needed to properly explore reflex structures. IX. CONCLUSION The MELS method provides accurate estimates of parameters since it takes advantage of an entire data record even though the individual segments are short. These results may have a clinical significance in the analysis of ocular nystagmus of all types (pursuit, optokinetic, etc.). The technique here allows greater insight into the functionality of various ocular reflexes, by providing quantitative measures of both saccadic and slow ocular dynamics from a single experimental record. In addition, it should be applicable to other reflexes (e.g., stretch reflexes) where the presence of hard nonlinearities such as “rectification” can functionally imitate the behavior of switched systems. Hence, the MELS method may be useful to estimate the coefficients of complex Hammerstein structure switched systems in biology.
443
ACKNOWLEDGMENT The authors would like to dedicate this work in loving memory of Margherita B. Rapagna (August 25, 1968–May 20, 2002). REFERENCES [1] A. T. Geerts and J. Schumacher, “Impulsive-smooth behavior in multimode systems part I: State-space and polynomial representations,” Automatica, vol. 32, no. 5, pp. 747–758, 1996. [2] R. Alur and D. Dill, “Automata for modeling real-time systems,” in Proc. ICALP, 1990, pp. 322–335. [3] R. Alur and T. Henzinger, “Modularity for timed and hybrid systems,” in Lecture Notes in Computer Science, A. Mazurkiewicz and J. Winkowski, Eds. Berlin, Germany: Springer-Verlag, 1997, vol. 1243, Proceedings CONCUR’97: Concurrency Theory, pp. 74–88. [4] B. Bérard, P. Gastin, and A. Petit, “On the power of nonobservable actions in timed automata,” in , ser. Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 1996, vol. 1046, Actes du STACS’96, pp. 257–268. [5] T. Henzinger, “The theory of hybrid automata,” in Proc. 11th Symp. Logic in Computer Science, 1996, pp. 278–292. [6] C. Tomlin, G. Pappas, and S. Sastry, “Conflict Resolution for Air Traffic Management: A Case Study in Multi-Agent Hybrid Systems,” Department of Electrical and Computer Science, UC Berkeley, Berkeley, CA, Tech. Rep. UCB/ERL/M96/38, July 1996. [7] I. Leontaritis and S. Billings, “Input-output parametric models for nonlinear systems part I: Deterministic nonlinear systems,” Int. J. Control, vol. 41, no. 2, pp. 303–328, 1985. [8] S. Chen and S. Billings, “Representations of nonlinear systems: The NARMAX model,” Int. J. Control, vol. 49, no. 3, pp. 1013–1032, 1989. [9] S. Billings and W. Voon, “Least squares parameter estimation algorithms for nonlinear systems,” Int. J. Syst. Sci., vol. 15, no. 6, pp. 601–615, 1984. [10] V. Panuska, “A stochastic approximation method for identification of linear systems using adaptive filtering,” in Proc. 9th Joint Automatic Control Conf.. Ann Arbor, MI, June 1968, pp. 1014–1021. , “An adaptive recursive least squares identification algorithm,” in [11] Proc. 8th IEEE Symp. Adaptive Processes, University Park, PA, Nov. 1969, p. paper 6e. [12] P. Young, “The use of linear regression and relaxed procedures for the identification of dynamic processes,” in Proc. 7th IEEE Symp. Adaptive Processes, Los Angeles, CA, Dec. 1968, pp. 501–505. [13] G. Goodwin and R. Payne, Dynamic System Identification: Experiment Design and Data Analysis, ser. Mathematics in Science and Engineering. New York: Academic, 1977, vol. 136. [14] A. Bemporad, A. Garulli, S. Paoletti, and A. Vicino, “A greedy approach to identification of piecewise affine models,” in Lecture Notes in Computer Science. ser. , O. Maler and A. Pnueli, Eds. Berlin, Germany: Springer-Verlag, 2003, vol. 2623, Hybrid Systems: Computation and Control, pp. 97–112. [15] G. Ferrari-Trecate, M. Muselli, D. Liberati, and M. Morari, “A clustering technique for the identification of piecewise affine systems,” Automatica, vol. 39, no. 2, pp. 205–217, 2003. [16] J. Roll, A. Bemporad, and L. Ljung, “Identification of piecewise affine systems via mixed-integer programming,” Automatica, vol. 40, no. 1, pp. 37–50, 2004. [17] K. Chun and D. Robinson, “A model of quick phase generation in the vestibulo-ocular reflex,” Biol. Cybern., vol. 28, pp. 209–221, 1978. [18] H. Galiana, “A nystagmus strategy to linearize the vestibulo-ocular reflex,” IEEE Trans. Biomed. Eng., vol. 38, no. 6, pp. 532–543, Jun. 1991. [19] R. Schmidt and F. Lardini, “On the predominance of anti-compensatory eye movements in vestibular nystagmus,” Biol. Cybern., vol. 23, pp. 135–148, 1976. [20] C. Rey and H. Galiana, “Transient analysis of vestibular nystagmus,” Biol. Cybern., vol. 69, pp. 395–405, 1993. [21] I. Leontaritis and S. Billings, “Input-output parametric models for nonlinear systems part II: Stochastic nonlinear systems,” Int. J. Control, vol. 41, no. 2, pp. 329–344, 1985. [22] S. Billings and S. Chen, “Extended model set, global data, and threshold model identification of severely nonlinear systems,” Int. J. Control, vol. 50, no. 5, pp. 1897–1923, 1989. [23] S. Kukreja, H. Galiana, H. Smith, and R. Kearney, “Parametric identification of nonlinear hybrid systems,” in Proc. BMES/IEEE-EMBS, vol. 21, Atlanta, GA, Oct. 1999, p. 991.
444
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 3, MARCH 2005
[24] S. Daniel-Berhe and H. Unbehauen, “Bilinear continuous-time systems identification via hartley-based modulating functions,” Automatica, vol. 34, no. 4, pp. 499–503, 1998. [25] J. Sjöberg, Q. Zhang, L. Ljung, B. D. A. Benveniste, P. Glorennec, H. Hjalmarsson, and A. Juditsky, “Nonlinear black-box modeling in system identification: A unified overview,” Automatica, vol. 31, no. 12, pp. 1691–1724, 1995. [26] T. Tsang and S. Billings, “Identification of continuous time nonlinear systems using delayed state variable filters,” Int. J. Control, vol. 60, pp. 159–180, 1994. [27] H. Unbehauen and G. Rao, “Continuous-time approaches to system identification—A survey,” Automatica, vol. 26, no. 1, pp. 23–35, 1990. [28] V. Verdult, “Nonlinear system identification: A state-space approach,” Ph.D. thesis, Twente Univ., Press, Twente, The Netherlands, 2002. [29] T. C. Hsia, System Identification: Least-Squares Methods. Lexington, MA: Lexington Books, D.C. Heath, 1977. [30] L. Ljung, System Identification: Theory for the User, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall., 1999. [31] T. Söderström and P. Stoica, System Identification. Upper Saddle River, NJ: Prentice-Hall, 1989. [32] P. Stoica and T. Söderström, “Asymptotic behavior of some bootstrap estimators,” Int. J. Control, vol. 33, no. 3, pp. 433–454, 1981. [33] S. Kukreja, “Parametric methods for nonlinear system identification,” Ph.D. dissertation, Dept. Biomed. Eng., McGill Univ., 2001. [34] R. Leigh and D. Zee, The Neurology of Eye Movements, 2nd ed. Philadelphia, PA: F.A. Davis, 1991. [35] R. Carpenter, Movements of the Eyes, 1st ed. London, U.K.: Pion, 1977. [36] A. Katsarkas, H. Galiana, and H. Smith, “Vestibulo-ocular reflex (VOR) biases in normal subjects and patients with compensated vestibular loss,” Acta Otolaryngol., vol. 115, pp. 476–483, 1995. [37] C. Rey and H. Galiana, “Parametric classification of segments in ocular nystagmus,” IEEE Trans. Biomed. Eng., vol. 38, no. 2, pp. 142–148, Feb. 1991. [38] J. Proakis and D. Manolakis, Digital Signal Processing: Principles, Algorithms, and Applications, 3rd ed. Upper Saddle River, NJ: Prentice-Hall, 1996. [39] H. Smith and H. Galiana, “The role of structural symmetry in linearizing ocular reflexes,” Biol. Cybern., vol. 65, pp. 11–22, 1991. [40] H. Galiana, H. Smith, and A. Katsarkas, “Comparing linear and nonlinear methods for the analysis of the vestibulo-ocular reflex,” Acta Otolaryngol., vol. 115, pp. 585–596, 1995. [41] J. M. Findlay and R. Walker, “A model of saccade generation based on parallel processing and competitive inhibition,” Behav. Brain Sci., vol. 22, pp. 661–721, 1999. [42] G. Golub, Matrix Computation, 2nd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1993. [43] G. Box and G. Jenkins, Time Series Analysis: Forecasting and Control, 1st ed. San Francisco, CA: Holden-Day, 1970. [44] R. Baloh, K. Jacobson, K. Beukirch, and V. Honrubia, “Horizontal vestibulo-ocular reflex after acute peripheral lesions,” Acta Otolaryngol. (Suppl.), vol. 468, pp. 323–327, 1989. [45] M. Korenberg, R. David, I. Hunter, and J. Solomon, “Automatic classification of protein sequences into structure/function groups via parallel cascade identification: A feasibility study,” Ann. Biomed. Eng., vol. 28, pp. 803–811, 2000. [46] M. Korenberg, S. Billings, Y. Liu, and P. McIlroy, “Orthogonal parameter estimation algorithm for nonlinear stochastic systems,” Int. J. Control, vol. 48, no. 1, pp. 193–210, 1988. [47] M. Korenberg, “A robust orthogonal algorithm for system identification and time series analysis,” Biological Cybernetics, vol. 60, pp. 267–276, 1989. [48] S. Kukreja, H. Galiana, and R. Kearney, “A bootstrap method for structure detection of NARMAX models,” Int. J. Control, vol. 77, no. 2, pp. 132–143, 2004.
Sunil L. Kukreja received the B.S. degree in electrical engineering from Johns Hopkins University, Baltimore, MD, in 1993 and the M.Eng. in 1997 and Ph.D. degree in biomedical engineering in 2001, both from McGill University, Montreal, QC, Canada. From 2001–2002, he was with the Division of Automatic Control, Department of Electrical Engineering, Linköpings universitet, Sweden. From 2002–2004, he was with the McConnell Brain Imaging Center at the Montréal Neurological Institute, McGill University. He is currently a National Research Council (USA) Research Associate with the Structural Dynamics Group in the Aerostructures Branch at NASA Dryden Flight Research Center, Edwards, CA. His current research interest include algorithmic development for nonlinear systems, multiple internal state Wiener systems, bifurcating nonlinear systems, continuous-discrete-continuous mappings, structure computation, order estimation, modeling and identification of nonlinear aeroelastic behavior for high-performance aircraft, identification for biology, and uncertainty modeling for robust control.
Robert E. Kearney (M’76–SM’92–F’01) received the Ph.D. degree in mechanical engineering from McGill University, Montreal, QC, Canada, in 1976. He is a Professor and Chair of the Department of Biomedical Engineering at McGill University. He maintains an active research program that focuses on using quantitative engineering techniques to address important biomedical problems. Specific areas of research include: The development of algorithms and tools for biomedical system identification; the application of system identification to understand the role played by stretch reflexes and joint mechanics in the control of posture and movement; and the development of bioinformatics tools and techniques for proteomics. Dr. Kearney is a Fellow the Engineering Institute of Canada, the American Institute of Medical and Biological Engineering and a recipient of the IEEE Millennium medal.
Henrietta L. Galiana (M’87–SM’93–F’02) received the Bachelor’s degree in electrical engineering (Honors) from McGill University, Montreal, QC, Canada, in 1966, followed by the Master’s Elect. Eng. degree (biomedical) in 1968. After a few years working with L. Young at Massachusetts Institute of Technology’s (MIT’s) Man-Vehicle Lab, and a 7–year sabbatical, she returned to doctoral studies and received the Ph.D. degree in biomedical engineering in 1981. Following a Post-Doc at McGill’s Aerospace Medical Res. Unit, with Geoffrey Melvill Jones, she accepted a staff position in the Department of Biomedical Eng., where she now holds the position of Full Professor. Her research interests focus on signal processing and the modeling of control strategies for the orientation of eyes and head, and related issues of platform coordination and sensory fusion. Theoretical predictions are tested in the vestibular clinic for patient evaluation, and by porting to biomimetic robot systems. She is a past President (2002) of the IEEE Engineering in Medicine and Biology Society, and currently serves on the IEEE TAB Strategic Planning and Review Committee.
