A New Multi-Channel Pilot Model Identification Method for Use in ...

AIAA 2006-6629

AIAA Modeling and Simulation Technologies Conference and Exhibit 21 - 24 August 2006, Keystone, Colorado

A New Multi-Channel Pilot Model Identification Method for Use in Assessment of Simulator Fidelity F.M. Nieuwenhuizen,∗ P.M.T. Zaal,∗ M. Mulder,† and M.M. van Paassen† Delft University of Technology, P.O. Box 5058, 2600 GB Delft, The Netherlands For simulator fidelity research, insight into pilot control behaviour is an important tool. As pilots use multiple cues for the perception of self-motion, the control behaviour can be described with multi-channel pilot models. For identification of these models in the frequency domain a method using Fourier Coefficients is commonly used. In this paper an identification method using MISO ARX models is introduced and compared with the previous method using simulations. This method explicitly accounts for the remnant of the pilot, yielding continuous estimates of the pilot describing functions with lower variance. It is shown that this results in better estimates of the cross-over frequencies, phase margins and pilot model parameters. The forcing functions are subject to fewer constraints and are not required to be multi-sine signals commonly used in human operator research.

I.

Introduction

Flight simulators have become the main tool in pilot training and therefore it is important that a simulator is able to induce the same pilot control behaviour as a particular aircraft in the real world. The fidelity of a simulator is best evaluated by looking at skill-based pilot control behaviour,1 that can be evaluated with objective techniques, such as mathematical models. The parameters of these models can be determined with system identification and parameter estimation techniques. The pilot can perceive the aircraft motion through visual and motion cues. This process can best be described by a multi-channel pilot model. For the identification of multi-channel pilot models, up until now a spectral-based method using Fourier Coefficients was commonly used.2–6 This method imposes constraints on the design of the disturbance and target forcing functions in terms of energy inserted into the system and only a limited number of points in the frequency domain can be identified. This paper introduces a multi-channel pilot model identification method using Multi-Input-Single-Output (MISO) ARX models, which builds on recent developments of single-channel model identification with SingleInput-Single-Output (SISO) ARX models.7–9 The principal advantage of this parametric method is that it explicitly accounts for the remnant in the pilot control signal, resulting in significantly lower variances in the obtained estimates of the pilot describing functions. The two multi-loop identification methods are compared using data from closed-loop simulations. A general outline of the identification problem is given in Section II. Next, the multi-channel identification methods using Fourier Coefficients and ARX models are introduced in Sections III and IV. In Section V, the identification methods are compared using data of a simulated closed-loop tracking and disturbance task. Finally, the results are discussed in Section VI.

II.

The identification problem

The identification problem entails the open-loop identification and parametrisation of the dashed pilot block from Figure 1, where Hpe represents the error response of the pilot, Hpx the dynamics state response and Hnm the pilot neuromuscular dynamics. The controlled system dynamics are given by Hc , here a double integrator, representing aircraft roll behaviour. The task performed by the pilot includes following a target ∗ Research † Associate

Associate, Control and Simulation Division. Professor, Control and Simulation Division.

1 of 9 American Institute of Aeronautics Copyright © 2006 by Delft University of Technology. Published by the American Institute of Aeronauticsand and Astronautics Astronautics, Inc., with permission.

signal, ft , and correction of a disturbance signal on the controlled dynamics, fd , which both consist of a finite number of harmonics at discrete frequencies. P ilot ft - e e- 6 x-

n Hp e

fd

-? e u- ? e - 6

x-

Hc

Hp x

Figure 1: Closed-loop system used for simulations The linear multi-channel pilot model used in this paper for the closed-loop simulations of a tracking and disturbance task is a combination of the models proposed by Hosman10 and Van der Vaart11 and is given in Figure 2. This model consists of two perception paths and the neuromuscular dynamics of the human operator. Both perception paths include a gain and a time delay. The first path describes the central visual part of perception of the visual error signal e on a compensatory display and also includes a lead-time constant for the perception of the error rate signal. The other path represents vestibular motion perception that uses information of the state signal x. It includes a double differentiator, as humans are capable of sensing the acceleration of the dynamics state signal, x, and the dynamics of the semi-circular canals, Hv . n e

x

- Kv (1 + jωτvl ) - e−jωτv - e- Hnm - ? e u - 6

-

- (jω)2 - Hv (ω) - Km - e−jωτm

Figure 2: Multi-channel pilot model The neuromuscular dynamics of the pilot are given by: Hnm =

2 ωnm 2 2 + 2ζ ωnm nm ωnm jω + (jω)

(1)

and the vestibular dynamics Hv are: Hv =

1 + jωτv1 1 + jωτv2

(2)

The non-linear part of the pilot behaviour is captured by the remnant, n, which is added to the linear part of the pilot control input. By comparing Figure 1 and 2 one can see that the pilot error response Hpe represents the central visual part of perception and the pilot state response Hpx the motion perception part. Both response functions also include the pilot neuromuscular dynamics. The response functions can be used in a parameter estimation procedure in which the parameters of the multi-channel model and the neuromuscular dynamics ˆp . ˆ p and H are determined by fitting the pilot model to the identified frequency responses H x e The identification using Fourier Coefficients entails some difficulties. As interpolation of signals from the frequencies of one forcing function to the frequencies of the other is required to come to a solution,2 the design of the forcing functions is elaborate. The discrete frequencies where energy is present should alternate between the forcing functions, but should also be close enough to avoid too large errors during interpolation. Also, the forcing functions should excite the human operator dynamics at the frequency range of interest. Moreover, the power content of the signals should be chosen carefully, avoiding possible cross-over regression 2 of 9 American Institute of Aeronautics and Astronautics

effects12 but still providing enough power content for identification. In case of identification using ARX models, the forcing function design is less strict. No interpolation is required and simulations show that less frequency content in the range of interest is still adequate for a successful identification.

III.

Identification using Fourier Coefficients

This method has been used in previous experiments and has been described thoroughly.2, 3, 13 The pilot control signal u in Figure 1 at an arbitrary frequency ν1j of forcing function fd is given in the frequency domain as:

U1 = Hpe ν1j E1 − Hpx ν1j X1 + N1

(3)

In order to be able to solve Equation 3 for the pilot describing functions a second equation is needed. Therefore, the Fourier transformed pilot control signal at the frequencies of the forcing function ft is inter˜2 . The contributions of the remnant noise, N1 and N ˜2 , are polated to the frequencies of fd and denoted by U assumed to be small as, generally, the Signal-to-Noise Ratios (SNR) are high at the input frequencies and are therefore neglected. This yields a set of equations at the frequencies of fd , given in Equation 4, from which the pilot describing functions can be solved. #" # " "
# Hpe ν1j E1 −X1 U1
(4) = ˜2 −X ˜2 ˜2 Hpx ν1j E U ˆ pe and H ˆ px However, the contributions of the remnant noise are in reality still present in the estimates H and will therefore have an influence on the bias and variance of the estimates. expressions for the Analytical ˆ ˆ bias and variance have been derived previously.2, 3 The variance of H and H are given in the following px pe equations. 
 ˆ Var H pe ν1j


 ˆ Var H ν px 1j


1 ˆ 2 (1 − 2ǫ1 + ǫ2 ) = H ǫ2 − ǫ21 + pe r˜2 2
ˆ p + 1 ǫ2 − ǫ21 + 1 (1 − 2ǫ1 + ǫ2 ) = H x Hc r˜2

(5) (6)

The expressions for expectations ǫ1 and ǫ2 are given by:14

ǫ1 (ν1j ; ζ)

=

ǫ2 (ν1j ; ζ)

=

 N1 (ν1j ; ζ) E Fd (ν1j ) + N1 (ν1j ; ζ) ( 2 ) N1 (ν1j ; ζ) E Fd (ν1j ) + N1 (ν1j ; ζ) 

(7) (8)

Expectations ǫ1 and ǫ2 can be computed analytically as a function of the SNR.3 When the SNR becomes large enough, i.e. > 5, the variance becomes very small. ˆ p can be approximated with: The variance of 6 H e   ˆ p    180 2 Var H e ˆp ≈ (9) Var 6 H e π ˆ 2 H pe ˆp . A similar expression holds for the variance of 6 H x

IV.

Identification using ARX models

This identification method is based on the concept of fitting a MISO ARX model on the measured input signals e and x and output signal u, such that the properties of Hpe and Hpx are accurately described in the

3 of 9 American Institute of Aeronautics and Astronautics

time and frequency domain, see Figure 3. The parameters of A, B1 and B2 of the ARX model have to be determined such that the following equation holds: u(t) =

B2 (q) 1 B1 (q) e(t) + x(t) + nw (t) A(q) A(q) A(q)

(10)

Here the remnant of the pilot, n, is assumed to be filtered white noise nw . The parameters of A and B can be determined using a standard least squares method. The orders of the A and B polynomials can be determined by physical insight into the system to be identified, by looking at the application of the identified model, or by calculating them. If for example a pilot model is fitted to the identified pilot describing functions, one could look at the model order of the pilot model to determine the order of the polynomials of the ARX model. For calculating the orders different techniques are available. An example is to take a range of orders for each polynomial and choosing the set of orders that produces the smallest final prediction error.15 nw

? 1 A

e x -

n

B1 A

-? e 6

u-

B2 A

Figure 3: Multi-input ARX model structure For identification of the pilot describing functions the sampling frequency should not be too high, as the high-frequency noise contributions are not of interest and should not be captured by the ARX model. On the other hand, it is important that the sampling frequency and thus the Nyquist frequency is high enough to capture all the useful information. The choice of sampling frequency for measuring the data is thus very important for the noise reduction in the estimate. The data should be resampled in order to eliminate the noise contributions. An antialias filter must be applied before the data is resampled in order to not let the folding effect distort the interesting part of the spectrum below the Nyquist frequency.15 The cutoff frequency of the filter should be equal to the Nyquist frequency of the resampled signal. ˆ p are now given by the polynomials of the estimated ARX model in the ˆ p and H The estimates H x e frequency domain: ˆ p (jω) = B1 (jω) H e A (jω)

(11)

ˆ p (jω) = B2 (jω) H x A (jω)

(12)

ˆp Off course, one is interested in the variance of the magnitude and phase of the frequency response of H e 15 ˆ ˆ ˆ and Hpx . The variance of the magnitude of the frequency response of Hpe or Hpx is given by:      2  2 ˆ p C3 ˆ p Im H ˆ p C2 ˆ p C1   Re H 2Re Im H H ˆ − + Var H 2 2 2 p = ˆ ˆ ˆ Hp Hp Hp

The variance of the phase in degrees is defined as:   2      2  ˆ p C3 ˆ p Im H ˆ p C2 ˆ p C1    180 2 Im H 2Re Re H H   ˆp = − + Var 6 H   4 4 4 π ˆ ˆ ˆ Hp Hp Hp 4 of 9 American Institute of Aeronautics and Astronautics

(13)

(14)

with

C1

=

C2

=

C3

=

Re

ˆp ∂H ∂θ

!

Im

ˆp ∂H ∂θ

!

Re

ˆp ∂H ∂θ

!

P (θ) Re

ˆp ∂H ∂θ

!∗

P (θ) Im

ˆp ∂H ∂θ

!∗

P (θ) Im

ˆp ∂H ∂θ

!∗

(15)

Here C1 , C2 and C3 are the entries of the covariance matrix of the real and imaginary part of the Fourier ˆ p . In these equations ∗ is the Complex conjugate transpose, P (θ) is the parameter ˆ p or H Coefficients of H x e ˆ p /∂θ is the differentiation of H ˆ p with covariance matrix of the ARX model, θ is the parameter vector and ∂ H respect to the parameter set.

V.

Results

The results in this paper are produced using data from 10000 closed-loop simulations with the pilot model given in Figure 2. Care should be taken when comparing the identification methods. The Fourier Coefficient method is a spectral-based non-parametric method, while the method using ARX models is a parametric method. The method using ARX models should perform better since more knowledge is incorporated into the estimators. The results in this section can, however, show the advantages and disadvantages of using a parametric method for identification of pilot frequency response functions. The parameters, which were used for the pilot model, determine an analytical pilot model which can be compared with the identified frequency response functions. The controlled dynamics, Hc , are a double integrator with constant gain, representing the roll dynamics of an aircraft. The pilot remnant, n, is modelled as white noise, filtered with a first order low-pass filter with a cut-off frequency of 5 rad/s. The properties of forcing functions fd and ft are similar as in previous research.6 2

2

10

10

1

1

10

Hˆ pe

Hˆ px

10

0

0

10

10

−1

−1

10

10

−2

10

−2

−1

10

0

10

10

1

10

200

200

100

100

0

10

1

10

0

ˆ px H

0 −100

−100

6

6

ˆ pe H

−1

10

−200

−200

−300

−300

−400 −1 10

0

10

−400 −1 10

1

10

f requency [rad/s]

Analytical FC ARX 0

10

1

10

f requency [rad/s]

Figure 4: Bode plot of the identified frequency responses The identification of the pilot response functions Hpe and Hpx for a single simulation can be found in Figure 4. Both the results from the Fourier Coefficient method and the method using ARX models are shown. From the figure one can see that the Fourier Coefficient method only gives an identification on 5 of 9 American Institute of Aeronautics and Astronautics

the frequencies inserted by the forcing function, whereas the method using ARX models gives a continuous estimate of the response functions. It is clear that the Fourier Coefficient method produces noisy results as the identified points are scattered around the line of the analytical model. The estimate identified using ARX models contains less noise as the remnant was explicitly accounted for in the ARX model. For the Fourier Coefficient method the standard deviation is given by the vertical bars and for the method using ARX models by the dashed lines. The standard deviation of the estimate from the method using ARX models is much smaller than the standard deviation in the estimate of the Fourier Coefficient method. 1

1

10

10

0

0

10


Hˆ pe


Hˆ px

10

−1

−1

10

std

std

10

−2

10

−3

10

−2

10

−3

−1

10

0

10

10

1

10

−1

10

1

10


ˆ px H 6

6

10




2

10

ˆ pe H

2

0

10

0

std

std

0

10

−2

10

10

FC calc. FC sim. ARX calc. ARX sim.

−2

−1

10

0

10

10

1

10

−1

10

f requency [rad/s]

0

10

1

10

f requency [rad/s]

Figure 5: Variance of the identifications (calculated and 10000 simulations) In Figure 5, the standard deviations obtained from the simulation are compared for both identification methods. Also, the analytically calculated standard deviations are given. One can see that the standard deviations from the ARX model estimate are much lower than the ones from the Fourier Coefficient estimate. It can also be seen that the mean analytically calculated standard deviations and the standard deviations of 10000 simulations coincide very well for the ARX model method. For the Fourier Coefficient method the assumptions used to create the equations of the standard deviations result in a slightly worse approximation of the real standard deviations of 10000 simulations. The identified frequency responses serve as input for a parameter estimation procedure in which the parameters of the multi-channel model and the properties of the neuromuscular system are estimated. The semicircular canal dynamics are taken constant. This leaves seven parameters to be estimated, given in Table 1. A simple cost criterion is used which consists of the squared error between the identified and parametrised model at a certain frequency scaled with the variance. The scaling serves as a means to put less emphasis on identified points with a large variance. Figure 6 shows the magnitudes of the frequency response of the parametric models and the analytical frequency responses for Hpe and Hpx . The estimated frequency responses show good results for both identification methods. The mean of the absolute error in the estimated parameters and their 95% confidence intervals are given in Figure 7. This figure shows that the errors in the parameters are smaller when the pilot model is fitted to the response functions estimated with the ARX model. Analysis of variance confirms that all changes in error are significant. Table 1 gives the mean values and the variance of the parameters that are calculated from 10000 simulations. One can clearly see that the variance of the parameters estimated using the identified frequency responses of the ARX model method is much lower and that less bias is present in the parameters. This can be attributed to the lower variance of the ARX model estimate and the fact that this estimate is continuous. For comparison of the identification methods and parameter estimations some statistical measures are used. The Root Mean Squared Error (RMSE) quantifies the error between the estimate and the true pilot response function, by taking the root of the summation of the squared errors divided by the number of 6 of 9 American Institute of Aeronautics and Astronautics

2

2

10

10

1

1

10

H˜ pe

H˜ px

10

0

0

10

10

−1

−1

10

10

−2

10

−2

−1

0

10

10

1

10

10

200

200

100

100

0

1

10

10

0

˜ px H

0 −100

−100

6

6

˜ pe H

−1

10

−200

−200

−300

−300

−400 −1 10

0

−400 −1 10

1

10

Analytical FC ARX

10

f requency [rad/s]

0

1

10

10

f requency [rad/s]

Figure 6: Bode plot of the parameter estimations

0.07

2

0.012

0.06

0.01 1.5

|Error|

0.05

0.012

0.05

0.5

0.2

0.01

0.04

0.4

0.03

0.3

0.02

0.2

0.01

0.1

0.008

0.008 0.15

0.04 1

0.006

0.006

0.03

0.1 0.004

0.02

0.004

0.5

0.05

FC par. 0.002 ARX par.

0.01 0

0.25

Kv

0

τvl

0

τv

0

0.002

Km

0

τm

0

ζnm

0

ωnm

Figure 7: Parameter errors and 95% confidence intervals (10000 simulations)

Table 1: Comparison of the estimated parameters

Kv τvl τv Km τm ζnm ωnm

Simulated 0.17 2.93 0.32 1.59 0.29 0.30 12.0

FC 0.11 4.42 0.33 1.40 0.30 0.28 12.2

ARX 0.17 3.06 0.31 1.57 0.28 0.28 11.6

FC var 2.40 10−3 1.98 10−0 1.61 10−3 5.94 10−2 3.44 10−3 7.01 10−3 1.07 10−0

ARX var 2.03 10−3 8.51 10−1 1.31 10−5 6.82 10−3 2.23 10−5 2.30 10−4 2.58 10−2

7 of 9 American Institute of Aeronautics and Astronautics

1

2.5

0.8

2

0.6

1.5

ωc

|Hpe |

data points. The Weighted Mean Squared Error (WMSE) is comparable to the RMSE, but weighted with the variance of the estimates. This gives a measure for the quality of fit in combination with the variance. Finally the Summed Mean Variance (SMV) is a summation of the variances divided by the number of data points. This is a measure for the amount of variance in the estimates. For both identification methods the statistics are determined using only the data on the input frequencies. The cross-over frequencies and phase margins are also used as a measure for the quality of fit. These are determined from the separate open-loop dynamics of the error response function and the state response function of the identifications and parameter estimations. Figure 8 gives the statistical measures and the cross-over and phase margins of 10000 simulations. The figure shows the means and the 95% conficence intervals of 10000 simulations. The statistics for the identification and parameter estimate of the method using ARX models always have the lowest value compared to the method using Fourier Coefficients, meaning that the identification and estimate are more accurate and have a lower variance. Also the cross-over frequency and phase margin are estimated more accuratly with the identification and parameter estimate resulting from the method using ARX models. The ARX model estimate is continuous, so there are less interpolation errors. A large error is present for the phase margins of the frequency responses estimated with the Fourier Coefficient method. This is because the already large interpolation error from determening the cross-over frequencies is adding to the interpolation error when determening the phase margins.

0.4

1

0.2

0.5

RM SE

W M SE

0

SM V

1

100

0.8

80

0.6

60

Hpe

Hpx

ϕp

|Hpx |

0

0.4

40

0.2

20

0

RM SE

W M SE

0

SM V

F C id. ARX id. F C par. ARX par. Hpe

Hpx

Figure 8: Means and 95% confidence intervals of the statistics (10000 simulations)

VI.

Discussion

The parametric multi-loop identification method using MISO ARX models performs well in identifying the multi-channel pilot model used during simulations. The method was compared with results from a non-parametric identification method using Fourier Coefficients. Based on the comparison, the following conclusions can be drawn. • The identification using ARX models gives a more accurate estimate compared to the method using Fourier Coefficients, as the remnant is accounted for in the ARX model. Also, this identification is continuous in the frequency domain, while the method using Fourier Coefficients only identifies the frequency response at the frequencies of the forcing functions. • The variance in the identification using ARX models is lower than in the identifications using Fourier Coefficients. The mean analytically calculated variance of both methods show good resemblance with

8 of 9 American Institute of Aeronautics and Astronautics

the variance found in multiple simulations and thus the analytical calculations can be trusted to provide accurate results. • The cross-over frequencies and phase margins, which are important metrics in evaluating human control behaviour, can be more accurately estimated when using the identification using ARX models. • In the case of identification using ARX models, the design of the forcing functions is less elaborate. One does not have to take the alternating frequencies of the forcing functions used in identification using Fourier Coefficients into account, as interpolation is not required. Also, the forcing functions are not required to be multi-sine signals, but should be deterministic. • The parametric model resulting from the identification using ARX models is more accurate compared to the parametric model resulting from the identification using Fourier Coefficients. The higher accuracy of the identification using ARX models and the smaller variance in the identified frequency response results in a better fit of the model parameters. The multi-channel pilot identification method using ARX models discussed in this paper can be extended. For the new pilot model identification method only ARX models are used, but it is in fact possible to use different Linear Time-Invariant model structures, such as the ARMAX or Output Error model structure. The use of, for example, the ARMAX model might lead to better results for the estimate of the remnant due to an additional polynomial. Also, when simulations are used to compare the identification methods, analytical calculations of the bias in the estimates are possible. These calculations should be compared with data from multiple simulations.

References 1 Rasmussen, J., “Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance Models,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, No. 3, 1983, pp. 257–266. 2 van Paassen, M. M., Biophysics in Aircraft Control. A model of the neuromuscular system of the pilot’s arm, Ph.D. thesis, Faculty of Aerospace Engineering, Delft University of Technology, 1994. 3 Mulder, M., Cybernetics of tunnel-in-the-sky displays, Ph.D. thesis, Faculty of Aerospace Engineering, Delft University of Technology, 1999. 4 Kaljouw, W. J., Mulder, M., and van Paassen, M. M., “Multi-loop Identification of Pilot’s Use of Central and Peripheral Visual Cues,” Proceedings of the AIAA Modelling and Simulation Technologies Conference and Exhibit, Providence (RI), No. AIAA-2004-5443, Aug. 16–19 2004. 5 Mulder, M., Kaljouw, W. J., and van Paassen, M. M., “Parametrized Multi-Loop Model of Pilot’s Use of Central and Peripheral Visual Motion Cues,” AIAA Modeling and Simulation Technologies Conference and Exhibit, San Francisco (CA), No. AIAA-2005-5894, Aug. 15–18 2005. 6 L¨ ohner, C., Mulder, M., and van Paassen, M. M., “Multi-Loop Identification of Pilot Central Visual and Vestibular Motion Perception Processes,” Proceedings of the AIAA Modelling and Simulation Technologies Conference, San Francisco, (CA), No. AIAA-2005-6503, Aug. 15-18, 2005. 7 Schouten, A. C., Proprioceptive Reflexes and Neurological Disorders, Ph.D. thesis, Faculty of Mechanical Engineering, Delft University of Technology, 2004. 8 de Vlugt, E., Identification of Spinal Reflexes, Ph.D. thesis, Faculty of Mechanical Engineering, Delft University of Technology, 2004. 9 Pintelon, R. and Schoukens, J., System Identification: A Frequency Domain Approach, IEEE, Inc., 2001. 10 Hosman, R. J. A. W., Pilot’s perception and control of aircraft motions, Ph.D. thesis, Faculty of Aerospace Engineering, Delft University of Technology, 1996. 11 van der Vaart, J. C., Modelling of perception and action in compensatory manual control tasks, Ph.D. thesis, Faculty of Aerospace Engineering, Delft University of Technology, 1992. 12 McRuer, D. T. and Jex, H. R., “A Review of Quasi-Linear Pilot Models,” IEEE Transactions on Human Factors in Electronics, Vol. HFE-8, No. 3, 1967, pp. 231–249. 13 van Paassen, M. M. and Mulder, M., “Identification of Human Operator Control Behaviour in Multiple-Loop Tracking Tasks,” Proceedings of the Seventh IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design and Evaluation of Man-Machine Systems, Kyoto Japan, Sept. 16–18 1998, pp. 515–520. 14 van Lunteren, A., Identification of Human Operator Describing Function Models with One or Two Inputs in Closed Loop Systems, Ph.D. thesis, Faculty of Aerospace Engineering, Delft University of Technology, 1979. 15 Ljung, L., System Identification Theory for the User , Prentice Hall, Inc., 2nd ed., 1999.

9 of 9 American Institute of Aeronautics and Astronautics