dead zone, and so on. A new system identification method which estimates the sensor errors due to nonlinearity, as well as system parameters, is proposed.
IFAC Workshop on Adaptation and Learning in Control and Signal Processing, and IFAC Workshop on Periodic Control Systems, Yokohama, Japan, August 30 – September 1, 2004
SYSTEM IDENTIFICATION IN THE PRESENCE OF NONLINEAR SENSORS Shuichi Adachi1) , Yasushi Okada1) and Jan M. Maciejowski2) 1) Department of Electrical and Electronic Engineering, Utsunomiya University, 7-1-2 Yoto, Utsunomiya, 321-8585, Japan 2) Engineering Department, Cambridge University
Abstract: We consider a system identification problem in which input and output signals are measured using sensors with static nonlinear characteristics, such as saturation, dead zone, and so on. A new system identification method which estimates the sensor errors due to nonlinearity, as well as system parameters, is proposed. The key idea is to formulate the identification problem as a constrained optimization problem. The system parameters are then estimated using a quasi-Newton method. Effectiveness of the proposed method is examined through numerical experiments. Keywords: modeling, system identification, constrained optimization problem.
1. INTRODUCTION System identification theory for linear systems has been well-established (Ljung, 1999) and many applications of system identification have been reported. On the other hand, system identification theory for nonlinear systems has not been established so systematically, because of its extremely wide scope (Nells, 2001). When system identification experiments are carried out in open-loop, it can often be assumed that the input signal is completely known without any error. However, when the input is determined by a feedback loop or the input signal is applied as an output of another system, it is necessary to measure the input signal as well as the output signal. Moreover, it may be difficult to use linear sensors because of various practical reasons, such as cost. In such cases, the input and output signals have to be measured using sensors with nonlinear characteristics, thresholds, etc. In this paper, a system identification problem is considered in which input and output signals are measured by using sensors with static nonlinearities such as saturation, dead zone, and so on. A new system identification method which estimates the sensor errors due to nonlinearity, as well as system parameters, is proposed. The key idea is to formulate the identification problem as a constrained optimization problem in which the cost function consists of the squared sum of prediction error and the nonlinear errors. Then both system parameters and nonlinear errors can be estimated by using numerical
Input signal
u(t )
Linear system
Sampling
Sampling
u (k ) Static nonlinearity
Measured input signal u (k )
Output signal y(t )
y (k ) nonlinear sensor
Static nonlinearity
Measured output signal y (k )
Figure 1: A system identification problem involving a linear system and nonlinear sensors
optimization methods, such as quasi-Newton or Gauss-Newton. The effectiveness of the proposed method is examined through numerical experiments. 2. IDENTIFICATION OF LINEAR SYSTEM USING NONLINEAR SENSORS 2.1 Problem description In this paper, the system identification problem shown in Fig.1 is considered. Plant to be identified is assumed to be a linear system, on the other hand the input and output signals are measured by using sensors with static nonlinearity such as saturation and dead-zone. Note that the set-up in Fig.1 is different from commonly considered systems such as the Hammer-
185
noted by ±L . We will assume L = L without loss of generality. The formulations above assume that
output L
-L
0
L
δ(k) ≥ 0
input
-L
y(k) + a1 y(k − 1) + · · · + an y(k − n) = b1 u(k − 1) + · · · + bn u(k − n) + w(k)(1) where u(k) is the input signal, y(k) is the output signal, and w(k) is a white noise. It is assumed that the order of the system n is known; this is rather restrictive, but one might have an estimate of n from a preliminary conventional identification step, for example. The sensor nonlinearities which we consider in this paper are saturation, dead-zone, and combined dead-zone with saturation. We assume that the nonlinear characteristics of the sensors are known exactly. 2.2 Sensors with saturation 2.2.1 Problem formulation The sensor characteristic shown in Fig.2 is considered. In Fig.1, the output signal measured by the nonlinear sensor is denoted by y¯(k) and the error between this and the signal y(k) at the output of the system is denoted by δ(k). Then an estimate of the actual output signal yˆ(k) is written as y¯(k) + δ(k), if y¯(k) = L y¯(k), if |¯ y (k)| < L yˆ(k) = y¯(k) − δ(k), if y¯(k) = −L Similarly, an estimate of the actual input signal u ˆ(k) can be written in terms of the measured input signal u ¯(k) and an error γ(k) as ¯(k) + γ(k), if u ¯(k) = L u u ¯(k), if |¯ u(k)| < L u ˆ(k) = u ¯(k) − γ(k), if u ¯(k) = −L where the linear range of the output sensor is denoted by ±L and that of the input sensor is de-
γ(k) ≥ 0
(2)
The equation error at time k denoted by e(k) is given by
Figure 2: Saturation nonlinearity
stein model, in which a nonlinear component is attached to input side of a linear system, which might be appropriate when a linear system is driven by a nonlinear actuator. We assume that the nonlinearities affect only the measurements, not the signals which actually enter or leave the system being identified. We assume that the input-output relationship of the system being identified is given by
and
e(k)
= yˆ(k) + a1 yˆ(k − 1) + · · · + an yˆ(k − n) ˆ(k − 1) − · · · − bn u ˆ(k − n) (3) − b1 u
It is noted that the actual input and the output signals can not be used to calculate the equation error. When quantization error is dominant in the input and output signals, Okao et al (2002) introduced a new term related to the quantization error into the system identification cost function in addition to the usual squared sum of the equation error. In this paper, we adopt the same framework, so the key idea is to formulate the identification problem as a constrained optimization problem in which the cost function consists of the squared sum of prediction error and the errors due to sensor nonlinearity. 2.2.2 Solution We solve the system identification problem by solving the following constrained optimization problem: min
N
e2 (k),
s.t. δ(k) ≥ 0, γ(k) ≥ 0
(4)
k=1
where N is the number of input-output samples. In order to transform the constrained optimization problem into the unconstrained one, a penaltymethod approach is taken. A new cost function which penalises negative saturation errors δ(k) and γ(k) is introduced: J=
N k=1
e2 (k) +
N
g(k) +
i=1
N −1
h(k)
(5)
i=1
where g(k) is a penalty function on δ(k) defined as follows: 2 ρδ (k), if δ(k) < 0 g(k) = (6) 0, if δ(k) ≥ 0 where ρ is a weighting constant. Similarly, a penalty function h(k) on γ(k) is defined by ξγ 2 (k), if γ(k) < 0 h(k) = (7) 0, if γ(k) ≥ 0 where ξ is a weighting constant.
186
3
Output
Output
3
0
−3 650
700 Number of data
−3 650
750
0
−3 650
700 Number of data
750
700 Number of data
750
20 0 −20 −40 1
10
10
2
10
2
Frequency [rad/s] 0 −100 −200 −300 1
10
Frequency [rad/s]
Figure 4: Identified frequency response functions for saturated signals (solid line: proposed method, dash-dotted line: conventional method, dotted line: true)
The optimization problem which minimizes Eq.(5) can be solved effectively by applying the quasiNewton method (Gill et al, 1981). Then both the parameters in Eq.(3) and the values of δ(k) and γ(k) can be estimated simultaneously. As initial values for parameters, the conventional leastsquares estimates are used, and initial values of δ(k) and γ(k) are set to zero. 2.2.3 A numerical example A 2nd-order vibrational system was assumed as the true system, whose parameters are a1 = −1.325, a2 = 0.8825, b1 = 0.2135, b2 = 0.2046. The input signal was a white Gaussian noise with distribution N (0, 0.82 ). Another white Gaussian noise, independent of the input signal, was used to perturb the output signal, as in Eq.(1). Nonlinear sensors with saturation were used with L = 1. The input and output signals are shown in Fig.3, and the number of samples used for identification
0
−3 650
750
Figure 3: Input and output signals measured by sensor with saturated nonlinearity (solid line: measured, dotted line : true)
Gain [dB]
700 Number of data
3
Input
Input
3
Phase [deg]
0
Figure 5: Estimated input and output signals for saturated data (solid line: estimated, dash-dotted line: measured, dotted line: true)
was N = 200. The BFGS (Broyden-Fletcher-Goldfarb-Shanno) formula was employed for approximating the Hessian in the quasi-Newton method. ρ = ξ = 10 were used in the penalty functions. The iteration of the optimization was terminated when the change of the cost function became less than 1 × 10−5 . The frequency response of the resulting identified system is shown in Fig.4, together with the results of using a conventional linear identification method. It is clear from the figure that the conventional method (dash-dotted lines) does not identify the frequency response, whereas the proposed method (solid lines) almost agrees with the true system (dotted lines). The estimated input and output signals are shown in Fig.5. In the proposed method, these signals were reconstructed by using both the estimated parameters and the estimated errors, and they agree very well with the original input and output signals. Define the parameter estimation error: PERR =
ˆ 2 θ ∗ − θ θ ∗ 2
(8)
where θ ∗ is the true value of the parameter vector and θˆ is its estimate. The mean value of PERR for the proposed method, evaluated over over 10 trials, was 0.0002, whereas PERR for the conventional method was 0.0552. It can be seen that the proposed method estimates the system parameters very accurately. 2.3 Sensors with dead-zone 2.3.1 Problem formulation Now consider a dead-zone characteristic, as shown in Fig.6, for the sensors. It is assumed that the
187
output Gain [dB]
20
-D 0 D
0 −20 −40
input
10
0
10 Frequency [rad/s]
1
10
2
1
10
2
Phase [deg]
0
Figure 6: Dead-zone nonlinearity
−100 −200 −300 0 10
10 Frequency [rad/s]
Output
3
Figure 8: Identified frequency response functions for dead-zone signals (solid line: proposed method, dotted line: initial value, dashed line: least-squares method, dash-dotted line: true)
0
−3 500
550 Number of data
600
Input
3
0
We now identify the system by solving the following constrained optimization problem:
−3 500
550 Number of data
600
min Figure 7: Input and output signals measured by sensor with dead-zone nonlinearity (solid line: measured signal, dotted line : true signal)
value D which defines the dead-zone is known. Again, the nonlinear characteristics of both input and output sensors are assumed to be the same. The measued values are first corrected as follows: ¯(k) + D, if u ¯(k) > 0 u 0, if u ¯(k) = 0 (9) u ¯ (k) = u ¯(k) − D, if u ¯(k) < 0 y¯(k) + D, if y¯(k) > 0 0, if y¯(k) = 0 (10) y¯ (k) = y¯(k) − D, if y¯(k) < 0
This correction is exact if |¯ u (k)| > D or |¯ y (k)| > D. Otherwise the true input and output signals need to be estimated, and we do this by writing: α(k), if |¯ u (k)| ≤ D (11) u ˆ(k) = u ¯ (k), if |¯ u (k)| > D β(k), if |¯ y (k)| ≤ D (12) yˆ(k) = y¯ (k), if |¯ y (k)| > D where α(k) and β(k) are new variables which satisfy the constraints: |α(k)| ≤ D, 2.3.2 Solution
|β(k)| ≤ D
(13)
N
e2 (k),
s.t.
|α(k)| ≤ D, |β(k)| ≤ D
k=1
(14) In order to solve this problem, two penalty functions ρ[|α(k)| − D]2 , if |α(k)| > D g(k) = 0, if |α(k)| ≤ D 2 ξ[|β(k)| − D] , if |β(k)| > D h(k) = 0, if |β(k)| ≤ D are introduced, where ρ and ξ are weighting constants. Then the constrained optimization problem can be transformed into the following unconstrained optimization problem: N N −1 N 2 e (k) + g(k) + h(k) (15) min k=1
i=1
i=1
This cost function can again be solved by the quasi-Newton method, yielding estimates of the system parameters, and of α(k) and β(k). As initial values, the conventional least-squares estimates obtained by correcting input and output measurements using Eqs.(9) are used, initial values of α(k) and β(k) being set to zero. 2.3.3 A numerical example The same true system as in Section 2.2.3 was identified, with dead-zones in the sensors (D = 0.5). The input signal was a uniform random signal between −2 and 2. The input and output signals are shown in Fig.7. ρ = ξ = 10 were used in the penalty functions. The termination condition for the optimization algorithm was the same as in Section 2.2.3.
188
3 Output
Output
3
0
−3 500
550 Number of data
−3 500
600
Input
Input
550 Number of data
600
550 Number of data
600
3
3
0
−3 500
0
550 Number of data
output L - L- D - D 0
D
−3 500
600
Figure 9: Estimated input and output signals for dead-zone data (solid line: estimated, dash-dotted line: measured, dotted line: true)
L +D input
0
Figure 11: Input and output signals measured by sensor with dead-zone and saturation nonlinearity (solid line: measured, dotted line : true)
Eqs.(9) and (10) are first applied to give an initial correction for the dead-zone. Then the estimates of the true input and output signals, u ˆ(k) and yˆ(k), are written in terms of the corrected input and output signals u ¯ (k) and y¯ (k):
-L
Figure 10: Dead-zone and saturation nonlinearity
The identified frequency response is shown in Fig.8. The solid line shows the results from the proposed method, the dotted line shows the initial estimate, calculated by the least-squares method based on the corrected input and output signals (using Eqs.(9) and (10)), the dashed line is the conventional least-squares method based on the measured input and output signals, and the dashdotted line is the true response. It is clear from the figure that the proposed method almost agrees with the true value. The estimated input and output signals are shown in Fig.9. It is clear that the proposed method can reconstruct the original input and output signals accurately. The mean value of PERR for the proposed method, over 10 trials, was 0.0003, whereas PERR for the conventional method was 0.0598. Again, the proposed method estimates the system parameters very accurately. 2.4 Sensors with dead-zone and saturation 2.4.1 Problem formulation We now consider sensors with combined deadzone and saturation characteristics, as shown in Fig.10. We assume that the values of L and ±D are known.
u ˆ(k) =
yˆ(k) =
α(k), u ¯ (k), (k) + γ(k), u ¯ u ¯ (k) − γ(k), β(k), y¯ (k), y ¯ (k) + δ(k), y¯ (k) − δ(k),
if if if if
|¯ u (k)| ≤ D D < |¯ u (k)| < L + D |¯ u (k)| ≥ D + L |¯ u (k)| ≤ −D − L
if if if if
|¯ y (k)| ≤ D D < |¯ y (k)| < L + D |¯ y (k)| ≥ D + L |¯ y (k)| ≤ −D − L
where α(k), β(k), γ(k), and δ(k) satisfy the constraints (13) and (2). 2.4.2 Solution We identify the system by solving the following constrained optimization problem: min
N
e2 (k)
s.t.
|α(k)| ≤ D, |β(k)| ≤ D,
k=1
γ(k) ≥ 0, δ(k) ≥ 0 In order to solve this problem, four penalty functions are introduced: ρd [|α(k)| − D]2 , if |α(k)| > D gd (k) = 0, if |α(k)| ≤ D 2 ξd [|β(k)| − D] , if |β(k)| > D hd (k) = 0, if |β(k)| ≤ D 2 ρs δ (k), if δ(k) < 0 gs (k) = 0, if δ(k) ≥ 0 2 ξs γ (k), if γ(k) < 0 hs (k) = 0, if γ(k) ≥ 0
189
3
0
Output
Gain [dB]
20
−20 −40 10
0
1
10 Frequency [rad/s]
10
−3 500
2
−100
Input
Phase [deg]
550 Number of data
600
550 Number of data
600
3
0
−200 −300 10
0
0
1
10 Frequency [rad/s]
10
0
−3 500
2
Figure 12: Identified frequence response functions for dead-zone and saturation signals (solid line: proposed method, dash-dotted line: least-squares method, dotted line: true)
Figure 13: Estimated input and output signals for dead-zone and saturation signals (solid line: estimated, dash-dotted line: measured, dotted line: true)
This leads to the following unconstrained optimization problem: N N −1 N min e2 (k) + gd (k) + hd (k)
istics, as well as system parameters, has been proposed. The effectiveness of the proposed method was examined through numerical experiments. This method is relevant when the nonlinear characteristics are non-invertible (such as dead-zones and saturations). If an invertible characteristic is known exactly, then it can be corrected exactly, after which standard identification methods can be applied. We have used a quadratic penalty function and a quadratic cost function. Future work will investigate the effects of using other kinds of penalty function. In this paper, the nonlinearity of the sensor was assumed to be known. Simultaneous estimation of the system dynamics and of nonlinear sensor characteristics is one of the goals of further research in this direction. Success of this would also make our approach relevant when imperfectly-known invertible nonlinearities are present.
i=1
k=1
+
N −1 i=1
gs (k) +
i=1 N
hs (k)
(16)
i=1
As initial values for parameters, the conventional least-squares estimates based on the corrected input and output signals (using Eqs.(9)) are used, and initial values of α(k), β(k), γ(k) and δ(k) are set to zero. 2.4.3 A numerical example The same true system as in Section 2.2.3 is considered. The nonlinear sensors with dead-zone and saturation were used with D = 0.5, L = 1. The input and output signals are shown in Fig.7. ρd = ξd = ρs = ξs = 10 were used in the penalty function. The identified frequency response is shown in Fig.12. It is clear from the figure that the proposed method almost agrees with the true value. The estimated input and output signals are shown in Fig.13. It is again clear that the proposed method can reconstruct the original input and output signals accurately. The mean value of PERR for the proposed method over 10 trials was 0.0003, whereas for the conventional method it was 0.0635.
REFERENCES Gill,P.E., W.Murray and M.H.Wright (1981). Practical Optimization. Academic Press, London Ljung, L. (1999). System Identification – Theory for the User (2nd Edition). Englewood Cliffs, NJ: Prentice Hall PTR. Nells,O. (2001). Nonlinear System Identification. Springer Okao,A., M.Ikeda, and R.Takahashi, (2002). System identification via data of finite wordlengths, Proc. of SICE 2002, pp.1734–1737.
3. CONCLUSIONS In this paper, a new identification method which estimates errors due to nonlinear sensor character-
190
