Proceedings of the 45th IEEE Conference on Decision & Control Manchester Grand Hyatt Hotel San Diego, CA, USA, December 13-15, 2006
ThIP8.9
Stochastic Optimal Control of a Servo Motor with a Lifetime Constraint Alexander Bogdanov∗ , Stephen Chiu∗ , Levent U. G¨okdere∗ and John Vian† ∗ Rockwell
Scientific Company, Thousand Oaks, CA 91360 Email:
[email protected],
[email protected],
[email protected] † Boeing Company, Seattle, WA 98124, Email:
[email protected]
Abstract— We consider a linear quadratic optimal regulation problem of a servo motor in the presence of stochastic load disturbance subject to a constraint that establishes a desired motor winding lifetime. To satisfy the constraint, a family of LQR control designs is parameterized with a single scalar performance weight that establishes a trade-off between performance and control power. Power analysis approach is then used to find the optimal value of the parameter that provides maximum disturbance rejection control in the given LQR family subject to the desired motor lifetime constraint.
I. NOMENCLATURE θ ω Tw , Ta α, β L W fm I Kt Cw Rw Rm Φv ω0 2 σu P Q, R µOL µCL ρ
motor output angular position motor output velocity winding and ambient air temperatures lifetime constants motor winding lifetime motor winding wear motor viscous friction coefficient moment of inertia at the motor output motor torque coefficient thermal capacitance from winding to air thermal resistance from winding to air motor phase-to-phase winding resistance power spectral density of the white noise inverse of the output disturbance correlation time control variance solution of the algebraic Riccati equation LQR design parameters open-loop system eigenvalues closed-loop system eigenvalues optimized parameter in the LQR design
III. MOTOR MODEL
II. INTRODUCTION Existing servo motor controllers are typically designed to maximize motor performance while providing sufficient stability margins. However, extracting high performance from an actuator results in decreased operational lifetime due to higher temperature and stress on the actuator components. This hidden cost of high performance control must be considered in controller designs when long-term maintenance-free autonomy is desired. Our work is focused on long-term lifetime prediction and control that ensures the desired lifetime while preserving stability. We assume that a new machinery component is initially damage-free and estimate its remaining life by modeling the accumulation of damage during its operation. In our past work [1], we utilized an adaptive control approach, in which the control performance was adjusted to maintain estimated residual lifetime of the machinery
1-4244-0171-2/06/$20.00 ©2006 IEEE.
component. We demonstrated in simulation tracking of the desired lifetime while maintaining stability by adaptation within a class of stabilizing controllers. In this paper, we consider a slightly different problem, in which we also parameterize a class of stabilizing controllers with a single parameter that affects the control performance and machinery lifetime. Here we take into account disturbance characteristics and optimize the parameter to obtain the best performing controller in the class that satisfies the desired lifetime constraint. Specifically, we focus on a regulation problem of a servo motor under stochastic external load subject to the constraint on its winding lifetime. While the mechanical part of the motor is modeled as a linear, second order dynamical system, the winding thermal model is quadratic in control, and the rate of wear is exponential with respect to the winding temperature. External load disturbance at the motor output is modeled as a first-order Markov process, a color noise resulting from low-pass filtering of Gaussian white noise with known power spectral density.
A typical brushless DC motor (Fig. 1) is described as a linear dynamic model with a state vector x = (θ, ω)T , control input u (motor current), and external load disturbance d x˙ 0 x Bu Ax B d u+ v + = 0 −ω0 ω0 0 d d˙ A
B
N
(1) where the disturbance is modeled as a first order Markov process driven by a Gaussian white noise v with mean E(v) = v¯ and power spectral density Φv . Explicitly, 0 1 Ax = 0 −I −1 fm 0 Bu = I −1 Kt 0 Bd = I −1 During the motor operation, the thermal behavior of the motor winding is modeled by a scalar nonlinear differential
4182
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThIP8.9
IV. CONTROL DESIGN
Fig. 1.
Typical brushless DC motor
equation that describes the evolution of the difference between winding temperature and ambient temperature ∆T = Tw − Ta as a function of the motor control input: ∆T˙ = At ∆T + Bt u2 −1 −1 −1 where At = −Cw Rw , Bt = Cw Rm . One of the primary failure modes of brushless DC motors is winding insulation failure. The life expectancy of motor winding insulation is a function of the winding temperature. The average life expectancy of winding insulation is a logarithmic function of the temperature: it is reduced approximately by half for every 10o C rise in temperature [2]. Lifetime of the new winding at constant temperature Tw = T (see details in references [2], [3]) is estimated by:
Control of winding temperature is one of the primary ways to prolong motor lifetime. In autonomous applications, desired operational lifetime can be a crucial requirement. Thus we consider a problem of a stabilizing control design subject to the desired lifetime constraint. More formally, given desired lifetime tf , and stochastic model and parameters of the disturbance E(v) = v¯, σ 2 (v) = σv2 , design a stabilizing controller that minimizes quadratic cost function ∞ T x Qx + Ru2 dτ J= t0
subject to the motor dynamics constraint (1) and the desired expected lifetime constraint E(tf ) = t¯f . Without the lifetime constraint, the linear quadratic regulator (LQR) solves the problem. To account for the lifetime constraint, we approximate the problem by parameterizing a family of LQR controllers with a single parameter ρ, and then optimize the parameter to satisfy the lifetime constraint. Explicitly, the cost function becomes ∞ T x Qx + ρRu2 dτ (3) J= t0
and the solution is −1
u = − (ρR) BTu P(ρ) x + Kt−1 d¯
Lnew (T ) = αe−βT
K
Assuming initially new winding and that damages incurred at different temperatures are linearly additive, residual fractional lifetime notion can be introduced as a remaining fraction of the new winding lifetime: t t dτ = 1−α−1 eβTw (τ ) dτ %Lres (t) = 1− L (T (τ )) new w t0 t0 An estimate of the absolute residual lifetime at time t and temperature Tw can be calculated under the assumption that winding temperature will stay constant in future (i.e. Tw (t1 ) = Tw (t), ∀t1 > t): t −1 βTw (τ ) e dτ Lres (Tw , t) = Lnew (Tw (t)) 1 − α t0
(2) Notice that rate of winding wear can be specified as another characteristic of the winding lifetime: ˙ (t) = W
(4)
1 = α−1 eβTw (t) Lnew (Tw (t))
and winding wear, or accumulated damage, is a fraction of the lifetime that is lost: t eβTw (τ ) dτ W (t) = 1 − %Lres (t) = α−1 t0
Note that given life expectancy tf , we have: W (t0 ) = 0, W (tf ) = 1, Lres (T, tf ) = 0.
where P(ρ) is the solution of the algebraic Riccati equation: PAx + ATx P − ρ−1 PBu R−1 BTu P + Q = 0
(5)
Thus we are looking for an optimal controller in the LQR class, given the fixed state cost weighting matrix Q. By introduction of ρ, we regulate the extent of how “expensive” control is. Longer desired lifetime dictates smaller control power (larger ρ) to keep winding temperatures lower. “Cheap” control (small ρ) provides higher performance at the expense of shorter motor lifetime. Thus higher performance and longer motor lifetime can be viewed as conflicting requirements, and the lifetime requirement sets a constraint on achievable performance. In the following we will find the minimum possible ρ that provides the desired lifetime. Given the LQR control design parameters Q, R, and ρ, the control is u(s) = −(k1 + k2 s)θ, and we can compute the control variance due to the stochastic disturbance. To do it, consider the transfer function v− > u: Wvu (s) =
ω0 k1 + k2 s Is2 + (fm + Kt k2 )s + Kt k1 s + ω0
(6)
Then variance of the control input due to the disturbance is: Φv ∞ u σu2 = ||Wvu (s)||22 Φv = W (jω)Wvu (−jω)dω 2π −∞ v (7)
4183
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThIP8.9
The integral in (7) can be computed via the residue theorem [4] due to the fact that (6) is strictly proper and has no poles on the imaginary axis [5]. Another way to compute the control variance is to use a state-space formulation [5]: given the closed-loop system z = (x, d)T driven by the white noise v z˙
=
x u
= Cz = −Kx
[7]. Notice that combined state and adjoint equations in the LQR problem are given by x˙ x Ax −ρ−1 Bu R−1 BTu = λ −Q −ATx λ˙ or, denoting χT = (xT , λT ),
(A − BKC)z + Nv
χ˙ = Ac χ The 2n = 4 eigenvalues of this system represent n = 2 optimal eigenvalues µCL of the LQR closed-loop system and n = 2 eigenvalues of the adjoint system. These eigenvalues are the roots of the equation
solve the Lyapunov equation for L: (A − BKC)L + L(A − BKC)T + NNT = 0
det(sI2n − Ac ) = 0
and then compute σu2
=
||Wvu (s)||22 Φv
= KCLC K Φv T
T
where K is the LQR gain matrix, u = −Kx. Given the control variance at fixed value of the LQR parameter ρ, the expected control power is 1 t 2 E(u2 ) = E lim u (τ )dτ = Kt−2 v¯2 + σu2 (ρ) t−>∞ 2t −t Now we can find the expected winding lifetime using first order Taylor approximation:
(8) E(L) = E αe−βTw (t) ≈ αe−βE(Tw ) where
(9)
which can also be expressed as a product of polynomials [7]: det(sI2n − Ac ) = ∆CL (s)∆CL (−s). The open-loop characteristic polynomial ∆OL (s) is defined by det(sIn − Ax ) = 0 It can be shown after some manipulations with matrices (see for example [7]) that for a single input system det(sI2n −Ac ) = (−1)n ∆OL (s)∆OL (−s) × 1+ρ−1 YT (−s)Y(s) (10) √ where Y(s) = RH(sIn − Ax )−1 Bu , Q = HT H. As ρ approaches infinity (the case of very expensive control), closed-loop poles asymptotically approach open-loop poles:
−1 E(Tw ) = Rm Rw E(u2 ) + Ta
lim det(sI2n − Ac ) = (−1)n ∆OL (s)∆OL (−s)
ρ→∞
A better approximation of the expected lifetime could be achieved by using the sigma-point approach, which is based on deterministic sampling for propagation of Gaussian random variables through nonlinear systems [6]. Given the desired expected lifetime, Ldes , the allowable control variance is then −1 σu2 des = −β −1 ln α−1 Ldes − Ta Rm Rw − Kt−2 v¯2 In the control design, we seek such a ρ that will result in a control variance corresponding to the desired lifetime. Proposition 1: Given a stabilizable system (1) that has no open-loop poles with positive real parts and is driven by the control law (4), and given a nonzero σu2 des and a finite Φv , there exists a unique finite ρ such that σu2 (ρ) = σu2 des . Proof: The proof is based on analysis of limiting values of closed-loop eigenvalues and eigenvectors as ρ goes to infinity. We can show that in the limit closed-loop eigenvalues and eigenvectors approach open-loop ones if the open-loop system does not have eigenvalues in the right half-plane. From that we can then derive a conclusion that the LQR gain matrix K becomes vanishingly small, that is limρ→∞ K = 0, which in turn provides limρ→∞ u = 0 when disturbance is bounded. It has been shown that the closed-loop eigenvalues and eigenvectors corresponding to limiting values of the control weight R can be found without explicitly solving for K
Since the LQR regulated closed-loop system is asymptotically stable, Re(µCL i ) < 0. This means that in the limit, when control is infinitely expensive, the set of closed-loop system eigenvalues must contain all open-loop eigenvalues OL with Re(µOL µOL i i ) < 0 and mirror images of all µj OL with Re(µj ) > 0. In our case, the motor model has one negative real eigenvalue Re(µOL 1 ) < 0 and one in the origin, = 0. As ρ approaches infinity, limρ→∞ µCL = 0. This µOL 2 2 means that infinitely expensive control in our case does not provide closed-loop stability, and the stabilizing solution of the Riccati equation does not exist for the infinitely large ρ. However since (10) has a limit at infinity and is continuous with respect to ρ, then for every small > 0 there exists δ such that for any finite ρ > δ, we have − < Re(µCL 2 ) < 0. Therefore, the eigenvalues of the closed-loop system can be made arbitrarily close to those of the open-loop system by sufficiently large ρ. Next, eigenvectors for the state and adjoint variables, xi and λi are solutions of the equation (µi I − Ax ) Bu R−1 BTu xi =0 (11) λi Qρ−1 (µi I + ATx ) which can be used to compute xi as ρ goes to infinity, considering only those roots of ∆OL (s)∆OL (−s) that are in the left half-plane. If µi is a stable root of the openloop system, then |µi I − Ax | = 0 and xi is an eigenvector
4184
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThIP8.9
of the open-loop system, and the mode is unaffected by control. Thus, modal control ui associated with xi must be zero: ui = −R−1 BTu Pxi = −Kxi = 0. If −µi is an unstable open-loop eigenvalue, then µi is a root of the adjoint system |µi I + ATx | = 0, and the closed-loop state eigenvector is xi = −(µi I − Ax )−1 Bu R−1 BTu λi . In our case, as ρ goes to infinity, one of the closed-loop eigenvalues approaches the origin in the left half-plane, and the corresponding closed-loop eigenvector approaches in the limit its open-loop counterpart. Thus in the limit, (µi I − Ax − Bu K)xi = (µi I − Ax )xi and limρ→∞ K = 0, limρ→∞ λi = limρ→∞ Pxi = 0, and all modal controls limρ→∞ ui = 0. Presenting an arbitrary state vector as a linear combination of eigenvectors, a corresponding control input is presented as a linear combination of modal controls: n u = i ki ui . From the discussion above, since (11) has a limit at infinity and is continuous with respect to ρ, then for every small > 0 there exists δ such that for any finite ρ > δ, we have 0 < max |ui | < and 0 < |u| < n max |ki |. Since the transfer function from the disturbance to the system state vector Wvx (s) is BIBO stable and well-defined for any K = (k1 k2 ) (consider (6) and note that Wvu (s) = KWvx (s)), bounded external disturbance causes bounded system reaction. This implies that all coefficients in u = n k u i i i are limited: max |ki | < M < ∞, and thus 0 < |u| < nM , which implies that there exists ρ such that σu2 (ρ) = σu2 des . As the last note, the root locus of (10) does not intersect itself, and ρ uniquely defines roots of the optimally regulated closed-loop system. Since the control input is scalar in our case, there is a one-to-one correspondence between eigenvalues of the closed-loop system and the gain K. Thus the gain is also uniquely defined by ρ. Remark 1 The proposition above is only correct in the case when the open-loop system has no eigenvalues in the right half plane. Since the poles of the optimally regulated closed-loop system have negative real parts, ||K||2 can not be made arbitrarily close to zero when the open-loop system is unstable. In that case, when ρ approaches infinity, the closed-loop system poles approach stable poles of the openloop system and the mirror images of the unstable poles. The gain matrix will approach its minimal value, which is not zero. This means that in the case of an unstable open-loop system the control variance σu2 has a lower bound defined by the disturbance variance and the minimum achievable gain Kmin = K|ρ=∞ . Remark 2 Note that for bounded ρ the closed-loop system always remains stable, although longer lifetime comes at a price of poorer performance. Since longer lifetime demands smaller ||K||2 , which causes the closedloop system poles to move towards the open-loop system poles, the extent of performance degradation is determined by how close the open-loop eigenvalues are to the imaginary axis. The last step in the control design is to numerically optimize ρ to provide the desired control variance.
1) Set ρ = 1 and solve the LQR problem to compute the gain matrix K. 2) Compute σu2 (ρ). If σu2 (ρ) < σu2 des , stop (nominal control design provides winding lifetime higher than desired). Otherwise do next step. 3) Numerically solve for ρ: σu2 (ρ) = σu2 des . In our simulations we used a simple bisections algorithm to find the solution. Note that depending on the desired lifetime and intensity of the disturbance, the solution may lead to poor performance when desired control variance is low (σu2 des ≈ 0), or the solution to the problem does not exist if σu2 des ≤ 0. In the latter case one has to soften requirements to the winding lifetime. The optimal regulation problem discussed in the paper can be extended to a more general case of tracking desired state trajectory xdes given that its statistics are known (e.g. typical operational profiles). For the tracking control problem, the optimized functional (3) turns into ∞ T e Qe + ρRu2 dτ J= t0
¯ where e = x − xdes and the solution is u = −Ke + Kt−1 d. In such a case, the described methodology readily applies due to linearity of the servo motor model. For example, assuming no correlation between commanded input and the external load disturbance and E(xdes ) = 0, σu2 = ||Wvu (s)||22 Φv + ||Wxudes (s)||22 Φxdes where Φxdes is a power spectral density of the commanded input (typical operational profile). The described optimization procedure then applies with only minor changes. V. SIMULATION RESULTS In this section, we present simulation results that demonstrate disturbance rejection given two different constraints on the motor winding lifetime. The motors are driving a linear electro-mechanical actuator under external load disturbance. The disturbance is modeled as a Markov process with the inverse of the correlation time ω0 = 10 driven by a white noise with mean value v¯ = 0.41 Nm, (at the motor v )2 . output), and power spectral density Φv = (0.25¯ In the first simulation, Fig. 2, the desired winding lifetime was set at 2.79·105 hours, which approximately corresponds to a mean winding temperature of 117o C. The optimal value of ρ was found to be ρ = 256, which resulted in the mean temperature of the winding 116.7o C over 300 seconds of operation time. Actual expected lifetime under this scenario is 3.4·105 hours, and the deviation from the desired lifetime is attributed to the linear approximation in (8). An estimate of the actual residual lifetime was obtained by using the instantaneous temperature according to (2). In the second simulation, Fig. 3, the desired winding lifetime was set at 3.94 · 105 hours, which approximately corresponds to mean winding temperature of 112o C. The optimal value of ρ was found to be ρ = 47524, which
4185
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
5
Winding lifetime
15
ThIP8.9
Residual Winding Lifetime
x 10
180
Desired Actual mean Actual estimated
10
Φ =(0.1v
)2
Φ =(0.2v
)
v
170
mean
v
2
mean
2
Φv=(0.25vmean)
160
5
0
0
50
100
150 Time, s
200
250
Temperature, C
2
300
Dynamic error Dynamic error
100 Speed error Position error
50 0
Φ =(0.5v v
)2
mean
130
110 0
50
100
150 200 Time, s. Winding temperature
160 Temperature, deg.
2
Φv=(0.4vmean) 140
120
−50 −100
Φv=(0.3vmean)
150
250
300
100 0
Actual Mean Desired
140
4
ρ
6
8
10 6
x 10
Winding temperature as a function of ρ at different disturbance
Fig. 4. levels
120
2
100 5
80
0
50
100
150 Time, s.
200
250
12
300
x 10
2
Φv=(0.1vmean)
10
Fig. 2. Estimated residual winding lifetime, disturbance rejection, and winding temperature at Ldes = 2.79 · 105 hours
2
Φv=(0.2vmean) Φ =(0.25v v
Winding lifetime
Lifetime, hr
6
2
Residual Winding Lifetime
x 10
Desired Actual mean Actual estimated
1.5 1
6
4
2
0
50
100
150 Time, s. Dynamic error
200
250
300 0
100
0
2
4
ρ
6
8
10 6
x 10
Fig. 5. Winding lifetime as a function of ρ at different disturbance levels
−50 −100
0
Speed error Position error
50 Error
Φ =(0.3v
0.5 0
0
50
140
100
150 200 Time, s Winding temperature
250
300 260
Actual Mean Desired
120
240 220 200
100 v 2
180
80
0
50
100
150 Time, s.
200
250
||Wu||2
Temperature
)2
mean )2 v mean Φ =(0.4v )2 v mean 2 Φ =(0.5v ) v mean
8
300
160 140 120
Fig. 3. Estimated residual winding lifetime, disturbance rejection, and winding temperature at Ldes = 3.94 · 105 hours
100 80 60
0
2
o
resulted in the mean temperature of the winding 111.7 C over 300 seconds of operation time. Notice the performance degradation (dynamic error) compared with Fig. 2. Fig. 4 and Fig. 5 show dependency of mean winding temperature and expected winding lifetime on the parameter ρ at different levels of disturbance intensity. The greater the disturbance intensity, the higher impact ρ has on the winding temperature and life expectancy. Dependence of ||Wvu (s)||22 on ρ is plotted in Fig. 6.
Fig. 6.
4
ρ
6
8
10 6
x 10
||Wvu (s)||22 as a function of ρ
VI. CONCLUSIONS In this paper we presented a design methodology for a motor controller with a constraint on motor winding lifetime. Controller parameterization in the class of LQR designs results in the best possible LQR design given a fixed
4186
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThIP8.9
state error weighting matrix Q. Stability is guaranteed by the LQR design, yet performance depends on the desired lifetime and disturbance intensity. The problem of lifetime control can also be cast in a different way: instead of imposing a hard constraint on the desired lifetime as considered in this paper, a tradeoff between the desired performance level and the lifetime can be used as a criterion of optimization, which remains in the scope of our work. VII. ACKNOWLEDGEMENTS This work was supported by Boeing internal R&D fund. R EFERENCES [1] L. Gokdere, A. Bogdanov, S. Chiu, K. Keller, and J. Vian, “Adaptive control of actuator lifetime,” in Proceedings of the IEEE Aerospace Conference, Big Sky, MO, March 2006. [2] D. Kaiser, “Advances in digital servodrive motor protection,” Motion system design, pp. 21–25, June 2003. [3] L. Gokdere, S. Chiu, K. Keller, and J. Vian, “Lifetime control of electromechanical actuators,” in Proceedings of the IEEE Aerospace Conference, Big Sky, MO, March 2005. [4] P. Franklin, Functions of complex variables. Englewood Cliffs, NJ: Prentice-Hall, 1958. [5] J. C. Doyle, B. A. Francis, and A. R. Tannenbaum, Feedback control theory. Macmillan Publishing Co., 1992. [6] S. Julier, J. Uhlmann, and H. Durrant-Whyte, “A new approach for filtering nonlinear systems,” in Proceedings of the American Control Conference, Seattle, WA, June 1995. [7] R. F. Stengel, Optimal control and estimation. New York, NY: Dover Publications, 1994.
4187
