Journal of Process Control 22 (2012) 2008–2025
Contents lists available at SciVerse ScienceDirect
Journal of Process Control journal homepage: www.elsevier.com/locate/jprocont
Automatic controller tuning via unfalsified control夽 Tanet Wonghong a,∗ , Sebastian Engell b,1 a b
Department of Electrical Engineering, School of Engineering, Bangkok University (Rangsit Campus), 12120 Pathumthani, Thailand Process Dynamics and Operations Group, Department of Biochemical and Chemical Engineering, TU Dortmund, 44221 Dortmund, Germany
a r t i c l e
i n f o
Article history: Received 1 March 2012 Received in revised form 24 June 2012 Accepted 6 August 2012 Available online 3 September 2012 Keywords: Adaptive control Unfalsified control Evolutionary algorithm Automatic controller tuning Continuous stirred tank reactor Nonlinear control Reactor control
a b s t r a c t In this paper, we present a new algorithm for the automatic adaptation of linear single-loop controllers. The algorithm is based on the idea of unfalsified control, where a set of controllers is defined and using measured data, the controller from the set is chosen online which provides the best performance. This scheme is extended here by an adaptation of the set of controllers after a change of the operating point of the plant. The original concept of unfalsified control relies on the computation of the performance of controllers that are not in the loop using so-called fictitious signals. However, the proposed cost function leads to erroneous results, in particular, it is not possible to detect closed-loop instability that may result from inserting a controller. Therefore a different cost function and a new method for online evaluation of controllers that are not in the loop without an explicit plant model are proposed. With this new cost function, the set of controllers can be adapted as well which is performed using an evolutionary algorithm. The method is demonstrated for a well-known example, the nonlinear non-minimum phase CSTR model with van der Vusse reaction scheme and PID controllers. Crown Copyright © 2012 Published by Elsevier Ltd. All rights reserved.
1. Introduction Adaptive control is an attractive, although up to now only very rarely implemented concept for the control of complex processes, e.g. bio-chemical transformations where it is difficult and timeconsuming to derive a reliable first principles based plant model and/or the plant dynamics change considerably over time. Unfalsified control is a data-driven direct adaptive control strategy which was introduced by Safonov and Tsao [1]. The main attractive feature of this approach is that it does not need an explicit model of a plant but the controller switches between a finite number of controllers in a set of possible controllers based solely upon measured plant input and output data. This data is used to determine the performance of the candidate controllers in the set without inserting them in the loop. The original unfalsified control algorithm in [1] consists of a set of candidate controllers, a falsification procedure, and a switching mechanism. The switching mechanism selects an active controller from the set of controllers based upon so-called fictitious reference and error signals. These signals are computed from the measured signals and the transfer functions of the currently active controller and the other controllers in the set. There
夽 This paper was not presented at any IFAC meeting. ∗ Corresponding author. Tel.: +66 2 902 0299; fax: +66 2 516 8553. E-mail addresses:
[email protected],
[email protected] (T. Wonghong),
[email protected] (S. Engell). 1 Tel.: +49 231 7555135; fax: +49 231 7555129.
are only few other direct data-driven adaptive control approaches. E.g. a direct adaptive fuzzy controller design for nonlinear systems is proposed by Labiod and Guerra [2]. No knowledge of a mathematical model of the processes is required and fuzzy systems are used to construct an unknown ideal controller and its adjustable parameters are updated using the gradient descent method to minimize the error between the unknown controller and the fuzzy controller. In contrast, gain-scheduling control (see, e.g. Klatt and Engell [3]) constitutes a pre-computed model-based controller adaptation law which is fixed once and for all [4] so it cannot react to changes in the plant dynamics and plant-model mismatch and requires a reasonably precise plant model for its development. An early version of online PID controller tuning via unfalsified control was demonstrated by Jun and Safonov [5]. The results showed that the adaptation of PID parameters worked for an unstable linear plant model without measurement errors. Later work by Engell et al. [6], Mauelli et al. [7] and Dehghani et al. [8] showed that the original scheme is flawed in the sense that it cannot detect the instability of controllers that are not inserted in the loop, hence it may switch to destabilizing controllers before it is detected that the performance is bad. The starting point of our work was the idea that if the performance of a controller which is not inserted in the loop can be detected from signals that are measured while another controller is in the loop, the approach could also be used to adapt the set of controllers and to compute an optimal controller based on the recorded data instead of only switching between a finite number of predefined controllers. This is attractive especially if a nonlinear plant
0959-1524/$ – see front matter. Crown Copyright © 2012 Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jprocont.2012.08.002
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
is controlled at different operating points with different dynamics. For such an optimization, it is crucial that the performance of a candidate controller is evaluated correctly and in particular that it is detected whether it destabilizes the loop. Otherwise clearly no optimization of the controller parameters is possible. Therefore in [6] a new cost function for controller performance monitoring based on the measured signals has been introduced. This idea is realized in combination with the -hysteresis switching algorithm [9,10] which restricts switching between controllers in the current set to the case where the performance of the new controller is better than that of the active controller by some threshold. The set of possible controllers is optimized using an evolutionary algorithm plus a diversity operator to provide back-up controllers that can cope with slightly changing plant dynamics. The evolutionary algorithm is called when a new operating point is commanded. Overall, the new adaptive control algorithm consists of four elements as follows: • Cost monitoring The cost monitoring algorithm computes the performance of all candidate controllers in the current set using the observed control error and the known reference signal and sends it to the switching mechanism. • -Hysteresis switching algorithm The switching algorithm controls the switching between the controllers based upon the performance evaluation by the cost monitoring algorithm. If there is a controller that performs better than the active one, then the controller is switched. • Evolutionary algorithm (EA) After a significant change of the set-point and collection of data over some period of time, an evolutionary algorithm is called which computes a new set of controllers (the population of the EA) based on the performance function used in the cost monitoring. The best controller is assigned as the initial active controller for the current operating point. • Diversity operator In the search process, the diversity in the population of controllers which the EA is investigating usually decreases. Therefore after convergence of the optimization, the optimal controller is modified deterministically to generate a set of controllers with sufficient diversity to react to changes of the plant dynamics or errors in the performance evaluation based on the signals that were recorded when the EA was started. In this paper we focus on the direct online tuning of PID controllers because of their widespread use. In contrast to model adaptation where a fixed model structure has to be assumed – if the model structure is simple enough, PID controllers can be computed directly from such models for a desired closed-loop behavior, which has, however, to be fixed by the designer aforehand – the approach is not limited in terms of the assumed plant dynamics. The only practical prerequisite of the application of our method is that one coarsely tuned PID controller is known for the plant under consideration that can be ‘diversified’ into a set of controllers by varying the parameters around the nominal value such that a range of operation conditions can be handled before the adaptation mechanism improves the controller set to provide good control performance. Theoretically, the approach can deal with unstable plants as long as at least one stabilizing controller is in the set. If the optimization of the controllers is performed fast enough, even stability of the closed-loop with at least one of the original controllers is not necessary, however realistically we do not propose to use a purely data based adaptive control law for an unstable plant. While [6–8] dealt with the detection of instability of the fictitious loop with a controller that is not currently active at the real
2009
Fig. 1. Adaptive control system ˚(P, C(t) (s)).
plant in the context of unfalsified control, this paper presents a complete adaptive control scheme based upon the idea originally proposed in [6] with a proof of stability and an application to a wellknown non-minimum phase reactor control benchmark. We also propose a scheme how to handle measurement noise and demonstrate its efficiency. This paper is structured as follows: first the idea of unfalsified control is explained and the elements of the new unfalsified adaptive control algorithm are presented in detail in Section 2. In Section 3, the stability of the loop is proven for a linear time-invariant plant if there is at least one non-destabilizing controller in the set of candidate controllers. The potential of the new algorithm is demonstrated by its application to the example of a nonlinear nonminimum phase CSTR model which is controlled by PID controllers in Section 4. In Section 5, the effect of measurement noise and the modification of the adaptation scheme to cope with stochastic measurement noise are discussed and demonstrated for the same example. The paper is based on the conference papers [6,11–13] and the doctoral dissertation [14].
2. The new unfalsified adaptive control algorithm 2.1. The adaptive control system We consider a SISO system in the continuous-time domain as shown in Fig. 1. A mapping ˚(P, C(t) (s)) : L2e → L22e that transforms r(t) → [u(t), y(t)]T is called an adaptive control system. We denote r(t) ∈ L2e as the reference signal or the external excitation.
2.2. The unknown plant P A black-box mapping P : L2e → L2e that transforms u(t) → y(t) is called an unknown plant P where u(t) is the observed plant input signal and y(t) is the observed plant output signal. We define a set of unknown linearized plant models of the unknown nonlinear plant P as
{P |Pj (s) =
Aj bj cj dj
} pj =(xsj ,usj )
(1)
where Pj (s) denotes the (unknown) linearized plant at the current operating point pj = (xsj , usj ). pj−1 and pj+1 are defined to be the previous operating point and the next operating point for pj . We define p0 (0, 0) as the operating point of P at the origin, i.e. xs0 = 0, us0 = 0 → ys0 = 0 is equivalent to r(t) = 0 → [u(t), y(t)]T = 0 and therefore the unknown initial state vector xp00 of P at the origin is 0. It is clear that at p0 , no adaptation is required because there is no external excitation. We assume that all unknown linearized plants are stable and that the adaptive control system ˚(Pj (s), C(t) (s)) has settled at the current operating point pj (y(t) ∼ = ysj , u(t) ∼ = usj ; t > tsj ) before the next operating point pj+1 is selected. tsj denotes the closed-loop settling time at pj .
2010
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
Fig. 2. Example of a series of operating points. Fig. 3. Basic feedback control scheme.
2.3. The size and the time of the excitation We define rj as the size of the excitation and tj as the time of the excitation for the current operating point pj . An example of a sequence of operating points is shown in Fig. 2 where the current set-point rj is commanded, the algorithm starts to perform an adaptation for the unknown plant with the unknown initial state vector xp0j at the current operating point pj . Our approach does not require the knowledge of Pj (s) and of xp0j but rj and tj must be known.
D =
The set of candidate controllers is defined as (2)
where the linear system Ci (s) ∈ K is called the ith candidate controller. We require that the controllers are proper and quotients of only two controllers in the set are stable.
(3)
(4)
where c(t) (t) is the inverse Laplace transform of C(t) (s) and ∗ is the convolution integral. A piecewise continuous mapping : R+ → M is called a switching signal which is generated by the unfalsified adaptive control algorithm to activate a candidate controller C(t+ ) (s) ∈ K. We define Ci(t) (s), ∀i(t) ∈ M − {(t)} as the non-active candidate controllers. 2.7. Initialization of a new active controller In this contribution, we assume a PID controller structure. Therefore we need to reset the state of the current active controller at the current switching time before implementing the next active controller in the loop. Otherwise, a high peak response due to the switching to the new active controller may result. The simulation results in [5] show that a large transient in the plant input and output signals occur if we do not handle this initialization properly.
The set of data is defined as d|d(t) = r(t)
C (s)Pj (s) 1 + C (s)Pj (s)
.
(7)
C (s)Pj (s) 1 + C (s)Pj (s)
(8)
Rj (s)
U (s) =
C (s) 1 + C (s)Pj (s)
Rj (s).
(9)
From (3), the error signal for C (s) can be computed as follows: Ej (s) = Rj (s) − Y (s) =
1 R (s). 1 + C (s)Pj (s) j
(10)
Note that the true error signal for C (s) can be computed using the observed plant output signal Y (s) because we know Rj (s). We now consider the case of a non-active candidate controller Ci (s) as shown in Fig. 3(b). Since Ci (s) is not in the feedback loop, a fictitious error signal of Ci (s) is computed as if it were in the feedback loop. Using U (s), we obtain U (s) = Ci (s)E˜ i (s) → E˜ i (s) = Ci−1 (s)U (s).
(11)
˜ i (s) of Ci (s) can be computed Moreover, a fictitious reference signal R using E˜ i (s) and Y (s) and (3): ˜ i (s) = E˜ i (s) + Y (s) = C −1 (s)U (s) + Y (s). R i
(12)
The fictitious reference signal and the fictitious error signal of Ci (s) can be defined in the time domain as
2.8. The set of data
D
(6)
and the plant input signal is
The adaptive control law is defined as
T
Thus the observed plant output signal is Y (s) =
2.6. The adaptive control law
y (t)
We consider the feedback control scheme for the (unknown) linearized system Pj (s) at the current operating point pj as shown in Fig. 3(a). Since Pj (s) is unknown, the available information only consist of the observed plant input signal u (t) u(t) and the observed plant output signal y (t) y(t) that have been measured while C (s) was in the feedback loop in tj ≤ t with the set-point rj (t). Rj (s) is the Laplace transform of the reference signal rj (t) that was applied to the feedback loop t ≥ tj . The closed-loop system is T (s) =
The true error signal is defined as
u(t) c(t) (t) ∗ e(t)
u (t)
where d (t) is called a truncated data signal vector or an observed data signal vector for a given time-window of length , = [ta , tb ] ∈ R+ .
2.5. The true error signal
e(t) r(t) − y(t).
d |d (t) = r (t)
2.9. The original fictitious signals
2.4. The set of candidate controllers
K {C|Ci (s), ∀i ∈ M = {1, 2, . . . , m}}
where d(t) ∈ L32e is called a data signal vector. The truncated data set is
u(t) y(t)
T
r˜i (t) ci−1 (t) ∗ u (t) + y (t), (5)
e˜ i (t) r˜i (t) − y (t) =
ci−1 (t) ∗ u (t).
(13) (14)
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
2011
2.9.1. The relationship of the fictitious reference signal and the true reference signal Inserting U (s) and Y (s) into (12), ˜ i (s) = R
C (s)S (s) Ci (s)Si (s)
˜ i (s)Rj (s) Rj (s) =
(15)
where S (s) = 1/(1 + C (s)Pj (s)) is the unknown sensitivity function of the closed-loop pair (C (s), Pj (s)) and Si (s) = 1/(1 + Ci (s)Pj (s)) is the unknown sensitivity function of the closed-loop pair (Ci (s), Pj (s)). ˜ i : L2e → L2e , rj (t) → r˜i (t) is called fictitious reference The mapping signal generator of Ci (s). ˜ i (s) for a PID controller structure 2.9.2. A PID controller structure, CiPID (s) = kpi (1 + (1/Tni s) + (Tdi s/ς s + 1)), is used with a small positive scalar ς . Using (12), we obtain ˜ i (s) = R
ς (Tni /kpi )s2 + (Tni /kpi )s (Tdi Tni + ς Tni )s2 + (Tni + ς )s + 1
U (s) + Y (s).
(16)
Realizing (16) by a state space representation, we have Fig. 4. Example of a fitness landscape using ˜Ji .
⎡
⎤
r ⎤ x1ji r x˙ 1ji rj ⎥ ⎢ ˜ P ID ⎢ x2i ⎥ ; Λ ˜ P ID = ⎣ x˙ r2j ⎦ = Λ i i ⎣ uφ ⎦ i r˜i (t) yφ
⎡
∞
The space of sampled, square-summable signals, i.e. x2 (k) < k=0 ∞, is called l2 -space. The norm of a sampled signal x(k) ∈ l2 is defined as
˜i 0 ˜i b A c˜i d˜i 1 (17)
with ˜i = A
(21)
k=0
⎡ ⎣
=
∞
x(k) l2 x2 (k).
0 1 − Tdi Tni + ς Tni −
and d˜ i =
⎤
1 −
Tni + ς
⎦ , b˜ i = 0 , c˜ i
Tdi Tni + ς Tni
ς Tni
Tdi Tn2i − ς 2 Tni
kpi (Tdi Tni + ς Tni )2
kpi (Tdi Tni + ς Tni )2
ς Tni
kp (Td Tn +ς Tn i
i
i
i
Wang et al. [10] proposed the cost function
1
˜Ji (t)
i
˜ PID (s) for the fictitious closed-loop pair (Ci (s), Pj (s)). Note that i is always stable since its two poles have negative real parts (Tni , Tdi , ς > 0). ˜ i (s) 2.9.3. Computation of the initial state vector of ˜ PID (s) at pj , we need to compute xrj . From the To implement 0 i i
assumption that the adaptive control system ˚(P, C(t) (s)) has settled at pj−1 , y(t) = ysj−1 and u(t) = usj−1 where t > tsj−1 . Using (17), we can compute (18)
˜ i is non-singular. Thus we define if A xr0j xpsij−1 , i
∀i ∈ M.
(19)
2.10. The original cost function ˜Ji (t)
x(t) L2
Pj (s) =
e−s (s + 1)3
and a PI-controller C0 (s) = 1 +1/3s at 25 time units after a reference step for all PI-controllers Ci (s) = kpi (1 + (1/Tni s)) in Fig. 4. It can be seen that the cost function has its global minimum for an unstable controller, and hence any optimization algorithm may terminate with this ‘best’ destabilizing controller. The reason why the fact that a controller yields an unstable closed-loop system is not detected is that due to the way the fictitious reference and error signals are computed, the unstable dynamics are not excited. Let us assume that a controller C1 (s) was in the feedback loop up to time and C1 (s) gives a stable closed-loop system. We test the controller C2 (s) which is destabilizing. From (11) and (12),
˜ 2 (s) = C −1 (s)U1 (s) + Y1 (s) = (1 + C −1 (s)P −1 (s))Y1 (s). R 2 2 j ˜ 2 (s) is The relationship between E˜ 2 (s) and R
∞
x2 (t)dt. 0
(22)
2.10.1. Problems when using ˜Ji (t) If the evaluation of the controllers is based upon the fictitious reference signal r˜i and the fictitious error signal e˜ i as in (22), the controller that minimizes the cost function may yield an unstable closed-loop system. As an example, the cost function ˜Ji with = 10−9 is plotted for the linear plant
E˜ 2 (s) = C2−1 (s)U1 (s) = (C2 (s)Pj (s))−1 Y1 (s)
The ∞ space of piecewise continuous, square-integrable signals, i.e. 0 x2 (t)dt < ∞, is called L2 -space. The norm of a continuoustime signal x(t) ∈ L2 is defined as
˜ri (t) 2=[0,t]
where is a positive scalar.
,
˜ PID (s) . xr0j ∈ R2 is the initial state vector ) i
˜ ˜ −1 xpsij−1 = −A i bi usj−1
˜ei (t) 2=[0,t] + u (t) 2=[0,t]
(20)
E˜ 2 (s) =
1 ˜ 2 (s) ˜ 2 (s) = S2 (s)R R 1 + C2 (s)Pj (s)
2012
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
and S2 (s) is unstable with (at least) one RHP pole p˜ . However, as ˜ 2 (s) = (1 + (C2 (s)Pj (s))−1 )Y1 (s) = R
1 + C2 (s)Pj (s) C2 (s)Pj (s)
Y1 (s),
˜ 2 (s) (the ficthe transfer function from Y1 (s) (the true signal) to R titious input) has a zero at p˜ and the unstable dynamics are not excited. If C2 (s) has a stable causal inverse,
˜e2 (t) 2 ≤ u1 (t) 2 for some constant = sup|C2−1 (jω)|2 . On the other hand, ω
˜ 2 (s) = R
[C2−1 (s) + Pj (s)]U1 (s)
and C2−1 (s) + Pj (s) is stable if Pj (s) is stable. Thus
˜r2 (t) 2 ≥ ˇ u1 (t) 2 with ˇ = inf |C2−1 (jω) + Pj (jω)|2 . If C2 (s) is causal and causally invert-
Fig. 5. Example of a fitness landscape using Ji∗ .
ω
ible, ˇ > 0 except when 1 + C2 (s)Pj (s) has a zero exactly on the jω axis. Therefore for → ∞
˜e2 (t) 2
˜r2 (t) 2
+
u1 (t) 2
˜r2 (t) 2
≤
+ ˇ
and ˜J2 will remain finite. So using the cost function ˜Ji it will in general not be possible to detect instability of the closed-loop caused by destabilizing controllers and ˜Ji is not suitable to optimize the controller parameters. 2.11. The new fictitious error signal ei∗ (t) 2.11.1. Definition In order to detect that instability of the closed-loop with a candidate controller without putting the controller in the loop and as a basis for an optimization of the controller parameters, a new performance measure that measures the true closed-loop performance of Ci (s) is needed. To avoid the cancelation of unstable poles and zeros, it must be based on the control error for the true reference signal that would result if the candidate controller was placed in the loop. A new procedure to compute this new fictitious error signal for a closed-loop pair (Ci (s), Pj (s)) was proposed in [6]: The sensitivity transfer function of a non-active controller Ci is Si (s)
Ei∗ (s) Rj (s)
=
1 1 + Ci (s)Pj (s)
(23)
where Ei∗ (s) is the Laplace transform of the resulting error ei∗ (t) and Rj (s) is the Laplace transform of the true reference signal rj (t) that was applied to the loop for the current operating point pj . However, we cannot compute ei∗ (t) directly since Pj (s) is unknown. A possible ˜ i (s) and E˜ i (s) and way is to compute it using R E˜ (s) 1 Si (s) = i = ˜ i (s) 1 + Ci (s)Pj (s) R
(27)
with the deconvolution matrix of Ci (s) of the form:
⎡ r˜ (k − l ) 0 ··· 0 mj i ⎢. ⎢. r˜i (k − lmj ) 0 0 ⎢. ˜ Ri = ⎢ ⎢ .. ⎢ . r˜i (k − lmj ) 0 ⎣ r˜i (k − 1) r˜i (k − 1)
r˜i (k)
e˜ i = e˜ i (k − lmj )
···
(26)
Practically (25) and (26) are solved using sampled signals as proposed in [6].
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
r˜i (k − lmj ) e˜ i (k)
···
s˜i (1)
T T
s˜i (lmj )
where lmj is the estimated settling time of the closed-loop system with the previous controller for the current operating point pj . Thus
e∗i = ei∗ (k − lmj )
···
ei∗ (k − 1)
ei∗ (k)
T
−1
˜i = Rj · R
· e˜ i
˜ i. where Rj is defined similar to R 2.12. The new cost function Ji∗ (t) The new cost function to measure the performance of a candidate controller Ci (s) is defined as
ei∗ (t) 2=[t ,t +t j
j
sj ]
+ u∗i (t) 2=[t ,t +t j
rj (t) 2=[t ,t +t j
(25)
···
e˜ i (k − 1)
˜ −1 ˜ i = s˜i (0) s˜ i = R i ·e
Ji∗ (t)
Then the signal of interest can be computed as ei∗ (t) = si (t) ∗ rj (t).
˜ i · s˜ i = e˜ i R
(24)
by solving: e˜ i (t) = si (t) ∗ r˜i (t).
2.11.2. Numerical computation of ei∗ (t) si (t) can be estimated from the measured data u (t) and y (t) via e˜ i (t) and r˜i (t). The deconvolution can be performed using sampled signals:
j
j
sj ]
(28)
sj ]
where u∗i (t) = ci (t) ∗ ei∗ (t). For the same example as in Fig. 4, the new cost function is plotted in Fig. 5. The stability region is clearly visible in Fig. 5 whereas in Fig. 4 destabilizing controllers seemed to yield a good performance. The unstable dynamics of a fictitious closed-loop pair (Ci (s), Pj (s)) are detected via Ji∗ (t) but not via ˜Ji (t). An optimal controller for the unknown plant Pj (s) can be obtained using Ji∗ (t).
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
2013
Fig. 6. Cost monitoring scheme. Fig. 7. -Hysteresis switching algorithm.
2.13. Falsified controllers and unfalsified controllers
2.15. The switching mechanism
Following the approach in Safonov and Tsao [1], an unsatisfactory closed-loop performance of a non-active candidate controller Ci (s) can be detected using the criterion Ji∗ (rj (t), u (t), y (t), t) = Ji∗ (ei∗ (t), u∗i (t), rj (t), t) < ˛
(29)
where ˛ is called the unfalsification threshold. If this criterion is met, Ci (s) is said to be an unfalsified controller. Otherwise, Ci (s) is said to be a falsified controller.
2.14. Cost monitoring: when to switch Using Ji∗ , the cost monitoring scheme evaluates the performances of all candidate controllers as shown in Fig. 6. At each sampling time, Ji∗ (k), ∀i ∈ M will be sent to the switching mechanism to decide on the next switching signal (k + 1). For the automatic controller tuning, the issue of when to switch the performance monitoring on and when to switch it off must be addressed. The numerical computation in the deconvolution (27) fails if r˜i (k − lmj ) tends to zero. Using (15) and (17), we obtain for PID controllers
= rj
kpi (ς + Tdi )
.
2.15.1. The -hysteresis switching algorithm A switching mechanism can be considered as a good algorithm for unfalsified control if it can discover the best unfalsified controller in a given set of controllers within a reasonable period of time. In [9], such an algorithm was proposed, the so-called hysteresis switching algorithm: 1. Initialize: choose > 0 and (tj0 ) = m ∈ M for a time-window of length = [tj0 , tj0 + tsj ]. 2. tjk :=tjk+1 ∗ if J(t (t ) ≥ minJi∗ (tjk ) + ) jk i∈M
then (tjk+1 ):=argminJi∗ (tjk )
s→∞
kp (ς + Td )
switches it off and replaces it with the best non-active candidate controller.
jk
˜ PID (s) ˜ PID (s)Rj (s) = rj lim r˜i (k − lmj ) = lim s i i s→∞
The switching mechanism plays an important role to implement the active controller C (s) in unfalsified control. It can be thought of as a switching supervisor that monitors the performance of all candidate controllers via the new cost function Ji∗ (t). In other words, at every time instant, if an active controller C (s) is performing significantly worse in the feedback loop than all non-active candidate controllers Ci (s) (using the comparison of J∗ and Ji∗ ), the supervisor
i∈M
(30)
˜ i is ill-conditioned. From If rj (kp (ς + Td )/kpi (ς + Tdi )) → 0, R (30), the computation cannot be performed if rj = 0 (no exter˜ i does not exist. Even nal excitation) because the inverse of R if there is enough excitation, the controller gain ratio from the active controller to the nonactive controller must be monitored. A large controller gain of a non-active controller can also lead to ill-conditioned matrices. In the implementation of the algorithm, two issues have to be taken care of: excitation size rj and window length = [k − lmj , k]. For the computation of the cost function Ji∗ , a sufficient amount of data is necessary, and for accurate results, the window length lmj should roughly match the settling time of the controlled system at each operating point pj . The data in the window must be generated by one fixed controller in the feedback loop, so an optimization algorithm can only start after a certain period of time after the last switching. Secondly, the computations will fail if the control error has converged to zero for most of the data within the win˜ i . So the activation of dow due to ill-conditioning of the matrices R an optimization algorithm is restricted to some interval after an excitation of the system by the change of the reference signal or when the real error is approaching to zero so that the monitoring of the real error can be used to trigger the optimization. In summary, the cost monitoring is switched on at the current excitation time tj and will be switched off at the current settling time tsj and then the optimization algorithm is activated.
else (tjk+1 ):=(tjk ). 3. Go to 2 until tjk = tj0 + tsj . It is equivalent to the block diagram in Fig. 7. Fig. 8 illustrates how the -hysteresis switching algorithm works. We assume that C1 (s) is a destabilizing controller and Ci (s), ∀i ∈ M − {1} are stabilizing controllers for an unknown linearized plant Pj (s). The candidate controller C1 (s) is chosen as the first active controller ((0) = 1) in the feedback loop. As time progresses, the -hysteresis switching algorithm compares the cost of C=1 (s) with the cost of all
Fig. 8. Example of the -hysteresis switching algorithm.
2014
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
non-active controllers Ci (s), ∀i ∈ M − {1} that are computed by the cost monitoring after it had been switched on at t = 0. During the period 0 ≤ t ≤ tu , C1 (s) is kept in the feedback loop because it is the best unfalsified controller. During tu < t < t:1→2 , C1 (s) is still kept because the switching criterion is not yet fulfilled, but C2 (s) now becomes the best unfalsified controller. That means C1 (s) will sooner or later be switched off and C2 (s) will be the new active ∗ controller whenever J=1 ≥ J2∗ + is true. It can be seen that C1 (s) is unfalsified for t < tf because its cost still remains below the unfalsification threshold ˛. Since only C1 is destabilizing for Pj (s), J1∗ (t) tends to infinity as t→ ∞ but Ji∗ (t), ∀i ∈ M − {1} remain bounded as t→ ∞. ∗ At t = t:1→2 , the condition J=1 ≥ J2∗ + is true and t:1→2 denotes the switching time. For t ≥ t:1→2 , C=2 (s) is kept in the feedback loop since then. Note that C1 (s) is falsified after t ≥ tf .
2.15.2. Introduction of the max function A small modification of Ji∗ is introduced for its practical utilization during the transient period. Without loss of generality, we assume that a time-window of length = [tj , tj + tsj ] is considered. Let us assume that the true performance of all candidate controllers can be precisely detected at a time around the unknown closedloop settling time tsj as shown in Fig 5. At the very beginning of that period, just after switching on the cost monitoring, the cost of a destabilizing controller is still finite (it may be even lower than the cost of a stabilizing controller) but getting larger and larger over time, i.e. the ratio ei∗ (t) 2 / rj (t) 2 → ∞. At the same time, the cost of the stabilizing controllers is of course bounded and getting smaller and smaller over time, i.e. the ratio ei∗ (t) 2 / rj (t) 2 → 0. If the data length is long enough, the true performances of all candidate controllers become reliable. This may lead to a situation where a non-active destabilizing controller is switched on in the feedback loop since the cost monitoring has not yet received sufficient data to assess the true performances of all candidate controllers. A poor closed-loop performance might be the result. For this reason, too many switches of active controllers are prevented using a nondecreasing cost function. In order to prevent such a situation, Ji∗ is modified using the max function Ji∗ () = max t min) of the best indif
vidual within a certain number of generations can be observed, the termination criterion of the evolution strategy is not fulfilled, and the population of parents selected individuals from the previous generation are used for the next iteration. 2.18. Suitable length time-window for the EA We assume that the active controller C (s) ∈ Kj is stabilizing the unknown linearized plant Pj (s) in the interval tj < t < tj+1 . Kj denotes
the current set of candidate controllers. A suitable time to activate the EA is around the unknown closed-loop settling time tsj . We fix a maximum computation length lm at tj∗ = tj + lm ≈ tj + tsj < tj+1 . The cost monitoring uses a time-window of length j = [tj , tj∗ ] and then the EA is activated at t = tj∗ . 2.19. Adaptive set of controllers A key point of this work is how the current set of candidate controllers can be adapted when a new operating point is commanded. K(t) denotes a controller set that varies with time. The set of initial candidate controllers K0 K(t = 0) is predefined for the start-up of the unknown plant P. Thus m candidate controllers are chosen freely at the beginning. In addition, we denote Kj = K(tj < t < tj∗ ) at the current operating point pj . After convergence of the ES, a set of often almost identical candidate controllers at pj results K∗j Kopt (t = tj∗ ).
(40)
K∗j
is the last population of the ES which used the observed data vector dj =[tj ,t ∗ ] (t) and the new cost function Ji∗ (t) as the fitness j
function. In general, after termination of the ES at tj∗ , the controllers in the set have converged to the vicinity of the best controller, so it is not a good idea to use the resulting population as the new set of controllers because within this set switching does not make sense and there is no diversity to cope with changes of the plant dynamics. Therefore the set of controllers is diversified to maintain variety within the set of the candidate controllers. The set of diversified candidate controllers at pj is defined as KF K(tj∗ < t < tj+1 ) j
(41) c∗j
∈ Rn is where F is a diversity operator. The optimal solution , i.e. other candidate used as a center to generate the new set KF j controllers are distributed around c∗j as shown in Fig. 10. We define KjF {ciF | ciF − c∗j ≤ F,
i = 1, 2, . . . , m − 1,
∀cF ∈ Rn }. i
(42)
Therefore, at the next operating point pj+1 , Kj+1 KF . j
(43)
For a PID controller structure, the search space is defined as KPID EA Kp × Tn × Td
(44)
2016
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
Fig. 11. Unfalsified adaptive control algorithm.
where Kp ⊂ R − {0} and Tn ⊂ R+ and Td ⊂ R+ . 2.20. The central data processing unit In Fig. 11, we illustrate the overall algorithm by a block diagram. There are two main loops, the control loop with the active controller and the intelligent loop (the dashed box) that computes a new switching signal and a new set of controllers. There are three tasks that the central data processing unit performs: • To detect a change of the current set-point and to provide time˜ i (s) for all candidate window data and compute initial states of controllers that are computed in the cost monitoring and in the EA. • To send a command to switch on and off the cost monitoring via the signal OCM (k) and to activate the EA via the signal OEA (k) for processing the controller adaptation. • To choose the currently active controller from the current set of candidate controllers by the -hysteresis switching algorithm and to compute the initial state of the next active controller at the current switching time. In general, this scheme performs two controller adaptations: • Switching between the candidate controllers in the set of controllers that can vary with time, • Recomputation of the set of controllers by the ES and the diversity operator online. This combination leads to a new method for automatic controller tuning without using an explicit plant model. The following elements of the controller must be chosen a priori:
gain controller should be active first. Afterwards, K(k) will be automatically generated by the algorithm. 2. The switching criterion • The modified cost function with the max operator is used in the cost monitoring to compute the cost level of all candidate controllers in K(k) in the -hysteresis switching algorithm to implement the next switching signal (k + 1). The cost monitoring is switched-on immediately after a new excitation is detected, and the cost monitoring will be switched-off if the algorithm receives enough data for the computation of the ˜. inverse of the matrix R i • The constant is chosen to prevent too fast switching of the active controllers. The smaller , the faster the switching. Note that the -hysteresis switching algorithm aims at keeping the currently active (stabilizing) controller in the loop as long as possible. Only if it starts to perform badly when comparing it to the best non-active controller, the algorithm will switch to the best non-active controller. 3. The computation length lm to activate the EA A window-length lm has to be chosen as a controller tuning parameter for each operating point. It can be decided during in operation when the operator thinks that enough data was obtained to optimize the process. In the simulation shown here, lm is fixed for all operating points. The EA uses this data set and the new cost function to compute the best controller parameter vector c* . The unfalsified adaptive controller algorithm switches c* into operation as the first active controller when the cost monitoring is switched-on at the next operating point. 4. The diversity operator F The diversity of the set of controllers usually is small after termination of the ES. To form a new set of controllers K(k) where c* is used as a center, the diversity operator F must be chosen.
3. Stability of the algorithm In the sequel, we prove that for a linear time-invariant stable plant with discrete transfer function P(z), the new cost function leads to the detection of instability for controllers which are not in the loop (that instability of the present controller is detected is obvious from the definition). This implies that if there is a stabilizing controller in the set of controllers, the switching mechanism will ultimately switch to a stabilizing controller. The proof is based on a few definitions and lemmas from linear control theory which are stated below for completeness sake. Definition 1. Any sequence x(j) with finite l2 -norm is said to belong to the space L. For a linear operator G : l2 → l2 , the H∞ -norm is defined as
G ∞ sup G(z)
z∈D
1. The initial set of candidate controllers K(0) There is no a systematic way to derive K(0). This initial set can be chosen freely, i.e. a suitable number of sufficiently diverse candidate controllers to make sure that there will be at least one stabilizing controller to start-up the physical plant P. The most important choice is that of the first active controller which is connected to the physical plant P while the unfalsified adaptive control algorithm has not yet received sufficient plant input–output data. If a destabilizing controller is active first, that obviously can lead to a saturation problem for an actuator and the desired output response can degrade very much during the start-up. This is why it is crucial to use available knowledge about the physical plant P. For example, if is known that P has a non-minimum phase behavior, to be on the safe side a low
where G(z) is the z-domain transfer function of G and D is the complement of the open unit disk in the complex plane. Definition 2. A scalar discrete-time closed-loop system is (internally) l2 -stable, if for all paths from external signals xi to internal signals yk , there is a constant Mki such that
L
y2 (j) j=0 k
L
x2 (j) j=0 i
as long as
≤ Mki , ∀L
L
x2 (j) j=0 i
and ∀xi with bounded l2 -norm
> 0, ∀L.
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
2017
Lemma 3. We assume that yj (j) and uj (j) result from the application of the controller Cj in the loop with the stable plant P shown in Fig. 12, and that the true closed-loop system is l2 -stable, i.e. Sj (z) and Cj (z)Sj (z) are H∞ -functions. Then, for any controller Ci that is not in the loop, we have Si (z) =
Fig. 12. Fictitious linear time-invariant closed-loop system.
Lemma 1. If a control loop is linear and time-invariant, then it is necessary and sufficient for l2 -stability that for all paths from external signals xi to internal signals yk Gki (z) =
∞
R(z)
E˜ i (z) = Ci (z)−1 Uj (z),
˜ i (z) = E˜ i (z) + Yj (z). R
Simple transfer function algebra.
Corollary 3.
j=0
E˜ i (z) ˜ i (z) R
=
with
Proof.
gki (j) · z −j
Ei∗ (z)
Let
∞
with
Si (z) =
j
yk (j) =
si (k) · z −k
k=0
gki (l) · xi (j − l) = gki (j) ∗ xi (j)
and assume that Cj (z)/Ci (z) = / 0 as z→ ∞ and r(0) = / 0. Then the impulse sequence si (k) can be computed from the measurements uj (k) and yj (k) by deconvolution:
l=0
is an H∞ -function, i.e. analytic for |z| > 1 and bounded for |z| ≥ 1. See Vidyasagar [26].
Proof.
Lemma 2. Consider the (fictitious) closed-loop system shown in Fig. 12 with linear plant P and controller Ci . If the plant P is stable, then it is necessary and sufficient for internal closed-loop stability that the transfer functions Si (z) =
Ei∗ (z) R(z)
=
1 1 + Ci (z)P(z)
∞
cˆi (k) · z −k
k=0
e˜ i (k) =
k
cˆi (l) · uj (k − l)
l=0
and Ci (z)Si (z) =
Cˆ i (z) = Ci (z)−1 =
r˜i (k) = e˜ i (k) + yj (k)
Ui∗ (z)
e˜ i (0) = r˜i (0) · si (0)
R(z)
are H∞ -functions.
e˜ i (1) = r˜i (1) · si (0) + r˜i (0) · si (1)
Proof. This follows directly from Lemma 1 and the stability of P(z).
e˜ i (2) = r˜i (2) · si (0) + r˜i (1) · si (1) + r˜i (0) · si (2)
If Si (z) and Ci (z)Si (z) are H∞ -functions,
Corollary 1.
L
e∗ (j)2 j=0 i
L
j=0
r(j)2
e˜ i (l) = r˜i (l) · si (0) + r˜i (l − 1) · si (1) + · · · + r˜i (0) · si (l)
∀L
≤ Me ,
.. . . = ..
and
from which it follows that:
L
u∗ (j)2 j=0 i L r(j)2 j=0
as long as Proof.
L j=0
si (0) = e˜ i (0)/˜ri (0)
∀L
≤ Mu ,
si (1) = [˜ei (1) − si (0) · r˜i (1)]/˜ri (0) si (2) = [˜ei (2) − si (0) · r˜i (2) − si (1) · r˜i (1)]/˜ri (0)
r(j)2 = / 0, ∀L.
.. .. .=.
Follows from Lemma 1.
Corollary 2. Consider the feedback loop in Fig. 12 and assume that P(z) is stable and that r(0) = / 0. Let
L
Ji∗ (L) =
.. .. .=.
e∗ (j)2 j=0 i
+
L
j=0
L
u∗ (j)2 j=0 i
2
.
si (m) · r˜i (l − m)]/˜ri (0)
convolution up to l−1
Then the closed-loop system is stable if and only if there are constants Me and Mu such that
∀L.
l−1 m=0
r(j)
Ji∗ (L) ≤ Me + Mu < ∞,
si (l) = [˜ei (l) −
(45)
Proof. If the loop is stable, Si (z) and Ci (z)Si (z) are H∞ -functions, hence Corollary 1 gives that Ji∗ (L) is bounded ∀L. Conversely, if (45) does not hold, either the path from r(j) to ei∗ (j) or the path from r(j) to u∗i (j) are unstable, or both. Thus the loop is unstable.
.. .. .=. / 0. From simple block diagram It only remains to show that r˜i (0) = algebra, ˜ i (z) = R
Cj (z)(1 + Ci (z)P(z)) Ci (z)(1 + Cj (z)P(z))
R(z),
hence if Cj (z)/Ci (z) = / 0 for z→ ∞ and r(0) = / 0, then r˜i (0) = / 0. Thus with Lemma 3, si (k) can be computed by deconvolution as stated above.
2018
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025 (k)
Lemma 4. Let s˜i (j) be the sensitivity impulse response that is computed from finite data r˜i (j), j = 0, . . . , k and e˜ i (j), j = 0, 1, . . . , k. Assume that the control loop with controller Cj has reached an equilibrium before time j = 0, i.e. r(j) = y(j) = 0 for j < 0. (k) Then the coefficients s˜i (j) that have been computed from finite input data up to sampling point k are equal to the true coefficients si (j) for j = 0, 1, . . ., k. Proof. Under the assumptions above, e˜ i (j), r˜i (j) vanish for j < 0, and it follows from Corollary 3 that the computation by deconvolution (k) is exact. Thus s˜i (j) = si (j) for j = 0, 1, . . ., k. Corollary 4. Under the conditions of Lemma 4, the computations of ∗(k) ∗(k) ei (j) and ui (j) by ∗(k)
ei
(j) =
j
(k)
s˜i (l) · r(j − l)
l=0
and ∗(k)
ui
(j) =
j
∗(k)
ci (l) · ei
(j − l)
l=0
Fig. 13. Continuous stirred tank reactor.
are exact for j = 0, 1, . . ., k. ∗(k)
Proof. s˜ik is exact up to time k, hence ei computed directly by convolution. Theorem 1.
k Ji∗ (k)
∗(k)
(j) and ui
(j) can be
Define
∗(k) e (j)2 j=0 i
+
k
j=0
k
∗(k) u (j)2 j=0 i
r(j)2
and assume r(0) = / 0. Then if the fictitious linear control loop in Fig. 12 is stable, Ji∗ (k) is bounded. Conversely, if • the linear time-invariant plant with transfer function P(z) is stable • r(j) = y(j) = 0 for j < 0 • u(j) and y(j) are measured without error for j ≥ 0 C (z) • lim j = / 0 z→∞ Ci (z)
• r(j) is bounded and R(z) does not vanish for |z| ≥ 1 then Ji∗ (k) will grow unboundedly if one of the transfer functions Si (z) or Ci (z)Si (z) are unstable, i.e. have a pole with |z| ≥ 1. Proof. The first part follows immediately from Corollary 4 and the definition of l2 -stability. For the second part, if one of the closed-loop transfer functions has a pole outside the unit circle ei∗ (j) and/or u∗i (j) will grow |p|k with |p| > 1 because this pole cannot be canceled by a zero of R(z), ∗(k) ∗(k) and as ei (j) = ei∗ (j) and ui (j) = u∗i (j) for j ≤ k and r(j) is bounded, ∗ Ji (k) will grow unboundedly. Remark 1. The condition that r(j) = y(j) = 0 for j < 0 can be replaced by the assumption that both signals have been constant for a sufficiently long period of time. Remark 2. Under the assumptions of Theorem 1, it cannot be ruled out that the closed-loop has poles on the unit circle if Ji∗ (k) < Mj , ∀k. Theorem 2. Let the assumptions of Theorem 1 be satisfied. Assume that there is at least one not destabilizing controller Cs (z) in the set and the unfalsification threshold is set sufficiently high. Then the switching mechanism will switch to one of these stabilizing controllers after a finite number of time steps.
Proof. Ji∗ (k)
Let ˛ be such that
< ˛,
∀k
for at least one of the not destabilizing controllers. From Theorem 1, Ji∗ (k) will exceed ˛ for all destabilizing controllers after a finite number of steps, hence the switching mechanism will finally choose one of the non-destabilizing controllers. Remark 3. It cannot be ruled out that a controller that generates oscillatory poles on the unit circle remains in the loop because then both the numerator and the denominator if the cost function grow proportionally to t. However, if another controller in the set gives a significantly better performance, it will be switched off after some time. 4. Application to a nonlinear system 4.1. A continuous stirred tank reactor model As an example of the application of the online tuning scheme to a nonlinear process, we investigate the well-known case study of the control of a CSTR with the van-der-Vusse reaction scheme. CSTRs are good examples for the use of the unfalsified adaptive control algorithm due to their highly nonlinear dynamics especially when consecutive and side reactions occur. A sketch of the reactor is shown in Fig. 13. As a benchmark problem for a nonlinear control design, we consider the production of cyclopentenal (B) from cyclopentadiene (A) as presented by [27]. Due to the strong reactivity of the educt and the product, dicyclopentadiene (D) is produced as a side product, and cyclopentanediol (C) as a consecutive product by the addition of another water molecule. The reaction scheme is k1
k2
A−→B−→C,
(46)
k3
2A−→D. The educt flow contains only A in a low concentration cA,in . We assume a constant density and an ideal residence time distribution within the reactor. The parameters of the model and the standard operation point are taken from [27] and [3]. We assume that the cooling power Q˙ k is applied in a range around its value at the standard operating point Q˙ ks = −4496 kJ/h. Control of the
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
2019
Loss of diversity after ES termination at p1
0.1
Td
0.09 0.08
0.04
0.035
0.03
0.025
0.02
3
T
kp
n
5
4.5
4
3.5
Other controllers
Diversity at p1
Best controller
T
d
0.2
0.1
0 0.05
0.04
0.03
0.02
0.01
1
4
3
2
T
6
5
7
8
k
n
p
Fig. 14. Controller set K2 for p2 (see above).
product concentration cB is specified such that any value in the range of 0.85 mol/l ≤cB ≤ 0.95 mol/l can be achieved without a steady state error for a value of the unmeasured inlet concentration cA,in = 5.1 mol/l and an inlet temperature ϑin = 130 ◦ C. The SISO nonlinear model which results from the mass balances for cA and cB and the energy balance for ϑ can be derived as
2. Operation scheme:
x˙ 1 = −k1 (x3 )x1 − k3 (x3 )x12 + (x1,in − x1 )u r(t) = x˙ 2 = k1 (x3 )x1 − k2 (x3 )x2 − x2 u x˙ 3 = −
1 [k1 (x3 )x1 HRAB + k2 (x3 )x2 HRBC + k3 (x3 )x12 HRAD ] Cp
+ u(x3,in − x3 ) +
⎧ 0 mol/l ⎪ ⎪ ⎪ ⎪ 0.9 mol/l(p1 ) ⎪ ⎪ ⎪ ⎪ ⎪ 0.95 mol/l(p2 ) ⎪ ⎪ ⎨ 0.85 mol/l(p3 )
⎪ ⎪ ⎪ 0.92 mol/l(p4 ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 0.88 mol/l(p5 ) ⎪ ⎪ ⎩
0.935 mol/l(p6 )
:
0 ≤ t < 0.15 h;
:
0.15 h ≤ t < 4.15 h;
:
4.15 h ≤ t < 6.15 h;
:
6.15 h ≤ t < 8.15 h;
:
8.15 h ≤ t < 11.15 h;
:
11.15 h ≤ t < 14.15 h;
:
14.15 h ≤ t < 16 h.
Q˙ ks Cp VR
y = x2 where x1 = cA , x2 = cB , x3 = ϑ, x1,in = cA,in , and x3,in = ϑin . We assume that the initial reactor temperature x30 = x3 (0) is equal to room temperature 25 ◦ C during the start-up. 4.2. Initialization of the algorithm The simulations are carried out under the following assumptions: 1. The CSTR model is used as the unknown plant P with a finite time delay due to an analytic instrument for the measurement of the product concentration of 0.02 h.
Note that we first move the process to the standard operating point cBs = 0.9 mol/l from the origin (start-up). Then we move to the second operating point after t ≥ 4.15 h, etc. 3. The PID controller structure is set up as follows: (a) A constant value of ς = 0.01 is chosen to smoothen the derivative term. (b) The initial set of PID candidate controllers consists of 27 candidate controllers: K0 = {ci (t = 0)|ci = [kpi , Tni , Tdi ]T , kpi ∈ {10, 50, 100}, Tni ∈ {0.1, 0.5, 1}, Tdi ∈ {0.001, 0.05, 0.1}} and the first active controller is chosen as c(0) = [10, 0.1, 0.001]T . (c) The search space for the PID candidate controllers is KPID EA = [−100, 100] × [0.01, 1] × [0.001, 0.1].
2020
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025 −1
Concentration of component B (mol ⋅ l ) 0.96 Set−points Automatic online controller tuning Fixed controller
0.95 0.94 0.93 0.92
B
c (t)
0.91 0.9
Fig. 16. Adaptive control system with measurement errors. 0.89 0.88
5. Unfalsified control with measurement errors
0.87 0.86 0.85 0.84
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Time (h) Fig. 15. Output response of the CSTR example with fixed and adaptive PID controllers.
(d) We define the diversity operator as KF (t) {ci (t)|ci = [kpi , Tni , Tdi ]T , j 1 kpi ∈ { kp∗ i , kp∗ i , Fkp∗ i }, F 1 Tni ∈ { Tn∗i , Tn∗i , FTn∗i }, F 1 Tdi ∈ { Td∗ , Td∗ , FTd∗ }} i i F i and F=2 is used. 4. The switching condition = 0.001 is used for the switching of an active controller and = 10−9 is used in Ji∗ . 5. The window length j = [tj , tj + lm ], j = 1, 2, 3, 4, 5, 6 is used where the length lm = 0.3 h is applied for the cost monitoring and the activation of the EA. 6. The ( + )-selection is used.
In a real situation, the observed output data is disturbed by measurement errors which affect the optimization of the controllers and the switching of the adaptive control law. In this section, the effect of these errors is analyzed and an approach to reduce the effect of measurement noise is presented [12]. 5.1. Measurement errors in the plant output We consider the scheme in Fig. 16 with a linear plant and a linear controller. In the Laplace domain:
Y (s) = P(s)U(s) + W (s) = P(s)C (s)(R(s) − Y (s)) + W (s) = T (s)R(s) + S (s)W (s) = Ytrue (s) + Yw (s)
(47)
with the active complementary sensitivity function T (s) =
C (s)P(s) 1 + C (s)P(s)
(48)
and the active sensitivity function S (s) =
1 . 1 + C (s)P(s)
(49)
The observed disturbed plant input signal is If the controller is fixed to the first active controller, a poor performance results as shown in Fig. 15 because one fixed controller cannot handle the nonlinear behavior of the CSTR model due to changes of the set-points. For this reason, the unfalsified adaptive control algorithm is applied to the CSTR model. The initial set K1 K0 is used at the first operating point p1 . The switching between the candidate controllers in K1 is performed during the interval 0.15 h ≤ t < 0.45 h and the EA is activated at t = t1∗ = 0.45 h as the first tuning point. Since c(0) is stabilizing P and the other candidate controllers in K1 are not better than this controller, it is kept in the loop up to t = t1∗ as shown in Fig. 20. After the termination of the ES, the optimized set of controllers at p1 results as shown in Fig. 14. The optimal controller c∗1 = [3.841, 0.023, 0.1]T is in the loop until p2 is selected at t = t2 = 4.15 h and it will be assigned as the new initial controller for p2 as shown in Fig. 20. According to the diversity operator, at the first tuning point, the new controller set K2 is also used for p2 as shown in Fig. 14. Note that the other candidate controllers are distributed around c∗1 according to F=2. The same procedure is automatically applied at the next operating points p2 , p3 , p4 , p5 , p6 , respectively. The optimal controllers c∗2 , c∗3 , c∗4 , c∗5 , c∗6 are switched on at each tuning point. So this algorithm leads to automatic PID controller tuning for the nonlinear CSTR model as shown in Fig. 20. The output performance is shown in Fig. 15. Note that a good performance can be achieved using the proposed algorithm.
U(s) = C (s)(R(s) − Y (s)) = C (s)(R(s) − Ytrue (s)) − C (s)Yw (s) = Utrue (s) − C (s)Yw (s).
(50)
Hence y(t) and u(t) consist of both deterministic and stochastic components: y(t) = ytrue (t) + yw (t)
(51)
and u(t) = utrue (t) − c (t) ∗ yw (t),
(52)
where ytrue (t) = t (t) ∗ r(t)
(53)
utrue (t) = c (t) ∗ (r(t) − ytrue (t)),
(54)
and yw (t) = s (t) ∗ w(t).
(55)
Therefore, the error propagation in the measured plant input–output data depends on the closed-loop performance of the active controller.
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
2021
Output Response 1.5
y(t)
1 0.5 r y
0
true
y −0.5 0
5
10
15
20
25
Time (sec) Input Response 3 u
true
u
u(t)
2
1
0 0
5
10
15
20
25
Time (sec) Fig. 17. Noisy plant input–output data for the example 5.2.4.
According to (11), the stochastic fictitious error signal E˜ i,w (s) of Ci (s) can be computed from (U(s), Y(s)):
5.2. Deconvoluion of noisy signals 5.2.1. Stochastic fictitious signals According to (12), the stochastic fictitious reference signal ˜ i,w (s) of Ci (s) using the noisy observed plant input–output data R (U(s), Y(s)) while the controller C (s) is active, is given by ˜ i,w (s) = C −1 (s)U(s) + Y (s) = C −1 (s)C (s)(R(s) − Y (s)) + Y (s) R i i =
E˜ i,w (s) = Ci−1 (s)U(s) = Ci−1 (s)C (s)(R(s) − Y (s)) = Ci−1 (s)C (s)(R(s) − Ytrue (s) − Yw (s)) = E˜ i,true (s) + E˜ i (s) where E˜ i (s) = −Ci−1 (s)C (s)S (s)W (s). Note that E˜ C (s) (s) =
Ci−1 (s)C (s)(R(s) − Ytrue (s) − Yw (s)) + Ytrue (s) + Yw (s)
∗ (t) −Yw (s) = / 0. According to (26), the new fictitious error signal ei,w of Ci (s) is
˜ i,true (s) + R ˜ i (s) =R ˜ i (s) = (1 − C −1 (s)C (s))S (s)W (s). Note that R ˜ C (s) = 0. where R i
∗ (s) = S˜ i,w (s)R(s). Ei,w
(56)
Fictitious Reference Signal 1.5
r1
1 0.5 0 0 6
5
10
15
20
25
5
10
15
20
25
5
10
15
20
25
r
2
4 2 0 −2 0 1.005
r3
1 0.995 0.99 0
Time (sec) Fig. 18. Fictitious reference signals for the example 5.2.4.
2022
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
New Fictitious Error Signal 200
*
e1
100 0 −100 −200 0 1.5
5
10
15
20
25
5
10
15
20
25
5
10
15
20
25
*
e2
1 0.5 0 −0.5 0 1.5
e*
3
1 0.5 0 −0.5 0
Time (sec) Fig. 19. True and noise corrupted fictitious error signals for the example 5.2.4.
S˜ i,w (s) can be obtained applying (25),
According to (57), s˜i,w (t) can be computed from u(t) and y(t) via e˜ i,w (t) and r˜i,w (t). The deconvolution of noisy signals is performed using sampled signals:
˜ i,w (s) E˜ i,w (s) = S˜ i,w (s)R
˜ ·˜ ˜i,w R i,w si,w = e
(57)
e˜ i,w (t) = s˜i,w (t) ∗ r˜i,w (t).
(58)
˜ ˜ s ˜i,true + e ˜i (R i,true + Ri ) · ˜ i,w = e
−1
−1
Trajectory of kp (h )/(mol ⋅ l ) 20
k
p
15 10 5 0
0
1
2
3
4
5
6
7
8 9 Time (h)
10
11
12
13
14
15
16
10
11
12
13
14
15
16
10
11
12
13
14
15
16
Trajectory of T (h) n
T
n
0.1
0.05
0
0
1
2
3
4
5
6
7
8 9 Time (h)
Trajectory of T (h) d
Td
0.1
0.05
0
0
1
2
3
4
5
6
7
8 9 Time (h)
Fig. 20. Automatic online PID controller tuning without measurement errors for the CSTR example.
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025 −1
2023
−1
Trajectory of kp (h )/(mol ⋅ l ) 30
k
p
20 10 0
0
1
2
3
4
5
6
7
8 9 Time (h)
10
11
12
13
14
15
16
10
11
12
13
14
15
16
10
11
12
13
14
15
16
Trajectory of Tn (h)
Tn
0.1
0.05
0
0
1
2
3
4
5
6
7
8 9 Time (h)
Trajectory of Td (h)
0.1
T
d
0.2
0 0
1
2
3
4
5
6
7
8 9 Time (h)
Fig. 21. Automatic online PID controller tuning under measurement errors for the CSTR example.
where
⎡
r˜i,true (0)
···
0
⎢. ⎢. r˜i,true (0) ⎢. ˜ Ri,true = ⎢ ⎢ . ⎢ ⎣ r˜i,true (l − 1) . . r˜i,true (l − 1)
r˜i,true (l)
⎡
˜ri (0)
0
⎢. ⎢. ˜ri (0) ⎢. ˜ Ri = ⎢ ⎢ . ⎢ ⎣ ˜ri (l − 1) . . ˜ri (l) and
⎡
e˜ i,true (0)
˜ri (l − 1)
⎤
⎢. ⎥ ⎢. ⎥ . ⎢ ⎥ ˜ ei,true = ⎢ ⎥ ⎣ e˜ i,true (l − 1) ⎦ e˜ i,true (l)
⎤
0
⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
0
0
r˜i,true (0)
0
···
r˜i,true (0)
···
0
0
0
˜ri (0)
0
···
˜ri (0)
⎡
˜ei (0)
˜ and e ˜ and in the vector e ˜ in the matrix R ˜ that error terms R make the linear equations system (58) solvable, i.e. ˜TLS , e ˜ e ˜TLS } = arg min R ˜ F {˜ sTLS , R ˜ s,˜ R,˜ e
˜ + R) ˜ ·˜ ˜ + e ˜. subject to (R s=e
5.2.2. Solution of the total least squares problem The conditions for the existence and the uniqueness of a TLS solution can be found in [29]. Write
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
˜e ˜] = U˙VT Z = [R where ˙ = diag( 1 , . . ., l+1 ) is a singular value decomposition of Z, 1 ≥ · · · ≥ l+1 are the singular values of Z. The partitioned matrices are defined as
⎡
⎤
⎢. ⎥ ⎢. ⎥ . ⎢ ⎥. ˜ and ei = ⎢ ⎥ ⎣ ˜ei (l − 1) ⎦ ˜ei (l)
The deconvolution problem (58) contains the error terms both in the matrix and in the RHS, and therefore it is not adequate to approach it as an ordinary least squares problem. Instead, we employ the total least squares (TLS) method as proposed by [28,29]. The total least squares method was originally proposed by [28]. The motivation causes from the asymmetry of the least squares (LS) ˜ is taken into account. method where no error term in the matrix R The idea of TLS is to find the minimal (in the Frobenius norm sense)
⎢ V11 ⎢ ⎣
.. .
V = ⎢···
v21
⎤
v12 ⎥ ···
.. . v22
⎡
⎢ ˙11
⎥ ⎢ ⎥, ˙ = ⎢··· ⎦ ⎣
021
.. . 012 ··· .. .
⎤ ⎥ ⎥ ⎥ ⎦
l+1
where V11 , ˙11 = diag(1 , . . . , l ) ∈ Rl×l , v12 , 012 ∈ Rl×1 , v21 , 021 ∈ R1×l , v22 , l+1 ∈ R. A TLS solution exists if and only if v22 is not zero. In addition, it is unique if and only if l = / l+1 . In the case when a TLS solution exists and is unique, the solution is given by ˜ sTLS = −
v12
v22
.
(59)
∗ (t) for controller C can Therefore, the new fictitious error signal ei,w i be computed by
e∗i,w = R · ˜ si,w = R · ˜ siTLS .
(60)
2024
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025
automatic online PID controller tuning under measurement errors is shown in Fig. 21 and the noisy output is shown Fig. 22. Note that a good performance is also obtained under measurement errors.
Concentration of component B (mol ⋅ l−1) 0.96 0.95
6. Conclusions and future work
0.94 0.93 0.92
B
c (t)
0.91 0.9 0.89 0.88 0.87 0.86 0.85 0.84
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Time (h) Fig. 22. Output response of the CSTR example with measurement noise.
5.2.3. Error propagation in the new error fictitious signal We consider the estimated sensitivity function under measurement error: S˜ i,w =
E˜ i,w = S˜ i,true + S˜ i . ˜ i,w R
(61)
Thus, we obtain E˜ i,w E˜ − i. ˜ i,w ˜i R R
S˜ i = S˜ i,w − S˜ i,true =
(62)
We can derive the error propagation in the new fictitious error signal as ∗ ∗ Ei∗ = Ei,w − Ei,true = R · S˜ i =
E˜ i,w 1 R− R. ˜ i,w 1 + Ci (s)P(s) R
(63)
Note that if Ci is destabilizing, Ei∗ blows up. 5.2.4. Example of estimated fictitious error signals We assume that P(s) = 1/(s + 1)3 and C (s) = 2(1 + 1/3s) and a unit step was applied to r(t). d = (r , u , y ) was observed up to = 25 s. We assume measurement errors in the plant output data as shown in Fig. 17. Three candidate controllers are tested:
1 3s 1 0.4 1 + 3s 1 2 1 + 3s .
1. C1 (s) = 10 1 + 2. C2 (s) = 3. C3 (s) =
This results in r˜i,true (t) and r˜i,w (t) as shown in Fig. 18. Note that ∗ the computation of r˜3,w (t) is error-free because C3 = C . ei,true (t) ∗ (t) are shown in Fig. 19. The closed-loop instability of C is and ei,w 1 detected because the unstable modes of the error propagation from (63) are excited and the performances of C2 and C3 are estimated well. 5.3. Unfalsified control with measurement errors applied to the CSTR model As an example, we run the algorithm with the nominal 3rd order non-isothermal CSTR model as we did in the previous section. Measurement errors are assumed as an i.i.d. Gaussian white noise. The
A new online tuning controller switching and tuning algorithm based upon ideas from unfalsified control was presented. This algorithm constitutes a new automatic fully data-based controller tuning method. The most important feature of the method is that the computation of controllers is done online without using an explicit plant model. The proposed algorithm consists of two elements: (1) switching between the controllers using the -hysteresis algorithm and (2) recomputation of the set of controllers using an evolution strategy and a diversity operator. Using this approach, the set of controllers changes to handle the nonlinear behavior of the physical plant. The key points of the technique are the use of a new fictitious error signal and of a modified cost function that can detect the instability of the control loop for non-active controllers. For linear time-invariant plants, i.e. for operation around a fixed operating point after a transient period, stability of the algorithm can be guaranteed if at least one non-destabilizing controller is in the set of candidate controllers. In this work, a PID controller structure was adapted online for a nonlinear nonminimum phase continuous stirred tank reactor (CSTR) model. A better and faster performance was achieved via this technique when comparing it with a fixed non-adaptive PID controller. In summary, the proposed technique is suitable if a plant model is too difficult or too expensive to derive, e.g. for bio-chemical processes. The combination of unfalsified control and online optimization is enabled by the introduction of a new cost function based upon a new fictitious error signal. Research on the following topics is suggested to further develop the new adaptive control scheme: 1. Analysis of the effect of deterministic disturbances on the adaptation. 2. Automatic detection of the points in time where the set of controllers should be adapted and monitoring of the sufficiency of the excitation of the plant dynamics. 3. Monitoring of the robustness of the controllers. 4. Automatic setting of the parameters of the algorithm. 5. Extension to multivariable problems. In principle, this is straight forward, however the computation of the controller performance requires a sufficient excitation of all channels of the plant. References [1] M. Safonov, T. Tsao, The unfalsified control concept and learning, IEEE Transactions on Automatic Control 42 (6) (1997) 843–847. [2] S. Labiod, T. Guerra, Direct adaptive fuzzy control of a class of MIMO nonlinear systems, International Journal of Systems Science 38 (8) (2007) 665–675. [3] K. Klatt, S. Engell, Gain-scheduling trajectory control of a continuous stirred tank reactor, Computers & Chemical Engineering 22 (1998) 491–502. [4] K. Tsakalis, P. Loannou, Linear Time Varying Systems: Control and Adaptation, Prentice-Hall, New Jersey, 1993. [5] M. Jun, M. Safonov, Automatic PID tuning: an application of unfalsified control, in: Proc. IEEE International Symposium on Computer Aided Control System Design, Hawaii, 1999. [6] S. Engell, T. Tometzki, T. Wonghong, A new approach to adaptive unfalsified control, in: Proc. European Control Conf., Kos, 2007, pp. 1328–1333. [7] C. Mauelli, S. Cheong, E. Mosca, M. Safonov, Stability of unfalsified adaptive control with non SCLI controllers and related performance under different prior knowledge, in: Proc. European Control Conf., Kos, 2007, pp. 702–708. [8] A. Dehghani, B. Anderson, A. Lanzon, Unfalsified adaptive control: a new controller implementation and some remarks, in: Proc. European Control Conf., Kos, 2007, pp. 709–716. [9] A. Morse, D. Mayne, G. Goodwin, Application of hysteresis switching in parameter adaptive control, IEEE Transactions on Automatic Control 37 (9) (1992) 1343–1354.
T. Wonghong, S. Engell / Journal of Process Control 22 (2012) 2008–2025 [10] R. Wang, A. Paul, M. Stefnovic, M. Safonov, Cost-detectability and stability of adaptive control systems, in: Proc. IEEE CDC-ECC Conf., Seville, 2005, pp. 3584–3589. [11] T. Wonghong, S. Engell, Application of a new scheme for adaptive unfalsified control to a CSTR, in: Proc. 17th IFAC World Congress, Seoul, 2008, pp. 13247–13252. [12] T. Wonghong, S. Engell, Application of a new scheme for adaptive unfalsified control to a CSTR with noisy measurements, in: Proc. International Symposium on Advanced Control of Chemical Processes (ADCHEM), Istanbul, 2009. [13] T. Wonghong, S. Engell, Real-time PI controller tunning via unfalsified control, in: Proc. IEEE Multi-conference on Systems and Control, Denver, 2011. [14] T. Wonghong, Adaptation by evolutionary algorithms in unfalsified control, Ph.D. Thesis, Technische Universität Dortmund, 2010. [15] I. Rechenberg, The evolution strategy. A mathematical model for darwinian evolution, in: F. Frehland (Ed.), Synergetics - From Microscopic to Macroscopic Order, Springer, Berlin, 1984, pp. 122–132. [16] H. Schwefel, Evolution and Optimum Seeking, John Wiley & Sons, New York, 1995. [17] T. Bäck, Evolutionary Algorithms in Theory and Practice: Evolution Strategies Evolutionary Programming Genetic Algorithms, Oxford University Press, New York, 1996. [18] J. Holland, Adaptation in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, 1975.
2025
[19] D. Goldberg, Genetic Algorithms in Search Optimization and Machine Learning, Addison-Wesley, Reading, 1989. [20] L. Fogel, A. Owen, M. Walsh, Artificial Intelligence through Simulated Evolution, Wiley, New York, 1966. [21] I. Rechenberg, Cybernetics solution path of an experimental problem, Royal Aircraft Establishment Library Translation 1122. [22] H. Schwefel, Evolutionsstrategie und numerische optimierung, Ph.D. Thesis, Technische Universität Berlin, 1975. [23] A. Eiben, J. Smith, Introduction to Evolutionary Computing, Springer-Verlag, Berlin, 2003. [24] H. Schwefel, Collective phenomena in evolutionary systems, in: Preprints of the 31st Annual Meeting of the International Society for General System Research, vol. 2, Budapest, 1987, pp. 1025–1033. [25] H. Schwefel, G. Rudolph, Contemporary evolution strategies, in: Proc. 3rd CEAL Advances in Artificial Life, Springer, Berlin, 1995, pp. 893–907. [26] M. Vidyasagar, Control System Synthesis: A Factorization Approach, The MIT Press, Massachusetts, 1987. [27] S. Engell, K. Klatt, Nonlinear control of a non minimum-phase CSTR, in: Proc. American Control Conf., San Francisco, 2005, pp. 2941–2945. [28] G. Golub, C.V. Loan, An analysis of the total least squares problem, SIAM Journal on Numerical Analysis 17 (1980) 883–893. [29] I. Markovsky, S.V. Huffel, Overview of total least squares methods, Signal Processing 87 (2007) 2283–2302.
