Time-Domain Control Design a Non-Differentiable Approach 1

Time-Domain Control Design a Non-Differentiable Approach Pierre Apkarian

∗

Dominikus Noll

†

Alberto Sim˜oes

‡

Abstract We present a new method to compute locally optimal feedback controllers for synthesis problems formulated in the time-domain. We minimize a time-domain performance objective subject to state or input time-domain constraints. The possibility to include state or input constraints in the design is very appealing from a practical point of view, in particular for plants subject to input saturations or other operational limits. Our method is based on a nonsmooth minimization technique, which can handle time-domain constraints constraints as hard constraints. We include a variety of numerical tests to demonstrate the validity of our approach.

Keywords: Nonsmooth optimization, structured controllers, PID tuning, time-domain constraints, time response shaping, envelope constraints.

Introduction

1

In traditional frequency based feedback control design for linear time-invariant systems closedloop performance specifications (like for instance limited overshoot, short settling or rise times), cannot be addressed directly. Instead, experienced designers know how to handle these specifications heuristically by introducing suitable frequency domain performance channels, which are then optimized using classical loop-shaping design techniques or more recent H∞ or H2 synthesis methods. A more recent trend in feedback control - referred to as iterative feedback tuning (IFT) - tries to address time-domain constraints more directly using closed-loop system responses z(t) to specific test input signals w(t) such as steps, ramps or other inputs. The design technique so generated is closer in spirit to optimal control methods than the standard frequency-domain methods. In this paper we follow this new line and discuss state and control constraints for system response trajectories z(t) and u(t) to control not only overshoot, settling- and rise-time, but also actuator saturation and other operational limits on the system. In contrast with a considerable ∗

ONERA-CERT, Centre d’´etudes et de recherches de Toulouse, Control System Department, 2 av. Edouard Belin, 31055 Toulouse, France - and - Institut de Math´ematiques, Universit´e Paul Sabatier, Toulouse, France Email: [email protected] - Tel: +33 5.62.25.22.11 - Fax: +33 5.62.25.27.84 † Universit´e Paul Sabatier, Institut de Math´ematiques, 118, route de Narbonne, 31062 Toulouse, France - Email: [email protected] - Tel: +33 5.61.55.86.22 - Fax: +33 5.61.55.83.85. ‡ ONERA-CERT, Centre d’´etudes et de recherche de Toulouse, Control System Department, 2 av. Edouard Belin, 31055 Toulouse, France - Email: [email protected] - Fax: +33 5.62.25.27.64.

1

body of work in progress [11, 13, 18, 15], we do not follow the least squares approach, which handles time-domain specifications as soft constraints by penalizing L2 integrals of the constraint violation [17, 1]. Penalty methods are inconvenient because a suitable penalty parameter has to be specified, and because ill-conditioning will often occur. Instead, we privilege a nonsmooth optimization technique, which may be seen as an exact penalization approach. This allows to handle time-domain specifications as hard constraints, for instance by minimizing the maximum constraint violation. For hard constraints our approach dispenses with the management of penalty parameters. In exchange, the new method is more demanding algorithmically, because it uses less standard nonsmooth optimization techniques, which are as a rule first-order methods. Our approach is reviewed in section 3. Setting up subgradient information in specific applications is explained in section 3.1. In section 5, we discuss several challenging applications, including control of a system with input amplitude and rate limitations, control of the large Brasilian power plant network, and model-free re-tuning of a fixed-structure controller when only experimental data are available to the designer.

Notation Let Rn×m be the space of n×m matrices, equipped with the corresponding scalar product hX, Y i = Tr(X T Y ), where X T is the transpose of the matrix X, Tr (X) its trace. Given an operator T , T ∗ is used to denote its adjoint operator on the appropriate space. The notation vec applied to a matrix stands for its usual column-wise vectorization. We use concepts from nonsmooth analysis covered by [9]. For a locally Lipschitz function f : Rn → R, ∂f (x) denotes its Clarke subdifferential at x while f ′ (x; h) stand for its directional derivative at x in the direction h. For functions of two variables f (x, y), ∂1 f (x, y) will denote the Clarke subdifferential with respect to the first variable. For differentiable functions f of two variables x and y the notation ∇x f (x, y) stands for the gradient with respect to the first variable.

2

Structured controllers synthesis in time-domain z

w P

y

u

K(κ)

Figure 1: Standard interconnection Consider a plant P in state-space form    x˙ A    P (s) : z = C1 y C2 2

B1 D11 D21

  B2 x   D12 w , D22 u

(1)

where x ∈ Rn is the state vector of P , u ∈ Rm2 the vector of control inputs, w ∈ Rm1 is a test signal, y ∈ Rp2 the vector of measurements and z ∈ Rp1 the controlled or performance vector. Without loss, it is assumed throughout that D22 = 0. We consider control laws of the form u = K(s)y with state-space realization K(s) = CK (sI − AK )−1 BK + DK ,

AK ∈ Rk×k ,

(2)

where k is the order of the controller, and where the case k = 0 of a static controller K(s) = DK is included. In order to unify the subgradient computations during what follows, we develop the formulas for static controllers. Formulas for dynamic controllers are then obtained by prior dynamic augmentation of the plant P (s), a standard procedure, where one substitutes:     A 0 B1 A→ , B1 → , etc. 0 0k 0 so that the dynamic controller for P (s) becomes a static controller   AK BK K := ∈ R(k+m2 )×(k+p2 ) . C K DK

(3)

for the augmented system. This allows to unify the setup notationally and is also favourable for the implementation. With this preparation, the state-space data of the closed-loop system in figure 1 are described as       A(K) B(K) A B1 B2 := + K [ C2 D21 ] . (4) C(K) D(K) C1 D11 D12 Structural constraints on the controller may now be defined by a matrix-valued mapping K(·) from a parameter space Rq to R(k+m2 )×(k+p2 ) . That is K = K(κ), where κ ∈ Rq denotes the independent variables in the controller parameter space Rq . For the time being we will consider free variation κ ∈ Rq , but the reader will be easily convinced that parameter restrictions under the form of mathematical programming constraints gI (κ) ≤ 0, gE (κ) = 0 could be added if need be. We will assume throughout that the mapping K(.) is continuously differentiable, but otherwise arbitrary. The reader is referred to [2] for some examples. The focus is on time-domain synthesis with structured controllers K(κ) for the plant in (1). That means we want to achieve the following: Find κ ∈ Rq such that • Internal stability: K(κ) stabilizes the original plant P (s) in closed-loop. • Performance: For all stabilizing controllers K(κ) with that structure, the closed-loop time response z(κ, t) to an input test signal w(t) with controller K(κ) satisfies the envelope constraints zi,min (t) ≤ zi (κ, t) ≤ zi,max (t), ∀t ≥ 0, i = 1, . . . , p1 . (5) There exist various optimization strategies to handle the specifications in (5). Consider for instance a partition of I := {1, . . . , p1 } into disjoint subsets S and H, i.e., I = S ∪ H, S ∩ H = ∅, where we think of S as the soft constraints, H the hard constraints. A possible form of the program is now minimize q κ∈R

maxi∈S maxt≥0 max {[zi (κ, t) − zi,max (t)]+ , [zi,min (t) − zi (κ, t)]+ }

subject to maxt≥0 zi (κ, t) − zi,max (t) ≤ 0, i ∈ H , maxt≥0 zi,min (t) − zi (κ, t) ≤ 0, i ∈ H , 3

(6)

where [.]+ denotes the threshold function [x]+ = max{0, x}. On the other hand when specifications i ∈ I are equally important, it is more natural to use the simpler cast max max max {[zi (κ, t) − zi,max (t)]+ , [zi,min (t) − zi (κ, t)]+ } . minimize q κ∈R

i∈I

t≥0

(7)

Note that in (6) and (7) we minimize a nonsmooth and semi-infinite objective function. Similarly, the constraint in (6) is nonsmooth and semi-infinite. Programs with such data do not fall within the range of traditional mathematical programming, and tailored optimization techniques must be developed. This is discussed in section 3. Programs (6) and (7) rely on simulated or possibly experimental data z(t), and the nature of the test signals w(t) should be specified at this stage. In tracking or set-point following problems w(t) will be a step input. Yet, the approach offers the flexibility to incorporate any deterministic input of practical interest such as ramps, sinusoids, stair sequences, etc. It is also possible tu use several input signals simultaneously. The constraints in (5) with upper and lower envelopes define in some sense templates for the shaping of the closed-loop response z(t). A typical case is illustrated in figure 2, where envelope or shape constraints on overshoot, damping, rise-time, settling-time and steady-state accuracy are imposed on the closed-loop response z(t). It is also possible and useful to formulate amplitude and rate constraints for the control signal u(t). See our discussion in section 4 and our illustration in the experimental section 5. z(t)

t

Figure 2: Shape-constraints on step response

3

Nonsmooth minimization technique

In this section we give a concise presentation of our optimization method. For a more detailed discussion we refer the reader to [3, 20], and to [19] for variations of the present technique. Introducing the notations fi (κ, t) := max{[zi (κ, t) − zi,max (t)]+ , [zi,min (t) − zi (κ, t)]+ }, i ∈ S and gi (κ, t) := max{[zi (κ, t) − zi,max (t)], [zi,min (t) − zi (κ, t)]}, i ∈ H, 4

program (6) takes the more familiar form minimize

f (κ)

κ

subject to g(κ) ≤ 0 ,

(8)

with f (κ) := maxt≥0 maxi∈S fi (κ, t) and g(κ) := maxt≥0 maxi∈H gi (κ, t). Following an idea in [20, 5], we introduce the so-called progress function for (8): F (κ+ , κ) = max{f (κ+ ) − f (κ) − µg(κ)+ ; g(κ+ ) − g(κ)+ },

(9)

where µ > 0 is some fixed parameter (with µ = 1 a typical value). We think of κ as the current iterate, κ+ as the next iterate or as a candidate to become the next iterate. The following properties of the progress function (9) are crucial for the understanding of our method. For a proof we refer to [5]. Lemma 1 a) Suppose κ ¯ is a local minimum of program (8), then κ ¯ is also a local minimum of F (·, κ ¯ ). In particular, this implies 0 ∈ ∂1 F (¯ κ, κ ¯ ). b) If κ ¯ satisfies the F. John necessary optimality conditions for (8), then 0 ∈ ∂1 F (¯ κ, κ ¯ ). c) Conversely, if 0 ∈ ∂1 F (¯ κ, κ ¯ ), then κ ¯ either satisfies the F. John necessary optimality condiitons for (8), or it is a critical point of constraint violation. Remark. A point κ ¯ ∈ Rq is called a critical point of constraint violation of (8) if 0 ∈ ∂1 F (¯ κ, κ ¯ ), but the constraint is violated, g(¯ κ) > 0. In the same vein, κ ¯ is called a local minimum of constraint violation if κ ¯ is a local minimum of F (·, κ ¯ ), but satisfies g(¯ κ) > 0. Such points mean failure to solve the given program. Moreover, in the neighborhood of such a point no further progess is possible, so a restart of the algorithm with a different initial seed is mandatory. Remark. The above lemma shows that we must nonetheless search for points κ ¯ satisfying 0 ∈ ∂1 F (¯ κ, κ ¯ ), because this is a necessary condition for a local minimum of (8). Excluding practically rare cases where κ ¯ is a critical point of the constraint violation g(¯ κ) ≥ 0, critical points κ ¯ of F (·, κ ¯) will also be critical points of the original program (8). Remark. From a practical point of view it is noteworthy to observe that a F. John critical point for (6) will, as a rule, be a KKT point. Indeed, Let κ ¯ be F. John critical, then there exist (σ, τ ) 6= (0, 0), σ ≥ 0, τ ≥ 0 such that 0 ∈ σ∂f (¯ κ) + τ ∂g(¯ κ) and g(¯ κ) ≤ 0, τ g(¯ κ) = 0. Then either σ > 0, in which case we can without loss take σ = 1, which says that κ ¯ is a KKT-point. Or we have σ = 0, in which case the remaining conditions say 0 ∈ ∂g(¯ κ) and g(¯ κ) = 0. In other words, we have a critical point of g just on the border of the feasible region, a highly unlikely event. Remark. Approximating a point κ ¯ with 0 ∈ ∂1 F (¯ κ, κ ¯ ) is based on an iterative procedure. Suppose the current iterate κ is such that 0 6∈ ∂1 F (κ, κ). Then it is possible to reduce the function F (·, κ) is a neighborhood of κ, that is, to find κ+ such that F (κ+ , κ) < F (κ, κ). Replacing κ by κ+ , we repeat the procedure. Unless 0 ∈ ∂1 F (κ+ , κ+ ), in which case we are done, it is possible to find κ++ such that F (κ++ , κ+ ) < F (κ+ , κ+ ), etc. The sequence κ, κ+ , κ++ , . . . so generated is expected to converge to the sought local minimum κ ¯ of (8). Remark. Finding the descent step κ+ away from the current κ is based on solving the tangent program at κ. Its name is derived from the fact that a first-order approximation Fb(·, κ) of F (·, κ) is built, which provides a descent direction dκ at κ, that is, d1 F (κ, κ; dκ) < 0, where d1 F denotes 5

the directional derivative of F (·, κ) at κ in direction dκ. The next iterate is then κ+ = κ + dκ, or possibly κ+ = κ + αdκ for a suitable stepsize α ∈ (0, 1) found by a backtracking line search. Remark. The choice of the progress function in (9) leads to a so-called phase I/phase II method. Indeed, as long as the current iterate κ is infeasibe, g(κ) > 0, we have g(κ)+ > 0 and the second term of the max function in (9) is dominant at κ+ = κ, because of −µg(κ)+ < 0 for the first term. Reducing F (·, κ) then amounts to reducing constraint violation, which represents phase I of the method. The choice of the constant µ > 0 may have an influence on the behavior of the method in phase I, but has been fixed to µ = 1 in our implementation. As soon as one feasible iterate κ is found, phase I terminates successfully, and phase II begins. Indeed, with κ feasible we have g(κ)+ = 0, and the next iterate κ+ (and therefore all consecutive iterates) will now stay strictly feasible. This is because F (κ, κ) = 0, while g(κ)−g(κ)+ = g(κ) ≤ 0, so that now the left hand branch in (9) is active at κ. Since the new κ+ has F (κ+ , κ) < F (κ, κ) = 0, we necessarily have g(κ+ ) = g(κ+ ) − g(κ)+ ≤ F (κ+ , κ) < 0, which means κ+ is strictly feasible. Moreover, that also means that in phase II the objective is minimized, because now f (κ+ ) − f (κ) − µg(κ)+ = f (κ+ ) − f (κ) ≤ F (κ+ , κ) < 0. This is because the term µg(κ)+ vanishes in phase II, so µ plays no role here. Let us now discuss the tangent program. In order to generate a first-order approximation b F (·, κ) of F (., κ) around κ, we need the set of active times for f : Tf (κ) := {t ≥ 0 : ∃i ∈ S, fi (κ, t) = f (κ)}.

Similarly, we define active times for g as the set Tg (κ) := {t ≥ 0 : ∃i ∈ H, gi (κ, t) = g(κ) > 0}. Notice that as a rule the sets Tf (κ) and Tg (κ) are finite or discrete, but it may happen that they contain contact intervals I = [t, t] with t < t and zi (κ, t) = zi,max (κ, t) for all t ∈ [t, t]. On such an interval the max-function is necessarily flat, i.e. ∇zi (κ, t) = ∇zi,max (κ, t) for all t ∈ I, giving ∇fi (κ, t) = 0 for t ∈ I. This means that even for infinite sets Tf (κ) and Tg (κ) the information required for gradient computations is finitely representable. Let us consider the practically important case where the active sets are finite. It obviously suffices to consider the case where f (κ) > 0, because for f (κ) = 0 there is nothing to optimize. As the active sets may be small, we consider finite extensions Tfe and Tge of the sets Tf and Tg , respectively. The idea here is that enriched sets capture more information on the closed-loop responses which results in a better tangent model. The proposed technique offers great flexibility to build such extensions, while guaranteeing convergence [5]. A general characterization is the following ∀t ∈ Tfe , ∃i ∈ S, fi (κ, t) > 0 . For all such t, the functions fi are differentiable in a neighborhood of κ. Indeed, fi (κ, t) = max{zi (κ, t) − zi,max (t), zi,min (t) − zi (κ, t), 0}, and only one component is active in this expression. We have  ∇κ zi (κ, t) if zi (κ, t) > zi,max (t) ∇κ fi (κ, t) = −∇κ zi (κ, t) if zi,min (t) > zi (κ, t) 6

For all t ∈ Tfe , we collect all pairs (φf , Φf ) := (fi (κ, t), ∇κ fi (κ, t)) and denote this finite set as Wf . A similar procedure is applied to the constraint functions gi . Times in Tge are characterized as ∀t ∈ Tge , ∃i ∈ H, gi(κ, t) > 0 . We deduce ∇κ gi (κ, t) =



∇κ zi (κ, t) −∇κ zi (κ, t)

if zi (κ, t) > zi,max (t) if zi,min (t) > zi (κ, t)

For all t ∈ Tge , we collect all pairs (φg , Φg ) := (gi (κ, t), ∇κ gi (κ, t)) and denote this finite set as Wg . With this preparation, a first-order approximation is obtained as   T T b F (κ + h, κ) := max max φf − f (κ) − µg(κ)+ + Φf h, max φg − g(κ)+ + Φg h , (φg ,Φg )∈Wg

(φf ,Φf )∈Wf

where h is the displacement in the controller parameter space Rq . Indeed, for an active index i the first-order approximation is fi (κ + h, t) ≈ fi (κ, t) + ∇κ fi (κ, t)T h, which leads to the term φf + ΦTf h on the left hand branch of Fb, and similarly for g and its branches on the right. This gives the tangent program minimize Fb(κ + h, κ) + 2δ khk2 . (10) q h∈R

Notice that the terms in Fb on the left of (10) are linear in the decision variable, so after getting rid of the finite max-operator, program (10) can be turned into a standard convex quadratic program (CQP), and can be efficiently solved using currently available codes. Current stateof-the-art CQP codes solve problems involving several hundreds of variables and constraints in less than a second. Note that the quadratic term in (10) can be used to capture second-order information, or it may be interpreted as a trust region radius management parameter. We refer the reader to [4, 21, 19, 5] for more details on the procedure, and to Polak [20] for a general view on phase I/phase II methods. The key facts about (10) have been established in [3] and we state them here without proof: • Whenever the solution h of (10) is nonzero, h is a descent direction of F (., κ) at κ. On the other hand, if the solution is h = 0, then 0 ∈ ∂1 F (κ, κ), and we are done. In particular, a stopping test may be based on the solution of the tangent program. • The direction h can be used in an Armijo line search defined as F (κ + α h, κ) − F (κ, κ) < γαF ′(., κ)(κ; h), where 0 < γ < 1, which terminates after finitely many steplength trials α ∈ (0, 1]. Here in both items we use the fact that ∂1 Fb(κ, κ) = ∂1 F (κ, κ).

Remark. Note that tangent program (10) operates in the space of controller parameter Rq , which we call the primal space. When q is greater than #Tfe + #Tge , it may be beneficial to work in the dual space. This is obtained by dualizing the convex QP in (10). We refer the reader to [2] for details. Having described the main features of our algorithm, we can state the pseudo-code: 7

Algorithm 1 Nonsmooth algorithm for program (6) Parameters: δ > 0, 0 < β, γ < 1. 1: Initialize. Compute a closed-loop stabilizing K(κ1 ). Put counter j = 1. b(κj , κj ) and return κj . Otherwise continue. 2: Stopping test. At counter j, stop if 0 ∈ ∂1 F 3: Compute descent direction. At counter j solve tangent program min Fb(κj + h, κj ) + 2δ khk2 .

h∈Rq

Solution is the search direction h. 4: Line search. Find α = β ν , ν ∈ N, satisfying the Armijo condition F (κj + αh, κj ) − F (κj , κj ) ≤ γαF ′ (·, κj )(κj , h) < 0. 5:

Update. Put κj+1 = κj + αh, increase counter j by 1 and loop back to step 2.

3.1

Simulated gradient computations

In order to make our conceptual algorithm more concrete, we need to clarify how gradients in (10) can be obtained. We first describe how gradients with respect to an arbitrary element Kij are computed and then obtain gradients with respect to the parameter κ via chain rules [10, 9]. Notice that the functions fi (κ, t) are differentiable with respect to κ except at points where several branches are active. Due to the special structure this only happens at the value 0, which is of no interest for optimization. In consequence f , even though nonsmooth, has the very favorable structure of a finite max-function. A similar argument applies to g. We note first that all signals in (1) are differentiable with respect to controller entries. Differentiating the state-space equations (1) with respect to Kij , we get the adjoint system of equations     

˙ (K, t) ∂x ∂Kij ∂z (K, t) ∂Kij ∂y (K, t) ∂Kij

∂x ∂u = A ∂K (K, t) + B2 ∂K (K, t) ij ij ∂x ∂u = C1 ∂Kij (K, t) + D12 ∂Kij (K, t) ∂x = C2 ∂K (K, t) ij

(11)

∂y ∂K = ∂K (K, t)y(K, t) + K ∂K (K, t) ij ij ∂y = yj (K, t)ei + K ∂Kij (K, t)

(12)

controlled by ∂u (K, t) ∂Kij

where ei stands for the i-th vector of the canonical basis of Rm2 . It follows that the partial ∂z (K, t) is the simulated output of the interconnection in figure derivative of the output signal ∂K ij 3, where the exogenous input w is held at 0, and the vector yj (K, t)ei is added to the controller output signal. We readily infer that at most (m2 + k) × (p2 + k) simulations are required in order to form the sought gradients. 8

0

P

∂z ∂Kij

∂y ∂Kij

∂u ∂Kij +

K

+

yj ei Figure 3: Interconnection for gradient computation

Often the computational burden associated with the gradients computation can be substantially reduced by keeping track of entries that are truly dependent on the parameter κ. For strongly structured controllers only a few entries in K(κ) are varying and thus fewer simulations are required. i , gradients with respect to the parameter κ are readily obtained Finally, given gradients ∂z ∂K ′ ∗ ∂zi using the chain rule K (κ) ∂K where K′ (κ)∗ is the adjoint of K′ (κ) and acts on elements of F ∈ R(m2 +k)×(p2 +k) using the formula ′

∗

K (κ) F =

h

T T Tr ( ∂K(κ) F) F ), . . . , Tr ( ∂K(κ) ∂κ1 ∂κq

iT

.

This yields the implementable form of the gradients  ∂K vec T ∂κ 1 ∂zi (t) ∂zi (t)   .. = ,  vec . ∂κ ∂K T ∂K vec ∂κq 

i = 1, . . . , p1 , t ≥ 0 .

The propose nonsmooth technique requires two essential ingredients at every iteration: the performance vector z(t) and its partial derivatives ∂z/∂Kij (t). These closed-loop responses can be generated from a state-space model (1). This is often referred to as the model-based approach. Yet, they can also be obtained from experiments carried out online with the real system, see figures 1 and 3, hence leading to a more realistic design. The latter option is often referred to as the model-free approach and is currently known as the IFT method [16]. In the application section of this paper, we investigate a model-free design where a model-based controller is retuned against noisy experimental data emulated through a more complex model with noisy inputs. According to our discussion above, both the proposed nonsmooth method and the IFT method become costly as soon as the the number of controller parameters gets sizable. For simpler SISO controllers, however, it has been shown in [14] that only one experiment is required for gradient computation no matter the order and structure of the controller. A similar analysis holds for the current technique and can be exploited to speed-up computations. 9

4

Control design with input amplitude and rate constraints

In this section we discuss a specific instance of synthesis with time-domain constraints, where rate and amplitude constraints are imposed on the control input. Such constraints arise regularly in practical designs, and this has generated intensive research in the past decade. Our approach differs from anti-windup schemes and is closer in spirit to the saturation avoidance philosophy. Admittedly with some sacrifice of performance, we try to keep signals at levels where the system dynamics remain linear. A standard trick to handle input amplitude and rate constraints simultaneously is to augment the plant with inputs as new states. This is shown schematically in figure 4.

z

w u˙

P

u

I s

y K(κ)

Figure 4: Augmentation of standard form The new control signal is now u, ˙ whereas u has become a state of the augmented plant. A state-space representation of the augmented standard form is readily obtained as      

x˙ u˙ z y u





    =    

A B2 B1 0 0 0 C1 D12 D11 C2 D22 D21 0 0 I

0 I 0 0 0



   x  u     w  ,  u˙

(13)

and synthesis with input amplitude and rate constraint reduces to a problem with amplitude constraint on the control signal u˙ and a state constraint on u. So far we have implicitly assumed that input u is measured. If this is not the case, the last row in (13) should be removed. Notice that the standard form (13) yields a control law of the form   y u˙ = [Ky Ku ] u with state-space realization 

x˙ K u˙



=



AK CK

BK,y DK,y 10

   xK BK,u  y . DK,u u

The original sought control law is then recovered under the form u = (sI − Ku (s))−1 Ky (s)y and has state-space realization      xK AK BK,u BK,y x˙ K  u˙  =  CK DK,u DK,y   u  . u 0 I 0 y On the other hand when u is not measured, we get u˙ = Ky (s)y, hence u = 1s Ky (s)y, which admits the state-space realization      x˙ K xK AK 0 BK,y  u˙  =  CK 0 DK,y   u  . u 0 I 0 y Note that in both cases, the resulting controller structure can be viewed as a ”filtered” version of the gain Ky (s). This is the slowing down effect due to amplitude and rate restrictions. We see that the structure of compensator Ky (s) is slightly altered in this formulation.

5

Applications

In this section, we illustrate our nonsmooth technique on a variety of examples.

5.1

Simple step following problem

We start our experiments with a simple step following problem borrowed from [12]. Consider a standard negative feedback interconnection of the plant G(s) and controller K(s), figure 5. For the purpose of comparison we do not use a prefilter in this preliminary study, i.e. F (s) = I. s+0.5 and must follow a step reference command with The plant has transfer function G(s) = s(s−2) minimum overshoot.

r

F (s)

+ −

K(s)

u

G(s)

y

Figure 5: Standard interconnection ???? ajouter w = r, et une fleche pour z = e dans le dessin ????? Time-domain constraints are formulated as −0.1 ≤ y(t) ≤ 1.1, 0.98 ≤ y(t) ≤ 1.02, 0.999 ≤ y(t) ≤ 1.001,

0 ≤ t ≤ 4, 4 ≤ t ≤ 10, t > 10 ,

(14)

and specify a settling-time of 4 seconds with a worst-case overshoot of 10%. These envelope constraints are drawn in figure 6. In a first study, the search is restricted to second-order controllers. 11

An initial stabilizing controller of order 2 is computed from scratch using our nonsmooth technique in [6]. The step response is then optimized to meet the above constraints using the technique of section 3. Figure 6 shows the system step response y(t) resulting from the first iteration of the nonsmooth technique. As can be seen, at this early stage the constraint is still violated, and we display times (’∗’ symbol in red) where this happens. These times t are selected to build the tangent subproblem for computation of the descent direction. As the algorithm loops on, the step response evolves and several iterations are shown in light-grey in figure 7. The solid line (in blue) in that figure is the final response, which meets all template constraints (14). A globally optimal solution has thus been obtained for problem (7) with zero value of the cost function and the associated second-order controller is described as

K(s) =

25.73s2 + 171.8s + 265.9 . s2 + 7.439s + 12.2

These preliminary results suggest that a simpler controller could perform just as well. Comparisons are given in figure 8 for a second- and a first-order controller K(s) = 68.04s+602 with the results in s+8.327 [12] with controller 12.27s3 + 75.82s2 + 183.3s + 240 . s3 + 4.73s2 + 12.5s + 28.82

1.8

1.8

1.6

1.6

1.4

1.4

1.2

1.2 step responses

step response

K(s) =

1 0.8 0.6

1 0.8 0.6

0.4

0.4

0.2

0.2

0 −0.2 0

0 5

10 15 time (sec)

20

−0.2 0

25

Figure 6: First iteration of nonsmooth method. Stars ’*’ indicate times where constraint is still violated.

5

10 15 time (sec)

20

25

Figure 7: Evolution of step responses during minimization. Final result in blue satisfies the constraints. 12

1.4 1.2

step responses

1 0.8 0.6 0.4 0.2 0 0

2

4

6

8

10

time (sec)

Figure 8: Comparison of step-responses. Solid red: optimal 2nd-order controller. Dashed-dotted green: optimal 1st-order controller, Dashed blue: 3rd-order (suboptimal) controller from [12].

5.2

Step following problem with input amplitude and rate constraints

We continue with the same example, but consider a more realistic set-up, where the step following problem is combined with a constraint on the input amplitude and with rate constraints. ??????????? This is where the LMI heuristic from [12] drops out with drums drumming and pipes piping, as it can not even accomodate this slight extension. ??????????? This is easily formulated via (6) according to our discussion in section 4. In order to avoid injecting pure step commands, which may result in unduly conservative designs when rate restrictions are present, 1 as shown in figure 5. We solve 3 different problems with control we use a prefilter F (s) = 0.3s+1 constraints |u(t)| ≤ 20, |u(t)| ˙ ≤ β = 10, 20, 30 . The associated controllers are all computed using the proposed nonsmooth method and are described as follows: Kβ=10 = Kβ=20 =

s2 s2

s + 13.72 30.89s + 467.5 30.89s + 467.5 = 2 + 18.1s + 91.81 s + 13.72 s + 18.1s + 91.81

s + 21.36 51.99s + 1324 51.99s + 1324 = 2 + 19.01s + 129.8 s + 21.36 s + 19.01s + 129.8

s + 63.53 102.7s + 7685 102.7s + 7685 = 2 + 60.46s + 415.3 s + 63.53 s + 60.46s + 415.3 Notice that in the second terms of these expressions, we explicit report the particular controller structure which springs from the input rate saturation, according to our discussion in section 4. The slowing down effect caused by the input rate saturation appears more pronounced as the constraints are more stringent. The corresponding time-domain simulations including step responses, control signal u(t) and control input derivative u(t) ˙ are presented in figures 9 to 17. Kβ=30 =

s2

13

We observe that when the input rate constraint is more severe, β = 10 and 20, the computed controllers do not meet the shape constraints. In those cases, the rate constraint is in some sense saturated ?????exhausted = fatigue ?????? in the transient part of the output response, and we have exactly maxt≥0 |u| ˙ = β. On the other hand, relaxing the constraint to β = 30 allows to satisfy all time-domain specifications. These results are corroborated by the frequency responses of each controller, since a noticeable reduction in bandwidth is observed when rate constraints are more demanding, see figure 18.

2

20

10 8

15 1.5

6

1

0.5

control input derivative

control input

step response

10 5 0 −5

4 2 0 −2 −4

−10 0

−6 −15

−0.5 0

2

4

6 time (sec)

8

10

−20 0

12

−8 2

4

6 time (sec)

8

10

−10 0

12

2

4

6 time (sec)

8

10

12

Figure 9: Step response with Figure 10: Control input u Figure 11: Control input |u| ˙ ≤ 10 with |u| ˙ ≤ 10 derivative u˙ with |u| ˙ ≤ 10

2

20

20

15

15

10

10

1

0.5

control input derivative

control input

step response

1.5

5 0 −5

5 0 −5

−10

−10

−15

−15

0

−0.5 0

2

4

6 time (sec)

8

10

−20 0

12

2

4

6 time (sec)

8

10

−20 0

12

2

4

6 time (sec)

8

10

12

Figure 12: Step response Figure 13: Control input u Figure 14: Control input with |u| ˙ ≤ 20 with |u| ˙ ≤ 20 derivative u˙ with |u| ˙ ≤ 20

2

20

30

15

20

1.5

1

0.5

control input derivative

control input

step response

10 5 0 −5

10

0

−10

−10 0

−20

−15 −0.5 0

2

4

6 time (sec)

8

10

12

−20 0

2

4

6 time (sec)

8

10

12

−30 0

2

4

6 time (sec)

8

10

12

Figure 15: Step response Figure 16: Control input u Figure 17: Control input with |u| ˙ ≤ 30 with |u| ˙ ≤ 30 derivative u˙ with |u| ˙ ≤ 30 14

Bode Diagram

30 20

Magnitude (dB)

10 0 −10 −20 −30 −1 10

0

10

1

2

10

10

3

10

Frequency (rad/sec)

Figure 18: Bode plots for all controllers solid (red): β = 30, dashed (blue): β = 20, dashed-dotted (green): β = 10

5.3

Power system oscillation damping

In the following example we discuss the control of a large dimension system, the oscillation damping of the power system presented in [22]. The system response oscillation is due mainly to a lightly-damped resonant mode, known as the NS (north-south) mode, which results from the interconnection of the Brazilian north and south sub-systems. In the closed-loop block diagram shown in figure 19, the measured and also the controlled output is the one-dimensional signal y and corresponds to the active power deviation, the control input u - also one-dimensional - represents the susceptance deviation. The disturbance w stands for the deviation in the mechanical power of a plant located at the north side of the interconnection.

Figure 20: Frequency response for Tw→y (solid: open-loop, dashed: final closed-loop)

Figure 19: Closed-loop system block diagram representation 15

We have used a model with 90 states corresponding to the worst-damped scenario in [22]. The NS mode presents a natural frequency of 1.08 rad/s and 3% damping, dominating the transient phase of the open-loop step response. This is confirmed by the magnitude of the frequency response for the open-loop transfer function Tw→y as displayed in figure 20.

This problem imposes some challenging design specifications. Firstly, from a performance perspective, the primary control objective is to guarantee oscillation damping with the lowest possible overshoot in the presence of disturbance. This must be achieved with a limited control effort deviation to avoid saturation of the Thyristor Controlled Series Compensator(TCSC) components. Secondly, a full-order controller is not acceptable, given the large dimension of the system. Hence, a reduced-order controller must be sought. Also, as typical in this kind of application, it is desirable that the controller possesses washout filtering properties to eliminate bias. With these specifications and constraints in mind, the controller structure was chosen of the form

K(s) =

s ˆ K(s), s+p

ˆ where K(s) is a 5th-order strictly proper transfer function, and the position of the real washout pole −p is also a decision variable of the optimization program. Altogether, this gives κ ∈ R36 .

The closed-loop step response and control input after the first iteration of the nonsmooth program are shown in figures 21 and 22. In order to achieve the desired level of oscillation damping, time-domain constraints were constructed as a piecewise constant approximation of a decaying exponential corresponding to 20% damping for the NS mode frequency. These shape constraints appear as step functions in figure 21. The exponential envelope has an offset equal to the asymptotic value of the open-loop step response, since the controller incorporates a washout filter. Regarding the control effort, constraints were introduced to limit the peak value, which avoids saturating TCSC components. In both figures we also display the times (indicated by the ’*’ symbols) where the response violates constraints either for the step response (left) or the control input (right). These times are used at the first iteration to compute the search direction of the nonsmooth algorithm. 16

Figure 21: Step response at first iteration

Figure 22: Control input at first iteration

Solid: closed-loop responses, dashed: constraints, *: selected violation times

The nonsmooth algorithm finds a feasible controller within 96 seconds cputime on a 2.8GHz Pentium processor with 1Gb RAM. By feasible we mean that the closed-loop step response satisfies both response and control input constraints, figures 23 and 24. As in the previous application, we note that the control input is saturated ???? exhausted = fatigu´e, ca ne va pas ??????, which suggests that better performance is not accessible unless the control input constraint is relaxed. The closed-loop transfer function Tw→y is drawn in figure 20 and the corresponding controller is obtained as

K(s) =

s(14.96s4 + 183.9s3 + 1.118e4s2 + 1.084e4s + 84.14) . s6 + 12.58s5 + 62.64s4 + 148.7s3 + 218.2s2 + 242.8s + 88.17 17

Figure 23: Final step response

Figure 24: Final control input

Solid: closed-loop responses, dashed: constraints

A careful examination of figure 23 (bottom left) reveals that the envelope constraints for the output response has been chosen to accommodate a low frequency oscillatory component. It turns out that this low frequency oscillation is caused by an almost uncontrollable mode with natural frequency of 0.26 rad/s. which generates a slowly oscillating behavior in the steady-state. The phenomenon is indicated by a dotted line in figure 23. By definition such a phenomenon cannot and should not be compensated by feedback. A practical design procedure should take the characteristics of the plant into account in order to avoid unrealistic solutions. The proposed design technique offers full flexibility in that respect.

5.4

Model-free design for a process with large dead time

In this section, the proposed technique is used first to synthesize a PID controller with a simple model of a process with large dead time. Next, the controller is retuned against experimental data emulated through a more complicate model with noisy inputs. Most design methods are model based and may perform poorly when confronted with the actual plant. An appealing feature of model-free approaches is that they rely only on experimental data and consequently inaccuracies in the mathematical model are no longer harmful. Moreover, the fact that there is no need to open the control loop is another attractive feature of IFT. A reasonable strategy appears to combine both model-based and model-free strategies. A first controller is computed using an identified model of the plant. If matching the result with experimental data turns out unacceptable, a model-free design is performed to further improve the controller. The benefits of the suggested procedure are twofold: • a complex accurate model is no longer needed, 18

• initializing the model-free design with a sensible controller should reduce the number of iterations in the tuning phase.

Figure 25: Block diagram of closed-loop system ????? ajouter w = r et une fleche pour z = e??????????? The process we consider to emulate experimental data is taken from [8] and is described by y (s) =

3 7 −6s e u (s) + v, 3s + 1 s + 7

(15)

where v is white Gaussian noise with zero mean and variance σ 2 = 0.01. The true dynamics (15) of the process are supposed unknown, and will be used solely as a black-box to generate experimental data of the real system during the re-tuning phase. For the model-based synthesis, a simple model of the process is constructed as a first-order transfer function in series with a 2nd-order Pad´e approximation for the dead time. The steadystate gain and bandwidth of the system are accurately modeled, but the dead-time has been underestimated to 5 seconds:  2  3 s − 1.2s + 0.48 y (s) = u (s) . (16) 3s + 1 s2 + 1.2s + 0.48 We are seeking a PID controller K (s) = Kp +

Ki Kd s + s 1 + εs

with the classical feedback interconnection shown in figure 25. Parameters for the initial controller K0 (s) are chosen as Kp = 0.09, Ki = 0.02, Kd = 0.01 and ε = 1. Figure 26 shows the closed-loop system responses and control signal for model (16) with controller K0 . Figure 26 also displays the time constraints of the desired closed-loop response that are imposed during the model-based synthesis. Note that the process dead time is easily captured by defining appropriate templates. A control effort constraint is also introduced, although it remains inactive in the absence of noise. Also shown are the closed-loop responses for a model-based controller K1 (s) computed using the nonsmooth technique. This controller meets all time-domain constraints, and is described by Kp = 0.199, Ki = 0.045468, Kd = 0.22304 and ε = 1.0507. In the next step, controller K1 (s) is tested with the true noisy process (15). The corresponding responses are shown in figure 27. Due to the discrepancy between model (16) and the true process (15), K1 (s) performs poorly. Compare figures 27.(a) and 26.(a). This leads us to re-tune the controller using experimental data and identical time constraints. Note that with such SISO controllers only 2 experiments are required at each iteration. The 19

14

6

12

5 10

4

y

u

8

6

3

4

2 2

1 0

−2

0

5

10

15

20

25

30

35

40

45

0

50

0

5

10

15

20

25

30

35

40

45

50

time (sec)

time (sec)

(a) (b) Figure 26: Closed-loop responses with model (16) (solid: final controller K1 , dash-dot: initial controller K0 ) ????????????? what is left, what is right ??????????????????? 12

6 10

5 8

4

y

u

6

3 4

2 2

1 0

0

5

10

15

20

25

30

35

40

45

0

50

0

5

10

15

20

25

30

35

40

45

50

time (sec)

time (sec)

(a) (b) Figure 27: Closed-loop responses with the real process and model-based controller K1 (s) 12

6 10

5 8

4

y

u

6

3 4

2 2

1

0

−2

0

5

10

15

20

25

30

35

40

45

0

50

0

5

10

15

20

25

30

35

40

45

50

time (sec)

time (sec)

(a) (b) Figure 28: Closed-loop responses with true plant and model-free controller model-free synthesis automatically adjusts to the true time-delay and no further information is required. The final PID parameters are obtained as Kp = 0.1994, Ki = 0.0395, Kd = 0.2872 and 20

ε = 0.7274. The initial overshoot has been significantly reduced, see Figure 28.(a). The specified control effort constraint becomes active, as can be seen from Figure 28.(b). The final constraints violation is γ = 0.27495.

6

Conclusion

We have described a nonsmooth algorithm to compute locally optimal solutions to constrained time-domain synthesis problems. The approach is flexible as it applies to many different scenarios and can capture almost any controller structure of practical interest. The proposed technique expands on our previous results which were restricted to minimization of a single cost function [7]. The possibility to include state or input constraints is very appealing from a practical point of view, particularly so for plants subject to control input saturations or to other operational limits. We have illustrated our nonsmooth design method on a variety of examples including step following problems with and without input constraints, damping control of a large dimension power system, and in re-tuning a fixed-structure controller through a model-free design when solely experimental data are available to the designer. These encouraging results already point to the more challenging goal where time- and frequencydomain constraints are combined. This will be the object of future research.

Ackowledgements The authors acknowledge financial support from the Fondation d’entreprise EADS (under contract Solving challenging problems in feedback control), from the Agence Nationale de Recherche (ANR) under project NT05 Guidage and from Agence Nationale de Recherche (ANR) under projet Controvert.

References [1] M. Akerblad, A. Hansson, and B. Wahlberg, Automatic tuning for classical stepresponse specifications using iterative feedback tuning, in Proc. IEEE Conf. on Decision and Control, Sydney, NSW, Australia, 2000, pp. 3347–3348. [2] P. Apkarian, V. Bompart, and D. Noll, Nonsmooth structured control design with application to PID loop-shaping of a process, Int. J. Robust and Nonlinear Control, 17 (2007), pp. 1320–1342. [3] P. Apkarian and D. Noll, Nonsmooth H∞ synthesis, IEEE Trans. Aut. Control, 51 (2006), pp. 71–86. [4] P. Apkarian, D. Noll, and O. Prot, Trust region spectral bundle method for nonconvex eigenvalue optimization, submitted, (2007). [5] P. Apkarian, D. Noll, and A. Rondepierre, Mixed H2 /H∞ control via nonsmooth optimization, to appear in CDC, (2007). 21

[6] V. Bompart, P. Apkarian, and D. Noll, Nonsmooth techniques for stabilizing linear systems, in Proc. American Control Conf., New York, NY, July 2007, pp. 1245–1250. [7]

, Control design in the time- and frequency-domain using nonsmooth techniques, Syst. Control Letters, (2008). to appear.

[8] F. D. Bruyne, Iterative feedback tuning for internal model controllers, Control Engineering Practice, 11 (2003), pp. 1043–1048. [9] F. H. Clarke, Optimization and Nonsmooth Analysis, Canadian Math. Soc. Series, John Wiley & Sons, New York, 1983. [10] R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, 1987. [11] M. Gevers, A decade of progress in iterative process control design: from theory to practice, J. Process Control, 12 (2002), pp. 519–531. [12] D. Henrion, S. Tarbouriech, and V. Kucera, control of linear sytems subject to timedomain constraints with polynomial pole placement and LMIs, IEEE Trans. Aut. Control, AC-50 (2005), pp. 1360–1364. [13] H. Hjalmarsson, Efficient tuning of linear multivariable controllers using iterative feedback tuning, Int. J. Adaptive Contr. and Sig. Process., 13 (1999), pp. 553–572. [14] H. Hjalmarsson and T. Birkeland, Iterative feedback tuning for linear time-invariant MIMO systems, in Proc. IEEE Conf. on Decision and Control, vol. 4, December 1998, pp. 3893–3898. [15] H. Hjalmarsson, M. Gevers, S. Gunnarsson, and O. Lequin, Iterative feedback tuning: theory and applications, IEEE Control Syst. Mag., 18 (1998), pp. 26–41. [16] H. Hjalmarsson, S. Gunnarsson, and M. Gevers, A convergent iterative restricted complexity control design scheme, in Proc. of the 33rd IEEE Conference on Decision and Control, Orlando, FL, 1994, pp. 1735–1740. [17] K. H. K, T. Fukuda, and T. S. T, Iterative feedback tuning of PID parameters: comparison with classical tuning rules, Control Eng. Practice, 11 (2003), pp. 1061–1068. [18] L. C. Kammer, F. D. Bruyne, and R. R. Bitmead, Iterative feedback tuning via minimization of the absolute error, in Proc. of the 38th IEEE Conference on Decision and Control, Phoenix, AZ, 1999, pp. 4619–4624. [19] D. Noll, O. Prot, and P. Apkarian, A proximity control algorithm to minimize nonsmooth and nonconvex semi-infinite maximum eigenvalue functions, submitted, (2008). [20] E. Polak, Optimization : Algorithms and Consistent Approximations, Applied Mathematical Sciences, 1997. [21] O. Prot, P. Apkarian, and D. Noll, Nonsmooth methods for control design with integral quadratic constraints, to appear in CDC, (2007). 22

[22] D. C. Savelli, P. C. Pellanda, N. Martins, N. J. P. Macedo, A. A. Barbosa, and G. S. Luz, Robust signals for the TCSC oscillation damping controllers of the brazilian northsouth interconnection considering multiple power flow scenarios and external disturbances, Proceedings of the 2007 IEEE PES General Meeting, (2007).

23