Algebraic Approach to Nonlinear Optimal Control Problems ... - J-Stage

SICE Journal of Control, Measurement, and System Integration, Vol. 11, No. 3, pp. 198–206, May 2018

Special Issue on SICE Annual Conference 2017

Algebraic Approach to Nonlinear Optimal Control Problems with Terminal Constraints: Sufficient Conditions for Existence of Algebraic Solutions Tomoyuki IORI ∗ , Yu KAWANO ∗∗ , and Toshiyuki OHTSUKA ∗ Abstract : This paper studies nonlinear finite-horizon optimal control problems with terminal constraints, where all nonlinear functions are rational or algebraic functions. We first extend a recursive elimination method, which decouples the Euler-Lagrange equations into sets of algebraic equations, where each set contains only the variables at the same time instant. Therefore, a candidate of an optimal feedback control law at each time instant is obtained by solving each set of algebraic equations. Next, we provide a sufficient condition such that each set of algebraic equations gives a unique local optimal feedback control law at each time instant. Illustrative and practical examples are provided to illustrate the proposed method and sufficient condition. Key Words : discrete-time systems, nonlinear systems, optimal control, commutative algebra.

1. Introduction Finite-horizon optimal control problems (FHOCPs) are one of the most important problems in systems and control. In particular, the FHOCPs with terminal constraints emerge in many practical situations [1] and are more general than FHOCPs without terminal constraints. For instance, in model predictive control, we repeatedly solve FHOCPs, and some terminal constraints guarantee asymptotic stability of the origin [2],[3]. Some classes of FHOCPs can be solved analytically under strong assumptions [1],[4] such as the linear quadratic case, but in general, FHOCPs are hard to solve both analytically and numerically. By assuming certain algebraic properties of FHOCPs, we can apply computer algebra tools to these problems, wherein the computational burdens entailed in solving the problems numerically is reduced by utilizing symbolic computation. Fotiou et al. [5] formulated polynomial FHOCPs, whose nonlinearity is described in terms of polynomial functions, as parametric optimization problems on the initial state and solved the KarushKuhn-Tucker conditions by introducing the concept of zerodimensional ideals in commutative algebra. They also proposed a method to solve the parametric optimization problems by applying cylindrical algebraic decomposition (CAD), where the global optimal solution can be obtained if it exists. Iwane et al. [6] combined CAD with dynamic programming and devised a method to compute the value function and optimal control law at each time instant by using CAD. In contrast to these ∗

∗∗

Department of Systems Science, Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan Jan C. Willems Center for Science and Control, Engineering and Technology institute Groningen, Faculty of Science and Engineering, University of Groningen, Nijenborgh 4, 9747 AG Groningen, The Netherlands E-mail: [email protected], [email protected], [email protected] (Received October 30, 2017) (Revised January 24, 2018)

related approaches, we consider rational FHOCPs, where all nolinear functions are rational or algebraic functions. Rational FHOCPs with terminal constraints can describe a wider range of optimal control problems than polynomial FHOCPs can. Ohtsuka [7] proposed a method to decouple the EulerLagrange equations (ELEs), which are the necessary conditions for optimality of FHOCPs, into sets of algebraic equations by using the concept of elimination ideals in commutative algebra. Each set consists of algebraic equations that involve only variables at a single time instant; this structure saves on computations. Ohtsuka also provided sufficient conditions for the existence and uniqueness of optimal feedback laws in the form of algebraic functions. These sufficient conditions guarantee the existence and uniqueness of optimal solutions for not only the given initial state but also the other initial states in some neighborhood of the given one. Moreover, by using the concept of zero-dimensional ideals, they can be made to characterize the optimal feedback laws as points of algebraic sets in the algebraic closure of the rational functions in the state. However, these sufficient conditions have not been extended to cases with terminal constraints. In such cases, the Lagrange multipliers associated with terminal constraints are introduced as additional variables. These additional variables are intrinsic to the problem; in contrast to state or costate variables, they cannot be explicitly expressed by the initial state and inputs. The sufficient conditions proposed in [7], which do not consider any terminal constraints or corresponding Lagrange multipliers, cannot be applied to problems with terminal constraints. We therefore tried to extend the sufficient conditions to guarantee the existence of the Lagrange multipliers by using results from sensitivity analysis. In this paper, first, we introduce a recursive elimination method for solving FHOCPs with terminal constraints where all nonlinear functions are rational or algebraic functions; note that this method has been presented in the preliminary form in conference proceedings [8],[9]. Next, we provide sufficient conditions for optimality of the problems. These conditions are

c 2017 SICE JCMSI 0003/18/1103–0198

SICE JCMSI, Vol. 11, No. 3, May 2018

less conservative than the classical sufficient conditions [1],[4]. Moreover, by applying the concept of zero-dimensional ideals to the outputs of the recursive elimination method, we characterize the optimal feedback laws in the form of algebraic functions. Notation: Throughout this paper, the subscript k denotes the time instant of discrete-time systems, while i denotes the components of a vector. For example, xk denotes a vector at time instant k, while xki denotes the ith component of the vector xk . To avoid confusion, xki is also denoted by xk,i if necessary. For a field K and vectors X = [X1 · · · Xn ]T and Y = [Y1 · · · Ym ]T , K[X, Y] and K(X, Y) denote the ring of polynomials and the field of rational functions, respectively, in the components of X and Y over K. For a field K, K denotes the algebraic closure of K. An ideal generated by the set of polynomials { f1 , . . . , f s } ⊂ K[X] is defined as f1 , . . . , f s := {a1 f1 + · · · + a s f s | a1 , . . . , a s ∈ K[X]}, and the polynomials f1 , . . . , f s are called generators of the ideal. For an ideal n n I ⊂ K[X], V(I) ⊂ K is the set of elements in K where all polynomials in I vanish and is called the algebraic set defined by I. If an ideal I is generated by { f1 , . . . , f s }, V(I) equals the set of elements where all the generators f1 , . . . , f s vanish [10]. In this case, V(I) is also denoted by V( f1 , . . . , f s ). For a scalarvalued function V(X), ∇X V denotes a column vector consisting of the partial derivatives of V with respect to Xi (i = 1, 2, . . . , n). That is, ∇X V = [∂V/∂X1 · · · ∂V/∂Xn ]T . We use ∇X V(Xk ) instead of ∇Xk V(Xk ) for a scalar-valued function V(Xk ) of variables depending on k.

2. Problem Formulation This paper considers the following finite-horizon optimal control problems. min φ(xN ) +

u1 ,...,uN−1

subject to

N−1

Lk (xk , uk )

(1)

k=0

xk+1 = fk (xk , uk ) for k = 0, . . . , N − 1,

(2)

x0 = x¯,

(3)

ψ(xN ) = 0,

(4)

where N ∈ Z+ denotes an optimization horizon of the problem, xk ∈ Rn and uk ∈ Rm denote the states and inputs of a dynamical system at each time instant k = 0, . . . , N, and x¯ denotes a given initial state. The scalar-valued functions φ : Rn → R and Lk : Rn × Rm → R denote the terminal cost and stage costs, respectively, and their sum is the cost function that will be minimized. Each equation (2) is a state equation of the system and consists of a vector-valued function fk : Rn × Rm → Rn . Equation (4) defines the set of terminal constraints, and ψ : Rn → Rl is a vector-valued function. We assume that the components of fk , ∇ x Lk , and ∇u Lk are rational functions and that the components of ψ and ∇ x φ are algebraic functions. The definition of algebraic functions is as follows. Definition 1 An analytic function ρ : U → R defined on an open set U ⊂ Rn is said to be an algebraic function if a nonzero polynomial Φ(y, Y) ∈ R[y, Y] exists such that Φ(y, ρ(y)) = 0 holds for all y ∈ U. The polynomial Φ(y, Y) can be regarded as an element of R(y)[Y] instead of R[y, Y], and ρ(y) is a root

199

of Φ(y, Y) ∈ R(y)[Y]. Therefore, ρ(y) is an element of R(y). Note that each component of the derivative ∂ρ(y)/∂y is also an algebraic function. Our algebraic approach is based on the ELEs. To simplify the description of the ELEs, we define the discrete-time Hamiltonian at time instant k as follows. Hk (xk , uk , pk+1 ) := Lk (xk , uk ) + pTk+1 fk (xk , uk ),

(5)

where pk ∈ Rn (k = 0, . . . , N) denote the costates. The corresponding ELEs to the FHOCP (1)–(4) for k = 0, . . . , N − 1 are xk+1 = fk (xk , uk ),

(6)

pk = ∇ x Hk (xk , uk , pk+1 ),

(7)

∇u Hk (xk , uk , pk+1 ) = 0, T ∂ψ(xN ) pN = ∇ x φ(xN ) + ν, ∂x

(8)

ψ(xN ) = 0,

(9) (10)

where ν ∈ Rl denotes a vector consisting of the Lagrange multipliers associated with the terminal constraints. If the terminal constraint (4) satisfies some constraint qualification, such as the linear independence constraint qualification, the ELEs are the necessary conditions for local optimality [11]; hence, we assume that the optimal control problem (1)–(4) has optimal solutions satisfying the ELEs. Accordingly, the solutions of the ELEs can be regarded as candidates of the optimal solutions. To apply the recursive elimination method, we have to recast the ELEs into a set of algebraic equations. Equations (6)– (8) are recast in [7], and (9) can be readily recast in the same manner as described in [7], because ∇ x φ and ∂ψ/∂x consist of algebraic functions. Therefore, only (10) remains to recast. For each component ψi ∈ R(xN ) of ψ, a polynomial Ψi (xN , zi ) ∈ R[xn , zi ] exists that satisfies Ψi (xN , ψi (xN )) = 0 for all points in the domain of ψi . The polynomial Ψi can be written in the following form. Ψi (xN , zi ) = zαi i dψi (xN , zi ) + ρψi (xN ), where αi is the minimum degree of zi in Ψi − ρψi , and dψi and ρψi are appropriate nonzero polynomials. If zi = ψi (xN ) = 0 holds, then Ψi (xN , ψ(xN )) = 0 and, consequently, ρψi (xN ) = 0.

(11)

Note that (11) is a necessary condition for the terminal constraint ψi (xN ) = 0 to hold. Therefore, we recast (10) into the form ρψ (xN ) = 0 where R[xN ]l ρψ (xN ) := T ρψ1 (xN ) · · · ρψl (xN ) by allowing some invalid solutions to appear. Now, we obtain the following set of algebraic equations from the ELEs. D xk (xk , uk )xk+1 − n xk (xk , uk ) = 0,

(12)

D pk (xk , uk )pk − n pk (xk , uk , pk+1 ) = 0,

(13)

nuk (xk , uk , pk+1 ) = 0,

(14)

ρN (xN , ν, pN ) = 0,

(15)

ρψ (xN ) = 0,

(16)

1 − yk d(xk , uk ) = 0,

(17)

200


where n xk ∈ R[xk , uk ]n , n pk ∈ R[xk , uk , pk+1 ]n , nuk ∈ R[xk , uk , pk+1 ]m , and ρN ∈ R[xN , ν, pN ]n are vectors of polynomials, D xk ∈ R[xk , uk ]n×n and D pk [xk , uk ]n×n are diagonal matrices of polynomials, yk is an additional scalar variable, and ¯ k , uk ) ∈ R[xk , uk ] is a polynomial. Equations (12)–(16) cord(x respond to (6)–(10), and (17) means that all denominators in the ELEs must not vanish; see Section 2 of [7] for details. Even though (12)–(17) are algebraic equations, it is still difficult to solve them; one of the difficulties is that most of them depend on the variables at different time instants. To get rid of this inconvenient structure, we can decouple the set of algebraic equations into sets of algebraic equations, where each set involves only variables at a single time instant and the Lagrange multiplier associated with the terminal constraint. This decoupling can be done using the mathematics of polynomials, i.e., commutative algebra and algebraic geometry. The decoupled equations are satisfied by the optimal solutions; that is, their solutions are candidates of the optimal solutions.

3. Recursive Elimination Method This section introduces the recursive elimination method for decoupling the set of algebraic equations (12)–(17). Each decoupled set involves the variables at time instant k, i.e., xk , uk , pk and the Lagrange multiplier ν, so that they can be solved independently when xk is specified. Ohtsuka proposed such a method for rational FHOCPs without terminal constraints and demonstrated its efficiency [7]. Here, we extend it to work on FHOCPs with terminal constraints. Before introducing the recursive elimination method [9], we will describe the basic notations of commutative algebra and algebraic geometry. An ideal I ⊂ R[X, Y] with X = [X1 · · · Xn ]T and Y = [Y1 · · · Ym ]T contains the set of the polynomials only depending on the variables in Y. This set I ∩R[Y] also becomes an ideal [10] and is called the elimination ideal of I with respect to X [12]. The calculation of the elimination ideal I ∩ R[Y] corresponds to the elimination of the variables X from the algebraic equations defined by the generators of the ideal I with the variables X and Y. In fact, the relationship between the algebraic sets V(I ∩ R[Y]) and V(I) is characterized by the following lemma [10]. Lemma 1 For an ideal I ⊂ R[X, Y], πY (V(I)) ⊂ V(I ∩ R[Y]) holds, where πY : Rn × Rm → Rm defined by (X, Y) → Y is the projection of V(I) onto the Y-space, i.e., πY (V(I)) = {Y ∈ Rm | ∃X ∈ Rn s.t. (X, Y) ∈ V(I)}. Generators of I ∩ R[Y] can be computed from those of I by using a Gröbner basis [10], which is a set of generators that has good properties for symbolic computations and whose computation algorithm has been implemented in various symbolic computation systems such as Mathematica and Maple. For FHOCPs with terminal constraints, the recursive elimination method in [8] is modified into Algorithm 1, which yields Theorem 1 and Corollary 1. Theorem 1 Denote the optimal solution of the FHOCP (1)–(4) N N−1 N , (ûk )k=0 , ( pˆ k )k=0 , and νˆ . Then, for the ideals Jk (k = by ( xˆk )k=0 0, . . . , N) in Algorithm 1, the following statements hold.

Algorithm 1 Recursive Elimination Method for FHOCP with Terminal Constraints Input: Algebraic equations (12)–(17) for FHOCP with terminal constraints Output: Algebraic equations Fk = 0 (k = 0, . . . , N) 1: Let F N be set of polynomials consisting of left-hand sides of (15) and (16), and let JN := F N ⊂ R[xN , ν, pN ], k := N − 1 2: while k ≥ 0 do 3: I¯k := D xk xk+1 − n xk , D pk pk − n pk , nuk , 1 − yk dk , Fk+1 4: Jk := I¯k ∩ R[xk , uk , pk , ν] 5: Let Fk be generators of ideal Jk , that is, Jk = Fk 6: k ←k−1 7: end while

( xˆ N , νˆ , pˆ N ) ∈ V(JN ), ( xˆk , uˆ k , pˆ k , νˆ ) ∈ V(Jk ) (k = 0, . . . , N − 1).

(18) (19)

That is, F N ( xˆ N , νˆ , pˆ N ) = 0,

(20)

Fk ( xˆk , uˆ k , pˆ k , νˆ ) = 0 (k = 0, . . . , N − 1).

(21)

Proof The proof is by induction. First, at k = N, the ideal JN is generated by the left-hand sides of (15) and (16); thus, the statement (18) holds. Suppose that ( xˆk , pˆ k , uˆ k , νˆ ) ∈ V(Jk ) at time step k. From the definition of I¯k−1 , we have ( xˆk−1 , pˆ k−1 , uˆ k−1 , yˆ k−1 , xˆk , pˆ k , uˆ k , νˆ ) ∈ V(I¯k−1 )

(22)

¯ xˆk−1 , uˆ k−1 ). By projecting V(I¯k−1 ) onto the where yˆ k−1 = 1/d( (xk−1 , pk−1 , uk−1 , ν)-space and by applying Lemma 1, ( xˆk−1 , pˆ k−1 , uˆ k−1 νˆ ) ∈ π(xk−1 ,pk−1 ,uk−1 ,ν) (V(I¯k−1 )) ⊂ V(Jk−1 )

(23)

is obtained, and the proof is completed by induction.

Corollary 1 Denote the optimal solution of the FHOCP (1)– N N−1 N (4) by ( xˆk )k=0 , (ûk )k=0 , ( pˆ k )k=0 , and νˆ , and define ideals Ik , Kk , and Wk and their generators GkI , GkK , and GW k as Ik = GkI := Jk ∩ R[xk , pk ], Kk = GkK := Jk ∩ R[xk , uk ], Wk = GW k := Jk ∩ R[xk , ν]. Then, the following statements hold. ( xˆk , pˆ k ) ∈ V(Ik ), ( xˆk , uˆ k ) ∈ V(Kk ), ( xˆk , νˆ ) ∈ V(Wk ). Proof Corollary follows readily from Theorem 1 and Lemma 1. Theorem 1 shows that all sets of polynomials Fk must vanish at the optimal solutions; that is, the solutions of equations Fk = 0 can be regarded as candidates of the optimal solutions. Moreover, Corollary 1 shows that generators of ideal Kk vanish at the optimal values xˆk and uˆ k , and thus it indicates that the set of equations GkK (xk , uk ) = 0 is an implicit representation of the optimal feedback control law at time instant k. Unlike the case without terminal constraints, we have to determine the value of an additional variable ν, namely, the Lagrange multiplier associated with the terminal constraint. The value of ν can be determined only when the value of the initial state x0 = x¯


is given, and thus ν can be implicitly represented by x0 . In Algorithm 1, ν is transferred from the polynomial set F N to F0 , and we obtain the implicit representation of ν by x0 as the set of algebraic equations GW 0 = 0 in Corollary 1. By solving the algebraic equations GkK ( x˜k , uk ) = 0 for a given state value x˜k , we can obtain the candidate values of the optimal input uˆ k ; they include the local optimal solutions if exist. In the following discussion, a candidate of the optimal value yˆ of y is denoted by y˜ . When the equations GkK ( x˜k , uk ) = 0 have solutions u˜ ak and u˜ bk , they may yield two different values of the a b and x˜k+1 from the state equation (2). next candidate state x˜k+1 These candidate state values in turn yields two different sets of K a K b ( x˜k+1 , uk+1 ) = 0 and Gk+1 ( x˜k+1 , uk+1 ) = 0, which equations Gk+1 may have different sets of candidate values of u˜ k+1 . That is, the N−1 branches out at the time of sequence of candidate inputs (˜uk )k=0 K solving Gk ( x˜k , uk ) = 0 for every k = 0, . . . , N − 1. Therefore, a tree that has the given initial value x˜0 = x¯ as its root is suitable for expressing the candidate values, and each node of the tree is associated with a set of u˜ k and x˜k+1 values. Moreover, we can compute the candidate stage cost corresponding to a specific set of x˜k and u˜ k values, and we can obtain the value of the cost function corresponding to a sequence of candidate inputs by accumulating the candidate stage costs and by adding the terminal cost. That is, each leaf of the tree has the candidate value of the cost function corresponding to the sequence of candidate inputs associated with its ancestors. Therefore, the global optimal solution can be obtained if it exists and satisfies the ELEs by picking the leaf associated with the minimum value of the cost function and picking the candidate inputs associated with its ancestors. We denote the tree associated with a given value of the initial state x¯ by T x¯ and denote the accumulated stage costs and candidate values of the state and input associated with a node v ∈ T x¯ by l(v), x(v), and u(v). The procedure described in the previous paragraphs is summarized as Algorithm 2. In this algorithm, the function CalcCands computes u˜ k and x˜k+1 from x(v) for a given node v, and if the third argument k does not equal N − 1, the function calls itself recursively. Therefore, by feeding it the N−1 , a root vroot such that x(vroot ) = x¯, sequence of ideals (Kk )k=0 and the time instance k = 0, this algorithm obtains T x¯ if the total number of candidate values is finite. Note that the sequences of candidate inputs obtained by Algorithm 2 may include invalid solutions that do not satisfy the ELEs. Moreover, the solutions of the ELEs may include one that is not locally optimal. Therefore, in the next section, we utilize the second-order sufficient conditions for local optimality to pick locally optimal solutions from the candidates.

4. Sufficient Conditions for Optimality Here, we introduce the second-order sufficient conditions for guaranteeing the local optimality of the solutions obtained by the recursive elimination method. These conditions are based on the sufficient conditions for local optimality in nonlinear programming [13], and their applicable range is wider than those of well-known sufficient conditions in FHOCPs with terminal constraints in [1],[4]. Moreover, the presented sufficient conditions also guarantee the uniqueness of the optimal solution in some neighborhood of a given initial state. First, to simplify the notation, we define the matrix-valued functions:

201

Algorithm 2 Recursive Computation of Candidate Values Input: Sequence of ideals (K j )N−1 j=0 , node v, time instance k Output: Tree that has node v as its root 1: function CalcCands((K j )N−1 j=0 , v, k) 2: Let U˜ be V(GkK (x(v), uk )), where GkK = Kk ∈ (K j )N−1 j=0 3: for each u˜ k ∈ U˜ do 4: if ∀i ∈ {1, . . . , n}, d xki (x(v), u˜ k ) 0 then 5: x˜k+1 := fk (x(v), u˜ k ) 6: Add new node v as child of v 7: x(v ) ← x˜k+1 8: u(v ) ← u˜ k 9: if k N − 1 then 10: l(v ) ← l(v) + Lk (x(v), u˜ k ) 11: CalcCands((K j )N−1 j=0 , v , k + 1) 12: else 13: l(v ) ← l(v) + Lk (x(v), u˜ N−1 ) + φ( x˜ N ) 14: end if 15: end if 16: end for 17: return Tree that has root v 18: end function

T ∂ fk ∂2 Hk ∂ fk + , Zab (S k+1 , k) := S k+1 ∂a∂b ∂a ∂b

(24)

where S k are n × n matrices for k = 0, . . . , N, characters a and b can be replaced by symbols x and u, and ∂/∂a denotes the derivative with respect to the symbol replacing a. For example, T ∂ fk ∂2 Hk ∂ fk + . (25) S k+1 Zux (S k+1 , k) = ∂u∂x ∂u ∂x Using these matrix-valued functions, we make the following assumption in order to state the sufficient conditions. N N−1 , inputs (ûk )k=0 , Assumption 1 Sequences of states ( xˆk )k=0 N ( pˆ k )k=0 , and νˆ exist such that they satisfy the ELEs (6)–(10), and the following matrix inequalities hold for k = 0, . . . , N − 1.

Zuu (S k+1 , k) > 0,

(26)

where matrices S k ∈ Rn×n in the function Zuu are determined by T −1 (S k+1 , k)Zuu (S k+1 , k)Zux (S k+1 , k) S k =Z xx (S k+1 , k) − Zux

(27) and the boundary condition: SN =

∂2 φ ∂2 ψ + νT 2 . 2 ∂x ∂x

(28)

In this assumption, the arguments of the partial derivatives of N N−1 Hk , fk , φ, and ψ are parts of sequences ( xˆk )k=0 , (ûk )k=0 , and N ( pˆ k )k=0 and νˆ . From the viewpoint of nonlinear programming, it can be shown that the ELEs (6)–(10) and the matrix inequalities (26) are sufficient conditions for local optimality; this is a straightforward consequence of Lemma 4 in the Appendix. Therefore, the existence and uniqueness of local optimal feedback laws are guaranteed by the following theorem. Theorem 2 Suppose that Assumption 1 holds and that the terminal constraint (4) satisfies the linear independence conN−1 and straint qualification. Then, for the sequences (ûk )k=0

202


N ( pˆ k )k=0 and the vector νˆ whose existence is assumed in Assump N−1 tion 1, unique sequences of differentiable functions u∗k (xk ) , k=0 N N p∗k (xk ) , and νk∗ (xk ) exist that are defined on some k=0 k=0 neighborhood of xˆk , and they satisfy u∗k ( xˆk ) = uˆ k , p∗k ( xˆk ) = pˆ k , and νk∗ ( xˆk ) = νˆ . Furthermore, some neighborhood of the given initial state xˆ0 = x¯ exists such that, for any initial state xCL 0 N in the neighborhood, the closed-loop trajectory xCL k k=0 given ∗ CL by u∗k (xCL k ), the sequence of costates given by pk (xk ), and the ∗ CL Lagrange multipliers given by νk (xk ) satisfy the ELEs and N−1 (26); that is, the set of differentiable functions u∗k (xCL k ) k=0 , N N ∗ CL p∗k (xCL k ) k=0 , and νk (xk ) k=0 gives a local optimal solution for xCL ¯. 0 in some neighborhood of x N N−1 , (ûk )k=l , Proof For l = 0, . . . , N − 1, the subsequences ( xˆk )k=l N ( pˆ k )k=l , and νˆ satisfy the ELEs and (26). From Lemma 4 in the N Appendix, a unique set of differentiable functions xkl (xl ) , k=l+1 N−1 N k k k ul (xl ) , pl (xl ) , and νl (xl ) exists that satisfy xl ( xˆl ) = xˆk , k=l k=l k ul ( xˆl ) = uˆ k , pkl ( xˆl ) = pˆ k , νl (xl ) = νˆ , the ELEs, and (26) for any xl in some neighborhood of xˆl . We define xll (xl ) := xl ; then, the uniqueness of these functions implies that

xkl1 (xll1 (xl )) = xkl2 (xll2 (xl )),

(29)

ukl1 (xll1 (xl )) = ukl2 (xll2 (xl )),

(30)

pkl1 (xll1 (xl )) = pkl2 (xll2 (xl )),

(31)

νl1 (xll1 (xl ))

(32)

=

νl2 (xll2 (xl )),

hold for any l1 , l2 ∈ {l, . . . , k} and for any xl in some neighborhood of xˆl . Now, let us define u∗k (xk ) := ukk (xk ) and fkCL (xk ) := fk xk , u∗k (xk ) for k = 0, . . . , N − 1. Since xˆk+1 = fkCL ( xˆk ), CL k+1 the uniqueness of xk+1 k (xk ) implies xk (xk ) = fk (xk ) for k = 0, . . . , N−1 and for any xk in some neighborhood of xˆk . Accordingly, we obtain the following equations for k = 0, . . . , N − 1 and for any x0 in some neighborhood of xˆ0 = x¯. xk0 (x0 ) = xkk−1 ◦ · · · ◦ x10 (x0 ) CL = fk−1 ◦ · · · ◦ f0CL (x0 ).

(33) (34)

N xk0 (x0 ) k=0

is the closed-loop trajectory N−1 given by the sequence of feedback control laws u∗k (xk ) , k=0 k ∗ k namely, xCL = x (x ). Similarly, we define p (x ) := p (x ) k k 0 0 k k k and νk∗ (xk ) := νk (xk ); then, That is, the sequence

k k k u∗k (xCL k ) = uk (x0 (x0 )) = u0 (x0 ), k k k p∗k (xCL k ) = pk (x0 (x0 )) = p0 (x0 ), k νk∗ (xCL k ) = νk (x0 (x0 )) = ν0 (x0 ),

(35) (36) (37)

hold from the definitions of u∗k , p∗k , and νk∗ and (30)–(32). There N fore, the closed-loop trajectory xCL k k=0 and corresponding in∗ CL ∗ CL puts uk (xk ), costates pk (xk ), and Lagrange multiplier νk∗ (xCL k ) are identical to the sequences xk0 (x0 ), uk0 (x0 ), and pk0 (x0 ) and the Lagrange multiplier ν0 (x0 ), which satisfy the ELEs and (26). Note that Theorem 2 is not a straightforward extension of the result in [7] because of the existence of an additional variable ν.

The result in [7] corresponding to Theorem 2 is based on sufficient conditions for optimality of FHOCPs without constraints, and these conditions cannot be applied to FHOCPs with terminal constraints. For those constrained FHOCPs, sufficient conditions for optimality have already been proposed [1],[4]. However, their applicable range is limited because they assume that ν is explicitly represented by each state xk . Hence, we provide less conservative conditions (Lemma 4) by getting rid of that assumption and prove Theorem 2 by using the new conditions. We can filter the sequences of candidate inputs obtained by Algorithm 2 by checking whether each candidate satisfies the ELEs and (26) and obtain locally optimal solutions. The uniqueness and differentiability of the optimal solution in some neighborhood also enables us to track the optimal solution from one state value to another: we can find a solution of GkK (xk , uk ) = 0 with xk xˆk from the optimal solutions xˆk and uˆ k by using numerical computations such as Newton’s method. Moreover, we can characterize the optimal feedback laws, costates, and Lagrange multiplier in Theorem 2 as algebraic functions of the state by placing assumptions on the ideals obtained in Corollary 1, which is the topic of the next section.

5. Existence of Algebraic Solutions u∗k (xk ),

When p∗k (xk ), and νk∗ (xk ) are algebraic functions of xk , each of their components can be regarded as a point in R(xk ). Therefore, u∗k (xk ) can be characterized as a point of the m algebraic set in R(xk ) defined by polynomials in R(xk )[uk ] instead of R[xk , uk ]; the same argument applies to p∗k and νk∗ . To characterize the algebraic sets, we will need additional notation from commutative algebra and algebraic geometry. For an ideal I ∈ R[X, Y] with X = [X1 · · · Xn ]T and Y = [Y1 · · · Ym ]T , the extension of the ideal I to R(X)[Y] is the ideal I e defined as I e := {b1 g1 + · · · + b s g s |b1 , . . . , b s ∈ R(X)[Y], g1 , . . . , g s ∈ I, s ∈ N}.

(38)

For an ideal J ⊂ R(X)[Y], J is called zero-dimensional if the m algebraic set V(J) ⊂ R(X) is a finite set. Zero-dimensional ideals are characterized by the following lemma [14]. Lemma 2 An ideal J ⊂ R(X)[Y] is zero-dimensional if and only if a nonzero polynomial hi ∈ R(X)[Yi ] exists for each i = 1, . . . , m such that J ∩ R(X)[Yi ] = hi holds. The polynomials hi (i = 1, . . . , m) are called minimal polynomials of Yi with respect to J and can be computed from generators of J by using a Gröbner basis. For an ideal J ⊂ R(X)[Y], the set of polynomials: √ J := {g ∈ R(X)[Y] | ∃s ∈ N s.t. g s ∈ J} (39) is also an ideal called the √ a √ radical of the ideal J. J is called radical ideal when J = J holds. Obviously, V(J) = V J holds. There is an algorithm to obtain the radical of a zerodimensional ideal [14]. Zero-dimensional radical ideals have good properties stated in the following lemma [7]. Lemma 3 If J ⊂ R(X)[Y] is a zero-dimensional radical ideal, m generators g1 , . . . , gm ∈ R(X)[Y], and it has exactly det ∂gi (Y)/∂Y j 0 for every Y ∈ V(J).

203


Lemma 3 shows that a set of generators g1 , . . . , gm exists such that the equations gi = 0 (i = 1, . . . , m) form a square system of polynomial equations in the variables Y whose Jacobian at each Y ∈ V(J) is a nonzero algebraic function of X; that is, the Jacobian does not vanish in an open and dense subset of X-space. Moreover, under additional mild assumptions, the Gröbner basis of a zero-dimensional radical ideal has a good structure for computing the generators in Lemma 3, which is known as the Shape Lemma [12]. Now, the functions u∗k , p∗k , and νk∗ are characterized as points of algebraic sets by the following theorem. Theorem 3 Suppose that Assumption 1 holds, that the terminal constraint (4) satisfies the linear independence constraint qualification, and that the extensions of the ideals Kke ⊂ R(xk )[uk ], Ike ⊂ R(xk )[pk ], and Wke ⊂ R(xk )[ν] are zero-dimensional ideals for k = 0, . . . , N − 1. Then, the optimal feedback laws u∗k (xk ), costates p∗k (xk ), and Lagrange multiplier νk∗ (xk ), whose existences are guaranteed by Theorem 2, are algebraic functions, and u∗k ∈ V(Kke ), p∗k ∈ V(Ike ), and νk∗ ∈ V(Wke ). Moreover, a set of generators of each ideal exists that form a square system of polynomial equations, and its Jacobian matrix is nonsingular for xk in an open and dense subset of some neighborhood of xˆk . Proof Theorem 2 shows that the set of u∗k ( x˜k ), p∗k ( x˜k ), and neighborhood of νk∗ ( x˜k ) is an optimal solution for x˜k in some ∗ xˆ . Therefore, Corollary 1 implies that x˜k , uk ( x˜k ) ∈ V(Kk ), k x˜k , p∗k ( x˜k ) ∈ V(Ik ), and x˜k , νk∗ ( x˜k ) ∈ V(Wk ). Since Kke at each time instant k is a zero-dimensional ideal, a minimal polynomial hi (xk , uki ) ∈ R(xk )[uki ] for each i = 1, . . . , m exists that vanishes on V(Kke ). Note that, from the definition of extensions of ideals, we can choose hi from Kk instead of Kke by multiplying some polynomial in R[xk ]. Therefore, x˜k , u∗k ( x˜k ) ∈ V(Kk ) for all x˜k in some neighborhood of xˆk im plies that hi xk , u∗ki (xk ) = 0 in the neighborhood, and thus, ∗ each component uki (xk ) is an algebraic function of xk . More∗ over, xk , uk (xk ) ∈ V(Kk ) implies that every polynomial in Kk m

vanishes at u∗k ∈ R(xk ) , and thus, all polynomials in Kke also vanish at u∗k , which implies u∗k ∈ V(Kke ). Now, Lemma 3 guarantees that the radical Kke has ex actly m generators as Kke = G¯ kK such that det ∂G¯ kK /∂u∗k ∈ R(xk ) is a nonzero algebraic function of xk , which implies that ∂G¯ kK (xk , u∗k (xk ))/∂uk ∈ Rm×m is nonsingular in an open and dense subset of some neighborhood of xˆk . The same argument can be applied to p∗k and νk∗ , thus completing the proof. Theorem 3 shows that we can guarantee the differentiability and nonsingularity of the algebraic state feedback laws in an open and dense subset of some neighborhood of xˆk by using the notion of zero-dimensional ideals. When we find u∗k ( x˜k ) for x˜k in the neighborhood, nonsingularity helps us to solve equations G¯ kK ( x˜k , uk ) = 0 numerically by using, for example, Newton’s method or the continuation method. Moreover, we can use the minimal polynomials hi (xk , uki ) of uki with respect to Kke to compute u∗k (xk ). In this case, we can compute u∗ki ( x˜k ) for x˜k by solving the univariate algebraic equation hi ( x˜k , uki ) = 0. Note that the computation of u∗ki ( x˜k ) can be performed independently for each i ∈ {1, . . . , m}.

6. Numerical Examples 6.1

Illustrative Example

Let us consider the following FHOCP with a terminal constraint. min

u1 ,...,uN−1

subject to

N−1 1 k=0

2

u2k

(40)

⎧ ⎪ ⎪ xk+1,1 = xk,2 , ⎪ ⎪ ⎨ xk,1 uk ⎪ ⎪ ⎪ ⎪ ⎩ xk+1,2 = −xk,1 + 1 + xk,1 ,

(41)

x0 = x¯,

(42)

xN = 0.

(43)

For the optimization horizon N = 4, we can obtain the polynomials F4 and F3 as F4 = {x41 , x42 , p41 − ν1 , p42 − ν2 }, 2 − x31 , F3 = {x32 , p32 − ν1 , p31 x31 − x31 2 − x31 , . . .}. u23 − u3 − x31

Some of the polynomials in F3 and all of the polynomials F2 , F1 , and F0 are omitted because of space limitations. The ideals Kk ⊂ R[xk , uk ] are obtained from Corollary 1 as 2 2 − x31 , u23 − x31 − u3 − x31 , K3 = x32 , u3 x31 − x31 2 2 − x21 , u22 − x21 − u2 − x21 , K2 = u2 x21 − x21 2 3 4 − 3u1 x11 + x11 + · · · , K1 = 2u21 x11 2 3 4 K0 = 2u20 x01 − 3u0 x01 + x01 + · · · .

By using Algorithm 2 in [9], we obtain 36 candidate optimal solutions for a given initial state x¯ = [−2.0, 3.0]T , and four of them satisfy the ELEs. Figure 1 shows the state trajectories T 1 , T 2 , T 3 , and T 4 given by these four sequences. The values of the cost functions corresponding to T 1 , T 2 , T 3 , and T 4 are 1.78, 8.50, 8.90, and 2.18. Since the dimension of the input is 1, the matrices Zuu (S k+1 , k) in (26) are scalars. Table 1 shows the corresponding values of Zuu (S k+1 , k) for each trajectory. All of the values of Zuu for T 1 are positive; hence, the sequence of candidate inputs giving T 1 is a locally optimal solution. The ideals K1 and K0 are generated by only one polynomial in R[xk , uk ], and their extensions are also generated by these polynomials. Therefore, from Lemma 2, K1e and K0e are zerodimensional ideals. Moreover, generators of these ideals are both square-free, which implies that these ideals are also radical. Then, the optimal state feedback laws can be obtained as

Fig. 1 State trajectories given by candidate inputs.

204


Table 1 T1 T2 T3 T4 T 1

Values of Zuu (S k+1 , k).

k=0

k=1

k=2

k=3

3.84 16.0 −157 −56.0 4.37

0.892 −15.0 −18.6 1.12 0.902

1.00 1.00 1.00 1.00 1.00

1.00 1.00 1.00 1.00 1.00

the roots of these generators, and we can calculate the optimal feedback laws explicitly because the degrees of the generators are at most 2. The following algebraic feedback laws are the roots of the generators of K1e and K0e and give T 1 . u∗0 (x0 ) = x01 + 1, u∗1 (x1 ) =

2 x11 (x11 − 1) 2 2x11 + 2x11 + 1

(44) ,

(45)

Note that the other inputs u2 and u3 are uniquely determined from only the state equation (41) and the terminal constraint (43). Note also that these optimal state feedback laws are calculated without any approximations. Theorem 2 guarantees that, for all x0 in some neighborhood of x¯ = [−2, 3]T , these algebraic feedback laws are also optimal. Indeed, for an initial state x¯ = [−2.4, 3.3] x¯, we obtain the trajectory T 1 that corresponds to the values of Zuu (S k+1 , k) in Table 1, which implies that the sequence of inputs giving T 1 is also a locally optimal solution. 6.2 Nonlinear Model Predictive Control Application In practice, an FHOCP is usually accompanied by nonlinear model predictive control (NMPC), in which an FHOCP is solved repeatedly and only the initial input of the optimal solution is utilized as the actual input. By solving the set of algebraic equations G0K (x0 , u0 ) = 0 obtained by Algorithm 1, we can obtain candidates of the initial optimal input immediately from the given initial state. Moreover, in the case of NMPC with a short enough sampling period, the state variation within the interval should be small. Therefore, from Theorem 2, the solution of each FHOCP possibly retains local optimality. As a practical example, let us examine stabilization of an inverted pendulum by NMPC. We define the continuous-time equation of motion of the pendulum as ¨ + θ(t) ˙ − sin θ(t) = u(t), θ(t)

control objective is to regulate the state to the origin and reduce control burdens. Thus, the stage costs and terminal constraint are defined as 1 Lk (xk , uk ) = u2k , ψ(xN ) = xN = 0. (48) 2 Since this terminal constraint specifies the terminal state, the terminal cost is omitted (φ(xN ) ≡ 0). For N = 2, G0K consists of only one polynomial: 2 3 + 20)u0 + 1193x01 + κ0 (x0 , u0 ) = 3(x01 2 x02 + 24060x01 + 2340x02 . 117x01

(49)

Since κ0 (x0 , u0 ) is linear in u0 , the root of κ0 (x0 , u0 ) as a polynomial of u0 can be computed symbolically, and this root is an explicit form of the feedback control law at k = 0. Moreover, the problem settings of this example give the values of the scalars Zuu (S 1 , 0) and Zuu (S 2 , 1) in (26) as Zuu (S 1 , 0) = Zuu (S 2 , 1) = 1 irrespective of the initial state, which means the obtained feedback control law is locally optimal, by Theorem 2. Figure 2 shows the trajectories of the system (46) with the feedback control law defined by κ0 (x(t), u(t)) = 0 and the free response without the control law. Each arrow in the figure shows the direction that each trajectory goes toward. The initial state is set to x0 = [1.0 0.0]T . In the controlled case, the state and control input are sampled and computed with a sampling period of 0.01 time units. That is, the sampling period of NMPC is smaller than that of the discretization. It is readily seen that the state of the controlled system is regulated to the origin.

Fig. 2 Trajectories of system (46) derived from controlled system (solid line) and its free response (dashed line).

(46)

where t ∈ R+ = [0, ∞) denotes continuous time, θ(t) ∈ R denotes the angle subtended by the pendulum and the vertical axis, and u(t) ∈ R denotes the controllable external torque. To apply the proposed method, we replace sin θ in (46) with (θ − 7/60θ3 )/(1 + 1/20θ2 ) by using Padé approximation, which often gives a better approximation of a function than one given by a truncated Taylor series and may still work outside of the convergence region of the Taylor series [15]. By using the forward difference approximation with a sampling period of Δt = 0.05, the discrete-time model can be obtained as ⎧ ⎪ ⎪ xk+1,1 = xk,1 + xk,2 Δt, ⎪ ⎪ ⎞ ⎛ ⎪ ⎨ 7 3 xk,1 − 60 xk,1 ⎟⎟⎟ ⎜⎜⎜ (47) ⎪ ⎪ ⎟ ⎜ ⎪ ⎪ ⎪ ⎩ xk+1,2 = xk,2 + ⎝⎜−xk,2 + 1 + 1 x2 + uk ⎠⎟ Δt, 20 k,1 where xk,1 and xk,2 denote the angle and angular velocity, respectively, and uk denotes the controllable external torque. The

7. Conclusion and Future Work We proposed a recursive elimination method for rational FHOCPs with terminal constraints. By applying the concept of elimination ideals from commutative algebra, the method decouples the ELEs into sets of algebraic equations, where each set involves the Lagrange multiplier associated with the terminal constraint and the variables at a single time instant. We also proposed an algorithm to solve the sets of algebraic equations and obtain candidates of local optimal solutions. By checking whether the presented sufficient conditions are satisfied or not for each candidate, we can pick local optimal solutions from the candidates. The sufficient conditions also guarantee the uniqueness of the optimal solution in some neighborhood of the given initial state. Moreover, we provided sufficient conditions for the existence of the optimal solutions in the form of algebraic functions of the states. By utilizing the concept of zero-dimensional

205


ideals and some additional assumptions, we can guarantee the nonsingularity of the optimal solutions in some neighborhood, which is a useful property for numerical computations. In the future, we plan to establish sufficient conditions for optimality using only the information at the initial time instant, i.e., the value of the initial state, which is suitable for model predictive control.

N−1 dence constraint qualification. Then, for the sequences (ûk )k=0 N and ( pˆ k )k=0 and the vector νˆ whose existence is assumed in Assumption 1, there exists a unique set of differentiable functions N N−1 N N (xk (x0 ))k=0 , (uk (x0 ))k=0 , (pk (x0 ))k=0 , and νk (x0 )k=0 that satisfy xk ( xˆ0 ) = xˆk , uk ( xˆ0 ) = uˆ k , pk ( xˆ0 ) = pˆ k , νk ( xˆ0 ) = νˆ , the ELEs, and the matrix inequalities (26) for x0 in some neighborhood of xˆ0 .

Acknowledgments

Now, let us consider the Lagrangian function of the FHOCP (1)–(4), defined as

This work was partly supported by JSPS KAKENHI Grant Numbers 15K18087 and 15H02257 and JST CREST Grant Number JPMJCR15K2. [1] [2]

[3]

[4] [5]

[6]

[7]

[8]

[9]

[10] [11]

[12] [13] [14] [15]

References A.E. Bryson, Jr. and Y.-C. Ho: Applied Optimal Control, Washington, Hemisphere, 1975. P. Scokaert, J. Rawlings, and E. Meadows: Discrete-time stability with perturbations: Application of model predictive control, Automatica, Vol. 33, No. 3, pp. 463–467, 1997. G. De Nicolao, L. Magni, and R. Scattolini: Stability and robustness of nonlinear receding horizon control, F. Allgöwer and A. Zheng, Eds., Nonlinear Model Predictive Control, Birkhäuser-Verlag, 2000. F.L. Lewis, D.L. Vrabie, and V.L. Syrmos: Optimal Control, 3rd edition, John Wiley & Sons, 2012. I.A. Fotiou, P. Rostalski, P.A. Parrilo, and M. Morari: Parametric optimization and optimal control using algebraic geometry methods, International Journal of Control, Vol. 79, No. 11, pp. 1340–1358, 2006. H. Iwane, A. Kira, and H. Anai: Construction of explicit optimal value functions by a symbolic-numeric cylindrical algebraic decomposition, Proc. of the 13th International Workshop on Computer Algebra in Scientific Computing, Lecture Notes in Computer Science, Vol. 6885, pp. 239–250, 2011. T. Ohtsuka: A recursive elimination method for finite-horizon optimal control problems of discrete-time rational systems, IEEE Trans. on Automatic Control, Vol. 59, No. 11, pp. 3081– 3086, 2014. T. Iori, Y. Kawano, and T. Ohtsuka: Algebraic approach to nonlinear finite-horizon optimal control problems of discrete-time systems with terminal constraints, Proc. of SICE Annual Conf., pp. 220–225, Kanazawa, Japan, Sept. 2017. T. Iori, Y. Kawano, and T. Ohtsuka: Algebraic approach to nonlinear finite-horizon optimal control problems with terminal constraints, Proc. of Asian Control Conf., pp. 1–6, Gold Coast, Australia, Dec. 2017. D. Cox, J. Little, and D. O’Shea: Ideals, Varieties, Algorithms, 4th edition, Springer, 2015. G.C. Goodwin, M.M. Seron, and J.A. De Doná: Constrained Control and Estimation: An Optimization Approach, Springer, 2005. M. Kreuzer and L. Robbiano: Computational Commutative Algebra., Vol. 1, Springer-Verlag, 2000. J.F. Bonnans and A. Shapiro: Perturbation Analysis of Optimization Problems, Springer, 2000. D. Cox, J. Little, and D. O’Shea: Using Algebraic Geometry, 2nd edition, Springer, 2005. ´ ApproxP. Graves-Morris: The Numerical Calculation of Pade imants, Lecture Notes in Mathematics, Vol. 765, pp. 231–245, Springer, 1979.

Appendix We prove the following lemma. Lemma 4 For the FHOCP (1)–(4), suppose that Assumption 1 holds and the terminal constraint (4) satisfies the linear indepen-

J¯ :=φ(xN ) + νT ψ(xN ) +

N−1

Hk (xk , uk , pk+1 ) − pTk+1 xk+1 .

(A. 1)

k=0

and consider the second-order terms in its Taylor expansion at N N−1 N the point ( xˆk )k=0 , (ûk )k=0 , ( pˆ k )k=0 , and νˆ : 2 2 1 ∂ φ T∂ ψ dxN + ν ˆ d2 J¯ = dxTN 2 ∂x2 ∂x2 ⎡ T ∂2 Hk ∂2 Hk ⎤ N−1 ⎢⎢⎢ ⎥⎥ dxk 1 dxk 2 ∂x∂u ⎥ ⎢⎢⎣ ∂x ⎥⎥⎦ , + 2 2 ∂ Hk ∂ Hk du du 2 k=0

k

∂u∂x

k

∂u2

(A. 2) where dxk and duk are infinitesimal changes from xˆk and uˆ k , N respectively. The sequence (dxk )k=0 is defined by the sequence N−1 (duk )k=0 and the following linearized system: ∂ fk ∂ fk dxk + duk , (A. 3) ∂x ∂u (A. 4) dx0 = 0, ∂ψ dxN = 0, (A. 5) ∂x where the partial derivatives of fk and ψ in (A. 3)–(A. 5) are N N−1 parts of sequences ( xˆk )k=0 and (ûk )k=0 . Lemma 4 can be deduced from a classical result in sensitivity analysis [13] if a real number α > 0 exists such that ⎡N−1 ⎤ ⎢⎢⎢ ⎥⎥⎥ d2 J¯ ≥ α ⎢⎢⎢⎣ duk 2 + dxk+1 2 ⎥⎥⎥⎦ (A. 6) dxk+1 =

k=0

holds for any feasible sequences of infinitesimal changes N N−1 (dxk )k=0 and (duk )k=0 . Therefore, we will prove that the premises in Lemma 4 imply the positivity of d2 J¯ under the constraints (A. 3)–(A. 5). To do so, we will make a change of variables: η := ΓN yN , [duT0

(A. 7) · · · duTN−1

dx1T

· · · dxTN ]T

where yN := RNm×N(n+m) is defined as Om×(N−1)n ΓN := Zuu Zux

∈R

Om×n

N(n+m)

and ΓN ∈

O(N−1)m×n

(A. 8)

and Zuu and Zux are

0 N−1 Zuu := block-diag Zuu , . . . , Zuu ∈ RNm×Nm ,

1 N−1 , . . . , Zux ] ∈ R(N−1)m×(N−1)n , Zux := block-diag[Zux k k for the matrices Zuu , Zux (k = 0, . . . , N − 1) in Assumption 1, and Ol1 ×l2 is an l1 × l2 zero matrix. To prove Lemma 4 by using the change of variables (A. 7), we introduce the following proposition. To simplify the notak . tion, we denote Zab (S k+1 , k) in (26) and the proof as Zab

206


Proposition 1 Suppose that Assumption 1 holds. For any seN−1 N ⊂ Rm and sequence (dxk )k=1 ⊂ Rn defined quence (duk )k=0 by the linearized system (A. 3)–(A. 5), the following statement holds. ΓN yN = 0 ⇔ yN = 0.

holds or, equivalently, ||ΓN yN ||2 ≥ γ||yN ||2 holds for any yN defined by (A. 3)–(A. 5). Therefore, ⎛N−1 ⎞ ⎜⎜⎜ ⎟⎟ 2 ¯ 2 2 2 2 ⎟ ⎜ ⎜ ||duk || + ||dxk+1 || ⎟⎟⎟⎠ d J ≥ β||η|| ≥ βγ||yN || = α ⎜⎝ k=0

(A. 9)

Proof Sufficiency (⇐) is trivial; thus, we will prove only necessity (⇒). The proof is by induction for the optimization horizon N. First, for N = 1, the matrix Γ1 is obtained as 0 0 , (A. 10) Γ1 = Zuu

holds for α := βγ > 0. The theorem thus follows from a classical result of sensitivity analysis [13], the proof completes.

Tomoyuki IORI

From the inequality > 0 in Asand y1 := sumption 1, Γ1 y1 = 0 implies du0 = 0, which also implies dx1 = 0 from the linearized state equation (A. 3) with dx0 = 0. Next, suppose yN = [duT0 · · · duTN −1 dx1T · · · dxTN ]T = 0. Then, [duT0

dx1T ]T .

0 Zuu

yN +1 = [0 · · · 0 duN 0 · · · 0 dxN +1 ]

Yu KAWANO

holds, and thus ΓN +1 yN +1 = 0 implies 0 ΓN +1 yN +1 = N = 0. Zuu duN

(A. 11)

k N > 0 in Assumption 1, Zuu duN = 0 imFrom the inequality Zuu plies duN = 0. Accordingly, the linearized state equation (A. 3) with dxN = 0 also implies dxN +1 = 0. Therefore, the proof is completed by induction.

Now, we prove Lemma 4 using Proposition 1. Proof Consider the following quantity that is identical to zero N−1 N and the sequence (dxk )k=0 defined by for any sequence (duk )k=0 (A. 3)–(A. 5): N−1 ∂ fk 1 T ∂ fk dxk+1 S k+1 dxk + duk − dxk+1 , (A. 12) 2 k=0 ∂x ∂u where the matrices S k ∈ Rn×n are defined in Assumption 1. By completing the square, the sum of d2 J¯ and the quantity (A. 12) can be written as a quadratic form: 1 T −1 η Zuu η 2 N−1 1 T k k T k −1 k + dxk Z xx − Zux Zuu Zux − S k dxk 2 k=0 2 1 T ∂2 φ T∂ ψ + dxN + νˆ − S N dxN 2 ∂x2 ∂x2 T −1 0 0 0 0 + dx0T Z xx − Zux Zux (A. 13) dx0 , Zuu

d2 J¯ =

where η = ΓN yN and yN = [duT0 · · · duTN−1 dx1T · · · dxTN ]T . From (27) and (28), the second and third terms on the right-hand side of (A. 13) vanish, and from (A. 4), the fourth term also vanishes. Now, Assumption 1 is such that Zuu > 0, which implies that a real number β > 0 exists such that 1 2 d2 J¯ = ηT Z−1 uu η ≥ β||η|| . 2 From Proposition 1, a real number γ > 0 exists such that γ = min||yN ||=1 ||ΓN yN ||2

He received his B.Eng. degree from Kyoto University, Japan, in 2016. Since 2016, he has been pursuing the M.Eng. degree with the Graduate School of Informatics, Kyoto University. His research interests include nonlinear control theory.

He received his M.S. and Ph.D. degrees in engineering from Osaka University, Japan, in 2011 and 2013, respectively. Since 2016, he has been a Researcher with the Engineering and Technology institute Groningen (ENTEG) at the University of Groningen, The Netherlands. In 2012, he was a Research Student at Tallinn University of Technology, Estonia. From 2013 to 2016, he was a Researcher with Kyoto University and JST CREST, Japan. In 2015, he was a Visiting Researcher at the University of Groningen. His research interests include analysis and model reduction of nonlinear systems and complex networks. He is a member of IEEE.

Toshiyuki OHTSUKA (Member) He received his B.Eng., M.Eng. and D.Eng. from Tokyo Metropolitan Institute of Technology, Japan, in 1990, 1992 and 1995. From 1995 to 1999, he worked as an Assistant Professor at the University of Tsukuba. In 1999, he joined Osaka University, and he was a Professor at the Graduate School of Engineering Science from 2007 to 2013. In 2013, he joined Kyoto University as a Professor at the Graduate School of Informatics. His research interests include nonlinear control theory and real-time optimization with applications to aerospace engineering and mechanical engineering. He is a member of IEEE, ISCIE, JSME, JSASS, and AIAA.