Some Insights on Synthesizing Optimal Linear Quadratic Controller ...

Some Insights on Synthesizing Optimal Linear Quadratic Controller Using Krotov’s Sufficiency Conditions Avinash Kumar and Tushar Jain

∗

arXiv:1807.02631v1 [math.OC] 7 Jul 2018

Abstract This paper revisits the problem of optimal control law design for linear systems using the global optimal control framework introduced by Vadim Krotov. Krotov’s approach is based on the idea of total decomposition of the original optimal control problem (OCP) with respect to time, by an ad hoc choice of the so-called Krotov’s function or solving function, thereby providing sufficient conditions for the existence of global solution based on another optimization problem, which is completely equivalent to the original OCP. It is well known that the solution of this equivalent optimization problem is obtained using an iterative method. In this paper, we propose suitable Krotov’s functions for linear quadratic OCP and subsequently, show that by imposing convexity condition on this equivalent optimization problem, there is no need to compute an iterative solution. We also give some key insights into the solution procedure of the linear quadratic OCP using the proposed methodology in contrast to the celebrated Calculus of Variations (CoV) and Hamilton-Jacobi-Bellman (HJB) equation based approach.

1

Introduction

Optimal control theory is a widely explored and still developing discipline of control engineering where the objective is to design a control law so as to optimize (maximize or minimize) a Performance Index (cost functional) while driving the states of a dynamical system to zero (Regulation problem) or to make output track a reference trajectory (Tracking problem) [1]. The generic optimal control problem (GOCP) is given as: GOCP. Compute an optimal control law u∗ (t) which minimizes (or maximizes) the performance index/cost functional: Z tf l(x(t), u(t), t)dt (1) J(x(t), u(t), t) = lf (x(tf )) + t0

subject to the system dynamics x(t) ˙ = f (x(t), u(t), t) with x(0) ∈ Rn ; t ∈ [t0 , tf ]

(2)

to give the desired optimal trajectory x∗ (t). Here, l(x(t), u(t), t) is the running cost, lf (x(tf )) is the terminal cost, x(t) ∈ X ⊂ Rn is the state vector and u(t) ∈ U ⊂ Rm is the control input to be designed. Also, lf : Rn → R and l : Rn × Rm × [t0 , tf ] → R are continuous. Clearly, the aforementioned corresponds to optimization of the cost functional subject to dynamics of the system considered and possibly constraints on input(s) and/or state(s), and hence the Calculus of Variations(CoV) is generally used to address optimal control design problems [1, 2]. The CoV based techniques traditionally work by assuming the existence of an optimal control law and subsequent derivation of the conditions which must be satisfied by such an optimal control law. Hence, only necessary conditions are found and in general sufficiency of these conditions is not guaranteed [2]. Furthermore, the obtained control law is generally only locally optimum [2]. Nevertheless, there are results available in the literature which provide restrictions under which the necessary conditions indeed become sufficient and the global optimal control law is obtained [3, 4, 5]. Note that, in solving optimal control design problems, the CoV method introduce the notion of so-called co-states [6]. Furthermore, in the solution procedure the states and co-states are assumed to be related by a linear time varying transformation [1]. A proper reasoning for this assumption does not seem to be available in the existing literature. Alongside CoV, another tool, namely dynamic programming (DP) (introduced by Bellman), has also been explored to solve optimal control problems [1]. The application of DP to optimal control design problems for continuous linear systems leads to the celebrated Hamilton-Jacobi-Bellman (HJB) equation which also gives a necessary condition for optimality [1]. Nevertheless, this equation also provides sufficiency under mild conditions on the optimal cost function [2, 6]. These conditions are: ∗ Avinash Kumar and Tushar Jain are with Indian Institute of Technology Mandi, School of Computing and Electrical Engineering, Himachal Pradesh 175005, India. email: [email protected], [email protected]

a) The optimal cost function actually exists b) The optimal cost function is continuously differentiable c) The gradient of cost function with respect to state vector equals costate which corresponds to the optimal trajectory For example, consider the optimal control design problem for the system [7]: x(t) ˙ = x(t)u(t) with performance measure as
J x(t), u(t), t = ∗

Z

1

∞



 x2 (t) + u2 (t) dt

For this problem, the optimal cost function is J (x (t), u∗ (t), t) = |x∗ (t)| and hence the HJB equation is not defined at x = 0 because of non-differentiability of J ∗ (x∗ (t), u∗ (t), t). Taking into consideration the aforementioned observations, a solution method for optimal control design which does not demand such conditions is desirable. As illustrated in this article, Krotov’s sufficiency conditions for global optimal control is indeed such a methodology [2]. Specifically, this methodology provides sufficient conditions for the global optimal control without using the notion of co-states and assumption that optimal cost function is continuously differentiable. Starting in the sixties, the results on sufficient conditions for the global optimum of optimal control problem were published by Vadim Krotov [8, 9]. The basic idea of development of these global methods is a total decomposition of the optimal control problem with respect to time via an appropriate selection of the so-called solving/bounding/Krotov’s function [2, 10, 11]. Once such a decomposition is obtained, the problem is reduced to a family of independent elementary optimization problems parameterised in time t. It has been shown in [2] that solving the two problems- original optimal control problem and the elementary optimization problems (resulting form decomposition)- is completely equivalent and once the solution is obtained, it is the global optimal solution. The key idea is that the solution of the latter optimization problems could be much easier than solving the original optimal control problem. The method, however, is abstract in the sense that the selection of Krotov’s function is not straightforward and usually it differs from one problem to the other [10]. The methodology has been applied to optimal control problems encountered in magnetic resonance systems [12], quantum control [13] etc. but is barely explored for engineering applications [11]. In this work, suitable Krotov’s functions are proposed for solving linear quadratic optimal control problems and then convexity conditions are imposed on the equivalent optimization problem (of functions parameterised in t) to compute the required global optimal control law. Thus, this work may lay a foundation for the procedure of selecting Krotov’s function (which is indeed a crucial step for the methodology to work) for more involved optimal control problems.

2

∗

Preliminaries and Problem Formulation

OCP 1. (Finite Horizon Linear Quadratic Regulation (LQR) problem): Compute an optimal control law u∗ (t) which minimizes the quadratic performance index/cost functional: h i h Z tf i T J(x(t), u(t), t) = 0.5 x (tf )F (tf )x(tf ) + 0.5 xT (t)Q(t)x(t) + uT (t)R(t)u(t)dt t0

subject to the system dynamics x(t) ˙ = A(t)x(t) + B(t)u(t) and drives the states of system to zero (Regulation). Here x(t0 ) = x0 is given, x(tf ) is free and tf is fixed. Also, Q(t)  0 and R(t) ≻ 0 ∀ t ∈ [t0 , tf ]. i) Solution using CoV technique: This technique comprises of four major steps: • Formulation of Hamiltonian function: The Hamiltonian for the considered problem is given as: 1 1 T x (t)Q(t)x(t) + uT (t)R(t)u(t) + λT (t)[A(t)x(t) + B(t)u(t)] 2 2 where λ(t) is the co-state vector. H(x(t), u(t), λ(t)) =

(3)

• Obtaining Optimal Control law using first order necessary condition: The optimal control law u∗ (t) is obtained as: ∂H =0 ∂u ⇒u∗ (t) = −R−1 (t)B T (t)λ∗ (t)

• Use of State and Co-state Dynamics and assumption of a transformation to connect state and co-state for all t ∈ [t0 , tf ]: The boundary conditions (i.e tf being fixed and x(tf ) being free) lead to the following boundary condition on λ(t): λ∗ (tf ) = F (tf )x∗ (tf ) (4) Next, the following transformation to connect co-state and state is assumed based on equation (4): λ∗ (t) = P (t)x∗ (t)

(5)

which gives the optimal control law as: u∗ (t) = −R−1 (t)B T (t)P (t)x∗ (t) • Obtaining Matrix Differential Riccati Equation: Finally, taking the derivative of equation (5) and substituting the state and costate relations the following matrix differential Riccati equation (MDRE) is obtained which P (t) must satisfy for all t ∈ [t0 , tf ]: P˙ (t) + P (t)A(t) + AT (t)P (t)) + Q(t) − P (t)B(t)R−1 (t)B T (t)P (t) = 0 ii) Solution using HJB equation: The HJB equation for this problem is given as:
∂J ∗ (x∗ (t), t) ∂J ∗ (x∗ (t), t) ∗ , u (t), t = 0 ; ∀t ∈ [t0 , tf ] + H x∗ (t), ∗ ∂t ∂x

(6)

where J ∗ is the optimal cost function, H is the Hamiltonian function as defined in (3) and u∗ (t) is the optimal control law. Also, the boundary condition is given as : J ∗ (x∗ (tf ), tf ) =

1 ∗T x (tf )F (tf )x∗ (tf ) 2

(7)

1 ∗T x (t)P (t)x∗ (t) 2

(8)

Next, a quadratic form of J ∗ (x∗ (t), t) is assumed as: J ∗ (x(t), t) =

where P (t) is a real, symmetric, positive-definite matrix to be determined. Using (8) in the HJB equation, we get: 1 ∗T 1 1 1 x (t)P˙ (t)x∗ (t) + x∗T (t)P (t)A(t)x∗ (t) + x∗T (t)Q(t)x∗ (t) + x∗T (t)AT (t)P (t)x∗ (t) 2 2 2 2 1 ∗T −1 T ∗ − x (t)P (t)B(t)R (t)B (t)P (t)x (t) = 0 2 This equation is valid for any x∗ (t), if: P˙ (t) + Q(t) + P (t)A(t) + AT (t)P (t) − P (t)B(t)R−1 (t)B T (t)P (t) = 0 Finally, P (tf ) = F (tf ) from (7) and (8) and thus the solution is same as that obtained using CoV. To summarize the results : The optimal control law is given by u∗ (t) = −R−1 (t)B T (t)P (t)x(t) where P (t) is the solution of matrix differential riccati equation (MDRE) P˙ (t) + P (t)A(t) + AT (t)P (t) + Q(t) − P (t)B(t)R−1 (t)B T (t)P (t) = 0 with boundary condition P (tf ) = F (tf ). OCP 2. (Finite Horizon Linear Quadratic Tracking(LQT) problem)- Compute an optimal control law u∗ (t) which minimizes the quadratic performance index/cost functional:

h i hZ J = 0.5 eT (tf )F (tf )e(tf ) + 0.5

tf

eT (t)Q(t)e(t) + uT (t)R(t)u(t)dt

t0

where e(t) , z(t) − y(t)

i

subject to the system dynamics x(t) ˙ = A(t)x(t) + B(t)u(t) y(t) = C(t)x(t) such that the output y(t) tracks the desired reference trajectory z(t). Here x(t0 ) = x0 is given, x(tf ) is free and tf is fixed. Also, Q(t)  0 and R(t) ≻ 0 ∀ t ∈ [t0 , tf ]. Proceeding in a way similar to the LQR case, the CoV and HJB equation based approach yield the optimal control law as: u∗ (t) = −R−1 (t)B T (t)P (t)x∗ (t) + R−1 (t)B T (t)g(t) where P (t) and g(t) satisfy:

with

P˙ (t) + P (t)A(t) + AT (t)P (t) − P (t)B(t)R−1 (t)B T (t)P (t) + C T (t)Q(t)C(t) = 0,  T g(t) ˙ + A(t) − B(t)R−1 (t)B T (t)P (t) g(t) + C T (t)Q(t)z(t) = 0 P (tf ) = C T (tf )F (tf )C(tf )

and g(tf ) = C T (tf )F (tf )C(tf ) Note, that in HJB approach the optimal cost function has to be guessed. Although the CoV and HJB based approaches as described above are widely employed for solving the optimal control problems, there are some assumptions associated with these approaches. Specifically, the CoV approach introduces the notion of co-states and uses the assumption that the state and costate functions are related by a linear time varying transformation (5) [6]. A justified reasoning for this assumption does not seem to be available in the literature. Similarly, the HJB based approach requires that the optimal cost function exists, is continuously differentiable and that its gradient with respect to the state is the co-state corresponding to the optimal trajectory. Moreover, generally the HJB equation is difficult to solve and approximate solutions are usually computed. Considering the aforementioned observations, it is desirable to find a solution strategy which does not require (i) the notion of co-states and the resulting assumptions; (ii) the continuous differentiability of optimal cost function; (iii) the assumption that the optimal cost function and co-state are related.

3

A new method for synthesizing optimal control law

In this section, the LQR and LQT problems are solved using the Krotov’s sufficiency conditions.

3.1

Krotov’s Sufficiency Conditions in Optimal Control

The underlying idea behind Krotov’s sufficient conditions for global optimality of control processes is the total decomposition of the original OCP with respect to time using the so-called extension principle [2]. 3.1.1

Extension Principle

Consider a scalar valued functional I(v) defined over a set M (i.e. v ∈ M) and the optimization problemProblem (i): Find v¯ such that d = inf v∈M I(v) where d , I(¯ v ). Instead of solving the Problem (i) directly, another equivalent optimization problem is solved. Let L(v, q) denotes the equivalent representation of the original cost functional, where q is the solving function. Then, a new problem is formulated over N, a super-set of M as: Equivalent Problem (i): Find v¯ such that e = inf v∈N L(v, q) where e , L(¯ v , q). Note that the set N is dependent upon the selection of q. In essence, the extension principle allows to compute the solution of the original problem via an appropriate selection of q and then solving the Equivalent Problem (i). The key point is that solving the equivalent problem can be much easier than solving the original problem [2]. Note that the successful realization of the extension principle relies on a proper selection of q. However, a general method to select q is not known yet. It is necessary to ensure that: I(v) = L(v, q) ∀ v ∈ M so that the optimizer v¯ is actually the optimizer of the original Problem (i) [2].

3.1.2

Application of Extension Principle to Optimal Control Problems

Now, we briefly discuss the application of Extension principle to optimal control problems and provide equivalent representations of these problems as explained by Krotov in [2]. The idea is to use extension principle and decomposition with respect to time t via suitable selection of q to obtain an equivalent representation of GOCP. Theorem 1. (Krotov’s Theorem) For GOCP, let q(x(t), t) be a continuously differentiable solving function. Then, there is an equivalent representation of (1) given as: Jeq (x(t), u(t)) = sf (x(tf )) + q(x0 , 0) +

Z

tf

s(x(t), u(t), t)dt

t0

where ∂q ∂q + l(x(t), u(t), t) + f (x(t), u(t), t) ∂t ∂x sf (x(tf ) , lf (x(tf )) − q(x(tf ), tf )

If x∗ (t), u∗ (t) is an admissible process i.e. x∗ (t) ∈ X and u∗ (t) ∈ U ∀ t ∈ [t0 , tf ] such that s(x(t), u(t), t) ,

s(x∗ (t), u∗ (t), t) =

min

x∈X(t),u∈U(t)

s(x(t), u(t), t) ∀ t ∈ [t0 , tf ]

and sf (x∗ (tf )) = min sf (x) x∈Xf

∗

∗

then (x (t), u (t)) is an optimal process. Here, Xf is the terminal set for admissible x(t) of (2) i.e. if x(t) is admissible then x(tf ) ∈ Xf . Proof. See [2, Section 2.3] for the proof. Although Theorem 1 provides a hint for calculating the global optimal control law and forms a basis for new algorithms to carry out the computation, the abstractness lies in the fact that the selection of q(x(t), t) is not at all trivial [2]. Notice, that q(x(t), t) can be any continuously differentiable function unlike to the optimal cost function used in the solution using HJB equation. For any q(x(t), t), the solution to the equivalent optimization problem given in Theorem 1 is generally computed using sequential methods, one of them is Krotov’s method, where a convergent sequence of admissible process is obtained whose limit is (x∗ (t), u∗ (t)). Such a sequence of processes is called an optimizing sequence [9, 10]. Considering the aforementioned observations, Krotov’s sufficiency conditions are indeed a more general case of HJB approach. Nevertheless, a general method to select q(x(t), t) still remains an open question. Our objective is to compute a direct solution of OCP 1 and OCP 2 using Krotov’s sufficiency conditions. In the following, we use 2J(x(t), u(t), t) as the cost functional instead of J(x(t), u(t), t) and the time variable t is dropped wherever it is required for the sake of simplicity.

3.2

Solution of OCP 1 (LQR problem)

The equivalent optimization problem for OCP 1 is given below.: Equivalent OCP 1. Compute an optimal control law u∗ (t) which (i)

min

(x,u)∈Rn ×Rm

s(x, u, t), ∀t ∈ [t0 , tf ], where s =

∂q ∂t

+

∂q ∂t [Ax

+ Bu] + xT Qx + uT Ru;

(ii) min sf (x), where sf = xT (tf )F (tf )x(tf ) − q(x(tf ), tf ). x∈Xf

The next proposition is one of the main results of the paper, where we propose a suitable solving function which will be useful in computing a direct solution to OCP 1. Proposition 1. For Equivalent OCP 1, let the solving function be chosen as q(x, t) = xT P x where P ≻ 0, ∀t ∈ [t0 , tf ]. Then, the following statements are equivalent: (a) s(x, u, t) and sf (x(tf )) are convex functions in (x, u) and x(tf ) respectively. (b) P satisfies the matrix inequalities:

(9)

(i) P˙ + P A + AT P + Q − P BR−1 B T P  0∀t ∈ [t0 , tf ] (ii) F (tf ) − P (tf )  0 Proof. With q selected as in (9), the function s(x, u, t) becomes: s = xT P˙ x + 2xT P [Ax + Bu] + xT Qx + uT Ru = xT P˙ x + xT P Ax + xT AT P x + 2xT P Bu + xT Qx + uT Ru Adding and subtracting the term xT P BR−1 B T P x, we get s = xT (P˙ + P A + AT P + Q − P BR−1 B)x + xT P Bu + uT B T P x + uT Ru + xT P BR−1 B T P x

(10)

˜ such that Since R ≻ 0, there exists a unique positive definite matrix [14], say R, ˜ 2 = R and R ˜ −1 R ˜ −1 = R−1 R Using (11) and rearranging the terms in (10), we get h i   ˜ +R ˜ −1 B T P x]T [Ru ˜ +R ˜ −1 B T P x s =xT P˙ + P A + AT P + Q − P BR−1 B T P x + Ru

(11)

(12)

Clearly the second term in (12) is strictly convex. Now, s is convex iff the following condition is satisfied P˙ + P A + AT P + Q − P BR−1 B T P  0, ∀t ∈ [to , tf ].

(13)

Moreover, with q(t) as in (9), sf (x(tf )) is given as

Similarly, sf is convex iff

sf = xT (tf )F (tf )x(tf ) − xT (tf )P (tf )x(tf )   = xT (tf ) F (tf ) − P (tf ) x(tf )

(14)

F (tf ) − P (tf )  0.

(15)

Corollary 1. With q(x, t) selected as in Proposition 1, the global optimal control law for OCP 1 is given by: u∗ = R−1 B T P x

(16)

where P ≻ 0, ∀t ∈ [t0 , tf ] is the solution of the matrix differential equation P˙ + P A + AT P + Q − P BR−1 B T P = 0

(17)

P (tf ) = F (tf )

(18)

with the final value

3.3

Solution of OCP 2 (LQT problem)

The equivalent optimization problem for OCP 2 is given below: Equivalent OCP 2. Compute an optimal control law u∗ (t) which (i)

min

(x,u)∈Rn ×Rm

s(x, u, t), ∀t ∈ [t0 , tf ], where s =

∂q ∂t

+

∂q ∂t [Ax

+ Bu] + eT Qe + uT Ru

(ii) min sf (x(tf )), where sf = eT (tf )F (tf )e(tf ) − q(x(tf ), tf ) x∈Xf

Another main result of the paper is given in the next proposition, where along the lines of proposition 1, we propose a suitable solving function, which will be useful in computing the direct solution to OCP 2. Proposition 2. For Equivalent OCP 2, let the solving function be chosen as q(x, u, t) , xT P x − 2g T x + φ where P ≻ 0, ∀t ∈ [t0 , tf ] Then, the following statements are equivalent: (a) s(x, u, t) and sf (x(tf )) are convex functions in (x, u) and x(tf ) respectively;

(19)

(b) P satisfies the following matrix inequalities: i) P˙ + P A + AT P − P BR−1 B T P + C T QC  0 ii) C T (tf )F (tf )C(tf ) − P (tf )  0 Proof. With q(x, t) chosen as in (19), the function s(x, u, t) is given as s = xT P˙ x − 2g˙ T x + φ˙ + (2xT P − 2g T )[Ax + Bu] + eT Qe + uT Ru ˙ = xT (P˙ + P A + AT P + C T QC)x + 2xT P Bu − 2g˙ T x − 2g T Ax − 2g T Bu + z T Qz − 2xT C T Qz + uT Ru + φ(t) Adding and subtracting the terms xT P BR−1 B T P x, g T BR−1 B T g and 2xT P BR−1 B T g, we get s = xT (P˙ + P A + AT P − P BR−1 B T P + C T QC)x − 2xT (g˙ + AT g + C T Qz − P BR−1 B T g) + xT P Bu −2g T Bu − 2xT P BR−1 B T g + uT B T P x + uT Ru + xT P BR−1 B T P x + g T BR−1 B T g ˙ + z T Qz − g T BR−1 B T g +φ(t)

(20)

˜ such that Since R ≻ 0, there exists a unique positive definite matrix [14], say R, ˜ 2 = R and R ˜ −1 R ˜ −1 = R−1 R

(21)

Using (21) in (20) and rearranging the terms, we get     s = xT P˙ + P A + AT P − P BR−1 B T P + C T QC x − 2xT g˙ + Ag + C T Qz − P BR−1 B T g       ˜ +R ˜ −1 B T P x − R ˜ −1 B T g + φ˙ + z T Qz − g T BR−1 B T g (22) ˜ +R ˜ −1 B T P x − R ˜ −1 B T g T Ru + Ru

The third term in (22) is positive definite and strictly convex. The second term is linear in x, and the fourth term is independent of x and u. The first term in (22) is quadratic in x and it is convex iff it is positive semi-definite. Hence, the function s(x, u, t) is convex iff P˙ + P A + AT P − P BR−1 B T P + C T QC  0 ∀t ∈ [t0 , tf ]

(23)

Next, with q(x(t), t) as in (19), sf (x(tf )) is then given as sf = eT (tf )F (tf )e(tf ) − xT (tf )P (tf )x(tf ) + 2g T (tf )x(tf ) − φ(tf )     = xT (tf ) C T (tf )F (tf )C(tf ) − P (tf ) x(tf ) + 2g T (tf ) − 2z T (tf )F (tf )C(tf ) x(tf ) − φ(tf ) + z T (tf )F (tf )z(tf ) (24) Similarly, sf is convex iff

C T (tf )F (tf )C(tf ) − P (tf )  0

(25)

The following corollary synthesizes the optimal control law for OCP 2. Corollary 2. With q(x, t) chosen as in Proposition 2, the global optimal control law for OCP 2 is given by: u∗ = −R−1 B T (P x − g)

(26)

where i) P is the solution of the matrix differential equation P˙ + P A + AT P − P BR−1 B T P + C T QC = 0

(27)

P (tf ) = C T (tf )F (tf )C(tf )

(28)

with boundary condition ii) g is the solution of the vector differential equation g˙ + Ag + C T Qz − P BR−1 B T g = 0

(29)

g(tf ) = C T (tf )F (tf )z(tf )

(30)

with boundary condition and

iii) φ is the solution of differential equation φ˙ + z T Qz − g T BR−1 B T g = 0

(31)

φ(tf ) = z T (tf )F (tf )z(tf )

(32)

with boundary condition

Proof. The third term in (22) is strictly convex and attains a minimum value when   ˜ +R ˜ −1 B T P x − R ˜ −1 B T g = 0 =⇒ u = −R−1 B T P x − g Ru For equivalent OCP 2, from Proposition 2.(b), P is now chosen as P˙ + P A + AT P − P BR−1 B T P + C T QC = 0, ∀t ∈ [t0 , tf ] and C T (tf )F (tf )C(tf ) − P (tf ) = 0. The equivalent OCP 2 is unbounded below [14] unless g˙ + Ag + C T Qz − P BR−1 B T g = 0 and g(tf ) = C T (tf )F (tf )z(tf ). Substituting (26-30) in (22, 24), yields the functions s(x, u, t) and sf (x(tf )) independent of x and x(tf ) respectively. Finally, φ is selected such that the residual terms (i.e. the term independent of x and u in (22) and term independent of x(tf ) in (24)) vanish.

4

Discussions

In this paper, we give some insights on synthesizing an optimal control law for linear systems. The solution to the linear optimal control problem has been widely addressed in the literature using the celebrated CoV/HJB methods. These methods synthesize the global optimal control law, which is unique and requires some forced assumptions. In order to address this issue, we solved the optimal control problem using Krotov’s sufficiency conditions, which does not require the equivalence of co-state and state at all time though they are equal only at t = tf , and the existence of the continuously differentiable optimal cost function. The idea behind Krotov’s formulation is that the original optimal control problem is translated into an another equivalent optimization problem (of functions parametrized in t) utilizing the so-called extension principle. The resulting optimization problem is highly nonlinear, which is generally solved using iterative methods [10, 11] to yield the globally optimal solution. The angle of our attack is to obtain a direct solution instead of iterative methods. This is achieved by imposing convexity conditions on the nonlinear optimization problem. As a byproduct, the selection of Krotov function now becomes very crucial. For linear systems, we propose suitable Krotov functions to obtain the global optimal control law for regulation and tracking problems.

References [1] D. Naidu, Optimal Control Systems. CRC Press, 2002. [2] V. Krotov, Global Methods in Optimal Control Theory. Marcel Dekker, 1995. [3] O. L. Mangasarian, “Sufficient conditions for the optimal control of nonlinear systems.,” SIAM Journal on Control, vol. 4, no. 1, pp. 139–152, 1966. [4] D. E. Kvasov and Y. D. Sergeyev, “Lipschitz global optimization methods in control problems,” Automation and Remote Control, vol. 74, no. 9, pp. 1435–1448, 2013. [5] M. I. Kamien and N. Schwartz, “Sufficient conditions in optimal control theory,” Journal of Economic Theory, vol. 3, no. 2, pp. 207–214, 1971. [6] M. Athans and P. L. Falb, Optimal control: an introduction to the theory and its applications. New York: Dover Publications, 2013. [7] R. W. Beard, G. N. Saridis, and J. T. Wen, “Approximate solutions to the time-invariant Hamilton-JacobiBellman equation,” Journal of Optimization theory and Applications, vol. 96, no. 3, pp. 589–626, 1998.

[8] V. F. Krotov, “Methods of solution of variational problems on the basis of sufficient conditions for absolute minimum. i,” Avtomatika i Telemekhanika, vol. 23, no. 12, pp. 1571–1583, 1962. [9] V. Krotov, “A technique of global bounds in optimal control theory,” Control and Cybernetics, vol. 17.2, no. 3, pp. 2–3, 1988. [10] I. Halperin, G. Agranovich, and Y. Ribakov, “Optimal control of a constrained bilinear dynamic system,” Journal of Optimization Theory and Applications, vol. 174, no. 3, pp. 803–817, 2017. [11] R. A. Rojas and A. Carcaterra, “An approach to optimal semi-active control of vibration energy harvesting based on mems,” Mechanical Systems and Signal Processing, vol. 107, no. 3, pp. 291–316, 2018. [12] M. S. Vinding, I. I. Maximov, Z. Toˇsner, and N. C. Nielsen, “Fast numerical design of spatial-selective RF pulses in MRI using Krotov and quasi-Newton based optimal control methods,” The Journal of Chemical Physics, vol. 137, no. 5, p. 054203 (10pp), 2012. [13] S. G. Schirmer and P. de Fouquieres, “Efficient algorithms for optimal control of quantum dynamics: the Krotov method unencumbered,” New Journal of Physics, vol. 13, no. 7, p. 073029 (35pp), 2011. [14] S. Boyd and L. Vandenberghe, Convex Optimization. UK: Cambridge University Press, 2004.