Optimization Letters DOI 10.1007/s11590-007-0056-3 ORIGINAL PAPER
A multi-parametric programming approach for constrained dynamic programming problems Nuno P. Faísca · Konstantinos I. Kouramas · Pedro M. Saraiva · Berç Rustem · Efstratios N. Pistikopoulos
Received: 19 January 2007 / Accepted: 25 May 2007 © Springer-Verlag 2007
Abstract In this work, we present a new algorithm for solving complex multi-stage optimization problems involving hard constraints and uncertainties, based on dynamic and multi-parametric programming techniques. Each echelon of the dynamic programming procedure, typically employed in the context of multi-stage optimization models, is interpreted as a multi-parametric optimization problem, with the present states and future decision variables being the parameters, while the present decisions the corresponding optimization variables. This reformulation significantly reduces the dimension of the original problem, essentially to a set of lower dimensional multi-parametric programs, which are sequentially solved. Furthermore, the use of sensitivity analysis circumvents non-convexities that naturally arise in constrained dynamic programming problems. The potential application of the proposed novel framework to robust constrained optimal control is highlighted. Keywords Dynamic programming · Constrained multi-stage models · Parametric programming
1 Introduction Multi-stage decision processes have attracted considerable attention in the open literature. With many applications in engineering, economics and finances, theory and
N. P. Faísca · K. I. Kouramas · B. Rustem · E. N. Pistikopoulos (B) Centre for Process Systems Engineering, Imperial College London, London SW7 2AZ, UK e-mail:
[email protected] P. M. Saraiva Gepsi, PSE Group, University of Coimbra, 3030-290 Coimbra, Portugal
123
N. P. Faísca et al.
(a)
(b)
Fig. 1 Multi-stage decision process
algorithms for multi-stage decision problems have been presented in the open literature [4,6]. A typical multi-stage decision making process, involving a discrete-time model and a convex stage-additive cost function, can be posed as follows [2,6]: xk+1 = f k (xk , u k ), xk ∈ X , u k ∈ Uk , k ∈ {0, 1, . . . , N − 1}, J (U ) = g N (x N ) +
N −1
gk (xk , u k ),
(1a) (1b)
k=0
where, k is the index of the time, xk is the state of the system at time k, X ⊆ Rn , u k denotes the optimization (decision) variable at time k, U {u 0 , u 1 , . . . , u N −1 }, Uk ⊆ Rm , f k describes the dynamic behaviour of the system and gk is the cost occurred at time k. Based on a sequence of stage-wise optimal decisions, the system transforms from its original state, x0 , into a final state, x N (as shown in Fig. 1). The set of optimal decisions, u ∗0 , u ∗1 , . . . , u ∗N −1 , and the corresponding path, x1∗ , x2∗ , . . . , x N∗ , optimize a pre-assigned cost function (1b). In other words, if the sequence of decisions is optimal the reward is maximum. Dynamic Programming is well-documented [4] as being a powerful tool to solve this class of optimization problems. Based on the optimality principle, the original problem disassembles into a set of problems of lower dimensionality, thereby significantly reducing the complexity of obtaining the solution. The value function for a general multi-stage optimization problem, as in (1), is given by: Vk (xk ) =
min
µk ,...,µ N
g N (x N ) +
N −1
gi (u i , xi ) ,
(2)
i=k
where u i = µi (xi ) ∈ Ui , and µi (·) is an admissible policy. Applying the optimality principle to Equation (2) results in the following recursive equation [2]: Vk (xk ) = min gk (u k , xk ) + Vk+1 (xk+1 ) . u k ∈Uk
123
(3)
A multi-parametric programming approach for constrained dynamic programming problems
From (3) we conclude that incumbent cost functions are a compound of all future cost functions, previously optimized, and the cost corresponding to the decision taken at the present time. Bellman [4] proved that this methodology solves the original problem to global optimality. The obvious advantage is that at each time step/stage the decision maker only takes decisions corresponding to it, provided that all future stages are optimized up to the incumbent stage. Although dynamic programming is a well-established methodology, a number of limitations can be identified, especially in the presence of hard constraints. As an example, consider the application of dynamic programming to unconstrained linearquadratic regulator control problems [16,19]: u 0 = K 0 ·x0 , u 1 = K 1 ·x1 , . . . , u N −1 = K N −1 · x N −1 , where the control action is set to be admissible, u k ∈ Uk , and K i are real matrices. In the presence of (hard) inequality constraints, non-linear decision laws result, which introduce non-convexities and hence significantly increase the complexity of the implementation as the use of specialised global optimization techniques is required. Borrelli et al. [8] presented an approach to address hard constrained multi-stage problems in a dynamic programming fashion. Based on multi-parametric programming theory [17] and Bellman’s optimality principle, the authors compute, for each stage, the corresponding control law, u k = µk (xk ), using multi-parametric programming algorithms [9,17]. The key idea is to incorporate the conditional piecewise linear function in the cost function of the previous stage, reducing it to a function of only the incumbent stage variables, u k−1 and xk−1 . However, as the objective function at each stage is a piecewise quadratic function of {xk , u k }, overlapping critical regions result, and a parametric global optimization procedure is thus required to obtain the explicit solution. In this work, we present a novel algorithm for the solution of constrained dynamic programs which effectively avoids the need for any global optimization procedure. The algorithm combines the principles of multi-parametric programming [17] and dynamic programming, and can readily be extended to handle uncertainty in the model data [5,11,15]. These developements are described in the following sections.
2 Constrained dynamic programming Consider the last stage of the decision chain depicted in Fig. 1(a), Stage N, and its corresponding optimization problem: min J N (u N −1 , x N −1 ) = g N (x N ) + g N −1 (x N −1 , u N −1 ),
u N −1
max s.t. u min N −1 ≤ u N −1 ≤ u N −1 ,
(4)
x Nmin ≤ x N ≤ x Nmax , x N = f N −1 (x N −1 , u N −1 ), where, x N , x N −1 ∈ Rn and u N −1 ∈ Rm . A Karush–Kuhn–Tucker (KKT) point for (4) satisfies the following system of equations [3]:
123
N. P. Faísca et al.
∇L (u N −1 , λ, µ, x N −1 ) = 0, λi ψi (u N −1 , x N −1 ) = 0, ∀ i = 1, . . . , p, ω j (u N −1 , x N −1 ) = 0, ∀ j = 1, . . . , q, p q λi ψi (u N −1 , x N −1 ) + µ j ω j (u N −1 ), L = J N (u N −1 , x N −1 ) + i=1
(5)
j=1
where λ and µ are vectors of the Lagrange multipliers of the inequalities and equalities of (4), respectively. Since x N −1 is a varying parameter, the solution of (5), u ∗N −1 , instead of being a point, corresponds to an optimal function, u ∗N −1 = µ∗N −1 (x N −1 ). The existence of such function depends on the conditions stated in Theorem 1, where we set x = x N −1 and u = u N −1 . Theorem 1 Basic Sensitivity Theorem [12]: let x0 be a vector of parameter values and (u 0 , λ0 , µ0 ) a KKT triple corresponding to (5), where λ0 is nonnegative and u 0 is feasible in (4). Also assume that (i) strict complementary slackness (SCS) holds, (ii) the binding constraint gradients are linearly independent (LICQ: Linear Independence Constraint Qualification), and (iii) the second-order sufficiency conditions (SOSC) hold. Then, in the neighbourhood of x0 , there exists a unique, once continuously differentiable function, z(x) = [u(x), λ(x), µ(x)], satisfying (5) with z(x0 ) = [u(x0 ), λ(x0 ), µ(x0 )], where u(x) is a unique isolated minimiser for (4), and ⎛ ⎜ ⎜ ⎝
du(x0 ) dx dλ(x0 ) dx dµ(x0 ) dx
⎞ ⎟ ⎟ = − (M0 )−1 N0 , ⎠
(6)
where, M0 and N0 are the jacobian of system (5) with respect to z and x: ⎛
∇2L ⎜ −λ1 ∇ T ψ1 ⎜ ⎜ .. ⎜ . ⎜ Tψ M0 = ⎜ −λ ∇ p p ⎜ ⎜ ∇ T ω1 ⎜ ⎜ .. ⎝ .
∇ψ1 −ψ1
··· ..
∇ψ p
. −ψ p
∇ω1
···
∇ωq
⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟, ⎟ ⎟ ⎟ ⎟ ⎠
∇ T ωq
2 N0 = (∇xu L , −λ1 ∇xT ψ1 , . . . , −λ p ∇xT ψ p , ∇xT ω1 , . . . , ∇xT ωq )T .
Note that M0 is always invertible (non-singular) because the solution for the homogeneous system of M0 is always zero [13, pp. 80–81]. From (6) it is possible to derive a general analytic expression for u N −1 , however, this is obviously limited to all but simplest cases. Nonetheless, Dua et al. [9] have recently proposed an algorithm to solve equation (6) for general convex problems in the entire range of the varying parameters. The algorithm is based on approximations of the non-linear optimal expression by a set of first-order expansions (Corollary 1) valid for different combinations of active constraints.
123
A multi-parametric programming approach for constrained dynamic programming problems
Corollary 1 First-order estimation of u(x), λ(x), µ(x), near x = x0 [13]: under the assumptions of Theorem 1, a first-order approximation of [u(x), λ(x), µ(x)] in a neighbourhood of x0 is, ⎡
⎤ ⎡ ⎤ u(x) u0 ⎣ λ(x) ⎦ = ⎣ λ0 ⎦ + (M0 )−1 · N0 · x + o(||x||), µ(x) µ0
(7)
where (u 0 , λ0 , µ0 ) = [u(x0 ), λ(x0 ), µ(x0 )], M0 = M(x0 ), N0 = N (x0 ), and φ(x) = o(||x||) means that φ(x)/||x|| → 0 as x → x0 . Each piecewise linear approximation, (7), is confined to regions defined by the feasibility and optimality conditions [9]. If ψ˘ corresponds to the non-active constraints, and λ˜ corresponds to the active constraints:
˘ ψ(u(x N −1 ), x N −1 ) ≤ 0 → Feasibility conditions, λ˜ (x N −1 ) ≥ 0 → Optimality conditions.
(8)
Consequently, the explicit expression is given by a conditional piecewise linear function [9,10]: u N −1 = µ∗N −1 (x N −1 ) , (9) or,
⎧ u N −1 = K1N −1 · x N −1 + C1N −1 ⎪ ⎪ ⎪ ⎪ ⎨u 2 2 N −1 = K N −1 · x N −1 + C N −1 . ⎪ ⎪ .. ⎪ ⎪ ⎩ L −1 L −1 · x N −1 + C NN−1 u N −1 = K NN−1
x N −1 ∈ C R 1N −1 , x N −1 ∈ C R 2N −1 ,
(10)
L
−1 x N −1 ∈ C R N N−1 ,
where, KiN −1 and CiN −1 are real matrices derived from (7), and C R iN −1 ⊂ Rn , is a set defined in (8). Note that similarly to Bellman’s procedure [4] we obtain a piecewise linear function for the incumbent states; the difference being the fact that we compute different decision laws in different regions of the states, C R iN −1 , because of the presence of hard constraints and as a result of different combinations of active constraints. Consider then Stage N-1, and its corresponding optimization problem: min J N −1 (u N −2 , x N −2 ) = g N (x N )
u N −2
+ g N −1 (x N −1 , u N −1 ) + g N −2 (x N −2 , u N −2 ), s.t. x N = f N −1 (x N −1 , u N −1 ), x N −1 = f N −2 (x N −2 , u N −2 ),
(11)
max u min N −2 ≤ u N −2 ≤ u N −2 , max x Nmin −1 ≤ x N −1 ≤ x N −1 , x N , x N −1 , x N −2 ∈ Rn , u N −1 , u N −2 ∈ Rm .
123
N. P. Faísca et al.
Proceeding as in the conventional dynamic programming procedure, we can incorporate the expression in (9) into (11), integrate the model information: xk+1 = A · xk + B · u k , express the incumbent cost function in terms of only {x N −2 , u N −2 }, and then optimize manipulating u N −2 . A and B are the system matrices, which correspond to the problem specifications and are considered constant. Note that since (9) is a nonlinear function of x N −1 , it is also a nonlinear function of x N −2 and u N −2 ; i.e., u N −1 = µ∗N −1 ( f N −2 (x N −2 , u N −2 )), therefore, the resulting problem, formulation (11), corresponds to a global optimization problem, which is obviously undesirable. However, note that (11) is convex with respect to u N −1 , u N −2 and x N −2 . We then take advantage of the following Lemma. Lemma 1 If a dynamic system is described by a convex function (1a) and we aim to minimise a convex stage-additive cost function (1b), then the dynamic programming recursive formula for the value function at stage k, Vk (xk ) = min gk (u k , xk ) + Vk+1 (xk+1 ) , u k ∈Uk
implies that the solution computed, satisfies the following inequalities,
(12)
xk ,u ∗k (xk ), u ∗k+1 (xk+1 ), . . . , u ∗N −1 (x N −1 ) ,
Vk (xk , u ∗k (xk ), u ∗k+1 (xk+1 ), . . . , u ∗N −1 (x N −1 )), ≤ Vk (xk , u k (xk ), u ∗k+1 (xk+1 ), . . . , u ∗N −1 (x N −1 )),
(13)
xk+1
Vk+1 (xk , u ∗k (xk ), u ∗k+1 (xk+1 ), . . . , u ∗N −1 (x N −1 )),
≤ Vk+1 (xk , u ∗k (xk ), u k+1 (xk+1 ), . . . , u ∗N −1 (x N −1 )),
(14)
xk+1
where, u i∗ , i ∈ {k, k + 1, . . . , N − 1}, is the optimal value of the optimization variable for the incumbent stage. Proof Since the cost functions at each stage are convex the proof is obvious. Since Vk+1 and Vk are convex functions, their interception will be unique. Thus, one concludes that the optimum at stage k may be obtained comparing the optimal function for stage k and stage k + 1, with u k+1 being a varying parameter at the k th stage optimization problem. In other words, similarly to Problem (4), Problem (11) is recast as a multi-parametric program [9,10,17], with both x N −2 and u N −1 being varying parameters: u N −2 = µ∗N −2 (x N −2 , u N −1 ) , (15) or, ⎧ u N −2 = K1N −2 · x N −2 + H1N −2 · u N −1 + C1N −2 , x N −2 , u N −1 ∈ C R 1N −2 , ⎪ ⎪ ⎪ ⎪ ⎨u 2 2 2 2 N −2 = K N −2 · x N −2 + H N −2 · u N −1 + C N −2 , x N −2 , u N −1 ∈ C R N −2 , (16) . ⎪ .. ⎪ ⎪ ⎪ ⎩ L −2 L −2 L −2 L −2 · x N −2 + H NN−2 · u N −1 + C NN−2 , x N −2 , u N −1 ∈ C R N N−2 , u N −2 = K NN−2
123
A multi-parametric programming approach for constrained dynamic programming problems
Fig. 2 Dynamic programming via multi-parametric programming j
j
j
j
where K N −2 , H N −2 and C N −2 are real matrices, and C R N −2 ⊂ Rn·m . Then, the explicit optimal decision law, u N −2 = µ∗N −2 (x N −2 ), for the incumbent stage, Stage N − 1, is computed by incorporating (10) in (16). On the other hand, the constraints are also reformulated since we have to consider the propagation of constraints along the horizon of decisions in order to get a consistent constraint satisfaction problem [1, see Appendix A]. Due to this fact, the final critical regions are defined as a union of the inequalities from (10) and (16), resulting in (i) realisable sets of inequalities and (ii) empty sets of inequalities. Empty sets are regions of the domain for which no feasible solution exists. Consequently, feasibility tests are performed here, after which empty regions are pruned and a compact set of regions is obtained. Note that in this way the need for global optimization problem is circumvented, as all possible combinations of policies are checked and the ones corresponding to inconsistent paths are systematically pruned. For instance, when incorporating region i into j the feasibility test is formulated as: min 0 x N −2 ,u N −2
s.t u N −1 = KiN −1 · x N −1 + CiN −1 , x N −1 = A · x N −2 + B · u N −2 ,
(17) j
x N −1 ∈ C R 1N −1 , x N −2 , u N −1 ∈ C R N −2 . If (17) does not have a solution, we conclude that it defines an empty region, which is discarded with its corresponding control policy. Figure 2 summarises the steps of the proposed overall algorithm. Remark 1 Within this methodology we may also assume unknown but bounded uncertainty in the data of matrices A,B of the dynamic model as follows: {A = ¯ ≤ δ1 A ≤ 1 | A|} ¯ and {B = B¯ + δ2 B; −2 | B| ¯ ≤ δ2 B ≤ 2 | B|}. ¯ A¯ + δ1 A; −1 | A| Hence, a previous step is required to the algorithm in Fig. 2. For the linear model and path constraints, the original optimization problem is recast by introducing the following constraints, as suggested in [5,11,15], in order to immunise the solution to uncertainty: min max ≤ Axk + Bu k ≤ xk+1 , xk+1
(18)
123
N. P. Faísca et al. max max A · xk +1 · |A| · |xk |+ B · u k + 2 · |B| · |u k | ≤ xk+1 +δ · max[1, |xk+1 |],
(19)
−A · xk + 1 · | − A| · |xk | + (−B) · u k + 2 · | − B| · |u k | min min ≤ −xk+1 + δ · max[1, |xk+1 |].
(20)
Details of the application of the proposed algorithm to robust control can be found in a forthcoming publication [14]. 3 Illustrative example In this section we revisit a popular optimal control problem [7,8,17,18]: min J {U, x} = U
x N
· P · xN +
N −1
[xk · Q · xk + u k · R · u k ],
k=0
s.t. xk+1 = A · xk + B · u k , k = 0, 1, . . . , N − 1, −2 ≤ u k ≤ 2,
k = 0, 1, . . . , N − 1,
(21a) (21b)
where, xk ∈ R2 , u k ∈ R, 0.7326 −0.0861 0.0609 N = 2; A = ; B= ; 0.1722 0.9909 0.0064 1.8588 1.2899 1 0 P= ; Q= ; R = 0.01. 1.2899 6.7864 0 1 Following the steps of the methodology proposed in Fig. 2: Step 1. Second stage—recast the second stage optimization problem as a multi-parametric program with x1 being the parameters: min J N = x2 · P · x2 + x1 · Q · x1 + u 1 · R · u 1 , u1
s.t. x2 = A · x1 + B · u 1 , −2 ≤ u 1 ≤ 2.
(22a) (22b) (22c)
A suitable multi-parametric programming algorithm [9,17] can be used to obtain its solution, resulting in the decision law: u 1 = f (x1 ), which comprises the critical regions listed in Table 1 and depicted in Fig. 3. Substituting the model information, xk+1 = Axk + Bu k (for x1 ), we obtain the critical regions listed in Table 2. Step 2. First stage—recast the first stage optimization problem as a multi-parametric program with x0 and u 1 being the parameters: min J N = x2 · P · x2 + x1 · Q · x1 + u 1 · R · u 1 + u 0 · R · u 0 , u1
123
(23a)
A multi-parametric programming approach for constrained dynamic programming problems Table 1 Explicit solution of u 1 = f (x1 ), Stage 2
Control action
Critical region
1
u1 = 2
+0.9961 · x11 + x12 ≤ −0.3292
2
u 1 = −6.051 · x11 −6.074 · x12
−0.9961 · x11 − x12 ≤ 0.3292
u 1 = −2
−0.9961 · x11 − x12 ≤ −0.3292
3
+0.9961 · x11 + x12 ≤ 0.3292
3 Feasible Region Fragments
4 CR001 CR002 CR003
3 2
x21
1 0 −1 −2 −3 −4 −4
−3
−2
−1
0
1
2
3
4
x11 Fig. 3 Map of critical regions for the second stage
Table 2 Explicit solution of u 1 = f (x0 , u 0 ), Stage 2
Control action
Critical region
1
u1 = 2
0.9019 · x01 + 0.9051 · x02 +
2
u 1 = −4.5376 · x01 −0.0809 · x02
+0.0671 · u 0 ≤ −0.3292 −0.90194 · x01 − 0.90514 · x02 − −0.067062 · u 0 ≤ 0.3292
−0.3723 · u 0 0.90194 · x01 + 0.90514 · x02 + +0.067062 · u 0 ≤ 0.3292 3
u 1 = −2
−0.90194 · x01 − 0.90514 · x02 + −0.067062 · u 0 ≤ −0.3292
s.t.
x2 = A · x1 + B · u 1 ,
(23b)
x1 = A · x0 + B · u 0 , −2 ≤ u 0 ≤ 2.
(23c) (23d)
123
N. P. Faísca et al. Table 3 Explicit solution of u 0 = f (x0 , u 1 ), Stage 1 Control action
Critical region
1
u0 = 2
0.9962 · x01 + x02 + 0.0490 · u 1 ≤ −0.2805
2
u 0 = −7.104 · x01 − 7.131x02 − 0.3494u 1
−0.9962 · x01 − x02 − 0.0490 · u 1 ≤ 0.2805 +0.9962x01 + x02 + 0.0490u 1 ≤ 0.2805
3
−0.9962x01 − x02 − 0.0490u 1 ≤ −0.2805
u 0 = −2
Again, using a suitable multi-parametric programming algorithm [9,17], we obtain the explicit decision law: u 0 = f (x0 , u 1 ), listed in Table 3. Step 3. Incorporate decisions u 1 , Table 2, into u 0 , Table 3, and express u 0 as function only of the incumbent state-space, x0 . After performing feasible tests in each of the nine generated regions we obtain results depicted in Fig. 4 and listed in Table 4. Note that each critical region in Fig. 4 corresponds to a different policy, however, many regions may have the same identical first-stage optimal decision, u 0 . In the example above, Table 4, only 3 different first-stage optimal decisions were identified. The implication of this in a closed-loop control implementation strategy, where only the first-stage decisions are updated, is that a very significant reduction of the number of critical regions (control laws) can take place, by merging the adjacent regions with identical first-stage control actions. In order to test the algorithm in different situations we have performed two independent modifications in the original illustrative example: (i) we make N = 100 and (ii) we introduce a path constraint in the first stage [18]: −0.5 ≤ x11 , x12 . In both cases 5 Feasible Region Fragments 2 CR001 CR002 CR003 CR004 CR005
1.5 1
x21
0.5 0 −0.5 −1 −1.5 −2 −2
−1.5
−1
−0.5
0
x11 Fig. 4 Final map of critical regions
123
0.5
1
1.5
2
A multi-parametric programming approach for constrained dynamic programming problems Table 4 Map of control policies’ regions 1 2
3
4
5
Control action
Critical region
u0 = 2
0.9965 · x01 + x02 ≤ −0.5119
u1 = 2 u0 = 2
−2 ≤ x01 , x02 −0.9965 · x01 − x02 ≤ 0.5119
u 1 = −5.479 · x01
0.9961 · x01 + x02 ≤ −0.3292
−5.498 · x02 − 0.8147 u 0 = −6.051 · x01 −6.074 · x02 u 1 = −3.014 · x01 −3.024 · x02
−2 ≤ x01 , x02 0.9961 · x01 + x02 ≤ 0.3292
u 0 = −2
+0.9965 · x01 + x02 ≤ 0.5119
u 1 = −5.479 · x01 −5.498 · x02 + 0.8147
−0.9961 · x01 − x02 ≤ −0.3292
u 0 = −2
−0.9965 · x01 − x02 ≤ −0.5119
u 1 = −2
x01 , x02 ≤ 2
−0.9961 · x01 − x02 ≤ 0.3292 −2 ≤ x01 , x02 ≤ 2
x01 , x02 ≤ 2
2 1.5 1
x20
0.5 0 −0.5 −1 −1.5 −2 −2
−1.5
−1
−0.5
0
0.5
1
1.5
2
x10 Fig. 5 Maps of feasible regions
the methodology was successfully applied, and the results are depicted in Figs. 5 and 6, respectively. 4 Computational complexity In this section, an analysis of the computational complexity of the proposed algorithm is presented and comparisons are performed with the direct solution of a multiparametric programming problem.
123
N. P. Faísca et al. 2 CR001 CR002 CR003 CR004 CR005 CR006 CR007 CR008 CR009
1.5 1
x20
0.5 0 −0.5 −1 −1.5 −2 −2
−1.5
−1
−0.5
0
0.5
1
1.5
2
x10 Fig. 6 Maps of feasible regions
We assume that all input and output variables, at each step, are bounded. We first consider the solution of the multi-stage problem as a single, large scale multiparametric program. For a problem with m optimization variables and N stages, the total number of inequality constraints, p Q P , is: p Q P = N · (2 · m + 2 · n),
(24)
The maximum number of regions, ηr , is bounded by [9]: ηr ≤
η−1
k!N k · (2 · m + 2 · n)k ,
(25)
k=0
where η
(2 · m · N + 2 · n · N )! . (2 · m · N + 2 · n · N − i)!i!
(26)
This can result in a rapid increase in the number of regions as N increases. However, in our proposed algorithm the dependence on N is eliminated, i.e., p D P = 2 · m + 2 · n ⇒ ηr ≤
η−1
k!(2 · m + 2 · n)k ,
(27)
k=0
where η
(2 · m + 2 · n)! . (2 · m + 2 · n − i)!i!
(28)
In other words, the large-scale multi-parametric program in our case is disassembled into a set of smaller problems, with the extra requirement of additional, albeit almost
123
A multi-parametric programming approach for constrained dynamic programming problems Table 5 Complexity comparison analysis for Example 1
Algorithm
mp-QP
mp-DP
Example 1
Horizon length, N = 2
# mp problems
1
2
# optimization variables
2
1
# inequalities
4
2
Horizon length, N = 100 # mp problems
1
100
# optimization variables
100
1
# inequalities
200
2
negligible computational cost, algebraic manipulations and feasibility tests. Note also that the computational performance of the proposed algorithm can be further improved by performing parallel computations. Table 5 illustrates these points. While for small-scale problems (N = 2) the computational advantages are not clear, for large-scale problems (N = 100) the computational savings are of several orders of magnitude. Comparisons of the proposed algorithm to the one presented in [7] are not meaningful, as their work requires solution of global optimization problems and involves a larger number of intermediate multi-parametric programs, naturally resulting in much higher computational requirements. 5 Conclusions We presented the main steps of a novel multi-parametric programming approach for the solution of general, constrained convex multi-stage problems. Through a literature example of optimal control problems, we highlighted how (i) we can use recently proposed multi-parametric programming theory and algorithms [17] to efficiently address constrained dynamic programming procedures, used in the context of multistage formulations, and (ii) we can avoid any need for global optimization methods by carefully posing and conducting feasibility tests, based on sensitivity analysis of the obtained parametric solutions. The work presented here establishes the foundations towards a comprehensive general theory for robust optimal control, which is described in a forthcoming publication. Acknowledgments Financial support from EPSRC (GR/T02560/01) and Marie Curie European Project PRISM (MRTN-CT-2004-512233) is gratefully acknowledged.
Appendix A Definition 1 Consistency of a Constraint Satisfaction Problem (CSP) [1]: consider a finite sequence of variables X := x1 , . . . , xn with respective domains D1 , . . . , Dn ,
123
N. P. Faísca et al.
together with a finite set C of constraints, each on a sub sequence of X . We write such CSP as C; DE , where DE := x1 ∈ D1 , . . . , xn ∈ Dn and call each construct of the form x ∈ D a domain expression. We now define the crucial notion of a solution to a CSP. Intuitively, a solution to a CSP is a sequence of legal values for all of its variables such that all its constraints are satisfied. More precisely, consider a CSP C; DE with DE := x1 ∈ D1 , . . . , xn ∈ Dn . We say that n-tuple (d1 , . . . , dn ) ∈ D1 × · · · × Dn satisfies a constraint C ∈ C on the variables xi1 , . . . , xim if D1 × · · · × Dn is a solution to C; DE if it satisfies every constraint C ∈ C. If a CSP has solution, we say that it is consistent and otherwise we say that it is inconsistent. References 1. Apt, Krzysztof, R.: Principles of Constraint Programming. Cambridge University Press, Cambridge (2003) 2. Ba¸sar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory. Academic, London (1982) 3. Bazaraa, M.S., Shetty, C.M.: Nonlinear Programming—Theory and Algorithms. Wiley, New York (1979) 4. Bellman, R.: Dynamic Programming. Dover Publications, Mineola (2003) 5. Ben-Tal, A., Nemirovski, A.: Robust solutions of linear programming problems contaminated with uncertain data. Math. Program. Ser. A 88, 411–424 (2000) 6. Bertsekas, D.P.: Dynamic Programming and Optimal Control, 3rd edn. Athena Scientific, Belmont (2005) 7. Borrelli, F., Bemporad, A., Morari, M.: Geometric algorithm for multiparametric linear programming. J. Optim. Theory Appl. 118(3), 515–540 (2003) 8. Borrelli, F., Batoti´c, M., Bemporad, A., Morari, M.: Dynamic programming for constrained optimal control of discrete-time linear hybrid systems. Automatica 41, 1709–1721 (2005) 9. Dua, V., Bozinis, A., Pistikopoulos, E.N.: A multiparametric programming approach for mixed-integer quadratic engineering problems. Comput. Chem. Eng. 26, 715–733 (2002) 10. Dua, V., Papalexandri, K.P., Pistikopoulos, E.N.: Global optimization issues in multiparametric continuous and mixed-integer optimization problems. J. Glob. Optim. 30, 59–89 (2004) 11. El-Ghaoui, L., Lebret, H.: Robust solutions to least-square problems with uncertain data matrices. SIAM J. Matrix Anal. Appl. 18, 1035–1064 (1997) 12. Fiacco, A.V.: Sensitivity analysis for nonlinear programming using penalty methods. Math. Program. 10, 287–311 (1976) 13. Fiacco, A.V.: Introduction to Sensitivity and Stability Analysis in Nonlinear Programming. Academic, New York (1983) 14. Kouramas, K.I., Faísca, N.P., Rustem, B., Pistikopoulos, E.N.: Design of robust parametric controllers for constrained multi-stage optimization problems (to be submitted to Automatica) (2007) 15. Lin, X., Janak, S.L., Floudas, C.A.: A new robust optimization approach for scheduling under uncertainty: I. Bounded uncertainty. Comput. Chem. Eng. 28(6–7), 1069–1085 (2004) 16. Mayne, D.Q., Rakovi´c, S.V., Findeisen, R., Allgöwer, F.: Robust output feedback model predictive control of contrained linear systems. Automatica 7(42), 1217–1222 (2006) 17. Pistikopoulos, E.N., Georgiadis, M.C., Dua, V.: Multi-parametric programming: theory, algorithms, and applications, vol. 1. Wiley-VCH, Weinheim (2006) 18. Pistikopoulos, E.N., Bozinis, N.A., Bemporad, A., Morari, M.: On-line optimization via off-line parametric optimization tools. Comput. Chem. Eng. 24, 183 (2000) 19. Rawlings, J.B.: Tutorial: model predictive control technology. In: Proceedings of the American Control Conference, pp. 662–676. San Diego, USA (1999)
123
