Robust Discrete Optimization Under Ellipsoidal Uncertainty Sets Dimitris Bertsimas
Melvyn Sim y
March, 2004
Abstract We address the complexity and practically ecient methods for robust discrete optimization under ellipsoidal uncertainty sets. Specically, we show that the robust counterpart of a discrete optimization problem with correlated objective function data is NP -hard even though the nominal problem is polynomially solvable. For uncorrelated and identically distributed data, however, we show that the robust problem retains the complexity of the nominal problem. For uncorrelated, but not identically distributed data we propose an approximation method that solves the robust problem within arbitrary accuracy. We also propose a Frank-Wolfe type algorithm for this case, which we prove converges to a locally optimal solution, and in computational experiments is remarkably eective. Finally, we propose a generalization of the robust discrete optimization framework we proposed earlier that (a) allows the key parameter that controls the tradeo between robustness and optimality to depend on the solution and (b) results in increased exibility and decreased conservatism, while maintaining the complexity of the nominal problem.
Boeing Professor of Operations Research, Sloan School of Management and Operations Research Center, Massachusetts Institute of Technology, E53-363, Cambridge, MA 02139,
[email protected]. The research of the author was partially supported by the Singapore-MIT alliance. y NUS Business School, National University of Singapore,
[email protected].
1
1 Introduction Robust optimization as a method to address uncertainty in optimization problems has been in the center of a lot of research activity. Ben-Tal and Nemirovski 1, 2, 3] and El-Ghaoui et al. 7, 8] propose e cient algorithms to solve certain classes of convex optimization problems under data uncertainty that is described by ellipsoidal sets. Kouvelis and Yu 15] propose a framework for robust discrete optimization, which seeks to nd a solution that minimizes the worst case performance under a set of scenarios for the data. Unfortunately, under their approach, the robust counterpart of a polynomially solvable discrete optimization problem can be N P -hard. Bertsimas and Sim 5, 6] propose an approach in solving robust discrete optimization problems that has the exibility of adjusting the level of conservativeness of the solution while preserving the computational complexity of the nominal problem. This is attractive as it shows that adding robustness does not come at the price of a change in computational complexity. Ishii et. al. 12] consider solving a stochastic minimum spanning tree problem with costs that are independently and normally distributed leading to a similar framework as robust optimization with an ellipsoidal uncertainty set. However, to the best of our knowledge, there has not been any work or complexity results on robust discrete optimization under ellipsoidal uncertainty sets. It is thus natural to ask whether adding robustness in the cost function of a given discrete optimization problem under an ellipsoidal uncertainty set leads to a change in computational complexity and whether we can develop practically e cient methods to solve robust discrete optimization problems under ellipsoidal uncertainty sets. Our objective in this paper is to address these questions. Speci cally our contributions include:
(a) Under a general ellipsoidal uncertainty set that models correlated data, we show that the robust
counterpart can be N P -hard even though the nominal problem is polynomially solvable in contrast with the uncertainty sets proposed in Bertsimas and Sim 5, 6].
(b) Under an ellipsoidal uncertainty set with uncorrelated data, we show that the robust problem
can be reduced to solving a collection of nominal problems with dierent linear objectives. If the distributions are identical, we show that we only require to solve r + 1 nominal problems, where r is the number of uncertain cost components, that is in this case the computational complexity is preserved. Under uncorrelated data, we propose an approximation method that solves the robust problem within an additive . The complexity of the method is O((ndmax)1=4;1=2 ), where dmax is the largest number in the data describing the ellipsoidal set. We also propose a Frank-Wolfe 2
type algorithm for this case, which we prove converges to a locally optimal solution, and in computational experiments is remarkably eective. We also link the robust problem with uncorrelated data to classical problems in parametric discrete optimization. (c) We propose a generalization of the robust discrete optimization framework in Bertsimas and Sim 5] that allows the key parameter that controls the tradeo between robustness and optimality to depend on the solution. This generalization results in increased exibility and decreased conservatism, while maintaining the complexity of the nominal problem.
Structure of the paper. In Section 2, we formulate robust discrete optimization problems under
ellipsoidal uncertainty sets (correlated data) and show that the problem is N P -hard even for nominal problems that are polynomially solvable. In Section 3, we present structural results and establish that the robust problem under ball uncertainty (uncorrelated and identically distributed data) has the same complexity as the nominal problem. In Sections 4 and 5, we propose approximation methods for the robust problem under ellipsoidal uncertainty sets with uncorrelated but not identically distributed data. In Section 6, we present the generalization of the robust discrete optimization framework in Bertsimas and Sim 5]. In Section 7, we present some experimental ndings relating to the computation speed and the quality of robust solutions. The nal section contains some concluding remarks.
2 Formulation of Robust Discrete Optimization Problems A nominal discrete optimization problem is: minimize
c0 x
subject to
x 2 X
(1)
with X f0 1gn. We are interested in problems where each entry c~j j 2 N = f1 2 : : : ng is uncertain and described by an uncertainty set C . Under the robust optimization paradigm, we solve minimize max c~ x ~ 0
c2C
subject to
x 2 X:
(2)
Writing c~ = c + s~, where c is the nominal value and the deviation s~ is restricted to the set D = C ; c, Problem (2) becomes: minimize c x + (x) (3) subject to x 2 X 0
3
where (x) = maxs~ 2D s~ x. Special cases of Formulation (3) include: 0
(a) (b)
D
= fs : s~j 2 0 dj ]g, leading to (x) = d x. 0
= fs : k;1=2sk2 g that models ellipsoidal uncertainty sets proposed by Ben-Tal and Nep mirovski 1, 2, 3] and El-Ghaoui et al. 7, 8]. It easily follows that (x) = x0 x, where is the covariance matrix of the random cost coe cients. For the special case that = diag(d1 : : : dn ), qP p i.e., the random cost coe cients are uncorrelated, we obtain that (x) = j2N dj x2j = d x: D
0
(c)
P
= fs : 0 sj dj 8j 2 J k2N ds ;g proposed in Bertsimas and Sim 6]. It follows that P in this case (x) = maxfS :jSj=;SJg j2S dj xj , where J is the set of random cost components. Bertsimas and Sim 6] show that Problem (3) reduces to solving at most jJ j + 1 nominal problems for dierent cost vectors. In other words, the robust counterpart is polynomially solvable if the nominal problem is polynomially solvable. Under models (a) and (c), robustness preserves the computational complexity of the nominal problem. Our objective in this paper is to investigate the price (in increased complexity) of robustness under ellipsoidal uncertainty sets (model (b)) and propose eective algorithmic methods to tackle models (b), (c). Our rst result is unfortunately negative. Under ellipsoidal uncertainty sets with general covariance matrices, the price of robustness is an increase in computational complexity. The robust counterpart may become N P -hard even though the nominal problem is polynomially solvable. D
k
k
p
Theorem 1 The robust problem (3) with (x) =
x0 x
(Model (b)) is N P -hard, for the following classes of polynomially solvable nominal problems: shortest path, minimum cost assignment, resource scheduling, minimum spanning tree.
Proof : Kouvelis and Yu 15] prove that the problem minimize maxfc1 x c2 xg 0
subject to
0
(4)
x 2 X
is N P -hard for the polynomially solvable problems mentioned in the statement of the theorem. We p show a simple transformation of Problem (4) to Problem (3) with (x) = x0 x as follows:
c x + c x c x ; c x c x + c x c x ; c x maxfc1 x c2 xg = max 1 2 2 + 1 2 2 1 2 2 ; 1 2 2 c x ; c x c x ; c x c x+c x 0
0
0
0
0
+ max
0
0
0
0
0
0
=
0
1
0
2
2
1
4
2
0
2 ; 1
0
2
2
0
c01 x + c02 x
c1 x ; c2 x + 2 2 q = c1 x +2 c2 x + 12 x0 (c1 ; c2 )(c1 ; c2 )0x: =
0
0
0
0
p
The N P -hard Problem (4) is transformed to Problem (3) with (x) = x0 x, c = (c1 + c2 )=2, p = 1=2 and = (c1 ; c2 )(c1 ; c2 )0. Thus, Problem (3) with (x) = x0x is N P -hard. We next would like to propose methods for model (b) with = diag(d1 : : : dn). We are thus naturally led to consider the problem G
= xmin c x + f (d x) 2X 0
(5)
x
models ellipsoidal uncertainty sets with
0
p
with f () a concave function. In particular, f (x) = uncorrelated random cost coe cients (model (b)).
3 Structural Results We rst show that Problem (5) reduces to solving a number of nominal problems (1). Let W = fd x j x 2 f0 1gng and (w) be a subgradient of the concave function f () evaluated at w, that is, f (u) ; f (w) (w)(u ; w) 8u 2 R: If f (w) is a dierentiable function and f 0 (0) = 1, we choose 0
8 > < f 0(w) if w 2 W nf0g (w) = > : f (dmindmin);f (0) if w = 0,
where dmin = minfj : d >0g dj . j
Theorem 2 Let
( ) = xmin (c + (w)d)0x + f (w) ; w (w) 2X
Z w
(6)
and w = arg minw2W Z (w). Then, w is an optimal solution to Problem (5) and G = Z (w ).
Proof : We rst show that G minw2W Z (w): Let x be an optimal solution to Problem (5) and w
= d x 2 W . We have 0
G
=
c0 x + f (d0x ) = c0 x + f (w
) = (c + (w )d)0 x + f (w ) ; w (w )
xmin (c + (w )d)0x + f (w ) ; w (w ) = Z (w ) w2W min Z (w): 2X
5
Conversely, for any w 2 W , let yw be an optimal solution to Problem (6). We have ( ) = (c + (w)d)0yw + f (w) ; w (w)
Z w
=
c0 yw + f (d0yw ) + (w)(d0yw ; w) ; (f (d0yw ) ; f (w))
c0 yw + f (d0yw )
(7)
xmin c x + f (d x) = G 2X 0
0
where inequality (7) for w 2 W nf0g follows, since (w) is a subgradient. To see that inequality (7) follows for w = 0 we argue as follows. Since f (v ) is concave and v dmin 8v 2 W nf0g, we have (
f dmin
) v ;vdmin f (0) + dmin f (v ) 8v 2 W nf0g: v
Rearranging, we have ( ) ; f (0) f (dmin) ; f (0) = (0) 8v 2 W nf0g v d
f v
min
leading to (0)(d yw ; 0) ; (f (d yw ) ; f (0)) 0. Therefore G = minw2W Z (w). Note that when dj = 2, then W = f0 2 : : : n 2g, and thus jW j = n + 1, In this case, Problem (5) reduces to solving n + 1 nominal problems (6), i.e., polynomial solvability is preserved. Speci cally, qP p for the case of an ellipsoidal uncertainty set = 2I , leading to (x) = j 2x2j = e x, we derive explicitly the subproblems involved. 0
0
0
Proposition 1 Under an ellipsoidal uncertainty set with (x) = G
p
e0 x,
= w=0min Z (w) 1:::n
8 p
0 > < minx2X c + 2pw e x + 2 w if w = 1 : : : n Z (w) = > : minx2X (c + e)0x if w = 0:
where
Proof : With (x) =
p
e0 x, we have f (w) =
p
(8) p
. Substituting (w) = f 0 (w) = =(2 w) 8 w 2 W nf0g and (0) = (f (dmin) ; f (0))=dmin = f (1) ; f (0) = to Eq. (6) we obtain Eq. (8). An immediate corollary of Theorem 2 is to consider a parametric approach as follows: w
Corollary 1 An optimal solution to Problem (5) coincides with one of the optimal solutions to the parametric problem:
minimize (c + d)0 x subject to
x 2 X
for 2 (e d) (0)]. 0
6
(9)
This establishes a connection of Problem (5) with parametric discrete optimization (see Gus eld 11], Hassin and Tamir 13]). It turns out that if X is a matroid, the minimal set of optimal solutions to Problem (9) as varies is polynomial in size, see Eppstein 9] and Fern et. al. 10]. For optimization over a matroid, the optimal solution depends on the ordering of the cost components. Since, as varies, ; it is easy to see that there are at most n2 + 1 dierent orderings, the corresponding robust problem is also polynomially solvable. For the case of shortest paths, Karp and Orlin 14] provide a polynomial time algorithm using the parametric approach when all dj 's are equal. In contrast, the polynomial reduction in Proposition 1 applies to all discrete optimization problems. More generally, jW j dmaxn with dmax = maxj dj . If dmax n for some xed , then Problem (5) reduces to solving n (n + 1) nominal problems (6). However, when dmax is exponential in n, an approach that enumerates all elements of W does not preserve polynomiality. For this reason, as well as deriving more practical algorithms even in the case that jW j is polynomial in n we develop in the next section new algorithms.
4 Approximation via Piecewise Linear Functions In this section, we develop a method for solving Problem (5) that is based on approximating the function f () with a piecewise linear concave function. We rst show that if f () is a piecewise linear concave function with a polynomial number of segments, we can also reduce Problem (5) to solving a polynomial number of subproblems.
Proposition 2 If f (w) w 2 0 e d] is a continuous piecewise linear concave function of k segments, 0
then
min (c + j d)0x
(10)
x2X
where j is the gradient of the jth linear piece of the function f ().
Proof : The proof follows directly from Theorem 2 and the observations that if f (w) w 2 0 e d] is 0
a continuous piecewise linear concave function of k linear pieces, the set of subgradients of each of the linear pieces constitutes the minimal set of subgradients for the function f . We next show that approximating the function f () with a piecewise linear concave function leads to an approximate solution to Problem (5). 7
Theorem 3 For W = w w] such that d x 2 W 8x 2 X , let g(w), w 2 W be a piecewise linear concave 0
function approximating the function f (w) such that ;1 f (w) ; g (w) 2 with 1 2 0: Let xH be an optimal solution of the problem:
minimize
c0 x + g (d0x)
subject to
x2X
(11)
and let GH = c xH + f (d xH). Then, 0
0
G
GH G + 1 + 2 :
Proof : We have that G
= xmin fc x + f (d x)g 2X 0
0
= c xH + f (d xH )
GH
c0 xH + g (d0xH ) + 2
0
0
= xmin fc x + g (d x)g + 2 2X 0
0
xmin fc x + f (d x)g + 1 + 2 2X 0
=
G
(12)
0
(13)
+ 1 + 2
where inequalities (12) and (13) follow from ;1 f (w) ; g (w) 2 . We next apply the approximation idea to the case of ellipsoidal uncertainty sets. Speci cally, we p approximate the function f (w) = w in the domain w w] with a piecewise linear concave function g (w) satisfying 0 f (w) ; g (w) using the least number of linear pieces.
Proposition 3 For > 0, w0 given, let = = and for i = 1 : : : k let 8 sp !2 9 < =2 w 1 1 0 2 wi = :2 i + 2 + 4 ; 2 :
(14)
Let g (w) be a piecewise linear concave function on the domain w 2 w0 wk ], with breakpoints (w g (w)) 2 f(w0 pw0) : : : (wk pwk )g. Then, for all w 2 w0 wk ] p
0
w
; g (w) :
8
Proof : Since at the breakpoints wi, g(wi) = pwi, g(w) is a concave function with g(w) pw 8w 2
w0 wk ]. For w 2 wi;1 wi], we have
p pwi ; pwi;1 (w ; wi;1 ) w ; g (w) = w ; wi;1 + w ; w i i;1 ( ) p
p
=
The maximum value of p
w
p
w
p
w
;
p
; g (w) is attained at
; g (w)
(p
w
; pw
; wi;1 p w + w
;p
wi;1
p
w
i;1
i
:
= (pwi + pwi;1)=2. Therefore,
w
; wi;1 p i;1 ; p w + w w
)
i i;1 8p 9 p p w + w ;1 2 > > < wi ; pwi;1 ; w i;1 = 2 = > ; pw + pw > 2 i i;1 : pw +3pw ;1 pw ;pw ;1 9 8 < pwi ; pwi;1 = 2 2 = : ; p p 2 wi + wi;1 i
i
i
i
i
i
(pwi ; pwi;1 )2 = 4(pw + pw ) i i;1 = =
(15)
where Eq. (15) follows by substituting Eq. (14). Since max
w2wi;1 wi ]
pw ; g(w) =
the proposition follows. Propositions 2, 3 and Theorem 3 lead to Algorithm 1.
Algorithm 1 Approximation by piecewise linear concave functions. Input: c d w w , f (x) = px and a routine that optimizes a linear function over the set X f0 1gn.
Output: A solution xH 2 X for which G c xH +f (d xH ) G + where G = minx2X c x+f (d x): Algorithm: 0
0
1. (Initialization) Let = = Let w0 = w Let 3 0s 2s p 1 s p w 1 1 w 1 k=6 66 2 + 4 ; 2 + 4 777 = O @ (ndmax) 4 A
9
0
0
where dmax = maxj dj and for i = 1 : : : k let
8 sp !2 9 =2 < w 1 1 2 wi = :2 i + 2 + 4 ; 2 :
2. For i = 1 : : : k solve the problem
!0
Zi = min c + p d p x2X wi + wi;1 Let xi be an optimal solution to Problem (16). 3. Output GH = Zi = mini=1:::k Zi and xH = xi
(16)
x:
:
Theorem 4 Algorithm 1 nds a solution xH 2 X for which G c xH + f (d xH ) G + : 0
0
Proof : Using Proposition 3 we nd a piecewise linear concave function g(w) that approximates within p
a given tolerance > 0 the function w. From Proposition 2 and since the gradient of the ith segment of the function g (w) for w 2 wi;1 wi] is pw ; pw i i;1 = : i = p p wi ; wi;1 wi + wi;1 we solve the Problems for i = 1 : : : k
!0 Zi = min c + p d x p x2X wi + wi;1 Taking GH = mini Zi and using Theorem 3 it follows that Algorithm 1 produces a solution within . Although the number of subproblems solved in Algorithm 1 is not polynomial with respect to the bit size of the input data, the computation involved is reasonable from a practical point of view. For example, in Table 1 we report the number of subproblems we need to solve for = 4, as a function of Pn d . and d e = j =1 j 0
5 A Frank-Wolfe Type Algorithm A natural method to solve Problem (5) is to apply a Frank-Wolfe type algorithm, that is to successively linearize the function f ().
Algorithm 2 The Frank-Wolfe type algorithm. Input: c d , 2 (d e) (0)], f (w), (w) and a routine that optimizes a linear function over the 0
set X f0 1gn. Output: A locally optimal solution to Problem (5).
Algorithm:
10
d0 e
k
0.01 0.01 0.01 0.01 0.001 0.001 0.001 0.001
10 100 1000 10000 10 100 1000 10000
25 45 80 121 80 141 251 447
Table 1: Number of subproblems, k as a function of the desired precision , size of the problem d e and = 4. 0
1. (Initialization) k = 0 x0 := arg miny2X (c + d)0y 2. Until d xk+1 = d xk , xk+1 := arg miny2X (c + (d xk )d)0y : 0
0
0
3. Output xk+1 :
We next show that Algorithm 2 converges to a locally optimal solution.
Theorem 5 Let x, y and z be optimal solutions to the following problems: x y
= arg umin (c + d)0u 2X
(17)
= arg umin (c + (d x)d)0 u 2X
(18)
0
= arg umin (c + d)0 u 2X
z
(19)
for some strictly between and (d x): (a) (Improvement) c y + f (d y) c x + f (d x): (b) (Monotonicity) If > (d x), then (d x) (d y). Likewise, if < (d x), then (d x) (d y). Hence, the sequence k = (d xk ) for which xk = arg minx2X (c+ (d xk 1 )d)0x is either non-decreasing or non-increasing. (c) (Local optimality) c y + f (d y) c z + f (d z ) 0
0
0
0
0
0
0
0
0
0
0
0
0
0
11
0
;
0
0
for all strictly between and (d x). Moreover, if d y = d x, then the solution y is locally optimal, that is y = arg min (c + (d y )d)0u u2X 0
0
0
0
and c0 y + f (d0 y) c0 z + f (d0z )
for all between and (d y ): 0
Proof : (a) We have c0 x + f (d0 x)
= (c + (d x)d)0x ; (d x)d x + f (d x) 0
0
0
0
c0 y + (d0x)d0 y ; (d0x)d0x + f (d0x)
=
c0 y + f (d0y ) + f (d0x)(d0y ; d0 x) ; (f (d0y ) ; f (d0 x)g
c0 y + f (d0y )
since () is a subgradient. (b) From the optimality of x and y, we have c0 y + (d0x)d0 y
c0 x + (d0x)d0 x
;(c y + d y ) ;(c x + d x): 0
0
0
0
Adding the two inequalities we obtain (d x ; d y)( (d x) ; ) 0: 0
0
0
Therefore, if (d x) > then d y d x and since f (w) is a concave function, i.e., (w) is non-increasing, (d y) (d x). Likewise, if (d x) < then (d y) (d x). Hence, the sequence k = (d xk ) is monotone. (c) We rst show that d z is in the convex hull of d x and d y. From the optimality of x, y, and z we obtain 0
0
0
0
0
0
0
0
0
0
0
0
c0 x + d0 x
c0 z + d0 z
c0 x + d0x
c0 z + d0z
c0 y + (d0x)d0 y
c0 z + (d0x)d0 z
c0 y + d0y
c0 z + d0z
12
From the rst two inequalities we obtain (d z ; d x)( ; ) 0 0
0
and from the last two we have (d z ; d y )( (d x) ; ) 0: 0
0
0
As is between and (d x), then if < < (d x), we conclude since () is non-increasing that d y d z d x. Likewise, if (d x) < < , we have d x d z d y ., i.e., d z is in the convex hull of d x and d y. Next, we have 0
0
0
0
0
0
0
0
0
0
0
0
c0 y + f (d0 y )
= (c + (d x)d)0 y ; (d x)d y + f (d y) 0
0
0
0
(c + (d x)d)0 z ; (d x)d y + f (d y ) 0
0
0
0
=
c0 z + f (d0 z ) + ff (d0y ) ; f (d0 z ) ; (d0x)(d0y ; d0 z )g
=
c0 z + f (d0 z ) + h(d0 z )
c0 z + f (d0 z )
(20)
where inequality (20) follows from observing that the function h( ) = f (d y) ; f ( ) ; (d x)(d y ; ) is a convex function with h(d y ) = 0 and h(d x) 0. Since d z is in the convex hull of d x and d y, by convexity, h(d z ) h(d y ) + (1 ; )h(d x) 0, for some 2 0 1]. Given a feasible solution, x, Theorem 5(a) implies that we may improve the objective by solving a sequence of problems using Algorithm 2. Note that at each iteration, we are optimizing a linear function over X . Theorem 5(b) implies that the sequence of k = (d xk ) is monotone and since it is bounded it converges. Since X is nite, then the algorithm converges in a nite number of steps. Theorem 5(c) implies that at termination (recall that the termination condition is d y = d x) Algorithm 2 nds a locally optimal solution. Suppose = (e d) and fx1 : : : xk g be the sequence of solutions of Algorithm 2. From Theorem 5(b), we have = (e d) 1 = (d x1 ) : : : k = (d xk ): 0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
When Algorithm 2 terminates at the solution xk , then from Theorem 5(c), c0 xk + f (d0 xk ) c0 z + f (d0 z )
(21)
where z is de ned in Eq. (19) for all 2 (e d) (d xk )]. Likewise, if we apply Algorithm 2 starting at = (0), and let fy1 : : : ylg be the sequence of solutions of Algorithm 2, then we have 0
0
= (0) 1 = (d y1 ) : : : l = (d yl )
0
0
13
and c0 yl + f (d0 yl ) c0 z + f (d0z )
(22)
for all 2 (d yl ) (0)]. If (d xk ) (d yl ), we have (d xk ) 2 (d yl ) (0)] and (d yl ) 2 (e d) (d xk )]. Hence, following from the inequalities (21) and (22), we conclude that 0
0
0
0
0
0
0
0
c0 yl + f (d0 yl ) = c0 xk + f (d0 xk ) c0 z + f (d0 z )
for all 2 (e d) (d xk )] (d yl ) (0)] = (e d) (0)]: Therefore, both yl and xk are globally optimal solutions. However, if (d yl ) > (d xk ), we are assured that the global optimal solution is xk , yl or in fx : x = arg minu2X (c + d)0 u, 2 ( (d xk ) (d yl ))g. We next determine an error bound between the optimal objective and the objective of the best local solution, which is either xk or yl . 0
0
0
0
0
0
0
0
Theorem 6 (a)Let W = w w], (w) > (w), X 0 = X \ fx : d x 2 W g, and 0
x1
= arg ymin (c + (w)d)0y 2X 0
(23)
x2
= arg ymin (c + (w)d)0y : 2X 0
(24)
Then 0 G
min fc x1 + f (d x1 ) c x2 + f (d x2 )g G0 + " 0
0
0
0
where 0 G "
and w
= ymin c y + f (d y) 2X 0 0
(25)
0
= (w)(w ; w) + f (w) ; f (w ) ) + (w)w ; (w)w : = f (w) ; f (w(w ) ; (w)
(b) Suppose the feasible solutions x1 and x2 satisfy x1
= arg ymin (c + (d x1 )d)0y 2X
(26)
x2
= arg ymin (c + (d x2 )d)0y 2X
(27)
0
0
such that (w) > (w), with w = d x1 , w = d x2 and there exists an optimal solution arg miny2X (c + d)0y for some 2 ( (w) (w)), then 0
G
0
min fc x1 + f (d x1 ) c x2 + f (d x2 )g G + " 0
0
0
where G = c x + f (d x ). 0
0
14
0
x
= (28)
10
9
8
7
(w* , g(w*))
6
5
4 (w* , f(w*)) 3
2
1
0
10
20
30
40
50
60
70
80
90
100
Figure 1: Illustration of the maximum gap between the functions f (w) and g (w).
Proof : (a) Let g(w), w 2 W be a piecewise concave function comprising of two line segments through
(w f (w)), (w f (w)) with respective subgradients (w) and (w). Clearly f (w) g (w) for w 2 W , and hence, we have ;" f (w) ; g (w) 0, where " = maxw2W (g (w) ; f (w)) = g (w ) ; f (w ), noting that the maximum dierence occurs at the intersection of the line segments (see Figure 1). Therefore, ( ) = (w)(w ; w) + f (w) = (w)(w ; w) + f (w):
g w
Solving for w , we have w
) + (w)w ; (w)w : = f (w) ; f (w(w ) ; (w)
Applying Proposition 2 with X 0 instead of X and k = 2, we obtain
min c y + g (d y ) = min fc x1 + g (d x1 ) c x2 + g (d x2 )g :
y2X 0
0
0
0
0
0
0
Finally, from Theorem 3, we have 0
G
min fc x1 + f (d x1 ) c x2 + f (d x2 )g G0 + ": 0
0
0
0
(b) Under the stated conditions, observe that the optimal solutions of the problems (26) and (27) are
respectively the same as the optimal solutions of the problems (23) and (24). Let 2 ( (d x2 ) (d x1 )) 0
15
0
such that x = arg miny2X (c + d)0y . We establish that c0 x + d0x
c0 x1 + d0x1
c0 x + (d0x1 )d0x
c0 x1 + (d0x1 )d0 x1
c0 x + d0x
c0 x2 + d0x2
c0 x + (d0x2 )d0x
c0 x2 + (d0x2 )d0 x2 :
Since (d x2 ) < < (d x1 ), it follows that d x 2 d x1 d x2 ] and hence, G = G0 and the bounds of (25) follows from part (a). Remark: If (d yl) > (d xk ), Theorem 6(b) provides a guarantee on the quality of the best solution of the two locally optimal solution xk and yl relative to the global optimum. Moreover, we can improve the error bound by partitioning the interval (w) (w)], with w = d yl , w = d xk into two subintervals, (w) ( (w) + (w))=2] and ( (w) + (w))=2 (w)] and applying Algorithm 2 in the intervals. Using Theorem 6(a), we can obtain improved bounds. Continuing this way, we can nd the globally optimal solution. 0
0
0
0
0
0
0
0
0
6 Generalized Bertsimas and Sim Robust Formulation Bertsimas and Sim 6] propose the following model for robust discrete optimization: Z
8 9 j :0 if xj = 0 or dj < . 17
(36)
By restricting the interval of can vary we obtain that l = 1 : : : r, is de ned for 2 dl dl+1] is ( ) = xmin c x+ 2X 0
Zl
X
j2Sl
Z
= min 0 minl=0:::r Zl ( ) where
(dj ; )xj + f (eJ x) 0
( ),
Zl
(37)
and for 2 d1 1): ( ) = xmin c x + f (eJ x) : 2X
Z0
0
(38)
0
Since each function Zl ( ) is optimized over the interval dl dl+1], the optimal solution is realized in either dl or dl+1 . Hence, we can restrict from the set fd1 : : : dr 0g and establish that Z
= l2Jf min0g c x +
X
0
j2Sl
(dj ; dl )xj + f (eJ x)dl: 0
(39)
Since eJ x 2 f0 1 : : : rg, we apply Theorem 2 to obtain the subproblem decomposition of (33). Theorem 8 suggests that the robust problem remains polynomially solvable if the nominal problem is polynomially solvable, but at the expense of higher computational complexity. We next explore faster algorithms that are only guarantee local optimality. In this spirit and analogously to Theorem 5, we provide a necessary condition for optimality, which can be exploited in a local search algorithm. 0
Theorem 9 An optimal solution, x to the Problem (32) is also an optimal solution to the following problem:
minimize
where l = arg min
l2Jf0g
X j2Sl
subject to
c0 y + y
X j2Sl
(dj ; dl )yj + (eJ x)dl 0
e0J y
2 X
(40)
(dj ; dl )xj + f (eJ x)dl . 0
Proof : Suppose x is an optimal solution for Problem (32) but not for Problem (40). Let y be the optimal solution to Problem (40). Therefore,
8 9
c0 y +
X X
j2Sl
(dj ; dl )yj + (eJ x)dl eJ y ; (eJ x)dl 0
=
c0 y +
c0 y +
min c y + l2Jf0g
X
j2Sl j2Sl 0
0
0
;
e0J x + f (e0J x)dl
;
0
0
0
0
0
0
l
(dj ; dl )yj + f (eJ y)dl 0
X
(dj ; dl )yj + f (eJ y)dl 0
8 9
