On Non-Convex Quadratic Programming with Box Constraints Samuel Burer∗
Adam N. Letchford† July 2008
Abstract Non-Convex Quadratic Programming with Box Constraints is a fundamental N P-hard global optimisation problem. Recently, some authors have studied a certain family of convex sets associated with this problem. We prove several fundamental results concerned with these convex sets: we determine their dimension, characterise their extreme points and vertices, show their invariance under certain affine transformations, and show that various linear inequalities induce facets. We also show that the sets are closely related to the boolean quadric polytope, a fundamental polytope in the field of polyhedral combinatorics. Finally, we present a ‘recursive’ result that enables one to interpret certain complex valid inequalities in terms of simpler valid inequalities. Keywords: non-convex quadratic programming — global optimisation — polyhedral combinatorics — convex analysis.
1
Introduction
Non-Convex Quadratic Programming with Box Constraints (QPB) is the problem of minimising a non-convex quadratic function of a set of variables, subject to lower and upper bounds on the variables. A QPB instance with n variables takes the form: min cT x + xT Qx : l ≤ x ≤ u, x ∈ Rn , where x is the vector of decision variables, c ∈ Zn is the vector of linear costs, Q ∈ Zn×n is the matrix of quadratic costs and l ∈ Zn and u ∈ Zn are the vectors of lower and upper bounds, respectively. ∗ Department of Management Sciences, Tippie College of Business, University of Iowa. E-mail:
[email protected] † Department of Management Science, Lancaster University, United Kingdom. E-mail:
[email protected]
1
As usual in the literature, we assume throughout this paper that the box constraints take the simple form x ∈ [0, 1]n . Any instance not satisfying this property can be easily transformed into one that does, using substitutions of the form x0i = (xi − li )/(ui − li ). Although QPB is a continuous optimisation problem, it is well-known to be N P-hard in the strong sense. It is therefore a fundamental problem in global optimisation (see Horst et al. [13]). A survey of research on QPB up to 1997 was given by De Angelis et al. [6]. More recent relevant papers include Yajima & Fujie [27], Vandenbussche & Nemhauser [25, 26], Burer & Vandenbussche [5], Anstreicher [1] and Anstreicher & Burer [2]. Most of the existing approaches to QPB begin by adding additional variables representing the quadratic terms. That is, for 1 ≤ i ≤ j ≤ n the variable yij is introduced, representing the product xi xj . Using such variables, the objective function can be linearised. Since the constraints yij = xi xj are however non-convex, they must be approximated by convex constraints, either linear (as in [22, 25, 26, 27]) or conic (as in [1, 2, 5]). The resulting relaxation can then be embedded within a branch-and-bound framework. In order to derive stronger relaxations in the (x, y)-space, it is natural to study the convex hull of feasible solutions to the problem, i.e., the set o n n+1 QP Bn = conv (x, y) ∈ [0, 1]n+( 2 ) : yij = xi xj (∀1 ≤ i ≤ j ≤ n) . Note that QP Bn , though convex, is not polyhedral even for n = 1: see Figure 1. Although a few authors have studied QP Bn explicitly [1, 2, 27], many fundamental questions about its structure remain unanswered. (In fact, an explicit description of QP Bn is not known even for n = 3). The goal of this paper is to make progress on this issue. The structure of the paper is as follows. In Section 2, we review the relevant literature. In Section 3, we explore some fundamental properties of QP Bn : its dimension, extreme points and vertices, and its invariance under certain affine transformations. In Section 4, we consider some well-known linear inequalities, the so-called RLT and psd inequalities, and determine the dimension of the corresponding faces of QP Bn . In Section 5, which is at the heart of the paper, we explore the connection between QP Bn and the so-called boolean quadric polytope BQPn , a fundamental polytope in the field of polyhedral combinatorics. Our main result is that every inequality valid for BQPn is valid for QP Bn . We also give a necessary and sufficient condition for an inequality to induce a facet of both BQPn and QP Bn , and show that a large class of inequalities meets this condition. Next, Section 6 presents a ‘recursive’ result that enables one to interpret valid inequalities for QP Bn in terms of valid inequalities for QP Bn−1 . This sheds some light on the structure of QP B3 . Finally, some concluding remarks are given in Section 7. 2
y11 1
0
p ppb p pp pp p p p pp p pp p p pp p p p pp p pp p p p p p p pp pp pppp p p p p p pp p p pp ppp pp p p p p p pp p pp pppp pp p p p p p p pp p pp p pp ppp pp bp p pp pp p p p p p p p p p p 0
x1
1
Figure 1: The convex set QP B1 . We assume throughout that the reader is familiar with the basics of polyhedral theory (see Nemhauser & Wolsey [15] or Schrijver [21]) and convex analysis (see Hiriart-Urruty & Lemar´echal [11]).
2
Key Concepts from the Literature
Some key concepts from the literature are now explained.
2.1
The RLT inequalities
It is well known that the constraint yij = xi xj , together with the bounds 0 ≤ xi ≤ 1 and 0 ≤ xj ≤ 1, imply the following four linear inequalities: yij ≥ 0, yij ≤ xi , yij ≤ xj , yij ≥ xi + xj − 1. These inequalities remain valid when i = j, in which case the second and third of them coincide. They have come to be known as RLT inequalities, because they can be derived using the so-called Reformulation-Linearisation Technique of Sherali & Adams [22]. Replacing the constraints yij = xi xj with the RLT inequalities, we obtain a Linear Programming (LP) relaxation of QPB. See Figure 2 for an illustration, again for the trivial case n = 1.
2.2
Using positive semidefiniteness
The idea of using Semidefinite Programming (SDP) to construct relaxations of non-convex quadratic programs goes back to the seminal papers of Shor [23] and Lov´ asz & Schrijver [14]. The basic idea is as follows.
3
y11 1
0
pp pb p ppppp p p p p p ppp p p p p ppp p p p p p ppp p p p pppp p p pp ppp pppp pp p p p p pp p pp pp p ppp p p p p pp p pp pp p ppp p p p p p p pp p ppp ppppppppppppppppppppppppppppppppppppppppppppppppppp bp p p p bpp 0
0.5
1
x1
Figure 2: Region defined by RLT inequalities when n = 1. We begin by defining the n × n symmetric matrix X = xxT . Note that, for any 1 ≤ i ≤ j ≤ n, Xij = yij . Since X is defined as the product of a vector and its transpose, it should be positive semidefinite (psd) in a feasible solution. In fact, we can say something stronger. The augmented matrix T 1 1 1 xT ˆ , = X := x X x x which contains X as a submatrix, should also be psd. ˆ is equivalent to imposing It is well-known that imposing psd-ness on X T psd-ness on the matrix X − xx , which in turn amounts to imposing the convex quadratic constraints bT Xb ≥ (bT x)2 for all b ∈ Rn . As noted for example by Ramana [18] and Yajima & Fujie [27], one can ˆ is also interpret the psd condition in terms of linear inequalities. Indeed, X psd if and only if: v T Xv + (2s)v T x + s2 ≥ 0 for all vectors v ∈ Rn and scalars s ∈ R. This is equivalent to imposing T
(2s)v x +
n X i=1
vi2 yii + 2
X
vi vj yij + s2 ≥ 0
(∀v ∈ Rn , s ∈ R). (1)
1≤i n. Proof. By induction, it suffices to prove that the inequality induces a face of QP Bn+1 of co-dimension k. Let F be the original face of QP Bn and let F 0 be the face of QP Bn+1 induced by the inequality. Since F has n+1 co-dimension k, it contains n + 2 + 1 − k affinely-independent extreme points of QP Bn . Each of these can be converted into an extreme point of QP Bn+1 by setting xn+1 = 0 and yi,n+1 = 0 for i = 1, . . . , n + 1. In this + 1 − k affinely-independent extreme points of way, one obtains n + n+1 2 QP Bn+1 that lie in F 0 . To complete the proof, we need another n + 2 such points. Let x1 , . . . , xn+1 ∈ Rn be the vectors mentioned in Lemma 3. We construct n + 1 modified vectors in Rn+1 , say x ˜1 , . . . , x ˜n+1 , by setting: • x ˜ki = xki for k = 1, . . . , n + 1 and i = 1, . . . , n, • x ˜kn+1 = 1 for k = 1, . . . , n + 1. Now note that, for k = 1, . . . , n + 1, we can construct an extreme point 0 (˜ xk , y˜k ) of QP Bn+1 that lies in F . These n + 1 extreme points, together n+1 with the original n + 2 + 1 − k ones, are easily shown to be affinely independent. Finally, we construct one more extreme point of QP Bn+1 as follows. Let x ¯ be identical to x ˜1 , apart from the fact that x ¯n+1 = 1/2. The corresponding extreme point of QP Bn+1 , say (¯ x, y¯), also lies in F 0 . It is affinely independent of the other points mentioned, since it is the only one that does not satisfy the equation yn+1,n+1 = xn+1 .
5
Facets from the Boolean Quadric Polytope
As mentioned in Subsection 2.4, Yajima & Fujie [27] proved that certain valid inequalities for BQPn are valid also for QP Bn . In this section, we extend this result in several ways. The key is a certain projection result, that we present in the next subsection. 13
5.1
BQPn as a projection of QP Bn
n+1 n Recall that QP Bn and BQPn ‘live’ in Rn+( 2 ) and Rn+( 2 ) , respectively. n+1 We let proj(x, y) denote the linear operator that projects points in Rn+( 2 ) n onto Rn+( 2 ) , by simply dropping the components yii for all 1 ≤ i ≤ n. Similarly, we let proj(S) and proj(QP Bn ) denote the projection of S and QP Bn , respectively, onto the same subspace. We will need the following lemma:
Lemma 4 Let (¯ x, y¯) be a member of S, and suppose that x ¯k ∈ (0, 1) for 0 1 some 1 ≤ k ≤ n. Let x and x be the vectors obtained from x ¯ by setting xk to 0 or 1, respectively, and let (x0 , y 0 ) and (x1 , y 1 ) be the corresponding extreme points of QP Bn . Then proj(¯ x, y¯) is a convex combination of proj(x0 , y 0 ) and proj(x1 , y 1 ). Proof. Let λ = x ¯k . One can check that x ¯i y¯ij
= λx1i + (1 − λ)x0i =
1 λyij
+ (1 −
0 λ)yij
(i = 1, . . . , n) (1 ≤ i < j ≤ n).
Thus, proj(¯ x, y¯) = λ proj(x1 , y 1 ) + (1 − λ) proj(x0 , y 0 ).
Our main result is then the following: Theorem 6 proj(QP Bn ) = BQPn . Proof. Clearly, proj(QP Bn ) is the convex hull of proj(S). Now let (¯ x, y¯) be a member of S, and suppose that (¯ x, y¯) is not binary. Then xk ∈ (0, 1) for some 1 ≤ k ≤ n. Lemma 4 shows that there exist two other members of S, say (x0 , y 0 ) and (x1 , y 1 ), such that proj(¯ x, y¯) is a convex combination 0 0 1 1 of proj(x , y ) and proj(x , y ). So proj(¯ x, y¯) cannot be an extreme point of proj(QP Bn ). Thus, proj(QP Bn ) is the convex hull of the members of S that are binary. It is therefore equal to BQPn . This yields the desired result immediately: Corollary 2 If a linear inequality is valid for BQPn , it is valid for QP Bn . This implies the above-mentioned result of Yajima & Fujie [27]. The following proposition shows that there is another link between QP Bn and BQPn . Proposition 7 Let F be the face of QP Bn defined by the equations yii = xi for all i. Then proj(F ) = BQPn .
14
Proof. The only members of S that satisfy yii = xi for all i are the binary ones. Thus, the extreme points of F are the binary members of S. Since proj(F ) is the convex hull of the projections of these binary members, it is equal to BQPn . Thus, BQPn is simultaneously a projection of QP Bn and a projection of a face of QP Bn . This fact too can be seen clearly in Figure 1: whether we project the whole of QP B1 or just the face F onto R, we still obtain the line segment defined by 0 ≤ x1 ≤ 1.
5.2
Which BQP facets yield QP B facets?
The RLT inequalities with j 6= i are examples of inequalities that induce facets of both BQPn and QP Bn . In this subsection, we give a necessary and sufficient condition for an inequality to have this property. We will need the following lemma: Lemma 5 Suppose we are given an inequality that induces a face of BQPn . Moreover, let (¯ x, y¯) be a member of S, and suppose that xk ∈ (0, 1) for some 1 ≤ k ≤ n. Let (x0 , y 0 ) and (x1 , y 1 ) be defined as in Lemma 4. Then (¯ x, y¯) 0 0 1 1 satisfies the inequality at equality if and only if (x , y ) and (x , y ) do. Proof. From Lemma 4, proj(¯ x, y¯) is a convex combination of proj(x0 , y 0 ) 1 1 and proj(x , y ). Thus, the slack of the inequality at (¯ x, y¯) is a convex combination of the slacks of the inequality at (x0 , y 0 ) and (x1 , y 1 ). We then have the following result: Theorem 7 Suppose an inequality induces a facet of BQPn . A necessary and sufficient condition for it to also induce a facet of QP Bn is the existence of n extreme points of QP Bn , say (x1 , y 1 ), . . . , (xn , y n ), such that: • each satisfies the inequality at equality; • xii ∈ (0, 1) for i = 1, . . . , n. • xij ∈ {0, 1} for i = 1, . . . , n and j 6= i. Proof. First we prove necessity. For any i ∈ {1, . . . , n}, there must exist an extreme point of QP Bn that lies on the face and such that xi is fractional. (If this were not so, then all extreme points of QP Bn lying on the face would satisfy the RLT inequality yii ≤ xi .) Now, by a repeated application of Lemma 5 with k 6= i, we can convert the ith such point into the desired point (xi , y i ). Next we prove sufficiency. Since the inequality induces a facet of BQPn , there exist n + n2 affinely-independent binary extreme points of QP Bn lying on the face. For the inequality to induce a facet of QP Bn , one 15
needs an additional n affinely-independent extreme points. To see that (x1 , y 1 ), . . . , (xn , y n ) are the desired points, note that, for any i, the point (xi , y i ) is the only point in the collection that does not satisfy the equation yii = xi . It is possible to express this condition entirely in terms of BQPn : Corollary 3 Suppose an inequality induces a facet of BQPn . A necessary and sufficient condition for it to also induce a facet of QP Bn is that there exist 2n vertices of BQPn , say (¯ x1 , y¯1 ), . . . , (¯ xn , y¯n ) and (ˆ x1 , yˆ1 ), . . . , (ˆ xn , yˆn ), with the following properties: • each satisfies the inequality at equality; • x ˆij = x ¯ij for i = 1, . . . , n and j 6= i, • x ¯ii = 0 and x ˆii = 1 for i = 1, . . . , n. Proof. To create the desired vertices of BQPn , it suffices to take the n extreme points of QP Bn described in Theorem 7, decompose each of them into two binary extreme points of QP Bn as in Lemma 5, and then project n the resulting 2n extreme points onto Rn+( 2 ) .
5.3
Clique, cut and cut-type inequalities
We now show that the clique, cut and cut-type inequalities induce facets of QP Bn We start by recalling the following two results of Padberg [16]: Proposition 8 (Padberg [16]) For any S ⊆ {1, . . . , n} with |S| ≥ 3, and any integer α with 1 ≤ α ≤ |S| − 2, the ‘clique’ inequality X X yij ≥ α xi − α(α + 1)/2 (2) {i,j}⊂S
i∈S
is valid and facet-inducing for BQPn . Proposition 9 (Padberg [16]) For any disjoint sets S and T with |S ∪ T | ≥ 3, the ‘cut’ inequality X X X yij + yij − yij ≥ 0 (3) {i,j}⊂S
{i,j}⊂T
i∈S,j∈T
is valid and facet-inducing for BQPn . Padberg remarked that, by applying the switching operation to the clique inequalities, one can obtain a more general class of inequalities. To our knowledge, the resulting inequalities were first described explicitly by Sherali et al. [24]: 16
Proposition 10 (Sherali et al. [24]) For any disjoint sets S and T with |S ∪ T | ≥ 3, and any integer α with 1 − |T | ≤ α ≤ |S| − 2, the inequality X X X yij − yij yij + i∈S,j∈T
{i,j}⊂T
{i,j}⊂S
≥α
X
xi − (α + 1)
i∈S
X
xi − α(α + 1)/2
(4)
i∈T
is valid and facet-inducing for BQPn . Yajima & Fujie [27] call the inequalities (4) cut-type inequalities. They reduce to clique and cut inequalities when T = ∅ and α ∈ {0, −1}, respectively. As mentioned in Subsection 2.4, Yajima & Fujie [27] proved that the cut-type inequalities are valid for QP Bn . We now show that, in fact, they induce facets of QP Bn : Proposition 11 The cut-type inequalities induce facets of QP Bn . Proof. Since the cut-type inequalities can be obtained by switching from the clique inequalities, it suffices to prove the result for the latter. One can check that a vertex Pof BQPn satisfies the clique inequality at equality if and only if it satisfies i∈S xi ∈ {α, α+1}. From this fact, it is easy to construct the 2n vertices of BQPn required by Corollary 3. The cut-type inequalities are in fact contained in an even larger class of valid inequalities, which to our knowledge first appeared in Boros & Hammer [4]: Proposition 12 (Boros & Hammer [4]) For any v ∈ Zn and s ∈ Z, all extreme points of BQPn satisfy (v T x + s)(v T x + s − 1) ≥ 0. Thus, the inequality n X
X
vi (vi + 2s − 1)xi + 2
i=1
vi vj yij ≥ s(1 − s)
(5)
1≤i 1. It follows that Qα,β (x, y) ≤ γ is not valid for the set defined by BQP3 , yii ≤ xi , and X xxT , which proves that this set is not QP B3 .
7
Concluding Remarks
Given the fact that QPB is a fundamental and much-studied problem in global optimisation, it is surprising that many of its basic properties were not established before now. We have addressed this gap in the literature, using the tools of polyhedral theory and convex analysis. There are some interesting topics for future research. For example, can one find an explicit description of QP B3 in terms of linear inequalities? And, if an inequality induces a facet of BQPn but not of QP Bn , can it be strengthened in some way so that it induces a facet of QP Bn ? Finally, the algorithmic implications of our results should be investigated.
21
Acknowledgement: The first author was supported in part by National Science Foundation Grant CCF-0545514, and the second author was supported by the Engineering and Physical Sciences Research Council under grant EP/D072662/1.
References [1] K.M. Anstreicher (2007) Semidefinite programming versus the reformulation-linearization technique for non-convex quadratically constrained quadratic programming. Working paper, Department of Management Sciences, University of Iowa. [2] K.M. Anstreicher & S. Burer (2007) Computable representations for convex hulls of low-dimensional quadratic forms. Working paper, Department of Management Sciences, University of Iowa. [3] F. Barahona, M. J¨ unger & G. Reinelt (1989) Experiments in quadratic 0-1 programming. Math. Program., 44, 127–137. [4] E. Boros & P.L. Hammer (1993) Cut-polytopes, Boolean quadric polytopes and nonnegative quadratic pseudo-Boolean functions. Math. Oper. Res., 18, 245-253. [5] S. Burer & D. Vandenbussche (2007) Globally solving box-constrained nonconvex quadratic programs with semidefinite-based finite branchand-bound. Comput. Optim. Appl., to appear. [6] P.L. De Angelis, P.M. Pardalos & G. Toraldo (1997) Quadratic programming with box constraints. In I.M. Bomze et al. (eds.) Developments in Global Optimization. Dordrecht: Kluwer. [7] C. De Simone (1989) The cut polytope and the boolean quadric polytope. Discr. Math., 79, 71–75. [8] C. De Simone (1996) A note on the Boolean quadric polytope. Oper. Res. Lett., 19, 115–116. [9] M.M. Deza & M. Laurent (1997) Geometry of Cuts and Metrics. Berlin: Springer. [10] M.R. Garey, D.S. Johnson & L.J. Stockmeyer (1976) Some simplified N P-complete graph problems. Theor. Comput. Sci., 1, 237–267. [11] J.B. Hiriart-Urruty & C. Lemar´echal (2004) Fundamentals of Convex Analysis. Berlin: Springer. [12] C. Helmberg & F. Rendl (1998) Solving quadratic (0,1)-programs by semidefinite programs and cutting planes. Math. Program., 82, 291–315. 22
[13] R. Horst, P.M. Pardalos & N.V. Thoai (2000) Introduction to Global Optimization, 2nd Edn. Dortrecht: Kluwer. [14] L. Lov´ asz & A.J. Schrijver (1991) Cones of matrices and set-functions and 0-1 optimization. SIAM J. Opt., 1, 166–190. [15] G.L. Nemhauser & L.A. Wolsey (1988) Integer and Combinatorial Optimization. New York: Wiley. [16] M.W. Padberg (1989) The boolean quadric polytope: some characteristics, facets and relatives. Math. Program., 45, 139–172. [17] G. Pataki (2000) The geometry of semidefinite programming. In H. Wolkowicz, R. Saigal & L. Vandenberghe (eds.) Handbook of Semidefinite Programming. Dortrecht: Kluwer. [18] M. Ramana (1993) An Algorithmic Analysis of Multiquadratic and Semidefinite Programming Problems. PhD thesis, Johns Hopkins University, Baltimore, MD. [19] F. Rendl, G. Rinaldi & A. Wiegele (2007) A branch-and-bound algorithm for max-cut based on combining semidefinite and polyhedral relaxations. In M. Fischetti & D.P. Williamson (eds.) Integer Programming and Combinatorial Optimization XII. Lecture Notes in Computer Science vol. 4513, Berlin: Springer. [20] I.G. Rosenberg (1972) 0-1 optimization and nonlinear programming. RAIRO, 2, 95–97. [21] A.J. Schrijver (1998) Theory of Linear and Integer Programming. New York: Wiley. [22] H.D. Sherali & W.P. Adams (1998) A Reformulation-Linearization Technique for Solving Discrete and Continuous Nonconvex Problems. Kluwer, Dordrecht. [23] N.Z. Shor (1987) Quadratic optimization problems. Tekhnicheskaya Kibernetika, 1, 128–139. [24] H.D. Sherali, Y. Lee & W.P. Adams (1995) A simultaneous lifting strategy for identifying new classes of facets for the Boolean quadric polytope. Oper. Res. Lett., 17, 19–26. [25] D. Vandenbussche & G.L. Nemhauser (2005) A polyhedral study of nonconvex quadratic programs with box constraints. Math. Program., 102, 531-557.
23
[26] D. Vandenbussche & G.L. Nemhauser (2005) A branch-and-cut algorithm for nonconvex quadratic programs with box constraints. Math. Program., 102, 559-575. [27] Y. Yajima & T. Fujie (1998) A polyhedral approach for nonconvex quadratic programming problems with box constraints. J. Global Optim., 13, 151-170.
24
