An Iterative Scheme for Valid Polynomial Inequality Generation in Binary Polynomial Programming Bissan Ghaddar?1 , Juan C. Vera2 , and Miguel F. Anjos??3 1
3
Department of Management Sciences, University of Waterloo, Waterloo, ON, Canada, N2L 3G1,
[email protected]. 2 Tilburg School of Economics and Management, Tilburg University, Tilburg, The Netherlands,
[email protected]. ´ D´epartement de Math´ematiques et g´enie industriel & GERAD, Ecole Polytechnique de Montr´eal, Montr´eal, QC, Canada H3T 1J4,
[email protected].
Abstract. Semidefinite programming has been used successfully to build hierarchies of convex relaxations to approximate polynomial programs. This approach rapidly becomes computationally expensive and is often tractable only for problems of small sizes. We propose an iterative scheme that improves the semidefinite relaxations without incurring exponential growth in their size. The key ingredient is a dynamic scheme for generating valid polynomial inequalities for general polynomial programs. These valid inequalities are then used to construct better approximations of the original problem. As a result, the proposed scheme is in principle scalable to large general combinatorial optimization problems. For binary polynomial programs, we prove that the proposed scheme converges to the global optimal solution for interesting cases of the initial approximation of the problem. We also present examples illustrating the computational behaviour of the scheme and compare it to other methods in the literature.
?
??
Research supported by a Canada Graduate Scholarship from the Natural Sciences and Engineering Research Council of Canada. Research partially supported by the Natural Sciences and Engineering Research Council of Canada, and by a Humboldt Research Fellowship.
1
Introduction
Semidefinite programming is now well recognized as a powerful tool for combinatorial optimization. Early research in this vein has yielded improved approximation algorithms and very tight bounds for some hard combinatorial optimization problems, see [22] and the references therein. As a consequence, interest in the application of semidefinite techniques to combinatorial optimization problems has continued unabated since the mid-1990s. This has included not only theoretical approximation guarantees but also the development and implementation of algorithms for efficiently solving semidefinite (SDP) relaxations of combinatorial optimization problems. Noteworthy developments in this direction include the biqmac solver for max-cut [35], an SDP-based branch-and-bound solver for max-k-cut [9], extremely tight bounds for the quadratic assignment problem [36], and exact solutions for single-row layout [1] as well as related quadratic linear ordering problems [11]. Since the seminal work of Lasserre [18], intense research activity has been carried out on polynomial programming (PP) and the related theory of moments. The main idea is the application of representation theorems to characterize the set of polynomials that are non-negative on a given domain. This research includes the recent work of de Klerk and Pasechnik [5], Lasserre [17, 18], Laurent [20, 21], Nie, Demmel, and Sturmfels [26], Parrilo [27, 29], Pe˜ na, Vera, and Zuluaga [31, 45], and the early work of Nesterov [24], Shor [39], and the S-Lemma of Yakubovich (see [33]) among others. Specifically for binary optimization, the specialization of Lasserre’s construction to binary PPs was shown to converge in a finite number of steps in [17] and the relationship of the Lasserre hierarchy to other well-known hierarchies was studied in works such as [19, 20]. All the PP-based approaches rely on the construction of sums-of-squares (SOS) certificates of nonnegativity for a suitable representation of the PP problem. While the resulting hierarchies yield very tight approximations of the original PP problem, from a computational perspective, these hierarchies all suffer from a common limitation, namely the explosion in size of the SDP relaxations involved. This fast growth in the size of the SDPs also affects formulations of combinatorial problems as PPs. One way to overcome this difficulty is to exploit the structure of the problem. This can be done either by taking advantage of symmetry as in Bai, de Klerk, Pasechnik, and Sotirov [2], de Klerk [4], de Klerk, Pasechnik, and Schrijver [6], de Klerk and Sotirov [7], and Gatermann and Parrilo [8], or by exploiting sparsity as in Kojima, Kim, and Waki [15], Waki, Kim, Kojima, and Muramatsu [41, 42], and several others [12, 13, 14, 16, 25]. However, in the absence of any favorable structure, the practical application of the SOS approach is severely limited. The motivation for our work is to devise ways to take advantage of the strength of the PP approach without depending on the presence of exploitable structure in the problem or paying a high computational price. To achieve this objective, it is imperative to avoid the growth of the complexity of the non-negativity certificates involved. For this purpose, instead of growing the degree of the certificates (which results in an exponential growth of the size of the relaxation), we fix the degree of the polynomials that we use and increase the set of polynomial inequalities describing the feasible set of the PP problem. These valid inequalities are then used to construct new certificates that provide better approximations. We obtain the valid inequalities by means of a dynamic inequality generation scheme (DIGS) that makes use of information from the objective function to dynamically generate polynomial inequalities that are valid on the feasible region of the PP problem. The result is an iterative scheme that provides better and better SDP approximations without growing the degree of the certificates involved. In this paper, we describe in some detail our proposed iterative scheme for general PP, and specialize it to binary PPs. Our method for the binary case can be seen as a generalization, from the linear case to higher degrees, of the lift and project methods of Balas, Ceria, and Cornu´ejols [3], Sherali and Adams [37], and Lov´ asz and Schrijver [23]. We prove that for binary PPs, the proposed scheme converges to the global optimal solution for interesting cases of the initial approximation of the problem. We also provide computational examples to highlight the advantages of the proposed scheme with respect to Lasserre’s approach as well as to the lift-and-project method of Balas-Ceria-Cornu´ejols. The potential impact of the methodology presented here is significant, since it provides a means to tightly approximate binary PPs that, unlike previously proposed hierarchies of SDP relaxations, is in principle scalable to large general combinatorial optimization problems.
1.1
Polynomial Programming Problem
Consider the general PP problem whose objective and constraints are multivariate polynomials, (PP-P) z = sup f (x) s.t. gi (x) ≥ 0
i = 1, . . . , m.
Let S = {x ∈ Rn : gi (x) ≥ 0, i = 1, . . . , m}, be the feasible set of (PP-P). We can rephrase (PP-P) as (PP-D) inf λ s.t. λ − f (x) ≥ 0
∀ x ∈ S.
(1)
The condition λ − f (x) ≥ 0 for all x ∈ S is N P-hard for most (interesting) choices of S. To obtain computable relaxations of (PP-D), one uses tractable relaxations of condition (1) that can be re-phrased in terms of a linear system of equations involving positive semidefinite matrices [18, 24, 29, 30, 28, 39, 43], second-order cones [10], or linear problems [19, 38, 45]. 1.2
Approximations Hierarchies for Polynomial Programs
Lasserre [18] introduced semidefinite relaxations corresponding to liftings of PPs into higher dimensions. The construction is motivated by results related to the representations of non-negative polynomials as SOS and the dual theory of moments. Lasserre shows that the global maximum of f (x) over a compact set S defined by polynomial inequalities reduces to solving a sequence of SOS representations for polynomials that are nonnegative on S. The convergence of Lasserre’s method is based on the assumption that {g1 (x), . . . , gm (x)}, the given description of S, allows the application of Putinar’s Theorem [34]. In particular, it assumes that S is compact. For ease of notation, define g0 (x) = 1 and G = {gi (x) : i = 0, 1, . . . , m}. For a given r > 0, a relaxation on the original polynomial program (PP-P) is obtained, µrG =
inf
λ
λ,σi (x)
s.t. λ − f (x) =
m X
σ0 (x)gi (x)
(2)
i=0
σi (x) is SOS of degree ≤ (r − deg(gi ))
i = 0, . . . , m.
Being an SOS of a given degree can be expressed in terms of SDP matrices, and thus the optimization problem (2) can be reformulated as a semidefinite program [39]. By increasing the value of r, one can build up a sequence of convex semidefinite relaxations of increasing size. Under mild conditions the optimal values of these problems converge to the global optimal value of the original non-convex problem (PP-P) [18, 29]. Using Lasserre’s approach for general polynomial programs, one may approach the global optimal value as closely as desired by solving a sequence of SDPs that grow in the size of the semidefinite matrices and in the number of constraints. The computational cost of the procedure clearly depends on both r and n, the number of variables. Problem (2) is computationally expensive in practice for r > 2 since the number of variables and constraints of (2) can be large, especially when using higher degree polynomials. For a problem with n variables and m inequality constraints, the optimization problem (2) has m + 1 semidefinite matrices of dimension O(nr ) and O(nr ) constraints. 1.3
Dynamic Approximations for Polynomial Programs
In this paper, we propose a scheme to dynamically generate valid polynomial inequalities for general PPs. Instead of growing r and increasing the size of the problem exponentially, we propose to fix r to a small value (mainly to d, the degree of f (x)) and improve the relaxation (2) by growing the set G, i.e., by adding valid polynomial inequalities to the description of S. Our approach makes use of information from the objective function to dynamically generate polynomial inequalities that are valid on the feasible region. Before diving into the technical details, we use an example to show how the method works.
Example 1. Consider the non-convex quadratic knapsack problem with n = 3 and d = 2: max 62x1 + 19x2 + 28x3 + 52x1 x2 + 74x1 x3 + 16x2 x3 s.t. 12x1 + 44x2 + 11x3 ≤ 66
(3)
x1 , x2 , x3 ∈ {0, 1}. The optimal value for (3) is z = 164. Let f (x) = 62x1 + 19x2 + 28x3 + 52x1 x2 + 74x1 x3 + 16x2 x3 . Setting r = 2 and replacing the condition xi ∈ {0, 1} with 0 ≤ xi ≤ 1 and x2i − xi = 0, we obtain the following relaxation min λ s.t. λ − f (x) = s(x) + a(66 − 12x1 − 44x2 − 11x3 ) +
3 X i=1
bi (1 − xi ) +
3 X
ci xi +
i=1
3 X
di (xi − x2i ),
i=1
s(x) is SOS of degree ≤ 2, a, bi , ci ∈ R+ , di ∈ R which has an objective value of 249.16. This is an upper bound on the optimal value of (3). If one wants to improve the value using Lasserre’s method, the hierarchy of SDPs shown in Table 1 must be solved. Table 1. Results for Lasserre’s hierarchy. The optimal solution is obtained with r = 6. 2 4 6 8 249.1 226.2 164.0 164.0 4×4(1) 10×10(1) 20×20(1) 35×35(1) 1×1(13) 4×4(13), 10×10(13), 20×20(13) total # of vars 26 135 925 3360 total # of constraints 10 35 84 165
r objective value psd matrices
By contrast, using our method we first generate a quadratic valid inequality p(x) = 0.809 − 0.388x1 − 0.037x2 − 0.361x3 − 0.099x1 x2 − 0.086x2 x3 and solve (2) with r = 2 again, this time taking G = {1, 66 − 12x1 + 44x2 + 11x3 , xi , 1 − xi , x2i − xi , xi − x2i , p(x)}. An objective function value of 243.22 is obtained. Performing this approach iteratively, one is able to improve the bound and obtain a tighter approximation of the original problem. After adding 11 inequalities we obtain an objective of 164.00 which is the optimal value of (3). In Table 2, the details on the size of the corresponding master problem and polynomial generation subproblem are given. Table 2. Results for the proposed Dynamic Inequality Generation Scheme. 0 1 2 3 ··· 9 10 11 i objective value 249.1 243.2 238.9 235.1 · · · 164.2 164.0 164.0 Master Problem psd matrices 4x×4(1) 4×4(1) 4×4(1) 4×4(1) · · · 4×4(1) 4×4(1) 4×4(1) non-negative vars 7 8 9 10 · · · 16 17 18 free vars 3 3 3 3 ··· 3 3 3 total # of vars 20 21 22 23 · · · 29 30 31 total # of constraints 10 10 10 10 · · · 10 10 10 Subproblem psd matrices 4×4(2) 4×4(2) 4×4(2) · · · 4×4(2) 4×4(2) 4×4(2) non-negative vars 14 16 18 · · · 30 32 34 free vars 20 20 20 · · · 20 20 20 total # of vars 54 56 58 · · · 70 72 74 total # of constraints 20 20 20 · · · 20 20 20
The key idea in the DIGS method is that if we can generate a new valid inequality for which we do not currently have a non-negativity certificate. We can use this new inequality, in combination with the previous ones, to generate new non-negativity certificates. This is formalized in Lemma 1. Before presenting Lemma 1 we define the notation we will use in the rest of the paper. Notation used in this paper: We use R[x] := R[x1 , . . . , xn ] (resp. Rd [x]) to denote the set of polynomials in n variables with real coefficients (resp. of degree at most d). Given S ⊆ Rn , we define P(S) (resp. Pd (S)) to be the cone of polynomials (resp. of degree at most d) that are non-negative over S. We denote by Ψ (resp. Ψd ) the cone of real polynomials (resp. of degree at most d) that are SOS of polynomials. Note PN Ψd := { i=1 pi (x)2 : p(x) ∈ Rb d c [x]}, with N = n+d d , and in particular Ψd = Ψd−1 for every odd degree d. 2
r For a given r > 0, define the approximation KG of Pd (S) as: |G| X r KG = gi (x)Ψr−deg(gi ) ∩ Rd [x]. i=0 r Lemma 1. Let r ≥ d and p ∈ Pd (S) \ KG . Then r r ⊆ Pd (S) and thus µrG ≥ µrG∪{p} . KG ( KG∪{p}
Thus, our inequality generation procedure is defined as d GIVEN G FIND p(x) ∈ Pd (S) \ KG .
(4)
In this procedure we need to tackle two problems: first, how to generate p(x) ∈ Pd (S) since somehow this d . This is the central point of the rest of the is our original problem; and second, how to ensure p(x) ∈ / KG paper. We explain how DIGS works for general PP in Section 2, and present a specialized (and more efficient) DIGS for binary PP in Section 3. We show some convergence results for Binary Quadratic Programming in Section 3.2. We also give examples comparing our method to Lasserre’s method and to the Balas, Ceria, and Cornu´ejols lift-and-project in Section 3.3.
2 2.1
General Case Dynamic Inequality Generation Scheme (DIGS)
To execute Procedure (4): d GIVEN G FIND p(x) ∈ Pd (S) \ KG
We will generate only polynomials for which we can ensure non-negativity on S. To do this we will generate d+2 d polynomials in KG . To ensure p(x) ∈ / KG , we use the dual optimal solution of (2), denoted by Y . We will abuse the notation and we identify Rd [x] with RN where N = n+d d , e.g. by identifying each polynomial d f (x) ∈ Rd [x] with its vector of coefficients f ∈ RN . In this way KG is a cone in RN . We endow RN with the n inner product h·, ·i such that for each f (x) ∈ Rd [x] and each u ∈ R , hf, Md (u)i = f (u), where Md (u) is the vector of monomials of u up to degree d. In this way, relaxation (2) and its SDP dual correspond to the conic primal-dual pair: inf λ λ d s.t. λ − f (x) ∈ KG
supY hf, Y i s.t. h1, Y i = 1 d ∗ Y ∈ (KG ) .
(5)
d ∗ From the definition of dual cone (KG ) , we have the following lemma, d Lemma 2. Let Y be a solution of (5). For all p ∈ KG , hp, Y i ≥ 0. d+2 d Thus to generate p(x) ∈ KG \ KG , one can solve the following SDP problem. We refer to this problem as the polynomial generating subproblem:
(PP-Sub) min hp, Y i d+2 s.t. p(x) ∈ KG
k p k≤ 1.
(6)
The normalization constraint is added since p(x) and cp(x) are equivalent inequalities for any c > 0. Moreover, without the normalization constraint, (PP-Sub) is unbounded. There are several options for choosing the norm k · k. In this paper we use the one that maximizes the `2 distance between Y and the set {Md (x) : p(x) = 0}. Example 2. min x1 − x1 x3 − x1 x4 + x2 x4 + x5 − x5 x7 − x5 x8 + x6 x8 s.t. x3 + x4 ≤ 1 x7 + x8 ≤ 1 0 ≤ xi ≤ 1
∀i ∈ {1, . . . , 8}.
The optimal objective value of the above problem is 0. We need to go at least up to r = 10 for Lasserre’s hierarchy to obtain the optimal value. However, for r = 8 even constructing the problem couldn’t be done within an hour. Using the dynamic scheme, we are able to use relaxations of degree 2 and add inequalities as in the previous example. In this case we stop after 50 inequalities due to the slow improvement in the bound, obtaining a lower bound of value -0.014 in 200.1 seconds. Figure 1 illustrates the bound improvement.
Fig. 1. DIGS lower bounds. The dotted line is the optimal objective value.
3
Binary Case
In this section we specialize the results presented in Section 2 to PPs where (some) all of the variables are binary; we refer to such problems as (mixed) binary polynomial programs (BPP). Algebraic geometry representation techniques are used to obtain a scheme that iteratively improves the bound and converges to the optimal objective value of the original binary polynomial program. Using the approach proposed in [31] and [44], we obtain a computationally cheaper subproblem for the binary case. Further, we present convergence results for some important cases. We prove that the resulting iterative scheme converges to the global optimal solution of the binary polynomial program when starting from the exact representation of the domain set excluding the binary constraints. As such a representation is not tractable in general, we show an initial approximation ensuring convergence for binary polynomial programs with a quadratic objective function and a set of linear constraints. 3.1
Specializing the Dynamic Inequality Generation Scheme
Using (PP-D), BPP can be reformulated as z = inf λ
(7) n
s.t. λ − f (x) ∈ Pd (D ∩ {−1, 1} ),
where D = {x : gi (x) ≥ 0, i = 1, . . . , m}. Without loss of generality we assume D ⊆ [−1, 1]n . Let Hj = {x ∈ Rn : xj ∈ {−1, 1}} and H = {−1, 1}n . Notice that H = ∩nj=1 Hj . To solve (7), we follow an approach similar to that for the general case presented in Section 2. Let G = {g0 (x), g1 (x), . . . , gm (x)} where g0 (x) = 1. Define QrG as the following approximation to Pd (D ∩ H): ! m n X X r 2 QG = gi (x)Ψr−deg(gi ) + (1 − xi )Rr−2 [x] ∩ Rd [x] i=0
i=1
and define the PP master problem ϕ = inf λ
(8)
λ
s.t. λ − f (x) ∈ QdG , which can be reformulated as an SDP. In this setting, for the polynomial generating subproblem (6), instead of solving the subproblem over Qd+2 as defined in Section 2.1, we use the following theorem to obtain a G computationally cheaper algorithm. Theorem 1. [31] For any degree d and compact set D, Pd (D ∩ Hj ) = ((1 + xj )Pd (D) + (1 − xj )Pd (D) + (1 − x2j )Rd−1 [x]) ∩ Rd [x]. From Theorem 1, it is natural to define the operator Cjd (Q) := (1 + xj )Q + (1 − xj )Q + (1 − x2j )Rd−1 [x] ∩ Rd [x], for any Q ⊆ Rd [x]. The following lemma is the key to our DIGS for the binary case. Lemma 3. Assume S = D ∩ H, and let Q ⊆ Pd (S). 1. For every j, Q ⊆ Cjd (Q) ⊆ Pd (S). 2. Moreover, if Pd (D) ⊆ Q then Pd (D ∩ Hj ) ⊆ Cjd (Q). 3. If Pd (D) ⊆ Q ( Pd (S) then for some j, Q ( Cjd (Q). Let Y be the dual optimal solution for (8). Define the j-th valid inequality generating subproblem as: ωj = min hp, Y i s.t. p(x) ∈
(9) Cjd (QdG )
k p k≤ 1. Using the master problem (8) and the subproblem (9) iteratively, we obtain a DIGS for the binary case. The algorithm terminates when for all indexes j, the subproblems have value equal to zero. In practice, we stop when for all j the value of the subproblem is sufficiently close to zero. The DIGS obtained for the binary case is computationally more efficient than the DIGS presented in Section 2.1 for the general case. The master problem in both cases is of the same size, but solving the subproblem (9) is basically of the same order as solving the master problem (8). Subproblem (9) has twice times the number of constraints of the master problem. This is much the number of variables and n+d+1 d+1 smaller than subproblem (6) obtained for the general case which has O(n2 /d2 ) times the number of variables and O(n2 /d2 ) times the number of constraints compared to the master problem. 3.2
Convergence Results
In this section, we first start by providing a proof of convergence for the case when the approximation QdG contains Pd (D) in the master problem. We use this theorem to show convergence of the DIGS for unconstrained binary polynomial programs with quadratic objective function, and for binary polynomial programs with quadratic objective function and linear constraints.
Notice that for the case where QdG is replaced with Pd (D) in the master problem (8), the subproblem (9) is equivalent to optimizing over Cjd (Pd (D)) = Pd (D ∩ Hj ), for a given index j. So intuitively, if the subproblem has a value of 0, it is because using the fact that xj is binary cannot help, i.e., the solution is already ”binary” in that coordinate. Thus if the value of all subproblems is zero we have converged to the optimal value. This intuition is formally expressed in Theorem 2. Recall from (5) that the dual optimal solution Y ∈ {X ∈ Pd (D)∗ : h1, Xi = 1}. Lemma 4. Let D ⊆ Rn be a compact set, then {X ∈ Pd (D)∗ : h1, Xi = 1} = conv(Md (D)). Theorem 2. Assume Pd (D) ⊆ QdG ⊆ Pd (S). Let ωj , ϕ, z be the optimal objective value of subproblem (9), master problem (8), and the original binary polynomial program (7) respectively. Assume D ⊆ [−1, 1]n and d ≥ 2. If ωj = 0 for all indexes j, then ϕ = z. Proof. Let (λ, Y ) be the optimal primal-dual solution of (8). Then ϕ = λ ≥ z and Y ∈ {X ∈ (QdG )∗ : h1, Xi = 1} ⊆P conv(Md (D)) using Lemma P 4. D is a compact set. By Caratheodory’s Theorem, Y can be written as Y = i ai Md (ui ) with a > 0, and each ui ∈ D. i i ai = 1 P P Notice that hf, Y i = i ai hf, Md (ui )i = i ai f (ui ). If ui ∈ H for all i, ϕ = hf, Y i ≤ z and we are done. To get a contradiction, assume uk ∈ / H for some k. Then there is j ≤ n such that uk ∈ / Hj . Consider p(x) = 1−x2j . d We have p ∈ Pd (Hj ) ⊆ Pd (D ∩ Hj ) ⊆ QG , and p(uk ) > 0. Therefore, ωj ≥ hp, Y i =
X i
ai hp, Md (ui )i =
X
ai p(ui ) ≥ ak p(uk ) > 0,
i
which is a contradiction. In the case of pure quadratic binary programming, taking D as the ball B = {x ∈ Rn : kxk2 ≤ n} and d = 2, d it follows from the S-lemma that P2 (D) = Ψ2 + (n − kxk2 )R+ 0 [x] and thus Pd (D) ⊆ QG ⊆ Pd (S) holds. From Theorem 2, we obtain convergence in the case of unconstrained binary quadratic programming. Theorem 3. When DIGS is applied to the case of pure binary quadratic programming, starting with G0 = {n − kxk2 , 1}, if all the subproblems have optimal value 0, then the objective function value of the master problem is equal to the optimal value of the original binary problem. In general, for S = {x : AT x = b}, the decision problem for P2 (S) is N P − hard, and thus we do not have an efficient description for P2 (S) unless P = N P. Thus we can not apply directly Theorem 2. However, a modification of the proof of Theorem 2 shows that if G contains (aTi x − bi )2 for each linear equality aTi x = bi , then the u’s constructed in the proof would be in S. Theorem 4. When DIGS is applied to a binary quadratic programming problem constrained to AT x = b, starting with G0 = {1, n − kxk2 , (aTi x − bi )2 , −(aTi x − bi )2 }, if all the subproblems have an optimal value of 0, then the optimal value of the master problem is equal to the optimal value of the original binary problem. 3.3
Examples
In this section, we provide several examples of BPPs and present computational results for DIGS. As our first two examples we consider the quadratic knapsack problem and the quadratic assignment problem. We report the objective function value at iteration zero and after performing 1, 5, and 10 iterations of DIGS. To solve these examples, we developed a MATLAB code that constructs and builds the resulting relaxations of the polynomial program and solves them using the SeDuMi solver [40]. As a reference for comparison we also present Lasserre’s results reporting the order of the relaxation to get the global optimal solution and the corresponding objective function value. In case where the time limit of one hour is reached, we report the bounds for Lasserre’s relaxation and the highest order r that could be solved within the time limit. To obtain a fair comparison, Lasserre’s relaxation was solved using the same code, on the same machine.
Example 3. Quadratic Knapsack Problem. Given n items with a non-negative weight wi assigned to each item i, and a profit matrix P , the quadratic knapsack problem maximizes the profit subject to a capacity constraint: max xT P x s.t. wT x ≤ c x ∈ {0, 1}n .
Table 3. Computational results for quadratic knapsack instances. Values in bold are optimal.
n 5 10 15 20 30
Optimal 370 1679 2022 8510 18229
r 4 4 4 2 2
Lasserre obj. t(sec) 370.0 2.1 1707.3 28.1 2022.0 1150.8 9060.3 2.9 19035.9 4.3
Iter. 0 413.9 1857.7 2270.5 9060.3 19035.9
Iter. 1 399.4 1821.9 2226.8 9015.3 18920.2
DIGS Iter. 5 370.0 1796.9 2180.4 8925.9 18791.7
Iter. 10 t(sec) 6.0 1791.5 7.2 2150.1 18.1 8850.3 35.4 18727.2 196.6
Table 3 presents computational results for small quadratic knapsack instances where the parameters are generated according to [32]. The results show that DIGS is much more efficient, in particular when n gets large. For n = 20, we are not able to go beyond r = 2 for Lasserre’s hierarchy in the given time limit. Example 4. Quadratic Assignment Problem. Consider the quadratic assignment problem where we want to allocate a set of n facilities to a set of n locations, with the cost being a function of the distance d and flow between the facilities f . The objective is to assign each facility to a location such that the total cost is minimized. This can be formulated as: P min i6=k,j6=l fik djl xij xkl P s.t. i xij = 1 1≤j≤n P 1≤i≤n j xij = 1 x ∈ {0, 1}n×n .
Table 4. Computational results for quadratic assignment instances. Values in bold are optimal.
n 3 4 5 6
Optimal 46 56 110 272
r 2 2 2 2
Lasserre obj. t(sec) 46.0 0.3 50.8 1.0 104.3 3.4 268.9 9.3
DIGS Iter. 0 Iter. 1 Iter. 5 Iter. 10 t(sec) 46.0 0.3 50.8 51.8 52.0 6.3 104.3 105.1 106.3 106.8 68.5 268.9 269.4 269.8 270.2 404.4
Table 4 presents computational results for small quadratic assignment instances where fik and djl are integers generated uniformly between 0 and 5. Using Lasserre’s hierarchy we can only go up to r = 2 for instances of dimension n ≥ 4 within one hour, while the dynamic scheme after 10 iterations improves significantly on the bounds of Lasserre’s r = 2 relaxation without as much computational effort. As a final example, we consider the maximum stable set problem. In this case, we compare DIGS with the lift-and-project method of Balas et al. [3]. This comparison provides a fair indication of the advantages of our method in terms of bound quality. For each instance we impose a 300 seconds time limit for each procedure. The upper bound for lift-and-project is compared to three approaches of DIGS. Linear refers to generating linear inequalities that are added to the master problem by using a non-negative multiplier. SOCP refers to
generating linear inequalities that are added to the master problem by using a polynomial multiplier that is in P1 (B) [10]. Quadratic refers to generating quadratic inequalities similar to the previous examples described. Since our methodology is a generalization of the lift-and-project method, our algorithm was used to obtain the Balas et al. results for the maximum stable set. Example 5. Stable Set Problem. Consider an undirected graph G(V, E) where V and E are the vertex set and the edge set respectively. Given a subset V¯ ⊆ V , then V¯ is called a stable set of G if there is no edge connecting any two vertices in V¯ . The maximum stable set problem is to find a stable set of maximal cardinality. Letting n = |V |, the maximum stable set problem can be formulated as a binary problem as follows: P (SS-D2) max i xi ∀(i, j) ∈ E
s.t. xi xj = 0 n
x ∈ {0, 1} . It can also be formulated as a linear problem by replacing the constraint xi xj = 0 by xi + xj ≤ 1, we refer to this problem as (SS-LP).
Table 5. Computational results for the stable set problem with a time limit of 300 seconds.
n Optimal 8 3 11 4 14 5 17 6 20 7 8 23 26 9 10 29 32 11 12 35 38 13 41 14 15 44 47 16 17 50
(SS-LP) UB 4.00 5.50 7.00 8.50 10.00 11.50 13.00 14.50 16.00 17.50 19.00 20.50 22.00 23.50 25.00
Balas et al. (SS-D2) Linear SOCP UB Iter. UB UB Iter. UB Iter. 3.00 197 3.44 3.00 186 3.00 126 4.00 160 4.63 4.00 139 4.00 130 5.02 135 5.82 5.02 114 5.01 91 6.22 121 7.00 6.23 84 6.09 63 7.46 104 8.18 7.43 68 7.25 45 8.81 88 9.36 8.61 50 8.36 33 10.11 77 10.54 9.84 37 9.60 25 11.65 65 11.71 11.10 24 10.87 17 13.03 56 12.89 12.37 18 12.20 14 14.48 49 14.07 13.49 13 13.32 10 16.05 43 15.24 14.80 8 14.74 7 17.69 39 16.42 15.88 7 15.77 6 19.10 34 17.59 17.19 6 17.09 5 20.78 29 18.77 18.39 4 18.26 4 22.18 27 19.94 19.52 4 19.42 4
Quadratic UB Iter. 3.02 49 4.05 109 5.14 82 6.30 54 7.42 38 8.67 22 9.96 14 11.18 10 12.53 6 13.66 4 14.85 4 16.26 1 17.30 1 18.59 1 19.77 1
Table 5 shows the upper bounds and the number of iterations performed within a time limit of 300 seconds. Lift-and-project performs the largest number of iterations for these instances since it utilizes linear programming which is computationally more efficient, however this efficiency comes at the expense of the bounds. For all instances the bounds obtained by DIGS using SOCP type of inequalities are the best bounds obtained within 300 seconds. These bounds are comparable with those from the Linear and Quadratic approaches, however Quadratic performs the least number of iterations and still achieves a competitive bound.
Bibliography
[1] M. F. Anjos and A. Vannelli. Computing globally optimal solutions for single-row layout problems using semidefinite programming and cutting planes. INFORMS J. Comp., 20(4):611–617, 2008. [2] Y. Bai, E. de Klerk, D. Pasechnik, and R. Sotirov. Exploiting group symmetry in truss topology optimization. Optimization and Engineering, 10(3):331–349, 2009. [3] E. Balas, S. Ceria, and G. Cornu´ejols. A lift-and-project cutting plane algorithm for mixed 0-1 programs. Mathematical Programming, 58:295–324, 1993. [4] E. de Klerk. Exploiting special structure in semidefinite programming: A survey of theory and applications. European Journal of Operational Research, 201(1):1–10, 2010. [5] E. de Klerk and D. Pasechnik. Approximation of the stability number of a graph via copositive programming. SIAM Journal on Optimization, 12(4):875–892, 2002. [6] E. de Klerk, D. Pasechnik, and A. Schrijver. Reduction of symmetric semidefinite programs using the regular∗ -representation. Mathematical Programming, 109(2-3):613–624, 2007. [7] E. de Klerk and R. Sotirov. Exploiting group symmetry in semidefinite programming relaxations of the quadratic assignment problem. Mathematical Programming, 122(2):225–246, 2010. [8] K. Gatermann and P. Parrilo. Symmetry groups, semidefinite programs, and sums of squares. Journal of Pure and Applied Algebra, 192(1-3):95–128, 2004. [9] B. Ghaddar, M. F. Anjos, and F. Liers. A branch-and-cut algorithm based on semidefinite programming for the minimum k-partition problem. Annals of Operations Research, to appear. [10] B. Ghaddar, J. Vera, and M. F. Anjos. Second-order cone relaxations for binary quadratic polynomial programs. SIAM Journal on Optimization, to appear. [11] P. Hungerl¨ ander and F. Rendl. Semidefinite relaxations of ordering problems. Technical report, AlpenAdria-Universit¨ at Klagenfurt, August 2010. [12] S. Kim, M. Kojima, M. Mevissen, and M. Yamashita. Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix completion. To appear in Mathematical Programming, 2009. [13] S. Kim, M. Kojima, and P. Toint. Recognizing underlying sparsity in optimization. Mathematical Programming, 9(2):273–303, 2009. [14] K. Kobayashi, S. Kim, and M. Kojima. Correlative sparsity in primal-dual interior-point methods for LP, SDP and SOCP. Applied Mathematics and Optimization, 58(1):69–88, 2008. [15] M. Kojima, S. Kim, and H. Waki. Sparsity in sums of squares of polynomials. Mathematical Programming, 103(1):45–62, 2003. [16] M. Kojima and M. Muramatsu. A note on sparse SOS and SDP relaxations for polynomial optimization problems over symmetric cones. Computational Optimization and Applications, 42(1):31–41, 2009. [17] J. Lasserre. An explicit equivalent positive semidefinite program for nonlinear 0-1 programs. SIAM Journal on Optimization, 12(3):756–769, 2001. [18] J. Lasserre. Global optimization problems with polynomials and the problem of moments. SIAM Journal on Optimization, 11:796–817, 2001. [19] J. Lasserre. Semidefinite programming vs. LP relaxations for polynomial programming. Mathematics of Operations Research, 27(2):347–360, 2002. [20] M. Laurent. A comparison of the Sherali-Adams, Lov´asz-Schrijver and Lasserre relaxations for 0-1 programming. Mathematics of Operations Research, 28:470–496, 2001. [21] M. Laurent. Semidefinite representations for finite varieties. Mathematical Programming, 109(Ser. A):1– 26, 2007. [22] M. Laurent and F. Rendl. Handbook on Discrete Optimization, volume 12, chapter Semidefinite programming and integer programming, pages 393–514. Elsevier, Amsterdam, Netherlands, 2005. [23] L. Lov´ asz and A. Schrijver. Cones of matrices and set-functions and 0-1 optimization. SIAM Journal on Optimization, 1:166–190, 1991. [24] Y. Nesterov. Structure of non-negative polynomials and optimization problems. Technical report, Technical Report 9749, CORE, 1997. [25] J. Nie and J. Demmel. Sparse SOS relaxations for minimizing functions that are summation of small polynomials. SIAM Journal on Optimization, 19(4):1534–1558, 2008.
[26] J. Nie, J. Demmel, and B. Sturmfels. Minimizing polynomials via sum of squares over the gradient ideal. Mathematical Programming: Series A and B, 106(3):587–606, 2006. [27] P. Parrilo. Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization. PhD thesis, Department of Control and Dynamical Systems, California Institute of Technology, Pasadena, California, 2000. [28] P. Parrilo. An explicit construction of distinguished representations of polynomials nonnegative over finite sets. Technical report, IFA Technical Report AUT02-02, Zurich - Switzerland, 2002. [29] P. Parrilo. Semidefinite programming relaxations for semialgebraic problems. Mathematical Programming, 96(2):293–320, 2003. [30] P. Parrilo and B. Sturmfels. Minimizing polynomial functions, algorithmic and quantitative real algebraic geometry. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, 60:83–89, 2003. [31] J. F. Pe˜ na, J. C. Vera, and L. F. Zuluaga. Exploiting equalities in polynomial programming. Operations Research Letters, 36(2), 2008. [32] D. Pisinger. The quadratic knapsack problem-a survey. Discrete Applied Mathematics, 155(5):623–648, 2007. [33] I. P´ olik and T. Terlaky. A survey of the S-lemma. SIAM Review, 49:371–418, 2007. [34] M. Putinar. Positive polynomials on compact semi-algebraic sets. Indiana University Mathematics Journal, 42:969–984, 1993. [35] F. Rendl, G. Rinaldi, and A. Wiegele. A branch and bound algorithm for max-cut based on combining semidefinite and polyhedral relaxations. Integer programming and combinatorial optimization, 4513:295– 309, 2007. [36] F. Rendl and R. Sotirov. Bounds for the quadratic assignment problem using bundle method. Mathematical Programming, Series B, 109:505–524, 2007. [37] H. D. Sherali and W. P. Adams. A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM Journal on Discrete Mathematics, 3(3):411– 430, 1990. [38] H. D. Sherali and C. H. Tuncbilek. Comparison of two reformulation-linearization technique based linear programming relaxations for polynomial programming problems. Journal of Global Optimization, 10(4):381–390, 1997. [39] N. Shor. A class of global minimum bounds of polynomial functions. Cybernetics, 23(6):731–734, 1987. [40] J. Sturm. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, 11-12, 1999. [41] H. Waki, S. Kim, M. Kojima, and M. Muramatsu. Sums of squares and semidefinite programming relaxations for polynomial optimization problems with structured sparsity. SIAM Journal on Optimization, 17(1):218–242, 2006. [42] H. Waki, S. Kim, M. Kojima, and M. Muramatsu. SparsePOP : a sparse semidefinite programming relaxation of polynomial optimization problems. ACM Transactions on Mathematical Software, 35(2):15, 2008. [43] H. Wolkowicz, R. Saigal, and L. Vandenberghe. Handbook of Semidefinite programming -Theory, Algorithms, and Applications. Kluwer, 2000. [44] L. Zuluaga. A conic programming approach to polynomial optimization problems: Theory and applications. PhD thesis, The Tepper School of Business, Carnegie Mellon University, Pittsburgh, 2004. [45] L. Zuluaga, J. C. Vera, and J. Pe˜ na. LMI approximations for cones of positive semidefinite forms. SIAM Journal on Optimization, 16(4), 2006.
