Dual-Optimal Inequalities for Stabilized Column Generation

OPERATIONS RESEARCH

informs

Vol. 54, No. 3, May–June 2006, pp. 454–463 issn 0030-364X eissn 1526-5463 06 5403 0454

®

doi 10.1287/opre.1060.0278 © 2006 INFORMS

Dual-Optimal Inequalities for Stabilized Column Generation Hatem Ben Amor GERAD and École Polytechnique de Montréal, C.P. 6079, succ. Centre-ville, Montréal, Quebec, Canada H3C 3A7, [email protected]

Jacques Desrosiers

HEC Montréal and GERAD, 3000, chemin de la Côte-Sainte-Catherine, Montréal, Quebec, Canada H3T 2A7, [email protected]

José Manuel Valério de Carvalho

Departamento de Produção e Sistemas, Universidade do Minho, Campus de Gualtar, 4710-057 Braga, Portugal, [email protected]

Column generation is one of the most successful approaches for solving large-scale linear programming problems. However, degeneracy difficulties and long-tail effects are known to occur as problems become larger. In recent years, several stabilization techniques of the dual variables have proven to be effective. We study the use of two types of dual-optimal inequalities to accelerate and stabilize the whole convergence process. Added to the dual formulation, these constraints are satisfied by all or a subset of the dual-optimal solutions. Therefore, the optimal objective function value of the augmented dual problem is identical to the original one. Adding constraints to the dual problem leads to adding columns to the primal problem, and feasibility of the solution may be lost. We propose two methods for recovering primal feasibility and optimality, depending on the type of inequalities that are used. Our computational experiments on the binary and the classical cutting-stock problems, and more specifically on the so-called triplet instances, show that the use of relevant dual information has a tremendous effect on the reduction of the number of column generation iterations. Subject classifications: production/scheduling: cutting-stock problems; programming: linear large-scale systems; column generation; stabilization; dual cuts. Area of review: Optimization. History: Received April 2003; revision received May 2004; accepted March 2005.

1. Introduction

reduced-cost columns, if any, that are added to the restricted master problem. If no such column exists, xk∗ k∗ is a pair of primal and dual-optimal solutions for P and D, respectively. Computing the minimum reduced cost should be performed in such a way that it is much less costly than evaluating the reduced cost of all columns explicitly. This is the case when the minimum reduced cost is given by the solution of a well-structured problem, usually called the oracle or subproblem. When columns of A are given as elements of a set , and the respective cost coefficients can be computed via a function c → , then the subproblem can be defined as minca − kT a a ∈ . This occurs for many well-known problems. For the cutting-stock problem, the feasible cutting patterns can be defined as the feasible integer solutions of a knapsack problem (Gilmore and Gomory 1961, Vance et al. 1994, Vanderbeck 1999), a longest-path problem over a disaggregated network (Valério de Carvalho 2005a), or a constrained shortest-path problem as in this paper. In vehicle routing and crew scheduling applications, or more generally, in many multicommodity flow problems, the subproblem is usually defined as a (constrained) flow or shortest-

Column generation is one of the most successful approaches in large-scale optimization (Lasdon 1970). It consists of solving huge linear programs by considering only a small subset of the variables at once. Consider the linear program P , also called the master problem, and its dual D, where A is an m × n matrix, c ∈ n , and b ∈ m : vP = min cT x Ax = b x 0 vD = max bT AT c At any iteration k of the column generation solution process, only a subset Ak of the columns of A is handled. By solving a restricted master problem, we obtain a primal feasible solution xk∗ , along with a vector k∗ of dual multipliers. With these multipliers, we compute the minimum reduced cost over all columns of matrix A to generate some negative 454

Ben Amor, Desrosiers, and Valério de Carvalho: Dual-Optimal Inequalities for Stabilized Column Generation Operations Research 54(3), pp. 454–463, © 2006 INFORMS

path problem; see, e.g., Desaulniers et al. (1998) and Mamer and McBride (2000). It is worth pointing out that solving the subproblem to optimality is only needed to prove optimality of the current primal and dual solutions. One can stop solving the subproblem whenever a negative reduced-cost column is found. Column generation users have experienced several instability features as the size and the difficulty level of the problems increase. Primal degeneracy is well known to cause slow convergence and long-tail effect to column generation procedures; see Gilmore and Gomory (1961). In general, the convergence of the dual variables towards an optimal set of multipliers presents a lot of oscillations, which results in useless and harmful moves in the dual space (Kallehauge et al. 2001). The box step method (Marsten 1975), bundle methods (Hiriart-Urruty and Lemaréchal 1993), and more general column generation/cutting planes stabilizing or regularizing approaches (du Merle et al. 1999, Ben Amor 2002, Neame 1999, du Merle and Vial 2002) show that controlling the dual-variable values during the solution process may be very helpful to make column generation much more efficient. In this paper, we address more specifically the use of relevant information on dual-optimal solutions as an alternative tool for stabilizing or regularizing column generation approaches. Gilmore and Gomory (1961) already proposed to replace equalities with greater or equal inequalities in the demand constraints of the cutting-stock problem, whereas Valério de Carvalho (2005b) adds cuts to the dual of its linear relaxation. In the former, the authors showed that the modified problem always has a solution with active demand constraints. Indeed, modifying the primal problem this way is equivalent to requiring nonnegativity of the dual variables, that is, at least half of the dual space is removed. In the latter, the author shows that it is always possible to build a primal feasible solution from any feasible solution to the corresponding modified primal problem. We show that these strategies are indeed special cases of a more general framework. This paper is organized as follows. We first present dual-optimal inequalities and their effects on the primal and dual formulations. Some efficiency issues are also discussed therein. In §3, we address the question of how to recover primal feasibility and optimality that might be lost when imposing new dual constraints. Sections 4, 5, and 6 are dedicated to computational experiments involving dual-optimal inequalities for cutting-stock applications. The results show that relevant dual information significantly reduces the number of column generation iterations.

2. Dual-Optimal Inequalities Assume that P and D have finite optimal solutions. Let D∗ be the set of optimal solutions for D and assume also that the dual space is restricted by a new set of constraints E T e that cuts off part of the dual space.

455

If D∗ ⊆ E T e, then E T e is called a set of dual-optimal inequalities. These are in fact valid inequalities on the dual-optimal polyhedron. Indeed, when added to the dual formulation, these constraints do not cut any dual-optimal solution. If the above condition is not necessarily satisfied for all ∈ D∗ , i.e., the condition changes to = D∗ ∩ E T e, then E T e may cut a nonempty subset of dual-optimal solutions. Therefore, E T e is called a set of deep dual-optimal inequalities. Indeed, a dual-optimal inequality is one that all dual-optimal solutions satisfy; a deep dual-optimal inequality is not necessarily satisfied by all of them, but is satisfied by at least one solution. denote Now consider adding E T e to D and let D the resulting problem. The corresponding primal problem is denoted P : vP = min cT x + eT y Ax + Ey = b x 0

y 0

= max b vD T

AT c E T e When the added constraints cut a nonempty subset of is a restriction of D. The primal the dual-feasible region, D space is then relaxed by introducing the vector of nonnegative variables y, and the cost function is also altered by introducing the term eT y incurred by these variables. Columns corresponding to the y-variables are used as static (as opposed to generated) columns in the primal formulation, that is, they are introduced prior to the column generation process. This ensures that at any column generation iteration, the dual multipliers computed at optimality of the corresponding restricted master problem satisfy the added constraint set E T e. Hence, this can be seen as a preprocessing technique. However, it can be implemented dynamically in the way that inequalities to be added are selected at the end of each column generation iteration. Redundant dual inequalities may also be eliminated to make the solution of the master problem more efficient. This would be useful, especially when it is not obvious to choose the best inequalities among a very large set. Intuitively, the tighter the dual-optimal inequalities, the more efficient is the column generation solution process. Indeed, the inequalities restrict the set of possible dual values, and the oscillations in the dual space are reduced both in magnitude and in frequency. Around the optimal multipliers, only relevant dual-cutting planes are generated to more rapidly describe the lower envelope of the Lagrangian dual function. Therefore, useless columns are not generated. From the primal point of view, we already know that the formulation is relaxed. Additionally, these static

Ben Amor, Desrosiers, and Valério de Carvalho: Dual-Optimal Inequalities for Stabilized Column Generation

456

Operations Research 54(3), pp. 454–463, © 2006 INFORMS

columns may have a special structure that makes the solution process more efficient. This is the case when these columns allow to take into account a number of columns that need not be generated explicitly (see §4). are respectively Optimal solutions for P , D, P , and D denoted by x∗ , ∗ , x∗ y∗ , and ∗ . The next proposition presents the obvious relationship between the optimal values of the original and modified problems when E T e are dual-optimal inequalities. Proposition 1. If D ∩ E e = , then vP = = vD = vP , and any optimal solution ∗ of D is vD also optimal for D. ∗

T

Let x∗ y∗ be an optimal solution for P and ∗ be Although ∗ is a dual-optimal an optimal solution for D. ∗ solution for D, x is not feasible for P whenever Ey∗ = 0 because A x∗ = b. Proposition 2. Let x y be an optimal solution for P and assume that E T e is a set of dual-optimal inequalities. Therefore, x ∗ is an optimal solution for P whenever Ey∗ = 0. ∗

∗

Proof. Let x∗ be an optimal solution for P and assume that Ey∗ = 0. Then, x∗ y∗ is feasible for P and x ∗ is feasible to P . Hence, cT x ∗ +eT y∗ cT x∗ +eT y∗ and cT x∗ cT x ∗ . Therefore, cT x∗ = cT x ∗ and x ∗ is an optimal solution for P . Note that it follows directly from the above proof that eT y∗ = 0 whenever x ∗ is feasible for P .

3. Determining an Optimal Primal Solution for P Let x∗ y∗ be an optimal solution for P and ∗ the corre Whenever Ey∗ = 0, sponding dual-optimal solution for D. ∗ ∗ x is not feasible for P as A x = b. Because we aim at obtaining an optimal solution for P , we need an efficient way to recover primal optimality given the solutions of the Next, we consider the primal and dual problems P and D. case where the added dual inequalities or the corresponding primal columns have a special interpretation that allows the building of an optimal solution for P from x ∗ . Proposition 3 (Valério de Carvalho 2005b). Assume that for any feasible solution x y for P , it is possible to build a feasible solution x for P such that cT x cT x + eT y. Then, for any optimal solution x∗ y∗ for P , the corresponding solution x∗ is an optimal solution for P . This situation occurs for the cutting-stock applications considered later on. However, when this is not the case, an efficient method to recover an optimal solution for P is needed. The procedure depends on the type of dual inequalities that are used. Assume first that E T e defines a set of dual-optimal inequalities. It becomes possible to obtain an optimal solution for P in a single phase by solving a slightly modified

). Given a vecproblem P (and the corresponding dual D T tor of scalars > 0, we replace E e in the definition by E T e + : of P and D vP = min cT x + e + T y Ax + Ey = b x 0

y 0

= max bT vD AT c E T e + Proposition 4. Let E T e be a set of dual-optimal inequalities and x∗ y∗ be an optimal solution for P . Then, y∗ = 0 and x ∗ is an optimal solution for P . Proof. Observe first that E T ∗ e + is a dual-optimal inequality for P . Then, by Proposition 1, any optimal is optimal for D. Moreover, it satsolution ∗ of D isfies E T ∗ e < e + . By complementary slackness conditions, y∗ = 0. Optimality of x ∗ for P follows by Proposition 2. Components of scalar vector have to be small enough to ensure the efficiency of column generation applied to P . Too-small values may cause numerical difficulties and/or ineffectiveness of the dual-inequality set, depending on the accuracy of the parameters used by the linear programming solver on hand. Assume now that E T e defines a set of deep dualoptimal inequalities. Given a dual-optimal solution ∗ for and a vector of scalars > 0 ∈ m , define the stabilized D ¯ ∗ , respectively: primal and dual problems P¯ ∗ and D vP¯ ∗ = min cT x − ∗ − T y1 + ∗ + T y2 Ax − y1 + y2 = b x 0

y1 0

y2 0

¯ ∗ = max bT vD AT c ∗ − ∗ + ¯ ∗ is constructed in such The stabilized dual problem D a way that the dual solution is restricted to a nonempty indeed box strictly containing an optimal solution ∗ of D, optimal for D by Proposition 1. The definition of the corresponding stabilized primal problem P¯ ∗ follows, adequately using nonnegative surplus y1 and slack variables y2 . Given that E T e defines a set of deep dual-optimal inequalities, an optimal primal solution for P can be found by using a two-phase approach. First, we solve P to get a dual-optimal solution ∗ for D; second, we solve the stabilized primal P¯ ∗ . The next proposition shows that this second phase leads to an optimal solution of P .


457


Proposition 5. Let E T e be a set of deep dual-optimal inequalities and ¯x∗ y1∗ y2∗ be a primal-optimal solution for P¯ ∗ . Then, x¯ ∗ is a primal-optimal solution for P . Proof. Every primal-optimal solution to P¯ ∗ satisfies complementary slackness with every dual-optimal solution ¯ ∗ . From Proposition 1, ∗ is an optimal solution to D ¯ ∗ . Hence, by complementary slackness, we must for D have y1∗ = y2∗ = 0 in any optimal solution to P¯ ∗ , and therefore x¯ ∗ is a primal-optimal solution for P . Note that the above proof does not depend on the actual ¯ ∗ , that is, ¯ ∗ might be equal dual-optimal solution of D ∗ to or different from . The efficiency of first solving the relaxed primal problem P by column generation to obtain ∗ depends mainly on the tightness of the dual inequalities. For the second phase, that is the solution of P¯ ∗ , Ben Amor (2002) gives both practical and theoretical arguments that plead in favor of the efficiency of column generation when initialized with a small box strictly containing a dual-optimal solution (see also §5). This efficiency is more likely to be strengthened by the use of the columns already generated while solving P . In the next sections, we show how dual-optimal inequalities may be applied, and their effect on column generation efficiency.

4. The Binary Cutting-Stock Problem Given a set of rolls of width L and unit demands for items of length wi i ∈ I = 1 m, the binary cutting-stock problem (BCS) aims at finding the cutting patterns that should be used to satisfy the demands while minimizing the total number of rolls utilized. A feasible cutting pattern is a subset C ⊂ I such that i∈C wi L. Let be the set of all feasible cutting patterns. The widely known formulation (Gilmore and Gomory 1961) uses binary variables: xp is equal to 1 if the cutting pattern p ∈ is used in the solution and 0 otherwise. We are interested in solving the linear relaxation of this problem. Consider the following primal and dual linear programming formulations PBCS and DBCS , respectively, where the binary coefficient aip i ∈ I p ∈ indicates if item i is used or not in pattern p: vPBCS = min xp p∈

p∈

aip xp = 1

i ∈ I

xp 0 p ∈ vDBCS = max i i∈I

i∈I

aip i 1

p ∈

The primal problem is solved by column generation. Indeed, the binary cutting-stock problem can also be formulated as a vehicle-routing problem in which the vehicles

and their capacity correspond to the large rolls to be cut into smaller items and their widths, respectively; the customers and their demands correspond to the items and their lengths, respectively; and the vehicle routes are the cutting patterns. These patterns are obtained by solving a binary knapsack problem, a special case of the shortest-path problem with time windows. In our case, it is formulated on an acyclic graph with a capacity window on each node, and solved by dynamic programming (Desrochers and Soumis 1988). Large bin packing BCS instances with integer item size tend to have several items of the same size, which are better replaced by a single item with demand equal to the number of duplicates (Vanderbeck 1999). This aggregation is equivalent to requiring that the dual variables corresponding to items of the same length be equal. This is strengthened by the following proposition, which shows that these conditions are indeed deep dual-optimal inequalities. It also shows that aggregation preserves the same linear programming bound for BCS. Proposition 6. For any binary cutting-stock problem for which wi = wj , i j ∈ I, there exists a dual-optimal solution for DBCS satisfying i∗ = j∗ . Proof. Let i j ∈ I be two items such that wi = wj and let ¯ ∈ m be an optimal solution for DBCS . Construct the vector of dual multipliers ∗ such that  ¯ + ¯ j   i # ∈ i j 2 #∗ =   otherwise ¯ # It is obvious that i∈I i∗ = i∈I ¯ i . We show that ∗ is feasible for DBCS , and hence optimal. Assume, without loss of generality, that ¯ i < ¯ j . Let i Ri represent a feasible cutting pattern containing item i but not item j, where Ri denotes the set of remaining items. Therefore, there also exists a feasible pattern j Ri , where item j replaces item i. Let i j Rij represent a feasible pattern containing both items i and j, where again Rij denotes the set of remaining items. The associated dual constraints are ¯ # 1 ¯ j + ¯ # 1 ¯ i + ¯ j + ¯ # 1 ¯ i + #∈Ri

#∈Ri

#∈Rij

∗

Now let ¯ be replaced by . Because ¯ i < i∗ = < ¯ j , the feasibility of the leftmost constraint is not compromised, as well as the two others, ensuring the feasibility of ∗ and its optimality. To complete the proof, the procedure is repeated as long as there exist items i and j such that wi = wj , but for which the dual variables differ. j∗

We tested two different ways to impose these dual constraints: constraint aggregation of identical items in the primal formulation and explicit addition of equality constraints in the dual formulation. Each way has different effects on problem size and structure. We address them separately.


458


Constraint Aggregation. In PBCS , demand constraints corresponding to items of the same size are aggregated into one constraint that has a right-hand side demand greater than 1 As a consequence, a feasible pattern may be used more than once and an item may appear more than once (see the cutting-stock formulation in the next section). The pricing problem network contains many fewer arcs because all nodes corresponding to identical-size items are represented as a layer having a single input node; for details, see Ben Amor (1997). The costs are modified by assigning the same dual value to all arcs leaving nodes corresponding to items having identical lengths. A feasible solution to the original problem PBCS is obtained by renumbering copies of any item and separating patterns used more than once into different patterns used only once. The constraint aggregation results in reducing (drastically for most of the test problems) the number of rows in the primal problem, the number of feasible cutting patterns (columns), and the size of the knapsack subproblem.

m ∈ 500 1000, wi i∈I ∼ U lmin lmax , lmin ∈ 1 20, and lmax = 100; and one class of triplets (t-problems) in which each roll is cut into exactly three orders without loss, comprising m = 501 items. The item lengths are all relatively close to L/3 and the (integer and continuous) optimal number of rolls for the triplet instances is m/3. Ten instances are generated for each class, and average values are computed for each class. The preprocessing procedure of Martello and Toth (1990) is applied to all cutting-stock instances. It consists of eliminating all pairs of items i j such that wi + wj = L. There always exists an optimal primal solution containing patterns formed by such pairs (this is also true for the integer solution). As a consequence, the total number of rows in the master problem is reduced. This number is denoted m in all tables. Due to their special structure, the preprocessing procedure has no effect on the triplets. All experiments are carried out on a Sun Ultra-10/440 (int95 = 18 1, fp95 = 22 7, 640M). Table 1 presents the results obtained for each class using standard column generation, constraint aggregation of items, and addition of equality constraints. lrs, mps, and sps give the cpu times (in seconds) spent in solving the linear relaxation, the master problem, and the subproblem, respectively (where lrs = mps + sps); itr is the number of column generation iterations needed to reach optimality, and % itr is the relative improvement in the number of column generation iterations. Because the subproblem is solved by dynamic programming as a constrained shortest-path problem, a number of negative reduced-cost columns is available at optimality. Hence, several columns are generated at each column generation iteration. It is worth noting that both methods use the same column generator, and this does not influence comparison between them. Table 1 clearly shows how aggregation of identical items drastically reduces the problem size in terms of rows m compared to the standard version. Hence, the

Equality Constraints. Ranking item sizes by decreasing order, for every item pair i i + 1 such that wi = wi+1 , BCS the equality constraint − i + i+1 = 0, we add in D that is, Om new dual constraints. This results in adding a priori in P BCS the corresponding zero-cost variables yi i+1 with coefficients −1 in row i, +1 in row i + 1, and 0 elsewhere. The number of columns is slightly increased, but the number of rows remains unchanged, as well as the size of the knapsack subproblem. The corresponding column in the primal allows for the use of a certain item i in an optimal solution in place of the identical size item i + 1. An optimal solution to PBCS is recovered by substituting item i + 1 to item i in any overcovered 1 pattern using item i. Computational Results. As test problems, we used the most difficult ones taken from the operations research library (Beasley 1990). These are four classes of uniformly generated problems (u-problems) with parameters L = 150, Table 1.

Computational results for the binary cutting-stock problem. Solution strategies

m

u-50020100

Standard column generation Constraint aggregation Equality constraints

251 2 51 7 251 2

u-100020100


u-5001100

Problems

lrs

mps

sps

itr

% itr

27 2 0 3 5 2

7 0 0 1 0 2

20 2 0 2 5 0

63 9 12 8 14 5

— 79 9 77 3

465 4 53 7 465 4

242 9 0 3 28 8

63 2 0 1 0 4

179 6 0 2 28 4

94 6 13 1 14 4

— 86 2 84 8


314 8 71 5 314 8

105 2 1 4 15 5

28 8 0 1 0 5

76 4 1 2 15 0

66 4 15 3 10 9

— 77 0 83 4

u-10001100


589 0 71 7 589 0

1040 3 1 4 60 7

409 9 0 1 0 7

630 4 1 3 60 0

92 3 10 2 7 8

— 89 0 91 6

t-501


501 0 194 3 501 0

522 4 20 6 276 7

57 2 2 5 6 1

465 2 18 2 270 6

239 3 124 2 133 1

— 48 1 44 4


459


instances become much easier to solve, and we observe an extraordinary decrease in all cpu times and in the number of column generation iterations. Regarding the explicit addition of the dual-equality constraints, we also observe a significant reduction (although smaller) in all cpu times even though m remains unchanged compared to the standard version and Om static columns were added to the master problem. More importantly, however, we observe on average the same decrease in the number of column generation iterations. This shows that these dual-optimal inequalities have an amazing stabilization effect on this column generation process. Even though the constraint aggregation and equality constraint models are equivalent, they have different behavior. Indeed, this is due to the subproblem solution. In both cases, columns are generated by solving constrained shortest-path problems, that is, by using a dynamic programming labeling algorithm. The aggregated version contains many fewer nodes and arcs; hence, it generates much less labels.

5. The Cutting-Stock Problem The cutting-stock problem (CS) is defined in the same way as the binary cutting-stock problem with the difference that demands bi i = 1 m have general positive integer values. As a consequence, an item may appear more than once in a feasible cutting pattern (aip i ∈ I p ∈ takes nonnegative integer values) and a feasible cutting pattern may be used more than once in a feasible solution (xp p ∈ takes nonnegative integer values). The knapsack constraint in the subproblem is replaced by i∈I

aip wi L

p ∈

vPCS = min

p∈

p∈

xp

aip xp bi

i ∈ I

xp 0 p ∈ vDCS = max bi i i∈I

i∈I

aip i 1

i 0

Proposition 7. For item i ∈ I and subset S ⊂ I, any optimal solution for DCS satisfies wi w# ⇒ i∗ #∗ #∈S

p ∈

i ∈ I

As a consequence of the transformation proposed by Gilmore and Gomory, the dual variables take only nonnegative values. It is easy to show by contradiction that these

#∈S

Proof. We give a proof by contradiction. Let x∗ ∗ be a ∗ pair of optimal primal-dual solutions and assume that i < ∗ #∈S # for item i and subset S such that wi #∈S w# . For any feasible cutting pattern i Ri involving item i, there is a feasible cutting pattern obtained by replacing i with set S, S Ri . These patterns define two dual constraints ∗ ∗ ∗ i∗ + # 1 and # + # 1 #∈Ri

Gilmore and Gomory (1961) also showed that the equality signs in the demand constraints can be replaced by greater or equal signs, and that the column generation solution process is slightly accelerated by this simple transformation. The linear relaxation PCS and its dual DCS are formulated as follows:

dual nonnegativity constraints are satisfied by all optimal solutions. Indeed, assume that the primal problem is formulated using equality constraints and ∗ is an optimal solution to the corresponding dual problem such that there exists an index k ∈ I for which k∗ < 0. Because bk > 0, the value of the dual objective setting k∗ = 0 improves on function while constraints i∈I aip i 1, p ∈ remain feasible. In fact, for each pattern containing item k, there is a reduced pattern containing the same set of items except item k. Because akp k∗ = 0 in both cases, the corresponding constraints are equivalent. Hence, a contradiction, and i 0, i ∈ I is a set of dual-optimal inequalities. We now address the dual subset inequalities introduced by Valério de Carvalho (2005b). Using Proposition 3, the author justifies the use of these cuts by the fact that it is possible to reconstruct a primal-optimal solution given an optimal solution of the modified primal problem. In this paper, we show that these dual cuts are in fact dual-optimal inequalities for the cutting-stock problem.

#∈S

#∈Ri

The assumption made above implies that i∗ + #∈Ri #∗ < ∗ ∗ #∈Ri # + #∈RS # 1, and the left-most constraint cannot be active at optimality. Applying complementarity slackness conditions, all primal variables corresponding to patterns containing item i must equal 0, a contradiction on the feasibility and optimality of x∗ . When S = 1, these subset inequalities have an interesting interpretation. Any optimal solution for D satisfies the ranking constraints w1 w2 · · · wm ⇒ 1∗ 2∗ · · · m∗ Additionally, applying this result to two items i and j of the same length, we obtain that all dual-optimal solutions to DCS satisfy wi = wj ⇒ i∗ = j∗ . Interestingly, however, this is not valid for the binary cutting-stock problem, even though it is often considered as a special case of the cutting-stock problem. Indeed, there is an implicit constraint while generating binary cutting patterns that an item cannot be cut more than once, whereas for the classical cutting-stock problem, an item may be cut an arbitrary integer number of times, independently of the demand for that item.

460


Example. Consider a binary cutting-stock problem with m = 2, w1 = 4, w2 = 4, W = 10, and demands equal to 1. There are only three feasible binary patterns for it: 0w1 w2 1 0w1 1 0w2 1, with corresponding nonnegative primal variables x1 , x2 , and x3 . The dual domain is defined by 1 2 0 1 + 2 1 1 1 2 1 It is easy to verify that the dual points 1 0 and 0 1 are dual-optimal solutions corresponding to the primal-optimal solution x1 = 1, x2 = x3 = 0. These dual solutions do not satisfy 1 = 2 even though w1 = w2 . Indeed, Proposition 6 only stipulates that there exists a dual-optimal solution for which 1 = 2 (obtained here as a convex combination of the above dual-optimal solutions). Now consider this instance as a classical cutting-stock problem, that is, we drop the binary restrictions from the pattern generation. Two new patterns are feasible: 0w1 w1 1 and 0w2 w2 1. These modify the dual space with two new constraints 2 1 1 and 2 2 1. The unique dual-optimal solution is now 1 = 2 = 1/2, a direct consequence of Proposition 7, where it is shown that all optimal solutions satisfy 1 = 2 . Although they do not cut any optimal solutions, dualoptimal inequalities are efficient if they cut a significant part of the dual space. Hence, at least some feasible points becomes of D are eliminated and the feasible domain of D strictly included in the feasible domain of D. Regarding the subset inequalities of Proposition 7, any dual solu tion such that wi #∈S w# and i < #∈S # is eliminated. Consider, for example, a fixed k ∈ S and let ak = maxp∈ akp >0 akp . The solution defined by k = 1/ak and i = 0 for i ∈ I\k is feasible for D but infeasible for D. Knowing that this is true for each l ∈ S where S satisfies l∈S wl wi for some i ∈ I\S, these dual-optimal inequalities eliminate the convex hull of these S points and the null dual vector. The number of subset inequalities is exponential, and one has to choose a reasonable number to be used. If S is fixed, the number of possible cuts is polynomial in m. The simplest cases are those for S 2. The number of ranking inequalities, that is, when S = 1, is m − 1. When S = 2, the number of possible dual constraints is Om3 . For the sake of efficiency, especially when m is large, Valério de Carvalho (2005b) uses only a single dual constraint, if any, for each i ∈ I. This inequality is generated using the smallest index j (corresponding to the largest width wj ), if such a value exists, for which there is a k, such that wi wj + wk . CS and columns This results in Om constraints added to D added to the primal PCS . The columns added to the primal problem P CS have a special meaning. Zero-cost variables yi S are introduced with coefficients −1 in row i, +1 in rows # ∈ S, and 0 otherwise. This allows the primal problem to replace item i with items of S in any feasible pattern containing i. Hence,


the presence of cutting patterns containing i along with the column corresponding to variable yi S implicitly ensures the presence of all corresponding feasible cutting patterns where i is replaced with the items of set S. Consequently, these columns do not need to be generated. This reasoning can be applied recursively to some items of S, and therefore a number of subset inequalities with S 3 are also implicitly considered. Finally, the above paragraph gives a straightforward way to reconstruct a primal-optimal solution. For any overcovered item i, the overcovering patterns must be modified by replacing i with items of set S for which yi S > 0. Computational Results. As test problems, we again used five classes for which 10 instances are generated and average values are computed: the same class of t-problems with m = 501, the two most difficult classes of u-problems with parameters m ∈ 500 1000 and lmin = 1, and two new classes defined below. The two classes of u-problems are obtained from the binary cutting-stock instances by aggregating items of the same size, resulting in items having different lengths and nonnegative integer demands. The total demand of all items remains at 500 and 1,000, respectively. The preprocessing procedure of Martello and Toth (1990) is next applied to the aggregated version, giving rise to the average numbers of distinct item lengths m provided in Table 2: 75.1 for u-5001100 and 71.7 for u-10001100. Therefore, the average total demand of all items is changed to 314 8 for the u-5001100 problems and 589.0 for the u-10001100 problems. The two new sets of test problems, denoted hba-5001100 and hba-10001100, are created in such a way that they are much more difficult to solve than the u-5001100 and u-10001100 classes. Starting from the aggregated version of u-problems with total demands 500 and 1,000, the new problems are constructed in two steps. For each item length wi with demand bi , minbi 10 different items are created with distinct lengths wi# = wi + 0 1# mod 10 for # = 0 minbi 10 − 1 and demand 1. If bi > 10, the last created wi9 item is reassigned a demand of bi − 9. Next, the preprocessing procedure of Martello and Toth (1990) is applied to the aggregated version, giving rise to the average numbers of distinct item lengths m provided in Table 2: 396.4 for hba-5001100 and 479.1 for hba-10001100. Building new item lengths before applying the Martello and Toth (1990) preprocessing procedure aims at keeping more distinct item lengths in the new classes. In fact, some items that could be eliminated by the preprocessing procedure have their length modified and the elimination criteria no longer applies. In the case of hba5001100 problems, the average demand per item length is 314 8/71 5 ≈ 4 4 < 10. Hence, the resulting number of distinct item lengths should be close to the total demand of u-5001100 problems after preprocessing, that is, 314.8; the actual number is m = 396 4. In the case of hba-10001100


461


problems, this is a litte bit different. u-10001100 problems have the same structure as u-5001100 problems: The number of distinct item lengths after aggregation is very close, that is, 71.5 and 71.7, respectively. However, because the total demand of u-10001100 is much larger (i.e., 1,000), the average demand per item length is 589 0/71 7 ≈ 8 2, a value much closer to 10. Therefore, relatively more items are eliminated by the preprocessing. This explains the fact that the actual average number of distinct item lengths of 479.1 is much smaller than 589.0. This is, however, larger than the corresponding number of 396.4 for hba-5001100. The results are presented in Table 2. Applying constraint aggregation to all these test-problem sets shows that the u-problems are very easy. For the hba-problems, the cpu time is very high, which is due to the large number of column generation iterations and the costly subproblem solution at each iteration. The t-problems need a relatively small amount of cpu time, but the number of column generation iterations remains quite high. We then used the two variants of the dual-optimal inequalities presented in the previous paragraphs: the ranking inequalities S = 1 and the subset inequalities defined with S 2. For the u-problems, the cpu time is slightly reduced, but we observe a quite significant improvement in the number of column generation iterations, that is, a decrease of 37.3% and 43.1% for u-5001100 problems, and 20.6%, and 18 6% for u-10001100 problems. This is very interesting and shows that dual-optimal inequalities may have a significant impact even on problems that are easy to solve. We observe the biggest improvements on the difficult hba-problems. The number of column generation iterations is divided by a factor varying between 5 and 7 (76.5% and 85.2%). The cpu time of the master problem is also significantly reduced. The total cpu time reduction varies from 30% to 46%. However, we observe that even if the subproblem cpu time is reduced due to the much smaller number of iterations, it remains quite high. This is due to the fact Table 2.

that in the very last column generation iterations, the knapsack subproblem is very costly when solved by dynamic programming as a constrained shortest-path problem. Finally, for the t-problems, we observe a slight improvement, especially on the number of column generation iterations, that is, 8.8%. Due to the special structure of these problems, there is no subset inequalities for S 2, so only the ranking inequalities can be used. The small impact of these cuts may be explained by the fact that item lengths are all close to L/3, and hence requiring a ranking of the dual variables does not affect their values significantly at the end of a column generation iteration.

6. Deep Dual-Optimal Inequalities for the Triplets The triplets in the t-problems are built in such a way that the loss is null at optimality. Table 2 shows that given their size, these problems are quite difficult and require a very high number of column generation iterations. However, there is a very interesting result for PCS instances with no loss. In fact, one can compute directly from the item lengths not only the optimal number of rolls m i=1 bi wi /L, i.e., the total length of all items divided by L, but also a dual-optimal solution. Proposition 8. Consider an instance of PCS with no loss at optimality. Then, i∗ = wi /L, i ∈ I is a dual-optimal solution. Proof. Assigning these values to the dual constraints a i∈I ip i 1, p ∈ simply replicates the knapsack constraint used to build the feasible patterns: Obviously, they are satisfied. The corresponding dual-objective function reaches the optimal value m b w i=1 i i /L. Therefore, this dual solution is optimal. As in Proposition 5, given a known dual-optimal solution ∗ , it is possible to use the following deep dual-optimal inequalities ∗ − ∗ + , where

Computational results for the cutting-stock problem.

Problems

Solution strategies

m

u-5001100

Constraint aggregation Ranking inequalities Subset inequalities

71 5 71 5 71 5

1 4 1 1 1 0

0 1 0 1 0 1

u-10001100


71 7 71 7 71 7

1 4 1 2 1 1

hba-5001100


396 4 396 4 396 4

hba-10001100


t-501

Constraint aggregation Ranking inequalities

lrs

mps

sps

itr

% itr

1 3 1 0 0 9

15 3 9 6 8 7

— 37 3 43 1

0 1 0 1 0 1

1 3 1 1 1 0

10 2 8 1 8 3

— 20 6 18 6

1153 7 813 5 747 9

64 8 26 9 26 1

1088 9 786 6 721 8

129 6 30 5 25 0

— 76 5 80 7

479 1 479 1 479 1

4301 4 2539 3 2324 8

190 1 43 1 41 9

4111 3 2496 2 2282 9

138 3 25 1 20 4

— 81 9 85 2

194 3 194 3

20 6 19 3

2 4 2 2

18 2 17 3

124 2 113 3

— 8 8


462


Table 3.

Computational results for the triplets.

Problems

Solution strategies

t-501

m

lrs mps sps

Constraint 194 3 20 6 aggregation Ranking 194 3 19 3 inequalities Box inequalities 194 3 7 9

itr

% itr

2 5

18 2 124 2

—

2 2

17 3 113 3

8 8

0 5

7 4

12 2 90 2

> 0 ∈ m is a vector of small scalars. These box inequalities restrict the dual values during all column generation iterations. The solution of problem P¯ ∗ is accelerated by noting that a basic solution is already available using the slack variables y2 = b, and for the corresponding basis (an identity matrix), the dual multipliers are almost optimal for small and take value ∗ + . Computational Results. We have tested the box inequalities on the t-problems with i∗ = wi /L, i ∈ I, and = 10−2 . The results are presented in Table 3, along with results previously obtained for these instances. There is a great improvement in all cpu times and especially in the number of column generation iterations, which are nearly divided by 10 compared to other methods. These problems, which are often considered to be difficult, are indeed very easy when solved adequately using a known dual-optimal solution.

7. Conclusion This paper discusses the usefulness of dual information for column generation efficiency. Dual-optimal inequalities do not cut any dual-optimal solutions, whereas the deep ones may remove a nonempty subset of these. The use of these inequalities results in restricting the dual space while relaxing the primal feasible set. Hence, the set of possible intermediate values for the dual multipliers is restricted and a dual-optimal solution is reached faster. However, primal feasibility may be lost, and one needs to recover a primal-optimal solution. In some cases, the dual inequalities have a special meaning for the application on hand and it is possible to build a primal-optimal solution given a solution of the modified problems. When this is not possible, we proposed two general methods to recover primal feasibility and optimality, one for each type of inequalities. The well-known aggregation technique used for the binary cutting-stock problem can be seen as an implicit way of implementing a set of deep dual-optimal inequalities. This implicit way drastically reduces the number of rows so that one cannot determine the real impact of the dual inequalities. Hence, we tested the explicit implementation of these dual constraints, with the consequence of an increase in the number of variables for the primal without changing the number of rows. We observed the same improvement in the reduction of the number of column

generation iterations, which confirms the importance of relevant dual information. For the cutting-stock problem, we showed that the subset inequalities introduced by Valério de Carvalho (2005b) to accelerate the column generation process are in fact dual-optimal inequalities. We also showed that the triplet instances, often considered to be difficult cutting-stock instances, are indeed very easy when solved using dual-optimal box inequalities derived from a readily computed dual-optimal solution. The computational results again show that significant dual information is clearly beneficial to the column generation solution process. A similar procedure has been proposed for the design of an efficient crossover method for linear programs (Ben Amor et al. 2006). Given an optimal-interior primal solution and a dual-optimal solution ∗ provided by an interior point algorithm, a primal-basic solution can be retrieved by re-solving the primal problem with a simplex algorithm while adding small dual box inequalities (e.g., ∗ ± 10−3 ) in the dual formulation. This simple procedure is very stable and quite fast. Finally, the cutting-stock problem is a nicely structured optimization problem, but many other combinatorial optimization problems also have a nice structure. The results obtained for the cutting-stock problem show that it is interesting to seek better characterizations of the dual-optimal solutions for other combinatorial problems.

Acknowledgments The work of the second author was supported by the Canadian Natural Science and Engineering Research Council under grant 6815-04. The work of the third author was partially supported by FCT—Fundação para a Ciência e a Tecnologia (Projecto POSI/ 1999/ SRI/ 35568) and by Centro de Investigação Algoritmi da Universidade do Minho, and was developed in the Grupo de Engenharia Industrial e de Sistemas. References Beasley, J. E. 1990. OR-Library: Distributing test problems by electronic mail. J. Oper. Res. Soc. 41 1060–1072. Ben Amor, H. 1997. Résolution du problème de découpe par génération de colonnes. Master’s thesis, École Polytechnique de Montréal, Montréal, Quebec, Canada. Ben Amor, H. 2002. Stabilisation de l’algorithme de génération de colonnes. Ph.D. thesis, École Polytechnique de Montréal, Montréal, Quebec, Canada. Ben Amor, H., J. Desrosiers, F. Soumis. 2006. Recovering an LP basis from an optimal dual solution. Oper. Res. Lett. Forthcoming. Desaulniers, G., J. Desrosiers, I. Ioachim, M. M. Solomon, F. Soumis, D. Villeneuve. 1998. A unified framework for deterministic time constrained vehicle routing and crew scheduling problems. T. G. Crainic, G. Laporte, eds. Fleet Management and Logistics. Kluwer, Norwell, MA, 57–93. Desrochers, M., F. Soumis. 1988. A generalized permanent labeling algorithm for the shortest path problem with time windows. INFOR 26 191–212.

Ben Amor, Desrosiers, and Valério de Carvalho: Dual-Optimal Inequalities for Stabilized Column Generation Operations Research 54(3), pp. 454–463, © 2006 INFORMS

du Merle, O., J.-Ph. Vial. 2002. Proximal ACCPM, a cutting plane method for column generation and Lagrangian relaxation: Application to the p-median problem. Technical report, Université de Genève, Geneva, Switzerland. du Merle, O., D. Villeneuve, J. Desrosiers, P. Hansen. 1999. Stabilized column generation. Discrete Math. 194 229–237. Gilmore, P. C., R. E. Gomory. 1961. A linear programming approach to the cutting-stock problem. Oper. Res. 9 849–859. Hiriart-Urruty, J.-B., C. Lemaréchal. 1993. Convex Analysis and Minimization Algorithms, Part 2: Advanced Theory and Bundle Methods, Vol. 306. Grundlehren der Mathematischen Wissenschaften. Springer, Berlin, Germany. Kallehauge, B., J. Larsen, O. B. G. Madsen. 2001. Lagrangean duality applied on vehicle routing with time windows. Technical report IMM-TR-2001-9, Informatics and Mathematical Modelling, Technical University of Denmark, Kgs. Lyngby, Denmark. Lasdon, L. S. 1970. Optimization Theory for Large Systems. Macmillan, London, UK. Mamer, J. W., R. D. McBride. 2000. A decomposition-based pricing

463

procedure for large-scale linear programs—An application to the linear multicommodity flow problem. Management Sci. 46(5) 693–709. Marsten, R. E. 1975. The use of the boxstep method in discrete optimization. Math. Programming Stud. 3 127–144. Martello, S., P. Toth. 1990. Knapsack Problems: Algorithms and Computer Implementations. Discrete Mathematics and Optimization. WileyInterscience, Chichester, UK. Neame, P. 1999. Nonsmooth dual methods in integer programming. Ph.D. thesis, University of Melbourne, Melbourne, Australia. Valério de Carvalho, J. M. 2005a. LP models for bin-packing and cutting stock problems. Eur. J. Oper. Res. 141(2) 253–273. Valério de Carvalho, J. M. 2005b. Using extra dual cuts to accelerate convergence in column generation. INFORMS J. Comput. 17(2) 175–182. Vance, P. H., C. Barnhart, E. L. Johnson, G. L. Nemhauser. 1994. Solving binary cutting stock problems by column generation and branch-andbound. Comput. Optim. Appl. 3(2) 111–130. Vanderbeck, F. 1999. Computational study of a column generation algorithm for bin packing and cutting stock problems. Math. Programming 86(3) 565–594.