Apr 11, 1997 - tion â and the analytic center cutting plane method. We conclude that the ... than subgradient optimization or Kelley's cutting plane method.
A Lagrangian Relaxation of the Capacitated Multi-Item Lot Sizing Problem Solved with an Interior Point Cutting Plane Algorithm O. du Merle
J.-L. Goffin
C. Trouiller
J.-P. Vial y
April 11, 1997
Abstract The capacitated multi-item lot sizing problem consists of finding a production schedule that minimizes over a finite number of periods the total production, holding inventory, and setup costs subject to demand and capacity constraints. The CLSP problem is NP-hard, while the problem of finding a feasible solution, which is polynomial if there are no set-up times, becomes NP-complete when set-up times are included. Approximate solutions can be obtained by heuristics. In this paper we consider an approach based on a Lagrangian relaxation of the capacity constraints. The relaxation is used in two ways. First, it generates a lower bound for the optimal value. Second, the primal and dual solutions of the relaxation (if available) are used to generate integer feasible solutions by primal or dual heuristics. We compare three methods of solving the Lagrangian relaxation: subgradient method, Kelley’s cutting plane method – also known as Dantzig–Wolfe decomposition – and the analytic center cutting plane method. We conclude that the analytic center cutting plane method performs as well, and sometimes better than subgradient optimization or Kelley’s cutting plane method. This
research has been supported by the Fonds National de la Recherche Scientifique Suisse (grant # 12-34002.92), NSERC-Canada, and FCAR-Qu´ebec. y The address of the the last author is D´epartement HEC, Universit´e de Gen`eve, Geneva, Switzerland; the address of the others is GERAD, Faculty of Management, McGill University, Montr´eal, Canada
1
Introduction The capacitated multi-item lot sizing problem (CLSP) is a model which aims at scheduling production of several products over a planning horizon, while minimizing linear production costs, holding inventory costs and specified setup costs subject to demand and capacity constraints. These costs may vary for each product and each period. A large amount of work has been devoted to the CLSP, because it is the core problem in the Aggregated Production Planning (APP) models used for determining the load and resource allocation in a production environment. These form inputs to Master Production Schedules and consequently to Materials Requirement Planning (MRP) in a “push” type manufacturing environment (see Bhatnagar et al. [6] for a review of these models). CLSP is known to be a NP–hard problem. Three groups of researchers pioneered work on this problem: Manne [31], using a linear programming model; Dzielinski and Gomory [15], based on Dantzig-Wolfe decomposition [9]; Lasdon and Terjung [27], using generalized upper bounding procedures. In these three papers, the solutions are approximate, not necessarily feasible and the reported costs are not necessarily correct, because setup times and costs are charged only once even when a batch is split between periods. Some algorithms reach an approximate solution in a single pass (see Dixon and Silver [12], Maes and VanWassenhove [30]). These algorithms are commonly called “smoothing” heuristics. The method proposed by Barany, Van Roy and Wolsey [4], solves CLSP without setup times optimally, using a cutting plane procedure followed by a Branch and Bound. So far, one of the most promising approaches to the CLSP seems to be the Lagrangian relaxation. Most of the studies about CLSP did not take the setup times into account, partly because some authors considered that including setup times is a trivial extension of CLSP without setup times. Trigeiro et al. [41] have shown that CLSP with setup times is a much more difficult problem to solve than the CLSP without setup times and than what had been reported in the literature. As they reported, the importance of properly accounting for the effect of setup times on capacity usage has been highlighted recently by the growing attraction of Just-In-Time manufacturing (JIT). In a JIT manufacturing environment, setup times are reduced as much as possible in order to permit the economic production of small lot sizes and therefore much lower level of work-in-process (i.e. stockless production). It leads to an increase in productivity (possible by automatization), an improvement of quality and a reduction of production lead times (hence, of customer response 2
times). But reducing setup times does not eliminate the need for research on manufacturing lotsizing. Setup times remain (and can sometimes be important) and must be taken into account in an accurate decision making model. Moreover, small setup times allow the firm to reduce the manufacturing cycle, thus increasing the number of setups. Very often, the total time allocated to setups is roughly the same (see Thomas and Weiss [40]). According to that, our model includes setup times. Thizy and Van Wassenhove [38] and Trigeiro et al.[41] developed a Lagrangian relaxation of the capacity constraints and updated the multipliers using a subgradient method, and heuristic based procedures to obtain feasible solutions of the CLSP. We are going to use the same relaxation, but updating the Lagrangian multipliers will be done by a cutting plane methodology. Those methods provide information that subgradient optimization does not, such as: duality gap, that guarantees the accuracy at termination, and dual variables of the Lagrangian formulation that can be used to design original heuristics to provide primal feasible solutions of CLSP and criterias to branch in an additional Branch and Bound procedure. In addition those iterative approaches allow to integrate multi-cuts at each step, and decrease significantly the number of iterations to compute the Lagrangian multipliers. What defines a a specific cutting plane algorithm is the choice of a proposition in the current polyhedral approximation where the oracle will be asked for some additional information. In this paper, we compare two different approaches. The first one is known as Kelley’s cutting plane algorithm [26] or Benders decomposition [5] and computes the optimal point of the current relaxation. By duality, this is equivalent is the primal decomposition principle, also called column generation algorithm, due to Dantzig and Wolfe [9]. This class of methods has been reported to perform well on some problems, but poorly on some others [14] and the performance may vary for different formulations of the same problem. Specific implementations of this cutting plane method have been shown to have extremely poor complexity bounds [35]. The second approach is based on interior point methodology, and uses the analytic center of a set of localization. (See Goffin et al. [19, 13]). This novel cutting plane method has been shown to achieve competitive practical performance (see [2, 3, 21, 28]) as well as better complexity bounds (see [1, 20, 36]). The advantages of the cutting plane methods over subgradient optimization are twofold: they allow a disaggregation of the objective that results in a considerable reduction of the number of iterations, and they provide dual variables that may be 3
used in primal heuristics in order to generate interesting integer feasible solutions. The heuristics used are either the dual heuristic due to Trigeiro et al.[41] or the primal heuristic studied by Trouiller[42], which expands on the works of Br¨annlund[7] and Thizy and Van Wassenhove [38]. The paper presents an extensive set of experiments on a series of problem tests provided by Trigeiro. The results show that the subgradient method behaves poorly on some difficult problems as compared with the cutting plane methods. On a standard implementation of the aggregated version ACCPM clearly dominates KCPM, especially on large problems. When disaggregation is exploited, both methods are on a par for moderate size problems; however ACCPM is increasingly more efficient, as the size of the problem grows. The larger problems involve up to 4 items, 225 periods. We conclude that the analytic center cutting plane method performs as well, and sometimes better than subgradient optimization or Kelley’s cutting plane method. Other approaches, not described here, include Lagrangean decomposition studied by Thizy [39], Millar and Yang [32] and Trouiller [42], or generic interior point integer programming codes (see Mitchell [33]). This paper is organized as follows. Section 2 describes the problem, section 3 gives the Lagrangian relataxion of the CLSP problem and section 4 deals with optimization techniques to solve the relaxation. In Section 4, we propose an heuristic to obtain primal feasible solutions of the CLSP and some numerical experiments are presented in Section 5.
1 Problem Description 1.1 Notations For i index of products (i = 1; 2 : : :N ) and define the following notations: parameters
t index of periods (t = 1; 2 : : : T ),
T the set of periods as well as its cardinality, N the set of products as well as its cardinality, 4
Dit = demand for product i in period t, Kt = time capacity in period t for production, si = setup time for product i, pi = unit processing time for product i, Sit = production setup cost for product i in period t, Cit = unit production cost for product i in period t, Hit = cost of holding a unit of product i between periods t and t + 1.
variables Xit
= quantity of product i produced in period t, Iit = inventory of product i carried from period t to period t + 1, it = 1 if product i is setup at period t, and 0 otherwise.
1.2 CLSP with Setup Times The capacitated lotsizing problem with setup times and costs can be formulated as the following problem (P ):
N X T X
(Sit it + CitXit + HitIit ) i=1 t=1 s.t. Iit = Ii;t?1 + Xit ? Dit N X (piXit + siit ) Kt i=1 T X Xit ( Dik )it k=t Ii;0 = Ii;T = 0
min
Xit ; Iit 0 it 2 f0; 1g
(1)
8i 2 N; t 2 T
(2)
8t 2 T 8i 2 N; t 2 T 8i 2 N 8i 2 N; t 2 T 8i 2 N; t 2 T:
(3) (4) (5) (6) (7)
The model assumes that the production in one period is immediately available to the customer. Each expression can be interpreted as follows: 5
(1) minimizes the sum of the setup, production and inventory costs for all the products over the planning horizon, (2) means that the inventory levels at the end of period t are equal to the inventory levels at the end of period t ? 1 plus the amount produced minus the customer’s demand in period t,
(3) insures that, in period t, the consumption of the production resource and setup resource cannot exceed the capacity available, (4) insures that a setup occurs if product i is produced in period t,
(5) reset the initial and final inventory levels to zero, (a simple transformation makes it easy to transform a general problem with non-zero initial and final inventory levels into (P )), (6) imposes nonnegativity restriction on the variables, and prohibits backlogging.
The capacity constraints (3) are considered as complicating constraints in that they prevent the problem from separating into N one product problems, for which efficient methods exist. Chen and Thizy [8] have proved that the problem is strongly NP–hard; simply stated, it means that the computational price to pay to refine the accuracy of polynomial time heuristics will be exorbitant for some problems. Hence, heuristic approaches do not dominate searches for optimum solutions. Trigeiro et al. [41] have shown that CLSP with setup times is much harder to solve than CLSP without setup times. Indeed, for example, it is easy to test if solutions of the CLSP without setup times exist or not just by computing cumulative demand and cumulative capacity. With setup times, the feasibility problem is NP–complete (Bin packing is a special case of CLSP with setup times, see [17, p. 226]). Thus, there is no “easy” way to solve the feasibility problem, let alone solving, the CLSP with setup times. An alternative formulation, which may be viewed in some circumstances as a realistic description of the actual problem, and in others as a computational device, introduces artificial variables Ot in the capacity constraints; these variables can be
6
interpreted as overtime. The modified objective and capacity constraints are:
N X T X
(Sitit + M Ot + Cit Xit + Hit Iit ) i=1 t=1 N X s.t. (piXit + siit ) Kt + Ot i=1
min
8t 2 T:
This problem is always feasible, and if the overtime is zero, it is feasible for the original problem P . The cost of overtime, M may be viewed as a penalty, identical in substance to the well-known big-M method for linear programming; it can also be used to drive a heuristic method towards achieving feasibility (see Trigeiro et al. [41] ).
2 Problem Reformulation: Lagrangian Relaxation and Column Generation Consider a combinatorial optimization problem formulated as the integer problem (P ): Z = min cT x s.t.
Ax b Dx d x0
(8) and integral;
where the constraints (8) are considered as the complicating constraints. For the methods that we will review to be appealing and effective, the set X = fx : Dx d; x 0 and integralg must satisfy two somewhat contradictory properties:
it should be simple enough that that optimizing a linear objective over it is not computationally intractable, usually because X separates as a Cartesian Q product of simpler sets X = j 2J Xj , it should be complex enough that the sets Xj ’s do not have the integrality property, which frequently means that the subproblems are NP-hard. 7
2.1 Dantzig-Wolfe Column Generation Assume that X = fx : Dx d; x 0 and integralg is finite and bounded, then X can be represented as X = fxk : k = 1; : : : ; @g. The list of points @ may be restricted to the list of integer boundary points of co(X ), the convex hull of X , which are the only points that can be obtained if X is described by the optimization of a linear form over it. This allows us to express (P ) as the following integer program (P ):
min s.t.
@ X
k=1
@ X
k=1
@ X
k cT xk k Axk b k = 1
(9)
k=1 k 0
@ X
k=1
k xk
for all k
2@
integral:
The integrality constraint of (9) may be replaced by (k : k 2 @ binary) if @ lists all of the points of X and not just the the boundary points or the extreme points of co(X ).
The linear programming relaxation of P , i.e. the problem without the integrality restrictions, is (P r ):
min s.t.
@ X
k=1
@ X
k=1
@ X
k cT xk k Axk b
k = 1 k=1 k 0
(10) for all k 2 @:
It is well known (Geoffrion [18]) that the linear programming relaxation P r of P is stronger or equal to the linear programming relaxation Pr of P , with equality occurring if the set X has the so-called integrality property. The integrality 8
property occurs if the optimization of a linear objective over the LP relaxation of X gives the same optimal objective as the optimization over X . The rationale for the column generation approach to integer programming is that, if the integrality property does not hold, the bounds given by the reformulation P r may be (dramatically) much better than the bounds given by Pr .
The column generation approach solves P r by the column generation method due to Dantzig and Wolfe [9]. The solution of the original integer problem P is then attempted by heuristics, or branch and bound methods, or mixtures of both.
2.2 Lagrangian Relaxation Consider the partial Lagrangian associated to P , (LRu ):
T T L(u) = min x c x + u (Ax ? b) s.t. Dx d x 0 and integral;
where u 0 is a vector of Lagrange multipliers.
Denote by Z the optimal value of P , and also of P . It is well known that L(u) Z . This is easy to show by assuming an optimal solution x to (P ) and observing that: L(u) cT x + uT (Ax ? b) Z
since Z = cT x , u 0 and Ax ? b 0. It is not possible to guarantee finding u for which L(u) = Z , but the fact that L(u) Z allows (LRu) to be used to provide lower bounds for (P ). The choice of u giving the best bound on problem (D) (see Fisher [16]):
Z is an optimal solution to the dual
L = max L(u):
This allows us to express (D) as the following linear program (D):
T ZD = max z;u z ? b u s.t. z cT xk + uT Axk u 0; 9
for all k
2@
whose linear programming dual is (P r ).
Problem (D) makes it clear that L(u) is the minimum of a finite family of linear functions. It is thus continuous and concave, but nondifferentiable at any u where (LRu) has multiple optima. However, L(u) is subdifferentiable. A vector y is called a subgradient of L(u) at u if it satisfies:
L(u) L(u) + yT (u ? u), for all u The vector (Axt ? b) is a subgradient at any u for which xt solves (LRu ). Any other subgradient is a convex combination of these primitive subgradients. Thus, the complementary slackness conditions of optimality is equivalent to the fact that u? is optimal in (D) if and only if 0 is a subgradient of L(u) at u?.
2.3 Disaggregation In many important instances the set X , and the constraints representing it, have a block structure that may be exploited. Assume now that (P ) has the following structure: X Z = min cTj xj j 2J X s.t. Aj xj b (11) j 2J for all j 2 J Dj xj d;
xj 0
and integral
Assume that Xj = fxj : Dj xj dj ; xj then Xj can be represented as Xj = fxkj
for all j
2 J:
0 and integralg is finite and bounded, : k = 1; : : : ; @j g.
10
The integer program (P ) becomes:
min s.t.
@j XX
j 2J k=1 @j XX
kj cTj xkj kj Aj xkj b
j 2J k=1 @j X kj = 1 k=1 kj 0 @j X kj xkj integral k=1
2J for all k 2 @ for all j 2 J for all j 2 J:
(12)
for all j
The linear programming relaxation (P r ) of (P ) is:
min s.t.
@j XX
j 2J k=1 @j XX
kj cTj xkj kj Aj xkj b
j 2J k=1 @j X kj = 1 k=1 kj 0
(13)
2J for all k 2 @ for all j 2 J: for all j
The linear programming dual of this gives the disaggregated form of the Lagrangian relaxation (D):
P z ? bT u ZD = max j 2J j zj ;u s.t. zj cTj xkj + uT Aj xkj u 0:
If we define then
for all k 2 @j for all
Lj (u) = min cT xk + uT Aj xkj ; k2@j j j ZD = max u0
X
j 2J
Lj (u) ? bT u:
11
j2J
2.4 Formulation of the Lagrangian Relaxation of the CLSP The application to the CLSP is done by identifying the matrix A of Section 2 with the set of capacity constraints (3), and matrix D with the remaining constraints. Then, the Lagrangian problem (D) to solve is:
L(u) = min
N X T X i=1 t=1
(Sit it + CitXit + hit Iit) +
N T X X ut( (siit + piXit ) ? Kt) t=1
i=1
subject to:
Iit = Ii;t?1 + Xit ? dit T X Xit ( dik )it k=t Ii;0 = Ii;T = 0 Xit ; Iit 0 it 2 f0; 1g ut 0
for all i 2 N; t 2 T
(14)
for all i 2 N; t 2 T
(15)
for all i 2 N for all i 2 N; t 2 T for all i 2 N; t 2 T for all t 2 T
(16) (17) (18) (19)
Li(u) subject to (19);
(20)
which is equivalent to:
T X
N X
t=1
i=1
L(u) = ? ut Kt + where:
T X Li(u) = min ((Sit + utsi )it + (Cit + utpi)Xit + hit Iit )
(21)
subject to (14); (15); (16); (17); (18):
(22)
t=1
Note that the case “no setup times” can be handled by setting the setup times sit ’s to 0 in the Li (u)’s without changing the complexity of the problem. L(u) decomposes into N uncapacitated single item lotsizing subproblems Li (u). They can be solved by the classic Wagner-Whitin dynamic programming algorithm [43], whose complexity is in O(T 2), for the simplest version. 12
The introduction of overtime simply amounts to replacing the nonnegativity constraint (19) by 0 ut M for all t 2 T: A subgradient of L at u is given by:
L 2 @ L(u) N X L (u) = ?K + Li(u) 0
0
0
i=1
(23)
where K is the vector of capacities, and
Li 2 @ Li(u) 0
(24) Li(u) = (siit (u) + piXit (u))t=1;:::;T Note that the tth component of L (u), if positive, represents the extent by which 0
0
the capacity constraint in period t is violated.
3 Optimization Techniques to Solve the Lagrangian Relaxation 3.1 Subgradient Optimization Subgradient optimization simply moves the current iterate u in the direction of the subgradient L0 (u) by a stepsize t to define the next iterate:
u+ = u + tL0(u): Subgradient optimization, while being extremely popular has drawbacks which can be summarized as: 1. although a theory about its convergence exists, the algorithm stops after a certain number of iterations and it is assumed that the optimal value is reached, or well-approximated, without any information on the accuracy of this approximation, 13
2. the performance of the algorithm relies on the fine tuning of a couple of parameters (original point, number of iterations without improvement before the step size is changed, . . . ) making necessary for the user to have some skill and experience, 3. “dual” variables, i.e. the Dantzig-Wolfe multipliers, are not available, making primal heuristics unavailable 4. no use of the disaggregated formulation has ever been proposed, 5. the algorithm, being an extension of the steepest ascent algorithm, is memoryless, i.e. forgets all of the information about the previously generated cutting planes.
3.2 Kelley’s Cutting Plane or Dantzig–Wolfe Column Generation These two methods are equivalent, by duality. They build an approximation to the problems D (or P r ) that uses a subset K of the complete set @. This subset corresponds to the K cutting planes (or dually the columns) that were previously generated. This defines a polyhedral approximation, or relaxation 1 , to D denoted as D K as the following linear program:
T ZDK = max z;u z ? b u s.t. z cT xk + uT Axk u 0; 1
for all k
2K
note that the word relaxation is used with a different meaning than previously
14
(25)
whose linear programming dual P K is a restriction of P r :
min s.t.
K X
k=1 K X k=1 K X
k cT xk k Axk b
k = 1 k=1 k 0
(26)
for all k
2 K:
An iteration of the Dantzig-Wolfe algorithm (resp. Kelley’s cutting plane method) 1. solves the restriction P K (resp. the relaxation D K ), 2. obtains an optimal dual vector uK and an optimal primal vector K , 3. computes the value and a subgradient of L(u) at uK , 4. checks for optimality, and if optimality conditions are not satisfied, appends the new column to P K (resp. the new row to D K ) and updates the set K . These methods can use the disaggregated formulation in a straightforward and classical fashion. It is important to note that the Dantzig-Wolfe multipliers K lead to a feasible P solution for the linear relaxation of the original problem P , as k Axk b, which can be rewritten as:
X
i2N
(si
X
k2K
k it (uk ) + pi
X
k Xit (uk )) Kt for all t = 1; : : : ; T ;
k2K k k P k2K k it (u ); k2K k Xit (u )) is a feasible, possibly fractional, solution to the original problem P .
P which says that (
3.3 The Bundle Method The bundle method can be viewed as a variant of Kelley’s cutting plane method, that adds a quadratic regularization to the current cutting plane approximation, and finds an approximately optimal dual vector uK . 15
Even if it is not standard in most implementations of bundle methods, it is possible to compute an approximately optimal primal solution K , the Dantzig-Wolfe multipliers.
3.4 The Analytic Center Cutting Plane Method By contrast with the previous methods, the ACCPM does not attempt to solve the cutting plane approximation D K , but computes an approximate center to the set of localization:
f(z; u) : z + bT u K ; z cT xk + uT Axk for all k 2 K; u 0g; where K is the best recorded value (i.e., a lower bound on the optimum), defined by: K = max L(uk ): k2K A similar localization set can be defined for the disaggregated formulation. For the clarity of exposition, let us denote the linear inequalities defining the set of localization by:
= fy : AT y + s = c; s 0g; note than this section’s notation is not related to the notation used in the rest of this paper. The set is assumed to be bounded with a nonempty interior. To the interior of this set of (m) linear inequalities is associated the dual potential
'D (y) =
m X j =1
ln sj ;
The analytic center y is the unique maximizer of the dual potential over the interior of and thus the necessary and sufficient first order optimality conditions for this maximizer imply that there exist x > 0 and s > 0 such that the following holds:
Ax = 0 AT y + s = c X s = e: 16
(27)
On the other hand, it is easy to check that minimizing the primal potential
n X T 'P (x) = n ln c x ? ln xj ; j =1 over the relative interior of the primal feasible region fx : Ax = 0; x 0g gives the same optimality conditions. This implies that the analytic center can be computed approximately by using a damped projected Newton method, known as the de Ghellinck Vial variant of Karmarkar’s original projective method. Because of warm start considerations (restoration step) a primal algorithm is used. An iteration of the ACCPM 1. computes an approximate analytic center of the localization set minimizing the primal potential, which is a pair (uK ; K ) of feasible primal and dual solutions, 2. computes the value and a subgradient of L(u) at uK , 3. checks for –optimality, and if optimality conditions are not satisfied, updates the set of localization with this new information, 4. takes a special restoration step to recover interior primal feasibility. The design of the restoration step is critical to the success of the method. A good restoration direction should strive to have the following properties: 1. recover in one step a point which is sufficiently centered to allow for the fast computation of the approximate analytic center for the updated problem, 2. use in the algebra the information available at the old analytic center. We use an extension to this framework of the direction proposed by Mitchell and Todd [34], extension which has been justified in some detail in [22]. In addition, the use of a primal method allows us to deal with deep and multiple cuts, which does not seem to be the case for dual or primal-dual methods. This is particularly important when the disaggregated formulation is used. Just as in the method of Dantzig-Wolfe the multipliers K lead to a feasible solution for the linear relaxation of the original problem P . 17
4 Primal and Dual Heuristics 4.1 Dual Heuristics Trigeiro et al. [41] designed a heuristic production smoothing procedure to generate feasible schedules. The purpose of this heuristic is to eliminate overtime created in some periods because the aggregation of the solutions from the subproblems does not fit exactly within a period’s available capacity. This heuristic is simple and myopic. It moves some batches or part of batches from a period where there is some overtime to an earlier period (if it is possible). It operates in 4 passes (backward, then forward, . . . ), and then stops even if it has not found a feasible schedule. The information from the Lagrangian solutions found earlier is not used to guide the search of the heuristic –because the subgradient optimization does not provide a dual vector of weights of the previous solutions. They solved problems with up to 24 items over 30 periods.
4.2 Primal Heuristics In our heuristic we utilize the weights of the subgradients as given by the projective algorithm to run a single-commodity network formulation based heuristic to find feasible solutions of the CLSP. Basic Construction: assume that we know the it? ’s in an optimal solution of the CLSP. Then, the CLSP (P ) from Section 1 turns out into the following problem:
min
8 >< Cit e where Cit = >: 1
N X T X (Ceit Xit + Hit Iit )
i=1 t=1 Iit = Ii;t?1 + Xit ? Dit N N X X piXit Kt ? siit? i=1 i=1 Ii;0 = Ii;T = 0
Xit; Iit 0
if it? if it?
=1 =0
.
18
8i 2 N; t 2 T 8i 2 N; t 2 T 8i 2 N 8i 2 N; t 2 T;
(28)
The following scaling of the inventory and production variables transforms this problem into a multi-commodity network flow:
Xit = piXit Iit = piIit : In practice, ? is unknown. We approximate ? by a rounding ~ of a combination of the fractional solution given in (3.2) and the new given by the last proposal returned by the oracle:
8 >< ~it := 1 >: 0
if (1 ? )(Bu)it + itnew
(29)
otherwise,
where B contains the previous ’s. A choice of = 0:5 and = :1 seems Sit in order to keep the original appropriate. We use an extra cost proportional to D it hierarchy of the costs and to choose as new periods of production the periods which minimize the fraction of setup cost per product. Note that substituting ~ for ? in problem (28) may render this problem infeasible. In order to push the problem towards feasibility we go through multiple passes of the heuristic. It should be clear that this procedure may fail as finding a feasible solution is NP-hard, and the basic step is polynomial.
Instead of setting the production cost Ceit to 1 when ~it Sit . Thus, proportional to D it
8 > eCit = < Cit >: Cit + Sit Dit
= 0, we put an extra cost,
if ~it
=1 if ~it = 0
(30)
where is a large number. This allows production of item i at time t even though ~it = 0. In addition, the network may be view as a single-commodity network and solved efficiently by appropriate algorithm. Basic step: 1. compute the C~ associated with the ~ according to (30),
2. solve (28) with ~ and the associated cost C~ , 19
3. the LP (28) has no feasible solution; STOP, the heuristic failed,
= 0 in the optimal solution set ~it = 0, if the optimal X is compatible with ~ (i.e. Xit > 0 implies ~it = 1), the heuristic outputs the feasible X with the set-up variables ~ if Xit > 0 for some ~it = 0, set ~it = 1 and return to 1.
4. for any Xit 5. 6.
The risk of failure increases if the set-up times represent a significant fraction of the capacity, and if the capacity is too tight for the given demand.
5 Computational Experiments We compare three solution methods: the subgradient method, Kelley’s cutting plane method (KCPM) and the analytic center cutting plane method (ACCPM), with two variants (aggregated and disaggregated) in the latter two cases. These methods are tested on a set of 751 problems provided by Trigeiro, which were randomly generated with sizes of 15 periods and 4 products up to 30 periods and 24 products with a wide range of difficulty (tightness of the capacity, high and low setup times and costs). For the subgradient method, we use Trigeiro’s code, without change in any of the settings. We use a home made code for KCPM in which the optimal solution of the linear programming P K is computed using the simplex solver of CPLEX 3.0. Finally, the ACCPM code is the one that is available on the WEB for academic research (Gondzio et al. [23]). The three implementations use the same heuristics (because of the lack of dual variables the subgradient method uses only the dual one): the dual heuristic provided by Trigeiro and the primal heuristic discribed in the previous section (the single-commodity network flow problem is solved using the subroutine netopt provided by CPLEX 3.0). We use two measures of accuracy:
The CLSP-gap which is the difference between the value of K = maxk2K L(uk ) and the best feasible solution to CLSP provided by the heuristics 20
The relaxation-gap which is the difference between an upper bound for P K and the value of K . (Again, the subgradient method does not provide this information.)
In the cutting plane methods, we compute L with a relative relaxation-gap less than 10?6 . The main computational effort is in the heuristics; less than 5% of the time is spent in the master program. It is thus fair to compare the methods on the number of iterations, rather than on CPU time (this is traditional in nondifferentiable optimization). Let us first focus on the CLSP-gap. Picture 1 summarizes the results in term of accuracy obtained with the subgradient and the cutting plane methods (ACCPM and KCPM); only results obtained with ACCPM in the disaggregated case are used, as similar results are obtained for the aggregated version or for KCPM. This picture shows the number of problems in the set that have been solved for any given level of CLSP-gap. We can see that the two methods appear to give similar results: more than 80% of problems have a CLSP-gap smaller than 5%. Nevertheless, let us remark that the number of iterations is much smaller with the disaggregated version of ACCPM or KCPM: the sum of the iteration number over all problems is 10202 with disaggregated ACCPM compared to 102728 with the subgradient method.
21
100 # of problems (%) 90
100 # of problems (%)
Cutting plane Subgradient
80
Cutting plane Subgradient
80
70 60
60
50 40
40
30 20
20
10 0
Rel. duality gap (%) 0
5
10
15
0
20
Figure 1: CLSP gap with all problems
Rel. duality gap (%) 0
5
10
15
Figure 2: CLSP gap with hard problems
Because the test problems are very small and often easy to solve, we selected a subset of the hardest problems (20 periods, 30 products) and plot the information about the CLSP-gap on picture 2. This picture shows that the good results for the subgradient method on average over all the problems hide difficulties that occur for harder problems (still of small size). Usually, when the lower bound computed by the subgradient method is far from the optimum of the Lagrangian relaxation, the solution found by the heuristic also deteriorates; thus losing on both sides, without any detection of this weakness. For the cutting plane methods (ACCPM and KCPM) neither heuristic seems to dominate the other. The second set of results compares an aggregated (see picture 3) and a disaggregated (see picture 4) implementation of ACCPM and KCPM. The same relaxationgap is required in the two methods. We plot the number of iterations required in one method against the other. In the aggregated version, ACCPM outperforms KCPM for the larger problems. Moreover, the shape of the picture looks like a logarithm function giving an indication of the comparative behavior for larger problems (polynomial vs exponential?). Similar results are obtained with the disaggregated implementation: the number of iterations significantly decreases as compared to the aggregated version; the larger problems require no more than 57 iterations for KCPM and 43 for ACCPM while 380 and 145 where needed respectively with the aggregated version. We took one of the problems for which KCPM performed better than ACCPM and we duplicated the data to obtain a problem 22
20
with 2; 3; : : : ; 15 times the original number of periods. Running ACCPM and KCPM gives picture 5, which seems to indicate divergent behaviors of ACCPM and KCPM on larger problems; this is an area that deserves a lot more investigation. 400 ACCPM
60 ACCPM
ACCPM versus KCPM
ACCPM versus KCPM
350
+
"70"
300
"141-15"
"540-2"
4
"540-3"
?
"141-30" "540-1"
250 200
50
30
px = y x8
150
? 4 ??? 4? 4??? 4 4 4 4444 ?? ? 4 ? 44 ?? 4 ?4?????44?4 ?? ?? ?4 ?? ?44 4444?444?44 4 ?44 44 ? 4?4? ?? ?4??4 ?4 4 4??444444 44?4 4 100 ? 4 4 ? 4?+? 44+ 4? ? 4 4+?4+4 ??? ? ? ? ? ?4 ?4?+ +++++4+444 ?+? ++ ??? 4 + + ? + + 4 4 4 4 4 + 4 4 ?++ ++4 +?4??? ? 4 ++?4 ++++++4 4?++????+??+4 +4 4?+4 +++++4 4 ? ?4 ++4 +4 ? +4 4 +4 4 +++++ ++ ? 50 ?4 4 +++4 ?4 ??4 ? +++?+?+? ? ?? 44? ? ? 4 + +4 ++4 ? ? ??4 + ?+4 4 ? ?? 4 4 ?4 4 ?+ ? ?? ? +++ + 44 ?+4 ???+4 4 +4 ? ? ??+ ?+ ?? 4 ?4 4 +?+4 +4 ? 4 ??4 +4 ? 4 ? 4 ?4 4??4 ?+4 ?+ 4 4 +4 ? 4 ?
? ? ?? ? ? ? ?
50
100
150
200
250
300
20
10
KCPM
0
0
4 4 4 ?4 4 ? ? + + + 4 4 ? 4 ? + 44 ? ? 4 4 + ++ + + ++ ? + ? 4 4 4 4 4 +? 4 ? ?? 4 ? 4? 4 ++? 4 ? 4? 4 ?? 4 4 + +4 4 + ? +? 4 + 4 ? 4? 4 ?+ ? ? + + + + + 4 + ++ ++ 4 4? 4 4? 4 ? ? 4 ? 4 ? ? + + + 4 + + 4 ++ ++? +? ++? 4 +? ? 4 4 ?? ? 4 ? + 4 + 44 + +? ++ 4 ? +4 4 + 4 + +? ++? +? ?? 4 ++ 4 + 4 ++ 4 ? ? +? 4 4 44 4 ? +? 4 4 + 4 ++?? 4 ++?? 4 + 4 4 4 + ++?? 4 4 ++ 4 ++ 4 +? 4 ++?? 4 4 ? 4 4 +4 4 ? ? +? 4 +?? 4 ++? 4 4 4 ? 4 ?? ?? 4 4 4 ++? 4 44 4 4 4 ++?? 4 4 +?? 4 4 +??? 4 +?? 4 ?+ 4 4 44 4?? ? ? ?4 4 ? 4 ? + 4 44 ? 4 4? 4 4
40
350
0
400
Figure 3: Using an aggregated oracle
0
10
20
30
40
"70" "141-15" "141-30" "540-1" "540-2" "540-3"
x=y
+ 4 ?
KCPM
50
Figure 4: Using a disaggregated oracle
800
ACCPM
ACCPM versus KCPM
700
CLSP problems x=y 3 ln (x + 1)
600 500 400 300 200 100 0
0
KCPM
200
60
400
600
800
1000
Figure 5: harder problems solved using disaggregation oracle We also analyze in more detail a few classical examples. In table 1, 2 and 3 we give a comparaison between disaggregated ACCPM, a bundle method and a subgradient method. The results for the bundle method and the subgradient method are from Br¨annlund’s PhD. thesis [7]. By construction, ACCPM returns the optimal lower bound attainable by the relaxation. Compared with the others methods, one can notice that the number of iterations needed is (generally) lower than the number of iterations used in both the subgradient and the bundle methods 23
(50). ACCPM method
Bundle method [7]
Lower bound
Best primal
Lower bound (50 eval.)
Tight
(36) 27907
30380
Medium-tight
(36) 24364
26690
Medium-loose
(30) 20293
20920
Loose
(25) 18872
19210
Capacity
Subgradient [7]
Best primal
Lower bound (50 eval.)
Best primal
Optimal primal
27721
30090
27491
30490
29740
24364
27060
24026
27430
26030
20293
20920
20236
20920
20920
(28) 18872
19800
18847
19800
19210
Table 1: No setup times, very high setup costs
Capacity
ACCPM method
Bundle method [7]
Subgradient [7]
Lower bound
Best primal
Lower bound (50 eval.)
Best primal
Lower bound (50 eval.)
Best primal
Optimal primal
Tight
(36) 7997
8480
7993
8530
7907
8710
8430
Medium-tight
(30) 7722
7970
7722
7970
7692
7920
7910
Medium-loose
(17) 7534
7660
(36) 7534
7610
7533
7660
7610
(9) 7464
7520
(23) 7464
7520
7464
7520
7520
Loose
Table 2: No setup times, high setup costs
Capacity
Projective method
Bundle method [7]
Subgradient [7]
Lower bound
Best primal
Lower bound (50 eval.)
Best primal
Lower bound (50 eval.)
Best primal
Optimal primal
Tight
(19) 2893
2920
2893
2900
2893
2900
2900
Medium-tight
(21) 2893
2930
2893
2930
2892
2930
2900
Medium-loose
(1) 2865
2865
2865
2865
2865
2865
2865
—
—
—
—
—
—
—
Loose
Table 3: No setup times, low setup costs On these problems, disaggregated ACCPM outperforms in terms of accuracy and speed (number of iterations) the above methods for the computation of the optimal Lagrangian multipliers and thus of the lower bound. General statements about the quality of the primal integer solution are harder to make, as should be expected from heuristics.
24
6 Conclusions We conclude that the analytic center cutting plane method performs as well, and sometimes better than subgradient optimization or Kelley’s cutting plane method, as regards accuracy. Somewhat contrary to expectations the analytic center cutting plane method also outperforms subgradient optimization in terms of computational time, in the test problems presented here. It is, thus, competitive with the best methods available today. The integration of all of this as a part of a branch and bound scheme richly deserves further investigation.
7 Acknowledgments The authors wish to thank Dr Trigeiro for making his code as well as his test problems available.
References [1] D. S. Atkinson and P. M. Vaidya, “A Cutting Plane Algorithm that Uses Analytic Centers”, “Nondifferentiable and Large Scale Optimization”, Mathematical Programming, Series B, 69 (1995) 1–43. [2] O. Bahn, J.-L. Goffin, J.-P. Vial and O. du Merle, “Experimental Behavior of an Interior Point Cutting Plane Algorithm for Convex Programming: An Application to Geometric Programming”, Discrete Applied Mathematics 49 (1994) 3–23. [3] O. Bahn, O. du Merle, J.-L. Goffin and J.-P. Vial, “A Cutting Plane Method from Analytic Centers for Stochastic Programming”, “Nondifferentiable and Large Scale Optimization”, Mathematical Programming, Series B, 69 (1995) 45–73. [4] I. Barany, T.J. VanRoy and L. Wolsey, “Strong Formulations Multi-Item Capacitated Lotsing”, Management Science 30 (1984) 1255–1261. 25
[5] J. F. Benders, “Partitioning Procedures for Solving Mixed-variables Programming Problems”, Numerische Mathematik 4 (1962) 238–252. [6] R. Bhatnagar, P. Chandra and S.K. Goyal, “Models for Multi-Plant Coordination”, EJOR 67 (1993) 141–160. [7] U.G. Br¨annlund, “Experiments with the Bundle Method on two Production Planning Problems”, PhD. Thesis (1991), Dept. of Mathematics, Kungliga Tekniska Hogskolan, Sweden. [8] W.H. Chen and J.M. Thizy, “Analysis of Relaxations for the Multi-Item Capacitated Lotsizing Problem”, Annals of Operations Research 26 (1990) 29– 72. [9] G. B. Dantzig and P. Wolfe, “The Decomposition Algorithm for Linear Programming”, Econometrica 29 (1961) 767–778. [10] E.V. Denardo, Dynamic Programming, Models and Applications, PrenticeHall (1982) [11] M. Diaby, H.C. Bahl, M.H. Karwan and S. Zionts, “A Lagrangean Relaxation Approach for Very-Large Scale Capacitated Lot-Sizing”, Management Science 38 (1992) 1329–1340. [12] P.S. Dixon and E.A. Silver, “A Heuristic Solution Procedure for the Multiitem, Single-level, Limited Capacity, Lotsizing Problem”, J. of Operations Management 2 (1981) 23–39. [13] O. du Merle, Points Int´erieurs et Plans Coupants: Mise en œuvre et D´eveloppement d’une M´ethode Pour l’Optimisation Convexe et la Programmation Lin´eaire Structur´ee de Grande Taille, PhD Thesis, HEC-Section of Management Studies, University of Geneva, Switzerland, March 1995. [14] O. du Merle, J.-L. Goffin and J.-P. Vial, “On the Comparative Behavior of Kelley’s Cutting Plane Method and the Analytic Center Cutting Plane Method”, Technical Report 1996.4, Department of Management Studies, University of Geneva, January 1996. [15] B.P. Dzielinski and R.E. Gomory, “Optimal Programming of Lotsizes, Inventories and Labor Allocations”, Management Science 11 (1965) 874–890.
26
[16] M.L. Fisher, “The Lagrangian Relaxation Method for Solving Integer Programming Problems,” Management Science 27 (1981) 1–18. [17] M. Garey and D. Johnson, Computers and Intractibility: a Guide to the Theory of NP-Completeness, Freeman and Co., San Fransisco (1979). [18] A. M. Geoffrion, “Lagrangean relaxation for integer programming, Mathematical Programming Study, 2 (1974) 82-114. [19] J.-L. Goffin, A. Haurie and J.-P. Vial, “Decomposition and nondifferentiable optimization with the projective algorithm”, Management Science 38 (1992) 284–302. [20] J.-L. Goffin, Z. Luo and Y. Ye, “Complexity Analysis of an Interior Cutting Plane Method for Convex Feasibility Problems”, SIAM J. Optimization 3 (1996) 638–652. [21] J.-L. Goffin, J. Gondzio, R. Sarkissian et J.-P. Vial, “Solving Nonlinear Multicommodity Flow Problems by the Analytic Center Cutting Plane Method”, Mathematical Programming 76 (1997) 131–154. [22] J.-L. Goffin and J.-P. Vial, ”Shallow, Deep and Very Deep Cuts in with the Analytic Center Cutting Plane Method”, GERAD Report G–96–20, May 1996, 18 pp. [23] J. Gondzio, O. du Merle, R. Sarkissian and J.-P. Vial, “ACCPM - A Library for Convex Optimization Based on an Analytic Center Cutting Plane Method, Technical”, Report 1995.34, LOGILAB/Management Studies, University of Geneva, Switzerland (1995). [24] C. Gonzaga, “Path-following Method for Linear Programming”, SIAM review, 34 (1992) 167-224. [25] L.A. Johnson and D.C. Montgomery, Operations Research in Production Planning, Scheduling and Inventory Control, Wiley, New-York, 1974. [26] J. E. Kelley, “The Cutting-plane Method for Solving Convex Programs”, Journal of the SIAM 8 (1960) 703–712. [27] L.S. Lasdon and R.C. Terjung, “An Efficient Algorithm for Multi-Item Scheduling”, Operations research 19 (1971) 946–969. 27
[28] A. Lisser, R. Sarkissian et J. P. Vial, “Survivability in Telecommunication Networks”, Universit´e de Gen`eve, Suisse (1994). [29] N. Karmarkar, “A new polynomial time algorithm for linear programming”, Combinatorica 4 (1984) 373–395. [30] J. Maes and L. VanWassenhove, “Multi-Item Single-Level Capacitated Dynamic Lotsizing Heuristics: a Computational Comparison (part I: static case)”, IEE Trans. 18 (1986) 114–123. [31] A.S. Manne, “Programming of Economic Lotsizes”, Management Science 4 (1958) 115–135. [32] H.H. Millar and M. Yang, “An Application of Lagrangean Decomposition to the Capacitated Multi-Item Lotsizing Problem,” Computers and Operations Research 20 (1993) 409–420. [33] J.M. Mitchell, “Computational experience with an interior point cutting plane algorithm”, Manuscript (1997). [34] J. E. Mitchell and M. J. Todd. ”Solving combinatorial optimization problems using Karmarkar’s algorithm” Mathematical Programming, 56, pp 245-284, 1992. [35] A.S. Nemirovsky and D.B. Yudin, “Problem Complexity and Method Efficiency in Optimization”, John Wiley, Chichester (1983). [36] Y. Nesterov, “Complexity estimates of some cutting plane methods based on the analytic barrier”, “Nondifferentiable and Large Scale Optimization”, Mathematical Programming, Series B, 69 (1995) 149–176. [37] M. Salomon, “Deterministic Lotsizing Models for Production Planning,” Lectures Notes in Economics and Mathematical Systems, 355 (1991), Springer-Verlag. [38] J.-M. Thizy and L.N Van Wassenhove, “Lagrangean Relexation for the Multi-item Capacitated Lot-Sizing Problem”, IIE Transactions 17 (1985) 308–313. [39] J.-M. Thizy, “Analysis of Lagrangean Decomposition for the Multi-Item Capacitated Lot-Sizing Problem”, INFOR 29 (1991) 271–283. 28
[40] L.J. Thomas and E.N. Weiss, “An Integrative Approach for Production Scheduling and Capacity Determination”, Production Management: Methods and Applications, B. Lev (Ed.), North-Holland (TIMS Series), Amsterdam, (1986). [41] W. Trigeiro, L.J. Thomas and J.O. McLain, “Capacitated Lotsizing with Setup Times”, Management Science 35 (1989) 353–366. [42] C. Trouiller, “Capacitated Multi-Item Lot Sizing with an Interior Point Cutting Plane Algorithm”, MSc Thesis in Computer Science, McGill University, November 1994, revised December 1995, 46 pages. [43] H. Wagner and T.M. Whitin, “Dynamic Version of the Economic Lotsize Model”, Management Science 5 (1958) 89–96.
29
