ACCPM | A Library for Convex Optimization Based on an ... - CiteSeerX

ACCPM | A Library for Convex Optimization Based on an Analytic Center Cutting Plane Methody J. Gondzioz, O. du Merle, R. Sarkissian and J.-P. Vial Logilab, HEC, Section of Management Studies, University of Geneva, 102 Bd Carl Vogt, CH-1211 Geneve 4, Switzerland Technical Report 1995.34 December 12, 1995

Keywords: Large scale convex optimization, Cutting plane method, Interior point

algorithm, Analytic center.

Hardware information: Any computer with C++ and FORTRAN 77 compilers. Software information: C++ and FORTRAN 77.

1 Introduction We are concerned in this note with the Gon Haurie and Vial's [7] Analytic Center Cutting Plane Method (ACCPM for short) for large-scale convex optimization. Its state-of-the-art implementation [10] is now available upon request for academic research use. Cutting plane methods for convex optimization have a long history that goes back at least to a fundamental paper of Kelley [14]. There exist numerous strategies that can be applied to \solve" subsequent relaxed master problems in the cutting planes optimization scheme. In the Analytic Center Cutting Plane Method, subsequent relaxed master problems are not solved to optimality. Instead of it, an approximate analytic center of the current localization set is looked for. The theoretical development of ACCPM started from Gon and Vial [9]. It was later continued in [7, 8] and led to a development of the prototype implementation of the method due to du Merle [15] that was successfully applied to solve several nontrivial convex optimization problems [2, 3]. This research has been supported by the Fonds National de la Recherche Scienti que Suisse, grant #12 ? 34002:92. y to appear in European Journal of Operational Research. z on leave from the Systems Research Institute, Polish Academy of Sciences, Newelska 6, 01-447 Warsaw, Poland. 

1

A need to solve very large optimization problems [6] motivated us to rewrite the program from scratch so as to make it more exible and easier to use for the general convex optimization problems. The new implementation of the method due to Gondzio, du Merle, Sarkissian and Vial [10] responds these needs. All recent theoretical and practical developments [16, 12] have been incorporated into it, resulting in the creation of a sophisticated optimization tool. We now make it available to the user, in the form of a callable library libaccpm.a.

2 Fundamentals of ACCPM The algorithm implemented in ACCPM combines two powerful optimization techniques: cutting plane methods [14] and interior point algorithms [13, 17]. Its detailed description is de nitely beyond the scope of this short note. The reader interested in the theory and the implementation details should consult [10] and the references therein. We shall nevertheless recall very brie y some basic concepts used within ACCPM. Although their understanding would surely be useful, it is not a pre-condition for an application of a black box type ACCPM library. We are concerned with the solution of a convex optimization problem minimize f (y ) (1) subject to y 2 Y: where f : Rm 7! R is a convex function, Y 2 Rm is a convex set such that Y  fy : f (y ) < 1g.

The Oracle

Since the problem is convex, both the epigraph of f and the set Y can be approximated by intersections of hyperplanes. The procedure called oracle rst tests whether y 2 Y . If y 2 Y , then the oracle generates a subgradient  2 @f (y ) at y with the property

f (y 0 )  f (y) + h; y 0 ? yi; 8y 0 2 Y:

(2)

This inequality is a supporting hyperplane for the function to be optimized; we shall call it an optimality cut. If y 62 Y , then the oracle generates a hyperplane (;  ) 2 Rm  R for Y that separates y from Y , i.e.,

h; yi <  and h; y0i  ; 8y0 2 Y: (3) This hyperplane improves the description of the feasible set Y ; we shall call it a feasibility cut. Suppose the oracle has been called at a given sequence of points fy lg; l 2 . Some of these points were feasible: y l 2 Y , l 2 opt; the others were not: y l 62 Y , l 2 fsb =  n opt. The oracle generated a set of optimality cuts at y l , l 2 opt that de ne a piecewise linear approximation f : Rm 7! R to the convex function f f (y) = lmax ff (yl) + hl; y ? ylig: (4) 2opt It also generated a set of feasibility cuts at y l , l 2 fsb that de ne an outer approximation Y of the feasible set Y (5) Y = fy : h l; yi   l; 8l 2 fsb g: 2

The relaxed master problem The program

minimize f (y ) (6) subject to y 2 Y : is an outer relaxation of the master problem (1). Note that Y is a polyhedron and f is a polyhedral (piecewise linear) function. The above program can thus be reformulated as a linear problem, the relaxed master program minimize  (7) subject to   f (y l ) + hl ; y ? y l i; 8l 2 opt; l l h ; yi   ; 8l 2 fsb: Its solution gives a lower bound for the master problem (1). Observe also that the best feasible solution in the generated sequence provides an upper bound for the master problem

ff (yl)g: u = l2min  opt

(8)

Localization set

For a given upper bound , let us consider the following polyhedral approximation called a localization set L() = f(; y) :   ; (9)   f (y l) + hl ; y ? y li; 8l 2 opt ; l l h ; yi   ; 8l 2 fsbg: It is the best (outer) approximation of the optimal set in (1). We are now ready to give a prototype cutting plane method [14]. Its main steps are the following: 1. Compute a point (; y) 2 L(u ) and an associated lower bound  . 2. Call the oracle at (; y). The oracle returns one or several cuts; if all of them are optimality cuts the oracle also generates an upper bound f (y ). 3. Update the bounds: (a) if y is feasible, u := minff (y ); ug. (b) l := maxf; l g: 4. Update the upper bound  in the de nition of the localization set (9) and add the new cuts. These steps are repeated until a feasible point to (1) is found such that u ? l falls below a prescribed optimality tolerance.

Multiple cuts

Let us observe that we have been concerned by now with a fairly general convex optimization problem (1). In many applications, one can bene t from two special structures: the objective is additive Xk f (y) = fi (y); (10) i=1

3

and the feasible set is the intersection of dierent subsets p \ Y= Y: j =1

(11)

j

In such a case, it is possible to modify the relaxed master program so that multiple cuts are introduced one at time:

Pk

i minimize i=1 subject to i  f (y l ) + hli ; y ? y l i; 8l 2 opt; i = 1; : : :; k; hjl ; yi  jl ; 8l 2 fsb; j = 1; : : :; p:

(12)

Analytic center

There exists a number of possible strategies that can be applied at step 1 of the cutting plane method. The optimal point strategy [4], for example, consists in solving every relaxed master program (7) to optimality. The central point strategy [9] consists in nding some \central point" in a localization set. The discussion of their advantages is beyond the scope of this note. Whatever strategy is used, however, we are concerned with a linear optimization problem (7) or, more precisely, with nding some point in the localization set. Localization set (9) is a polytope in Rm+1 . It is de ned by a set of n inequalities that can be written as

L = fy : AT y  cg;

(13)

where a vector (y;  ) 2 Rm+1 is, for brevity, denoted with y and A 2 R(m+1)n . We can represent the localization set in an equivalent form that use explicit dual slack variables s 2 Rn

L = fy : AT y + s = c; s  0g:

(14)

We associate a (dual) potential function to the set of linear inequalities (13) D =

Xn ln s :

j =1

(15)

j

The analytic center is a (unique) maximizer of the dual potential over the interior of L. It is easy to show that it is an interior point (x > 0; s > 0; y ) that satis es

Ax = 0; = c; XSe = e;

AT y + s

(16)

where X and S are diagonal matrices with the elements xj and sj , respectively and e is the n-vector of all ones. In other words, an analytic center is the primal and dual feasible point such that all complementarity products xj sj ; j = 1; :::; n, are \centered" to one. Note that

e D = e

Pnj

=1

ln sj

4

=

Yn s :

j =1

j

Thus the analytic center is a point that maximizes the product of slacks in L. This gives it a natural geometrical interpretation: analytic center is a point that maximizes the product of distances to all faces of the polytope L. (Note, by the way, that repeating a given constraint in a de nition of L does not change the geometry of the polytope, but it does change its analytic center | the repeated constraint moves the analytic center away.) ACCPM employs the de Ghellinck and Vial's [5] potential reduction algorithm to nd an analytic center of the current localization set.

3 How to use ACCPM We are aware that ACCPM represents a fairly complicated optimization technique. Hence to facilitate its usage, we provide the user with four carefully designed examples of its applications. They start from a simple unconstrained quadratic optimization and, step by step, get more complicated up to a constrained quadratic programming problem with a separable objective and intersection type de nition of the feasible set. We hope to have demonstrated with these examples quite a sophisticated usage of the library in a still comprehensive way. Some of potential ACCPM users may have already implemented a variant of the cutting plane method (e.g., Dantzig-Wolfe approach, bundle method, Elzinga-Moore approach, etc.). For such users, a replacement of the implemented technique with a routine that nds an analytic center might turn out to be a half-an-hour exercise. The inverse is also true: any ACCPM implementation can be easily transformed into another cutting plane approach. Whenever a cutting plane algorithm is thoughtfully designed, such a modi cation reduces, in practice, to a replacement of the get AC routine from libaccpm.a with an appropriate get type routine. To build a new application, the user has to install the libaccpm.a library and follow the examples.

Installation of ACCPM

ACCPM library is written in C++. An exception are the Cholesky factorization routines written in FORTRAN 77: the sparse one due to Gondzio [11] and the dense one from LAPACK [1]. The library and examples of its application are placed in a directory accpm. This directory contains a read.me le, a PostScript version of the most recent revision of ACCPM's user's guide, useaccpm.ps and two subdirectories. The rst, called applications contains four quadratic programming examples QP1, QP2, QP3, and QP4. The second, called ac contains the ACCPM library libaccpm.a. The whole directory accpm has been archivized (with a Unix command tar cvf accpm.tar accpm) that resulted in creating a le accpm.tar. The library is distributed in such a form. There are three versions available of the accpm.tar le: accpm.tar.ibmxlC is a version compiled with the IBM's xlC compiler for IMB RISC 6000 computer and its successors, accpm.tar.ibmg++ is a version compiled with the gnu C++ compiler for IMB RISC 6000 computer, and accpm.tar.sung++ is a version compiled with the gnu C++ compiler for SUN SPARC workstation. To install ACCPM library, the authorized user is rst of all expected to get the appropriate 5

le accpm.tar for the computer and compiler available. (For example, on the IBM RISC 6000 computer with the xlC compiler, this would require executing the command cp accpm.tar.ibmxlC accpm.tar.) Once accpm.tar le has been placed it in the current directory, the user should execute the following command tar xvf accpm.tar accpm. This should create a subdirectory accpm in the current directory. To complete the installation, the user should enter this directory cd accpm

and run the make

command. If this operation is successfully completed, the applications subdirectories QP1, QP2, QP3, and QP4 will contain executable qp programs. It might happen that when linking the program, the loader will not be able to nd appropriate libraries liblapack.a and libblas.a. In such a case, the reader will have to install them either in a public place or in the current (application) directory. The public domain FORTRAN source les for both these libraries can be found in netlib (ftp://netlib.att.com). Note that at the beginning of makefile, several important parameters are de ned that the user may want to change. They de ne the names of compilers: F77 and CXX, the ags (options) for compilation: FFLAGS and CFLAGS and the names (and paths to) dierent libraries: LAPACK, BLAS, LM, LC, LF77. All these names are exported to makefiles in subdirectories.

Callable routines of the ACCPM library

ACCPM is a library of routines that handle a part of the logic of the cutting plane algorithm. Those routines update the localization set, nd an approximate analytic center to it and execute a number of more technical actions. Their callable subset is de ned in an ACCPM include le accpm.inc that should be included in all user's les which make a call to libaccpm.a routines. The include le has the following form /* structure types */ struct Problem; struct Work; /****** ACCPM Callable Library Routines ******/ extern void init problem(Problem Work int char & double& double double int int double double ); extern void phaseI phaseII(Problem double ); extern void init mytime(void); extern double mytime(void); extern void get AC(Problem Work char int double double& int int int double int int double double int double double extern void get info(Problem int& int& int& double& double& int&); extern void get x primal(Problem double int&); extern void change no of subpb(Problem int); extern void set compute binf(int Problem ); extern void set testcolumn(int Problem ); extern void set push bound(int Problem ); extern void set weight(int Problem );

;

;

;

;

;

;

;

; ; ; ;

 ; ; ;

;

; ;

 ; ; 

;

; ;

; ; ;   



6

; ;

;

;



; ;

 ;

;

;

;

;

; ; ; ;

);

i.e., it contains the prototypes of all callable routines.

Running ACCPM

Assume a new user's application has already been compiled and linked. The executable program needs the ACCPM speci cations le input.dat to be present in the current directory. This le contains a set of technical parameters that control the execution of ACCPM (e.g., determines the requirements imposed on the quality of approximate analytic center, chooses appropriate data structures to handle cuts, chooses the preferable factorization technique to be applied in the interior point algorithm, and, nally, provides ACCPM with useful hints to determine its initial storage requirements). Below we give an example of such a le chosen appropriately to the quadratic programming applications supplied with ACCPM. QTEST Verbose sparseG factor L explicit B dense GUB nonz in cut densityB densityL exp iters max iters

:

1 0 0 0 1 1 2 2 0 100 100

4 Handling large scale problems There are no particular limits of the size of problems to be solved, except a total amount of operational memory available on a given computer. The origin of the problem has clearly an essential in uence on these limits. A 64MB workstations, for example, can handle really large problems. If cuts are stored as dense columns, the rough limits are: m  500 and the total number of cuts n  10; 000. However, if cuts display sparse structure, these limits can be pushed much further. The largest problems solved in [12] had m = 5; 000 and the number of cuts went over n = 100; 000. By the appropriate setting of parameters in input.dat le, the user can help better storage management within ACCPM.

5 Further reading ACCPM is a sophisticated optimization tool. Although its new implementation bene ts from all experience gathered so far, we are aware that its application is not always straightforward. Potential user of ACCPM should undoubtedly consult at least its user's guide, useaccpm.ps. More advanced usage of the library will require careful reading of the survey [10].

References

[1] Anderson E., Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov and D. Sorensen, LAPACK Users' Guide, SIAM, Philadelphia,

7

[2] [3] [4] [5] [6]

[7] [8] [9] [10] [11] [12]

[13] [14] [15] [16] [17]

1992. Bahn O., J.-L. Gon, J.-P. Vial and O. du Merle, Experimental behaviour of an interior point cutting plane algorithm for convex programming: an application to geometric programming, Discrete Applied Mathematics 49 (1994) 3-23. Bahn O., O. du Merle, J.-L. Gon and J.-P. Vial, A cutting plane method from analytic centers for stochastic programming, Mathematical Programming 69 (1995) 45-73. Dantzig G.B. and P. Wolfe, The decomposition algorithm for linear programming, Econometrica 29, 4 (1961) 767-778. de Ghellinck G. and J.-P. Vial, A polynomial Newton method for linear programming, Algorithmica 1 (1986) 425{453. Gon J.-L., J. Gondzio, R. Sarkissian and J.-P. Vial, Solving nonlinear multicommodity ow problems by the analytic center cutting plane method, Technical Report 1994.21, Department of Management Studies, University of Geneva, September 1994, revised November 1995, Mathematical Programming (to appear). Gon J.-L., A. Haurie and J.-P. Vial, Decomposition and nondierentiable optimization with the projective algorithm, Management Science 38, 2 (1992) 284-302. Gon J.-L., A. Haurie, J.-P. Vial and D.L. Zhu, Using central prices in the decomposition of linear programs, European Journal of Operational Research 64 (1993) 393-409. Gon J.-L. and J.-P. Vial, Cutting planes and column generation techniques with the projective algorithm, Journal of Optimization Theory and Applications 65 (1990) 409-429. Gondzio J., O. du Merle, R. Sarkissian and J.-P. Vial, An advanced implementation of the analytic center cutting plane method and its applications, Technical Report, Department of Management Studies, University of Geneva, December 1995 (in preparation). Gondzio J., Implementing Cholesky factorization for interior point methods of linear programming, Optimization 27 (1993) pp. 121-140. Gondzio J., R. Sarkissian and J.-P. Vial, Using an interior point method for the master problem in a decomposition approach, Technical Report 1995.30, Department of Management Studies, University of Geneva, Switzerland, October 1995, revised in May 1996, to appear in European Journal of Operational Research. Gondzio J. and T. Terlaky, A computational view of interior point methods for linear programming, in: Advances in Linear and Integer Programming, Beasley J.E. (ed.), pp. 106-147, Oxford University Press, Oxford, England, 1996. Kelley J.E., The cutting plane method for solving convex programs, Journal of the SIAM 8 (1960) 703-712. Merle O. du, Interior points and cutting palnes: a development and implementation of methods for convex optimization and large scale structured linear programming, Ph.D Thesis, Department of Management Studies, University of Geneva, Geneva, Switzerland, 1995 (in French). Merle O. du, J.-L. Gon, and J.-P. Vial, On the Comparative Behavior of Kelley's Cutting Plane Method and the Analytic Center Cutting Plane Method, Technical Report 1996.4, Department of Management Studies, University of Geneva, Switzerland, March 1996. Roos C. and J.-P. Vial, Interior Point Methods, in: Advances in Linear and Integer Programming, Beasley, J.E. (ed.), Chapter 3, Oxford University Press, Oxford, England, 1996.

8