An active-set strategy in an interior point method for linear programming

Mathematical Programming 59 (1993) 345-360 North-Holland

345

An active-set strategy in an interior point method for linear programming Kao ru Tone Graduate School of Policy Science, Saitama University, Urawa, Japan Received 19 November 1990 Revised manuscript received 14 November 1991

We will present a potential reduction method for linear programming where only the constraints with relatively small dual s l a c k s - - t e r m e d "active c o n s t r a i n t s " - - w i l l be taken into account to form the ellipsoid constraint at each iteration of the process. The algorithm converges to the optimal feasible solution in O(,/~L) iterations with the same polynomial b o u n d as in the full constraints case, where n is the n u m b e r of variables and L is the data length. If a small portion of the constraints is active near the optimal solution, the computational cost to find the next direction of movement in one iteration may be considerably reduced by the proposed strategy.

Key words: Linear programming, interior point method, active set strategy.

1. Introduction

Since the epoch-making breakthrough by Karmarkar [11], interior point methods for linear programming have been studied in many aspects. Karmarkar introduced the potential function for linear programming for the first time. Since then many researchers have studied and extended it with significant theoretical successes. (See Gonzaga [8], Ye [15], Freund [5], Todd and Ye [14], Kojima, Mizuno and Yoshise [12] among others.) Ye [15] has developed an O(x/nL) iteration potential reduction algorithm based on the primal-dual potential function. (Gonzaga [9] and Freund [5] have presented similar results.) The purpose of this paper is to introduce an active-set strategy in Ye's dual potential reduction method. Usually, a linear program in inequality constrained form has many redundant constraints in the optimal solution. If we deal with all constraints, we are forced to solve simultaneous linear equations related to the entire coefficient matrix of the problem. The proposed active-set strategy may allow us to Correspondence to: Prof. K. Tone, Graduate School of Policy Science, Saitama University, Urawa, Saitama 338, Japan. This research was partially done in June 1990 while the author was visiting the Department of Mathematics, University of Pisa.

346

K. Tone / Active-set LP algorithm

reduce the cost, still keeping the same order of polynomial convergence property. In Section 2, we will define the problem and the potential functions along with some preliminary lemmas. We are mainly concerned with the dual form of linear programming. In Section 3, an active set strategy for finding the direction of movement in the scaled dual space will be introduced. Also, we will discuss some properties related to the expansion of the active set. In Section 4, we will introduce primal variables. In Section 5, an algorithm based on the active set strategy will be presented. In Section 6, we will analyze the convergence property of the algorithm and show that it terminates in O(,/-ffL) iterations, where n is the number of variables and L is the data length. Then, in Section 7, we will observe the case when the active set constitutes a basis and show that the interior point method can be managed by the basis factorization techniques of the simplex method coupled with a sequence of rank-one changes to matrices. Historically, several research strategies have been used for incorporating the interior point methods with a suitable partition of variables--basic and nonbasic or promising and non-promising--using the simplex technologies. Shanno and Marsten [13] tried to use a basis for the null space of the constraint matrix to solve the least-squares problem for the search direction. Goldfarb and Mehrotra [6, 7] developed a relaxed version of Karmarkar's method that uses inexact projections and which has the same polynomial-time complexity as the original method. They showed that a direction for the relaxed algorithm can be obtained by inexactly solving a least-squares problem involving a basis for the null space of the constraint matrix. Our method has the same features with theirs in the decomposition of variables and in the approximate search direction. However, they deal with the full constraint matrix, while ours deals principally with the promising columns of the matrix and introduces the non-promising columns adaptively to obtain a fixed amount of reduction in the potential function. Zikan and Cottle [16] proposed the box method to select promising columns to keep in. This approach uses a subset of the columns based on an interior iterate and replaces the ellipsoid by a box. The method's novelty lies primarily in the simple scheme it uses to produce the search directions. Like the simplex method, it is finite in the nondegenerate case. Recently, Dantzig and Ye [3] proposed a build-up interior point method that has very close relations with the present one in that they partition the columns based on the closeness to the current iterate, form an inscribed ellipsoid using only the "close" columns, calculate the descent direction based on the ellipsoid and add the closest blocking constraint to the close columns if any. They applied this philosophy to Dikin's dual affine algorithm [4] and demonstrated that the algorithm either terminates in a finite number of iterations with the optimal primal and dual solutions or converges to them in the limit. The differences exist in the parent algorithms (Dikin's for Dantzig and Ye and Ye's dual potential reduction one for ours), the way to add columns to the set of "close" columns and hence in the complexity bounds of the algorithms.

K. Tone / Active-set LP algorithm 2. Problem

and potential

347

functions

We are concerned with the linear program whose primal form is (P)

minimize

~ = cTx

subject to

x~{x~Rn:Ax=b,x~>O},

(2.1)

where e c ~ ", A c R mxn, and b c ~ m are given. Its dual form is (D)

maximize

z = yTb

subject to

yc{yc~m:c-AVy>~O}.

(2.2)

We denote the dual slack variables by

s = c-ATy>~O.

(2.3)

As far as notations are concerned, the superscript v denotes the transpose, e the vector of all ones and Aj the j t h column vector of A. The upper case letter (X) designates the diagonal matrix of the vector (x) in lower case. The optimum value of ~ (and hence z) is denoted by z*. For (P) and (D), we assume that (1) the relative interiors of the feasible regions of (P) and (D) are nonempty and we have an interior feasible x ° and yO for (P) and (D) such that

A x °=b,

x°>0,

(2.4)

and

s o = c - A V y ° > 0;

(2.5)

(2) A has full row rank and has no zero columns; and (3) the objective functions of (P) and (D) are not constant on the feasible regions. Associated with the programs, we consider the following potential functions: - the dual potentialfunction for an interior feasible (y, s), n

4~D(Y, ~) = P ln(~-- b-ry) - 2 ln(sj); j=l

(2.6)

and -

the primal-dual potentialfunetion for an interor feasible pair (x, y, s), n

0pD(X, S) = p ln(xVs) -- ~ ln(xjsj);

(2.7)

j=l

where ~ is an upper bound to z* and p is a positive parameter. We have the relation 0pD(X, S) = 4~D(Y, ~) -- ~ ln(xj), j--1

for ~ = cTx.

(2.8)

348

K. Tone / Active-set LP algorithm

The gradient vector of qSi~ (with respect to y) is ~7~bD(y, 2 ) --

(2.9)

b + A S - ~ e.

P

- bTy

The following lemma is due to Ye [15]. Lemma 1 (Ye [15]). F o r a n y two points s o= c - - A Y y ° > O, s 1= c - - A T y I > O l[ (S°)-l(s

1 - s°)ll

and

(2.10)

< 1,

we have

&D(y 1, ~) - qSD(y°, e) ~0,

Ary°+s°=c,

s°>0,

with the primal objective value 2o= cWx o and the dual objective value z ° = bTy °. We partition the index set J = {1, 2 , . . . , n} into/3 a n d / 3 in the following way: o (1) Let us reorder the index set J in the ascending order of sj , s° 0. The conclusion is: Lemma 3. I f y < y¢, then ( y ( y ) , s ( y ) ) of (3.6) and (3.9) is an interior feasible dual

solution.

[]

Here, we will try to expand the active set /3 and show that y¢ of (3.11) is non-decreasing with respect to/3. Let the sets /3 and fi be defined by (3.2) and (3.3). We e x p a n d / 3 by adding an element h ~ fi and define /3 + 1 =/3 u {h}

(3.18)

/3 + 1 =/3 -{h}.

(3.19)

and

For/3 + 1 a n d / 3 + 1, we define A/3+1, A ~ , same way as for/3 and ft.

S~+ 1, S g0; i , _h~+l, X ~

and Y¢+1 in the

Lemma 4.

(3.20) Proof. Since _he = min{dVA~(S°~)-2AV~d: Ildll = 1} and similarly for _h]3+l, it is easy

to see that _h~+l~>_A¢. A similar formula for A~ shows X~+~ O. Let

[p~(~o)] (4.11)

P = [ pg(~o) j , with pt~(ff °) =

P

o

1

ffO_bry o ( S g ) x ~ + e g = O

(by (4.7)).

(4.12)

The last relation shows that t h e / 3 part of p is always zero and so the corresponding c o m p o n e n t s of s o and x 1 can be viewed as centered (see, e.g., [12]). Lemma 6. Let p = n + u x / - n (with u > l )

and 0 < a < l . is an interior primal feasible solution and we have

I f Ilplto O D ( y ( y ) , ~o).

(5.3)

Let 6 * = &D(y °, fro)_ CbD(y(y*), ~o).

(5.4)

If 6* < 3, then expand/3 by adding an element of/3 with the smallest sjo and redefine/3 and ft.

354

K. Tone / Active-set LP algorithm

Go back to the beginning of inner iteration. If 8"/> 6, then let y l : y(z,*), X1 = X 0,

s1 : s(~,*),

~1 = if0.

(5.5) (5.6)

end dual step. else if []p~(~0)]1 < % then begin primal step: Compute xfi1

of (4.7).

Let

X 1-'~FX1[3]

(5.7)

~ = cTx 1,

(5.8)

Lx~-J'

yl=yO,

s l = s o.

(5.9)

end primal step. end inner iteration. end iteration. Now replace (x °, yO, s o) with (x 1, yl, S 1) and repeat until (x0)Ts0 < 2 L is satisfied. Remark 5.1. We need not compute the scalar r/° in (5.1) explicitly but can include the term into % since our objective is to minimize the dual potential function on the line segment. The expression of y ( y ) in this form is applied in concordance with (3.6). Remark 5.2. In the dual step, we can expand the set fi not one by one but by a certain number of elements in/3 at a time.

6. Convergence Lemma 8. The inner iteration in A l g o r i t h m A ends after at most n - m additions to fl a n d comes to satisfy 6* >~ (%

Proof. In the dual step, we have Ilpll = IIp~(~°)ll/> ~. If we set, in (5.1), Y < Yp = 1/~/l+,(J_hi3 ,

(6.1)

K. Tone / Active-setLP algorithm

355

then, by L e m m a 3, (y(y), s(y)) satisfies (3.16) and (3.17) and is dual feasible and hence by L e m m a 1, we have

,~D(y(3'), ? ) - ~(yO, ~o) l 1 and (x k, s k) ( k = 0 , 1, 2 , . . . ) be a series of interior primal- dual feasible pairs with CpD(X °, S°) = O (x/nL). I f for a positive 3 independent of n, the relation

t~pD(Xk+l, Sk+l) < (~pD(Xk, S k)

-- 3

holds for each k, then in O(~,.,/-nL) iterations, we have c T x k -- bTy k = ( x k ) T s k