A long step barrier method for convex quadratic ... - CiteSeerX

A long step barrier method for convex quadratic programming Report 90-53

K.M. Anstreicher D. den Hertog C. Roos T. Terlaky

Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics and Informatics Technische Universiteit Delft Delft University of Technology

ISSN 0922-5641

Copyright c 1990 by the Faculty of Technical Mathematics and Informatics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, microfilm, or any other means without permission from the Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands. Copies of these reports may be obtained from the bureau of the Faculty of Technical Mathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone +3115784568. A selection of these reports is available in PostScript form at the Faculty’s anonymous ftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl

DELFT UNIVERSITY OF TECHNOLOGY

REPORT 90{53

A LONG STEP BARRIER METHOD FOR CONVEX QUADRATIC PROGRAMMING K. M. Anstreicher, D. den Hertog, C. Roos, T. Terlaky

ISSN 0922{5641 Reports of the Faculty of Technical Mathematics and Informatics 90{53 Delft 1990 i

K.M. Anstreicher:

Department of Operations Research, Yale University, 84 Trumbull St., New Haven, CT 06520, U.S.A. D. den Hertog, C. Roos and T. Terlaky, Faculty of Technical Mathematics and Informatics, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands.

This work is completed with the support of a research grant from SHELL. The fourth author is on leave from the Eotvos University, Budapest, and partially supported by OTKA No. 2116.

c 1990 by Faculty of Technical Mathematics and InforCopyright matics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, micro lm or any other means without written permission from Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands.

1

Abstract In this paper we propose a long{step logarithmic barrier function method for convex quadratic programming with linear equality constraints. After a reduction of the barrier parameter, a series of long steps along projected Newton directions are taken until the iterate is in the vicinity of the center associated with p the current value of the barrier parameter. We prove that the total number of iterations is O( nL) or O(nL), dependent on how the barrier parameter is updated.

Key Words: convex quadratic programming, interior point method, logarithmic barrier function, polynomial algorithm.

1 Introduction Karmarkar's [14] invention of the projective method for linear programming has given rise to active research in interior point algorithms. At this moment, the variants can roughly be categorized into four classes: projective, ane scaling, path{following and potential reduction methods. Researchers have also extended interior point methods to other problems, including convex quadratic programming (QP). Kapoor and Vaidya [12], [13] and Ye and Tse [31] extended Karmarkar's projective method for QP. They proved that their method requires O(nL) iterations, where L is the size of the problem. Dikin [5] already proved convergence for the ane scaling method applied to QP. Ye [29] further analysed this method. Monteiro and Adler [21] developed a primal{dual small{step path{following method for QP. Ye [28], Ben Daya and Shetty [1], and Goldfarb and p Liu [8] proposed small{step logarithmic barrier methods for QP. All these methods require O( nL) iterations. Renegar and Shub [26] simpli ed and uni ed the complexity analysis for these methods. Monteiro et al. [22] proposed a small{step path{following method for QP in which only the primal{dual ane scaling direction is used. Nesterov and Nemirovsky [24] proposed small{step barrier methods for smooth convex programming problems. In Chapter 5 of their monograph they also work out long{step barrier methods for convex quadratic programming problems with linear inequality constraints. Jarre [10], [11] and Mehrota and Sun [18], [19] analysed a small{step path{following method for quadratically constrained QP and smooth convex problems, based on Huard's method of centers. Den Hertog et al. [2] extended this method to a long{step method. In [3] they also proposed a long{step method for smooth convex problems, based on the logarithmic barrier function approach. It is well{known that QP can be formulated as a Linear Complementarity Problem (LCP). So, methods for solving LCP's are indirect methods for QP. However, the transformation from QP to LCP may make these methods inecient for QP, due to the increase in the size of the matrices involved. Many interior point methods for solving LCP have appeared in the literature: e.g. [16], [17], [20], [30]. Kojima et al. [15] gave a uni ed approach for interior point methods for LCP. In this paper we propose a long{step logarithmic barrier method for QP with linear equality constraints. For a suitable proximity criterion, we show that the Newton process quadratically converges, which makes it possible to simplify the analysis signi cantly compared with [2], [3] and [24]. It may be noted that the latter papers deal with inequality constraints, and hence cannot be applied directly to our problem with equality constraints. Moreover, it will appear that the iteration bound derived in this paper is much better than the one obtained by Nesterov and Nemirovsky [24]. Our method is an extension of Den Hertog et al.'s [4] method for linear programming. Steps along projected Newton directions are taken, until the iterate is suciently close to the current center. After that, the barrier parameter is reduced and the process is repeated. We will use the null{space formulation for the projected Newton direction. It is shown that the total number of iterations is O(pnL) or O(nL), dependent on how the barrier parameter is updated. The paper is organized as follows. In Section 2 we prove some properties of the central path. In Section 3 proximity criterions are introduced, and some properties of nearly centered points are proved. The quadratic convergence property will appear to be a good expedient for proving the other properties. Then, in Section 4 we describe our algorithm and in Section 5 we will derive an upper bound for the total number of iterations. Finally, in Section 6 we end up with some remarks.

1

Notation. As far as notations are concerned, e shall denote the vector of all ones and I the identity matrix. Given an n-dimensional vector x we denote by X the n  n diagonal matrix whose

diagonal entries are the coordinates xj of x; xT is the transpose of the vector x and the same notation holds for matrices. Finally kxk denotes the l2 norm.

2 Properties of the central path We consider the convex quadratic programming problem in the following standard form: (QP )

8 > < > :

min q (x) = cT x + 21 xT Qx Ax = b; x  0:

Here Q is a symmetric, positive semi{de nite n  n matrix, A is an m  n matrix, b and c are m? and n? dimensional vectors respectively; the n?dimensional vector x is the variable in which the minimization is done. The dual formulation to (QP ) is: (QD)

8 > < > :

max d(x; y ) = bT y ? 21 xT Qx AT y ? Qx + s = c; s  0;

where y is an m?dimensional vector. It is well{known that for all x and y that are feasible for (QP ) and (QD) we have

d(x; y)  z   q(x); where z  denotes the optimal objective value for (QP ). Optimality holds if and only if the complementary slackness relation xT s = 0 is satis ed. Without loss of generality we assume that all the coecients are integer. We shall denote by L the length of the input data of (QP ). We make the standard assumption that the feasible set of (QP ) is bounded and has a nonempty relative interior. In order to simplify the analysis we shall also assume that A has full rank, though this assumption is not essential. We consider the logarithmic barrier function n 1 T T X f (x; ) := c x + 2 x Qx ? ln xj ; j =1

where  is a positive parameter. The rst and second order derivatives of f are rf (x; ) = c + Qx ? X ?1e;



r f (x; ) = 1 Q + X ? : 2

2

2

(1)

Consequently, f is strictly convex on the relative interior of the feasible set. It also takes in nite values on the boundary of the feasible set. Thus it achieves a minimum value at a unique point. The necessary and sucient rst order optimality conditions for this point are:

AT y ? Qx + s = c; s  0; Ax = b; x  0; Xs = e;

(2) (3) (4)

where y and s are m? and n?dimensional vectors respectively. Let us denote the unique solution of this system by (x(); y (); s()). Then the primal and dual central path is de ned as the solution set x() and y () respectively, for  > 0. It is easy to see that the duality gap in (x(); y (); s()) satis es

x()T s() = n: (5) Hence, if  ! 0 then x() and (x(); y ()) will converge to optimal primal and dual solutions

respectively. The following lemma states that the primal objective decreases along the primal path and the dual objective increases along the dual path. The rst part is a classical result of Fiacco and McCormick [6]. The second part also follows from this result, since (x(); y ()) is the minimizer of the (convex) dual logarithmic barrier function. Our proof is completely dierent from the classical one.

Lemma 1 The objective q(x()) of the primal problem (QP ) is monotonically decreasing and the objective d(x(); y ()) of the dual problem (QD) is monotonically increasing if  decreases.

Proof:

To prove the rst part of the lemma it suces to show that

q(x())0 = cT x0 + xT Qx0  0; (6) where the prime denotes the derivative with respect to . Using that x() and y () satisfy (2){(4) and taking derivatives with respect to  we obtain AT y 0 ? Qx0 + s0 = 0; (7) 0 Ax = 0; (8) 0 0 Xs + Sx = e: (9) Now, using (2) and (8), we have

cT x0 = (s ? Qx + AT y)T x0 = sT x0 ? xT Qx0: Using (9), (4), (7) and (8) respectively, this results into

cT x0 + xT Qx0 = sT x0 = eT Sx0 = (Sx0 + Xs0)T Sx0 = (x0)T S 2x0 + (x0)T s0 = (x0)T S 2x0 + (x0)T Qx0  0:

3

Thus, the rst part of the lemma follows because of (6). To prove the second part of the lemma it suces to show that

d(x(); y())0 = bT y0 ? (x0)T Qx  0:

(10)

Multiplying (9) by AS ?1 we obtain

AS ?1 Xs0 + Ax0 = AS ?1e; which reduces to AX 2s0 = b. Now, taking the inner product with y 0 results into

bT y 0 = (AT y 0 )T X 2s0 = (x0)T QX 2s0 ? (s0 )T X 2s0 = (x0)T QX (e ? Sx0) ? (s0)T X 2s0 : Consequently, we have

bT y0 ? (x0)T Qx = ?(x0 )T Qx0 ? (s0 )T X 2s0  0: Together with (10), this proves the second part of the lemma.

2

3 Properties near the central path To measure the distance to the central path of non{centered points, we will introduce three measures. The rst is analogous to Roos and Vial's [27] appealing measure for linear programming

(x; ) := min k X (c + Qx ? AT y) ? ek: y

(11)

Loosely speaking,  (x; ) measures the deviation from optimality condition (4). The unique solution of the minimization problem in the de nition of  (x; ) is denoted by y (x; ) and the corresponding slack variable by s(x; ) (i.e. s(x; ) = c + Qx ? AT y (x; )). It can easily be veri ed that x = x() () (x; ) = 0 () y(x; ) = y(): In the sequel of this paper we will deal with the scaled version of (QP ): (QP )

8 > < > :

min q (x) = cT x + 21 xT Qx Ax = b; x  0;

where A = AX , c = Xc, Q = XQX , and x is the current iterate. Hence, a step starting at e in (QP ) corresponds to a step starting at x in (QP ). The scaled versions of rf (x; ) and r2 f (x; ) will be denoted by g = g (x; ) and H = H (x; ) respectively. In the algorithm, described in the following section, we will do steps along projected Newton directions with respect to f . Gill et al. [7] give two alternative (equivalent) forms for this direction:

4

 the range{space form p(x; ) = ?H ?1 (I ? AT (AH ?1 AT )?1 AH ?1 )g;

(12)

p(x; ) = ?Z (Z T HZ )?1 Z T g;

(13)

 the null{space form

where Z is an n  (n ? m) matrix, with independent columns, such that AZ = 0. The second and third measure for the distance to the central path are kp(x; )k and kp(x; )kH (x;), where the latter is de ned by kp(x; )k2H (x;) = p(x; )T H (x; )p(x; ). We note that because H (x; ) is positive de nite, k:kH (x;) de nes a norm. In the sequel of the paper we will sometimes write  and p instead of  (x; ) and p(x; ) for briefness' sake. We remark that although all of the three measures will be used in the analysis, only kpkH will be used in the algorithm. We will work with the null{space form for p, because it facilitates the analysis very much. In the analysis we will also assume that Z T Z = I , hence Z is orthonormal. In this case we have the following well{known properties:

Property 1. ZZ T is the projection onto the null{space of A; Property 2. kZk = kk for any ; Property 3. kZ T xk  kxk for any x, with equality if x is in the null{space of A. The following lemma shows that there is a close relationship between the three measures.

Lemma 2 For given x and , kpk  kpkH = ?pT g   . Proof: Using the null{space form (13) it follows that ZZ T g = ?ZZ T HZZ T p. Property 1 and the fact 2

2

2

that Ap = 0 imply ZZ T p = p. Consequently, we may write

?pT g = ?pT ZZ T g = (ZZ T p)T HZZ T p = pT Hp = pT (I + 1 Q)p  kpk : 2

This proves the rst inequality of the lemma. Using the de nition of p it follows that

?gT p

=

gT Z

?1 1 T I + Z QZ Z T g



 ?1  kZ T gk2 k I + 1 Z T QZ k  kZ T gk2; 

5



where the last inequality follows because the eigenvalues of I + 1 Z T QZ equal to 1. Moreover,

ZT g

=

ZT

= ZT

?1

are all less than or

!

c + Qe ? e  ! c + Qe ? AT y ? e ; 

where the last equality holds for any y , because AZ = 0. Putting y equal to y (x; ), it follows from Property 3 that kZ T g k   . Hence it follows that ?g T p   2 . This proves the lemma. 2 Now we will prove some fundamental lemmas for nearly centered points. The following lemma shows quadratic convergence in the vicinity of the central path for all of the three measures.

Lemma 3 If kp(x; )k < 1 then x = x + Xp(x; ) is a strictly feasible point for (QP ). Moreover, kp(x; )k  kp(x; )kH x;  (x; )  kp(x; )k  kp(x; )kH x;  (x; ) : (

2

)

2

(

)

2

Proof:

It is easy to see that Ax = b. Moreover, x = X (e + p) > 0, because kpk < 1. This proves the rst part of the lemma. Using the de nition of  (x; ) we have   (x; ) = min k X (I + P ) c + QX (e + p) ? AT y ? ek

 (I + P ) c + Q(e + p) ? AT y  ? ek; = min k y  y

where So

(14)

!

?p = Z (I + 1 Z T QZ )?1Z T c +Qe ? e : !

Q(e + p) = Qe + QZ (I + 1 Z T QZ )?1 Z T e ? c +Qe :

Next consider the projection of Q(e + p) onto the null-space of A (using Property 1):

ZZ T Q(e + p) = ZZ T Qe + ZZ T QZ (I + 1 Z T QZ )?1 Z T e ? c +Qe = Z = Z

"

"

Z T Qe +



!

 (I + 1 Z T QZ ) ? I (I + 1 Z T QZ )?1 Z T e ? c +Qe

Z T Qe + Z T e ? Z T Qe ? Z T c ? (I

= ZZ T (e ? c) ? p:

6

!#

+ 1 Z T QZ )?1 Z T e ? c +Qe

!#

Note that there are certainly m{dimensional vectors u and v such that

Q(e + p) = ZZ T Q(e + p) + AT u; and

(e ? c) = ZZ T (e ? c) + AT v: Setting y = u ? v we then obtain

Q(e + p) ? AT y = (e ? c) ? p:

(15)

Using this y in (14), it follows that

(x; )  k(I + P )(e ? p) ? ek = kPpk  kpk2: This proves the middle inequality of the lemma. The rest follows immediately from Lemma 2.

2

Lemma 4 If kpkH < 1 then 2 f (x; ) ? f (x(); )  1 ?kpkkpHk2 :

H

Proof:

The barrier function f is convex for xed , whence

f (x; ) ? f (x + Xp; )  ?g T p = kpk2H ;

(16)

where the equality follows from Lemma 2. Now, let x0 := x and let x0 ; x1; x2;


denote the sequence of points obtained by repeating Newton steps, starting at x0 . Then we may write, using Lemma 3 1 X

f (x; ) ? f (x(); ) =

i=0

1 X



i=0

f (xi; ) ? f (xi+1 ; )



kpkH+1 2i

 1 ?kpkkpHk : H 2

2

Lemma 5 If kpkH < 1; then

p jq(x) ? q(x())j  kpkH1 ?(1 k+pkkpkH )  n: H

7

2

Proof:

Since q (x) is convex, we have

rq(x)T Xp  q(x + Xp) ? q(x)  rq(x + Xp)T Xp:

(17)

For the left{hand side expression we can derive the following lower bound, using that rq (x) = c + Qx = X ?1 g + X ?1 e

p rq(x)T Xp = gT p + eT p  ?kpkH ? kpkH n; 2

(18)

where the last inequality follows from Lemma 2. Now, using (15), we derive an upper bound for the right{hand side expression in (17):

rq(x + Xp)T Xp = cT p + pT Q(e + p) = cT p + pT (p AT y + e ? c ? p) = eT p ? kpk  eT p  kpkH n:

(19)

Consequently, substitution of (18) and (19) into (17) yields

p jq(x) ? q(x + Xp)j  kpkH (1 + kpkH ) n: Again, let x := x and let x ; x ; x ;


denote the sequence of points obtained by repeating Newton steps, starting at x . Then we may write, using Lemma 3 0

0

1

2

0

jq(x) ? q(x())j =  



1 X

i=0

1 X i=0

1 X i=0

q(xi ) ? q(xi+1 )

q(xi ) ? q(xi+1 )

p kpkH i(1 + kpkH i) n 2

2

1 X p  (1 + kpkH ) n kpkH i i k p k H (1 + kpkH ) p  1 ? kpkH  n: 2

=0

2

4 The Algorithm In our long{step algorithm a linesearch along the Newton direction is done until the iterate is suciently close to the current center. After that, the barrier parameter is reduced, and the process starts again. In the next section we will show that a linesearch along the Newton direction reduces the barrier function value by a constant if kpkH  21 . This enables us to derive an upper bound for the number of steps between reductions of .

8

If kpkH < 12 we can give bounds on f (x; ) ? f (x(); ), jq (x) ? q (x())j and cT x ? z  by Lemmas 4 and 5.

Algorithm Input:

0 is the initial barrier value, 0  2O(L); t is an accuracy parameter, t = O(L);  is the reduction parameter, 0 <  < 1; x0 is a given interior feasible point such that kp(x0; 0)kH (x0;0 )  21 ;

begin

x := x0;  := 0 ; while  > 2?t do

begin (outer step) while kpkH  do begin (inner step) ~ := arg min> ff (x + Xp; ) : x + Xp > 0g 1 2

x := x + ~Xp

0

end (inner step)  := (1 ? ); end (outer step) end. For nding the initial point that satis es the input assumptions of the algorithm we refer the reader to e.g. Ye [28]. Later on the centering assumption kp(x0; 0)kH (x0;0 )  21 will be relaxed.

5 Convergence analysis of the Algorithm In this section we will derive upper bounds for the total number of outer and inner iterations.

Theorem 1 After at most K = O( L ) outer iterations, the algorithm ends up with a primal solution such that q (x) ? z   2?O L . Proof: We can derive an upper bound for the gap q (x) ? z  after K outer iterations from (5) and Lemma 5: p q(x) ? z  = q(x( )) ? z  + q(x) ? q(x( ))   (n + 3 n); ( )

K

where

K = (1 ? )K 0 .

K

K

2

This means that q (x) ? z   2?O(L) certainly holds if

Taking logarithms we require

p (1 ? )K 0 (n + 23 n)  2?O(L):

p O (L) + ln(n + 32 n) + ln 0 K : ? ln(1 ? ) 9

Since we have assumed that 0  2O(L), and since   ? ln(1 ? ), this certainly holds if K = O( L ).

2

We note, that this nal primal solution can be rounded to an optimal solution for (QP ) in O(n3) arithmetic operations. (See e.g. [25].) The following lemma is needed to derive an upper bound for the number of inner iterations in each outer iteration. It states that a sucient decrease in the value of the barrier function can be obtained by taking a step along the Newton direction.

Lemma 6 Let  := (1 + kpkH )? . Then
f := f (x; ) ? f (x + Xp; )  kpkH ? ln(1 + kpkH ): 1

Proof:

We write down the Taylor expansion for f :

f (x + Xp; ) = f (x; ) + g T p + 12 2 pT Hp + where tk denotes the k?order term in the Taylor expansion. Since n kX tk = (?) pki ;

k

n X i=1

k=3

tk ;

i=1

we nd, by using Lemma 2, n kX k jtk j  k jpijk  k i=1

n X

jpij

2

!k

2

 k kpkk  k kpkkH : k

k

Using Lemma 2, we have for the linear and quadratic term in the Taylor series pT g + 12 2 pT Hp = ( 12 2 ? )kpk2H : So we nd n k X f (x + Xp; )  f (x; ) + ( 12 2 ? )kpk2H + k kpkkH k=3  f (x; ) ? kpk2H ? ln(1 ? kpkH ) ? kpkH : Hence


f  (kpk2H + kpkH ) + ln(1 ? kpkH ): (20) The right hand side is maximal if  =  = (1 + kpkH )?1 . Substitution of this value nally gives
f  kpkH ? ln(1 + kpkH ):

2

This proves the lemma.

10

Theorem 2 Each outer iteration requires at most

? p
N = 111? n + 3 n + 11 3

inner iterations.

Proof:

This proof is a generalization of Gonzaga's [9] proof for the linear case. Let us consider the (k +1)'st outer iteration. The starting point is then (xk ; k ), with kp(xk ; k?1 )kH (x ; ?1 ) < 12 . Let N denote the number of inner iterations. For each inner iteration we know, according to Lemma 6, that the decrease in the potential function value is at least 1:
f  12 ? ln(1 + 21 ) > 11 Following the N inner iterations, we have xk+1 with kp(xk+1 ; k )kH (x +1; ) < 12 . So, we have k

k

Equivalently,

k

k

1 N: f (xk+1 ; k )  f (xk ; k ) ? 11

1 N  f (xk ;  ) ? f (xk+1 ;  ): (21) k k 11 Now we will derive an upper for the right hand side. The de nition of f (x; ); x > 0 implies that

f (x; k ) = f (x; k?1 ) + q(x) ? q(x) k k?1   1 q ( x ) ?1 = f (x; k?1 ) +  k?1 1 ?  = f (x; k?1 ) +  q (x) : 1?  k?1

Using this we obtain

f (xk ; k ) ? f (xk+1 ; k ) = f (xk ; k?1 ) ? f (xk+1 ; k?1 ) + 1 ?   1 (q(xk ) ? q(xk+1 )): (22) k?1 Because xk and xk+1 are approximately centered with respect to x(k?1 ) and x(k ) respectively, using Lemma 5 for the rst and Lemma 1 for the second inequality we nd p p q(xk ) ? q(xk+1 )  q(x(k?1 )) + 23 k?1 n ? q(x(k )) + 23 k n p = q (x(k?1 )) ? q (x(k )) + 32 (2 ? )k?1 n  (q(x(k?1)) ? d(x(k?1); y(k?1))) p ? (q(x(k )) ? d(x(k)p; y(k ))) + 3k?1 n = k?1 n ? k n + 3k?1 n ? p
= k?1 n + 3 n :

11

Secondly, using Lemma 4 (with kpkH = 12 ), and the fact that x(k?1 ) minimizes f (x(k?1 ); k?1 ), we obtain

f (xk ; k?1 ) ? f (xk+1 ; k?1 ) = f (xk ; k?1 ) ? f (x(k?1 ); k?1) +f (x(k?1 ); k?1) ? f (xk+1 ; k?1 )  f (xk ; k?1) ? f (x(k?1 ); k?1)  13 : Hence, substitution of the last two inequalities into (22) yields

? p
f (xk ; k ) ? f (xk+1 ; k )  1 ?  n + 3 n + 31 :

Substitution of this inequality into (21) yields the lemma. Combining Theorem 1 and 2, the total number of iterations turns out to be given by  11 ?n + 3pn
+ 11  O(L): 1? 3 This makes clear that  if we take  = O( p1n ) then O(pnL) iterations are needed;

2 (23)

 if we take  = O(1) then O(nL) iterations are needed.

6 Concluding remarks

6.1 Computing the Newton direction

Even though our analysis is based on the null{space form for the Newton direction, in practice either the null{space or the row{space form can be used. It is obvious that the null{space form is more ecient when the number of linear constraints is relatively large compared to the number of variables. The row{space form is ecient when the number of linear constraints is small compared to the number of variables. In our analysis we assumed that Z is orthonormal. However, the search direction doesn't change if Z is any basis for the nullspace of A. So, in practice we don't have to do all the work to nd an orthonormal Z on each iteration. For example, if a Z is found such that AZ = 0, then Z = X ?1 Z satis es AXZ = 0. We refer the reader to Gill et al. [7] for the numerical aspects.

6.2 Obtaining dual feasible solutions

At the end of each sequence of inner iterations we have a primal feasible x such that kpkH  1. The following lemma shows that a dual feasible solution can be obtained by performing an additional full Newton step, and projection.

12

Lemma 7 Let x = x + Xp(x; ). If kp(x; )kH x;  1 then  := (x; )  1 and y := y(x; ) is dual feasible. Moreover, the duality gap satis es p p (n ?  n)  q(x) ? d(x; y)  (n +  n): (

)

Proof:

By Lemma 3 we have  (x; )  kp(x; )k2H (x;)  1. By the de nition of s(x; ) = c + Qx ? AT y(x; ) we have   (x ; ) = k X s(x ; ) ? ek  1:



This implies s(x ; )  0, so y (x; ) is dual feasible. Moreover, (x )T s(x ; ) X s(x; ) X s(x; )



? n = eT (

? e)  kekk

  Consequently, using that (x)T s(x; ) = q (x) ? d(x; y ) p p (n ?  n)  q(x) ? d(x; y)  (n +  n):

? ek = pn:

6.3 Small{step path{following methods

2

Small{step path{following methods start at a nearly centered iterate and after the parameter is reduced by a small factor, a unit Newton step is taken. The reduction parameter is suciently small, such that the new iterate is again nearly centered with respect to the the new center. Small{ step barrier methods for convex quadratic programming have been given by Ye [28], Goldfarb and Liu [8], and Ben Daya and Shetty [1]. The following lemma shows that if  is small, then we obtain such a small{step path{following method. Lemma 8 Let x := x + Xp(x; ) and  := (1 ? ), where  = 101pn . If (x; )  21 then (x;  )  12 .

Proof:

Due to the de nition of our measure we have  (x; ) = k Xs(x;  ) ? ek

  k Xs(x; ) ? ek = k 1 ?1  ( Xs(x; ) ? e) + ( 1 ?1  ? 1)ek p  1 ?1  ((x; ) +  n)  1 ?1 1 ( 12 + 101 ) 10 2 = : 3

13

Now we can apply the quadratic convergence result (Lemma 3) (x ; )  (x; )2  49 < 21 :

2

6.4 Relaxing the initial centering condition

The initial centering condition kp(x0; 0)kH (x0;0 )  21 can be relaxed to

p f (x0; 0) ? f (x(0 ); 0)  O( nL)

if  = O( p1nL ) is used, and to

f (x0; 0 ) ? f (x(0); 0)  O(nL) if  = O(1) is used. This holds because of Lemma 6. For the last case the assumption is equivalent with the assumption x0j  2?L for all j . This can easily be veri ed. Since x(0 ) is primal feasible, it can be written as a convex combination of basic feasible solutions. The coordinates xj of each basic feasible solution satisfy xj  2L . Moreover, q (x0 ) ? q (x(0))  2O(L). Therefore

f (x0; 0) ? f (x(0 ); 0) = q(x ) ?q(x(0 )) ? 0

0

n X j =1

ln x0j +

n X j =1

ln xj (0 )  O(nL):

6.5 Results for the LP case

It is worthwile to look at the results for the LP case, for which Q = 0. In this case the projected Newton direction (13) reduces to p = ?ZZ T g , which coincides with the scaled projected gradient direction. It is easy to verify that the three measures kpk, kpkH and  are exactly the same. (i.e. in Lemma 2 equalities hold instead of inequalities). Hence, the resulting algorithm and results are the same as in [4].

6.6 Barrier methods for LCP

It is well{known (see e.g. Murty [23]) that the Linear Complementarity Problem (LCP) (LCP )

8 > < > :

y = Mx + q; x; y  0 xT y = 0;

is completely equivalent with the following quadratic programming problem (QP )

8 > < > :

min xT y y = Mx + q; x; y  0:

If M is positive semi{de nite, then this problem is a equivalent to a convex quadratic programming problem, since xT y = 21 xT Qx + q T x, where Q = M + M T . Consequently, the algorithm proposed in this paper can also be applied to positive semi{de nite LCP's.

14

6.7 Comparison with Nesterov and Nemirovsky's results

In Chapter 5 of their monograph [24], Nesterov and Nemirovsky analysed long{step barrier methods for QP with linear inequality constraints. Their analysis is totally dierent from ours: it is not based on changes in the barrier function value, for example. On one hand their analysis is more general, but on the other hand it is also very complicated. From their analysis it can be extracted that the total number of iterations is at most O(( 1 + n4 7 )L ln n). Note that the iteration bound (23) is better than this one. Our bound is much better if we deal with real long{step algorithms (i.e.  is large). For example:  If we take  = O( p1n ) pthen Nesterov and Nemirovsky require O(pnL ln n) iterations. The dierence with our O( nL) iteration bound is not so signi cant in this case.  If we take  = O(1) then Nesterov and Nemirovsky require O(n4L ln n) iterations. This bound is much worse than our O(nL) iteration bound.

References [1] Ben Daya, M., Shetty, C.M. (1988), Polynomial Barrier Function Algorithms for Convex Quadratic Programming, Research Report, No. J 88{5, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta. [2] Den Hertog, D., Roos, C., Terlaky, T. (1990), A Potential Reduction Method for a Class of Smooth Convex Programming Problems, Report No. 90{01, Faculty of Mathematics and Informatics/Computer Science, Delft University of Technology, Delft, Holland. [3] Den Hertog, D., Roos, C., Terlaky, T. (1990), On the Classical Logarithmic Barrier Method for a Class of Smooth Convex Programming Problems, Report No. 90{28, Faculty of Mathematics and Informatics/Computer Science, Delft University of Technology, Delft, Holland. p [4] Den Hertog, D., Roos, C., Vial, J.-Ph. (1990), An O( n) Complexity Reduction for Long Step Path{following Methods, Revised version of Report No. 89{85, Faculty of Mathematics and Informatics/Computer Science, Delft University of Technology, Delft, Holland. [5] Dikin, I.I. (1967), Iterative Solution of Problems of Linear and Quadratic Programming, Doklady Akademiia Nauk SSSR 174, 747{748. [6] Fiacco, A. V. and McCormick, G. P. (1968), Nonlinear Programming, Sequential Unconstrained Minimization Techniques, Wiley and Sons, New York. [7] Gill, P.E., Murray, W. and Wright, M.H. (1988), Practical Optimization, Seventh Printing, Academic Press, Inc., San Diego, California. [8] Goldfarb, D. and Liu, S. (1988), An O(n3L) Primal Interior Point Algorithm for Convex Quadratic Programming, Manuscript, Department of Industrial Engineering and Operations Research, Columbia University, New York, NY. [9] Gonzaga, C.C. (1989), Large{Steps Path{Following Methods for Linear Programming: Barrier Function Method, Report ES{210/89, Department of Systems Engineering and Computer Sciences, COPPE{Federal University of Rio de Janeiro, Rio de Janeiro, Brasil.

15

[10] Jarre, F. (1987), On the Convergence of the Method of Analytic Centers when applied to Convex Quadratic Programs, Report No. 35, Schwerpuntkprogramm Anwendungsbezogene Optimierung und Steuerung, Institut fur Angewandte Mathematik und Statistik, Universitat Wurzburg, Wurzburg, West{Germany. [11] Jarre, F. (1989), The Method of Analytic Centers for Smooth Convex Programs, Dissertation, Institut fur Angewandte Mathematik und Statistik, Universitat Wurzburg, Wurzburg, West{ Germany. [12] Kapoor, S. and Vaidya, P. (1986), Fast Algorithms for Convex Quadratic Programming and Multicommodity Flows, Proc. 18th Annual ACM Symp. Theory Comput. 147{159. [13] Kapoor, S. and Vaidya, P.M. (1988), An Extension of Karmarkar's Interior Point Method to Convex Quadratic Programming, Technical Report, Department of Computer Science, University of Illinois at Urbana{Champaign, Urbana. [14] Karmarkar, N. (1984), A New Polynomial{Time Algorithm for Linear Programming, Combinatorica 4, 373{395. [15] Kojima, M., Megiddo, N., Noma, T., Yoshise, A. (1990), A Uni ed Approach to Interior Point Algorithms for Linear Complementarity Problems, Research Report RJ 7493 (70008), IBM Almaden Research Division, San Jose, California. [16] Kojima, M., Mizuno, S., Yoshise, A. (1987), A Polynomial Time Algorithm for a Class of Linear Complementarity Problems, Research Report, Department of Information Sciences, Tokyo Institute of Technology, Tokyo, Japan. p [17] Kojima, M., Mizuno, S., Yoshise, A. (1988), An O( nL) Iteration Potential Reduction Algorithm for Linear Complementarity Problems, Research Report, Department of Information Sciences, Tokyo Institute of Technology, Tokyo, Japan. [18] Mehrota, S. and Sun, J. (1988), On Computing the Centers of a Convex Quadratically Constrained Set, Technical Report 88{11, Department of IE/MS, Northwestern University, Evaston IL. [19] Mehrota, S. and Sun, J. (1988), An Interior Point Algorithm for Solving Smooth Convex Programs Based on Newton's Method, Technical Report 88{08, Department of IE/MS, Northwestern University, Evaston IL. [20] Mizuno, S. (1990), An O(n3L) Algorithm Using a Sequence for a Linear Complementarity Problem, Journal of the Operations Research Society of Japan Vol. 33, No. 1. [21] Monteiro, R.D.C., Adler, I. (1987), Interior Path Following Primal{Dual Algorithms, Part II: Convex Quadratic Programming, Department of Industrial Engineering and Operations Research, University of California, Berkeley, California. [22] Monteiro, R.D.C., Adler, I., Resende, M.G.C. (1988), A Polynomial{Time Primal{Dual Ane Scaling Algorithm for Linear and Convex Quadratic Programming and its Power Series Extension, Report ESRC 88{8, Department of Industrial Engineering and Operations Research, University of California, Berkeley, California.

16

[23] Murty, K.G. (1988), Linear Complementarity, Linear and Nonlinear Programming, Heldermann Verlag Berlin, Germany. [24] Nesterov, Y.E., and Nemirovsky, A.S. (1989), Self{Concordant Functions and Polynomial Time Methods in Convex Programming, Moscow. [25] Papadimitriou, C.R., and Steiglitz, K. (1982), Combinatorial Optimization: Algorithms and Complexity, Prentice{Hall, Englewood Clis, New Jersey. [26] Renegar, J., and Shub, M. (1988) Simpli ed Complexity Analysis for Newton LP Methods, Technical Report No. 807, School of Operations Research and Industrial Engineering, Cornell University, Ithaca, New York. [27] Roos, C. and Vial, J.{Ph. (1988), A Polynomial Method of Approximate Centers for Linear Programming, to appear in Mathematical Programming. [28] Ye, Y. (1987), Further Development on the Interior Algorithm for Convex Quadratic Programming, Engeneering{Economic Systems Department, Stanford University, Stanford. [29] Ye, Y. (1987), Interior Algorithms for Linear, Quadratic and Linearly Constrained Convex Programming, Ph.D. Dissertation, Engeneering{Economic Systems Department, Stanford University, Stanford. [30] Ye, Y. and Pardalos, P. (1989), A Class of Linear Complementarity Problems Solvable in Polynomial Time, Department of Management Sciences, The University of Iowa, Iowa City, Iowa. [31] Ye, Y. and Tse, E. (1986), A Polynomial{time Algorithm for Convex Quadratic Programming, Manuscript, Engeneering{Economic Systems Department, Stanford University, Stanford.

17