Primal-dual target-following algorithms for linear ... - CiteSeerX

Primal-dual target-following algorithms for linear programming Report 93-107

B. Jansen C. Roos T. Terlaky J.-Ph. Vial

Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics and Informatics Technische Universiteit Delft Delft University of Technology

ISSN 0922-5641

Copyright c 1993 by the Faculty of Technical Mathematics and Informatics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, microfilm, or any other means without permission from the Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands. Copies of these reports may be obtained from the bureau of the Faculty of Technical Mathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone +3115784568. A selection of these reports is available in PostScript form at the Faculty’s anonymous ftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl

DELFT UNIVERSITY OF TECHNOLOGY

REPORT Nr. 93{107 Primal{Dual Target{Following Algorithms for Linear Programming

B. Jansen, C. Roos, T. Terlaky, J.{Ph. Vial

ISSN 0922{5641 Reports of the Faculty of Technical Mathematics and Informatics Nr. 93{107 Delft, November 23, 1993 i

B. Jansen, C. Roos and T. Terlaky, Faculty of Technical Mathematics and Informatics, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. e{mail: [email protected], [email protected], [email protected] J.{Ph. Vial, Department of Commercial and Industrial Economics, University of Geneva, Geneva, Switzerland. e{mail: [email protected]. This work is completed with the support of a research grant from SHELL. The rst author is supported by the Dutch Organization for Scienti c Research (NWO), grant 611-304-028. The third author is on leave from the Eotvos University, Budapest, and partially supported by OTKA No. 2116. The fourth author is supported by the Swiss National Foundation for Scienti c Research, grant 12-34002.92.

c 1993 by Faculty of Technical Mathematics and Copyright Informatics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, micro lm or any other means without written permission from Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands. ii

Contents 1 Introduction

1

2 The {space

4

v

2.1 Representation of primal{dual pairs in the {space 2.2 Proximity to the central path 2.3 Target{sequences v

: : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

3 The ( ){space

4 5 5

6

x; s

3.1 The Newton step in the ( ){space 3.2 Proximity to the target point 3.3 Analysis of the Newton step 3.3.1 Feasibility 3.3.2 Eect on the proximity measure 3.3.3 Eect on the duality gap 3.3.4 Updating the target x; s

: : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : :

4 An algorithm using Dikin steps

12

4.1 The Dikin step in the {space 4.1.1 Feasibility and step size 4.1.2 Proximity to the central path 4.2 Algorithm 4.3 Analysis 4.4 Improving the complexity bound 4.5 Centering 4.6 General{order scaling v

6 8 9 9 10 11 11

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

5 Further applications of the target{following concept iii

13 13 14 15 16 17 19 20

21

5.1 Path{following methods 5.2 Ecient centering 5.3 Computing a weighted center 5.3.1 Mizuno's algorithm 5.3.2 Atkinson and Vaidya's algorithm

: : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

6 Concluding remarks

21 22 23 24 25

28 Abstract

In this paper we propose a method for linear programming with the property that, starting from an initial non{central point, it generates iterates that simultaneously get closer to optimality and closer to centrality. The iterates follow paths that in the limit are tangential to the central path. Along with the convergence analysis we provide a general framework which enables us to analyze various primal{dual algorithms in the literature in a short and uniform way. Key words: interior{point method, primal{dual method, target{following, centering, Dikin steps. Running title: Target{Following Methods for LP.

iv

1 Introduction The purpose of this paper is two{fold. Firstly, it proposes a new primal{dual path{following method for linear programming that may start at a non{central point and that eciently combines centering and moving towards optimality. Secondly, it oers a framework for the convergence analysis of primal{dual interior point methods. This framework is general enough to apply to very diverse existing methods and still yield simple convergence proofs. To be more speci c we introduce the linear programming problem in standard form min f cT x : Ax = b; x  0 g and its dual

max f bT y : AT y + s = c; s  0 g: The optimal solutions are characterized by the system of equations1

Ax = b; x  0; = c; s  0; xs = 0:

AT y + s

(1)

This system can be viewed as the limit of the parametrized system of equations

Ax = b; x  0; = c; s  0; xs = e;

AT y + s

(2)

where  > 0 is a parameter, and e is the vector of all ones. Under the assumption that there exists a primal{dual interior feasible pair, for xed  the system (2) has a unique solution, named the {center. Moreover, the set of solutions as  varies describes a smooth curve named the central path. This path has been extensively studied by McLinden [23], Megiddo [24] and Bayer and Lagarias [3]. The best convergence results for interior point methods are obtained by path{following schemes. Those schemes, when initiated at, or close to, the central path, generate, by means of Newton steps, a sequence of iterates that closely approximate the central path, while achieving at each step a signi cant decrease of the duality gap. The cumulated decreases yield the complexity bound. To implement the method one has to nd rst an initial (central) point. One way to deal with this issue is to reformulate the original problem in such a way that some privileged point, say the vector e of all ones, be feasible and perfectly centered in the new formulation. This was the approach rst proposed by Renegar [31] and by Monteiro and Adler [29]. Alternatively, one can transfrom the problem to a homogeneous self{dual formulation, see Ye et al. [34] or Jansen et al. [18], which has the advantage that there exists a pair of positive primal{dual feasible solutions, which need not be feasible for the original problem pair. The solution of this problem provides the solution to the original problem pair. As far as notation is concerned, if x; s 2 IRn then xT s denotes the dot product of the two vectors, whereas p xs, x and x for  2 IR denote the vectors obtained from componentwise operations. 1

1

Our goal here is to propose an algorithm that can start from any initial interior feasible pair and decreases the duality gap and the non{centrality at each iteration. We shall assume from now on that an initial interior point has been produced. Note, that we might use a self{dual reformulation for this and hence allow for infeasible (but positive) points. Let us mention that centering may serve other useful purposes than just producing an initial (centered) point for a standard path{following method. A sequence of iterates that approximate the central path will generate points converging to the analytic center of the optimal face. It is well{known that this center is a strictly complementary solution, thereby de ning the optimal partition of indices that characterizes the optimal set. Weighted centers may be used to elicit the optimal partition; the analytic center however has the advantage of being neutral and free from the arbitrariness of the choice of the weights. This turns out to be very important in performing the asymptotic analysis of algorithmic sequences, e.g., [9, 10, 11]. On the other hand, Adler and Monteiro [1], Jansen et al. [15], (see also [12]) showed the value of the knowledge of the optimal partition for correct and meaningful sensitivity analysis. Some methodsphave been proposed for centering alone. Den Hertog [13] and Mizuno [25] propose methods in O( n) iterations, while Gon and Vial [8] give an algorithm in O(n) iterations. The former methods can be coupled withpa standard path{following method to obtain optimality for thepLP{problem, converging in O( nL) iterations. The best overall result is an algorithm in O( nL) iterations. An alternative, that by{passes the centering phase, is a weighted path{following method, approximating the weighted path that goes through the initial non{central point. Weighted path{ following methods reported in the literature are Ding and Li [4], Den Hertog et al. [14], Mizuno [26],p Roos and Den Hertog [32], and Gon and Vial [7], which all have a complexity bound of O( !n L), where ! is the square root of the ratio between the smallest to the largest complementarity pair (a measure of eccentricity) of the initial point. This is not as good a complexity result as the one that is obtained by performing sequentially the centering phase and then the optimality phase. The other drawback of a weighted path{following approach is that the iterates always keep away from the central path. Our objective is to propose a new method that remedies the disadvantage of limiting sequences of non{central points. It combines centering and optimality and eventually produces nearly centered primal-dual pairs. Starting away from the central path, it follows a path that is tangent to the central path in the limit. To de ne this trajectory it is best to introduce a new space, named hereafter the v {space, made of the square roots of the complementary pairs: p vi = xisi . One easily realizes that the image of the central path in the v{space is the main diagonal and that the image of the weighted p 0 0path that passes through an initial point (x0; s0) is 0 the positive ray passing through v = x s . We are interested in a trajectory whose image in the v {space passes through v 0 and is tangent to the main diagonal at the origin of the positive orthant. In this paper we suggest to use the vector eld of the primal{dual Dikin direction and its associated set of trajectories. This direction was introduced by Jansen et al. [17]; it is de ned as the solution of the minimization problem

min f v T
v :

v ?1
v

 1 g:

This problem can be interpreted as the one of nding the direction in the v {space that aims at 2

a maximum decrease of the duality gap within Dikin's ellipsoid in the v {space. It is easy to see that the solution
v is a vector proportional to ?v 3 . So the equation of the trajectory passing through v 0 and tangent to the vector eld is

0 v(t) = p 0v 2 ; t  0: (v ) t + 1 For t = 0, v (0) = v 0 and for t ! 1, v (t) tends to zero tangentially to the vector e.

To analyze this new algorithm, we were led to focus on a few general concepts. The basic algorithmic step in path{following primal{dual interior point methods is a Newton step in the (x; s) space. This step is de ned with respect to some target point in the v {space. The fundamental property in interior point methods is that the step is feasible (i.e., preserves the interior point property) if the current iterate (x; s) is close enough to the target v , where closeness is de ned with respect to some appropriate measure of proximity. With this in mind, we can de ne the concept of a target{sequence, by which we mean any sequence of vectors in the v {space. Traceable target{sequences are target{sequences with the property that: (i) it is approximated, in the sense of the above mentioned proximity measure, by a sequence of points in the (x; s)-space; (ii) successive points in the (x; s) space are obtained by some `easy' computations such as one or a few Newton steps. If the target{sequence converges to some point, then we may enforce convergence of the associated (x; s) sequence to the target limit. By applying this analysis to a special sequence that approximates the Dikin trajectory pn in the v {space, we were able to prove convergence of the new path{following method in O( ! L) iterations. The complexity bound is not better than the one obtained for weighted path{following algorithms, but the new algorithm has the advantage of generating, in theory and in practice, increasingly centered pairs. The striking feature of the convergence analysis of the new algorithm is that it is essentially performed in the v {space. We were able to express a simple condition on the target{sequence to be traceable by a sequence of primal{dual pairs (x; s). By verifying that the target{sequence that approximates Dikin's trajectory in the v {space satis es the condition, we easily derived the complexity bound. This methodology can also be used to analyze other target{sequences. It turns out that most primal{dual interior point methods can be interpreted in terms of target{ sequences. It suces then to check if the condition of the main theorem are satis ed to immediately derive the complexity bound. In this way, we were able to analyze and prove convergence of a great variety of algorithms such as the standard path{following methods [29, 30], [22], the weighted path{following methods [4], algorithms for computing analytic centers [13, 25] and algorithms for computing weighted centers [27, 2]. The convergence proofs are strikingly short and thereby demonstrate the unifying value of an analysis focusing on the v {space. While this paper was under completion, we discovered that a similar idea of target{following was introduced by Mizuno [25, 27]. He uses the notion of {sequences, which form a special class of our target{sequences. He applies his methodology to linear complementarity problems. Moreover, Den Hertog's algorithm (Section 5.2) is also found in [25], and the algorithm for computing a weighted center in Section 5.3.1 is not new, but is to be attributed to [27]. The paper is organized as follows. In Section 2 we concentrate on the v {space and introduce some formal notions. In Section 3 we consider the (x; s){space and analyze the Newton step to a target. In Section 4 we apply the general results from the previous section to the new algorithm 3

combining the centering phase with the optimality phase by Dikin steps. Finally, in Section 5 we analyze in the new framework other algorithms from the literature.

2 The {space v

2.1 Representation of primal{dual pairs in the {space v

For each positive primal{dual pair (x; s) satisfying the feasibility constraints in the primal and the dual problem, de ne p (x; s) := xs: Hence, 2 IRn+ , the space of positive vectors of size n. We make the standard assumption in interior point methods, namely that there exists a feasible pair (x; s) for which x > 0 and s > 0. The following theorem establishes a one{to{one correspondence between pairs (x; s) and positive vectors in IRn . It was proved in Kojima et al. [21]. Here we present a simple alternative proof.

Theorem 2.1 For each v 2 IRn+ there exists a unique positive primal{dual feasible pair (x; s) such that (x; s) = v:

Proof: The function g : IR+ ! IR de ned by

g(t) := t ? ln t ? 1; t > 0: is strictly convex, and minimal for t = 1 with g (1) = 0: Now consider h(x; s) :=

n X

j =1

(3)

vj2 g(tj );

P

with tj = (xj sj )=vj2. If we restrict the domain of h to positive primal{dual pairs, then j tj is linear in x and s, and h is convex. It can easily be veri ed that the rst order conditions for optimality are given by the system

Ax = b; x  0; = c; s  0; (4) 2 xs = v : Clearly, if (~x; y~; s~) is the (unique) minimizing point then (~x; s~) is a primal{dual interior feasible pair such that (~x; s~) = v: 2 AT y + s

Because of the one{to{one correspondence we can now identify the space of positive primal{dual feasible pairs with IRn+ . Henceforth, we will denote the latter by the term v {space, keeping in 4

mind that the transformation depends on the LP{problem under consideration. Recall that x lies on the central path of (P ) if and only if for some dual feasible y the slack vector s := c ? AT y is such that xs is a scalar multiple of the all one vector e. In that case y lies on the central path of (D). We call the pair (x; s) centered if this happens. Clearly, the pair (x; s) is centered if and only if v = (x; s) is a scalar multiple of the all one vector e. Thus we conclude that in the v {space the straight half line e;  > 0; represents the centered pairs. Also note that if v = (x; s), then kvk2 = xT s. So in the v{space the points with constant norm represent all positive feasible primal{dual pairs with a xed duality gap. Note, that all optimal pairs (x; s) correspond to the vector v = 0.

2.2 Proximity to the central path In the v {space the central path is just a straight line, namely v = e for  > 0. Furthermore, every weighted path is also a straight line, since the ratio between all the elements of the weight{ vector do not change. Atkinson and Vaidya [2] discuss how the eciency of Newton's method is aected by such dierences in the elements of a weight{vector. They give a simple example which demonstrates that when the ratio between the largest and the smallest weight increases, then the region where Newton's method converges gets smaller. Hence, a natural way of measuring the closeness of a point to the central path appears to be this ratio. Henceforth we denote min(v) ! := !(v) := max ; (5) (v) where we use the notation min(u) = mini (ui ) for any vector u and likewise max(u) = maxi (ui). Note that 0 < !  1, with equality if and only if v is on the central path.

2.3 Target{sequences We will use the one{to{one correspondence between points in the v {space and primal{dual interior feasible pairs in the following way. Given a point in the v {space and its primal{dual (x; s) pair, we will choose a dierent v and compute (approximately) the new pair (x; s) by Newton's method. A similar general methodology was used by Mizuno [25, 27] for the linear complementarity problem. In fact this framework can be used to interpret almost every interior point method: set a target in the v {space and try to reach the target by way of Newton steps. In the standard path following methods the targets are points on the central path. Then the (traceable) target{sequence is determined by v(0) = e; v(k+1) = (1 ? k )v(k); for certain values  > 0 and 0  k  1, where k is the iteration number. A weighted{path following algorithm has a given v(0) > 0 and sets v(k+1) = (1 ? k )v(k): However, the one{to{one correspondence between points in the v {space and positive primal{dual pairs (x; s) suggests that, to solve the linear programming problem, we can follow any sequence 5

of targets fv(k)g in the v {space, for which eT (v (k))2 tends to zero, hence leads to optimality. For this reason we construct a sequence of primal{dual pairs f(x(k) ; s(k))g, that are close to the exact primal{dual pairs determined by v(k). The computations should be 'easy', for instance by way of (truncated) Newton steps. As we will see later, the same methodology of tracing points in the v {space by Newton steps can be used to solve other problems, like computing weighted centers. A target{sequence may consist of an in nite as well as a nite number of targets. We mention that Mizuno [25, 27] uses so{called {sequences, which form a special class of our target{sequences. We now de ne a target{following algorithm as an algorithm that generates iterates (x(k); s(k) ) which are close to their corresponding targets v(k). Note that a target{sequence can be predetermined, but also adaptively constructed during the algorithm.

3 The (

{space

x; s)

In this section we will analyze the Newton step for computing a point close to the target point. We will introduce a quantity that measures the closeness of a point to the target, and prove some nice properties of this quantity. We point out that this proximity measure is completely in the spirit of the Roos{Vial measure [33], and the primal{dual measures as discussed in [19]. After the rst version of this paper was written, we found out that it also appears in Mizuno [25, 27].

3.1 The Newton step in the ( ){space x; s

Let (x; s) be a pair of primal{dual interior feasible solutions, and let v be the corresponding point in the v {space. Furthermore, let v be the current target point in the v {space. Our aim is to nd an approximate solution of the following system of equations

Ax = b; = c; xs = v2:

AT y + s

We apply Newton's method to this system, and obtain the following relations for the corresponding displacements in the x-, y { and s{ spaces:

A
x = 0; = 0; x
s + s
x = v2 ? v2:

AT
y +
s

These are the search directions which will be used. It is not dicult to obtain explicit expressions for the search direction vectors
x;
y and
s. For the analysis below it will be convenient to use a dierent formulation. To this end we introduce the vector

p d := xs?1: 6

Using d we can rescale both x and s to the same vector, namely v : d?1x = ds = v: We also use d to rescale
x and
s: px := d?1
x; ps := d
s: Note that the orthogonality of
x and
s implies that px and ps are also orthogonal. Now we may write x
s + s
x = xd?1d
s + sdd?1
x = v(px + ps): Hence, Newton's direction is determined by the following linear system: ADpx = 0 T DA py + ps = 0   px + ps = v?1 v2 ? v2 : Denoting we have





pv := v?1 v2 ? v2 ; px + ps = pv ;

and px and ps are simply the orthogonal decomposition of pv in the nullspace of AD and the row space of AD respectively. We mention here that this is the last time that the data A; b; c explicitly appear in this paper, and that the data only comes in via an initial starting point and via the number L (the number of bits needed to represent the data). This has the great advantage that we work completely in the v {space from now on. A disadvantage is that we are not able to explore e.g. special structures in the data. We also use the vector qv , de ned by

qv := px ? ps: Note that the orthogonality of px and ps implies that kqv k = kpv k : We also have px = 12 (pv + qv ); ps = 12 (pv ? qv ); whence pxps = 14 (p2v ? qv2): (6) The product px ps plays an important role in the analysis. It represents the second order eect in the Newton step, which needs to be small to prove eciency of Newton's method. For that purpose we relate the euclidean and the in nity norms of this product to the norm of pv as follows (a similar lemma for the case where v is on the central path is proved by Mizuno et al. [28]). 7

Lemma 3.1 One has kpxps k1  14 kpv k2 and kpxpsk  2p1 2 kpv k2. Proof: Using (6) we may write

    kpxps k1  14 max kpv k21 ; kqv k21  14 max kpv k2 ; kqv k2 = 14 kpv k2 :

Using (6) once more we obtain

2

kpxpsk2 = eT (pxps)2 = 161 eT (p2v ? qv2)2 = 161

p2v ? qv2





2



2   1  16 p2v + qv2  161 kpv k4 + kqv k4 = 81 kpv k4 : 2

This proves the lemma.

3.2 Proximity to the target point In the analysis of target{following algorithms we will need a measure for the proximity of the current iterate v to the current target point v. For this purpose we introduce the following proximity measure: (v; v) := 2 min1 (v) kpv k :

Note that this measure is not symmetric in the iterate v and the target v. The same measure was used by Mizuno [25, 27] to measure the distance between two successive targets, but he uses a dierent proximity measure for the proximity of a primal{dual pair to its target in the v{space. De ning u := v?1v; (7) the measure can be rewritten as



   (v; v) = 2 min1 (v)

v?1 v2 ? v2

= 2 min1 (v)

v u ? u?1

: (8)

Let us indicate that if v2 = e for some positive  then this amounts to

(v; v) = 21

u ? u?1

; which, up the factor 12 , is equal to the proximity measure used in [19]. The next lemma relates the proximity measure to the ratio vi =vi and shows that componentwise the elements of v cannot dier too much from the elements of v.

Lemma 3.2 Let  := (v; v) and u as de ned in (7). Then one has 1 ()  ui  (); i = 1; : : :n;

with

p

() :=  + 1 + 2: 8

Proof: From we get,

1

v u ? u?1 

 1 min(v)

u ? u?1

= 1

u ? u?1

 = 2 min (v) 2 min(v ) 2

So, for each i; 1  i  n;

1

u ? u?1

 :

2

?2  u?i 1 ? ui  2;

Since ui is positive, this is equivalent to

?2ui  1 ? u2i  2ui; or

u2i ? 2ui ? 1  0  u2i + 2ui ? 1:

One easily veri es that this is equivalent to

()?1  ui  (): 2

This proves the lemma.

3.3 Analysis of the Newton step 3.3.1 Feasibility We proceed by investigating when the (full) Newton step to the target point v can be made without becoming infeasible. So, we want to know under which conditions the new iterates x := x +
x and s := s +
s are positive. The next lemma gives a simple condition on (v; v) which guarantees that the property is met after a Newton step.

Lemma 3.3 The Newton step is feasible if

pvxps

1 < 1. This condition is satis ed if 2

(v; v) < 1:

Proof: Let 0    1 be a step length along the Newton direction. We de ne x() = x + 
x and s() = s + 
s. The new iterate is x := x(1) and s := s(1). Then we have x()s() = (v + px)(v + ps) = v2 + v(px + ps ) + 2pxps  p x ps 2 2 2 2 2 2 = v + (v ? v ) +  px ps = v (1 ? ) + v e +  v2 : 9

(9)



We obtain that x()s() > 0 if

pxvps

1 < 1 and   1, which proves the rst statement. The condition on  follows from the observations 2

p p

kp p k 2

x2 s

 x s 12  kpv k 2 = 2: v 1 min(v) 4 min(v)

where the last inequality follows from Lemma 3.1.

2

Letting  = 1 in (9) and denoting (v )2 = x s we get the useful relation (v )2 = v2 + pxps :

(10)

3.3.2 Eect on the proximity measure We are interested in how close a Newton step brings us to the target point v. In this respect the following lemma is of interest. It shows that if the current iterate v is close enough to the target point v, the Newton step ensures quadratic convergence of the proximity measure.

Lemma 3.4 Assume that  := (v; v) < 1 and let v result from a Newton step at v with respect

to v. Then one has

4 (v; v)2  2 (1? 2) :

Proof: From Lemma 3.3 we know that x and s are feasible. For the calculation of   we need v . From (10) we have (v )2 = v2 + pxps : Consequently, using Lemma 3.1, we get min(v )2  min(v)2 ? kpxps k1  min(v)2 ? 1=4 kpv k2 = min(v)2 (1 ?  2):

Using these two relations and (8) we may write

  2 (v; v)2 = 4 min1(v)2

(v)?1 v2 ? (v)2

2

= 4 min1(v )2

(v )?1 (pxps )

(px ps )k2 :  4 min1(v)2 kmin (v )2

Substitution of the bounds derived in Lemma 3.1 and (11) yields

(v; v)2 

kpv k4 : 32 min(v)2 min(v)2(1 ?  2 ) 1

10

(11)

Performing the substitution

kpv k = 2 min(v);

yields,

4 (v; v)2  2 (1? 2) ;

which proves the lemma.

2

p

One easily checks that for  < 2p=3 it holds  (v ; v) <  , implying convergence of the sequence of Newton steps, while for  < 1= 2 holds  (v ; v) <  2, guaranteeing quadratic convergence.

3.3.3 Eect on the duality gap The Newton step has another important consequence, namely that the duality gap after the step has the same value as the gap in the target point point v. Formally we have:

Lemma 3.5 Let the primal{dual feasible pair (x; s) be obtained after a full Newton step with

respect to v. Then the corresponding duality gap achieves its target value, namely (x)T s = kvk2.

Proof: We recall from (10) (v )2 = v2 + px ps . Hence, using that px and ps are orthogonal, we may write (x)T s = eT (v )2 = eT v2 + pTx ps = eT v2 = kvk2 :

2 This lemma has two important implications. The rst is that if subsequent Newton steps would be taken, with v xed, then the duality gap would remain constant! Furthermore, since in so{ called short{step methods one full Newton step is taken with respect to each target, the lemma implies that we do not have to bother about the duality gap in the iterates themselves, but that it suces to consider the duality gap in the targets.

3.3.4 Updating the target To conclude this section we will analyze the eect on the proximity measure of a Newton step followed by an update in the target. Although it seems more natural to analyze the eect of an update in the target followed by a Newton step with respect to the new target, for the analysis both alternatives are equivalent. We will do the analysis in a very general setting, such that in the next sections it will be an easy task to apply this theorem and derive polynomial complexity bounds for various applications. 11

Theorem 3.1 Let v and v be such that  := (v; v) < 1=2. Let v be obtained from v by a full Newton step with respect to v and let v be arbitrary. Then p min(v) :   (v ; v )  26 (v; v) + p1 min (v ) 2 6

Proof: First, from Lemma 3.3 it follows that v  is well{de ned. By de nition we have

 2  2

1   (v ; v ) = 2 min(v)

(v ) v? (v )

: Recall from (10) that (v )2 = v2 + px ps and from (11) that min(v )2  min(v )2(1 ?  2 ): Using these and Lemmas 3.1 and 3.2 gives

 2 2

p p

( v ) ? v  v  1 1

  (v ; v )  2 min(v)

v v

+ 2 min(v)

vx s



 (v; v)

vv

+ 2 min(v1) min(v) p1 kpv k2 2 2 1 2 min ( v )  (v; v)((v; v)) + p 2 2 min(v  ) min(v ) min(v) p  2  (v; v)((v; v)) + min (v ) 2(1 ?  2 ) ;

(12)

where the last inequality follows from (12). Finally, from Lemma 3.4 we obtain

2 (v; v)  p  2 : 2 (1 ?  ) p p p Substituting   1=2 yields  2 = 2(1 ?  2 )  1=(2 6) and ( (v ; v))  6=2. This gives the bound. 2

In the next section we will apply this lemma in the following way. Given v close to v such that (v; v) < 1=2, we need to determine how far v can be pushed away from v such that v will be in the region of quadratic convergence around v , in other words, such that  (v ; v) < 1=2. The lemma implies that the order of the update is completely determined by the proximity  (v ; v ) between the targets and by the ratio min(v )=min(v ).

4 An algorithm using Dikin steps In this section we will perform the convergence analysis for a new method which follows targets determined by so{called Dikin steps, introduced by Jansen et al. [17]. The idea of the algorithm 12

is to simulate the behavior of the new primal{dual ane method [17] by taking Dikin steps in the v{space and tracing the resulting targets with Newton steps. As explained in the Introduction, the method has the property that centering and striving for optimality is done at the same time, hence combines the centrality and optimality phases.

4.1 The Dikin step in the {space v

We rst derive some interesting properties of the Dikin step in the v {space.

4.1.1 Feasibility and step size Recall from the Introduction that the Dikin step
v at v is determined by the problem

min f vT
v :

v?1
v

 1 g:

One easily veri es that
v is uniquely determined and given by
v = ?v3=kv2k. Hence, we de ne for the new target in the v {space

!

v2 ; e ?  kv k kv2k where  is some positive number. Since we require v to be positive it is well de ned only if

v2

 < max := max(v)2 : De ning the step size  by  :=   ; max we have 0 <  < 1 and ! 3 2 v  v  v := v ?  max(v)2 = v e ?  max(v)2 : 3 v := v ?  v2 = v

Note that each element of v is smaller than the corresponding element of v. This property is important, since, as we saw in Lemma 3.5 the Newton process in the (x; s){space forces equality between the duality gap and eT (v )2 . So the duality gap will be decreasing. In fact we have the obvious result 2!   min ( v )  kvk (1 ? )  kv k  kvk 1 ?  max(v)2 = kvk 1 ? ! 2 ; (13) where ! is de ned by (5).

13

4.1.2 Proximity to the central path The Dikin step has some further interesting properties. With   13 , rst, it preserves the ranking of the coordinates of v, second, it is monotonic in the ratio ! . These results are summarized in the next lemmas.

Lemma 4.1 Assume that and that   13 : Then

0 < v1  v2 


 vn ; 0 < v1  v2 


 vn :

Proof: Let i < j . We have

  vj ? vi = vj ? vi ? v2 vj3 ? vi3 n    2 2 = (vj ? vi ) 1 ? v2 vj + vi vj + vi n  (vj ? vi) (1 ? 3)  0:

Thus it follows that vj  vi , with equality if vj = vi . This proves the lemma.

2

Remark 4.1 An alternative proof of Lemma 4.1 can be given using the function f (t) = t(1 ? t2 )=(1 ? ). Assuming w.l.o.g. that vn = 1, then vi = f (vi) is the value after the Dikin step,

where the maximal component of v is rescaled to 1. This function is monotonically increasing and concave for   1=3.

In the sequel we shall use   1=3, henceforth we may assume that the coordinates of v are ranked as in Lemma 4.1. So v1 is the smallest and vn the largest element of v. Consequently, ! = v1=vn .

Lemma 4.2 Assume that   31 and let !  := !(v): Then 2! 1 ?  !   ! = 1 ?  !  ! ; and   1 ? !   1 ? ! (1 ? ! ) : 1? Proof: 14

(14) (15)

Since   13 , Lemma 4.1 implies that !  = v1 =vn : Hence, by using the de nition of v1 and vn we get  v1 1 ? ! 2 2! v  1 ?  !   1 ! = v = v 1 ?  = 1 ?  !  ! : n

n

For the second inquality, note that 3 2 1 ? !  = 1 ? 1 1??! ! = 1 ?  1??! + ! 2) ! 2)  ( ! + !  1 ?  (1 + !  + !  (1 ? ! ) = 1 ? 1 ?  (1 ? ! ) = 1?

 This proves the lemma.



 1 ? 1?!  (1 ? ! ):

2

If we use a value  > 1=3, the ranking of v may not be preserved and the proof of Lemma 4.2 does not go through. However, it is still possible to prove the monotonicity of ! for   1=2. We will not do this here, since this property will not be used in the analysis.

4.2 Algorithm Taking for
x and
s the displacements according to a full Newton step with respect to the target point v, we can now formally state the algorithm as follows.

Algorithm Input

(x(0); s(0) p):(0)the(0)initial pair of interior feasible solutions; (0) v := x s .

Parameters

" is the accuracy parameter;  is the step size (default value

begin

! (0) 6pn ).

x := x(0); s := s(0); v := pxs; while xT s > " do v := v ?  vvn ; x := x +
x; s := s +
s; 3 2

end end.

15

4.3 Analysis From Section 3 it is clear that the only thing remaining to analyze a target{following method, is to guarantee that a suciently large step{size in the v {space can be taken. For this, we should check for which value of  the conditions of Theorem 3.1 hold. So we need to investigate the eect on the proximity measure of a Newton step followed by an update of the target point, in this case, of a Dikin step. Let v be the new target point in the v {space, obtained from a Dikin step with step size   13 at v. So

!

2 3 v := v ?  vv2 = v e ?  vv2 : n n

Lemma 4.3 Let v result from a Dikin step with step size   13 at v. Then v1  1 and (v; v)  1 pn : v1 1 ?  1 ?  ! Proof:

The rst bound follows from

v1 = v1(1 ? ! 2)  v1(1 ? ):

(16)

By de nition and Lemma 4.1 we have

  (v; v) = 21v

v?1 (v)2 ? v2

: 1

(17)

Since v < v, we have v + v < 2v  2vn e. So, also using the de nition of v , we get



?1   2 2

v (v ) ? v =

v?1(v + v)(v ? v)

3

 2vn

v?1 vv2

pn: n  2v n

Using (16) we get This proves the lemma.

(v; v) 

pn = 1  p n 2(1 ? )v1 2vn 1 ?  ! : 1

p

2

Assuming that  (v; v) < 1=2, it easy to check that  = ! =(6 n) gives  (v ; v) < 1=2 by Theorem 3.1. Since, p ! increases during the course of the algorithm (see Lemma 4.2), the default value  = ! (0)=(6 n) guarantees that one Newton step per target update is sucient. We proceed by considering the reduction of the duality gap in the algorithm. Recall from Lemma 3.5 that after a full Newton step the duality gap attains its target value. So we only need to consider successive target values eT v2 . Using this, we prove the following theorem. 16

Theorem 4.1 Assume that the step size  has its default value 6!pn . Let p (0) (0) (0) (0) (0) (0)

v := x s

Then, after at most

and ! := ! (v ):

pn (x(0))T s(0) ! O (!(0))3 ln "

iterations the algorithm stops, and we obtain a primal{dual interior feasible pair (x; s) that satis es

xT s  ":

Proof:

At the start of the algorithm the duality gap is given by

2

(x(0))T s(0) =

v(0)

: If, as before, the target point at the beginning of some iteration is denoted as v and the end of the same iteration as v , then we have, by (13)

  kvk  kvk 1 ? ! 2 :

Hence, at the iteration under consideration the duality gap is reduced by at least the factor



2

1 ? ! 2 :

Since !  ! (0) by (14), this factor is smaller than



2

1 ? (! (0))2 :

Substitution of the default value for , we nd that at each iteration the duality gap is reduced by at least the factor (0))3 !2 ( ! 1 ? 6pn : From this the theorem easily follows. 2

p

From the theorem we see that the target{following algorithm runs in O( nL) iterations whenever ! (0) = (1).

4.4 Improving the complexity bound Unfortunately, when ! (0) is smaller than (1), the complexitypbound of this target{following algorithm is highly aected. For instance, when ! (0) = (1= n) we obtain only an O(n2 L) algorithm. 17

However, the rather straightforward analysis of the previous section can be improved signi cantly to yield a bound of (0))T s(0) !! pn 1 1 ( x O  ! (0) ln ! (0) + ln iterations. The fact is that in our previous analysis we bounded ! by its initial value ! 0, without considering that it is increasing at each step. Actually, ! will reach a value close to 1 in a limited number of steps. From that point on we can use in the analysis this new value to bound ! from below. The rst goal is thus to bound the number of iterations to have ! close to 1.

Lemma 4.4 Let   1=3. After at most

  O 1 ln !1(0) updates of the targets using Dikin steps we have ! 2  1=2.

Proof:

Using (14) we have for ! 2  1=2 1 ? ! 2  1 ? =2 = 1 + =2 : 1? 1? 1? So ! 2  1=2 will certainly hold if



2k = 2 1 + 1 ?  (! (0))2  1=2;

or equivalently, if

  1=2  = 2 2k ln 1 + 1 ?   ln (0) 2 : (! ) 

Using ln(1 + t) > t=2 for t < 1, this will certainly be satis ed if  1=2  = 2 k 1 ?   ln (!(0))2 : Hence we nd that the number of iterations required is at most 2(1 ? ) ln  1  ;  2(! (0))2 which is of the order speci ed in the lemma.

2

By Lemma 4.1 we know that  = 6!pn is an acceptable choice. Thus we reach a point with  p !  1=2 in O ! n ln 1=! 0 iterations; in that process v and hence eT v2 decreases. Then on, p p we can use  = 1=(12 n) and we need O( n ln((x0)T s0 )=) more iterations to terminate. We proved the following theorem. 0

0

18

Theorem 4.2 The algorithm tracing the targets determined by Dikin steps requires at most (0))T s(0) !! p 1 ( x 1 ln + ln O n 

! (0) ! (0)

iterations to obtain an {approximate solution.

So we have saved a factor 1=(! (0))2 in the complexity.

4.5 Centering The centering property, that is the fact that the iterates get close to the central path, is an important property in all interior point methods. Let us de ne 'close to the central path' by requiring that the iterate is in the region of quadratic convergence of some point on the central path. We can relate this 'closeness' to the value of ! as follows. n , then there exists a target{point v on the central path such that Lemma 4.5 If !p:= !(v)  n+1

 := (v; v) < 1= 2.

Proof:

Let v2 = e, then  reduces to



1

pv ?1 ? v

: p 2

It is easy to show that this measure is minimal for  = kv k = v ?1 with value q p1 kvk kv?1k ? n: 2

p

Hence we will have   1= 2 if

kvk kv?1k ? n  1:

p p Using the bounds kv k  n max(v ) and v ?1  n=min(v ), this implies that it suces to have 1  n + 1; ! n which implies the lemma. 2 The next lemma estimates the number of updates needed to reach a target with !  n=(n + 1).

Lemma 4.6 After at most iterations we have !  n=(n + 1).

  1 O ! (0) ln(n + 1)

19

Proof:

From equation (15) we need k to satisfy (1 ? ! k ) 

(0) 1 ? 1!? 

!k

(1 ? ! (0))  n +1 1 :

Taking logarithms and using ln(1 ? t)  ?t for t < 1 we obtain that k should satisfy k  1!?(0) ln((n + 1)(1 ? ! (0))); which gives the order in the lemma.

2

4.6 General{order scaling Instead of Dikin steps, we can let the steps be de ned by r{order scaling in the following sense

v = v

!

2r e ?  vv2r : n

(18)

In this light, the Dikin step has r = 1 and weighted path{following has r = 0. Again it is easy to analyze the resulting algorithms, which can be viewed as the family of target{following algorithms simulating the family of primal{dual ane scaling algorithms analyzed in Jansen et al. [16]. First it is easy to see that kvk  kvk(1 ? ! 2r ): It is left to the reader to verify the following lemmas.

Lemma 4.7 If  < 2r1+1 then the ranking in v is preserved and ! is monotonically increasing. Lemma 4.8 Let v result from v by a target{update using (18) with step size  < 1=(2r + 1). Then pn 1  v1  p 1  and  (v ; v )  p : v1 1? 1 ?  ! (0) Assuming r = O(1), we nd that the algorithm using r{order scaling for the target update requires ! pn 1 O (!(0))2r+1 ln  iterations to obtain an {approximate solution. In a similar way as in Lemma 4.4 and Theorem 4.2 we can improve the convergence analysis for r = O(1) and improve the complexity bound to

pn 1 ! O ! (0) ln  : 20

5 Further applications of the target{following concept In this section we apply the general ingredients from Section 3 to various primal{dual algorithms in the literature. The reader should recall that the only missing element to complete the convergence analysis of a target{following method, is to determine the stepsizes that can be taken. In all applications to follow the maximal stepsize is obtained from the condition that after a Newton step the iterate should be close to an updated target, in the sense that it belongs to the region of quadratic convergence around the target.

5.1 Path{following methods In this subsection we will perform the convergence analysis for the standard primal{dual logarithmic barrier method which follows targets on the central path determined by k = (1 ? )k?1 ; v(k) = pk e: Of course, this method and its complexity are well established in the literature. The only purpose of this subsection is to elucidate the use of targets with a simple and well{known example. Note that the duality gap in the target v(k) is equal to nk .

p Lemma 5.1 Let v = p e; using the target{update v = 1 ? v, we have min(v) p 1 1 pn: = and  (v ; v) = p  min(v ) 1? 2 1? Proof: The rst statement is trivial. The second follows from



(1 ?  ) e ? e 1

 (v; v ) = p

pe

2 (1 ? ) = p 1 kek 2 1? p = p 1  n: 2 1?

p

2

We can combine Lemma 5.1 with Theorem 3.1 which gives that  (v ; v) < 1p=2 for  = 1=(3 n). Since kvk2 = (1?) kvk2 , we get by Lemma 3.5 the well{known bound of O( n ln 1=) iterations for the short step logarithmic barrier method. In a similar way we can analyze an algorithm following a weighted path. The advantage of such an algorithm is that it can start at any interior point, not necessarily close to the central path.

p Lemma 5.2 Let v be given and ! = min(v)=max(v); using the target{update v = 1 ? v, we have p min(v) p 1 )  p 1  n : and  ( v ; v  = min(v ) 1? 2 1 ?  ! 21

Proof:

The rst statement is trivial. The second follows from

2 ? v2

(1 ?  ) v 1

 (v; v ) = p

v 2 1 ?  min(v ) kvk = p 1 2 1 ?  min(v ) pn  1  2p1 ?  ! :

2

As is clear from this lemma, in the maximal step size wep have to take into account ! . So the number of iterations required for the algorithm will be O( !n ln 1=) which is by a constant factor worse than the number of iterations for the central path following algorithm. The bound is in accordance with what was obtained in [4, 32, 14]. In the next application we shall analyze an alternative algorithm which is better complexity{ wise: instead of following the weighted path the method consists rst in centering and then using the standard (central) path{following algortihm.

5.2 Ecient centering The second application of the target{following concept is the problem of ecient centering as considered by Den Hertog [13] and Mizuno [25]. The problem is stated as follows: given an arbitrary interior feasible point (x; s) compute a point close to the central path. In this section we give an easy analysis of the algorithm, independently proposed by Den Hertog and Mizuno. The idea of the algorithm is to successively increase the smaller elements of the v {vector until they become equal to the largest element. More speci cally, let (v (0))2 = xs be given; update v to obtain v as follows: p vi = max(vi; 1 +  min(v)); i = 1; : : :; n; (19) if min(v ) > max(v ), then we set v = max(v)e which is on the central path. The goal of the algorithm is to obtain a vector which is a multiple of the all{one vector. Since  max(v) 2 1  max(v) 2  1 +  min(v) ; min(v ) or equivalently, (! )2  (1 + )! 2 ; it follows that this will require at most O( 1 ln 1=! (0)) iterations. The appropriate value of  is determined by the following lemma.

Lemma 5.3 Let v be given; using the target{update (19) we have min(v)  1 and  (v; v)  1 p n: min(v )

2

22

Proof:

If we are not at the last iteration, then from (19) we have for all i p vi  1 +  min(v)  min(v); when v = max(v)e at the last iteration we have vi  min(v); hence the rst bound. Let J be the set of indices for which vi is increased. Then we have vi = vi for i 62 J and (vi)2 ? vi2   min(v)2 for i 2 J: Consequently,

 2 2

1  (v; v ) = 2 min(v)

(v ) v? v



2 eJ

1  min ( v )

 2 min(v)

v

p  12  n: where eJ is the 0{1 characteristic vector of indices in J . 2

p

Combining this result with Theorem 3.1 gives that we can take  = 1=(3 n) to have  (v ; v ) < p 1=2. So we obtain that the algorithm needs at most O( n ln 1=! (0)) iterations. If we combine the above scheme with a the standard primal{dual path{following algorithm we obtain a global algorithm for the LP{problem needing at most (0))2 !! p n max ( v 1 (20) O n ln ! (0) + ln  iterations, starting from any interior feasible point. This is done by rst centering, and then working to optimality. Note that in the centering phase the duality gap in subsequent target points increases, but is bounded by n max(v (0))2. It is interesting to consider the seemingly equivalent scheme of moving the larger components of v downwards. One can check that the analysis does not yield as good a bound as before. Due to the asymmetry of the proximity measure, there is a factor ! that appears in the bound on  (v; v). It is also clear that if we combine the ecient centering scheme with pa standard path{following algorithm, we can reach the target (min(v ))e with a complexity in n. So the observed asymmetry is not intrinsic to the problem.

5.3 Computing a weighted center In this application we discuss two algorithms to nd an approximate solution to the KKT{system Ax = b T A y+s = c xs = w2; 23

where w is a prespeci ed weight{vector. Approximate means that we will compute a feasible pair (x; s), such that (v; w)  1=2; p where v = xs as usual. We make the assumption that a point on or close to the central path is available. Note that we might use the centering algorithm of the previous subsection to nd such a point. This problem is closely related to the one considered by Atkinson and Vaidya [2], Freund [6] and Gon and Vial [8], namely to obtain the weighted analytic center of a polytope. If one sets b = 0, then y solving the system will be the weighted center in the dual space; when c = 0, then x will be the weighted center in the primal space. We will rst analyze an algorithm proposed by Mizuno [27], which is somehow the dual of the algorithm for nding a center as discussed in the previous subsection. Then we give a simpli ed analysis of the algorithm proposed by Atkinson and Vaidya and we extend their algorithm to the case where both the primal and dual weighted centers are computed simultaneously.

5.3.1 Mizuno's algorithm Assume that we start close to the center e, with  = max(w2). The aim is to get close to the weighted center w. The rst target point is set to v = max(w)e. We will now gradually decrease the elements of our vector v untill they all reach the correct value wi. This will be performed by updating the target as follows: p vi = max(wi; 1 ?  vi ): (21) Each component vi is decreased until it reaches its nal value wi .

Lemma 5.4 Let v be obtained from v from an update of the target using (21). Then min(v) p 1 )  p 1 pn:  and  ( v ; v  min(v ) 1? 2 1? Proof: p The rst bound is trivial. The components of v that are decreased by a factor 1 ?  have not yet achieved their nal value wi . Since they all start with the same value, they have all been reduced by the same cumulated factor and thus p vi = 1 ? vi =) vi = min(v): So we have for all i that j(vi)2 ? vi2 j   min(v )2. Hence

 2 2

1  (v; v ) = 2 min(v)

(v ) v? v



2 e

 min ( v ) 1

 p1 ?  min(v)

v

2 p  2p11?   n: 24

p

2

Using Theorem 3.1 gives that  (v ; v) < 1=2 holds for  = 1=(3 n). The number of iterations to be performed is determined by the condition (1 ? )k max(w)2  min(w)2;

!

which means that

(w)2 : k  2 ln max min(w)2 So pwe obtain that the number of Newton steps to compute the weighted center is at most O( n ln 1=!(w)).

5.3.2 Atkinson and Vaidya's algorithm The approach from the previous subsection is dierent from the one proposed by Atkinson and Vaidya [2]. Assuming that w2  e and w2 integral, they suggest to start with a weight vector v(0) = e, and to successively increase the weights by the use of a scaling technique a la Edmonds and Karp [5]. The basic idea is to recursively solve the given problem with all weights wi2 replaced by the maximum of 1 and b w2i c. Let p = blog2 max(w2)c. Then any of wi2 can be written in binary notation as wi2 = bi bi : : :bip ; where bij 2 f0; 1g for all i; j . Elements of the weight{vector w2 which do not need p digits for their binary description, start by convention with a string of zeroes. Now, at iteration k the target point is given by (vi(k))2 = bi bi : : :bik ; where we set vi(k) = 1 if bi bi : : :bik = 0. Note that an update of the target to get v(k) from v(k?1) amounts to doubling the target (i.e. adding a zero to the binary description) and possibly adding 1 (if bik = 1) or substracting 1 (if bi bi : : :bik = 0). From now on, we denote for ease of notation v := v(k?1) and v := v(k). Then the technique boils down to a scheme that updates vi in one of the following ways. 2

0

1

0

0

1

1

0

1

8 > 2vi2 ? 1 if i 2 I1 = f i : bi bi : : :bik? = 0 g > < (vi)2 = > 2vi2 if i 2 I2 = f i : bi bi : : :bik? 6= 0 and bik = 0 g > : 2vi2 + 1 if i 2 I3 = f i : bi bi : : :bik? 6= 0 and bik = 1 g:

Observe that

0

1

1

0

1

1

0

1

1

i 2 I1 =) vi = vi = 1:

The number of updates is determined by the condition 2k  max(w)2;

which implies that there will be blog2 max(w2)c + 1 updates of the target point. 25

(22)

(23)

In [2] a pure dual algorithm is used which means that doubling all weights does not change the position of the dual weighted center. Hence, the only Newton steps needed are to get from 2v 2 to (v )2, which are quite close to each other. Let (x; s) and v be given such that  (v; v) < 1=2. Since b = 0, by setting (v + )2 = 2v 2; x+ = 2x; s+ = s; we have a feasible pair (x+ ; s+ ) for which

(v+; v+) = (v; v) < 1=2: So in the analysis we just have to consider the target{sequence that leads from v+ to v. For this purpose we use the following scheme. Let j = 0 and (~v (0))2 = (v + )2 = 2v 2. De ne

8 >  > < vipn if i 2 I1 i = > 0 if i 2 I2 > : ? v pn if i 2 I3; i

where  > 0 is a certain constant. Update v~(j ) for j  1 in the following way: (~vi(j ))2 = (1 ? i )(~vi(j ?1))2:

(24)

Of course, we do not overshoot the target{value. The conditions

 )j (2v2)  2v2 ? 1 (1 ? v p i i n

i 2 I1

i

(1 + v pn )j (2vi2)  2vi2 + 1

i 2 I3

i

determine the number of updates to be performed. For i 2 I1 , it suces to have

!

p v  vi2 : i n j   ln 2v2 2 i ?1

Since from (23) we have vi = 1 it follows that at most

pn  ln 2

iterations are needed for i 2 I1. For i 2 I3 we have the following: j satisfying pn  1 + 21v2 1 + vj i i p

suces. This leads to the condition that j  2nvi suces; using pthe fact that vi  1, this proves that the number of updates to be performed is not larger than n=2. We need to show now that the speci c choice of the update guarantees that one Newton step suces per update. 26

Lemma 5.5 De ne  := =pn. Let v~(j) be obtained from v~(j?1) from an update of the target

using (24). Then

min(~v (j ?1)) p 1  1? min(~v (j ))

3 : and  (~v (j ?1); v~(j ))  p 2 1?

Proof:

For ease of notation, let v~ = v~(j ?1) and v~+ = v~(j ). It is easy to see that min(~v + )2  (1 ? )min(~v)2 :

Hence we have

0

! 11=2

(~v+)2 ? v~2

X v~i2(1 ? i ) ? v~i2 2A 1 1 + @ (~v; v~ ) = 2 min(~v+)

v~

 p v~i 2 1 ?  min(~v ) i2I [I

0 0 11=2 11=2   2 X X @ @ pn vv~i A = p 1 (i v~i )2 A = p 1 2 1 ?  min(~v) i2I [I 2 1 ?  min(~v ) i2I [I i 1 0 0 q 2 12 1=2 X p 1 @ 2vi + 1 A CA  p 1  3 n  2p1 ?  min(~v)  B @ vi 2 1 ?  min(~v ) 1

1

p

1

3

1

3

3

i2I1 [I3

3: 2 1 ?  min(~v) Since min(~v)  1, the lemma follows. =

2

Using Theorem 3.1, from the lemma it follows that for  = 1=7 we can get close to v~(j ) from a point close to v~(j ?1) in one full Newton step. So the entire algorithm performs at most

O(pn log2 max(w))

(25)

Newton steps, and for this pure dual algorithm we get the same complexity as in [2] using a much simpler analysis. We will now analyze the same strategy for the problem of nding the primal and dual weighted centers simultaneously. The number of Newton steps needed to get close to a new target is more than one now, since the update of v is big. We use the same de nition (22) for the new target. Again, to nd an iterate in the quadratic convergence region of v , another sequence of targets is constructed by which we reach v from v. The following scheme is used. Let j = 0 and v~(0) = v. De ne 8 >  : ? vipn if i 2 I2 [ I3; where  > 0 is a certain constant. Update v~(j ) for j  1 in the following way: (~vi(j ))2 = (1 ? i )(~vi(j ?1))2: 27

(26)

Note that the proof of Lemma 5.5 is easily adapted for this sequence, and that the result remains the same. Using the condition (~vi(j ))2  (vi)2 or (1 + v pn )j vi2  2vi2 i

 )j v2  2v2 + 1 (1 + v p i n i i

i 2 I2 i 2 I3

and using the fact that vi  1,pit is easy to see that this implies that the number of updates must be of the order O(max(v) n), so an upperbound expressed in the data is

O(max(w)pn):

The reader should notice here that we have shown that the algorithm has a total complexity of

O(max(w)pn log2 max(w)) Newton steps. This is a factor max(w) worse than the result (25) above and in [2]. This dierence

can be explained by noticing that doubling all weights does not have any eect in a pure primal or dual method, but has quite an eect in a primal{dual method.

6 Concluding remarks In this paper we developed a new interior point method for linear programming that can start from any positive feasible primal{dual pair, and works for optimality and centrality at the same time. The analysis also led to a collection of tools for analyzing various primal{dual interior point methods, which we put in the general framework of target{following methods. This made it possible to analyze in a simple way various existing algorithms in the literature. The algorithms analyzed in this paper can all be classi ed as short step methods. In a subsequent paper [20] we introduce a primal{dual barrier function which enables us to analyze also large step target{ following algorithms.

References [1] I. Adler and R. D. C. Monteiro. A geometric view of parametric linear programming. Algorithmica, 8:161{176, 1992. [2] D. S. Atkinson and P. M. Vaidya. A scaling technique for nding the weighted analytic center of a polytope. Mathematical Programming, 57:163{192, 1992. [3] D. A. Bayer and J. C. Lagarias. The nonlinear geometry of linear programming, Part II : Legendre transform coordinates. Transactions of the American Mathematical Society, 314(2):527{581, 1989. 28

[4] J. Ding and T. Y. Li. An algorithm based on weighted logarithmic barrier functions for linear complementarity problems. Arabian Journal for Science and Engineering, 15(4):679{ 685, 1990. [5] J. Edmonds and R.M. Karp. Theoretical improvements in algorithmic eciency for network

ow problems. Journal of the ACM, 19:248{264, 1972. [6] R. M. Freund. Projective transformations for interior{point algorithms, and a superlinearly convergent algorithm for the w{center problem. Mathematical Programming, 58:385{414, 1993. [7] J. L. Gon and J.-Ph. Vial. Short steps with Karmarkar's projective algorithm for linear programming. Manuscript, Department d'Economie Commerciale et Industrielle, Universite de Geneve, Geneve, Switzerland, 1990. (A version with no weight will appear in SIAM Journal on Optimization, January 1994). [8] J. L. Gon and J.-Ph. Vial. On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm. Mathematical Programming, 60:81{92, 1993. [9] C. C. Gonzaga and R. A. Tapia. On the convergence of the Mizuno{Todd{Ye algorithm to the analytic center of the solution set. Technical Report 92{36, Dept. of Mathematical Sciences, Rice University, Houston, TX 77251, USA, 1992. [10] C. C. Gonzaga and R. A. Tapia. On the quadratic convergence of the simpli ed Mizuno{ Todd{Ye algorithm for linear programming. Technical Report 92{41, Dept. of Mathematical Sciences, Rice University, Houston, TX 77251, USA, June 1992. Revised November 1992. [11] C. C. Gonzaga and R. A. Tapia. A quadratically convergent primal{dual algorithm for linear programming. Technical Report, Dept. of Mathematical Sciences, Rice University, Houston, TX 77251, USA, 1993. [12] H. J. Greenberg. The use of the optimal partition in a linear programming solution for postoptimal analysis. Technical Report, Mathematics Department, University of Colorado, March 1993. [13] D. den Hertog. Interior Point Approach to Linear, Quadratic and Convex Programming, Algorithms and Complexity. PhD thesis, Faculty of Mathematics and Informatics, TU Delft, NL{2628 BL Delft, The Netherlands, September 1992. (To appear at Kluwer Publishers). [14] D. den Hertog, C. Roos, and T. Terlaky. A polynomial method of weighted centers for convex quadratic programming. Journal of Information & Optimization Sciences, 12(2):187{ 205, 1991. [15] B. Jansen, C. Roos, and T. Terlaky. An interior point approach to postoptimal and parametric analysis in linear programming. Technical Report 92{21, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2628 CD Delft, The Netherlands, April 1992. [16] B. Jansen, C. Roos, and T. Terlaky. A family of polynomial ane scaling algorithms for positive semi{de nite linear complementarity problems. Working Paper, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2600 GA Delft, The Netherlands, 1993. 29

[17] B. Jansen, C. Roos, and T. Terlaky. A polynomial primal{dual Dikin{type algorithm for linear programming. Technical Report 93{36, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2600 GA Delft, The Netherlands, May 1993. [18] B. Jansen, C. Roos, and T. Terlaky. The theory of linear programming : Skew symmetric self{dual problems and the central path. Technical Report 93{15, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2600 GA Delft, The Netherlands, 1993. (To appear in Optimization). [19] B. Jansen, C. Roos, T. Terlaky, and J.-Ph. Vial. Primal{dual algorithms for linear programming based on the logarithmic barrier method. Technical Report 92{104, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2600 GA Delft, The Netherlands, 1992. (To appear in JOTA, Vol. 83). [20] B. Jansen, C. Roos, T. Terlaky, and J.-Ph. Vial. Long{step primal{dual target{following algorithms for linear programming. Working Paper, Faculty of Technical Mathematics and Informatics, TU Delft, NL{2600 GA Delft, The Netherlands, 1993. [21] M. Kojima, N. Megiddo, T. Noma, and A. Yoshise. A uni ed approach to interior point algorithms for linear complementarity problems, volume 538 of Lecture Notes in Computer Science. Springer Verlag, Berlin, Germany, 1991. [22] M. Kojima, S. Mizuno, and A. Yoshise. A primal{dual interior point algorithm for linear programming. In N. Megiddo, editor, Progress in Mathematical Programming : Interior Point and Related Methods, pages 29{47. Springer Verlag, New York, 1989. [23] L. McLinden. The analogue of Moreau's proximation theorem, with applications to the nonlinear complementarity problem. Paci c Journal of Mathematics, 88:101{161, 1980. [24] N. Megiddo. Pathways to the optimal set in linear programming. In N. Megiddo, editor, Progress in Mathematical Programming : Interior Point and Related Methods, pages 131{ 158. Springer Verlag, New York, 1989. [25] S. Mizuno. An O(n3 L) algorithm using a sequence for linear complementarity problems. Journal of the Operations Research Society of Japan, 33:66{75, 1990. [26] S. Mizuno. A rank{one updating interior algorithm for linear programming. Arabian Journal for Science and Engineering, 15(4):671{677, 1990. [27] S. Mizuno. A new polynomial time method for a linear complementarity problem. Mathematical Programming, 56:31{43, 1992. [28] S. Mizuno, M. J. Todd, and Y. Ye. On adaptive step primal{dual interior{point algorithms for linear programming. Technical Report 944, School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY 14853{3801, USA, October 1990. (To appear in Mathematics of Operations Research). [29] R. D. C. Monteiro and I. Adler. Interior path following primal{dual algorithms : Part I : Linear programming. Mathematical Programming, 44:27{41, 1989. 30

[30] R. D. C. Monteiro and I. Adler. Interior path following primal{dual algorithms : Part II : Convex quadratic programming. Mathematical Programming, 44:43{66, 1989. [31] J. Renegar. A polynomial{time algorithm, based on Newton's method, for linear programming. Mathematical Programming, 40:59{93, 1988. [32] C. Roos and D. den Hertog. A polynomial method of approximate weighted centers for linear programming. Technical Report 89{13, Faculty of Mathematics and Informatics, TU Delft, NL{2628 BL Delft, The Netherlands, 1989. [33] C. Roos and J.-Ph. Vial. A polynomial method of approximate centers for linear programming. Mathematical Programming, 54:295{305, 1992. p [34] Y. Ye, M. J. Todd, and S. Mizuno. An O( nL){iteration homogeneous and self{dual linear programming algorithm. Technical Report, Dept. of Management Science, University of Iowa, Iowa City, IA 52242, USA, June 1992.

31