A family of polynomial affine scaling algorithms for positive semi-definite linear complementarity problems Report 93-112
B. Jansen C. Roos T. Terlaky
Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics and Informatics Technische Universiteit Delft Delft University of Technology
ISSN 0922-5641
Copyright c 1993 by the Faculty of Technical Mathematics and Informatics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, microfilm, or any other means without permission from the Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands. Copies of these reports may be obtained from the bureau of the Faculty of Technical Mathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone +3115784568. A selection of these reports is available in PostScript form at the Faculty’s anonymous ftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl
DELFT UNIVERSITY OF TECHNOLOGY
REPORT Nr. 93{112 A Family of Polynomial Affine Scaling Algorithms for Positive Semi{Definite Linear Complementarity Problems
B. Jansen, C. Roos, T. Terlaky
ISSN 0922{5641 Reports of the Faculty of Technical Mathematics and Informatics Nr. 93{112 Delft, December 10, 1993 i
B. Jansen, C. Roos and T. Terlaky, Faculty of Technical Mathematics and Informatics, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. e{mail:
[email protected],
[email protected],
[email protected] This work is completed with the support of a research grant from SHELL. The rst author is supported by the Dutch Organization for Scienti c Research (NWO) under grant 611-304-028. The third author is on leave from the Eotvos University, Budapest, and partially supported by OTKA No. 2116.
c 1993 by Faculty of Technical Mathematics and InforCopyright matics, Delft, The Netherlands. No part of this report may be reproduced in any form, by print, photoprint, micro lm or any other means without written permission from Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands.
ii
Contents 1 Introduction
1
2 The search directions
2
3 The algorithm
4
4 General results for r 0
4
5 The proof of polynomial complexity if r > 0
8
6 Polynomial complexity if r = 0
12 Abstract
In this paper the new polynomial ane scaling algorithm of Jansen, Roos and Terlaky for LP is extended to PSD linear complementarity problems. The algorithm is immediately further generalized to allow higher order scaling. These algorithms are also new for the LP case. The analysis is based on Ling's proof for the LP case, hence allows an arbitrary interior feasible pair to start with. With the scaling of (x )T s n Jansen et al. the complexity of the algorithm is O( (1? ) ln ), where 2 is a uniform bound for the ratio of the smallest and largest coordinate of the iterates in the primal{dual space. Finally we show that Monteiro, Adler and Resende's polynomial complexity result for the classical primal{dual ane scaling algorithm can easily be derived from our analysis. In addition our result is valid for arbitrary not necessarily centered, initial points. (0)
2
(0)
2
Key words: interior{point method, ane scaling method, linear complementarity
problem.
iii
1 Introduction In this paper we consider the positive semide nite (PSD) linear complementarity problem (LCP) as follows: ? Mx + s = q; x 0; s 0; xs = 0: (1) where M is a given n n real PSD matrix, q 2 IRn and xs denotes the componentwise product of the vectors x and s. The set of feasible and positive feasible vectors is denoted respectively by
F = f (x; s) j ? Mx + s = q; x 0; s 0 g; F = f (x; s) j ? Mx + s = q; x > 0; s > 0 g: We shall assume throughout that F = 6 ;. 0
0
Scaling is one of the most important techniques in modern polynomial time optimization methods. The rst ane scaling algorithm, of Dikin [2], remained unnoticed for a long time. After Karmarkar [6] initiated the dynamically developing eld of interior point methods (IPMs), ane scaling became one of the basic concept in IPMs. Primal or dual ane scaling methods were studied by e.g., Barnes [1], Vanderbei et al. [13], Tsuchiya [12] and Saigal [11]. A primal{ dual ane scaling algorithm for linear programming (LP) was analyzed by Monteiro, Adler and Resende [10]. For a general framework of IPMs for LCP see [7]. Recently, the authors proposed a new primal{dual ane scaling method for LP [5] . Given a nearly centered primal{dual interior feasible pair, they de ne an ane scaling direction (dx; ds) as the steepest descent in the norm induced by Dikin's ellipsoid in the primal{dual space. Provided the step along this direction be small enough, the next iterate is still nearly centered. This is due to the fact that this new ane scaling direction has a centering component; more precisely, it is tangent to a curve that tends to the central path. Having a well centered initial pair, the complexity of the algorithm is proved to be O(nL) iterations (compared with the O(nL2) complexity bound of the classical primal{dual ane scaling algorithm [10]). Ling [8] gave a new analysis of the new ane scaling method, allowing any interior starting point and proving that the complexity of the algorithm is O( n L), where 2 is uniform bound for the ratio of the smallest and largest coordinate of the product x(k) s(k) during the algorithm. 2
The aim of this paper is to generalize the approach in [5] in two ways. First, we consider the LCP instead of the LP problem. Secondly, we analyze a family of ane scaling methods of which the algorithm in [5] is just a special case. It is also shown that the classical ane scaling algorithm can be derived as the limiting case of our family and the use of our analysis provides a new, simple proof for the polynomial complexity (see also Monteiro, Adler and Resende [10] and Mizuno and Nagasawa [9]) of the classical ane scaling algorithm, using an arbitrary initial interior point. Hence as in [5] we apply and further extend Dikin's original scaling approach in the primal{dual (xs){space. It may be recalled from [5] that the resulting search directions are not a linear combination of the classical ane scaling and centering directions, as is usual in the context of IPMs (see [4]).
1
The paper is organized as follows. The generalized Dikin{type search directions are derived and discussed in Section 2. The algorithmic frame is presented in Section 3 and the convergence analysis is presented in Section 4. Finally, in Section 5 we derive the polynomial complexity of the classical primal{dual ane scaling algorithm with suitable step size. Throughout, we shall use kkp (p 2 [1; 1]) to denote the lp norm on IRn , with kk denoting the Euclidean norm kk2 . E will denote the identity matrix, e will be used to denote the vector which has all its components equal to one. Given an n{dimensional vector d, we denote by D the n n diagonal matrix whose diagonal entries are the coordinates dj of d. If x, s 2 IRn then xT s denotes the dot product of the two vectors. Further, xs and x for 2 IR will denote the vector resulting from componentwise operations.
2 The search directions In [5], the new ane scaling direction (x; s) for LP is obtained by minimizing the duality gap over a suitable ellipsoid which is called the primal{dual Dikin ellipsoid. In this paper we generalize this approach to LCPs and we also generalize the scaling by introducing a parameter r > 0 which is called the degree of scaling. We remark that r = 1 will give the algorithm studied in [5] for LP and the classical ane scaling algorithm is obtained with the value r = 0. Let a strictly feasible pair (x; s) 2 F 0 be given. To determine the search direction (x; s) for a xed value of r we consider minimize ((xs)r )T (x?1 x + s?1 s) subject to ?M x + s = 0; (2) kx?1x + s?1 sk 1: This minimization problem has a unique solution, as we now will show. It is convenient to introduce some notations. For each positive primal{dual pair (x; s) de ne v := (xs) 2
1
Hence we have Further let us denote
x = dv px := d?1 x pv := px + ps
and d := (xs?1 ) : 1 2
and s = d?1 v: and and
ps := ds; M := DMD:
These relations imply that xs + sx = xd?1 ps + sdpx = v (px + ps ) = vpv ; hence xT s + sT x = v T pv ;
2
(3)
and
x?1 x + s?1 s = (xs)?1 (xs + sx) = v ?2vpv = v ?1 pv :
Using these notations problem (2) can be reformulated as follows: minimize (v 2r?1)T (px + ps ) subject to ?Mpx + ps = 0; kv?1(px + ps)k 1:
(4)
We can eliminate ps from (4), by using ps = Mpx . Now using (3) pv = px + ps = (E + M )px:
Hence (4) is equivalent to minimize (v 2r?1)T pv subject to kv ?1pv k 1: This is a trivial optimization problem with a unique solution, namely pv = ?
v 2r+1 kv2rk :
(5)
From now on, pv will have the meaning (5). Thus we nd px = (E + M )?1 pv
and ps = M (E + M )?1 pv :
(6)
Rescaling to x and s we thus nd x = D(E + DMD)?1pv ;
s = MD(E + DMD)?1 pv :
(7)
We can now also calculate the optimal value of (2), namely T
r
((xs)r )T (x?1x + s?1 s) = (v 2r?1)T pv = ? ekv v2r k = ?kv 2r k: 4
Note, that (x; s) is the unique solution of the system of equations
?M x + s = 0; sx + xs = ? vkvr r k : 2 +2 2
If we compare this with the classical equation system of the primal{dual ane scaling method, we see that at the right hand side of the second equation we have ? vkvr r k instead of ?v 2 . See, e.g., [7]. So classical ane scaling occurs for r = 0, whereas r = 1 gives the new ane scaling direction proposed in [5]. 2 +2 2
3
3 The algorithm The algorithm is initialized with (x(0); s(0)) 2 F 0 and repeatedly makes steps in the direction (x; s), using a xed step size , until the error in complementarity reaches some prescribed value . For each degree of scaling r > 0 one can de ne an algorithm, which formally is stated as follows.
Algorithm Input
(x0; s0): the initial pair of interior feasible solutions; r > 0: the degree of scaling;
Parameters
" is the accuracy parameter; is the step size;
begin
x := x(0); s := s(0) ; while xT s > " do calculate x and s from (5) and (7); x := x + x; s := s + s;
end end.
4 General results for r 0 Given (x; s), the new iterates will be denoted by x^ = x + x and s^ = s + s respectively, where is the step size. So we have v^2 := x^s^ = xs + (xs + sx) + 2 xs = v 2 + vpv + 2 px ps : As a consequence, the new error in complementarity is given by x^T s^ = eT v^2 = eT v 2 + v T pv + 2 pTx ps = eT v 2 ?
eT v 2r+2 2 T kv2rk + px ps:
(8)
To be able to bound the error in complementarity we need to bound the last two terms in (8). Note that due to the positive semide niteness of matrix M , one has that M is also positive semide nite, hence pTx ps = pTx Mpx 0:
4
From (8) it is clear that for an estimate of the error in complementarity after one iteration, we need an estimate for pTx ps . Later on we also need an estimate for kpx ps k1 . For both purposes the following lemma is useful.
Lemma 4.1 Let px; ps and pv be de ned as in (5) and (6). One has (i) kpv k kvk1 kvk; (ii) 0 xT s = pTx ps kpv k ; (iii) kxsk1 = kpx ps k1 kpv k : 2
4
2
4
Proof:
(i): Since pv = ? vkvr r k , the inequalities kpv k kv k1 kv k are obvious. 2 +1 2
(ii): To prove the other inequalities, we introduce the notation qv = px ? ps . One has kqv k2 = kpxk2 + kpsk2 ? 2pTx ps = kpv k2 ? 4pTx ps kpv k2: Consequently xT s = pTx ps = 41 (kpv k2 ? kqv k2 ) 14 kpv k2 ; which proves (ii). (iii): Using that we write
(9)
1 4
px ps = (p2v ? qv2 );
kpxpsk1 14 max(kpvk1; kqv k1) 14 max(kpv k ; kqv k ) = 41 kpv k : 2
2
2
2
2
The last inequality follows from (9). This completes the proof of the lemma.
2
We introduce some further notations. Since our algorithm can start in any interior feasible point, the complexity will depend on the ratio between the smallest and largest coordinate of v (cf. [8]). This ratio will be denoted by ! (v ). So we de ne min(v ) ; ! (v ) := max(v ) where max(v ) denotes the largest coordinate of v and min(v ) denotes the smallest coordinate of v . If ! (v ) , there are , 2 (0; 1) such that e v 2 e;
with
= 2 :
(10)
A crucial part of the analysis is to give an upper bound for the second term in (8) #(r) := v T pv = ?
eT v 2r+2 kv2rk :
where v is given. Later on we also will give conditions which ensure that ! (v ) will remain bounded during the algorithm.
5
Lemma 4.2 Let v 2 IRn be an arbitrary vector. Depending on the value of r, the following +
bounds hold for #(r).
If 0 r 1; then #(r) ? kpvkn : r? If 1 r and ! (v ) ; then #(r) ? pn kv k2:
(i) (ii)
2
2
Proof:
2
(i): It is obvious that #(0) = ? kpvkn . Hence it is enough to show that the derivative of #(r) is nonpositive as long as 0 r 1. Let us rst di erentiate the nominator and denominator separately. n X (eT v 2r+2)0 = 2 vi2r+2 ln vi : 2
i=1
10 0v Pn r u n n X X u kv rk0 = @t vi rA = 2kv4 rk vi r ln vi = 2 i kv vrik ln vi : 2
4
4
2
i=1
4
=1
2
i=1
The sign of #0(r) is determined by the nominator of the derivative, which is given by
? 2
n X i=1
vi
r
2 +2
kv
r k ln v
2
! n X 2 4r r +1 2 i ? 2r kv k vi kv k ln vi = i=1
n 2 X 4r r+1k2 ? v 2r+2 kv 2r k2 ln v : v k v i i i kv2rk i=1
Now we may write 2
n X i=1
vi r kv r+1k2 ? vi2r+2 kv 2r k2 4
= 2 = =
n X i;j =1
n X
i;j =1 n X i;j =1
ln vi = 2
n n X X
vi4r vj2r+2 ? vi2r+2vj4r ln vi
i=1 j =1
(vi vj )2r+2(vi2r?2 ? vj2r?2) ln vi
(vivj )2r+2 (vi2r?2 ? vj2r?2) ln vi + (vj2r?2 ? vi2r?2 ) ln vj
(vivj )2r+2(vi2r?2 ? vj2r?2) ln vvi : j
The last expression is nonpositive for r 1 and nonnegative for r > 1, hence #(r) is monotone non{increasing if 0 r 1 and monotone non{decreasing if r > 1. Since #(0) = ? kpvkn we have #(r) ? kpvkn if 0 r 1. The rst part of the lemma is proved. 2
2
(ii): Using (10) one has T
r
?#(r) = kev rvkkvk kvk 2
6
2 +2
2
2
r?1 T
r ? kvekkvvk kvk r? pn kvk : 1
2
4
2
2
2
2
2
The last inequality follows from eT v 4 kv2k p1 ; = kv2kkvk2 kvk2 n
2
where the Cauchy{Schwartz inequality is used. The proof is completed.
Remark: Observe that the above lemma is trivial if r = 1. In that case #(1) ? kpvkn is an 2
immediate consequence of the Cauchy{Schwartz inequality. This was the last step in the above proof. Now we can guarantee a decrease in the error of complementarity.
Lemma 4.3 (i) If 0 r 1 and pn then T x^ s^ = kv^k 1 ? p kv k : 2 n 2
?2
(ii) If 1 r and 2pn then 2r
(11)
2
2
x^T s^ = kv^k2
2r ?2 1 ? 2pn
!
kvk :
(12)
2
Proof:
From (8) one has: T
r
kv^k = kvk ? e kvv rk + pTx ps = kvk + #(r) + pTx ps: 2
2
2 +2
2
2
2
(i): Using Lemma 4.1 and Lemma 4.2 we obtain
kv^k 2
2 1 ? pn + 4
!
kvk : 2
Since p2n , it follows that 1 ? p + 1 ? p + p = 1 ? p : n 4 n 2 n 2 n This proves the rst part. 2
7
2
(ii): We use Lemma 4.1 and Lemma 4.2 again
kv^k 2
2r ?2 2 1 ? pn + 4
!
kvk : 2
?2
Since 2pn , it follows that 2r
r?2
r?2
r?2
r?2
1 ? pn + 4 1 ? pn + 2pn = 1 ? 2pn : The lemma is proved. 2
2
2
2
2
2
Lemma 4.3 makes clear that the algorithm will converge if the step size can be bounded away from zero, since this will guarantee a xed reduction of kv k2. If the lower bound for is suciently large, then the algorithm will be polynomial. We proceed with a condition on the step size, that guarantees feasibility of the new iterates. Let us say that the step size is feasible, if the new iterates are positive. Then we may state the following result.
Lemma 4.4 Let 0 , x( ) = x + x, s( ) = s + s and v ( ) = x( )s( ). If is such that v ( ) > 0 for all satisfying 0 , then the step size is feasible. 2
2
Proof:
If satis es the hypothesis of the lemma then x( ) and s( ) cannot vanish for any 2 [0; ]. Hence, by continuity, x( ) and s( ) must be positive for any such . 2
5 The proof of polynomial complexity if r > 0 The next theorem makes clear that, with a suitable step size, the new iterates not only stay feasible, but also that the ratio of the smallest and largest coordinate of v will remain bounded by , i.e., ! (v ) stays valid for all the iterates. The proof goes along the same lines as the proof of Theorem 3 in [8] for the LP case with r = 1.
Theorem 5.1 If (x; s) 2 F , 0 < < 1, r > 0, !(v) and 0 0s 1 p 1 r r 0 min @2 @ 1 + ? p A ; n ; 4 (1 ? p ) A 0
2
2
n
n
then (^x; s^) 2 F 0 and ! (^v).
8
r+1
2
2
(1 + 2) n
(13)
Proof:
The hypothesis of the theorem provides three upper bounds for the step size . As we will see below, the rst upper bound guarantees feasibility of the new iterates, the last guarantees that ! (^v) , both under the premise that the second bound holds. Let and be de ned as in (10). We remind that by (8) v^2 = x^s^ = v 2 ?
v 2r+2 2 kv2rk + pxps:
One easily veri es that the function '(t) = t ?
tr+1 kv2rk
v r k . The second upper bound for is monotonically increasing on the interval [0; ] if (rk+1) r now guarantees the monotonicity of ', because 2
kv rk r kek = r pn : (r + 1) r (r + 1) r r + 1 2
2
Using this monotonicity property together with (10) one has the inequalities
!
!
r+1 r+1 v 2r+2 ? 2r e v 2 ? 2r ? 2r e: kv k kv k kv k
So the minimal and maximal coordinates of v^2 are bounded by min(^v 2) ? k vr r k ? 2 kpxps k1 ; r max(^v 2) ? k v r k + 2 kpxps k1 : +1
2
+1
(14)
2
Now applying Lemma 4.1 and observing that kpv k2 one has kpxpsk1 41 : Hence from (14) and (15)
(15)
min(^v 2) ? k vr r k ? 14 2 ; r max(^v 2) ? k v r k + 41 2 :
(16)
+1
2
+1
2
Lemma 4.4 implies that the new iterates will be feasible if min(^v 2) > 0. After dividing by , this amounts to the following condition on : r
1 ? k ? 0: v 2r k 42 Since
2
r k r ek p = kv2rk n kv 2r k
9
p1n ;
this certainly holds if
1 ? pn ? 42 0: 2
Elementary calculations make clear that this condition is satis ed, due to the rst upper bound on in the theorem. So the new iterates are feasible. Now ! (^v) will certainly hold if
!
1 r+1 1 r+1 1 ? 2r + 2 2 ? 2r ? 2 : kv k 4 kv k 4 On dividing this by = , we see that this is equivalent to 2
r r 2 1 ? 2r + 1 2 1 ? 2r ? 2 : kv k 4 kv k 4
By rearranging one has
1 + 2 4( r ? r ) :
kv rk
2
i.e.,
From (10), the de nition of and ,
2
42( r ? r ) : (1 + 2)kv 2r k
p
(17)
p
r n kv 2r k r n:
Hence So (17) will certainly hold if
r ? r kv2rk
1 ?pn : r
2
42(1 ? p2r ) ; (1 + 2) n which is guaranteed by the third upper bound for in the theorem. Hence it is proved that the new iterate is at least as well centered as the old one. 2
We have that for each 0 < < 1 there exists a 0 < such that all the iterates of the algorithm gives a feasible primal{dual solution for which ! (^v) . Now we are ready to present the complexity of the algorithms. The proof can be done in the usual way.
Theorem 5.2 If > 0 is given, (x
; s(0)) 2 F 0, and satis es the conditions of Lemma 4.3 and Theorem 5.1, then the algorithm stops, with a solution (x; s) for which (x)T s and ! (v ) holds, after at most p
T s(0)
(i) 2 n ln (x ) p (ii) 2 rn? ln (x (0)
(0)
2
2
(0)
iterations if 0 < r 1, iterations if 1 < r.
)T s(0)
10
Proof:
(i) By Lemma 4.3, in each iteration the duality gap reduces by at least the factor 1 ? 2p n :
(18)
So, after k steps the error in complementarity will be less than if
1 ? 2p n
Taking logarithms one obtains
k ln 1 ?
which is certainly true if
k
(x(0))T s(0) :
p ; ln 2 n (x )T s
(0)
(0)
?k 2p n ln (x )T s ; p 2 n ln (x )T s : k (0)
or, equivalently,
(0)
(0)
(0)
(19)
This proves the rst part of the theorem. The proof of part (ii) is analogous.
2
In the following corollaries we will use the notation !02 = ! (x(0)s(0)).
Corollary 5.1 Let us take (x
(0)
; s(0)) such that !0 = p12 holds. ?r
(i) If 0 < r 1 and n 4 then we may choose = 4(13?p2n ) , hence T the complexity of our algorithm is O( 1?n2?r ln (x ) s ). (ii) If r = 1 and n 4 then we may choose = 2p1 n , hence T the complexity of our algorithm is O(n ln (x ) s ). (iii) If 1 < r and n is suciently large then we may choose = 2r 4pn , hence T the complexity of the algorithm is O(22r?2 n ln (x ) s ). (0)
(0)
(0)
(0)
Corollary 5.2 Let us take (x
(0)
(0)
; s(0)) such that !0 holds. If r = 1 and n is suf2 2 ciently large then we may choose = 2 (1p?n ) , hence the complexity of the algorithm is
O(
n
?2 ) ln
2 (1
( (0) )T (0)
x
s
(0)
).
11
6 Polynomial complexity if r = 0 In this section we show that, with suitable step size, the classical primal{dual ane scaling algorithm is polynomial. The obtained complexity bound, O(nL2 ), is the same as obtained by Monteiro, Adler and Resende [10] and by Mizuno and Nagasawa [9]. Our approach enjoys the advantages of these two results. In the case of LP and convex quadratic programming problems our complexity result is the same as in the above mentioned papers. We use a xed step size as it is in Monteiro, Adler and Resende's [10] paper, but we do not use any potential function which determines the actual step-size as presented by Mizuno and Nagasawa [9]. Contrary to the assumptions in [10], as in [9] our analysis allows an arbitrary, not necessarily centered, starting point. So from now on we assume that r = 0. For keeping the discussion simple we assume, as in the previous section, that n 4. It is easily veri ed that Lemma 4.3 and Lemma 4.4 still apply in the present case. Theorem 5.1, however, is not valid if r = 0. In fact, by taking the limit of the bounds in Theorem 5.1 as r tends to zero one obtains that the step size also tends to zero. Below we show that by making a positive step, (i.e., > 0) ! (v ) may well decrease, but the decrease can be bounded from below. In fact, this is the contents of the next lemma.
Lemma 6.1 If (x; s) 2 F and 0
1 1 0 0s p ! ( v ) ! ( v ) 0 min @2! (v ) @ 1 + n ? pn A ; nA 2
then (^x; s^) 2 F 0 and
2 1 + ! (^v 2) 1 + ! (pv n) : 1 + 4(pn?) 2
(20) (21)
Proof:
It may be clear from the proof of Theorem 5.1 that the given bounds (20) on guarantee the feasibility of the new iterate (^x; s^). So it remains to show that (21) holds. First observe that (16) holds also for r = 0. Hence, by using the notation ! 2 = ! (v 2) = with and such that e xs e, one has (1 ? pn ) ? 4 4! 2 (pn ? ) ? 2 pn 2 ! (^v ) = 4(pn ? ) + 2 pn : (1 ? pn ) + 4 2
2
By rearranging the terms the inequality (21) directly follows.
2
Now we are ready to prove the polynomial complexity of the classical primal{dual ane scaling algorithm for positive semide nite LCPs. We will denote by (x(k); s(k)) the iterate after k iterations and for simplicity we use the notation !k2 := ! (x(k)s(k) ).
12
Theorem 6.1 Let an initial interior point (x
(0) ; s(0)) 2 F 0, with 1 !0 and 0 < < (x(0))T s(0) ~ be given. We de ne parameters L and as follows. (x(0))T s(0) ; := 2 + 1 ; L~ := ln !02 nL~ p and we assume that 2nL~ !0 . Then, taking = tp1nL~ , where t is the (unique) real number in the interval [ ; + 2n1L~ ) such that 2tnL~ 2 is integral, after 2tnL~ 2 = O( n!L~ ) iterations the algorithm yields a solution (x; s) such that (x)T s and ! (xs ) !2 . 2
2
2 0
2 0
Proof:
For the moment we make the assumption that in each iteration of the algorithm the step size
= tp1nL~ satis es the conditions of Lemma 6.1. Later on we will justify this assumption. Taking logarithms in (21), and substituting the given value of , we obtain
p
!
p
+ !02 k ln 1 + p k k 2 n k2 n p ln 11 + = = 2 2 2 ~ ~ !k 4( n ? ) 4( n ? ) 4t nL ? 4tL 4tL~ tnL~ ? 1 : Hence we have !k2 !2 as long as 2 0
k ~ ~
4tL tnL ? 1
ln 1 + !! : 1+
2 0
2 0
(22)
2
Since f ( ) := ln((1 + )=(1 + =2)) is a concave function, and f (0) = 0; f (1) 41 , one has ln 1 + !! 0 !40 : 1+ 2 As a consequence, the inequality (22) is certainly satis ed if k !02 tL~ tnL~ ? 1 : 2
2
2 0
(23)
We conclude that, to maintain the inequality !k2 !2 , the total number of iterations must satify (23). 2 0
Since Lemma 4.3 is valid, the proof of Theorem 5.2 makes clear (see (18) and (19)) that the algorithm stops after at most k iterations, where pn (x(0))T s(0) = 2tnL~ 2 ; k 2 ln
and then we have (x(k) )T s(k) . (Note that the de nition of t guarantees that 2tnL~ 2 is integral.) So, as far as the gap reduction is concerned, the algorithm needs not more than 2tnL~ 2 iterations. This number of iterations will respect the bound (23) if 2tnL~ 2 !02 tL~ tnL~ ? 1 :
13
Dividing both sides by !02tnL~ 2 this reduces to the inequality t , which clearly is satis ed by the value assigned to t in the theorem. It remains to show that in each iteration of the algorithm the speci ed step size satis es the p condition (20) of Lemma 6.1. First, observe that n is equivalent to tnL~ 1: Since t p ~ and nL 1, we have n. So it remains to deal with the condition that for each k, with 1 k 2tnL~ 2 , 1 0s 2!k @ 1 +
!k2 !k A ? pn : n
Using n 4, we have p!kn !2k 12 : Therefore, since 2
p
1 + 2 ? > 1 if 0 < 43 ;
it is sucient that !k , for each k. As we have seen before, for the given step size p p we have !k p! 2 for each k. So is it sucient that satis es p! 2 . This amounts to !0 t nL~ 2. p Due to the assumption in the theorem that 2nL~ !0 , this certainly holds if t satis es !02t 2. Since !02 2 and t it is obvious that t sati es this inequality. Hence the proof of the theorem is complete. 2 0
0
Remarks: We can make the results of Theorem 6.1 more concrete as follows. If we choose a centered starting point, i.e., ! = 1, then 2 < 3 and so also 2 < t 3. 0
Hence in that case the algorithm needs at most 6nL~ 2 iterations. Let L denote the size of the LCP (1). If we assume that for the initial point (x(0))T s(0) = O(2L) and = O(2?L ), then we solve the LCP in O(2tnL2) iterations. If in addition the starting point is centered, then we have O(6nL2 ) complexity. Finally note, that although for arbitrary, not necessarily centered, starting points our analysis proves the polynomial complexity of the classical primal{dual ane scaling algorithm, the constant t depends on the (non)centrality of the initial point. A less centered initial point results in a larger t value. Clearly as !0 tends to zero, then t goes to in nity. This behaviour is in conformance with the results of Section 5.
14
References [1] E. R. Barnes. A variation on Karmarkar's algorithm for solving linear programming problems. Mathematical Programming, 36:174{182, 1986. [2] I. I. Dikin. Iterative solution of problems of linear and quadratic programming. Doklady Akademii Nauk SSSR, 174:747{748, 1967. Translated in : Soviet Mathematics Doklady, 8:674{675, 1967. [3] O. Guler, and Y. Ye. Convergence behavior of some interior{point algorithms. Mathematical Programming, 60:215{228, 1993. [4] D. den Hertog, and C. Roos. A survey of search directions in interior point methods for linear programming. Mathematical Programming, 52, 481{509, 1991. [5] B. Jansen, C. Roos, and T. Terlaky. A polynomial primal{dual Dikin{type algorithm for linear programming. Report Nr. 93{36, Technische Universiteit Delft, April 1993. Submitted to Mathematics of Operations Research. [6] N. K. Karmarkar. A new polynomial{time algorithm for linear programming. Combinatorica, 4:373{395, 1984. [7] M. Kojima, N. Megiddo, T. Noma, and A. Yoshise. A uni ed approach to interior point algorithms for linear complementarity problems, volume 538 of Lecture Notes in Computer Science. Springer Verlag, Berlin, Germany, 1991. [8] P. D. Ling. A new proof of convergence for the new primal{dual ane scaling interior{point algorithm of Jansen, Roos and Terlaky. Technical Report, University of East Anglia, August 1993. [9] S. Mizuno, and A. Nagasawa. A primal{dual ane{scaling potential{reduction algorithm for linear programming. Mathematical Programming, 62, 119{131, 1993. [10] R. D. C. Monteiro, I. Adler, and M. G. C. Resende. A polynomial{time primal{dual ane scaling algorithm for linear and convex quadratic programming and its power series extension. Mathematics of Operations Research, 15, 191{214, 1990. [11] R. Saigal. A three step quadratically convergent implementation of the primal ane scaling method. Technical Report 93{9, Dept. of Industrial and Operational Engineering, University of Michigan, Ann Arbor, MI 48109{2117, USA, February 1993. [12] T. Tsuchiya and M. Muramatsu. Global convergence of the long{step ane scaling algorithm for degenerate linear programming problems. Research Memorandum 423, The Institute of Statistical Mathematics, 4{6{7 Minami{Azabu, Minato{ku, Tokyo 106, Japan, January 1992. Revised September 1992. [13] R. J. Vanderbei, M. S. Meketon, and B. A. Freedman. A modi cation of Karmarkar's linear programming algorithm. Algorithmica, 1(4):395{407, 1986.
15
