linear complementarity problems - CiteSeerX

Chapter 10

NEW COMPLEXITY ANALYSIS OF PRIMAL-DUAL NEWTON METHODS FOR P() LINEAR COMPLEMENTARITY PROBLEMS Jiming Peng, Cornelis Roos and Tamas Terlaky

Abstract In this paper, we consider a primal-dual Newton method for linear

complementarity problems (LCP) with P ()-matrix. By using some new analysis tools, we prove polynomial complexity of the large update method without using a barrier or potential function. Our analysis is based on an appropriate proximity measure only. This proximity measure has not been used in the analysis of a large update method for LCP before. Our new analysis provides a uni ed way to analyse both large update and small update methods. The polynomial complexity of the method of nding a maximally complementarity solution is discussed as well.

Keywords: Linear complementarity problem, interior point method, primal-dual method, polynomial complexity, ane variational inequalities.

1.

INTRODUCTION

In this paper we deal with a class of Linear Complementarity Problems (LCP): ?Mx + s = q; x  0; s  0; xs = 0: (10.1) where M is an n  n real matrix, q 2 0; s0 = Mx0 + q > 0: This assumption is very usual in the IPM literature on LCPs' [9]. The basic idea of primal-dual IPMs is to replace the complementarity condition in (10.1) by the parameterized equation xs = e, where e denotes the all-one vector and  > 0. Thus we consider the system s = Mx + q; x  0; s  0; xs = e: (10.2) If the matrix M is a P0 matrix and LCP (10.1) is strictly feasible, then the above parameterized system has a unique solution [9]. IPMs follow the central path approximately. Let us brie y indicate how this goes. Without loss of generality, we can assume that (x(); s()) is known for some positive . Then  is reduced to + = (1 ? ) for some  2 (0; 1), and by using Newton's method one construct a new point (x; s) that is `close' to (x(+ ); s(+ )). This process is repeated until  is small enough. The method is said to be a large update method if the parameter  is independent of the dimension number n (for example,  = 0:5), and small update if  depends on the dimension number n (for example  = p1n ). The proximities play an important role in the analysis of interior point methods since we use them to keep control the `distance' from the current 1 L is the input length of the problem [9].

New Analysis of Newton Methods for LCP

251

iterates to the current -centers. The three proximity measures are de ned as follows: n xs X ( i i ? 1 ? log xi si ); i=1



; K (xs; ) :=

xs ? e 

r xs r 

1 (xs; ) := 2

 ? xs

; q th

(xs; ) :=

(10.3) (10.4) (10.5)

q where xs ; xs and xs denote the vectors whose i component is xi si ; q xi si and q  respectively. The rst measure  [5] has turned   xi si out to be appropriate for the analysis of large update methods, while the second measure has been used by many authors to analyze the behavior of the path-following methods based on full Newton step, which are the simplest small update methods. Some variants of  are also applied in the analysis of the so-called potential reduction methods [13]. One reason for this is that  has some barrier properties while K does not. In [12], we have used the third measure  which is introduced by Jansen et al., [5] to analyze the complexity of the primal-dual newton method for linear programming and presented a uni ed proof for both small and large update methods. We notice that a variant ! of the proximity (xs; ) had been used by Kojima et al. in [9]. In [11], Mizuno and Nagasawa also used the proximity function 42 . However, both [9] and [11] consider the proximity  with  = xnT s depending on the current iterate while we cast  as a free parameter. We also mention that in a recent work [18], Zhao principally cast  as such a free parameter, but he updates  by choosing k+1 the smallest  2 (0; k ) such that (xs; k+1 )   where  is a constant (see Algorithm 3.2 in [18]). Hence, contrary to our approach, Zhao's algorithm never returns to a close vicinity of the central path, thus it is not a large update method. It operates in a large neighborhood, but the parameter  is updated typically by a small number. Further, the subproblem of determining the stepsize  in Zhao's method is more complicated than what we consider here. The rst aim of this work is to analyze the complexity of primaldual Newton methods for LCP based on the proximity measure  only which can be viewed as an extension of the results in [12] from Linear Optimization (LO) to LCP. An interesting topic in the IPMs literature is to identify an exact solution from an approximate solution. In this

252

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

paper, among others, we study the complexity of getting a maximally complementarity solution. The paper is organized as follows: In Section 2, we introduce some notations which will be used in this paper and state our interior point method. In Section 3, we give a simple analysis of the primal-dual Newton method. Complexity of nding the maximally complementarity solution is discussed in Section 4. Finally we close this paper by some conclusions.

2.

PRELIMINARIES

We rst introduce some notations and de nitions. A matrix M 2 is said to be a P () matrix if X X (1 + 4) xi [Mx]i + xi [Mx]i  0; 8x 2 0 such that (x0 s0 ; 0 )   .

begin x := x0 ; s = s0 ;  = 0 ; while n   do begin  := (1 ? ); while (xs; )   do x := x + 
x; s := s + 
s;

end end end Remark: In the update of the iterate, we demand that the damping parameter  is chosen such that the measure function  decreases suciently. Theorem 10.5 give a default value for .

3.

COMPLEXITY OF THE PRIMAL-DUAL NEWTON METHOD

We divide this section into three parts. In the rst section, we will estimate the decrease of the proximity after a damped Newton step. The second subsection is devoted to analyze the decrease of the proximity under some additional conditions. We summarize these results and

254

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

give the complexity of the algorithm in the last subsection. primal-dual Newton method,complexity

3.1

ESTIMATE OF THE PROXIMITY AFTER A DAMPED NEWTON STEP

Let + = (x+ s+; ) (with x+ = x + 
x; s+ = s + 
s) be the centrality measure of the new updated point. We want to estimate the decrease of +2 ? 2 (for simplicity, we often use  = (xs; )). Let us denote I+ = fi :
xi
si  0; i 2 I g; I? = I ? I+ . Since M is a P ()-matrix, we have (1 + 4) which implies

X

i2I+

(1 + 4)

Let us denote

+ =


xi
si +

X i2I+

X i2I+

dxi dsi +

X

i2I?

X i2I?


xi
si  0;

dxi dsi  0:

dxi dsi ; ? = ?

X i2I?

dxidsi:

Lemma 10.1 We have +  2 ; ?  (1 + 4)2 : Proof. From the de nitions of dx; ds , we get +  41 and that

X

i2I+

(dxi + dsi )2  14 kdx + ds k2 = 2 ;

?  (1 + 4)+  (1 + 4)2 :

2

By the de nition of + , we have 4+ 2 = =

n X

i=1 n X i=1

1

u2i + (1 ? u2i ) + 2 dxidsi + u2 + (1 ? u2 ) + 2 dxds ? 2 i

g((1 ? )u2i + (1 + dxi dsi));

i

!

i i

(10.7)

New Analysis of Newton Methods for LCP

255

where g(t) = t + 1t ? 2 is a convex function in (0; 1). If  2 [0; 1] is suciently small such that 1 + dxi dsi > 0 for all i = 1; 2; : : : ; n, then by the convexity of g(t) and (10.6) we have 4+ 2  (1 ? )

n X

g(u2i ) + 

n X

g(1 + dxi dsi)

i=1 i=1 ! n X 1 = (1 ? )42 +  1 + dxi dsi + 1 + dx ds ? 2 i i i=1 n n s x X X = (1 ? )42 + 2 dxi dsi ? 2 1 +diddix ds : (10.8) i i i=1 i=1 1 x s ? , then it easy to see that e + d d > 0. It follows from

If  < (10.7) that

4+ 2  (1 ? )42 + 2

n X i=1

x s X dxidsi 2 X ?di di +  x s i2I+ 1 + + i2I? 1 + di di x s2 + 3 X (di di )

dxi dsi ? 2

 (1 ? )42 + 2 + ? 2 1 +  +  3 2

+ 3  2

+ + ? :  (1 ? )42 + 1 +  + 1 ? ?

x s i2I? 1 + di di

(10.9)

The above inequality gives Theorem 10.2 Let  = (xs; ) and  = max(+; ?). For all 0   < 1 , one has 3 2 4+ 2  (1 ? )42 + 1 2?2 2 : (10.10)

Proof. From the choice of  we know that

3 +2  3 2 ; 1 + + 1 + 

and that

3 ?2  3 2 : 1 ? ? 1 ? 

The theorem follows from the above two inequalities and (10.8).

2

By Lemma 10.1, one has   (1 + 4)2 , thus the next corollary follows

256

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

Corollary 10.3 One has

3 + 4)2  4 (10.11) 4+ 2  (1 ? )42 + 1 2?(1 2 (1 + 4)2  4 : Remark 10.4 Let us consider the special case where a full Newton step ( = 1) can be made. If the matrix M is positive semide nite, then Corollary 10.3 implies 4 4+2  1 2? 4 : This is exactly the same result as Theorem II.51 of [13] for the linear optimization problem (see also Remark 5 in [12]). The result means that forp monotone LCP, the Newton process is quadratically convergent if   22 . Theorem 10.5 If   1 and  = 2(1+41 )2 then +2 ? 2  ? 12(1 5+ 4) : Proof. Since  = 2(1+41 )2  1 , it is a feasible step size. It follows from Corollary 10.3 that 3 + 4)2  4 +2 ? 2  ?2 + 2(1 ? (1 2 (1 + 4)2 4 )  ? 2(1 +1 4) + 12(1 +1 4)2  ? 12(1 5+ 4) :

The theorem is proved.

3.2

THE CASE kU ?1k1  1

2

In this section, we estimate the decrease of the proximity after a damped Newton step under the assumption that ku?1 k1  1. First, we give a technical result about the function g(t). Lemma 10.6 If y  1 and z  0. Then for all  < min(1; p1z ), we have 4 2 g((1 ? )y + (1 ? z))  (1 ? )g(y) + 1 ? z 2 z :

Proof. By the assumptions, it is easy to see we have 0  (1 ? 2 z )((1 ? )y + (1 ? z ))  y ? y +  ? 2 z < y;

New Analysis of Newton Methods for LCP

which implies (1 ? )(y ? 1)

257

? )(y ? 1)  (1 ? 2(1 z)(y ? y +  ? 2 z)

y

= 1 ?12 z ? y ? y +1  ? 2 z :

The lemma follows from the above inequality.

2

Now we are ready to state our main result in this section. Theorem 10.7 If ku?1 k1  1 and   min(1; p1 ), then 4 2

3 2

4+2  (1 ? )42 + 1+ + 1 ?  2  :

Proof. Similarly to the proof of inequality (10.8), one can easily show X g((1 ? )u2i +  + 2 dxi dsi) i2I+ 3 + X  (1 ? )g(u2i ) + 1 +  :

that

+

i2I+

By Lemma 10.6, one has X g((1 ? )u2i +  + 2 dxi dsi) i2I?



X

(1 ? )g(u2i ) +

X 4[dxi dsi]2

2 x s i2I? 1 +  di di 4 2 X  (1 ? )g(u2i ) + 1 ? 2? : ? i2I?

i2I?

The theorem follows from the above two inequalities.

2

A direct consequence of the above theorem is Corollary 10.8 If   1 and ku?1 k1  1. Then the stepsize  = 1 1+(1+4) is feasible. Moreover we have 2 4+2 ? 42  ? 1 + (13+ 4) :

258

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

Proof. The proof follows from some direct calculus, so the details are

omitted here.

3.3

2

ITERATION BOUND FOR THE PRIMAL-DUAL NEWTON METHOD

Now we are going to bound the computational complexity of our method. First let us consider the number of inner iterations between two successive updates of the parameter . Assume (xs; )   , i.e. the iterate is centered, the algorithm update  by + = (1 ? ). In this case, the proximity may increase and so one need to estimate how large the proximity  becomes. primal-dual Newton method,iteration bound The following upper bound is from Lemma IV.36 of [13]

2 (xs; + ) 

p

( n + 2)2 : 4(1 ? )

(10.12)

Now we have

Lemma 10.9 Let (x; s; )   and   1. Then after an update of the barrier parameter no more than  3(1 + 4)   p 2 5(1 ? ) n + 4 n + 4 iterations are needed to recenter. Proof. By (10.12), after the update, (xs; + )2

p

2 (2  +  n )  4 (1 ? ) :

5 . Hence after Each damped Newton step decreases 2 by at least 12(1+4 ) at most !' & p 12(1 + 4) (2 +  n)2 ?  2 5 4 (1 ? ) iterations, the proximity will have passed the threshold value  . This implies the lemma. 2

Consequently, we have

New Analysis of Newton Methods for LCP

259

Theorem 10.10 If   1, the total number of iterations required by the primal-dual Newton algorithm is no more than

 & 1 n0 '  3(1 + 4)   p 2 5(1 ? ) n + 4 n + 4  log  :

Proof. It can easily be shown that the number of barrier parameter updates is given by (cf. Lemma II.17, page 116, in [13])

&

'

1 log n0 :  

Multiplication of this number by the bound in Lemma 10.9 yields the theorem. 2 Omitting the round o brackets in Theorem 10.10 does not change the order of magnitude of the iteration bound. Hence we may safely consider the following expression as an upper bound for the number of iterations: 0!   p n 3(1 + 4  ) O 5(1 ? ) n + 4 n + 4 2 log  : Thus, if  = 21 and  = 1 the bound becomes

!

!

 n p  n0 6(1 + 4  ) n0 ; n + 4 log O + 4 = O (1 + 4  ) n log 5 2  

the best bound known for large-update methods. p Note thatp the order ofis magnitude does not change if we take  = n, or  = O ( n), which practically more attractive. An interesting choice is also  = n? 41 and 1  = n 4 , leading to the bound

O

3 (1 + 4)n 4

!

0 log n :

From the theoretical point of view, however, the best choice is  = n? 21 and  = 1, giving the best known iteration bound 0! p n O (1 + 4) n log  :

260

4.

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

COMPLEXITY OF FINDING A MAXIMALLY COMPLEMENTARITY SOLUTION

We will study the complexity of nding a maximally complementarity solution of (10.1) in the present section. First we introduce some de nitions and concepts. Let ? = fx : x  0; s(x) = Mx + q  0; xs(x) = 0g be the solution set of (10.1). Let us de ne the sets B := fi 2 I : xi > 0 for some x 2 ?g; N := fi 2 I : si (x) > 0 for some x 2 ? g; T := fi 2 I : xi = si(x) = 0 for all x 2 ? g: Then the index sets B; N and T form the so-called optimal partition of I (see Lemma 3.1 in [4]). It is known that the solution set ? is convex [1]. If x 2 ? and xB > 0; sN (x) > 0, then we call x a maximally complementarity solution of LCP. The rst condition number for LCP we use here is de ned by [4]: x := min maxfxi g; s := min maxfsi (x)g LCP LCP i2B x2? i2N x2? x s LCP = minfLCP ; LCP g = i2min maxfx + si (x)g: B[N x2? i Observe that by de nition, LCP > 0. Now we have Lemma 10.11 2 Let (xs; ) be as it is de ned in Section 2 with some positive  > 0. One has

p

xT s  (n + 22 + 2 n + 2 ):

(10.13)

Proof. It is easy to see that v vn u n n n u X X X u uX 1 kuk2 = u2i = n + ui(ui ? u )  n + t u2i t (ui ? u1 )2 i=1

i=1

= n + 2kuk: This inequality implies that

i

i=1

i=1

p kuk   + n + 2 :

2 This lemma improves slightly a similar result in the book [13] (page 213).

i

261

New Analysis of Newton Methods for LCP

It follows from the de nitions of  and u that

p

xT s = kuk2  (n + 22 + 2 n + 2 ):

2

This completes the proof of the lemma.

To identify the optimal partition (B; N; T ), we need to knowphow large xi and si are in these dierent index sets. Denote k = ( + k +  2 )2 , we have

Theorem 10.12 If    for some positive ;  > 0. Then it holds (1 + 4)n ; i 2 B ; ; s  xi  (1 +4LCP i )n 1 LCP (1 + 4)n ; i 2 N: si  (1 +4LCP )  ; xi   n 1

LCP

Proof. The proof is similar to that of Theorem 3.1 in [4]. For completeness, we give it here. First we consider the case i 2 B . Let us assume x^ 2 ? and hence s^ = M x^ + q. Since M 2 P (), and all x; s; x^ and s^ are nonnegative, we get

(x ? x^)T (s ? s^) = (x ? x^)T M (x ? x^)  ?4 = ?4

 ?4

X

i2I+(x?x^)

X

i2I+(x?x^)

X i2I+(x?x^)

(x ? x^)i [M (x ? x^)]i

((xs)i ? (xs^)i ? (^xs)i + (^xs^)i ) (xs)i  ?4xT s:

This inequality means

xT s^ + sT x^  (1 + 4)xT s  (1 + 4)n ; where the last inequality follows from Lemma 10.11 and the de nition of k . Hence it holds max(si x^i ; xi s^i)  (1 + 4)n ; 8i 2 I:

(10.14)

Let us rst take any j 2 B . Due to the de nition of LCP , we can choose x^ such that x^j is maximal and hence x^j  LCP . It follows from (10.14) that (10.15) s  (1 + 4)n : j

LCP

262

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

On the other hand, since    , one can easily verify that p 2 2 =  ; 8i 2 I: (xs)i = u2i  ( + 1 +  ) 1 The above inequality and (10.15) give

xj  (1 +4LCP )n 1 : By appropriately choosing x^, the same bound can be derived for all j 2 B . The case i 2 N follows similarly. The proof of the theorem is nished. 2 We need the following notation to de ne our second condition number for LCP. For any matrix A 2 A > eT (z + y) = 1; u; y; z  0; > > > > >

! >
=  (A; C ) := max

u;v > v corresponding to nonzero >
> > 1 > > elements of ( u; v ) are > > > : ; linearly independent > For given b 2 x^i ; 8i 2 N: Letting I1 = fi : x^i  s^ig; I2 = fi : x^i < s^i g: We have B  I1 and N  I2 under the assumption (10.17). Let us de ne ( H (^x) = min(^x; s^) = s^i if i 2 I1; x^i if i 2 I2: Since    , one can easily show that 1  u2   ; 8i = 1; 2; : : : ; n; which implies Hence

1

i

1

  (^xs^)   ; 8i = 1; 2; : : : ; n: i 1 1

(10.18)

[H (x())]i  p1 p; 8i = 1; 2; : : : ; n: Now consider the linear system 8 > < Mx ? s = ?q (10.19) xI2 = 0 ?xI1  0; > : sI1 = 0 ?sI2  0: It is easy to see that the feasible set of this system is the solution set of the LCP. Replacing the equalities in the above system by Mx ? s = ?q; xI2 = HI2 (^x); sI1 = HI1 (^x);

264

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

we get a new system which has at least a feasible point (^x; s^). Now applying Lemma 10.13 and (10.18), we get the conclusion of the theorem.

2

From Theorem 10.12 and Theorem 10.15 we know that if

LCP p1p1 < (1 +4LCP )n 1

and

p (1 + 4)n < LCP LCP 1 p1 ; then we get a complete separation of the variables. Both of the above inequalities hold if 2 (10.20)  <  2 (1+LCP 4)2 2 3 : LCP

n 1

This implies that if    with suciently small  > 0 satisfying (10.20), then we can determine the optimal partition B; N and T . By Theorem 10.10 we get Theorem 10.16 Let   1. To identify the optimal partition index sets (B; N; T ), the primal-dual Newton algorithm needs at most

 3(1 + 4)   & 1 nLCP 2 (1 + 4)2 2 3 ' p n 1 2 2 5(1 ? ) n + 4 n + 4  log LCP

iterations. If    and  is suciently small, then we can apply the rounding procedure in [4] to locateQa maximally complementarity solution of LCP. Let us denote (M ) := nj=1 kM:j k. Following the proof of Theorem 5.1 in [4], one can easily check that if 2   n 2 (1 + 4)2LCP 2n 31 kM k21 2 (M ) ; LCP

then the rounding procedure in [4] can nd a maximally complementarity solution in strongly polynomial time. This shows that Theorem 10.17 Let   1. To nd a maximally complementarity solution, the primal-dual Newton algorithm needs at most

 3(1 + 4)  2 (1 + 4)2 2 3 kM k2 2 (M ) '  & 1 n2LCP p 1 n 1 2 2 5(1 ? ) n + 4 n + 4  log LCP

New Analysis of Newton Methods for LCP

265

iterations, and then a maximally complementary solution can be identi ed in strongly polynomial time. It should be noted that if M and q are integral, then by Lemma 3.2 and Lemma 3.6 in [4], we have LCP  (1M ) and LCP  n(M ): This gives Corollary 10.18 Assume that M and q are integral. Let   1. Then the total number of iterations required by the primal-dual Newton algorithm to nd a maximally complementarity solution is at most  3(1 + 4)  pn + 4 2   1 log[n4(1 + 4)2 2 3kM k2 6(M )] : n + 4  n 1 1 5(1 ? ) 

5.

CONCLUDING REMARKS

Based on some standard and elementary tools, we have analyzed the behavior of a primal-dual Newton method for linear complementarity problems and discussed its complexity. First, by using the convexity of the measure function, a very simple proof of the polynomial complexity of IPMs with large update is presented. Particularly, this new analysis can be used to improve slightly the results of both small and large update interior point methods, and thus provides a uni ed way for studying both small and large update interior point methods. These tools can also be applied to other interior point algorithm variants, i.e., the logarithmic barrier method, or the potential reduction method. The complexity of nding an exact maximally complementarity solution is also discussed. Our results show that if a strictly feasible starting point is available, then the primal-dual Newton method can identify not only the optimal partition sets but also an exact maximally complementary solution in polynomial time. In the analysis of the algorithm, we assume that a strictly feasible starting point is available. For general LCP with P ()-matrix, it is not easy to nd a strictly feasible point. In [9], Kojima et al. proposed the big-M method to get a strictly feasible starting point for P () LCP. The method can be stated as below. Denote

!

!

!

M = M En ; q = q ; x = x : ?En 0 q^ x^

266

HIGH PERFORMANCE OPTIMIZATION TECHNIQUES

One get an arti cial LCP (10.21) s = M x + q  0; x  0; xT s = 0: For this arti cial LCP, Kojima et al. proved that M is a P () matrix if and only if M is a P () matrix. Further, it was showed [9] that if q^ > (M )e, then the arti cial LCP (10.21) has a strictly feasible point. Since the self-dual embedding technique has successfully solved the initialization problem in linear optimization [16, 13], we naturally hope to generalize the embedding technique from LO to LCP. In this situation we usually get an extended matrix as follows

!

M= MT v ; ?v for some v 2 0. It is easy to see that M is positive semidefinite for any v and any  0 provided M is positive semide nite. One can also show that if M is a P -matrix, then there exists a constant 0 depending on M and v such that under the condition that > 0 M is also a P -matrix. However, this is not true for the P ()-matrices. For example, let us choose

0 BB 01 M =B B@ 8 ?1

?4 ?1 0 0 2

1 0 ?2 0 ?4 4

1 CC CC ; A

where is any positive number. M is not a P () matrix, since if we choose x = (1= ; 1; ?1; 1= ); then 4 X xi (M x)i  0; 1  i  4; and xi (M x)i < 0: But

i=1

0 1 0 ? 4 ? 1 B C M =B @ 1 0 0 CA

8 0 0 is a P ()-matrix with  = 314 . This example shows the embedding model does not work well for P () LCP. So we stay with the following question:

New Analysis of Newton Methods for LCP

267

Question: How to construct a big-M free initialization procedure for P () LCPs.

Some extensions of our results are also interesting. For instance, consider the ane variational inequality problems (AVIP) de ned as follws F (x)T (y ? x)  0; 8y 2 X where X is a polyhedral set de ned by X = fx : Ax  b; x  0g with A 2