Approximation algorithms for quadratic programming - CiteSeerX

Approximation algorithms for quadratic programming Minyue Fu Department of Electrical and Computer Engineering, The University of Newcastle Newcastle, NSW 2308, Australia Zhi-Quan Luo Department of Electrical and Computer Engineering McMaster University Hamilton, Ontario, Canada L8S 4K1 and Yinyu Ye Department of Management Sciences The University of Iowa Iowa City, Iowa 52242, U.S.A. November 24, 1998

Abstract We consider the problem of approximating the global minimum of a general quadratic program (QP) with n variables subject to m ellipsoidal constraints. For m = 1, we rigorously show that an -minimizer, where error  2 (0; 1), can be obtained in polynomial time, meaning that the number of arithmetic operations is a polynomial in n, m, and log(1=). For m  2, we present a polynomial-time (1 ? m12 )-approximation algorithm as well as a semide nite programming relaxation for this problem. In addition, we present approximation algorithms for solving QP under the box constraints and the assignment polytope constraints.

Key words. Quadratic programming, global minimizer, polynomial-time approximation algorithm

The work of the rst author was supported by the Australian Research Council; the second author was supported in part by the Department of Management Sciences of the University of Iowa where he performed this research during a research leave, and by the Natural Sciences and Engineering Research Council of Canada under grant OPG0090391; the third author was supported in part by NSF grants DDM-9207347 and DMI-9522507, and by the Iowa Business School Summer Grant. 

1

1. Introduction Consider the general quadratic programming (QP) problem (QP)

Minimize q(x) := 12 xT Qx + cT x Subject to x 2 F ;

where Q 2   jj. By the positive semi-de niteness of Q +  I , we have

kx k < kx k = r: We consider two cases. Case I. In the rst case we assume



  p 1 ? 8 n   jj or   jj + 8p n  :

Using the relation (2.5) and simplifying, we obtain

kx k ? kx k = xT ( )(I ? (Q +  I )(Q + I )? (Q +  I ))x  = xT ( ) 2( ?  )(Q + I )? ? ( ?  ) (Q + I )? x : 2

2

2

1

2

2

Next we consider a diagonalization of Q and bound the resulting diagonal entries of the above expression by using the smallest eigenvalue . This gives

kx k ? kx k ) ! ) (  ?  2(  ?   ( ? jj) ? (( ? jj)) kx k 2

2

2

2

5

2

= = =



!

( ?  )2 2( ?  )  2 ? ( ?  ) + ( ? jj) (( ?  ) + ( ? jj))2 kx k ( ?  )2 + 2( ?  )( ? jj) r2 (( ?  ) + ( ? jj))2 ! 2  1 ? (( ? ( ) +?(jj)? jj))2 r2 !  =8pn)2 (  1 ? (( ?  ) +  =8pn)2 r2 ;

where the in last step we used the assumption   jj + 8p n  . Therefore, if we have  ?   0 =8, then pn0 =) + (pn0=)2 pn0 (2 2 2 2 2 kx k ? kx k  (1 + (pn0 =))2 r   r2: (2.6) On the other hand, note that

q(x ) ? q(x ) = 21 xT Qx + cT x ? 21 xT Qx ? cT x

= 12 (Qx + c)T (x ? x ) + 21 (Qx + c)T (x ? x ) = ? 12 xT (x ? x ) ? 21  xT (x ? x ) = ? 12 ( ?  )xT (x ? x ) ? 12  (kx k2 ? kx k2 ):

(2.7)

Now we use the bound (2.6), the assumption  ?   0  =8 and the fact kx k  kx k = r to obtain:

q(x) ? q(x ) p 0   r20=8 + r2 n pn0  2 = (0 =4 +  ) r2 =2 pn0 2 0  ( =4 +  )(q(0) ? q(x));

where the last step is due to (2.4). Thus, if we select

2 0  2pn+ 1=4 ;

then x is feasible for BQP(r) and

q(x ) ? q(x )  (q(0) ? q(x )); i.e., x is an -minimizer to x . 6

Case II. In this case, we have   1 ? 8p n  < jj or  < jj +  8p n : p Again, if we have  ?  < 0  =8, then  ? jj < 0  =8 +  =8 n. However, unlike Case I, we nd kx k is not suciently close to r. When we observe this fact, we do the following computation,

essentially due to Vavasis and Zippel [29], to enhance x . Let q, kqk = 1, be an eigenvector associated with the eigenvalue . Then, one of the unit vector ej , j = 1; :::; n ? m, pmust have jeTj qj  1=pn. (In fact, we can use any unit vector q to replace ej as long as qT q  1= n. A randomly generated q will do it with high probability.) Now we solve for y from (Q + I )y = ej and let x = x + y; where  is chosen such that kxk = r. Note we have (Q + I )x = ?c + ej ; and in the computation of x and y, matrix Q + I needs to be factorized only once. It is easy to show that kyk  pn(1? jj) and jj  2r( ? jj)pn  2r(0 =8 + =8pn)pn: Then, we have from (2.7) q(x) ? q(x ) = 21 (Qx + c)T (x ? x ) + 12 (Qx + c)T (x ? x ) = 12 (Qx + c ? ej )T (x ? x ) + 12 eTj (x ? x ) ? 12  xT (x ? x ) = ? 21 xT (x ? x ) + 12 eTj (x ? x ) ? 12  xT (x ? x ) = ? 12 (x +  x )T (x ? x ) + 12 eTj (x ? x ) = ? 21 ( ?  )xT (x ? x ) + 12 eTj (x ? x );

where the last step follows from kxk = kx k = r. Now we use  ?  < 0  =8 and the preceding upper bound on  to estimate the right hand side: q(x) ? q(x )  r20=8 + 2(0 =8 + =8pn)r2pn p = (0 =4 + n0 =2 + =2) r2 =2  (0=4 + pn0 =2 + =2)(q(0) ? q(x )); 7

where the last step is due to (2.4). Thus, if we choose

0  pn + 1=2 ;

then x is feasible for BQP(r) and

q(x) ? q(x )  (q(0) ? q(x ))  (z ? z ); i.e., x is an -minimizer of q(x) over B(r). Hence, the bisection method will terminate with an -minimizer of BQP(r) in at most O(log(0 = ) + log(1=) + log n) steps, or in a total of O(n3 (log(0 = ) + log(1=) + log n)) arithmetic operations.

Theorem 1 The total running time of the bisection algorithm for generating an -minimal solution to the ball-constrained QP is bounded by O(n3 (log(0 = )+log(1=)+log n)) arithmetic operations.

The polynomial complexity in Theorem 1 can be further improved. In particular, we [31] developed a Newton-type method for solving (BQP(r)) and established an arithmetic operation bound O(n3 log(log(0 = ) + log(1=0 ))) to yield a  such that 0   ?   0 . To compute an -minimizer of BQP(r), we rst nd an approximate  to the absolute value of the least eigenvalue jj and an approximate eigenvector q to the true q, such that 0   ? jj  0 and qT qk  1 ? 0. This approximation can be done in O(n3 log(log(1=0 ))) arithmetic operations. Then, we will use q to replace ej in Case II (i.e., kx k < r) to enhance x() and generate a desired approximation. Otherwise, we know  >  and, using the method in [31], we will generate a  2 (; 0 ) such that j ? j  0=8 in O(n3 log(log(0 =) + log(1=0 ))) arithmetic operations.

3. The multiple-ellipsoid constrained QP In this section we consider the QP problem Minimize q(x) := 21 xT Qx + cT x Subject to x 2 F ;

(3.1)

whose feasible region F is de ned by the intersection of multiple ellipsoids, namely,

F := fx 2 r > 0, respectively. Moreover, let x be a -minimizer of BQP(r) and

q(x) ? q(x (r))  (q(0) ? q(x (r))): Then, for any z such that q(x (R))  z  q(x (r)

q(x) ? z  (1 ? (1 ? )(r=R)2 )(q(0) ? z):

Proof. We have q(x) ? z = q(x) ? q(x (r)) + q(x (r)) ? z  (q(0) ? q(x (r))) + q(x (r)) ? z = ?(1 ? )(q(0) ? q(x (r)) + q(0) ? z = ?(1 ? )(q(0) ? q(x (R))) + (q(0) ? z )  ?(1 ? )(q(0) ? z) + (q(0) ? z) = (1 ? (1 ? ))(q(0) ? z )); where

(0) ? q(x (r)) :  := qq(0) ? q(x (R))

According to Theorem 2, we have   (r=R)2 .

9

Q.E.D.

Theorem 2 shows that the larger is the ratio r=R, the closer is the approximate minimizer x (r) to the global minimum of q(x) over F . This suggests that we should nd two ellipsoids of the same orientation and center, one containing F with radius R while the other contained in F with radius r, so that the ratio r=R is maximized. In what follows, we shall use the analytic center based inscribing ellipsoid for F and show that when we increase the radius r to R = mr, then the inscribing ellipsoid becomes a containing ellipsoid of F , where m is the number of quadratic inequalities de ning F . This result should be contrasted with the result of Lowner-John [25, Page 205] which says that for each full dimensional compact convex body in