The recent book by Boyd et al 4] contains many examples from system and control theory. ... Recent work on primal-dual methods for semide nite programming.
On Extending Some Primal-Dual Interior-Point Algorithms From Linear Programming to Semide nite Programming Yin Zhang Department of Computational and Applied Mathematics Rice University Houston, Texas 77005, U.S.A. October, 1995 (Revised November, 1996)
Abstract This work concerns primal-dual interior-point methods for semide nite programming (SDP) that use a search direction originally proposed by Helmberg-Rendl-Vanderbei-Wolkowicz [5] and Kojima-Shindoh-Hara [11], and recently rediscovered by Monteiro [15] in a more explicit form. In analyzing these methods, a number of basic equalities and inequalities were developed in [11] and also in [15] through di erent means and in di erent forms. In this paper, we give a concise derivation of the key equalities and inequalities for complexity analysis along the exact line used in linear programming (LP), producing basic relationships that have compact forms almost identical to their counterparts in LP. We also introduce a new formulation of the central path and variable-metric measures of centrality. These results provide convenient tools for deriving polynomiality results for primal-dual algorithms extended from LP to SDP using the aforementioned and related search directions. We present examples of such extensions, including the long-step infeasible-interior-point algorithm of Zhang [25].
Keywords: Semide nite programming, Primal-Dual interior-point methods. Abbreviated Title: Extending primal-dual algorithms to SDP
Research supported in part by DOE DE-FG02-93ER25171-A001
1
1 Introduction In this paper, we consider primal-dual interior-point algorithms for solving the semide nite program (SDP) min C X (1.1) s.t. Ai X = bi; i = 1; 2; ; m X 0; where C , X , and Ai are symmetric matrices in Rnn , bi 2 R, C X = tr(CX ), and X 0 means that X is positive semide nite. For any m n matrix A, vec A denotes the mn-vector obtained from stacking the columns of A one by one from the rst to the last. The norm k k is by default the 2-norm for vectors and spectral norm for matrices unless speci ed otherwise, such as k kF for Frobenius norm. In this paper, we will use extensively Kronecker-product notations (see Appendix A). Under certain assumptions, the optimality conditions for (1.1) are
0 T BB A y + vec Z ? vec C @ A(vecX ) ? b XZ
1 CC = 0; X; Z 0; A
(1.2)
where AT = [vec A1 vec A2 vec Am ], bT = [b1 b2 bm ], y 2 Rm and Z 2 Rnn are the dual variables. The zero on the right-hand-side of the equation in (1.2) means that every term on the left-hand side is zero. We say that a point (X; y; Z ) is feasible if it satis es the rst two linear equations in (1.2), X 0 and Z 0; and strictly feasible if in addition X and Z are symmetric positive de nite (s.p.d.). It is known that if a strictly feasible point exists, then (1.2) has a solution. If we require that all the matrices in SDP be diagonal, then (1.1) reduces to the familiar standard form linear program. Semide nite programming arises in many scienti c and engineering elds. The recent book by Boyd et al [4] contains many examples from system and control theory. Many applications in combinatorial optimization are included in [1]. In addition, many problems in semide nite programming come from eigenvalue optimization (see [18, 19], for example). Semide nite programming has recently attracted active research from the interior-point methods community. A rather comprehensive list of references for research works and applications in this eld up to early 90's can be found in [1]. Some more recent works include [3, 5, 7, 12, 16, 22, 23]. Primal-dual interior-point methods, especially the infeasible ones, have proven to be the most ecient methods in linear programming1 , and many polynomiality results exist for these methods. In this paper we will use linear programming as the representative of the class of problems with the common basic characteristics in terms of primal-dual methods, including convex quadratic programming, monotone linear complementarity problem and so on. 1
2
Extending existing results to semide nite programming is a promising area from both theoretical and practical points of view. Recent work on primal-dual methods for semide nite programming include [2, 11, 17, 24]. The analysis for semide nite programming has appeared to be more dicult than that for linear programming. A large part of the theoretical diculty has to do with the issue of maintaining symmetry in linearized complementarity, as will be discussed in the next section. Very recently, Monteiro [15] proposed two closely related linearized complementarity equations and used the rst one to successfully extend two feasible-interior-point primal-dual algorithms to semide nite programming, including the Kojima-Mizuno-Yoshise [10] long-step feasible algorithm. It was pointed out by Kojima [8] that the search directions of Monteiro are in fact special cases of those proposed earlier by Kojima, Shindoh and Hara [11], and one of the directions was also proposed independently by Helmberg, Rendl, Vanderbei and Wolkowicz [5]. In [11], the authors analyzed a short-step path-following algorithm and an infeasible-interior-point short-step potential-reduction algorithm. Consequently, some of the results in [15] coincide with those in [11] but with alternative and shorter proofs. Nevertheless, Monteiro's more explicit formulation does o er certain advantage over the formulation of Kojima, Shindoh and Hara. As we will demonstrate in this paper, Monteiro's formulation reveals some deep similarity between linear programming and semide nite programming and allows insightful complexity analysis for primal-dual methods for semide nite programming. Based on Monteiro's second linearized complementarity equation, which we prefer to the rst one for computational reasons, we rst derive in this paper a set of key equalities and inequalities central to the analysis of primal-dual methods for semide nite programming. These equalities and inequalities are not completely new in the sense that they ultimately have similar functionalities as those derived by Kojima, Shindoh and Hara [11] (and some of those by Monteiro [15] due to the aforementioned equivalence). However, our concise analysis follows the exact line of analysis for linear programming and produces familiar and compact results. In this paper, we also introduce a new formulation of the central path and a variable-metric measure of centrality. Together these results make the task of extending polynomiality results to semide nite programming a much amenable process for many existing algorithms. We present some examples of such easy extensions. Our analysis starts from the following simple identity which opens the door to the entire development of the paper.
Proposition 1.1 Let vectors u; v; r 2 Rp and nonsingular matrices E; F 2 Rpp satisfy Eu + Fv = r: If S = FE T is s.p.d., then
kD?T uk2 + kDvk2 + 2u v = kS ? 21 rk2; 3
(1.3)
where
D = S ? 12 F = S 21 E ?T and D?T = S ? 21 E = S 12 F ?T :
Proof: Pre-multiply both sides of Eu + Fv = r by S ? 21 and then take 2-norm squared.
(1.4)
2
This paper is organized as follows. In Section 2, we discuss the issue of linearizing the complementarity condition. Section 3 contains our derivations of basic equalities and inequalities. In Section 4, we introduce a new de nition of central path and two of its neighborhoods suitable for semide nite programming and present some useful properties. Section 5 is a brief discussion on the 1 1 role played by the condition number of Z 2 XZ 2 in determining the order of iteration-complexity. Section 6 contains some complexity results as examples to demonstrate the power of the basic equalities and inequalities in extending polynomiality results to semide nite programming. The paper concludes with some remarks in Section 7.
2 Linearization Since primal-dual methods are Newton-like methods applied to the optimality system (1.2), at a given point (X; y; Z ), an increment direction (X; y; Z ) should satisfy
AT y + vec Z = vec Rd ?(AT y + vecZ ? vec C );
(2.1)
Avec X = rp ?(Avec X ? b):
(2.2)
and
Equivalently, one can write (2.1) as m X i=1
yi Ai + Z = Rd = ?
m X i=1
!
yi Ai + Z ? C :
(2.3)
Hence Z will be symmetric whenever Z is. The linearization of the complementarity condition X+ Z+ = 0 at (X; Z ), or more generally the perturbed complementarity condition X+ Z+ = I for 2 [0; 1], where
= X Z=n is the so-called duality gap2, has proven to be a delicate issue due to the need of maintaining symmetry in X . The most obvious linearization is
XZ + XZ + X Z = I;
(2.4)
Strictly speaking, is the duality gap only when (X; y; Z ) is feasible for some y. But for convenience we will use the term regardless of feasibility. Moreover, since not all semide nite programs have equal primal and dual optimal objective values, the term \duality gap" is also used, in that context, to refer to the di erence between the two. 2
4
The new duality gap at the updated point (X+ ; y+ ; Z+ ) = (X; y; Z ) + (X; y; Z ) will be
+ = (X + X ) (Z + Z )=n = (1 ? ) + (X Z + Z X + X Z )=n + 2 (X Z )=n: If (X; Z ) satis es (2.4), we have Therefore,
X Z + Z X + X Z = n:
(2.5)
+ = (1 ? + ) + 2 (X Z )=n
(2.6)
It is critical to realize that (2.5) and (2.6) still hold true if the equation (2.4) is similarly transformed and symmetrized. Let us de ne the similarly transformed symmetrization operator h i HP (M ) = 21 PMP ?1 + (PMP ?1 )T : (2.7) for a given nonsingular matrix P . Therefore, (2.5) and (2.6) hold true if (X; Z ) satis es the following similarly transformed and symmetrized equation
HP (XZ + XZ + X Z ) = I
(2.8)
For interior-point methods, it is imperative to maintain symmetry of the X matrices, which in turn requires symmetry of X at each iteration. Since the three matrix products in the left-hand side of (2.4) are generally nonsymmetric, neither will be X in (2.4). On the other hand, X in (2.8) does enjoy symmetry thanks to the symmetrization. In [2], Alizadeh, Haeberly and Overton used pure symmetrization without a similar transformation, i.e., they used P = I in equation (2.8), leading to (Z X + XZ ) + (X Z + ZX ) = 2I ? XZ ? ZX:
(2.9)
Using the symmetrized linearized-complementarity equation (2.9), Alizadeh, Haeberly and Overton presented promising numerical results in [2] showing that the algorithm based on (2.9) is the most robust among a number of primal-dual algorithms. However, (2.9) does seem to have its diculties. Since Proposition 1.1 does not hold for the matrices in (2.9), our analysis in this paper does not seem to be applicable to search directions resulting from (2.9). Moreover, (2.9) necessitates solving a Lyapunov equations, i.e., equations of the form AX + XA = B for matrix X (see [2] for more details). 1 1 Recently, Monteiro [15] proposed to use P = X ? 2 or P = Z 2 in (2.8) for the similar transformations. He analyzed some primal-dual feasible-interior-point algorithms using the rst choice 5
P = X ? 12 and established polynomial complexity for them. Monteiro's linearized complementarity
equations, special cases of those of Kojima, Shindoh and Hara in more explicit forms, motivated our analysis in this paper. 1 Out of the two scaling matrices proposed by Monteiro, we prefer the second one P = Z 2 to the rst one for computational reasons that will be made clear at the end of this section. From now 1 on, we omit the subscript P from HP in (2.7) whenever P = Z 2 . 1 Substituting P = Z 2 into (2.8), and pre-multiplying and post-multiplying the resulting equation p H (XZ + X Z + XZ ) = I by 2Z 21 , we obtain 2Z (X )Z + ZX (Z ) + (Z )XZ = Rc ;
(2.10)
where
Rc = 2(Z ? ZXZ ) = 2Z 21 (I ? Z 12 XZ 12 )Z 21 : Equation (2.10) can also been obtained from linearizing, at (X; Z ), the equation
(2.11)
1 h(ZX Z ) + (ZX Z )T i = Z; + + + + 2 i.e., the symmetrization of the equation Z (X+ Z+ ? I ) = 0. Since Z varies from iteration to iteration, the direction (X; y; Z ) resulting from using (2.10) is not equivalent to a Newton direction for a xed nonlinear system. In Kronecker-product notation, (2.10) is 2(Z Z )vec X + (ZX I + I ZX )vec Z = vec Rc :
(2.12)
E = 2Z Z
(2.13)
F = ZX I + I ZX:
(2.14)
Let us de ne and Then (2.12) becomes
E vecX + F vec Z = vec Rc : It is worth noting that E is s.p.d, but F is generally nonsymmetric.
(2.15)
With the linearized complementarity (2.15), together with (2.1) and (2.2), we have the linear system satis ed by the increment (X; y; Z ) at point (X; y; Z ):
2 T I 3 0 vec X 0 A 7B 66 4 A 0 0 75 B@ y vec Z E 0 F
1 0 CC BB vec Rd A = @ rp vec Rc
1 CC A:
(2.16)
Just like in linear programming, one can solve the linear system (2.16) by the following procedure: 6
Procedure-A:
End
(1) [A(E ?1F )AT ]y = rp + A[E ?1(F vec Rd ? vec Rc )]. P (2) Z = Rd ? mi=1 yi Ai . (3) X = Z ?1 ? X ? [X (Z )Z ?1 + (X (Z )Z ?1)T ]=2.
This procedure was essentially used in both [5] and [11]. It is important to observe that E ?1 = (Z ?1 Z ?1 )=2 is relatively easy to compute. This is the reason why out of the two scaling schemes we prefer to use the second scaling P = Z 12 in (2.7) over 1 the rst one P = X ? 2 , which would lead to E = XZ I + I XZ and necessitate the solution of Lyapunov equations. In addition to the easy inversion of E , another computational advantage of (2.10) over (2.9) is that Cholesky, rather than LU, factorization can be utilized in the rst step of the above procedure, as is indicated by the following simple fact, which was also observed in [5].
Proposition 2.1 The matrix E ?1F = (X Z ?1 + Z ?1 X )=2 is s.p.d. Therefore A(E ?1F )AT is s.p.d. whenever A has full rank.
Proof: See Appendix A.
2
The following proposition states that if the primal equality constraints are consistent, then so is the linear system (2.16). It is worth noting that for the purpose of establishing complexity one does not need to assume that A is full rank, which we do not in this paper.
Proposition 2.2 Let X and Z be s.p.d. If b is in the range of A, then equation (1) in Procedure-A
is consistent and, consequently, so is (2.16).
Proof: Observe that under the assumption of the proposition rp and hence the whole righthand side of equation (1) are in the range of A. Since the matrix E ?1F is s.p.d, we can de ne A^ = A(E ?1F )1=2 and rewrite equation (1) as A^ A^ T y = A^ r for some vector r, which is clearly consistent. Consequently, there exists (X; y; Z ) that satis es (2.16). 2 The following proposition can be easily veri ed for matrices E and F de ned in (2.13) and (2.14), respectively. It allows us to use Proposition 1.1, thus paving the way to the development of the paper. 7
Proposition 2.3 The matrix S = FE T is s.p.d. and can be written as S = E 12 F^E 21 ;
(2.17)
F^ = E ? 21 FE 21 = Z 12 XZ 21 I + I Z 21 XZ 12 :
(2.18)
where F^ is s.p.d. given by
3 Basic Relationships In this section, we will assume that X and Z are s.p.d. and (X; y; Z ) satis es (2.16). When (X; y; Z ) is feasible, i.e., Rd = 0 and rp = 0, it is easy to check that X Z = 0. 1 1 1 1 Due to similarity, matrices XZ , ZX , Z 2 XZ 2 and X 2 ZX 2 all have the same spectrum, which we denote by and arrange as 1 2 n > 0: 1 1 Moreover, we de ne the spectral decomposition of Z 2 XZ 2 as
Z 12 XZ 21 = QQT ; where QT Q = I and
(3.1)
= diag(1; 2; ; n):
Before we proceed, let us recall that
= X Z=n =
n X i=1
i=n
and S , D and Rc are de ned by (2.17), (1.4) and (2.11), respectively.
3.1 General Relationships
Lemma 3.1
kD?T vecX k2 + kDvecZ k2 + 2X Z = kS ? 12 vec Rck2:
Proof: The equality follows immediately from Propositions 1.1 and 2.3. Lemma 3.2 n 1! n ( ? )2 X X 1 i 2 2 ? = 1 ? 2 + n n: kS 2 vec Rck = i=1
Moreover, if n for 2 (0; 1), then
kS ? 12 vec Rck2 min
i
i=1 i
! k ? I k2F ; (1 ? 2 + 2= )n :
8
2
Proof: Observe that
p vec Rc = 2E 21 (Q Q)vec(I ? )
(3.2)
F^ = (Q Q)( I + I )(QT QT ):
(3.3)
The proof of the rst part follows directly from the fact kS ? 21 vec Rck2 = (vecRc)T S ?1(vec Rc); Proposition 2.3, (3.2) and (3.3). The second part is obvious.
2
and
Lemma 3.3 p 1 1 ? kH (X Z )kF kZ 2 (X Z )Z 2 kF 2 kD?T vec X k2 + kDvec Z k2 ;
where = 1=n is the spectral condition number of the matrix Z 2 XZ 2 . 1
1
Proof: We start with kH (X Z )kF kZ 12 (X Z )Z ? 21 kF kZ 12 (X )Z 21 kF kZ ? 12 (Z )Z ? 12 kF :
(3.4)
Consider
kD?T vec X k2 = (vecX )T ES ?1E (vecX ) 1 1 = (vec X )T E 2 F^?1 E 2 (vec X ) 1 1 1 1 = 2[vec (Z 2 (X )Z 2 )]T F^?1 [vec (Z 2 (X )Z 2 )]: Since F^ has eigenvalues fi + j g (see (2.18) and Appendix A) kD?T vec X k2 1 kvec (Z 21 (X )Z 12 )k2 = 1 kZ 21 (X )Z 12 k2F ; 1
or
1
p kZ 12 (X )Z 21 kF 1kD?T vec X k:
(3.5)
By similar calculation we have
kDvecZ k2 nkvec (Z ? 21 (Z )Z ? 12 )k2; or
kZ ? 12 (Z )Z ? 12 kF p1 kDvec Z k:
Combining (3.4), (3.5) and (3.6), we have
kZ
1 2
1 (X Z )Z ? 2 kF
Finally, observe that
n
(3.6)
s
1 kD?T vec X kkDvecZ k: n
kD?T vec X kkDvecZ k 21 kD?T vec X k2 + kDvecZ k2 ;
which completes the proof.
9
2
3.2 Feasible Case First consider (X; y; Z ) is feasible. In this case, X Z = 0. Thus the following corollary follows directly from Lemmas 3.1, 3.2 and 3.3.
Corollary 3.1 Let (X; Z ) be feasible and n for 2 (0; 1). Then ! p 2 k ? I k 2 F kH (X Z )kF 2 min
; (1 ? 2 + = )n :
3.3 Infeasible Case Complexity analysis for infeasible-interior-point algorithms started in [25] where a couple of basic relationships were observed for monotone horizontal linear complementarity problem. These results were later modi ed into di erent forms by various authors. For the sake of clarity, we restate some of the original results in the context of linear programming.
Lemma 3.4 Let (x; y; z), (x0; y0; z0) be such that (x; z) > 0, (x0; z0) > 0 and for 2 [0; 1] AT y + z ? c = (AT y0 + z0 ? c); Ax ? b = (Ax0 ? b): Moreover, let (x; y; z )+(x; y; z ) and (xf ; yf ; zf ) both satisfy the equations Ax = b and AT y + z = c. Then
[x + (x0 ? xf )] [z + (z0 ? zf )] = 0; [(x ? xf ) ? (x0 ? xf )] [(z ? zf ) ? (z0 ? zf )] = 0;
(3.7) (3.8)
x z = ? [x (z0 ? zf ) + (x0 ? xf ) z ] ? 2 (x0 ? xf ) (z0 ? zf );
(3.9)
or equivalently,
[x (z0 ? zf ) + (x0 ? xf ) z] = x z + 2x0 z0 + (1 ? )(x0 zf + xf z0) ?x zf ? xf z + (1 ? )2xf zf :
(3.10)
The proof is mostly direct substitutions, so we omit it here. We note that the equalities (3.7) and (3.8) are taken directly from the proofs of Lemmas 6.1 and 6.2 in [25] where they appeared as inequalities in the more general context of monotone linear complementarity problems 3 . Furthermore, if we assume that (xf ; yf ; zf ) = (x; y; z); x0 ? x 0; z0 ? z 0; Equality (3.7) corresponds to the rst inequality after (6.8) in [25], and (3.8) to the one before (6.1) in [25]. The equivalence can be seen after the substitution uk = xk ? (x0 ? u0 ) given in Lemma 4.1 of [25]. 3
10
where (x ; y; z) is a solution of the linear program, as was done in [25], then it follows immediately from (3.10) that, noting 2 [0; 1], [x (z0 ? z ) + (x0 ? x) z] x z + x0 z0 + (x0 z + x z0 ): (3.11) To translate the above results to semide nite programming, let us likewise introduce two points: the rst point (X0; y0; Z0), where X0 and Z0 are s.p.d., satis es for some 2 [0; 1], and along with (X; y; Z ), the equations
AT y + vec Z ? vec C = (AT y0 + vec Z0 ? vec C ); A(vecX ) ? b = (A(vec X0) ? b);
(3.12) (3.13)
and the second point (X; y; Z) is a solution to (1.2). Under these analogous conditions, the above basic relationships in linear programming can be trivially extended to semide nite programming after changing the vectors into corresponding matrices along with the change in inner product. Hence to save space we will recycle Lemma 3.4 and inequality (3.11) in our analysis for semide nite programming without copying them into matrix forms. The following lemma is a direct extension of Lemma 6.2 in [25]. Lemma 3.5 For any (X0; y0; Z0) satisfying (3.12) and (3.13), and a solution (X; y; Z) to (1.2),
kD?T vec X k2 + kDvecZ k2
q
+ 2 +
2
;
(3.14)
= (kD?T vec (X0 ? X )k + kDvec(Z0 ? Z )k); = kS ? 21 Rck2F + 2 2 (X0 ? X) (Z0 ? Z ):
(3.15) (3.16)
where
Proof: First de ne
1
t = kD?T vec X k2 + kDvecZ )k2 2 :
From (3.9), we have X Z = ? (D?T vec X )T Dvec (Z0 ? Z) ? (Dvec Z )T D?T vec (X0 ? X) ? 2(X0 ? X) (Z0 ? Z) ? kD?T vec X kkDvec(Z0 ? Z)k ? kDvec Z )kkD?T vec (X0 ? X)k ? 2(X0 ? X) (Z0 ? Z) ? (kDvec (Z0 ? Z)k + kD?T vec (X0 ? X)k)t ? 2(X0 ? X) (Z0 ? Z) = ?t ? 2 (X0 ? X) (Z0 ? Z ): 11
Substituting the above and t2 into the rst equation in Lemma 3.1, we obtain
t2 ? 2t ? 0:
p
The quadratic 2 ? 2 ? has a unique positive root at + = + 2 + and it is positive for > +, hence we must have t + , which proves the lemma. 2 Now we need to estimate the quantities and in Lemma 3.5. Let us denote
0 = X0 Z0 =n: In the sequel, we will selectively use the following conditions (in addition to (3.12) and (3.13) which we always assume) that are also direct extensions of corresponding conditions in [25]:
=0 1; X0 ? X 0 and Z0 ? Z 0; X0 = Z0 = I;
(3.17) (3.18) (3.19)
where (X; y ; Z) is a solution to the semide nite program (1.2) and trX + trZ :
n
We observe that a condition similar to condition (3.17) was rst used by Kojima, Megiddo and Mizuno [9]. We also mention that we impose conditions (3.18) and (3.19) mainly for the sake of simplicity. They can be somewhat weakened by extra work. Conditions (3.18) and (3.19) are imposed to obtain polynomial complexity bounds. In general, a priori information on the size of a solution is not available, thus a very large value for may have to be taken to accommodate the worse-case scenario. This is particularly problematic in semide nite programming where the size of a solution may become excessively large (see [21], Section 1.6, Example 4). Fortunately, in practice a very large value for does not appear to be necessary for good convergence behavior.
Lemma 3.6 Under condition (3.17), 2 (X0 ? X) (Z0 ? Z) n: Moreover, if n for some 2 (0; 1), then
(3 ? 2 + 2= )n:
Proof: The rst inequality follows from the calculation 2 (X0 ? X) (Z0 ? Z ) = 2(X0 Z0 ? X0 Z ? X Z0 + X Z) (=0)(X0 Z0 ? X0 Z ? X Z0) (=0)X0 Z0 = n; 12
(3.20)
where we used the fact that matrices X , Z, X0 and Z0 are all s.p.d. Inequality (3.20) follows directly from Lemma 3.2 and (3.16). 2
Lemma 3.7 Under conditions (3.17) and (3.18), pn 2 + X0 ZX + ZZ0 X : n 0 0 In addition, if condition (3.19) also holds and n , then r 3n : Proof: First we calculate kD?T vec(X0 ? X)k2 = (vec(X0 ? X))T (ES ?1E )vec(X0 ? X) 1 1 = (vec (X0 ? X))T (E 2 F^?1 E 2 )vec (X0 ? X) 1 1 1 1 = 2(vec Z 2 (X0 ? X )Z 2 )T (F^ ?1 )vec (Z 2 (X0 ? X)Z 2 ) (1=n)kvec(Z 12 (X0 ? X)Z 21 )k2 1 1 = (1=n)kZ 2 (X0 ? X)Z 2 k2F (1=n)[tr(Z 12 (X0 ? X)Z 21 )]2 = (1=n)[tr((X0 ? X )Z )]2 = (1=n)[(X0 ? X) Z ]2 ; where we used the fact that for symmetric matrix M 0, kM kF tr(M ). Similarly,
kDvec (Z0 ? Z)k2 = (vec(Z0 ? Z))T (E ?1SE ?1)vec(Z0 ? Z) = (vec (Z0 ? Z ))T (E ?1F )vec (Z0 ? Z ) 1 1 = (1=2)(vec(X 2 (Z0 ? Z )X 2 )T 1 1 1 1 (X ? 2 Z ?1 X ? 2 I + I X ? 2 Z ?1 X ? 2 ) vec (X 21 (Z0 ? Z)X 21 ) (1=n)kvecX 21 (Z0 ? Z)X 21 k2 1 1 = (1=n)kX 2 (Z0 ? Z )X 2 k2F (1=n)[tr(X 21 (Z0 ? Z)X 12 )]2 = (1=n)[tr(X (Z0 ? Z ))]2 = (1=n)[X (Z0 ? Z )]2: Therefore,
q
q
1=n [X (Z0 ? Z) + (X0 ? X ) Z ] = 1=n 13
(3.21)
where
= [X (Z0 ? Z ) + (X0 ? X) Z ]:
To estimate , we invoke (3.11) to get
X Z + X0 Z0 + (X0 Z + X Z0) n + (=0)n0 + (n)=(n0)(X0 Z + X Z0) n[2 + (X0 Z + X Z0)=X0 Z0]: Substituting the above into (3.21), we obtain
X Z + X Z 0 0 1=nn 2 + : X0 Z0 q
This proves the rst inequality. Under condition (3.19), we calculate that X0 Z + X Z0 = [tr(X) + tr(Z )] = tr(X) + tr(Z) 1:
X0 Z0
2 n
Therefore.
n
q
3n 1=n:
Finally, using the condition n , we complete the proof.
2
Lemma 3.8 Under conditions (3.17), (3.18) and (3.19), if n for 2 (0; 1), then for n 4, kD?T vec X k2 + kDvecZ k2 38n2= :
(3.22)
Proof: Observe that when n 4, by Lemmas 3.6 and 3.7, 2 + 10n2 = : Therefore, by Lemma 3.5
kD?T vec X k2 + kDvecZ k2
q
+ 2 +
2
p (3 + 10)2n2= 38n2= ;
which completes the proof. 2 Under slightly di erent conditions and using some variants of Lemma 3.4, Kojima, Shindoh and Hara [11] derived their Lemma 7.6, which has similar functionality as our Lemma 3.8 above. Our straightforward approach leads to a shorter derivation.
14
4 Central Path and Its Neighborhoods We rst need to introduce some more notation. Let (M ) 2 Cn be the vector of eigenvalues of the matrix M 2 Rnn , i.e., the elements of (M ) form the spectrum of M . For convenience, if all eigenvalues are real we will assume the order
1 (M ) 2(M ) n?1 (M ) n (M ): We also de ne
X ( ) = X + X; Z ( ) = Z + Z; ( ) = X ( ) Z ( )=n:
4.1 Variable-Metric Measures of Centrality The following simple fact is somewhat obvious. We include a proof for completeness.
Proposition 4.1 For M 2 Rnn with a real spectrum, nonsingular P 2 Rnn and a scalar , HP (M ) = I , M = I:
Proof: Suppose that the equality holds on the left. We must have PMP ?1 = I + G for some skew-symmetric matrix G. Since the spectrum of M is real, we must have G = 0, otherwise M would have an eigenvalue + k (G) for a nonzero pure imaginary number k (M ), contradicting the realness assumption for the spectrum of M . The converse is obvious. 2 n n Hence the central path XZ = I is equivalent to HP (XZ ) = I for any nonsingular P 2 R . 1 In particular, at a given point (X; Z ) and for P = Z 2 , the central path can be de ned, for variable (X+ ; Z+ ), as H (X+Z+) = + I trH (Xn +Z+ ) I: (4.1) Moreover, the centrality of (X ( ); Z ( )) can be measured by the magnitude of the quantity (4.2) H (X ( )Z ( )) ? ( )I H (X ( )Z ( )) ? trH (X ( )Z ( )) I:
n
It should be noted though that in (4.1) the scaling matrix Z 12 changes from iteration to iteration, making (4.1) a variable-metric de nition of the central path and (4.2) a variable-metric measure of centrality. It is extremely convenient, from a theoretical point of view, to use the new de nition (4.1) for the central path in primal-dual methods for semide nite programming because of its connection with the linearized complementarity (2.10), as is indicated by the following proposition obtained by direct calculation. 15
Lemma 4.1 Let (X; y; Z ) and (X; y; Z ) satisfy the linear system (2.16). Then H (X ( )Z ( )) = (1 ? )H (XZ ) + I + 2 H (X Z ); 2 ( ) = (1 ? + ) + n X Z: Corresponding to the two most popular centrality conditions in linear programming
kx z ? ek and x z ?; we have for semide nite programming and
kH (XZ ) ? I kF ; 2 (0; 1);
(4.3)
(H (XZ )) ?;
(4.4)
where ? can be in nity in which case the condition is one-sided. More precisely, these conditions mean that the steplength should satisfy
kH (X ( )Z ( )) ? ( )I kF +( ); + 2 (0; 1);
(4.5)
or
+( ) (H (X ( )Z ( ))) ?+ ( ): (4.6) where the values of the parameters + , + and ?+ may or may not be di erent from those of ,
and ? at the previous iteration. For a xed steplength parameter = + , let us de ne
X+ = X ( +); Z+ = Z ( + ); + = ( + ): 1
If Z+ is s.p.d. (see Lemma 4.7 later in this section), then Z+2 will be the next scaling matrix P in 1 (2.7) to be used in the next step. We will denote the operator HP corresponding to P = Z+2 by H+. In general, 1 1 H (X+Z+) 6= H+ (X+ Z+ ) Z+2 X+Z+2 ; though we do have, as observed by Monteiro in [15], for P = Z 12 Z+? 2 1
H (X+Z+ ) = HP (H+(X+Z+)):
(4.7)
The following lemma guarantees a smooth transition from one iteration to the next for the variable-metric centrality conditions (4.5) and (4.6).
Lemma 4.2 Assume that both Z and Z+ are s.p.d. Then 16
1. kH+ (X+ Z+ ) ? + I kF kH (X+Z+ ) ? + I kF ; 2. n (H (X+Z+ )) (H+ (X+ Z+ )) (X+ Z+ ) 1 (H (X+Z+ )):
Proof: The rst inequality was proved by Monteiro in [15] (see equation (32) in the proof of
Theorem 4.1). The second set of inequalities follows from the fact that the real part of the spectrum of a real matrix is contained between the largest and the smallest eigenvalues of its Hermitian part (see P. 187 of [6], for example). 2 We point out that although Monteiro [15] did not explicitly use the variable-metric central path (4.1), nor the variable-metric centrality conditions (4.5) and (4.6), he made the necessary connections, using the relations in Lemma 4.2, between the variable-metric central path de nition (4.1) and the more traditional central path de nition (XZ ) = e that he used.
4.2 Results for Steplength Selections Subtracting the second equation times identity from the rst one in Lemma 4.1, we obtain H (X ( )Z ( )) ? ( )I = (1 ? )[H (XZ ) ? I ] + 2[H (X Z ) ? X n Z I ]: (4.8) In order to estimate terms like the second term in the right-hand side of (4.8), we de ne ! = kH (X Z )k + p2 jX Z j: F
n
(4.9)
Lemma 4.3 Let = 0 if (X; y; Z ) is feasible and = 1 otherwise. Then p ! ! 2 + pn (kD?T vec X k2 + kDvecZ k2 ): Proof: If (X; y; Z ) is feasible, then X Z = 0 and the lemma follows from Lemma 3.3. If (X; y; Z ) is infeasible, then the lemma follows from the fact that jX Z j kD?T vec X kkDvecZ k 21 (kD?T vec X k2 + kDvec Z k2): This completes the proof. From (4.10) and Lemma 3.8, we obtain the estimate 2
jX Z j 19n ;
(4.10)
2 (4.11)
which will be useful later. In the sequel, we demonstrate that the use of (4.5) and (4.6) enables us to obtain explicit rules for steplength selections almost exactly identical to those in linear programming. This fact greatly facilitates the extensions of many, if not most, primal-dual algorithms from linear programming to 17
semide nite programming. The lemmas in this subsections are all straightforward extensions of well-know steplength selection rules in linear programming. For the strong centrality condition (4.3), which usually leads to the so-called short-step algorithms, the following result is useful, especially for variants of the Mizuno-Todd-Ye predictorcorrector algorithm [14] (which should not be classi ed as a short-step algorithm, though).
Lemma 4.4 Let (X; Z ) satisfy (4.3). Then (4.5) holds for all 2 [0; 1] satisfying h( ) = ( ? +)(1 ? ) ? + + ! 2 0:
(4.12)
Moreover, (4.5) holds when 1. = 0, + > and
2. (X; y; Z ) is feasible, = = 1, and
p
2
1 + 1 + 4!=( + ? )
;
2(1 + )1=2 : + 2(1 ? )3=2
(4.13)
(4.14)
Proof: It follows from (4.8) and (4.3) that kH (X ( )Z ( )) ? ( )I kF ? +( ) (1 ? ) + 2(kH (X Z )kF + jX Z j=pn) ? + ((1 ? + ) + 2X Z=n) ( ? +)(1 ? ) ? + p + 2 (kH (X Z )kF + (1= n + + =n)jX Z j) h( ): Therefore, h( ) 0 implies (4.5). When = 0 and + > , the right-hand side of (4.13) is the unique positive root of h( ). Hence (4.13) is equivalent to h( ) 0. For the second case, observe that when = = 1, h( ) 0 is equivalent to ! + : (4.15) Condition (4.3) implies
kH (XZ ) ? I kF = k ? I kF ;
and hence
(1 ? ) i (1 + ): When (X; Z ) is feasible, from X Z = 0 and from Corollary 3.1, we have p k ? I k2 p 22 2p F ! 2
2 = 2 18
with 1 ? and (1 + )=(1 ? ). Substituting these bounds into (4.15), we obtain (4.14). 2 It is easy to check that the inequality (4.14) is satis ed by
= 0:45 and + = 0:3: These values will be used later in Section 6.3. We are actually more interested in the weak centrality condition (4.6), which usually leads to the so-called long-step algorithms that are much more ecient in practice. The following lemma gives an inequality for nding lower bounds on the steplength parameter when the weak centrality condition (4.6) is in use. It is worth observing that no eigenvalue calculations are necessary in order to nd the lower bound.
Lemma 4.5 Let (H (XZ )) satisfy (4.4) and 0 < + < 1 < ? ?+ . Then the inequalities g( ) = (1 ? )( ? +) + (1 ? + ) ? ! 2 0; (4.16) G( ) = (1 ? )(?+ ? ?) + (?+ ? 1) ? ! 2 0; (4.17) imply, respectively,
n (H (X ( )Z ( ))) + ( ) and 1(H (X ( )Z ( ))) ?+ ( ): Moreover, centrality condition (4.6) holds for all 2 [0; 1] satisfying
min(1 ? + ; ?+ ? 1) !:
Proof: Using Lemma 4.1, the de nition (4.9) for ! and condition (4.4), we calculate n(H (X ( )Z ( ))) ? +( ) (1 ? )[ n(H (XZ )) ? ] + g( ) g( ): Hence, g ( ) 0 implies n (H (X ( )Z ( ))) + ( ). Similarly, we can derive that G( ) 0 implies 1 (H (X ( )Z ( ))) ?+ ( ). Moreover, ignoring the rst term, which is nonnegative, in g( ) we see that (1 ? +)=! is sucient for g( ) 0. Similarly, (?+ ? 1)=! is sucient for G( ) 0. This completes the proof. 2
For infeasible-interior-point algorithms, we need to impose condition (3.17) on the steplength parameter . Since is updated by a factor of (1 ? ) at each iteration, in terms of , condition (3.17) can be written as ( ) (1 ? ): (4.18) It is worth noting that this relationship always holds for feasible algorithms.
Lemma 4.6 Condition (4.18) is satis ed for all positive if X Z 0; otherwise under the conditions of Lemma 3.8 it is satis ed for 2 [0; 1] such that jXn Z j 19n : 19
Proof: Condition (4.18) is equivalent to 2
( ) ? (1 ? ) = + n X Z 0;
which always holds if X Z 0. If X Z < 0, then condition (4.18) is equivalent to n=jX Z j, which, together with (4.11), proves the lemma. 2
4.3 Positive De niteness of X and Z Finally, we need to ensure the positive de niteness of the matrices X and Z at each iteration. Speci cally, we need to show that X+ and Z+ are s.p.d.
Lemma 4.7 Let + 2 [0; 1] be the largest value such that for all 2 [0; +] condition (4.18), and
either condition (4.5) or (4.6), are satis ed. Then both X+ and Z+ are s.p.d. unless + = 1 and + = 0 in which case (X+; y+ ; Z+) is a solution to (1.2).
Proof: It suces to verify for the weaker condition (4.6). We recall that the matrices X and Z are s.p.d. and note that condition (4.18) implies ( ) > 0 for 2 [0; 1). If + = 1 and (1) = 0, then (X+ ; y+ ; Z+ ) is clearly a solution. So we assume that ( ) > 0 for 2 [0; +]. Since the eigenvalues
of H (X ( )Z ( )) are continuous with respect to and H (X (0)Z (0)) is s.p.d., it follows from (4.6) and continuity that for 2 [0; + ] all the eigenvalues of H (X ( )Z ( )) remain positive. Hence by Lemma 4.2 all of those of Z 21 X ( )Z ( )Z ? 12 , and in turn those of X ( )Z ( ), remain positive since they are bounded below by the smallest eigenvalue of the Hermitian part H (X ( )Z ( )). This implies that all the eigenvalues of both X ( ) and Z ( ) remain positive and completes the proof.
2
5 Condition Number of Z XZ 1 2
1 2
Most bounds derived in this paper depend on | the condition number of the matrix Z 2 XZ 2 , whose order in turn depends on how close XZ is to the central path. We include some well-known facts concerning below as a proposition for the sake of convenience. 1
Proposition 5.1 If XZ satis es condition (4.3) for < 1, then : 11 + ? On the other hand, if XZ satis es condition (4.4), then
min( n; ?) : 20
1
In general, for short-step algorithms using the centrality condition (4.3), , being O(1), will not contribute to the order of iteration-complexity. However, for algorithms using the centrality condition (4.4), ? has to be chosen as a constant independent of n in order to preserve the iterationcomplexity order. When using the popular one-sided weak centrality condition, in which ? = 1 such as in Kojima-Mizuno-Yoshise feasible path-following algorithm [10] and in Zhang's infeasibleinterior-point algorithm [25], we have = O(n), thus increasing the iteration-complexity order.
6 Examples of Extensions To demonstrate the power of the results obtained in the previous sections, in this section we give some examples of extending polynomiality results to semide nite programming. Among them are the Kojima-Mizuno-Yoshise path-following algorithm [10], the infeasible-interior-point algorithm of Zhang [25], both long-step algorithms using the one-sided weak centrality condition (4.4), and the Mizuno-Todd-Ye predictor-corrector algorithm [14]. The Kojima-Mizuno-Yoshise algorithm has been extended to semide nite programming by Monteiro [15] and the Mizuno-Todd-Ye algorithm by Nesterov and Todd [17]. We will describe the algorithms and then give their complexity results for semide nite programming. Since we have obtained the essential ingredients for complexity analysis that are in forms similar to those in linear programming, the proofs naturally bears much similarity to their counterparts in linear programming. In the sequel, we will always assume that 2 (0; 1).
6.1 Feasible Path-Following Algorithms Algorithm-A:
Choose a strictly feasible point (X; y; Z ), ; 2 (0; 1), 0 < < 1 < ?, independent of n, such that either condition (4.3) or condition (4.4) is satis ed. Let 0 = X Z=n. Repeat until 0 , do (1) Find a solution (X; y; Z ) to (2.16). (2) Choose the largest + 1 such that either (4.5) or (4.6) is satis ed by all 2 [0; + ]. (3) (X; y; Z ) ( (X; y; Z ) + + (X; y; Z ). (4) = X Z=n.
End
In the algorithm, either (4.5) or (4.6) is consistently used throughout the process. When (4.6) is used with ? = 1, the algorithm is the Kojima-Mizuno-Yoshise long-step path-following algorithm [10] | the rst long-step polynomial primal-dual feasible-interior-point algorithm, which has recently been extended to semide nite programming by Monteiro in [15]. 21
Proposition 6.1 Assume that a strictly feasible point (X; y; Z ) to (1.2) exists. Algorithm-A terminates, asymptotically, in at most O(np ln 1 ) iterations, where
8 > < 1; if (4.5) is used; p = > 1; if (4.6) is used and ? < 1; : 1:5; if (4.6) is used and ? = 1:
Since the proof of Proposition 6.1 is similar to, and simpler than, that of Proposition 6.2 below, we omit it here.
6.2 Infeasible Path-Following Algorithms Algorithm-B:
Choose (X; y; Z ) satisfying (3.17)-(3.19), ; 2 (0; 1), 0 < < 1 < ?, independent of n, such that either condition (4.3) or condition (4.4) is satis ed. Let 0 = X Z=n and = 1. Repeat until 0 , do (1) Find a solution (X; y; Z ) to (2.16). (2) Calculate the largest ^ 1 such that (4.18), and either (4.5) or (4.6), are satis ed by all 2 [0; ^ ]. (3) Choose + to minimize ( ) = X ( ) Z ( )=n in [0; ^ ]. (4) (X; y; Z ) ( (X; y; Z ) + + (X; y; Z ). (5) = X Z=n and ( (1 ? + ) .
End
In the algorithm, either (4.5) or (4.6) is consistently used throughout the process. When (4.6) is used with ? = 1, the algorithm is an extension of the infeasible-interior-point algorithm studied in [25]. We note that in steps (2) and (3) it is not necessary to pick the exact maximizer ^ and minimizer + . We observe that because of condition (4.18), the decrease of infeasibility is never slower than that of duality gap. Hence a suciently small duality gap will ensure a small infeasibility, justifying the termination condition in Algorithm-B.
Proposition 6.2 Assume that a solution to (1.2) exists. Algorithm-B terminates, asymptotically,
in at most O(np ln 1 ) iterations, where
8 > < 2; if (4.5) is used; p = > 2; if (4.6) is used and ? < 1; : 2:5; if (4.6) is used and ? = 1: 22
Proof: We will prove the proposition for condition (4.6) only since the proof for condition (4.5) is
very similar except that (4.12) in Lemma 4.4 would be used instead of Lemma 4.5. We note that Step (1) in Algorithm-B is well-de ned by the virtue of Proposition 2.2. Consider step (2) in Algorithm-B. It follows from Lemmas 4.5 and 4.6 that ^ in step (2) exists and satis es : ; ^ min min(1 ? ; ? ? 1) ! 19n
Furthermore, Lemmas 3.8 and 4.3 ensures the existence of a constant 1 such that (1 ? ) n 1 ^ pn2 ; 1 2 0; 19 : (6.1) The upper bound on 1 will be utilized soon. Now consider step (3) in Algorithm-B. Observe from the expression of ( ) in Lemma 4.1 that + = ^ if X Z 0. On the other hand, if X Z > 0, then (1 ? )n + = min ^ ; 2X Z : Since + minimizes ( ) in [0; ^ ] and 2 [0; ^], clearly, ( + ) ( ). At any two consecutive p iterations, invoking (4.11) and substituting = 1 =( n2 ) we have
( + )=
( )= = 1 ? [1 ? ? X Z=(n)] p 1 ? [1 ? ? jX Z j 1 =( n3 )] p 1 ? [1 ? ? 19 1=( n)] p 1 ? [ 1 =( n2 )][1 ? ? 19 1=( n)] p = 1 ? 2 =( n2 );
where 2 = 1 [1 ? ? 19 1=( n)] > 0 in view of the upper bound on 1 in (6.1). This means that p the duality gap is reduced at each iteration by at least a factor of 1 ? 2=( n2 ). By a standard p argument, we conclude that 0 in at most O( n2 ln 1 ) iterations. Finally, the iteration-complexity orders for the cases ? < 1 and ? = 1 both follow from Proposition 5.1 and the discussion after it. 2 Infeasible-interior-point algorithms require weaker conditions to operate and they are usually more ecient in practice. However, their theoretical complexities are higher than their counterparts in feasible-interior-point algorithms.
23
6.3 Mizuno-Todd-Ye Predictor-Corrector Algorithm
Algorithm-C:
Choose a strictly feasible point (X; y; Z ) such that condition (4.3) is satis ed with = 0:3. Let 0 = X Z=n. Repeat until 0 , do (1) Find a solution (X; y; Z ) to (2.16) for = 0. (2) Choose the largest + 1 such that (4.5) is satis ed with + = 0:45 by all 2 [0; +]. (3) (X; y; Z ) ( (X; y; Z ) + + (X; y; Z ). = X Z=n. (4) Find a solution (X; y; Z ) to (2.16) for = 1. (5) (X; y; Z ) ( (X; y; Z ) + (X; y; Z ).
End
At each iteration, the algorithm solves two linear systems to compute two di erent steps: the predictor-step computed in (1), and the corrector-step in (4). Using a di erent linearized complementarity equation, Nesterov and Todd [17] have recently extended the Mizuno-Todd-Ye predictor-corrector algorithm to semide nite programming.
Proposition 6.3 Assume that a strictly feasible point (X; y; Z ) to (1.2) exists. Algorithm-C terminates in at most O(n 2 ln 1 ) iterations. 1
Lemma 4.4 contains all the needed ingredients for the proof of Proposition 6.3, which is a direct translations from its counterparts for linear programming. We provide an outline of the proof in the next paragraph and refer the interested reader to [14] for more details. Being a feasible algorithm, the goal of Algorithm-C is to drive the duality gap to zero. Observe that the predictor-step ( = 0) decreases the duality gap by a factor of 1 ? and the correctorstep ( = = 1) does not alter the duality gap. At the beginning of any iteration, the iterate satis es the centrality condition (4.3) with = 0:3. Firstly, (4.13) in Lemma 4.4 ensures that the p step-length for the predictor-step is at least the order of 1=O( n) since
+ ? = 0:45 ? 0:3 = 0:15; ! = O(n); (1 + 0:3)=(1 ? 0:3) 2: Secondly, (4.14) in Lemma 4.4 ensures that after the corrector-step the next iterate will again satisfy p the centrality condition (4.3) with = 0:3. Hence, the algorithm guarantees a factor of 1 ? 1=O( n) p reduction in duality gap at every iteration, leading to the O( n ln 1 )-iteration complexity.
6.4 Other Algorithms A number of other primal-dual algorithms can be readily extended to semide nite programming. For example, extensions of the Mizuno-Todd-Ye predictor-corrector algorithm to infeasible starting 24
point should have an O(n ln 1 )-iteration complexity. Recently, Potra and Sheng [20], and Kojima, Shida and Shindoh [13] have independently extended infeasible-interior-point predictor-corrector algorithms to semide nite programming with O(n ln 1 )-iteration complexity.
7 Concluding Remarks In summary, we consider that the present paper brings three additions to the research on primaldual interior-point methods for semide nite programming: 1. We provided a concise derivation of the most basic equalities and inequalities for complexity analysis, almost completely parallel to the simple derivation for linear programming. Consequently, the derived results have familiar and highly compact forms. 2. We introduced variable-metric measures of centrality, making the task of extending polynomiality results to semide nite programming much amenable for many existing primal-dual algorithms. 3. As an example, we extended a long-step infeasible-interior-point polynomial algorithm to semide nite programming for the rst time. This gives another theoretical indication that semide nite programming, though more dicult than linear programming, is susceptible to practically more ecient algorithms. We observe that for the infeasible algorithm, Algorithm-B, a stopping criterion can be derived from, among a number of possibilities, inequality (3.22) to detect \infeasibility". Whenever (3.22) fails, we can safely conclude from Lemma 3.8 that there exists no solution (X; Z) satisfying the conditions (3.18) and (3.19). In particular, there is no solution (X; Z) satisfying (X; Z) 21 (I; I ); where > 0 and X0 = Z0 = I . Finally, we mention that results in this paper are applicable to more general problems such as the monotone linear complementarity problem in symmetric matrices studied by Kojima, Shindoh and Hara in [11].
Acknowledgment I sincerely thank the following colleagues: Renato Monteiro for pointing out to me a costly error in my rst manuscript; Masakazu Kojima for bringing to my attention the equivalence of the search directions in [11] and [15]; and Steve Wright for his service in maintaining the interior-point mailing list and archive. I am also grateful to two anonymous referees for their constructive comments. 25
References [1] F. Alizadeh. Interior point methods in semide nite programming with applications to combinatorial optimization. SIAM Journal on Optimization, 5 (1995), pp. 13-51. [2] F. Alizadeh, J. A. Haeberly and M. Overton. Primal-dual interior-point methods for semidefinite programming. Draft, Computer Science Department, New York University, 1994. [3] F. Alizadeh, J. A. Haeberly and M. Overton. Complementarity and nondegeneracy in semidefinite programming. 1995. To appear in Mathematical Programming. [4] S. Boyd, L. Ghoui, E. Feron and V. Balakrishnan. Linear matrix inequalities in system and control theory. Society for Industrial and Applied Mathematics, Philadelphia, 1994. [5] C. Helmberg, F. Rendl, R. J. Vanderbei, and H. Wolkowicz. An interior-point method for semide nite programming. SIAM Journal on Optimization 6 (1996) pp.342-361. [6] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, 1991 [7] F. Jarre. An interior-point method for minimizing the maximum eigenvalue of a linear combination of matrices. SIAM Journal on Control and Optimization 31 (1993), pp. 1360-1377. [8] M. Kojima. Private communication. [9] M. Kojima, N. Megiddo, and S. Mizuno. A primal-dual infeasible-interior-point algorithm for linear programming. Mathematical Programming 61 (1993) pp.263-280. [10] M. Kojima, S. Mizuno, and A. Yoshise. A primal-dual interior point method for linear programming. In Nimrod Megiddo, editor, Progress in mathematical programming, interior-point and related methods, pages 29{47. Springer-Verlag, New York, 1989. [11] M. Kojima, S. Shindoh, and S. Hara. Interior-point methods for the monotone linear complementarity problem in symmetric matrices. Research Report #B-282, Dept. of Mathematical and Computing Sciences, Tokyo Institute of Technology 2-12-1 Oh-Okayama, Meguro-ku, Tokyo 152, Japan, April 1994 (revised April 1995). [12] M. Kojima, S. Shindoh, and S. Hara. Reduction of monotone linear complementarity problems over cones to linear programs over cones. Research Report #296, Dept. of Mathematical and Computing Sciences, Tokyo Institute of Technology 2-12-1 Oh-Okayama, Meguro-ku, Tokyo 152, Japan, January 1995. [13] M. Kojima, M. Shida and S. Shindoh. Global and local convergence of predictor-corrector infeasible-interior-point algorithms for semide nite programs. Research Report #B-305, Dept. 26
of Mathematical and Computing Sciences, Tokyo Institute of Technology 2-12-1 Oh-Okayama, Meguro-ku, Tokyo 152, Japan, October, 1995. [14] S. Mizuno, M. J. Todd, and Y. Ye. On adaptive step primal{dual interior{point algorithms for linear programming. Mathematics of Operations Research 18 (1993) 964-981. [15] R.C. Monteiro. Primal-Dual Path Following Algorithms for Semide nite Programming. Working paper, School of Industrial Engineering, Georgia Institute of Technology, 1995. [16] Y. E. Nesterov and A. S. Nemirovskii. Interior Point Methods in Convex Programming { Theory and Applications. Society for Industrial and Applied Mathematics, Philadelphia, 1994. [17] Y.E. Nesterov and M. Todd. Primal-dual interior-point methods for self-scaled cones. Technical Report 1125, School of Operations Research and Industrial Engineering, Cornell University, Ithaca, New York, 14853-3801, 1995. [18] M. L. Overton. On minimizing the maximum eigenvalue of a symmetric matrix. SIAM J. on Matrix Analysis and Applications Vol.9:256-268, 1988. [19] M. L. Overton. Large-scale optimization of eigenvalues. SIAM J. Optimization Vol.2:88-120, 1992. [20] F. A. Potra and Rongqin Sheng. A superlinearly convergent primal-dual infeasible-interiorpoint algorithm for semide nite programming. Reports on Computational Mathematics, No.78, Department of Mathematics, The University of Iowa, October 1995. [21] M. Ramana. An exact duality theory for semide nite programming and its complexity implications. Technical Report RRR 46-94, RUTCOR, Rutgers University, 1994. [22] F. Rendl, R. J. Vanderbei, and H. Wolkowicz. Interior-point method for max-min eigenvalue problems. Manuscript, Program in Statistics and Operations Research Princeton University, 1993. [23] L. Vandenberghe and S. Boyd. Semide nite programming. SIAM Review, 38 (1996) pp.49-95. [24] L. Vandenberghe and S. Boyd. A primal-dual potential reduction method for problems involving matrix inequalities. Mathematical Programming, Series B, 69 (1995) pp.205-236. [25] Yin Zhang. On the convergence of a class of infeasible interior-point algorithms for the horizontal linear complementarity problem. SIAM J. Optimization Vol.4:208-227, 1994.
27
A Appendix We list some important operation rules for Kronecker products that are used in the present paper. We refer interested reader to Chapter 4 of the excellent book by Horn and Johnson [6]. We use (A) to denote the spectrums of A. 1. A B = [aij B ]. 2. vec (AXB ) = (B T A)vec X . 3. (A B )T = AT B T . 4. (A B )?1 = A?1 B ?1 . 5. (A B )(C D) = AC BD. 6. (A) = fi g; (B ) = fi g ) (A B ) = fi j g.
28
