On Commutative Class of Search Directions for Linear ... - CiteSeerX

CS-00-02

On Commutative Class of Search Directions for Linear Programming over Symmetric Cones

Masakazu Muramatsu

The University of Electro-Communications 1-5-1 Chofugaoka, Chofu-shi, Tokyo, 182-8585 JAPAN.

June 20, 2000

Abstract The Commutative Class of search directions for semidefinite programming is first proposed by Monteiro and Zhang [13]. In this paper, we investigate the corresponding class of search directions for linear programming over symmetric cones, which is a class of convex optimization problems including linear programming, second-order cone programming, and semidefinite programming as special cases. Complexity results are established for short, semi-long, and long step algorithms. We then propose a subclass of Commutative Class of search directions which has polynomial complexity even in semi-long and long step settings. The last subclass still contains the NT and HRVW/KSH/M directions. An explicit formula to calculate any member of the class is also given.

Key words: Symmetric Cone, Primal-dual Interior-Point Method, Jordan Algebra, Polynomial Complexity Affiliation: Department of Computer Science, The University of Electro-Communications

1

1. Introduction In this paper, we consider linear programming problems over symmetric cones. Nesterov and Todd [16] first proposed this optimization problem under the name of convex programming for self-scaled cones, and established polynomial complexity of primal-dual interior-point method applied to this problem using the so-called NT direction. This optimization problems include usual linear programming (LP), semidefinite programming (SDP), and second-order cone programming (SOCP) as special cases. Exact definitions of symmetric cone and linear programming over the cone will be given in the following sections. It is well-known in the research field of Lie algebra that symmetric cone has a deep connection with Euclidean Jordan algebra. Faybusovich [4, 5] used Euclidean Jordan algebra to analyze linear programming over symmetric cones. Sturm [18] obtained several fundamental inequalities needed for the analysis of interior-point methods in terms of Euclidean Jordan algebra. We can find a different algebraic approach for convex programming similar to ours in Schmieta and Alizadeh [17]. In semidefinite programming, a special case of our problems, Monteiro [10], Zhang [21] and Monteiro and Zhang [13] proposed a family of search dierction derived by scaling, which is called MZ-family of search directions. In Monteiro and Zhang [13], a subfamily of MZ-family called Commutative Class, or MZ∗ family, is also proposed for which complexity of the algorithm is derived. The same class is also proposed by Gu [7] from a different point of view with the name TTT family. In this paper, we consider Commutative Class of search directions for linear programming over symmetric cones. Based on the techniques developed in Monteiro and Zhang [13], Monteiro and Tsuchiya [12], Tsuchiya [19, 20], and Sturm [18], we analyze complexity of short, semi-long, √ and long step algorithms using search directions of this class. For short step algorithm, we prove O( n log 1/ǫ) complexity of the algorithm. For long step, the same complexity result as in Monteiro and Zhang [13] for semidefinite programming is derived, which depends on a ‘condition number’ of the scaled point. Then we propose a subclass of the Commutative Class called Power Class of search directions and show that the long and semi-long step algorithms using direction in this family have polynomial complexity. The Power Class still includes HRVW/KSH/M direction ([8, 9, 10]) and NT direction as special cases. Finally, we give an explicit formula to calculate search directions in Power Class. This paper is organized as follows. In Section 2, we give a summary of Euclidean Jordan algebra and symmetric cones. Most of lemmas in this section are cited from textbook [3] of Euclidean Jordan algebra by Faraut and Kor´ anyi. In Section 3, we introduce the linear programming over symmetric cones and MZ-family of search directions. The aim of Section 4 is to define neighborhoods of center path appropreately. In the course, we view several scaling invariant properties of the scaled points. In Section 5, we prove some more properties needed for the proof of complexity of the algorithm with the assumption that primal and dual variables are ‘commutative’. Then in Section 6, a subfamily of MZ-family is introduced and complexity of short and long step algorithms using that subfamily is shown. In Section 7, we define a subfamily of Commutative Class and prove polynomial complexity of algorithm using semi-long and long steps. Also we discuss how to calculate the search direction, and give an explicit formula for search direction of this class. 2. Euclidean Jordan algebra and Symmetric Cone In this section, we briefly review the structure of Euclidean Jordan algebra and its associated symmetric cone, which will be extensively used in the analysis of this paper. We strongly recommend Chapter I to IV of the book [3] by Faraut and Kor´ anyi for complete and detailed discussion on what is described in this section. A finite dimensional vector space V over R is called Jordan algebra if a multiplication ◦ is defined on V having the following properties: (J1) x ◦ y = y ◦ x (J2) [L(x2 ), L(x)] = 0 where L(x) is a linear transformation of V defined by L(x)y = x ◦ y and [A, B] = AB − BA. Note that in general x ◦ (y ◦ z) 6= (x◦ y)◦ z, but we can show that xk for a positive integer k is well-defined. Throughout the paper, we assume that V has an idenity element e satisfying e ◦ x = x ◦ e = x for all x ∈ V . Such identity element is unique.

2

If there exists a positive definite bilinear form (·|·) on V which is associative, namely, (L(x)y|z) = (y|L(x)z) for all x, y, z ∈ V,

(1)

then the Jordan algebra is Euclidean. In the following, we assume that V is a Euclidean Jordan algebra. Note that (1) implies that L(x) is a symmetric linear transformation with respect to (·|·) for any x ∈ V . A c ∈ V is called idempotent if c ◦ c = c. Idempotents c and c′ are orthogonal if c ◦ c′ = 0. Note that if two idempotents c and c′ are orthogonal in this sense, then (c|c′ ) = (L(c)c|c′ ) = (c|L(c)c′ ) = 0. Furthermore, we have the following lemma. Lemma 1. If c and c′ are two orthogonal idempotents, then [L(c), L(c′ )] = 0. Proof : This is Lemma IV.1.3 of Faraut and Kor´ anyi [3].



If an idempotent c cannot be expressed by a sum of other two non-zero idempotents, then c is primitive. We cannot choose an arbitrary number of primitive orthogonal idempotents. The number is obviously bounded by the dimension of V as a vector space. We denote the maximum possible number of primitive orthogonal idempotents by r, which is called rank of V . The rank of V is in general different from the dimension of V denoted by n in the following. Theorem 2 (Spectral Decomposition). For x ∈ V , there exists a set of orthogonal primitive idempotents {c1 , . . . , cr } and real numbers {λ1 , . . . , λr } such that x=

r X

λi ci .

i=1

Proof : This is Theorem III.1.2 of Faraut and Kor´ anyi [3].



In this paper, when we consider spectral deomposition, we always assume that kci k = (ci |ci ) = 1 for i = 1, . . . , r. The real numbers λ1 , . . . , λr and {c1 , . . . , cr } are frame P called spectral values2 and P Jordan 2 of x, respectively. If {c , . . . , c } is a Jordan frame, then e = c and we have kek = kc k = r. 1 r i i i i P If x = i λi ci is the spectral decomposition, then x2 = x ◦ x =

kxk2 =

r X

r X

λ2i ci ,

i=1

λ2i .

i=1

We also define x−1 :=

r X i=1

x1/2

λ−1 i ci if λi 6= 0 for all i.

r p X := λi ci if λi > 0 for all i. i=1

Obviously, we have x ◦ x = x ◦ x = e and x1/2 ◦ x1/2 = x. The symmetric cone Ω associated with V is the set of elements whose spectral values are all positive. Therefore, x1/2 exists for x ∈ Ω. The linear transformation P (x) := 2L(x)2 − L(x2 ) is called quadratic expression. The following properties of quadratic expression will be thoroughly used in the analysis of this paper. −1

−1

Proposition 3. 1. P (x) is invertible if and only if x is invertible. 2. P (x)−1 = P (x−1 ) if x is invertble. 3. P (x)x−1 = x if x is invertble. 4. P (x)1/2 = P (x1/2) if x ∈ Ω. 5. (P (x)y)−1 = P (x−1 )y−1 if x and y are invertible. 6. P (P (x)y) = P (x)P (y)P (x).

3

Proof : See Faraut and Kor´ anyi [3], Proposition II.3.1 and II.3.2 for all statements except 4. From 6, we have for x ∈ Ω, P (x) = P (P (x1/2)e) = P (x1/2 )P (e)P (x1/2) = P (x1/2 )2 . Since L(x) is symmetric, P (x) is also symmetric, thus P (x)1/2 = P (x1/2).  The automorphism group of Ω is defined by G(Ω) := { g ∈ GL(V ) | gΩ = Ω } ,

where GL(V ) is the general linear group of V . We denote the connected component of identity of G(Ω) by G in the following. Proposition 4. 1. For any x, y ∈ Ω, there exists g ∈ G such that gx = y. 2. g ∈ G can be decomposed as g = P (x)k where x ∈ Ω and k ∈ K := G ∩ O(V ). Here, O(V ) is the orthogonal group of V . Any k ∈ K is an automoriphim, which means that k is an invertible linear transformation of V such that kx ◦ ky = k(x ◦ y)

holds for any x, y ∈ V . 3. If g ∈ G, then P (gu) = gP (u)g∗ where g∗ is adjoint of g. 4. If g ∈ G, then (gx)−1 = (g∗ )−1 x−1 . Proof : See Faraut and Kor´ anyi [3], the discussion on page 5 for 1, Theorem III.5.1 for 2, Proposition III.5.2 for 3, and Theorem III.5.3 for 4.  Pr Theorem 5 (Pierce Decomposition). For x ∈ V , let x = i=1 λi ci be its spectral decomposition,

and Pij

Vii := { θci | θ ∈ R } , i = 1, . . . , r   1 1 Vij := y ∈ V L(ci )y = y, L(cj )y = y , 1 ≤ i < j ≤ r, 2 2 be the orthogonal projection onto Vij for i ≤ j. Then we have M V = Vij .

(2) (3)

(4)

i≤j

Furthermore, we have

Pii = P (ci ),

(5)

Pij = 4L(ci )L(cj ),

(6)

and L(x) = P (x) =

r X

i=1 r X

λi Pii +

X λi + λj i