Computation of Condition Numbers for linear programming problems using Pe˜ na’s Method Joo-Siong Chai ∗, and Kim-Chuan Toh
†
March 2, 2005
Abstract We present the computation of the condition numbers for linear programming (LP) problems from the NETLIB suite. The method of Pe˜ na [Technical report, Center for Applied Mathematics, Cornell University, May 1998] was used to compute the bounds on the distance to ill-posedness ρ(d) of a given problem instance with data d, and the condition number was computed as C(d) = kdk/ρ(d). We discuss the efficient implementation of Pe˜ na’s method and compare the tightness of the estimates on C(d) computed by Pe˜ na’s method to that computed by the method employed by Ord´on ˜ ez and Freund [SIAM J. Optimization, 14 (2003), pp. 307–333]. While Pe˜ na’s method is generally much cheaper, the bounds provided are generally not as tight as those computed by Ord´on ˜ ez and Freund. As a by-product, we use the computational results to study the correlation between log C(d) and the number of IPM iterations taken to solve a LP problem instance. Our computational findings on the preprocessed problem instances from NETLIB suite are consistent with those reported by Ord´on ˜ ez and Freund. Keywords: Conic optimization, interior-point methods, complexity, Renegar condition number, linear programming AMS subject classification: 65F15, 65K05, 90C25
1
Introduction
Ever since Renegar [13] introduced the distance to infeasibility and condition measure of a conic linear system, a number of authors have been studying their properties and connections to some behavioral characteristics of the optimization problems involving the conic system, such as [1], [3], [6], [7], [10], [11], [12]. ∗
High Performance Computing for Engineered Systems (HPCES), Singapore-MIT Alliance, 4 Engineering Drive 3, Singapore 117576. (
[email protected]). † Department of Mathematics, National University of Singapore, 2 Science Drive 2, Singapore 117543, Singapore (
[email protected]).
1
In [12] the authors proposed an algorithm for computing solutions to a conic system Ax = b,
x ∈ C,
(1)
where C is a closed convex cone. This problem was reformulated as an optimization problem to be solved via a suitable interior-point method (IPM). The algorithm in [12] was later generalized to general conic systems in [11]: Ax − b ∈ CY ,
x ∈ CX ,
where CX and CY are closed convex cones. Building on the key results developed in [10], [12], [13], Pe˜ na [9] proposed a scheme to effectively estimate the distance to infeasibility of the conic linear system (1). In particular, the scheme involves solving an analytic center problem and then approximates the distance to infeasibility along the direction of an eigenvector corresponding to the minimum eigenvalue associated with the Hessian of a self-concordant barrier function evaluated at the analytic center. More recently, in [7], Ord´on ˜ ez and Freund (abbreviated as OrF) extended the theory of condition measures and their properties from the conic format to handle more general structured convex optimization in the following ground-set format: n
o
min cT x : Ax − b ∈ CY , x ∈ P ,
(2)
where P can be any closed convex set. In addition, they developed an approach to compute the distance to ill-posedness ρ(d) for linear optimization data instances d = (A, b, c) of the ground-set model, as well as estimating C(d). In particular, they present a method to compute ρ(d) for linear optimization problems by solving 2n + 2m LPs of size roughly that of the original problem. The computational cost in estimating C(d) can thus be extremely expensive, especially when n + m is large. The main difference between Pe˜ na’s method and OrF’s method for computing the condition number C(d) of an LP instance with data d = (A, b, c) lies in the norms used in computing ρ(d) and kdk. To distinguish between quantities computed by these two methods, we will put a superscript “J” onto quantities computed by Pe˜ na’s method, and a superscript “F ” for those computed by OrF’s method. The main objective of this paper is to present our work on computing the condition numbers for LP problems from the NETLIB suite [5]. We used Pe˜ na’s method to compute the bounds on the distance to ill-posedness ρJ (d), and compute the condition number as C J (d) = kdkJ /ρJ (d). The advantage of this method is that the computational cost is much more moderate compared to the method of Ord´on ˜ ez and Freund [7]. Specifically, regardless of how large m + n is, only 6 second order cone programs with roughly the same size and structure as the original LP need to be solved. While Pe˜ na’s method is computationally attractive, its efficient implementation is nevertheless more involved than OrF’s method. In particular, the method involves solving nonlinear conic programs, in contrast to OrF’s method which requires only the solution of numerous LPs. Furthermore, in estimating ρ(d), Pe˜ na’s method requires the computation of the smallest eigen-pairs of certain large, possibly ill-conditioned, symmetric matrices. Our paper will address the implementation of these additional computational complexities in Pe˜ na’s method. A natural question one might ask is whether the estimates produced by Pe˜ na’s method and OrF’s method on C(d) are comparable in their tightness. We find that
2
the bounds computed by Pe˜ na’s method are generally not as tight as those computed by OrF’s method. For the former method, the ratios between the bounds are generally greater than 2 (but less than 100), whereas for the latter method, the ratios are generally less than 2. Thus, although OrF’s method is much more expensive than Pe˜ na’s method in computing bounds for condition numbers, it does have the advantage that the bounds provided are much tighter. As a by-product, we use the computational results to study the connection between C J (d) and some behavioral characteristics of LP, such as the number of IPM iterations taken to solve a LP problem. Our findings are consistent with those reported in [7]. The paper is organized as follows. In section 2, we review some relevant theoretical results which lead to the method that we used in our computation of the distance to ill-posedness for LP problems. We then show in section 3 how to apply Pe˜ na’s method to approximate the distance to primal and dual infeasibility for a given LP problem, and then use these approximations to compute the condition number for the LP problem. In section 4, we derive a relationship between C J (d) and C F (d) based on standard Lp norm inequalities. In section 5, we address numerical issues arising from the computation ρJ (d). This is followed by a discussion of our computational results and findings in section 6. Finally, we conclude the report in section 7. For a vector x, kxk2 denotes the q vector 2-norm. For a matrix M , kM k2 denotes the matrix 2-norm defined by kM k2 = eigenvalue.
2
λmax (M M T ), where λmax (·) denotes the largest
Geometry of distance to infeasibility
It has been shown in [12] that a certain relationship exists between the distance to infeasibility of the conic system (1) and the analytic center of a homogenized form of the system. In this section, we shall review some of the relevant theoretical development and results which ultimately lead to the method we used in our computation of the distance to ill-posedness for the LP problems in the NETLIB suite.
2.1
Analytic center problem
Pe˜ na and Renegar [12] derived an optimization problem corresponding to the conic system (1) by means of homogenization and relaxation. Homogenizing the conic system (1) gives Ax − tb = 0 kxk22 + t2 ≤ 1 x ∈ C, t ≥ 0. Note that under the mapping (x, t) → x/t, the solutions of this homogeneous system for which t 6= 0 are in one-to-one correspondence with the solutions of (1). By introducing a relaxation variable y and a positive variable δ whose value bounds kyk2 , the system is thus reformulated as the following optimization problem (with zero
3
optimal objective value): min
δ
s.t.
Ax − tb = y kxk22 + t2 ≤ 1 kyk2 ≤ δ x ∈ C, t ≥ 0.
x,t,y,δ
(3)
Let fC (x) be a self-concordant barrier function for int C, and f (x, t) := fC (x) − ln(t) − ln(1 − kxk22 − t2 ).
(4)
Note that, by Proposition 3 of [11], f (x, t) is a self-concordant barrier function for {(x, t) : kxk22 + t2 < 1, x ∈ int C, t > 0}. By Proposition 6 in [12], if (1) is well-posed (i.e. has strictly feasible solutions) then the central path for (3) converges to the analytic center (¯ x, t¯) of the bounded region F = {(x, t) : Ax = tb, kxk22 + t2 < 1, x ∈ int C, t > 0}.
(5)
That is, (¯ x, t¯) is the minimizer of f (x, t) in the subspace {(x, t) : Ax = tb}. However, in practical implementation, an IPM solver might not follow the central path of (3) exactly, and thus might not converge to the analytic center of F if (3) has more than one solution. Hence, to solve (3), we instead solve the following analytic center problem directly: min {f (x, t) : Ax = tb} .
2.2
(6)
Distance to infeasibility
There is a close connection between the analytic center (¯ x, t¯) of F and the distance to infeasibility of the conic system (1). Define the distance to infeasibility ρJ (d) of (1) as the smallest perturbation of the data d = (A, b) that yields an infeasible system. That is, ρJ (d) = inf{k∆A, ∆bk2 : (A + ∆A, b + ∆b) ∈ I},
(7)
where I = {(A, b) : the system (1) is infeasible}. The distance to infeasibility yields a notion of condition number that has proven to be useful in convex optimization, see [1], [3], [6], [7], [10], [11], [12]. Assuming that the conic system (1) is well-posed (i.e. ρJ (d) > 0), the analytic center (¯ x, t¯) exists and is related to ρJ (d) as follows (Thm 2 [12]). Proposition 2.1 Assume ρJ (d) > 0 and let z¯ = (¯ x, t¯) be the analytic center of F in (5). Then q ρJ (d) z )−1 [A − b]T ) ≤ ρJ (d), ≤ λmin ([A − b]H(¯ 4ϑ + 1
where H denotes the Hessian of f , a self-concordant barrier function for {(x, t) : kxk22 + t2 < 1, x ∈ int C, t > 0}, and ϑ denotes the barrier parameter of f .
4
As it turns out, the multiplier 4ϑ+1 in Proposition 2.1 can be replaced by ϑ. While the factor is 4ϑ+1 according to the general theory of self-concordance as used by Pe˜ na, the factor can be reduced when C is a self-scaled cone. And according to the theory of self-scaled cones [4], we can use the factor ϑ instead.
2.3
Geometric properties
The method that we used to approximate the distance to infeasibility is due to Pe˜ na [9], which is motivated by the geometric intuition underlying the following characterization of the distance to infeasibility originally due to Renegar [14]: ρJ (d) = inf{kyk2 : Ax − tb = y, x ∈ C, t ≥ 0, kxk22 + t2 ≤ 1 is inconsistent}. This result can be interpreted geometrically as follows: consider the set S = {Ax − tb : x ∈ C, t ≥ 0, kxk22 + t2 ≤ 1}. The above characterization states that ρJ (d) is the Euclidean distance from the origin to the boundary of S. Given a fixed vector v ∈ 0. is related to the restricted distance ρJ1 (A, b) as follows (cf. Prop 13 [11]). Proposition 2.3 Assume A is such that ρJ1 (A, b) > 0 and {(x, t) : Ax−tb = 0, kx1 k+ t2 ≤ 1, x ∈ C, t ≥ 0} is nonempty and bounded. Let z¯ = (¯ x, t¯) be the analytic center of (10). Then ρJ1 (A, b) q z )−1 [A − b]T ) ≤ ρJ1 (A, b), ≤ λmin ([A − b]H1 (¯ 4ϑ + 1 where H1 denotes the Hessian of f1 , and ϑ denotes the barrier parameter of f1 . Again, the multiplier 4ϑ + 1 in Proposition 2.3 can be replaced by ϑ when C is a self-scaled cone. Renegar’s characterization of the distance to infeasibility may be extended to the restricted version in a straightforward manner: ρJs (d) = inf{kyk2 : [A1 A2 ]
"
x1 x2
#
− tb = y, x ∈ C, t ≥ 0, kx1 k22 + t2 ≤ 1 is inconsistent}.
Accordingly, Proposition 2.2 may be extended to the restricted case.
6
Proposition 2.4 Given v ∈
