MAX-MIN EIGENVALUE PROBLEMS, PRIMAL-DUAL INTERIOR POINT ALGORITHMS, and TRUST REGION SUBPROBLEMS Franz Rendl
Robert J. Vanderbei
y
Henry Wolkowicz
z
University of Waterloo Department of Combinatorics and Optimization Faculty of Mathematics Waterloo, Ontario N2L 3G1, Canada e-mail
[email protected] November 13, 1993 Technical Report CORR 93-30
Abstract
Two Primal-dual interior point algorithms are presented for the problem of maximizing the smallest eigenvalue of a symmetric matrix over diagonal perturbations. These algorithms prove to be simple, robust, and ecient. Both algorithms are based on transforming the Technische Universit at Graz, Institut fur Mathematik, Kopernikusgasse 24, A-8010 Graz, Austria. Research support by Christian Doppler Laboratorium fur Diskrete Optimierung. y Program in Statistics & Operations Research, Princeton University, Princeton, NJ 08544. Research support by AFOSR through grant AFOSR-91-0359. z Research support by the National Science and Engineering Research Council Canada.
problem to one over the cone of positive semide nite matrices. One of the algorithms does this transformation through an intermediate transformation to a trust region subproblem. This allows the removal of a dense row. Key words: Max-min eigenvalue problems, trust region subproblems, Loewner partial order, primal-dual interior point methods, quadratic 0,1 programming, graph bisection. AMS 1991 Subject Classi cation: Primary 65K15, Secondary 49M35, 90C48.
1
1 INTRODUCTION Consider the max-min eigenvalue problem (MMP ) ! := vmax (C b ? Diag (v)); t e=0 min
(1.1)
where C b is an (n +1) (n +1) real symmetric matrix, v 2 0 is the step length. For X , when the error jjZ ?1 ? X jj is small, then a reasonable heuristic is to check Z + tDiag (y) 0; for X: Let Z = RtR be the Cholesky factorization of Z . Then the above is equivalent to nding t > 0 such that I tR?t(Diag (y))R?1 0: With this change, the problem is equivalent to the step length problem in ordinary linear programming, i.e. suppose that the matrix Diag (y) is not negative semide nite (in which case the step length can be in nite and so is chosen to be 1). Then we get that the step for Z is 1 tZ := minf1; (R?t (Diag (y))R?1) g; max while, if the matrix Diag (y) is not positive semide nite (in which case the step length can be in nite and so is chosen to be 1) the step for X is 1 tX := minf1; ? (R?t (Diag (y))R?1) g: min By the above we see that we can use any upper bound for the maximum eigenvalue and any lower bound for the minimum eigenvalue. Note that for an n n symmetric matrix K , see [20], s 2 p trace K K 2 ; max(K ) n + n ? 1 tracen K ? trace n 15
s
2 trace K 2 p trace K trace K min(K ) n ? n ? 1 : n ? n We can nd the traces using Z ?1, e.g.
trace (R?t (Diag (y))R?1) = trace Diag (y)Z ?1 = ytdiag (Z ?1): Alternatively, to nd the bounds, we could rst shift R?t (Diag (y))R?1 by a multiple of the identity to guarantee that it is positive de nite and then calculate its norm. We then shift by a multiple of the identity to guarantee that it is negative de nite and nd the norm of the negative of the matrix. By shifting back, we get the values for the largest and smallest eigenvalues. Alternatively, we could calculate the largest and smallest eigenvalues of Diag (y)Z ?1 directly. Note that if tX = 1, then X is a good step for X . However, if tX < 1, then X + tX X is not necessarily equal to Z ?1(Z + Diag (tX y))Z ?1: This situation almost never arose in practice. But an extra safeguard was added to ensure that X remain positive de nite. After nding these maximum steplengths, we then multiply the step length by .90 to guarantee that we do not get too close to the boundary. Because of the nonlinearity in algorithm 1, we then take one steplength for both variables; while algorithm 2 uses di erent steplengths in the primal and dual variables. From numerical tests, it appears that the step length 1 for X is usually not too large, i.e. it does not usually lose positive de niteness. But, the step length for Z is much too large immediately after is decreased. When is not decreased, the step length 1 seems good for both variables. An ecient line search could be done using a Lanczos type algorithm to calculate the smallest eigenvalues of Cy ? y; Cy + y. Or inverse iteration can be used so the matrices from the Cholesky factorization do not get inverted.
4.3 THE ALGORITHM
The algorithms presented above di er only in the optimality conditions that Newton's method solves. We present the following outline for the algorithm 1: 16
Algorithm 4.1
Suppose that C b is given. Find initial estimate y so that Z = C ? Diag (y) is positive de nite and well conditioned. Find an initial and set X = Z ?1 so that X is well conditioned, e.g. set = jjZjjZ?jj1jj : Repeat the following steps.
1. If the convergence criteria is satis ed, e.g. if < given tolerance, then:
(a) If X is not positive de nite, then recalculate it using (4.12) with tX y. XZ ) jje?diag(xy xty )?diag(X )jj (b) Recalculate = maxf trace( g; to ensure 10n ; 10n primal feasibility. (c) STOP if the convergence criteria, < given tolerance, is still satis ed. 2. Solve the Newton system for the directions y; Z; X: 3. Find a step length that preserves positive de niteness for the primal and dual variables X; Z , and update the variables. 4. Check the error in the current Newton system with the current . If this XZ ) error has decreased from the last iteration, then update = trace( 10n :
5 CONCLUSION We have derived two primal-dual algorithms for maximizing the minimum eigenvalue of a diagonally perturbed symmetric matrix. Both algorithms have been tested extensively. Test results for the second algorithm are given in [14]. The results for the rst algorithm, which is based on the trust region subproblem, were similar. Therefore, if the matrix has a dense row, the rt algorithm should be used since the dense row can be eliminated.
References [1] F. ALIZADEH. Combinatorial optimization with semide nite matrices. In Proceedings of the Second Annual Integer Programming and Combinatorial Optimization Conference, Carnegie-Mellon University, 1992. 17
[2] J. CULLUM, W.E. DONATH, and P.WOLFE. The minimization of certain nondi erentiable sums of eigenvalues of symmetric matrices. Mathematical Programming Study, 3:35{55, 1975. [3] J. FALKNER, F. RENDL, and H. WOLKOWICZ. A computational study of graph partitioning. Technical Report CORR, Department of Combinatorics and Optimization, Waterloo, Ont, 1992. Submitted to Math. Progr. [4] D.M. GAY. Computing optimal locally constrained steps. SIAM J. Sci. Statist. Comput., 2:186{197, 1981. [5] R. HORN and C. JOHNSON. Matrix Analysis. Cambridge University Press, New York, 1985. [6] R. HORN and C. JOHNSON. Topics in Matrix Analysis. Cambridge University Press, New York, 1991. [7] A. P. KAMATH and N. K. KARMARKAR. A continuous method for computing bounds in integer quadratic optimization problems. Journal of Global Optimization, 2(3):229{241, 1992. [8] Y. E. NESTEROV and A. S. NEMIROVSKY. Interior Point Polynomial Methods in Convex Programming : Theory and Algorithms. SIAM Publications. SIAM, Philadelphia, USA, 1993. [9] D. OUELLETTE. Schur complements and statistics. Linear Algebra and Its Applications, 36:187{295, 1981. [10] M.L. OVERTON. On minimizing the maximum eigenvalue of a symmetric matrix. SIAM J. Matrix Analysis and Applications, 9:256{268, 1988. [11] M.L. OVERTON. Large-scale optimization of eigenvalues. SIAM J. Optimization, 2:88{120, 1992. [12] M.L. OVERTON and R.S. WOMERSLEY. Second derivatives for optimizing eigenvalues of symmetric matrices. Technical Report 627, Computer Science Department, NYU, 1993. 18
[13] S. POLJAK and H. WOLKOWICZ. Convex relaxations of 0-1 quadratic programming. Technical Report 93-18, DIMACS, 1993. [14] F. RENDL, R. J. VANDERBEI, and H. WOLKOWICZ. A primal-dual interior point method for the max-min eigenvalue problem. Technical Report CORR 93-20, SOR 93-15, Department of Combinatorics and Optimization, Waterloo, Ont, 1993. [15] D.C. SORENSEN. Trust region methods for unconstrained minimization. In M.J.D. Powell, editor, Nonlinear Optimization 1981. Academic Press, London, 1982. [16] R.J. STERN and H. WOLKOWICZ. Inde nite trust region subproblems and nonsymmetric eigenvalue perturbations. Technical Report CORR 92-38, University of Waterloo, Waterloo, Canada, 1992. [17] L. VANDENBERGHE and S. BOYD. Primal-dual potential reduction method for problems involving matrix inequalities. Technical report, Electrical Engineering Department, Stanford University, Stanford, CA 94305, 1993. [18] H. WOLKOWICZ. Some applications of optimization in matrix theory. Linear Algebra and its Applications, 40:101{118, 1981. [19] H. WOLKOWICZ. Explicit solutions for interval semide nite linear programming. Research Report CORR 93-29, University of Waterloo, 1993. [20] H. WOLKOWICZ and G.P.H. STYAN. Bounds for eigenvalues using traces. Linear Algebra and its Applications, 29:471{506, 1980. [21] B. YANG and R. J. VANDERBEI. The simplest semide nite programs are trivial. Technical Report SOR-93-??, Program in Statistics & Operations Research, Princeton University, Princeton, NJ, 1993.
19
