Numerical properties of block Cholesky-like methods for solving symmetric quasidefinite linear systems Felicja Okulicka-Dłużewska, Ryszard Kozera, and Alicja Smoktunowicz
Citation: AIP Conference Proceedings 1978, 110005 (2018); doi: 10.1063/1.5043763 View online: https://doi.org/10.1063/1.5043763 View Table of Contents: http://aip.scitation.org/toc/apc/1978/1 Published by the American Institute of Physics
Numerical Properties of Block Cholesky-Like Methods for Solving Symmetric Quasi-Definite Linear Systems Felicja Okulicka-Dłu˙zewska1,a) , Ryszard Kozera2,3,4,b) and Alicja Smoktunowicz1,c) 1
Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, 00-662 Warsaw, Poland. 2 Faculty of Applied Informatics and Mathematics, Warsaw University of Life Sciences - SGGW, Nowoursynowska 159, 02-776 Warsaw, Poland. 3 School of Computer Science and Software Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, Perth, WA 6009, Australia. 4 Faculty of Computer Science, Vistula University, Stokłosy 3, 02-787 Warsaw, Poland. a)
Corresponding author:
[email protected] b)
[email protected] c)
[email protected]
Abstract. This paper presents a comparison of two Cholesky-like algorithms for solving symmetric quasi-definite system Mz = f . This pair of methods computes the block factorization M = RT DR, where R is upper triangular and D is a diagonal matrix.
INTRODUCTION We study the numerical properties of two block methods for solving the symmetric saddle point problem Mz = f , where ! ! ! A B x b M= , z= , f = . (1) y c BT −C Through the paper we assume that A ∈ Rm×m and C ∈ Rn×n are symmetric positive definite, B ∈ Rm×n , where n ≤ m. Note that M is nonsingular, so there exists a unique solution z∗ = (x∗T , yT∗ )T of Mz = f . We see that M is symmetric but is indefinite, in general. M is called a symmetric quasi-definite (sqd) by Vanderbei [1]. Such problems arise in many applications, e.g., in optimization, in the solution of PDEs, weighted least squares (image restoration), Finite Element (FE) formulations of consolidation problem ([1], [2], [3], and [4]). Vanderbei [1] used quasi-definite matrices within barrier methods for linear and quadratic programming. Vanderbei [1] proved that for any permutation P, PT MP has a ”Cholesky factorization” LDLT , where L is a unit lower triangular and D is a diagonal matrix with both positive and negative elements. However, such factorization may be unstable. Gill, Saunders, and Shinner [3] analyzed the numerical properties of methods using this decomposition. They applied the results of the Golub and Van Loan [5] (see also [6], p. 142) for positive definite systems (unsymmetric, in general). See Theorem 5.1 from [3], where the following bound of the relative error of the computed solution z˜ to the system (1) follows: k˜z − z∗ k ≤ L1 (m, n) ε M Φ(M). (2) kz∗ k Here ε M denotes machine precision, L1 (m, n) is slowly growing function on m and n, and Φ(M) is defined as follows Φ(M) = κ(M)(1 + ω(M)), where ω1 (M) =
kBT A−1 Bk , kMk
ω(M) = max {ω1 (M), ω2 (M)}, ω2 (M) =
kBC −1 BT k . kMk
International Conference of Numerical Analysis and Applied Mathematics (ICNAAM 2017) AIP Conf. Proc. 1978, 110005-1–110005-4; https://doi.org/10.1063/1.5043763 Published by AIP Publishing. 978-0-7354-1690-1/$30.00
110005-1
(3)
(4)
Throughout this paper, k · k is the matrix or vector two–norm depending upon context, and κ(M) = kM −1 k · kMk denotes the condition number of the matrix M. Notice that ω1 (M) and ω2 (M) are related to the Schur complements (−S ) = M\A of A in M, and (−G) = M\(−C) of −C in M, where S = C + BT A−1 B, G = A + BC −1 BT . Clearly, S is symmetric and positive definite. We have by [7] that kS k ≤ kMk(1 + ω1 (M)) and κ(S ) ≤ κ(M)(1 + ω1 (M)). It may happen that M is well-conditioned and A is ill-conditioned. The following example illustrates the latter. ! A B are defined as follows: Example 1 Let t > 1 and A, B, C and M = BT −C A=
t 1
1 1
! ,
B=
−1/t t
! ,
C = 1.
We see that M is symmetric quasi-definite matrix. Notice that for t = 1010 we have κ(A) ≈ 1010 and κ(M) ≈ 1.
ALGORITHMS We consider the following decomposition of the quasi-definite matrix M = RT DR: ! ! ! ! A B RT11 0 D1 0 R11 R12 M= = , 0 −D2 0 R22 BT −C RT12 RT22
(5)
where R11 , R22 are upper triangular and D1 , D2 are diagonal matrices. Algorithm I Compute the RT DR decomposition of M, where D1 and D2 are the identity matrices. ◦ Find the Cholesky decomposition A = RT11 R11 . ◦ Solve the triangular system RT11 R12 = B for R12 . ◦ Compute S = C + RT12 R12 . ◦ Find the Cholesky decomposition S = RT22 R22 . 2. Find a solution z to (1) by solving two triangular systems: RT w = f, Rz = D−1 w. 1.
Algorithm II Compute the RT DR decomposition of M, where R11 and R22 are upper unit triangular matrices. ◦ Find the Cholesky decomposition A = RT11 D1 R11 . ◦ Solve the triangular system RT11 D1 R12 = B for R12 . ◦ Compute S = C + RT12 D1 R12 . ◦ Find the Cholesky decomposition S = RT22 D2 R22 . 2. Find a solution z to (1) by solving two triangular systems: RT w = f, Rz = D−1 w. 1.
In practice, S can be ill-conditioned, i.e., the condition number κ(S ) = kS −1 k · kS k can be large. If the computed matrix S˜ in floating point arithmetic is numerically singular then Cholesky’s method can fail. For these reasons, the LDLT variant of Cholesky’s method (which avoids extracting square roots) for the matrices A and S may be preferred. If this method is efficiently implemented then it requires the same space and computational complexity as Cholesky’s LLT method. Algorithm I and Algorithm II have similar numerical properties. However, we can obtain a better bound than given in (2). If z˜ denotes the computed solution to Mz = f by either Algorithm I or Algorithm II and they run to completion then we get k˜z − z∗ k ≤ L2 (m, n) ε M κ(M)(1 + ω1 (M)), (6) kz∗ k where L2 (m, n) is slowly growing function on m and n, and ω1 (M) is given in (3)). Similar result in the error analysis is also proved for the partitioned Cholesky-like method [7].
110005-2
2
kS k kS k 22 k Notice that ω1 (M) is the same order as kMk . Unfortunately, the ratio kMk = kRkMk can be arbitrarily large. For ! 1 the ratio is of order −1 , since we have the following RT DR factorization example, for the matrix M = 1 −1 1 ! 1 1 2 0 − 2 1 0 2 1 1 M = − 1 . −1 2 −1 2 0 −1 2 (1 + ) 0 (1 + )
NUMERICAL EXPERIMENTS Numerical tests were performed in MATLAB, version 8.4.0.150421 (R2014b), with ε M ≈ 2.2 · 10−16 . Let z∗ be the exact solution to Mz = f of (1) and let z˜ be the computed approximation to z∗ . In all experiments, we chose z∗ = (xT , yT )T by MATLAB commands x = randn(m, 1) and y = randn(n, 1). The distribution is N(0, 1) and the MATLAB command randn(’state’,0) is used to reset the random number generator to its initial state. In sequel we computed the vector f = Mz∗ and find a numerical solution z˜ first by Algorithm I and next by Algorithm II. We use here the forward reduction and back substitution methods for solving triangular systems RT w = f, Rz = D−1 w. We report the following statistics for the algorithms: ˜ kM−R˜ T DRk kMk
•
dec =
•
res(˜z) =
kM˜z− f k kMk k˜zk
(the decomposition error), bound dec = ε M (the backward stability error), stab(˜z) =
kS˜ k kMk ,
k˜z−z∗ k κ(M) kz∗ k
(the forward stability error).
We used the following test matrices: 1.
P = pascal(k) is a k × k Pascal matrix defined by P = (pi j ), pi j =
2.
(i + j − 2)! , i, j = 1, . . . , k, (i − 1)!( j − 1)!
H = hilb(k) is a k × k Hilbert matrix defined by H = (hi j ), hi j =
1 , i = 1, . . . , k, j = 1, . . . , k. i+ j−1
Example 2 For generating A(m × m), B(m × n), C(n × n), and the vectors x(m × 1) and y(n × 1) we used the following MATLAB code: randn(’state’,0); A=pascal(m);C=pascal(n); B=t*invhilb(n); x=randn(m,1);y=randn(n,1); The results for m = n = 10 are given in Tables 1 and 2. Here 0 , t ∈ R is a parameter that can be used to produce B to have a different magnitude, and MATLAB function invhilb generates the inverse of the Hilbert matrix. We observe that in the above example A, C and M are ill-conditioned. When we use Algorithm I for t = 10−6 , then the Cholesky algorithm breaks down because the computed Schur complement S˜ is not positive definite in working precision.
CONCLUSION • • • •
We analyzed two block methods for solving symmetric quasi-definite linear systems (1). Algorithm I is based on the LLT Cholesky factorization for symmetric positive definite matrices. Extensive numerical experiments in MATLAB indicate that Algorithm II which uses the LDLT variant of Cholesky’s method may be preferred. In order to improve the results we can use iterative refinement methods [8]. Possible applications of Algorithms I and II can be found e.g. in [9].
110005-3
TABLE 1. Results for Example 2 and Algorithm I. Here κ(A) = κ(C) = 4.15 · 109 .
t
1e-10
1e-09
1e-08
1e-07
1e-06
κ(M) κ(S˜ ) dec bound dec ε M ω1 ε M ω2 res(˜z) stab(˜z)
1.00e+07 2.04e+12 1.83e-11 6.86e-11 6.86e-11 6.86e-11 8.36e-12 1.71e-13
2.93e+06 8.52e+13 1.00e-09 6.86e-09 6.86e-09 6.86e-09 4.84e-10 1.21e-11
3.74e+06 7.85e+15 1.71e-07 4.84e-07 4.84e-07 4.84e-07 9.73e-08 2.04e-09
1.47e+07 2.77e+17 1.31e-06 4.84e-06 4.84e-06 4.84e-06 5.93e-07 1.68e-08
8.29e+07 8.18e+18 4.84e-05 4.84e-05 -
TABLE 2. Results for Example 2 and Algorithm II. Here κ(A) = κ(C) = 4.15 · 109 .
t
1e-10
1e-09
1e-08
1e-07
1e-06
κ(M) κ(S˜ ) dec bound dec ε M ω1 ε M ω2 res(˜z) stab(˜z)
1.00e+07 2.04e+12 3.09e-11 4.00e-12 6.86e-11 6.86e-11 8.26e-12 2.23e-13
2.93e+06 8.52e+13 3.44e-09 8.06e-10 6.86e-09 6.86e-09 1.15e-09 3.47e-11
3.74e+06 7.81e+15 2.88e-07 7.58e-10 4.84e-07 4.84e-07 6.62e-08 2.32e-12
1.47e+07 2.23e+17 3.58e-06 7.61e-11 4.84e-06 4.84e-06 3.46e-07 3.90e-09
8.29e+07 7.30e+18 2.3e-05 7.62e-12 4.84e-05 4.84e-05 1.33e-06 1.17e-07
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9]
R. J. Vanderbei, SIAM J. Optim. 5, 100–113 (1995). M. Benzi, G. H. Golub, and J. Liesen, Acta Numer. 14, 1–137 (2005). P. E. Gill, M. A. Saunders, and J. R. Shinnerl, SIAM J. Matrix Anal. Appl. 17, 35–46 (1996). F. Okulicka-Dłu˙zewska, Lecture Notes Comput. Sc. 4310, 100–105 (2007). G. H. Golub and C. F. V. Loan, Linear Algebra Appl. 28, 85–97 (1979). G. H. Golub and C. F. V. Loan, Matrix Computations (The Johns Hopkins University Press, Baltimore and London), pp. 142–143. M. Rozloˇzn´ık, F. Okulicka-Dłu˙zewska, and A. Smoktunowicz, SIAM. J. Matrix Anal. & Appl. 36(2), 727– 757 (2015). A. Smoktunowicz and A. Smoktunowicz, Appl. Numer. Math. 67, 220–229 (2013). L. Noakes and R. Kozera, Lecture Notes Comput. Sc. 2616, 419–436 (2003).
110005-4
