Iterative Methods and Preconditioning for Large and Sparse Linear

Iterative Methods and Preconditioning for Large and Sparse Linear Systems with Applications

Contents

Preface

xi

List of Figures

xiii

List of Tables

xv

List of Algorithms

xvii

1 Introduction and Motivations 1.1 1.2 1.3

1

Notes on error analysis . . . . . . . . . . . . . . . . . . . . . Sparse matrices . . . . . . . . . . . . . . . . . . . . . . . . . On parallel computing and hardware acceleration . . . . . .

2 Some iterative algorithms for linear systems 2.1 2.2 2.3 2.4

21

Iterative algorithms and simple one-dimensional techniques . Krylov subspace algorithms, orthogonal projections: CG and GMRES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Krylov subspace algorithms, oblique projections: BiCG, QMR, CGS, BiCGstab, BiCGstab(l) . . . . . . . . . . . . . . . . . Which Krylov subspace method should I use? . . . . . . . .

3 General purpose preconditioning strategies 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8

Generalities on preconditioning . . . . . . . . . . . . . Krylov iterative methods for preconditioned iterations On Jacobi, SOR, and SSOR preconditioners . . . . . Incomplete factorization preconditioners . . . . . . . . Approximate inverse preconditioners . . . . . . . . . . Preconditioning sequences of linear systems . . . . . . Parallelizing preconditioners and available software . Which general purpose preconditioner should I use? .

2 10 16

22 37 57 70 75

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

76 78 89 91 114 142 158 161

ix

x

Iterative methods and preconditioning

4 Preconditioners for some structured linear systems 4.1 4.2 4.3 4.4

Toeplitz and block Toeplitz systems Notes on multigrid preconditioning Complex symmetric systems . . . . Saddle point systems . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

163 . . . .

. . . .

. . . .

. . . .

. . . .

A A Review of Numerical Linear Algebra A.1 Vector and matrix norms . . . . . . . . . . . . . . . . . . . . A.2 Eigenvalues and singular values . . . . . . . . . . . . . . . . B Data sets and software codes B.1 Test matrices and data sets . . . . . . . . . . . . . . . . . . . B.2 Numerical Linear Algebra software . . . . . . . . . . . . . . .

164 198 221 229 275 279 284 293 293 297

Bibliography

303

Index

351

Preface

This book on iterative methods for linear equations can be used as a tutorial and a reference for those who need to solve sparse and/or structured large linear systems of algebraic equations. It can also be used as a textbook for courses in iterative methods or as source material for a course in numerical analysis at the graduate level. It is assumed that the reader is familiar with calculus, elementary numerical analysis, linear algebra, and the main ideas of direct methods for the numerical solution of dense linear systems as described in, e.g., [270]. This text is completed with a bibliography with more than 550 titles where the interested reader can find more details and proofs of results considered here. The focus is on some real life applications and then on a certain number of methods suitable for them. This book can be used for a Master’s and/or Ph.D. level graduate course in science and engineering and can be useful to researchers, from any field of engineering, health, physical, chemical and biological sciences, etc. Chapter 1 contains some basic notions of error analysis and sparse matrices. In Chapter 2 there are basic facts on some fundamental iterative algorithms for linear systems, focusing in particular on Krylov subspace methods. Chapter 3 recalls the definition and some general purpose preconditioning strategies. Finally, in Chapter 4, a few examples of preconditioners for Toeplitz and Toeplitz–like, complex symmetric and saddle point systems are considered, together with some introductory notes on multigrid preconditioners. A short review of linear algebra is also included; see Appendix A. A number of computational examples and exercises are provided. The former are intended to validate the theoretical results and to give a sense of how the algorithms perform and not to give a complete picture of performance or to be a suite of test problems. Some test matrices and existing test sets are mentioned in Appendix B1. Finally, a few numerical linear algebra software packages and repositories are cited in Appendix B2, together with a short description of the codes shipped with the book and how to get them.

D. Bertaccini and F. Durastante

xi