The Matrix Cookbook. Kaare Brandt Petersen. Michael Syskind Pedersen.
Version: January 5, 2005. What is this? These pages are a collection of facts ...
The Matrix Cookbook Kaare Brandt Petersen Michael Syskind Pedersen Version: January 5, 2005 What is this? These pages are a collection of facts (identities, approximations, inequalities, relations, ...) about matrices and matters relating to them. It is collected in this form for the convenience of anyone who wants a quick desktop reference . Disclaimer: The identities, approximations and relations presented here were obviously not invented but collected, borrowed and copied from a large amount of sources. These sources include similar but shorter notes found on the internet and appendices in books - see the references for a full list. Errors: Very likely there are errors, typos, and mistakes for which we apologize and would be grateful to receive corrections at
[email protected]. Its ongoing: The project of keeping a large repository of relations involving matrices is naturally ongoing and the version will be apparent from the date in the header. Suggestions: Your suggestion for additional content or elaboration of some topics is most welcome at
[email protected]. Acknowledgements: We would like to thank the following for discussions, proofreading, extensive corrections and suggestions: Esben Hoegh-Rasmussen and Vasile Sima. Keywords: Matrix algebra, matrix relations, matrix identities, derivative of determinant, derivative of inverse matrix, differentiate a matrix.
1
CONTENTS
CONTENTS
Contents 1 Basics 1.1 Trace and Determinants . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Special Case 2x2 . . . . . . . . . . . . . . . . . . . . . . . . . 2 Derivatives 2.1 Derivatives 2.2 Derivatives 2.3 Derivatives 2.4 Derivatives 2.5 Derivatives
of of of of of
a Determinant . . . . . . . . . . . . an Inverse . . . . . . . . . . . . . . . Matrices, Vectors and Scalar Forms Traces . . . . . . . . . . . . . . . . . Structured Matrices . . . . . . . . .
3 Inverses 3.1 Exact Relations . . . . . 3.2 Implication on Inverses . 3.3 Approximations . . . . . 3.4 Generalized Inverse . . . 3.5 Pseudo Inverse . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
5 5 6
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
7 7 8 9 11 12
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
14 14 14 15 15 15
4 Complex Matrices 17 4.1 Complex Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 17 5 Decompositions 20 5.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . 20 5.2 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . 20 5.3 Triangular Decomposition . . . . . . . . . . . . . . . . . . . . . . 21 6 General Statistics and Probability 22 6.1 Moments of any distribution . . . . . . . . . . . . . . . . . . . . . 22 6.2 Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 7 Gaussians 7.1 One Dimensional . . . . . . 7.2 Basics . . . . . . . . . . . . 7.3 Moments . . . . . . . . . . 7.4 Miscellaneous . . . . . . . . 7.5 One Dimensional Mixture of 7.6 Mixture of Gaussians . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
24 24 25 27 29 29 30
8 Miscellaneous 8.1 Functions and Series . . . . . . . . . . . . . 8.2 Indices, Entries and Vectors . . . . . . . . . 8.3 Solutions to Systems of Equations . . . . . 8.4 Block matrices . . . . . . . . . . . . . . . . 8.5 Matrix Norms . . . . . . . . . . . . . . . . . 8.6 Positive Definite and Semi-definite Matrices
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
31 31 32 35 36 37 38
. . . . . . . . . . . . . . . . . . . . . . . . . . . . Gaussians . . . . . . . .
. . . . . .
Petersen & Pedersen, The Matrix Cookbook (Version: January 5, 2005), Page 2
CONTENTS
8.7 8.8
CONTENTS
Integral Involving Dirac Delta Functions . . . . . . . . . . . . . . Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A Proofs and Details
39 40 41
Petersen & Pedersen, The Matrix Cookbook (Version: January 5, 2005), Page 3
CONTENTS
CONTENTS
Notation and Nomenclature A Aij Ai Aij An A−1 A+ A1/2 (A)ij Aij a ai ai a
