The singular values are ordered s1 ¸ s2 ¸ ::: ¸ sK ¸ 0, where K = minfM;Ng. The uk's .... [5] Jain, Anil K. Fundamentals of Digital Image Processing, Prentice Hall, ...
Matrix Expansion by Orthogonal Kronecker Products Je®ery C. Allen The American Mathematical Monthly June-July 1995, page 538{540 Introduction. The singular-value decomposition (SVD) provides an expansion of a real M £ N matrix A by orthogonal outer products [3, page 144]: A=
K X
sk uk vTk :
k=1
The singular values are ordered s1 ¸ s2 ¸ : : : ¸ sK ¸ 0, where K = minfM; N g. The uk 's and the vk 's are the orthonormal right and left singular vectors: uTk uk0
=
vTk vk0
=
(
1 k = k0 ; 0 k= 6k 0
where uT denotes the vector transpose. This note demonstrates that by a simple rescanning of the matrix A, the SVD can also produce a variety of expansions in terms of orthogonal Kronecker products. Notation. Vectors are denoted by a bold lowercase letter. The \vec" of a matrix is the vector that results by stacking the columns. For example, vec([x1 ; x2 ]) =
"
x1 x2
#
:
Mat(M; N ) denotes the linear space of real M £N matrices. BlkMat(M1 ; N1 ; M2 ; N2 ) denotes the liner space of real M1 £ N1 block matrices with M2 £ N2 blocks. The inner-product of two matrices is given by hU; V i = vec(U)T vec(V ). The Kronecker product of two matrices is U − V = [um;n V ] [3, page 243]. For example, "
u1;1 u1;2 u2;1 u2;2
#
−V =
"
u1;1 V u2;1 V
u1;2 V u2;2 V
#
:
Note that in U − V , every element of U multiplies every element of V . Thus, the Kronecker product contains the same products as the outer product vec(V )vec(U )T . The map taking Kronecker products to outer products is the rescanning function. 1
The Rescanning Function. Let M1 , M2 , N1 , and N2 be positive integers. Use these to de¯ne the mapping block : Mat(M2 N2 ; M1 N1 ) ! BlkMat(M1 ; M2 ; N1 ; N2 ) 2 6 6
block([a1 ; : : : aM1 N1 ]) = 6 6 4
A1 A2 .. .
AM1 +1 AM1 +2 .. .
AM1 A2M1
::: ::: .. .
A(N1 ¡1)M1 +1 A(N1 ¡1)M1 +2 .. .
: : : AN1 M1
3
7 7 7; 7 5
where each block Ak is a real M2 £ N2 matrix determined from ak by vec(Ak ) = ak . Thus, block merely rescans a matrix of size M2 N2 £ M1 N1 into an M1 £ N1 block matrix with M2 £ N2 blocks. It is straight-forward to verify the following: (B-1) block is a linear mapping which is one-to-one and onto. (B-2) block preserves the matrix inner product. (B-3) block maps outer products onto Kronecker products. To illustrate the last claim, let U 2 Mat(M1 ; N1 ) and V 2 Mat(M2 ; N2 ). Then A = U − V is a M1 £ N1 block matrix with M2 £ N2 blocks Am+(n¡1)M1 = um;n V for m = 1,. . . , M1 and n = 1,. . . , N1 . In matrix form, this can be written block(vec(V )vec(U)T ) = U − V: Since block is an isometry mapping outer products to Kronecker products, it should come as no surprise that block lifts the SVD expansion to the Kronecker expansion. The Kronecker Expansion. Let A 2 Mat(M; N ) where M = M1 M2 and N = N1 N2 . Let K = minfM1 N1 ; M2 N2 g. Then there are matrices U1 , . . . , UK 2 Mat(M1 ; N1 ), matrices V1 , . . . , VK 2 Mat(M2 ; N2 ), and numbers s1 ¸ : : : ¸ sK ¸ 0 such that A=
K X
sk Uk − Vk
k=1
where the following orthogonality conditions hold: hUk ; Uk0 i = hVk ; Vk0 i =
(
1 k = k0 : 0 k= 6k 0
Proof: By (B-1), block is invertible. Set A = block¡1 (A). Write the SVD of matrix A in the form A=
K X
sk vk uTk :
k=1
2
Let Uk 2 Mat(M1 ; N1 ) be determined from vec(Uk ) = uk . Likewise, let Vk 2 Mat(M2 ; N2 ) be determined from let vec(Vk ) = vk . Then by (B-1) and (B-3), the Kronecker expansion is obtained A = block(A) =
K X
sk block(vk uTk )
k=1
=
K X
sk Uk − Vk :
k=1
By (B-2), it is straight-forward to verify the orthogonality of the Uk 's and the Vk 's. Remarks. Since the SVD also determines optimal reduced rank approximations, best approximations using a ¯xed number of Kronecker products can be obtained from this Kronecker expansion. The motivation for this Kronecker expansion or \block" SVD arose in an image-processing application. Illuminating discussions of image processing and applications of the SVD, block matrix computations, and Kronecker products are found in Gonzalez and Wintz [6] or Jain [5]. An excellent treatment of the Kronecker product is found in Horn and Johnson [3]. Further generalizations of the Kronecker product and signal-processing applications are found in [4], [7], [2], [1].
References [1] Andrews, H. C. and J. Kane, Kronecker Matrices, Computer Implementation, and Generalized Spectra, Journal of the Association for Computing Machinery, Volume 17, Number 2, (1970) 260{268 [2] Henderson, Harold V. and S. R. Searle, The Vec-Permutation Matrix, the Vec Operator and Kronecker Products: A Review Linear and Multilinear Algebra, Volume 9 (1981), 271{288 [3] Horn, Roger A. and Charles R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, (1991) [4] Hyland, David C. and Emmanuel G. Collins, Jr., Block Kronecker Products and Block Norm Matrices in Large-Scale Systems Analysis, SIAM Journal of Matrix Analysis and Applications, Volume 10, Number 1, (1989) 18{29 [5] Jain, Anil K. Fundamentals of Digital Image Processing, Prentice Hall, (1989) [6] Gonzalez, Rafael C. and Paul Wintz, Digital Image Processing, Addison-Wesley Publishing Company, (1977) [7] Regalia, Phillip A. and Sanjuit K. Mitra, Kronecker Products, Unitary Matrices and Signal Processing Applications, SIAM Review, Volume 31, Number 4, (1989) 586{613
3
