Goal: Reliable algorithm for calculating ranks, null spaces and pseudoinverse solutions for large sparse matrices in the presence of errors. I. Numerical rank ...
CALCULATING RANKS, NULL SPACES AND PSEUDOINVERSE SOLUTIONS FOR SPARSE MATRICES USING SPQR Leslie Foster Department of Mathematics, San Jose State University
October 28, 2009, SIAM LA 09
L ESLIE F OSTER D EPARTMENT OF M ATHEMATICS , S AN J OSE S TATE U NIVERSITY CALCULATING RANKS, NULL SPACES AND PSEUDOINVER
O UTLINE Goal: Reliable algorithm for calculating ranks, null spaces and pseudoinverse solutions for large sparse matrices in the presence of errors I. II. III. IV. V.
Numerical rank, Numerical Null Space, . . . Tools: SPQR, subspace iteration Algorithm Numerical Experiments Conclusions
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
N UMERICAL R ANK , N UMERICAL N ULL S PACE
For m × n matrix A and a tolerance tol Numerical rank: r = no. of singular values > tol Numerical Null Space Basis: n × (n − r ) matrix with orthonormal columns with ||AX|| tol T References: Chan, Vogel, Gotsman, Toledo L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
A LGORITM SPQR_NULL
SPQR_NULL: returns accurate numerical rank, orthogonal null space basis, and pseudoinverse solution “accurate numerical rank” means that when the estimated numerical rank is not correct, a flag is returned warning the user
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
A LGORITHM SPQR_NULL, CONT ’ D Input: m × n matrix A and tolerance tol Use SPQR twice to construct approximate complete orthogonal decomposition b 0 T A = Q1 P2 Q2T P1T + E, 0 0 with Q1 , Q2 , P1 , P2 orthogonal. Know ||E||F . b for numerical null Use SSI to find ⊥ basis X b. T b usually has a small, often 0, space of T nullity. L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
A LGORITHM SPQR_NULL, CONT ’ D The ⊥ nullspace basis of A is b 0 X . Save in factored form. X = P1 Q2 0 I Use singular value estimates and ||E||F to confirm that the rank is correct, or return warning if not possible to confirm. b is Use ⊥ factorization and, when T numerical singular, deflation to calculate pseudoinverse solution. L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
M ATRIX T EST S ET SJSU Singular Matrix Database at http://www.math.sjsu.edu/ singular/matrices/ 767 numerically singular matrices (with tolerance max(m, n) eps(normest(A)) ) Matrices come from real world applications or have characteristic features of real world problems Most matrices from UF Sparse Matrix Collection (Davis) L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
P ROPERTIES OF T EST S ET M ATRICES :
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
P ROPERTIES OF T EST S ET M ATRICES ( CONT ’ D ):
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
ACCURACY, CORRECT RANKS :
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
ACCURACY, CORRECT RANKS OR WARNING :
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
ACCURACY OF R ANK C ALCULATION :
warning flag had no false positives L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
ACCURACY OF N ULL S PACE BASES :
QR based method overall as good as SVD L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
RUN T IMES : SPQR_NULL VERSUS MATLAB’ S SVD:
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
RUN T IMES : SPQR_NULL VERSUS MATLAB’ S SVDS: For a run over 372 matrices with smaller dimensions MATLAB’s SVDS required 8 hours SPQR_NULL required 1.5 minutes SVDS error flag reported an error for 89% of the matrices MATLAB’s SVDS not useful for finding null spaces bases of matrices L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
RUN T IMES : SPQR_NULL VERSUS SPQR:
SPQR_NULL is slower than SPQR L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
M EMORY U SE FOR N ULL S PACE BASES
Overall SPQR_NULL null space bases requires less memory than dense bases L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
C ONCLUSIONS / F URTHER WORK : SPQR_NULL reliably determines rank, null space basis and psuedoinverse solutions Much faster than MATLAB’s SVD More reliable than SPQR and MATLAB’s SVDS More features than SPQR Further work: Related algorithms Applications L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
RUN T IMES : SPQR_NULL VERSUS SPQR + SSI
maximum number total time total time numerical of (hours) for (hours) for nullity matrices SPQR_NULL SPQR + SSI 500 479 6.7 4.2 1000 517 8.5 16.6 1500 550 10.8 42.7 394336 738 21.8 not available TABLE : Total times (sum of the run times for the matrices in the indicated subsets of our test set) for SPQR_NULL and SPQR + SSI
L ESLIE F OSTER
SIAM L INEAR A LGEBRA C ONFERENCE 2009, O CT. 26-31, 2009, M
