Fast algorithms for positive definite matrices structured by orthogonal polynomials Volker H¨osel a,∗ , Rupert Lasser a,b a Munich
University of Technology, Centre of Mathematics, Boltzmannstraße 3, 85748 Garching, Germany
b GSF-National
Research Center for Environment and Health,Institute of Biomathematics and Biometry, Ingolst¨ adter Landstraße 1, 85764 Neuherberg, Germany
Abstract Positive definite matrices structured by orthogonal polynomial systems allow a Cholesky type decomposition of their inverse matrices in O(n2 ) steps. The algorithm presented in this paper uses the three-term recursion coefficients and the mixed moments of the involved polynomials. Key words: Structured matrices, fast matrix algorithms, orthogonal polynomials, Cholesky decomposition, Levinson algorithm 2000 Mathematics Subject Classification: 15A09
1
Introduction
Techniques which allow efficient triangular decompositions and inversions of structured matrices are important for many applications and have been studied for a long time (for example [7], [14]). Involved algorithms are usually considered as fast if they do the job for n-th order matrices in O(n2 ) steps or less. So the classical Cholesky decomposition of general positive definite symmetric matrices is not fast as it requires O(n3 ) steps [5]. Fast algorithms for non sparse matrices exist for certain structured types. These types include Toeplitz and Hankel matrices and more general matrices with a displacement structure [13], [10], [8]. ∗ Corresponding author. Email addresses:
[email protected] (Volker H¨osel),
[email protected] (Rupert Lasser).
Preprint submitted to Linear Algebra and its Applications
2 May 2007
We introduce in this paper a fast algorithms for a large class of symmetric positive definite matrices. These matrices need not to be of classical displacement type and fast decomposition algorithms have not been published before. A strategy of fitting together a modified Chebyshev algorithm [4] and an algorithm presented in [2] allows a fast triangular decomposition of the inverse. The types of matrices mentioned here occur for example as covariance matrices of certain classes of non stationary processes. These so called Pn -weakly stationary processes have been studied by the authors and other researchers since the end eighties, see for example [11],[9],[12]. The presented algorithm parallels somewhat the Levinson algorithm used in the prediction theory of classical weakly stationary processes [3].
2
(Ps , µ)-structured matrices
We start from a fixed system of real polynomials (Ps )s∈N with deg(Ps ) = s, having positive leading coefficients and being orthogonal with respect to a positive Borel measures π on the real axis with infinite support. Such polynomials satisfy a three-term recursion xPk (x) = ak Pk+1 (x) + bk Pk (x) + ck Pk−1 (x)
P−1 (x) = 0,
k = 0, 1, . . .
P0 (x) = P0 > 0
with recursion coefficients ak > 0, ck > 0 and bk ∈ R [1]. We assume that the recursion coefficients as well as the linearization coefficients g(k, l, s) in the equation Pk (x)Pl (x) =
k+l X
g(k, l, s)Ps (x)
s=|k−l|
are explicitly known. For a lot of polynomial systems such explicit representations can be found in the literature (for example [15], [6]). Next, we chooseR another Borel measure µ on R with infinite support and finite moments µk = R xk dµ(x) of all orders k = 0, 1, . . .. With these settings we call an (n + 1) × (n + 1)-matrix Mn (Ps , µ)-structured if it has the form 2
R P0 (x)P0 (x)dµ(x) · · · R P0 (x)Pn (x)dµ(x) .. .. ... . . . R R R
R
R
Pn (x)P0 (x)dµ(x) · · ·
R
Pn (x)Pn (x)dµ(x)
We denote the entries of Mn as d(k, l) and set d(k) :=
R R
Pk (x)dµ(x).
As the orthogonal polynomials {Pk }nk=0 and the defining measure µ can be chosen arbitrarily except for the stated restrictions, this definition allows for a large amount of structured matrices. A simple example is provided by the Chebyshev polynomials of the first kind Tk (x). The recursion formula of the polynomials implies d(k, l) = 1/2(d(|k − l|) + d(k + l)), showing that (Ts , µ)-structured matrices have a specific Toeplitz-plus-Hankel structure. In the following we will present an algorithm which yields a decomposition of the inverse T M−1 n = Ln Dn Ln into a product of a lower triangular matrix Ln , its transposed LTn and a diagonal matrix with positive entries. The calculation required is O(n2 ) and therefore linear equations Mn xn+1 = bn+1 with given (n + 1)-vector bn+1 can be solved fast for xn+1 . Let φk be the monic polynomials associated to the measure µ and let αk , βk be the recursion coefficients in the three term recursion xφk = φk+1 + αk φk + βk φk−1
k = 0, 1, . . .
φ−1 (x) = 0, φ0 (x) = 1. Further, let cφP (k, l) be the connection coefficients of the expansion φk (x) =
k X
cφP (k, l)Pl (x)
l=0
and σk,l =
Z
φk (x)Pl (x)dµ(x)
R
3
the mixed moments. Note, that one defines cφP (k, l) = 0 for k < l and that σ(k, k) and cφP (k, k) are not zero for k = 0, 1, . . .. Proposition 1 The inverse M−1 n of a (Ps , µ)-structured matrix Mn always exists and has a decomposition Mn −1 = LTn Dn Ln with
cφP (0, 0)
0
··· ...
cφP (1, 0) cφP (1, 1) Ln = .. .. . .
cφP (n, 0)
···
0 .. . 0
· · · cφP (n, n)
and a diagonal matrix Dn with k-th diagonal element 1/(σk,k · cφP (k, k)). Proof. dent: Pn
i=0
The inverse M−1 n exists since the columns of Mn are linear indepen-
λi d(s, i) = 0 for all s = 0, . . . , n means that for those s Z
Ps (x)Q(x)dµ(x) = 0
R
with the polynomial Q(x) = ni=0 λi Pi (x) is valid. But this is only possible if Q(x) is the zero polynomial and λi = 0 for i = 0, . . . , n. P
For the decomposition first observe that Z
(φk (x))2 dµ(x) =
R
Z R
φk (x)
k X
cφP (k, l)Pl (x)dµ(x) = cφP (k, k)σk,k .
l=0
The last equality holds since the orthogonality of the φk (x) with respect to µ implies σk,l = 0 for l < k. Now, again using orthogonality one gets for 0 ≤ k, l ≤ n cφP (k, k)σk,k · δk,l =
Z
φk (x)φl (x)dµ(x) =
k X l X
cφP (k, s)cφP (l, t)d(s, t).
s=0 t=0
R
With the convention cφP (k, l) = 0 for k < l this is just the matrix equation T D−1 n = Ln Mn Ln
4
−1 which implies Dn = (LTn )−1 M−1 n Ln and thus T M−1 n = Ln Dn Ln .
The task is now to calculate the coefficients cφP (k, l) and σk,l for 0 ≤ k, l ≤ n from the matrix Mn . Actually our algorithm needs the moments d(k) for k = 0, . . . , 2n − 1. Mn shows d(k) only up to k = n. But this does not spoil anything: Proposition 2 The (n+1)×(n+1) entries of a (Ps , µ)-structured matrix are uniquely given by the moments d(k), k = 0, . . . , 2n. Vice versa these (2n + 1) moments can be calculated from the matrix in O(n2 ) steps. Proof.
The linearization of Pk (x)Pl (x) yields d(k, l) =
k+l X
g(k, l, s)d(s),
s=|k−l|
showing that d(k, l), 0 ≤ k, l ≤ n can be calculated from d(k), 0 ≤ k ≤ 2n. Now, let the matrix and thus d(k, l), 0 ≤ k, l ≤ n be given. Then d(k, 0) = P0 d(k) is already at hand for k = 0, . . . , n. Since g(k, l, k + l) 6= 0 is always valid, one obtains the missing moments d(n + k), 1 ≤ k ≤ n successively from
n+k−1 X 1 d(n, k) − d(n + k) = g(n, k, s)d(s) . g(n, k, n + k) s=n−k
The required calculation is obviously O(n2 ).
Remark 1 With respect to this observation one could also have the viewpoint that d(k), k = 0, . . . , 2n are given beforehand and Mn is derived. If the (Ps )s∈N define a hypergroup on N0 one might start from a sequence being positive definite with respect to the hypergroup. In this case, the measure µ needs not to be stated explicitely. Its existence follows from a Bochner type theorem (compare for example [12]).
3
The algorithms
The first ingredient of our algorithm is a modified Chebyshev algorithm ([4],p.76). It provides the recursion coefficients αk , βk from the mixed moments σk,l . As 5
we do not require that the Pn are monic we get slightly different formulas compared to [4]. Proposition 3 The recursion coefficients {αk , βk }n−1 k=1 , can be calculated from the 2n−2 values {d(k)}2n−1 k=0 and {ak , bk , ck }k=0 as follows:
Start with the initialization: σ−1,−1 = 1, σ−1,l = σl,−1 = 0,
l = 0, 1, . . . , 2n − 2;
σ0,l = d(l), l = 0, 1, . . . , 2n − 1; β0 = a−1 = c0 = 0 and continue for k = 1, 2, . . . , n − 1; l = k, k + 1, . . . , 2n − k − 1 with the recursion for the mixed momentums σk,l = al σk−1,l+1 + [bl − αk−1 ]σk−1,l + cl σk−1,l−1 − βk−1 a0 σk−2,l
(1)
and the coefficients βk = ak−1
Proof.
σk,k σk−1,k−1
;
αk = bk + ak
σk,k+1 σk−1,k − ak−1 . σk,k σk−1,k−1
(2)
For the mixed moments one finds
σk,l =
Z
φk (x)Pl (x)dµ(x)
R
Z
= [xφk−1 (x) − αk−1 φk−1 (x) − βk−1 φk−2 (x)]Pl (x)dµ(x) R
=
Z
φk−1 (x)[al Pl+1 (x) + bl Pl (x) + cl Pl−1 (x)]dµ(x) − αk−1 σk−1,l − βk−1 σk−2,l
R
= al σk−1,l+1 + [bl − αk−1 ]σk−1,l + cl σk−1,l−1 − βk−1 σk−2,l . The orthogonality of φk with respect to µ gives σk,l = 0 for all k > l. Especially one has for all k ≥ 1 0 = σk+1,k−1 = ak−1 σk,k − βk σk−1,k−1 and thus βk = ak−1
σk,k σk−1,k−1
.
Likewise one gets 0 = σk+1,k = ak σk,k+1 + [bk − αk ]σk,k − βk σk−1,k 6
giving σk−1,k σk,k+1 − βk σk,k σk,k σk,k+1 σk−1,k = b k + ak − ak−1 . σk,k σk−1,k−1
αk = bk + ak
With the received recursion coefficients αk and βk we calculate as next step the connection coefficients cφP (k, l) of the expansion. This can be done with an algorithm given by Askey [2] for connecting monic polynomials. We again have to rewrite it, as the polynomials Ps need not be monic. Proposition 4 The connection coefficients {cφP (k, l)}kl=0 for k = 1, . . . , n can be n calculated from {αk , βk }n−1 k=0 and {ak , bk , ck }k=0 as follows:
Start with
1 P0 = c0 = 0, cφP (k, l) = 0 for l > k or k < 0 or cφP (0, 0) =
and the conventions β0 = a−1 l < 0.
Continue for 1 ≤ k ≤ n − 1 and l = 0, . . . , k + 1 with cφP (k + 1, l) = al−1 cφP (k, l − 1) + (bl − αk )cφP (k, l) +cl+1 cφP (k, l + 1) − βk cφP (k − 1, l). Proof.
cφP (0, 0) =
1 P0
follows from
φ0 (x) = 1.
The three term recursions of φk and Pk show for k = 0, . . . , n − 1 φk+1 (x) = xφk (x) − αk φk (x) − βk φk−1 (x) =x =
k X
cφP (k, l)Pl (x) − αk
l=0 k X
k X
cφP (k, l)Pl (x) − βk
l=0
k−1 X l=0
cφP (k, l)[al Pl+1 (x) + bl Pl (x) + cl Pl−1 (x)]
l=0
−αk
k X
cφP (k, l)Pl (x) − βk
l=0
k−1 X l=0
7
cφP (k − 1, l)Pl (x).
cφP (k − 1, l)Pl (x)
Now, we reorder according to Pl . Hereby we shift indices and use the above conventions.
φk+1 (x) =
k+1 X
[al−1 cφP (k, l − 1) + (bl − αk )cφP (k, l)
l=0
+cl+1 cφP (k, l + 1) − βk cφP (k − 1, l)]Pl (x) These equations show an explicit representation of the connection coefficients. The technique for a fast decomposition of our structured matrices now follows from combining the given algorithms. We thereby get rid of the coefficients αk and βk . Theorem 1 The coefficients {cφP (k, l)}0≤k,l≤n and the mixed moment {σk,k }nk=0 required for the decomposition Mn −1 = LTn Dn Ln can be found from the entries of Mn , and the three term recursion coefficients 2 {ak , bk , ck }2n−2 k=0 , with the following algorithm having O(n ) complexity: First Step: Calculation of d(s) For k = 0, . . . , n d(k) = d(k, 0)/P0 . For k = 1, . . . , n:
n+k−1 X 1 d(n, k) − g(n, k, s)d(s) d(n + k) = g(n, k, n + k) s=n−k
Second Step: Calculation of σk,l σ−2,−2 = σ−1,−1 = 1, a−1 = c0 = 0. For l = 0, 1, . . . , 2n − 2 σ−1,l = σl,−1 = 0. For 0 ≤ l < k ≤ n σk,l = 0. For
l = 0, 1, . . . , 2n − 1 σ0,l = d(l).
For k = 1, . . . , n;
l = k, . . . 2n − k − 1 8
σk,l = al σk−1,l+1 + [bl − bk−1 − ak−1 + cl σk−1,l−1 − ak−2
σk−1,k σk−2,k−1 + ak−2 ]σk−1,l σk−1,k−1 σk−2,k−2
σk−1,k−1 σk−2,l . σk−2,k−2
Third Step: Calculation of cφP (k, l) cφP (0, 0) = 1/P0 , cφP (−1, −1) = cφP (−2, −2) = 0. For 0 ≤ l ≤ n cφP (−1, l) = cφP (l, −1) = cφP (−2, l) = cφP (l, −2) = 0. For 0 ≤ k < l ≤ n cφP (k, l) = 0. For k = 1, . . . , n − 1,
l = 0, . . . , k + 1
cφP (k + 1, l) = al−1 cφP (k, l − 1) + [bl − bk − ak + cl+1 cφP (k, l + 1) − ak−1
σk,k+1 σk−1,k + ak−1 ]cφP (k, l) σk,k σk−1,k−1
σk,k σk−1,k−1
cφP (k − 1, l).
Proof. The calculation of d(k) is given in Proposition 2. The representation of the mixed moments results from the modified Chebyshev algorithm by inserting (2) into (1). Plugging the representations of αk and βk in the algorithm presented in Proposition 4 gives the stated recursions for cφP . All steps have complexity O(n2 ).
References
[1] R. Askey, Orthogonal polynomials, Mem. Amer. Math. Soc. 18(213), Providence, 1979. [2] R. Askey, Orthogonal expansions with positive coefficients II, SIAM J. Math. Anal. 2 (1971), 340-346. [3] P. Delsarte and Y. Genin, On the role of orthogonal polynomials on the unit circle in digital signal processing applications (115-133), in: Orthogonal polynomials, theory and practice, P. Nevai (ed.), NATO Advanced Science Institutes Series C: Mathematical and Physical Sciences, 294. Kluwer, 1990. [4] W. Gautschi, Orthogonal polynomials, computation and approximation, Numerical Mathematics and Scientific Computation. Oxford University Press, 2004.
9
[5] G. H. Golub and C. F. van Loan, Matrix computations, 2nd edn., The John Hopkins University Press, 1989. [6] M.E.H. Ismail, Classical and quantum orthogonal polynomials in one variable, Cambridge University Press, Cambridge, 2005. [7] V. Olshevsky (ed.), Fast algorithms for structured matrices, theory and applications, AMS-IMS-SIAM Joint summer research conference on fast algorithms in mthematics, computer science and engineering, 2001, Amer. Math. Soc., Contemp. math.(323), 2003. [8] G. Heinig and K. Rost, New fast algorithms for Toeplitz-plus-Hankel matrices, SIAM Journal Matrix Anal. Appl. 25(3) (2003), 842-857. [9] V H¨osel, On the estimation of covariance functions of Pn -weakly stationary processes, Stoch. Anal. Appl. 16(4) (1998), 607-629. [10] T. Kailath and A. H. Sayed, Displacement structure, theory and applications, SIAM Review 37 (1995), 297-386. [11] R. Lasser, A modification of stationarity for stochastic processes induced by orthogonal polynomials, in : H. Heyer (ed.), Probability measures on groups IX, Lecture Notes Math. 1379, Springer, Berlin, 1989, 185-191. [12] V. H¨osel and R. Lasser, Prediction of weakly stationary sequences on polynomial hypergroups, Annals of Probability 31(1)(2003), 93-114. [13] V. Y. Pan. Structured Matrices and Polynomials, unified superfast algorithms, Birkh¨auser, Boston, 2001. ¨ [14] I. Schur, Uber Potenzreihen, die im Innern des Einheitskreises beschr¨ankt sind, J. Reine Angew. Math. 147 (1917), 205-232. [15] G. Szeg¨o, Orthogonal Polynomials, Amer. Math. Soc., New York, 1959.
10
