SPECTRAL FACTORIZATION OF LAURENT ... - CiteSeerX

SPECTRAL FACTORIZATION OF LAURENT POLYNOMIALS Tim N.T. Goodman Department of Mathematical Sciences, University of Dundee Dundee DD1 4H, Scotland, U.K. email: [email protected] Charles A. Micchelli IBM Research Division, T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 U.S.A. email: [email protected] Giuseppe Rodriguez and Sebastiano Seatzu Department of Mathematics, University of Cagliari Viale Merello 92, 09123 Cagliari, Italy email: [email protected], [email protected] Abstract. We analyse the performance of ve numerical methods for factoring a Laurent

polynomial, which is positive on the unit circle, as the modulus squared of a real algebraic polynomial. It is found that there is a wide disparity between the methods, and all but one of the methods are signi cantly in uenced by the variation in magnitude of the coecients of the Laurent polynomial, by the closeness of the zeros of this polynomial to the unit circle, and by the spacing of these zeros. Keywords: spectral factorization, Toeplitz matrices, Euler-Frobenius polynomials, Daubechies wavelets. AMS subject classi cation: 12D05, 15A23.

1. Introduction

Our recent interest in the asymptotic behaviour of the Gram-Schmidt iteration for orthonormalization of a large number of integer translates of a xed function [11, 12] and also in techniques used for wavelet construction, cf. [7] and [20], has led us to experiment numerically with several existing algorithms to factor a Laurent polynomial, assume to be nonnegative on the unit circle, as the modulus squared of a real algebraic polynomial. In this paper we present a numerical study of the ve algorithms that we are aware of for the solution of this problem. The rst and second authors were partially supported by NATO grant CRG 950849, the second author also by EPSRC grant GR/K 54779 VF, and the last two authors by the Italian Ministry of University and Scienti c and Technological Research and by the Italian National Research Council. Typeset by AMS-TEX 1

2

Let

a(z) =

m X j =?m

aj zj

(1.1)

be a Laurent polynomial of degree m such that aj = aj = a?j , j = 1; 2; : : : ; m. We denote the space of all such functions by Sm . When a 2 Sm has the property that

a(z)  0; jzj = 1 the problem we consider is to nd real numbers 0; 1; : : : ; m such that

(z) =

m X j =0

(1.2)

j zj

has all roots outside the unit disc and such that

(z) (z?1 ) = a(z); z 2 C n f0g: (1.3) To guarantee the uniqueness of the factorization we impose the normalization (0) > 0 on the polynomial (z). The existence of such a factorization is well-known and attributed to Fejer [17, pg. 117]. Our concern here is to present some numerical experiment with several algorithms which are available for computing (z). In this regard, we use m for the space of all real algebraic polynomials of degree at most m. In our numerical simulation we are especially interested in the factorization of the Laurent polynomials which are positive on the unit circle. Speci cally, those two Laurent polynomials associated both to the orthogonalization process of B-splines with integer knots, cf. [11], and the generation of the Daubechies re nable function, cf. [7]. In the rst case the Laurent polynomial which we shall factor is given by the equation 2m ?m X z a(z) = (2m)! E2m(z) = M2m+1 (k + 1)zk?m k=0

(1.4)

m m + j    2j X i sin 2 dm (e ) = j j =0

(1.5)

where E2m is the Euler-Frobenius polynomial of degree 2m and M2m+1 is the forward B-spline of degree 2m + 1 with integer knots, cf. [19, pg. 11]. In the second case, we want to factor the polynomial

which can be easily written in the equivalent form

dm(z) =

m X j =?m

2

?1)j

4(

m X



2?2r m r+ r r=jj j



2r

r ?j

3  5 zj ;

(1.6)

3

cf. [7]. Furthermore, introduce the polynomial

bm (x) =

m m + j X j =0

j



xj

(1.7)

which is the binomial series for (1 ? x)?(m+1) , truncated after m + 1 terms, the same recently studied by Shen and Strang [18]. Since x = (1 ? cos )=2 and z = ei , we obtain that z + z?1 = 2(1 ? 2x) and therefore bm (x) = dm (z). Note that, as a result of the rule z + z?1 = 2(1 ? 2x), each zero of bm (x) gives two zeros of dm (z). Hence from each pair we choose the zero outside the unit circle and obtain the polynomial (z) that we look for. Besides these two important examples we will also consider the spectral factorization of carefully chosen positive Laurent polynomials which reveal certain numerical properties of the ve algorithms we have studied. Finally we add that the third method, among the ve that we shall describe below, can be used under the hypothesis (1.2), while the remaining four are applied only if a(z) > 0 for jzj = 1. 2. The algorithms

Bauer method. The rst method we wish to describe is due to F. Bauer [2, 3]. His idea is to form the banded and symmetric bi-in nite Toeplitz matrix A = (ai?j )i;j2Z by the coecients of a(z). When the Laurent polynomial a satis es (1.2) the matrix A is positive semide nite on `2(Z). Moreover the factorization (1.3) corresponds to the factorization A = ??T where ? = ( i?j )i;j2Z is the banded lower triangular matrix formed by the coecients of (z). Let A+ be the semi-in nite compression of the bi-in nite matrix A, given by A+ = (Aij )i;j2Z + and An, n 2 Z+, be the sequence of nite compressions of A de ned by An = (Aij )i;j=0;1;::: ;n; n = 0; 1; 2; : : : : If, in particular, a(z) is strictly positive on the unit circle, the bi-in nite matrix A is positive de nite on `2(Z). As a result, each nite compression An , n 2 Z+ has a unique Cholesky factorization LnLTn = An

4

where Ln is a lower triangular matrix with positive diagonal elements. As we increase n to n +1 the matrix An+1 agrees with An on its rst n +1 rows and columns. Likewise Ln+1 has its rst n + 1 rows and columns equal to those of Ln. Therefore we may consider Ln as the n-th nite section of a semi-in nite matrix L which is the unique Cholesky factorization of A+ , that is, A+ = LLT . Bauer proved that as n ! 1, the elements of Ln approach those of ? exponentially, that is, there is a constant c > 0 and a  2 (0; 1) such that for all i; j 2 Z+

j`ij ? i?j j  ci:

(2.1)

Based on this observation, we design the following algorithm. Choose an  > 0 depending on the precision desired and (1) compute the Cholesky factor Lm+1; (2) compute the last row of Lm+i+1 for i = 1; 2; : : : ; (3) stop the iteration as soon as max fj`m+i+1;j+1 ? `m+i;j j : j = i; i + 1; : : : ; i + mg  : This method, as a consequence of inequality (2.1), has linear convergence. For the importance of this algorithm for orthonormalization of the integer translates of a xed function see [11, 12].

Wilson method. The second method which we consider is due to Wilson [21]. His idea

is to rewrite equation (1.3) as the following equivalent system of quadratic equations m ?i X j =0

j j+i = ai ; i = 0; 1; : : : ; m:

(2.2)

This approach consists in the use of a Newton-Raphson method for solving system (2.2). Wilson proved that, if a(z) has no zeros on the unit circle, for a suitable and easily made choice of starting values 0(0); 1(0); : : : ; m(0), his iteration is self-correcting and always converges quadratically to the required solution. Let us review some of the details in his analysis. For k = 0; 1; : : : , let (k) = ( 0(k); 1(k); : : : ; m(k)) be the vector of the coecients of a polynomial (k)(z) at the k-th step and denote by a = (a0 ; a1 ; : : : ; am ) the vector appearing in the right hand side of system of equations (2.2). Given (k), the new vector (k+1) is the solution of the linear system (T1(k) + T2(k)) (k+1) = c(k) + a where T1(k) and T2(k) are (m + 1)  (m + 1) matrices de ned as (T1(k))ij = [ i(+k)j ]; (T2(k))ij = [ j(?k)i]; i; j = 0; 1; : : : ; m

(2.3)

5

with



i; if i = 0; 1; : : : ; m 0; otherwise and c(k) is an (m + 1)-vector given by the formulas [ i] =

c(k) = T1(k) (k) = T2(k) (k): Alternatively, we may think of Wilson's algorithm as an iteration within the class of polynomials of degree m. Thus we try to construct a sequence (k), k 2 Z+ of polynomials in m which will converge to the solution of equation (1.3). The idea is based on the following consideration. Suppose we have found a polynomial (k) which approximates . We obtain a next approximation (k+1) by considering the residual u(k) := (k+1) ? (k), demanding that (1.3) is satis ed approximately 





(k)(z) + u(k)(z) (k)(z?1 ) + u(k)(z?1 ) ' a(z)

and discarding the second order term u(k)(z)u(k) (z?1 ). This gives Wilson's recursion

(k+1)(z) (k)(z?1 ) + (k+1)(z?1 ) (k)(z) = a(z) + (k)(z?1 ) (k)(z): (2.4) Let us review some of his conclusions about this iterative method. The rst point to note is that (k+1)(z) will be uniquely determined provided that (k)(z?1 ) and (k)(z) have no common zeros. To explain this, we choose any 2 m and consider the linear mapping ? : m ! Sm de ned by ?(f )(z) = f (z) (z?1 ) + f (z?1 ) (z): Obviously, if (z0) = (z0?1 ) = 0 for some z0 2 C n f0g then ?(f )(z0 ) = 0 and so ?(m ) 6= Sm . However, when (z) and (z?1 ) have no common zeros then ?(m ) = Sm . To check this we suppose ?(f ) = 0 for some f 2 m , that is f (z) (z?1 ) = ?f (z?1 ) (z); z 2 C n f0g: (2.5) Hence, whenever (z) = 0 it follows that f (z) = 0 and so f must have a factorization as f = h where h is a polynomial. But then equation (2.5) implies h(z) = ?h(z?1) and so h = f = 0. Returning to the iteration (2.4) we need an initial polynomial (0) that guarantees at each stage of the algorithm (k)(z) and (k)(z?1 ) have no common zeros. Wilson observed the following fact. If (0) has all its zeros outside the unit circle, then for each k 2 Z+, (k) has its zeros outside the unit circle. This is proved inductively on k. Suppose it is true for (k). To advance the induction hypothesis we observe (2.4) implies that n

(k+1)(z) (k+1)(z?1 ) = (k+1)(z) ? (k)(z)

on

o

(k+1)(z?1 ) ? (k)(z?1 ) + a(z)

6

and so for jzj = 1 we have that (k+1) 2 (z)

This implies that and also that

=

(k+1) (z)

(k+1) 2 (z) (k+1) (z)



2 ? (k)(z) + a(z):

 a(z) > 0; jzj = 1

> (k+1)(z)



? (k)(z) ;

jzj = 1:

(2.6) (2.7) (2.8)

Therefore, by Rouche's theorem (k) and (k+1) have the same number of zeros inside the unit circle. This advances the step. The convergence of the iteration (2.4) is based on the following observation. From (2.4) we have that (k+1) 2 Re (k) (z) = 1 + (ak()z) 2 ; jzj = 1:

(z) j (z)j

(2.9)

Since a(z) > 0 on jzj = 1 we see, on recalling (2.7), that 1 < Re (k+1)(z)  1; jzj = 1: 2

(k)(z)

(2.10)

But (k+1)= (k) is analytic inside the unit disk and so Re( (k+1)= (k)) is harmonic there. This means that (2.10) holds for all jzj  1. Since (0)(z) is zero free in jzj  1 it is of one sign on the interval [?1; 1]. Thus we obtain by our above remark, inductively on k that the sequence f (k)(t) : t 2 [?1; 1]g is nonincreasing. Hence it follows that lim (k)(z) = (z); z 2 C

k!1

for some polynomial 2 m which is necessarily zero free inside the unit circle. To see that iteration converges quadratically we set

e(k) = (k) ? ; k 2 Z and observe from (2.4) that

e(k+1)(z) (k)(z?1 ) + e(k+1)(z?1 ) (k)(z) = e(k)(z)e(k) (z?1 ): This can be rewritten as ?(e(k+1))(z) = e(k)(z)e(k) (z?1 ) ? e(k+1)(z)e(k) (z?1 ) ? e(k+1)(z?1 )e(k)(z):

(2.11)

7

Since (z) and (z?1) have no common zeros there is a positive constant  > 0 such that k?(f )k1  kf k1 ; f 2 m ; where kf k1 := maxfjf (z)j : jzj  1g: Hence equation (2.11) gives us the inequality

ke(k+1)k1  2ke(k+1)k1ke(k)k1 + ke(k)k21 or, when ke(k)k1 < =2,

(k) 2 ke(k+1)k1   ?ke2ke(kk1)k

1

from which quadratic convergence follows. Wilson considers his method as an improvement of the Bauer method, which has linear convergence. However, the computational complexity of the Wilson method is not always less than that of the Bauer method. Suppose, for example, that we need nB iterations to obtain a xed precision by the Bauer method. At each iteration we must perform O(m2 =2) oating point operations, and so the computational complexity is O(nB m2=2). With the same hypothesis we suppose that the Wilson method needs nW iterations, and then, since at each step we have to execute O(m3 =3) operations, the computational complexity is O(nW m3=3). Hence the Wilson method is really an improvement of the Bauer method whenever the number of the iterations needed in the Bauer method is large enough with respect to the degree of the polynomial (z), that is, if nW < 3 : n 2m B

Concerning the implementation of the Wilson method, we found that our numerical results were greatly improved if at the k-th iteration the solution of linear system (2.3) is obtained by the formula 1

(k+1) = (k) +  (k+1) 2

where (k+1) is the solution of linear system

(T1(k) + T2(k))(k+1) = a: Indeed, our numerical simulation showed that the condition number of matrix T (k) = T1(k) + T2(k) strongly depends on the iteration index k, and that for some values of m it tends to become extremely large, thereby amplifying the roundo errors for the vector ck .

8

Roots method. The third method we consider is based on the computation of the

zeros of the Laurent polynomial a(z). In order to improve the eectiveness of the method, in view of the symmetry of its coecients, we write a(z) in the following form

a(z) = a0 +

m X j =1

?




aj zj + z?j :

By making use of the expansion ?1 k  ?
k kX k ? k ? 1 zk?2j z +z = z+z ? j j =1

and applying the change of variable

w = z + z?1 we obtain the polynomial

c(w) := a(z(w)) =

m X j =0

cj w j

whose coecients c0; c1; : : : ; cm can be expressed recursively in terms of the original coecients a0 ; a1 ; : : : ; am . The zeros wi, i = 1; 2; : : : ; m of the polynomial c(w) can then be evaluated by computing the eigenvalues of its companion matrix by a QR method [10]. Next, we solve the m quadratic equations z + z?1 = wi ; i = 1; 2; : : : ; m and then, taking into account their multiplicity, we obtain 2m roots zi , i = 1; 2; : : : ; 2m of a(z), that we order by decreasing modulus. As the coecients of a(z) are real and symmetric, if zi is a root so are zi and zi?1 , so that if zi is outside the unit circle, zi?1 is inside and if zi is on the unit circle, so is zi?1 . Furthermore, since a(z)  0 for jzj = 1, the roots on the unit circle have even multiplicity. As a result, the complex roots o the unit circle actually come four at a time: zi and zi outside, zi?1 and z?i 1 inside. Complex roots on the unit circle also come four at a time: zi twice and zi twice. Real roots on the unit circle come two at a time (even multiplicity). The polynomial (z) that we are looking for can then be constructed by taking all the roots zj (including zj ) outside the unit circle and also taking one of each double root z^j on the circle: mY ?r r Y j

(z) = j z :=  (z ? zj ) (z ? z^j ) j =1 j =1 j =0 m X

9

where

=

v u u sgn(am )t(

?1)m am



r Y

j =1

zj

mY ?r j =1

z^j

?1

:

This method is in fact the standard way the existence of the Fejer factorization is proved. We note that in several practical situations (like the Euler-Frobenius case) the coef cients fcj gm j =0 can be severely \unbalanced", in the sense that maxj jcj j  minj jcj j. In such cases it is useful to apply a scaling technique to the polynomial c(w) before computing its zeros. In this regard we adopted, except in the Daubechies case, the following scaling procedure. First, we let m X c ( w ) c~(w) := c = c~j wj 0 j =0

and then make the change of variables y = !. To obtain a polynomial in y that has \better" scaled coecients we choose the value  = 2h where h = blog2 jc~mj=mc is the integer which minimizes the quantity

j1 ? 2?hm c~mj: This choice ensures that the binary representation of  will not degrade the accuracy of the coecients. Concerning the spectral factorization of the Daubechies polynomials, as already pointed out by Shen and Strang [18], it is much more eective to evaluate directly the zeros of the polynomial bm (z) de ned in (1.7). In this case, taking into account the expression of the coecients of bm , the scaling factor turns out to be  = 4.

MinPh method. The fourth method we have tested is the minimum phase factori-

zation. This algorithm was rst brought to our attention by Wayne Lawton of the Institute of System Science in Singapore. The idea of the algorithm is the following observation. Let a(z) have no zeros on the unit circle. For any g 2 m normalized so that g(0) = 1 we consider the minimization problem min g2 ;g(0)=1 m

Z



?

jg(ei )j2 a(d ei ) :

Then the solution of this minimization problem is given by (z) gopt(z) :=

(0) where is the polynomial in (1.3) which we seek.

10

For the proof of this observation we only need to verify that Z  gopt(ei )e?ij + gopt(e?i )eij d = 0; j = 1; 2; : : : ; m:

(ei ) (e?i ) ? Indeed, because has no zeros inside the unit circle, for each j = 1; 2; : : : ; m we have that Z  gopt(ei )e?ij + gopt(e?i )eij d = 2 Z  eij d

(ei ) (e?i )

(0) ? (ei ) ? Z 2 zj?1 dz = 0 = i (0) jzj=1 (z ) by the Cauchy integral formula. The actual algorithm solves a linear system for the coecients of g. Indeed, if we let

g(z) = 1 ? and  = (1; 2 ; : : : ; m )T then

m X j =1

j zj

T  = b; where T = (ti?j )i;j=1;2;::: ;m is the Toeplitz matrix whose elements are given by Z  cos(i ? j ) d; i; j = 1; 2; : : : ; m ti?j = ? a(ei ) and b = (b1 ; b2 ; : : : ; bm )T is the vector with components Z  cos k d; k = 1; 2; : : : ; m: bk = ? a(ei ) The elements of the matrix T can be approximated by the DCT. The solution of the above system identi es the polynomial g(z), and then (z) since r

(z) = (0)g(z) and (0) = ? am : m Our numerical experiments with the minimum phase algorithm use an implementation of the above procedure provided to us by Dr. Lawton. Other normalization rules for g(z) can be adopted as well. This is convenient, in particular, whenever m is very small. In this case it is more eective to evaluate  as speci ed below p (1)

(z) = g(z); where  = jga(1) j:

11

Cepstral method. The nal method we consider, very popular in signal processing,

goes by the name of the cepstral algorithm [4, 16]. It is based on some results on the factorization of an absolute convergent Fourier series on the unit circle, discovered independently by M.G. Krein [13] and A. Calderon, F. Spitzer and H. Widom [5]. Let us review some of the ideas associated with this method. If the Laurent polynomial a(z) is strictly positive on the unit circle, by the WienerLevy theorem concerning trigonometric series, the function log a(z) can be written as an absolutely convergent Laurent series

b(z) := log a(z) =

X

j 2Z

bj zj :

(2.12)

Moreover, the coecients bj , as those of a(z), are even and decay exponentially since log a(z) has an extension as an analytic function in some annulus r?1 < jzj < r, r > 1 containing the unit circle. Splitting the Laurent series in the form

b(z) = b? (z) + b+(z) where

b? (z) :=

10 X j =0

and

b+(z) :=

b?j z?j ; jzj  1

1 X j =0

0

bj zj ; jzj  1

we obtain the factorization (1.3) of the required type, where

(z) := exp(b+ (z)) is analytic and zero free in the disk fz : jzj  rg and normalized so that (0) > 0. The prime on the sums above indicates that the rst term is halved and also it can be seen that above is indeed a polynomial of degree at most m. An implementation of this method, making intensive use of the Fast Fourier Transform (FFT), is widely used in signal processing [16, chapter 12]. The associated algorithm consists of the following three steps: (1) sample a(z) on equispaced points on the unit circle (zk = eik , k = 2k n ,k= 0; 1; : : : ; n ? 1); (2) compute the inverse DFT of log a(ei ) to obtain the vector b = (bj )jn==2??n=1 2; (3) take the vector b+ = (b0 =2; b1; : : : ; bn=2?1 ) and compute the inverse DFT of the function exp(b+(ei )).

12

The vector so gained gives us the Fourier coecients of the factor (ei ). A Matlab [14] implementation of this algorithm has been recently proposed by Kevin Amaratunga [1] for the fast computation of the coecients of the Daubechies re nable function. Our routine is an adaptation of the original version, kindly sent us by the author. We note that this method can be used as well, with no more computational eort, whenever we have to compute the Fourier coecients of ( (ei ))?1 , as happens for example in the orthonormalization of a large number of functions which are the integer translates of a xed function [11, 12]. These coecients are in fact the Fourier coecients of the function expf?b+(ei )g. 3. Numerical simulation

Our numerical simulations concern the factorization of Euler-Frobenius polynomials (1.4), wavelet polynomials (1.5) and other nonnegative Laurent polynomials which highlight some properties of the methods studied. We shall denote by Em a measure of the error in the calculated approximation to the polynomial in (1.3). Two choices of this will be described later. The ve methods have been implemented adopting the following computational strategies: (1) In the Bauer and in the Wilson method we xed at 400 the maximum number of iterations. Hence, if we denote by  the tolerance value that we have xed, then our stopping criterion for the iterative processes is either Em   or the maximum number of iterations is reached. (2) In the Roots method the zeros of the polynomials have been computed by the routine roots of Matlab [14]. As this routine implements the QR algorithm [10] in order to maximize the precision achievable, we use it without any check on the number of iterations. (3) Both in the MinPh and in the Cepstral methods the FFT has been used with several choices of points. Speci cally, for each example, the numerical simulation has been carried out using 2k (k = 6; 7; : : : ; 11) points. The numerical results quoted refer to the optimal number of points (nopt), that is the number of points that produces the minimum value of Em . For the sake of clarity, we comment separately on the three classes of examples we have considered. Euler-Frobenius polynomials. Since the accuracy in the computation of the coecients fM2m+1 (k + 1)g strongly in uences the quality of the factorization, we investigated the performance of two dierent methods for computing the values assumed by a cardinal B-spline M2m+1 on its knots. The rst one, which is based on the recurrence relation for B-splines, cf. [9, pg.131], allows us to obtain a stable evaluation of M2m+1(k + 1) by repeatedly forming positive linear combinations of positive quantities. The second method, which is taken from [6], adapted and analyzed further in [8] for the computation of wavelet integrals, reduces the problem of computing a B-spline at

13

its knots to an eigenvector problem. Speci cally, let

(x) =

nX +1 j =0





aj (2x ? j ); aj := 2?n n + 1 ; j = 0; 1; : : : ; n + 1 j

be the re nement equation for the cardinal B-spline  = Mn of degree n, which is known to satisfy the equality (1) + (2) +


+ (n) = 1: (3.1) Therefore, we get the eigenvector relation  = A

where the vector  is given by  = ((1); (2); : : :

; (n))T

and the elements of the n  n matrix A are de ned as

Aij = a2i?j ; i; j = 1; 2; : : : ; n: Since the subdivision scheme based on the matrix A converges, cf. [15], it follows from [8] that  is the unique eigenvector of A corresponding to its largest eigenvalue which is one. Therefore, the power method can be successfully used to compute . An alternative algorithm using the symmetry of  proceeds as follows. Set n = 2m+1 and let ak be the k-th column vector of the matrix A. Since

(j ) = (2m + 2 ? j ); j = 1; 2; : : : ; m

(3.2)

we have that  = (a1 + a2m+1 )(1) +


+ (am + am+2 )(m) + am+1 (m + 1):

Therefore, introducing the m-vector ^ =



^1; ^2; : : : ; ^m

T

whose components, proportional to those of , are ^k := (k)=(m +1), and the m  m matrix A^ = [^a1 ; a^2; : : : a^m ] whose columns are de ned by the relation a^k = ak + a2m+2?k , we obtain the linear system of equations (I ? A^)^ = am+1:

14

The solution of this system of equations, taking into account (3.2) and the normalization condition (3.1), gives us the eigenvector that we seek. In our numerical experiments we let m = 1; 2; : : : ; 100 and found out that for each m the coecients computed by the three mentioned algorithms agree at least on the rst 8 signi cant digits (working in double precision on Matlab [14]). The two eigenvector algorithms proved to be a bit faster than the rst one, even though the computational complexity for each of the three algorithms is O(m3 ). In particular the power method seems to be convenient for large values of m, as we found out that the maximum number of iterations performed to reach convergence grows very slowly (for 1  m  100 this number was always  26). Since the zeros of the spectral factor for the Euler-Frobenius polynomial are simple, negative, less than ?1 and (1) > 0, cf. [19], all the coecients of must be positive. Hence, as a preliminary test, we checked whether or not each method preserves this property. method m

Bauer Wilson Roots MinPh Cepstral 43 43 32 8 10 Table 1

The maximum values of m for which each method furnishes positive coecients f j gmj=0 is reported in Table 1. This table shows that the MinPh and Cepstral methods guarantee the positivity of the coecients of the spectral factor only for rather small values of m, compared to the other three algorithms. This fact suggests that there could be signi cative dierences in the performance of the ve methods, at least in this class of examples. Let (z) be the spectral factor of Euler-Frobenius polynomial (1.4) and ~(z) the numerically computed spectral factor. Let m X

a~(z) = ~(z)~ (z?1 ) =

j =?m

a~j zj

be the corresponding numerically computed Laurent polynomial. To assess the eectiveness of the algorithms, we adopted the following two indices

Em = and

Em; =

v u 200 uX t

k=1

v u m  u X aj t

j =?m

ja(zk ) ? a~(zk )j

? a~j 2

(3.3)

aj

,v u 200 X 2 u t

k=1

ja(zk )j2

(3.4)

15

where

 ; k = 1; 2; : : : ; 200: zk = eik ; k = k 100 Our numerical experiments suggested that we should consider each method only for m in the range 1  m  45, and the degree of the Euler-Frobenius polynomials to be  91, because for larger values all methods give unreliable results. 0

10

−5

10

MinPh Bauer

−10

Cepstral

10

Wilson −15

Roots

10

1

5

10

15

20 25 radius = 1

30

35

40

45

0

10

MinPh

−5

10

Cepstral

−10

Bauer

10

Wilson −15

Roots

10

1

5

10

15

20 25 radius = 2

30

35

40

45

0

10

MinPh

−5

10

Cepstral

Bauer

−10

10

Wilson

−15

Roots

10

1

5

10

15

20 25 radius = 10

30

35

40

45

Figure 1

The two indices introduced above, well de ned since the coecients of a(z) in (1.4)

16

are positive, furnish two dierent measures of comparison for the algorithms. Indeed, while Em gives a measure of the error in the recovery of the coecients of the Laurent polynomial a(z), Em; gives us an estimate of the distance of ~a(z) from a(z) on a circle of radius . In this sense, they respectively give a global and a local measure of the error in the factorization. The behaviour of the error index Em; in the MinPh and Cepstral methods, unlike the other three methods, strongly depends on the radius  of the sampling circle. This result is evident in Figure 1, where we reported the values of Em; versus m for each method and for  = 1; 2; 10. Though the MinPh algorithm produces by far the less reliable results with respect to the other methods, it is clear that its performance enhances in a neighborhood of the unit circle. Furthermore, the Cepstral method gives a very accurate factorization when  is close to one. The reason for this is that the FFT does not resolve quantities whose relative magnitude exceeds the computer oating point accuracy (roughly 10?16 in double precision). This fact is illustrated in Figure 2, where, for m = 20, we report the coecients of the factor numerically computed by the Wilson, MinPh and Cepstral methods (the results returned from Bauer and Roots are graphically undistinguishable from those returned by Wilson). The fast decay of the coecients j is a direct consequence of the fast decay of the coecients aj . Table 2 reports the values of the ratio a0=am , rounded to two digits, for some values of m. This behaviour is not surprising. In fact, since a0 = M2m+1(m + 1) is \close" to 1, am = M2m+1(1) = 1=(2m + 1)! is \close" to zero. 0

10

−5

10

MinPh

−10

10

m 5 10 20 30 40

−15

10

−20

Cepstral

10

−25

10

−30

10

−35

10

Wilson

−40

10

a0=am 1:6
107 1:5
1019 7:1
1048 8:9
1082 8:8
10119

−45

10

−50

10

1

3

5

7

9

11

13

Figure 2

15

17

19

21

Table 2

The values of Em; for the other three algoritms are good for all the values of  taken into consideration, even though the results for the Bauer method degrade for increasing m sooner than those of the Wilson and Roots methods. The index Em , measuring the relative error between the true coecients of a(z) and those numerically computed, emphasizes the dierent performance of the ve methods over the whole complex plane. The plot of Em versus m are reported in Figure 3. In these tests the tolerance value of  required by the rst two algorithms was xed

17

at 10?10, and each method has been tested only for those values of m for which the numerically computed spectral factor has positive coecients, that is for m ranging from one to the values listed in Table 1. Figure 3 shows that, as already pointed out, the MinPh and Cepstral methods are reliable only for very small values of m. On the contrary, while the Roots method is eective for m  32, the remaining two methods give reliable results for larger values of m. The breakdown of the Roots algorithm is due to the fast clustering towards zero of the roots of the Euler-Frobenius polynomials. In fact, starting from m = 33 the smaller root of a(z) returned by the routine roots of Matlab is exactly zero. An indication of this clustering as the degree of the polynomial increases is given in Figure 6, where its zeros interior to the unit circle are plotted for m = 20. 0

10

−2

10

MinPh

−4

10

Cepstral

−6

10

Bauer

−8

10

−10

10

−12

Wilson

10

−14

Roots

10

−16

10

1

5

10

15

20

25

30

35

40

45

Figure 3

A pecularity of the Wilson method is the \sawtooth" pro le of the error curve. In fact, to reach the prescribed accuracy it may be necessary, for a certain degree m , to perform an additional iteration with respect to m = m ? 1. When this happens, quadratic convergence drops the error to values much lower than the required accuracy, generating a new peak in the error curve. The numbers of the iterations required by the rst two methods for various values of m, reported in Figure 4, show that while the Wilson method reaches the prescribed accuracy for m  39, the Bauer method converges only for m  22. Finally, the computational complexity of the rst three algorithms is illustrated in Figure 5, where

18 8

400

10

350

7

10

300

Bauer

6

10

250

Bauer

Wilson

Wilson Roots

5

10

200 4

10

150 3

10

100

2

10

50 0

1

5

10

15

20

25

Figure 4

30

35

40

45

10

5

10

15

20

25

30

35

40

45

Figure 5

the number of ops ( oating-point operations) performed to compute the spectral factor is plotted as a function of m. This gure shows in particular that the Roots method (when reliable) requires a lower number of ops, and that the Wilson method is less computationally expensive than the Bauer method, when both converge. In conclusion we think that if m  30, a hypothesis generally satis ed in applications, the rst three methods are all reliable, even though the best performance is provided by the Roots and Wilson algorithms.

Wavelet polynomials. The eectiveness of the algorthms in this case has been measu-

red by using the same error indices (3.3) and (3.4) that we used in the Euler-Frobenius case. Note that the coecients aj used in (3.3) are very accurate, because they have been computed symbolically from the Laurent expansion (1.6) of polynomial dm (z). It is worthwhile to note the most signi cant dierences in the implementation of the ve algorithms. Namely, while the rst two methods require the numerical evaluation of the coecients of the Laurent polynomial (1.6), both the MinPh and Cepstral algorithms are applied directly to polynomial (1.5). The Roots algorithm instead, as already pointed out, has been implemented by the same strategy used in [18], i.e. by looking for the roots of the polynomial bm (x) after applying a scaling technique. We remark that there is an alternative procedure which gives essentially the same results without rescaling the coecients. It consists in evaluating the zeros of the polynomial ~bm (x) = xm bm (x?1 ), whose roots are reciprocal to those of bm (x) and that can be obtained by simply reversing the order of the coecients of bm . As the zeros of bm are inside the unit circle, this transformation ensures that the zeros of the new polynomial are better separated than those of bm . Figure 7 shows the displacement of the zeros of bm (x) (O markers) and of ~bm (x) (X markers) for m = 10. Concerning the location of the zeros of bm on the complex plane, Shen and Strang [18] proved the following result.

Theorem.

For

m = 1 the only zero of bm (x) is given by x = ?1=2, while for m > 1

19 1 3

0.8 2

0.6

Imaginary part

Imaginary part

0.4 0.2 0 −0.2

1 0 −1

−0.4 −0.6

−2

−0.8 −3

−1 −1

−0.5

0

0.5

1

−5

−4

−3

−2

−1

0

1

2

3

Real part

Real part

Figure 6

Figure 7

all the zeros satisfy the inequalities:

jxj < 12 ;

j4x(1 ? x)j > 21=(m+1):

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

−0.1

−0.1

−0.2

−0.2

−0.3

−0.3

−0.4

−0.4

−0.5 −0.5

0

Figure 8

0.5

−0.5 −0.5

0

0.5

Figure 9

A geometric visualization of this result is given in Figures 8 and 9. Speci cally, Figure 8 shows the location of the zeros of b80 in the domain delimited by the curves jxj = 1=2 and j4x(1 ? x)j = 21=81 , while Figure 9 exhibits the displacement of the zeros of bk , k = 5; 10; : : : ; 80 with respect to the limiting curve j4x(1 ? x)j = 1. For m > 80 the precision of the zeros furnished by the Roots algorithms gradually degrades and for m  85 some of the zeros fall outside the region de ned in the above theorem. Although these two gures have already been reported in the paper by Shen and Strang, we show them here to better highlight the performance of the Roots method in the spectral factorization of wavelets polynomials.

20 0

0

10

10

Wilson

−2

Cepstral

MinPh

10

−4

MinPh Bauer

−4

10

10

−6

Roots

10

−8

Cepstral

−6

Roots

10

−8

10

10

Bauer

−10

−10

10

10

−12

−12

10

10

−14

−14

10

10

−16

10

Wilson

−2

10

−16

5

10

15

20

25 30 radius = 1

35

40

45

10

5

10

15

20

25 30 radius = 10

35

40

45

Figure 10

The main results concerning the performance of the ve tested methods are shown in Figure 10 and Figure 11, where we plot Em; for  = 1; 10 and Em respectively against m. Primarily we note that, although the Roots algorithm allows one to evaluate with sucient accuracy the zeros of the Daubechies polynomials for m  80, all the methods, with respect to our quality index, give unreliable results for m > 50. This fact suggests that, for growing values of m, the recovery of the coecients of the polynomial dm (z) from its zeros is an increasingly ill-conditioned problem. The graph of Em;1 reported in Figure 10 shows that the quality of the spectral factor computed by the Cepstral algorithm is comparable with those of the Roots method on the unit circle, while the values of Em;10 con rm that the factorization performed by the Cepstral method becomes less trustworthy on a circle of radius 10. On the contrary the results furnished by the MinPh method are poor even on the unit circle. The values of Em , plotted in Figure 11, are in accordance with those of Em;10, and show that the best results were obtained by the Roots method and that the Wilson algorithm, when numerically convergent, reaches a comparable accuracy. The main advantage of the Roots method in this case is that the coecients of the polynomial bm are analytically given and, thanks to one of the two mentioned strategies, it is possible to estimate accurately the zeros of bm , and consequently those of dm , for large values of m. We note that the Cepstral method is much more eective in this case than in the Euler-Frobenius case. This dierent performance is due to the behaviour of the ratio max j j j= min j j j, j = 0; 1; : : : ; m, that for 1  m  30 in this case does not exceed 1013, while in the previous example is close to 1081. The Bauer method, in this as well as in the preceding example, generates good results only for moderately large values of m. Its main drawback, due to the linear convergence, is that the large number of iterations necessary to reach convergence, as m increases, tends to propagate too much the rounding errors. Moreover, for m > 28, the matrix

21 0

10

−2

10

MinPh

Cepstral

−4

10

Bauer

−6

10

Roots

−8

10

−10

10

Wilson

−12

10

−14

10

−16

10

1

5

10

15

20

25

30

35

40

45

50

Figure 11

An to be factorized at each step is not numerically positive de nite. The Wilson method is numerically more stable. It reveals instability whenever at some iteration the conditioning of the linear system to be solved is very large, and this happens only for large values of m. On the contrary the MinPh algorithm is reliable only for very small degree. In fact the condition number of the Toeplitz matrix involved in the linear system to be solved grows rapidly with m, and furthermore this matrix is numerically singular for m > 28. Other Laurent polynomials. The numerical simulation concerning the spectral factorization of the Euler-Frobenius and the Daubechies polynomials, suggests to us the following conjectures: (1) The Bauer and Wilson methods are both reliable, even though the Wilson algorithm is more eective than the Bauer method. In particular the Bauer algorithm fails when the matrix An to be factorized at some iteration n becomes numerically not positive de nite. Furthermore propagation of rounding errors often leads to numerical divergence. (2) The Roots method is very eective whenever the zeros of the Laurent polynomial that we must factorize are numerically well separated. (3) The Cepstral and, especially, the MinPh methods have a poor performance if the ratio max jaj j= min jaj j is very large. This fact is a direct consequence of the use of the FFT, as already pointed out. On the basis of these comments we present some examples which tend to con rm their

22

validity in general. To this end we constructed a set of test polynomials, (z), having roots outside the unit disc and then we evaluated the coecients of the corresponding Laurent polynomials a(z) = (z) (z?1 ). The diculty in recovering (z) by the spectral factorization of a(z) essentially depends on the minimum distance among its zeros, on their distance from the unit circle and the value of the ratio max j j j= min j j j. Given a test problem, we recover the spectral factor and estimate the eectiveness of the results by the error index

em

v u m u X =t

!2

j =?m

j ? ~j ;

j

 =



j

j ; j 6= 0 1; j = 0

where f j g and f ~j g are the coecients of the true factor and of the factor ~ that we have computed. 70

0

10

10

−2

60

10

10

MinPh

−4

Cepstral

10

50

10

−6

10

40

10 −8

10

Roots

−10

30

10

Bauer

10

20

10

−12

10

Wilson

−14

10

10

10

0

−16

10

1

5

10

15

20

25

30

35

40

45

Figure 12

50

10

1

5

10

15

20

25

30

35

40

45

50

Figure 13

For the purpose of reproducing the particular pattern of the zeros of the EulerFrobenius polynomials, we consider the Laurent polynomial whose spectral factor is

1(z) =

m X j =0

j(1)zj

:=

m Y k=1

(z ? (1 + k)); m = 1; 2; : : : ; 50:

Figure 12 shows that the behaviour of the error index em is very similar to that of Em for the Euler-Frobenius polynomials. This example is somehow \easier" to solve for the Roots method since the clustering of the roots is less dramatic than in the EulerFrobenius case. Furthermore, the Bauer, Wilson and Roots methods give very low errors for 1  m  50. On the contrary, the MinPh and Cepstral methods become quickly unreliable because the ratio ? := max j j j= min j j j increases rapidly as m grows, as shown in Figure 13 where this ratio has been plotted in logarithmic scale.

23 0

10

−2

10

1

Bauer

Roots

0.8 0.6

−4

10

0.4 Imaginary part

−6

10

MinPh −8

10

Cepstral

−10

10

0.2 0 −0.2 −0.4 −0.6

−12

10

−0.8 −14

10

−1 Wilson

−16

10

1

5

10

15

20

25

30

35

40

45

50

55

−1

60

−0.5

0

0.5

1

Real part

Figure 14

Figure 15

To emphasize the relevance of the ratio max j j j= min j j j, especially for the Cepstral and the MinPh methods, we consider the following polynomial

2(z) := 10 +

m X j =1

zj ; m = 1; 2; : : : ; 60:

In this case the ratio ? is 10 and, as shown in Figure 14, both the MinPh and the Cepstral methods give very good results. Indeed, they are better than those given by Bauer and, at least for m  40, they are comparable with those of Wilson. On the contrary the error generated by the Roots method grows rather rapidly with m, as a consequence of the progressive clustering of the zeros of 2(z). This is evident in Figure 15 where the zeros of 2(z) are depicted for m = 30. 10

0

10

10

9

−2

10

−4

10

10

8

10

7

10 −6

Roots

10

6

10

5

−8

10

10

−10

4

10

Bauer

10

3

10

−12

10

Wilson

2

10

Cepstral

−14

10

1

10

MinPh

0

−16

10

1

5

10

15

20

25

30

Figure 16

35

40

45

50

10

1

5

10

15

20

25

30

35

40

45

50

Figure 17

In order to investigate the in uence of the distance of the zeros from the unit circle, we applied the ve algorithms to the recovery of the sequence of polynomials of constant

24

degree 30

30 1

3 (z) := 1 + n ? z30 ; n = 1; 2; : : : ; 50; having its zeros equispaced on a circle whose radius tends to 1 as n goes to in nity. The results are shown in Figure 16. The rst observation is that the two methods based on the FFT give very good results as the radius gets closer to one, taking advantage of the decreasing ratio ? = (1 + n1 )30 , displayed in Figure 17. The Roots method seems to be totally insensitive on the radius of the circle. Indeed its results are inaccurate for each value of n, because the roots are quite close. The Bauer and Wilson methods have no particular trouble in recovering 3(z). Wilson produces a null error for several values of n, as the breaks in the dashed line of Figure 16 show, while Bauer exhibits a degradation in the results only for n > 35, as the distance of the zeros from the unit circle becomes less than 1=35. 



0

0

10

10

−2

−2

10

Roots

10

Roots

−4

Bauer

−4

10

10 MinPh

−6

10

−8

−6

10

MinPh

−8

Cepstral

10

10

−10

−10

10

Cepstral

10 Wilson

−12

10

−12

10

−14

−14

10

10 Bauer

−16

10

1

Wilson

−16

5

10

15

20

25

30

Figure 18

35

40

45

50

10

1

5

10

15

20

25

30

35

40

45

50

Figure 19

The last example emphasizes the dependence of the results on the distance from the unit circle. The spectral factor sought for is

4;(z) := m ? zm ; where m = 1; 2; : : : ; 50 and  > 1: The values of the error index em are drawn in Figure 18 and 19 for  = 2 and  = 1:01 respectively. In this case the radius  of the circle is xed, while the density of the zeros increases with m. Among the ve methods, only Bauer is sensitive to the decreasing of the distance of the zeros from the unit circle. In fact, while for  = 2 it converges in a small number of iterations giving extremely low or null errors, for  = 1:01 it cannot reach the required accuracy. Conversely, the Wilson, MinPh and Cepstral methods take advantage of the smaller radius, while the performance of the Roots method, as already pointed out, is aected only by the increasing number of zeros. Conclusions. Our numerical experiments make it evident that the performance of the methods in computing the spectral factorization of a Laurent polynomial may be

25

in uenced by the following three parameters: the ratio max jaj j= min jaj j, the minimum distance of the zeros of a(z) from the unit circle and the minimum distance between the zeros of a(z). Among the ve methods that we have studied, only the Wilson method is not signi cantly in uenced by these parameters, and can be considered as an all purpose method. However, it suers from numerical instability whenever, at some iteration, the conditioning of the linear system that we have to solve is very large. The Bauer method is in uenced by the distance of the zeros of a(z) from the unit circle, but it is independent of the values of the two other mentioned parameters, as the last examples emphasizes. This conclusion is not surprising. Indeed, for moderately large values of m, it is generally reliable provided that the matrices that we have to factorize at each iteration are strictly numerically positive de nite, and this depends, of course, on the distance of the zeros of a(z) from the unit circle. The Bauer method is generally less eective than the Wilson method, because they are both iterative methods with linear and quadratic convergence respectively. The Roots method, implemented with care, can be very eective. Nevertheless, its performance is strongly in uenced by the minimum distance between the zeros of a(z). Concerning the last two methods, that is, the Cepstral method and the MinPh methods, the rst one is the most reliable. However, both methods are crucially in uenced by the ratio max jaj j= min jaj j and the results are acceptable only if this ratio is not too large. This is particularly evident in the last two examples. Is is worthwhile to remark that whenever the coecients ratio is large, the reliability of the results furnished by these two algorithms, and especially by the Cepstral method, decreases as the distance from the unit circle increases. In other words, while equation (1.3) is satis ed near the unit circle, its validity degrades as the distance from it increases. We think that the popularity of the Cepstral method in Signal analysis depends mainly on its easy and ecient implementation, due to the use of the FFT, and on the absence of a very large ratio max jaj j= min jaj j in commonly encountered applied problems. References

[1] K. Amaratunga, A fast Matlab routine for calculating Daubechies lters, Wavelet Digest 4(4) (1995). [2] F.L. Bauer, Ein direktes Iterations Verfahren zur Hurwitz-zerlegung eines Polynoms, Arch. Elektr. Uebertragung 9 (1955), 285{290. [3] F.L. Bauer, Beitrage zur Entwicklung numerischer Verfahren fur programmgesteuerte Rechenanlagen, II. Direkte Faktorisierung eines Polynoms, Sitz. Ber. Bayer. Akad. Wiss. (1956), 163{203. [4] B.P. Bogert, M.J.R. Healy and J.W. Tukey, The quefrency alanysis of time serier for echoes: cepstrum pseudo-autocovariance, cross-cepstrum and saphe cracking, Proc. Symposium Time Series Analysis (M. Rosenblatt, eds.), John Wiley and Sons, New York, 1963, pp. 209{243. [5] A. Calderon, F. Spitzer and H. Widom, Inversion of Toeplitz matrices, Illinois J. Math. 3 (1959), 490{498. [6] A.S. Cavaretta, W. Dahmen and C.A. Micchelli, Stationary subdivision, Mem. Amer. Math. Soc. 93 (1991), 1{186. [7] I. Daubechies, Ten Lectures on Wavelets, vol. 61, CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM, Philadelphia, PA, 1992.

26 [8] W. Dahmen and C.A. Micchelli, Using the re nement equation for evaluating integrals of wavelets, SIAM J. Numer. Anal. 30(2) (1993), 507{537. [9] C. de Boor, A Practical Guide to Splines, vol. 27, Applied Mathematical Sciences, Springer-Verlag, New York, 1978. [10] G.H. Golub and C.F. Van Loan, Matrix Computations, second edition, The John Hopkins University Press, Baltimore, 1989. [11] T.N.T. Goodman, C.A. Micchelli, G. Rodriguez and S. Seatzu, On the Cholesky factorization of the Gram matrix of locally supported functions, BIT 35(2) (1995), 233{257. [12] T.N.T. Goodman, C.A. Micchelli, G. Rodriguez and S. Seatzu, On the limiting pro le of exponentially decaying functions, preprint (1996). [13] M.G. Krein, Integral equations on the half-line with kernel depending upon the dierence of the arguments, Uspehi Mat. Nauk. 13(5) (1958), 3{120 (Russian); English translation, AMS Translations 22 (1962), 163{288. [14] Matlab Version 4.2c, The MathWorks Inc., South Natick, MA, 1994. [15] C.A. Micchelli, Mathematical Aspects of Geometric Modeling, vol. 65, CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM, Philadelphia, PA, 1995. [16] A.V. Oppenheim and R.W. Shafer, Discrete-Time Signal Processing, Prentice Hall Signal Processing Series, Englewood Clis, NJ, 1989. [17] F. Riesz and B. Sz.-Nagy, Functional Analysis, Frederick Ungar, New York, 1955. [18] J. Shen and G. Strang, The zeros of the Daubechies polynomials, Proc. Amer. Math. Soc. (1996). [19] I.J. Schoenberg, Cardinal Spline Interpolation, vol. 12, CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM, Philadelphia, PA, 1973. [20] G. Strang and T. Nguyen, Wavelets and Filter Banks, Wellesley-Cambridge Press, Wellesley, MA, 1996. [21] G. Wilson, Factorization of the covariance generating function of a pure moving average process, SIAM J. Numer. Anal. 6(1) (1969), 1{7.