Optimal preconditioners for systems defined by functions of Toeplitz ...

Optimal preconditioners for systems defined by functions of Toeplitz matrices

arXiv:1802.03622v1 [math.NA] 10 Feb 2018

Sean Hon1 Mathematical Institute, University of Oxford, Radcliffe Observatory Quarter, Oxford, OX2 6GG, United Kingdom

Abstract We propose several circulant preconditioners for systems defined by some functions g of Toeplitz matrices An . In this paper we are interested in solving g(An )x = b by the preconditioned conjugate method or the preconditioned minimal residual method, namely in the cases when g(z) are the functions ez , sin z and cos z. Numerical results are given to show the effectiveness of the proposed preconditioners. Keywords: Toeplitz matrices, Functions of matrices, Circulant preconditioners, PCG, PMINRES MSC: 65F08, 65F10, 65F15, 15A16, 15B05 1. Introduction Motivated by [16] in which the authors proposed some optimal preconditioners for certain functions of general matrices, we show that g(c(An )), where c(An ) is the optimal circulant preconditioner for An first proposed in [9], is an effective preconditioner for g(An ). Specifically we are interested in the cases when g(z) are the trigonometric functions ez , sin z and cos z. A crucial application of eAn , for example, arises from the discretisation of integro-differential equations with a shift-invariant kernel [17]. Solving those equations is often required in areas such as the option pricing [12, 28]. Related work on computing the exponential of a block Toeplitz matrix arising in approximations of Markovian fluid queues can also be found in [2].

1

Email address: [email protected] (Sean Hon) The research of the author was partially supported by Croucher Foundation.

Preprint submitted to Journal

February 13, 2018

Over the past few decades, preconditioning for Toeplitz matrices with circulant matrices has been extensively studied. Strang [30] and Olkin [22] were the first to propose using circulant matrices as preconditioners in this context. Theoretical results that guarantee fast convergence with circulant preconditioners were later given by [7]. Other circulant preconditioners such as optimal preconditioners [9], Huckle’s preconditioners [14] and superoptimal preconditioners [31] were then developed for certain classes of Hermitian and positive definite Toeplitz matrices generated by positive functions f . The restriction on f was later relaxed for example in [5, 29, 11]. Some work had also been done on preconditioning for Hermitian indefinite Toeplitz systems [6], non-Hermitian Toeplitz systems [15] and nonsymmetric Toeplitz systems [25]. For references on the development of preconditioning of Toeplitz matrices we refer to [20, 4]. Throughout this work we consider the case when f is a 2π-periodic continuous complex-valued function as analysed in [8], thus the corresponding Toeplitz matrix An [f ] is in general complex and non-Hermitian. Consequently g(An [f ]) is also a non-Hermitian complex matrix. We let cn [f ] be the optimal circulant preconditioner [9] derived from An [f ]. Using g(cn [f ]) as the preconditioner, we can then apply CG to the normal equations system [g(cn [f ])−1 g(An [f ])]∗ [g(cn [f ])−1 g(An [f ])]x = [g(cn [f ])−1 g(An [f ])]∗ g(cn [f ])−1 b. We also consider the special case in which we can use MINRES for the Hermitian indefinite g(An [f ]) with the preconditioner g(cn [f ]). Given a circulant matrix Cn , we remark that g(Cn ) is also a circulant matrix. By the diagonalisation Cn = Fn∗ Λn Fn , where Fn is a Fourier matrix [10] in which the entries are given by [Fn ]jk = √1n e−2πijk/n with j, k = 0, 1, . . . , n − 1, we have g(Cn ) = Fn∗ g(Λ)Fn . Therefore, for any vector d the product g(Cn )−1 d can by efficiently computed by several Fast Fourier Transforms (FFTs) in O(n log n) operations [3]. It must be noted however that fast matrix vector multiplication with the matrix g(An [f ]) is not readily archived by circulant embedding. For eAn [f ] , the matrix vector multiplication can be computed efficiently for example by a fast algorithm in O(n log n) operations [18]. This paper is outlined as follows. In section 2 we first provide some lemmas on bounds for the spectra of An [f ] and cn [f ]. In section 3 we also give several lemmas on functions of matrices. In section 4 we provide the 2

main results on the preconditioned matrix g(cn [f ])−1 g(An [f ]). In section 5 we present numerical results to demonstrate the effectiveness of the proposed preconditioners. 2. Spectra of cn [f ] and An [f ] Denote by C2π the Banach space of all 2π-periodic continuous complexvalued functions equipped with the supremum norm k · k∞ . For all f ∈ C2π , we let Z π 1 f (θ)e−ikθ dθ, k = 0, ±1, ±2, . . . , ak = 2π −π be the Fourier coefficients of f . Let An [f ] be the n-by-n complex Toeplitz matrix with the (j, k)-th entry given by aj−k . The function f is called the generating function of the matrix An [f ]. We then have   a0 a−1 · · · a−n+2 a−n+1  a1 a0 a−1 a−n+2     .. ..  ..  . a1 a0 .  An [f ] =  . .   .. .. an−2 . . a−1  an−1 an−2 · · · a1 a0 We also let cn [f ] be the n-by-n optimal circulant preconditioner [9] for An [f ], namely   c0 cn−1 · · · c2 c1  c1 c0 cn−1 c2     .. ..  ..  . c1 c0 .  cn [f ] =  .    . . . . cn−2 . . cn−1  cn−1 cn−2 · · · c1 c0 defined by ( ck =

(n−k)ak +kak−n , n

cn+k ,

0 ≤ k < n, − n < k < 0.

Lemma 2.1. [8, Lemma 1 and 3] If f ∈ C2π we have kAn [f ]k2 ≤ 2kf k∞ and kcn [f ]k2 ≤ 2kf k∞ 3

n = 1, 2, . . . .

Lemma 2.1 states that the 2-norm of the circulant matrix and that of the Toeplitz matrix generated by f are bounded by a constant which is independent of n. Lemma 2.2. [8, Theorem 1] Let f ∈ C2π . Then for all  > 0 there exists a positive integer M > 0 such that for n > 2M , we have cn [pM ] − An [pM ] = Un − Wn , where pM is a trigonometric polynomial such that kf − pM k∞ < , An [pM ] ∈ Cn×n is the Toeplitz matrix generated by pM , cn [pM ] ∈ Cn×n is the optimal circulant preconditioner for An [pM ], rank Un ≤ 2M and kWn k2 ≤

1 M (M + 1)( + kf k∞ ). n

Lemma 2.2 indicates that the difference between the circulant matrix and the Toeplitz matrix generated by a trigonometric approximation to f can be decomposed into the sum of a matrix of low rank and a matrix of small norm. In the next section, this lemma is used to prove that the difference between the matrix exponential of a circulant matrix and that of a Toeplitz matrix can also be decomposed in a similar fashion. 3. Preliminaries on matrix functions In this section we introduce the preliminaries that will be used in the following section. Definition [13] For any An ∈ Cn×n , 1 1 2 An + A3n + · · · , 2! 3! 1 1 1 cos An = In − A2n + A4n − A6n + · · · 2! 4! 6! eAn = In + An +

and sin An = An −

1 3 1 1 An + A5n − A7n + · · · . 3! 5! 7! 4

Lemma 3.1. [13, Theorem 10.2] For any An , Bn ∈ Cn×n e(An +Bn )t = eAn t eBn t for all t if and only if An Bn = Bn An . Definition [13] Given a vector norm on Cn , the corresponding subordinate matrix norm is defined by kAn k = max x6=0

kAn xk . kxk

Definition [13] The norm k · k is called consistent if kABk ≤ kAkkBk for all A ∈ Cm×n and B ∈ Cn×p . Lemma 3.2. [13, Problem 10.3] For any subordinate matrix norm and any An , Bn ∈ Cn×n , we have keAn − eBn k ≤ kAn − Bn kemax (kAn k,kBn k) . Lemma 3.3. [13, Theorem 10.1] For An ∈ Cn×n , let Pr,s

r X 1 An s ( )] . =[ i! s i=0

Then for any consistent matrix norm keAn − Pr,s k ≤

kAn kr+1 kAn k e sr (r + 1)!

and lim Pr,s = lim Pr,s = eAn .

r→∞

s→∞

Lemma 3.4. [13, Theorem 10.10] For An ∈ Cn×n and any subordinate matrix norm, e−kAn k ≤ keAn k ≤ ekAn k n = 1, 2, . . . . Lemma 3.5. [24] For any circulant matrix Cn ∈ Cn×n , the absolute value circulant matrix |Cn | is defined to be 1

1

|Cn | = (Cn∗ Cn ) 2 = (Cn Cn∗ ) 2 = Fn∗ |Λn |Fn , where |Λn | is the diagonal matrix in the eigenvalue decomposition of Cn with all entries replaced by their magnitudes. 5

4. Spectra of the preconditioned matrices In this section, for Hermitian g(An [f ]) we show the preconditioned matrix g(cn [f ])−1 g(An [f ]) can be decomposed into the sum of a unitary matrix, a matrix of low rank and a matrix of small norm for sufficiently large n under some assumptions. For non-Hermitian g(An [f ]), we consider its normal equations system and also show that the preconditioned matrix can also be decomposed in a similar way. 4.1. Spectra of (ecn [f ] )−1 eAn [f ] We first provide some lemmas concerning to the matrix exponential and then give the main results concerning to the preconditioned matrix (ecn [f ] )−1 eAn [f ] . Corollary 4.1. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. We have k(ecn [f ] )−1 k2 ≤ e2kf k∞ n = 1, 2, . . . . Proof By Lemma 3.1, we have ecn [f ] e−cn [f ] = ecn [f ]−cn [f ] = In . Thus e−cn [f ] is the inverse of ecn [f ] . We then have k(ecn [f ] )−1 k2 = ke−cn [f ] k2 . Using lemmas 2.1 and 3.4, we have ke−cn [f ] k2 ≤ ekcn (f )k2 ≤ e2kf k∞ .  We are now ready to give our main results on the spectrum of (ecn [f ] )−1 eAn [f ] . Theorem 4.2. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. For all  > 0, there exist positive integers N and M such that for all n > N ecn [f ] − eAn [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, kEn [f ]k2 ≤ . 6

Proof Since f ∈ C2π , by Weierstrass theorem [26, Theorem 6.1], for any  > 0 there exists M ∈ N and a trigonometric polynomial pM (θ) =

k=M X

ρk eikθ

k=−M

such that kf − pM k∞ ≤ .

(1)

For all n > 2M we decompose cn [pM ] An [pM ] An [pM ] cn [pM ] − − eAn [f}] , ecn [f ] − eAn [f ] = |ecn [f ] − {ze } + |e {z e } + |e {z G1

B

G2

where An [pM ] ∈ Cn×n is the Toeplitz matrix generated by pM and cn [pM ] ∈ Cn×n is the optimal circulant preconditioner for An [pM ]. We first want to show that G1 + G2 is of small norm. Using Lemmas 2.1, 3.2 and (1) we have kG1 + G2 k2 ≤ kecn [f ] − ecn [pM ] k2 + keAn [f ] − eAn [pM ] k2 ≤ kcn [f ] − cn [pM ]k2 emax (kcn [f ]k2 ,kcn [pM ]k2 ) + kAn [f ] − An [pM ]k2 emax (kAn [f ]k2 ,kAn [pM ]k2 ) ≤ (kcn [f ] − cn [pM ]k2 + kAn [f ] − An [pM ]k2 )e2 max (kf k∞ ,kpM k∞ ) ≤ (2kf − pM ]k∞ + 2kf − pM ]k∞ )e2 max (kf k∞ ,kpM k∞ ) ≤ (4e2 max (kf k∞ ,kpM k∞ ) ). (2) We further rewrite cn [pM ]

B=e |

K K K K X X X X 1 1 1 1 i i i cn [pM ] + cn [pM ] − An [pM ] + An [pM ]i − eAn [pM ] , − i! i! i! i! i=0 {z } |i=0 {z i=0 } |i=0 {z } B1

D

B2

where K is a positive integer. We are now to measure the norm of B1 + B2 . Using Lemmas 2.1, 3.3 and

7

(1), we have kB1 + B2 k2 ≤ kecn [pM ] −

K K X X 1 1 cn [pM ]i k2 + k An [pM ]i − eAn [pM ] k2 i! i! i=0 i=0

kcn [pM ]k2K+1 kcn [pM ]k2 kAn [pM ]kK+1 2 + ≤ e ekAn [pM ]k2 (K + 1)! (K + 1)! K+1 K+1 kcn [pM ]k2 kAn [pM ]k2 ≤ ( + )e2kpM k∞ (K + 1)! (K + 1)! (2kpM k∞ )K+1 2kpM k∞ ≤ 2e =: K (K + 1)! which converges to zero as K goes to infinity. Therefore for a given K > 0, there exists an integer K such that kB1 + B2 k2 ≤ K ≤ .

(3)

We next show that D can be decomposed into a sum of a matrix of low rank and a matrix of small norm. Firstly, we observe that cn [pM ] − An [pM ] = Un − Wn , where n−M ρM n

      Un =   n−M   n ρ−M  .. ..  . . n−1 ρ−1 · · · n

n−M ρ−M n

8

··· .. .

n−1 ρ1 n

.. .



   n−M  ρ M n       

and 

0

 1  ρ1  n  ..  .  M  ρM n Wn =         

1 ρ n −1

..

.

..

.

..

.

..

.

··· .. . .. . .. . .. . .. .

M ρ n −M



..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

..

.

.

..

.

..

..

.

..

M ρ n M

. ···

        .  M  ρ −M  n ..  .    1 ρ−1  n

1 ρ n 1

0

Rewrite D as K K X X 1 1 i D = cn [pM ] − An [pM ]i i! i! i=0 i=0

=

K X 1 (cn [pM ]i − An [pM ]i ) i! i=1

=

K i−1 X 1 X ( cn [pM ]j (cn [pM ] − An [pM ])An [pM ]i−1−j ) i! j=0 i=1

K i−1 X 1 X = ( cn [pM ]j (Un − Wn )An [pM ]i−1−j ) i! i=1 j=0 i−1 K i−1 K X X 1 X 1 X j i−1−j = ( cn [pM ] Un An [pM ] )+ ( cn [pM ]j Wn An [pM ]i−1−j ) . i! j=0 i! j=0 i=1 i=1 | {z } | {z } J

Rn [f ]

9

Using Lemmas 2.1 and 2.2, we can estimate the norm of J: kJk2

K i−1 X 1X = k cn [pM ]j Wn An [pM ]i−1−j k2 i! i=1 j=0

≤

i−1 K X 1 X k cn [pM ]j Wn An [pM ]i−1−j k2 i! j=0 i=1

i−1 K X 1X kcn [pM ]kj2 kAn [pM ]k2i−1−j ≤ kWn k2 i! j=0 i=1

≤ kWn k2

i−1 K X 1X (2kpM k∞ )j (2kpM k∞ )i−1−j i! j=0 i=1

K i−1 X 1X = kWn k2 (2kpM k∞ )i−1 i! i=1 j=0

= kWn k2

K X i=1

≤ kWn k2

∞ X

1 (2kpM k∞ )i−1 (i − 1)! 1 (2kpM k∞ )i−1 (i − 1)!

i=1 2kpM k∞

= kWn k2 e 1 ≤ M (M + 1)( + kf k∞ )e2kpM k∞ . n

(4)

We now show that rank Rn [f ] ≤ 2KM by first investigating the structure of Rn [f ]. Similar to the approach used in the proof of Lemma 3.11 in [21], simple computations show that   ♦ ··· ♦ ♦ ··· ♦  .. . .. ..   . ♦ .. . ♦ .   ♦ · · · ♦ ♦ · · · ♦     α β , cn [pM ] Un An [pM ] =       ♦ · · · ♦ ♦ · · · ♦    .. .. ..  .. . ♦ . . ♦ . ♦ ··· 10

♦

♦ ···

♦

where the diamonds represent the non-zero entries which appear only in the four (α + 1)M by (β + 1)M blocks in the corners, provided that n is larger than 2 max(α + 1, β + 1)M . Since the rank of i−1 K X 1 X ( cn [pM ]j Un An [pM ]i−1−j ) Rn [f ] = i! j=0 i=1

P j K−1−j is determined by that of K−1 which is a block maj=0 cn [pM ] Un An [pM ] trix with only four non-zero KM by KM blocks in its corners, it follows that the rank of Rn [f ] is less than or equal to 2KM if we assume n > 2KM . Considering (4), we pick N := max {M (M + 1)(1 +

kf k∞ 2kpM k∞ )e , 2KM }, 

and it follows that for all n > N we have kJk2 ≤ . Further combining this result with (2) and (3), we conclude that for all n > N kEn [f ]k2 = kG1 + B1 + J + B2 + G2 k2 ≤ (4e2 max (kf k∞ ,kpM k∞ ) + 2).  Corollary 4.3. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. For all  > 0, there exist positive integers N and M such that for all n > N bn [f ] + E bn [f ], (ecn [f ] )−1 eAn [f ] = In + R where bn [f ] ≤ 2M, rank R bn [f ]k2 ≤ . kE Proof By Theorem 4.2, we know that for all  > 0, there exist positive integers N and M such that for all n > N ecn [f ] − eAn [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, 11

kEn [f ]k2 ≤ . By Corollary 4.1 we know that k(ecn [f ] )−1 k2 is uniformly bounded with respect to n, so that we have (ecn [f ] )−1 eAn [f ] = In + (ecn [f ] )−1 (eAn [f ] − ecn [f ] ) = In + (ecn [f ] )−1 (−Rn [f ]) + (ecn [f ] )−1 (−En [f ]) . | {z } | {z } cn [f ] R

cn [f ] E

The result follows.



Remark Since An [f ] is Hermitian when f is real-valued, we can write An [f ] = ZnT Dn Zn where Dn is a diagonal matrix with real eigenvalues di being the eigenvalues of An [f ]. We immediately see that eAn [f ] = ZnT eDn Zn is positive definite as its eigenvalues are all of the form edi > 0. Thus CG can be used in this case. Next, for the more general case when eAn [f ] is non-Hermitian, we can use CG for the normal equations system with the preconditioner ecn [f ] . Corollary 4.4. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. For all  > 0, there exist positive integers N and M such that for all n > N [(ecn [f ] )−1 eAn [f ] ]∗ [(ecn [f ] )−1 eAn [f ] ] = In + Rn [f ] + E n [f ], where rank Rn [f ] ≤ 4M, kE n [f ]k2 ≤ . Proof By Corollary 4.3, we know that for all  > 0 there exist positive integers N and M such that for all n > N bn [f ] + E bn [f ], (ecn [f ] )−1 eAn [f ] = In + R where bn [f ] ≤ 2M, rank R bn [f ]k2 ≤ . kE 12

We then have [(ecn [f ] )−1 eAn [f ] ]∗ [(ecn [f ] )−1 eAn [f ] ] cn [f ] + E cn [f ])∗ (In + R bn [f ] + E bn [f ]) = (In + R b [f ]∗ (In + R bn [f ] + E bn [f ]) + (In + E bn [f ]∗ )R bn [f ] = In + R |n {z } Rn [f ]

∗

∗

b [f ] + E bn [f ] + E b [f ] E bn [f ] . + E {z n } |n E n [f ]

It immediately follows that rank Rn [f ] ≤ 4M and kE n [f ]k2 ≤ 2 + 2.  Since [(ecn [f ] )−1 eAn [f ] ]∗ [(ecn [f ] )−1 eAn [f ] ] in Corollary 4.4 is Hermitian, by Weyl’s theorem we know that its eigenvalues are mostly close to 1 when n is sufficiently large. 4.2. Spectra of (sin cn [f ])−1 sin An [f ] and (cos cn [f ])−1 cos An [f ] In this subsection we directly show that similar results hold for (sin cn [f ])−1 sin An [f ] and (cos cn [f ])−1 cos An [f ] using the theorems on (ecn [f ] )−1 eAn [f ] . Theorem 4.5. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. For all  > 0, there exist positive integers N and M such that for all n > N sin cn [f ] − sin An [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, kEn [f ]k2 ≤ . Proof By Theorem 4.2, we know that for all  > 0 there exist positive integers N and M such that for all n > N ecn [f ] − eAn [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, kEn [f ]k2 ≤ . 13

Using the fact that sin An =

eiAn −e−iAn 2i

for any An ∈ Cn×n , we write

eicn [f ] − e−icn [f ] eiAn [f ] − e−iAn [f ] )−( ) 2i 2i e−icn [f ] − e−iAn [f ] eicn [f ] − eiAn [f ] )−( ) ( 2i 2i ecn [if ] − eAn [if ] ecn [−if ] − eAn [−if ] ( )−( ) 2i 2i Rn [if ] + En [if ] Rn [−if ] + En [−if ] ( )−( ) 2i 2i Rn [if ] − Rn [−if ] En [if ] − En [−if ] ( )+( ). 2i 2i | {z } | {z }

sin cn [f ] − sin An [f ] = ( = = = =

Rn [f ]

En [f ]

Since Rn [if ] and Rn [−if ] are both block matrices with only four non-zero M by M blocks in their corners, we see that the rank of Rn [f ] is less than or equal to 2M . Also En [if ] − En [−if ] k2 2i kEn [if ]k2 + kEn [−if ]k2 ≤ 2 ≤ .

kEn [f ]k2 = k

 Corollary 4.6. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. If k(sin cn [f ])−1 k2 is uniformly bounded with respect to n, then for all  > 0 there exist positive integers N and M such that for all n > N b n [f ] + Ebn [f ], (sin cn [f ])−1 (sin An [f ]) = In + R where b n [f ] ≤ 2M, rank R kEbn [f ]k2 ≤ .

14

Proof By Theorem 4.5, we know that for all  > 0, there exist positive integers N and M such that for all n > N sin cn [f ] − sin An [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, kEn [f ]k2 ≤ . By the assumption that k(sin cn [f ])−1 k2 is uniformly bounded with respect to n, we have (sin cn [f ])−1 sin An [f ] = In + (sin cn [f ])−1 (sin An [f ] − sin cn [f ]) = In + (sin cn [f ])−1 (−Rn [f ]) + (sin cn [f ])−1 (−En [f ]) . {z } | {z } | b n [f ] R

Ebn [f ]

The result follows.



Remark From k(sin cn [f ])−1 k2 = max | i

1 | sin λi

we know that k(sin cn [f ])−1 k2 could be arbitrarily large since sin λi could be close to zero, where λi is the i-th eigenvalue of cn [f ]. Therefore, we have needed to assume that k(sin cn [f ])−1 k2 is uniformly bounded with respect to n. Consider now the special case when sin An [f ] is Hermitian. Unlike the case with the matrix exponential, we cannot use CG for sin An [f ] as it is not positive definite in general. By the diagonalisation of sin An [f ] = ZnT (sin Dn )Zn where Dn is a diagonal matrix with real eigenvalues di being the eigenvalues of An [f ], as before, we see that its eigenvalues are all of the form −1 ≤ sin di ≤ 1. Krylov subspace methods like MINRES [23] together with a Hermitian positive definite preconditioner | sin cn [f ]| should be used [32, Section 5], where | sin cn [f ]| is the absolute value circulant preconditioner [24, 19] of sin cn [f ]. Corollary 4.7. Let f ∈ C2π be a real-valued function. Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. If k(sin cn [f ])−1 k2 is uniformly bounded with respect 15

to n, then for all  > 0 there exist positive integers N and M such that for all n > N e n [f ] + Een [f ], | sin cn [f ]|−1 sin An [f ] = Qn + R where Qn is unitary and Hermitian, e n [f ] ≤ 2M, rank R kEen [f ]k2 ≤ . Proof Using a similar approach proposed in [24], we want to show that | sin cn [f ]| is an effective Hermitian positive definite preconditioner for sin An [f ] under the assumptions. As sin cn [f ] is a circulant matrix we write sin cn [f ] = Fn∗ (sin Λn )Fn where sin Λn is the diagonal matrix with the eigenvalues of sin cn [f ]. We then immediately have | sin cn [f ]| = Fn∗ | sin Λn |Fn = Fn∗ (sin Λn )Fn Fn∗ (sign(sin Λn ))−1 Fn {z } | Qn

= sin cn [f ]Qn ,

(5)

λi where sign(sin Λn ) = diag( | sin ) = diag(±1) and Qn is both unitary and sin λi | 2 involutory (i.e. Qn = In ). It is noted that | sin λi | 6= 0 for i = 1, 2, . . . , n by the assumption, so that sign(sin Λn ) is well defined. By Corollary 4.6, we know that for all  > 0 there exist positive integers N and M such that for all n > N

b n [f ] + Ebn [f ], (sin cn [f ])−1 (sin An [f ]) = In + R where b n [f ] ≤ 2M, rank R kEbn [f ]k2 ≤ . Using (5), we have b [f ] + Q Eb [f ] . | sin cn [f ]|−1 sin An [f ] = Qn sin cn [f ]−1 sin An [f ] = Qn + Qn R | {zn } | n {zn } e n [f ] R

16

Een [f ]

Since Qn is unitary, we know e n [f ] = rank(Qn R b n [f ]) = rank R b n [f ] ≤ 2M rank R and kEen [f ]k2 = kQn Ebn [f ]k2 = kEbn [f ]k2 ≤ . The result follows.



Remark Since | sin cn [f ]| is also a circulant matrix, | sin cn [f ]|−1 d for any vector d can be efficiently computed by several FFTs in O(n log n) operations. For the more general case when sin An [f ] is non-Hermitian, we can use CG for the normal equations system with the preconditioner sin cn [f ]. Corollary 4.8. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. If k(sin cn [f ])−1 k2 is uniformly bounded with respect to n, then for all  > 0 there exist positive integers N and M such that for all n > N [(sin cn [f ])−1 sin An [f ]]∗ [(sin cn [f ])−1 sin An [f ]] = In + Rn [f ] + E n [f ], where rank Rn [f ] ≤ 4M, kE n [f ]k2 ≤ . Proof By Corollary 4.6, we know that for all  > 0 there exist positive integers N and M such that for all n > N cn [f ] + Ebn [f ], (sin cn [f ])−1 sin An [f ] = In + R where b n [f ] ≤ 2M, rank R kEbn [f ]k2 ≤ .

17

We then have [(sin cn [f ])−1 sin An [f ]]∗ [(sin cn [f ])−1 sin An [f ]] b n [f ] + Ebn [f ])∗ (In + R b n [f ] + Ebn [f ]) = (In + R b [f ]∗ (In + R b n [f ] + Ebn [f ]) + (In + Ebn [f ]∗ )R b n [f ] = In + R |n {z } Rn [f ]

+ Ebn [f ] + Ebn [f ]∗ + Ebn [f ]∗ Ebn [f ] . | {z } E n [f ]

It immediately follows that rank Rn [f ] ≤ 4M and kE n [f ]k2 ≤ 2 + 2.  Since [(sin cn [f ])−1 sin An [f ]]∗ [(sin cn [f ])−1 sin An [f ]] in Corollary 4.8 is Hermitian, by Weyl’s theorem, we know that its eigenvalues are mostly close to 1 when n is sufficiently large. iAn −iAn for any An ∈ Cn×n , we have the following Because cos An = e +e 2 similar theorem and corollaries for cos An [f ]. Theorem 4.9. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. For all  > 0, there exist positive integers N and M such that for all n > N cos cn [f ] − cos An [f ] = Rn [f ] + En [f ], where rank Rn [f ] ≤ 2M, kEn [f ]k2 ≤ . Corollary 4.10. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. If k(cos cn [f ])−1 k2 is uniformly bounded with respect to n, then for all  > 0 there exist positive integers N and M such that for all n > N | cos cn [f ]|−1 cos An [f ] = Qn + Ren [f ] + Een [f ], where Qn is unitary and Hermitian, rank Ren [f ] ≤ 2M, kEen [f ]k2 ≤ . 18

For the more general case when cos An [f ] is non-Hermitian, we can use CG for the normal equations system with the preconditioner cos cn [f ]. Corollary 4.11. Let f ∈ C2π . Let An [f ] ∈ Cn×n be the Toeplitz matrix generated by f and cn [f ] ∈ Cn×n be the optimal circulant preconditioner for An [f ]. If k(cos cn [f ])−1 k2 is uniformly bounded with respect to n, then for all  > 0 there exist positive integers N and M such that for all n > N [(cos cn [f ])−1 cos An [f ]]∗ [(cos cn [f ])−1 cos An [f ]] = In + R n [f ] + E n [f ], where rank R n [f ] ≤ 4M, kE n [f ]k2 ≤ . Since [(cos cn [f ])−1 cos An [f ]]∗ [(cos cn [f ])−1 cos An [f ]] in Corollary 4.11 is Hermitian, by Weyl’s theorem, we know that its eigenvalues are mostly close to 1 when n is sufficiently large. 5. Extension to analytic functions of Toeplitz matrices Lemma 5.1. [13, Theorem 1.18] Let h be analytic on an open subset Ω ⊆ C such that each connected component of Ω is closed under conjugation. Consider the corresponding matrix function h on its natural domain in Cn×n , the set D = {An ∈ Cn×n : Λ(An ) ⊆ Ω}. Then the following are equivalent: (a) h(A∗n ) = h(An )∗ for all An ∈ D. (b) h(An ) = h(An ) for all An ∈ D. (c) h(Rn×n ∩ D) ⊆ Rn×n . (d) h(R ∩ Ω) ⊆ R. Lemma 5.2. [13, Theorem 4.7] Suppose h has a Taylor series expansion h(z) =

∞ X

ak (z − α)k ,

k=0 (k)

where ak = h k!(α) , with radius of convergence r. If An ∈ Cn×n then f (An ) is defined and is given by h(An ) =

∞ X

ak (An − αIn )k

k=0

19

if and only if the distinct eigenvalues λ1 , · · · , λs of An satisfy one of the conditions (a) |λi − α| < r, (b) |λi − α| = r and the series for h(ni −1) (λ), where ni is the index of λi , is convergent at the point λ = λi , i = 1, . . . , s. Lemma 5.3. [13, Theorem 4.8] Suppose h has a Taylor series expansion h(z) =

∞ X

ak (z − α)k ,

k=0 (k)

where ak = h k!(α) , with radius of convergence r. If An ∈ Cn×n with ρ(An − αIn ) < r then for any matrix norm k · k K−1 X

kh(An )−

ak (An −αIn )k k ≤

k=0

1 max k(An − αIn )K h(K) (αIn + t(An − αIn ))k. K! 0≤t≤1

We first assume that the condition in Lemma 5.2 is satisfied so h(An ) can be represented by a Taylor series of An . Replacing Lemma 3.3 with Lemma 5.3, we can show that h(cn [f ]) − h(An [f ]) can be decomposed into a sum of a matrix of certain rank and a small norm matrix in a similar manner to theorem 4.2. Further assuming the boundedness of h(cn [f ]), we can prove a similar decomposition for h(cn [f ])−1 h(An [f ]) or its normal equations matrix. 6. Numerical results In this section, we demonstrate the effectiveness of our proposed preconditioners g(cn [f ]) for the systems g(An [f ])x = b using CG, MINRES and GMRES [27]. Throughout all numerical tests, eAn [f ] is computed by the MATLAB built-in function expm whilst sin An [f ] and cos An [f ] are computed by funm. Also, we use the function pcg to solve the Hermitian positive definite systems g(An [f ])x = b, and g(An [f ])∗ g(An [f ])x = g(An [f ])∗ b, where b is generated by the function randn(n,1), with the zero vector as the initial guess. For Hermitian indefinite systems, we use the function minres. 20

As a comparison, GMRES is also used for some systems and it is executed by gmres. The stopping criterion used is krj k2 < 10−7 , kbk2 where rj is the residual vector after j iterations. Example 1: We first consider eAn [f ] , where An [f ] is generated by several functions f with moderate kf k∞ . Table 1 shows the numbers of iterations needed for eAn [f ] with An [f ] generated by f (θ) = 43 θ cos(θ) with or without preconditioners. It is clear that the proposed precondtioner is efficient for speeding up the rate of convergence of CG. In Figure 1 (a) and (b), the contrast between the spectra of the matrices is shown. In Figure 1 (c), we see that the eigenvalues of the preconditioned matrix are highly clustered near 1. By the analysis on the rate of convergence of preconditioned CG for highly clustered spectrum given in [1], the preconditioned matrix can be regarded as having an ”efficient” condition number b/a, where [a, b] is the closed interval in which most of the eigenvalues are clustered. Therefore, a fast convergence rate for preconditioned CG is expected due to the cluster of eigenvalues at 1 and the small number of outliers. Table 1: Numbers of iterations with CG for eAn [f ] with the generating f (θ) = 34 θ cos(θ).

n 128 256 512 1024

In 224 325 414 491

ecn [f ] 20 21 25 26

In Figure 2, we further show the spectrum of eAn [f ] before and after applying the preconditioner ecn [f ] with different n. We observe that the highly clustered spectra seem independent of n.

21

0

10

20

30

40

50

60

70

40

50

60

70

(a) 0

10

20

30

(b) -0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

(c) −1

Figure 1: The spectrum of (a) eAn [f ] and that of (b) (ecn [f ] ) eAn [f ] . (c) The zoom-in spectrum of (b). An [f ] is generated by f (θ) = 43 θ cos(θ) and n = 512.

(a)

0

-0.2

(b)

0

0

-0.2

(c)

10

0

-0.2

20

0.2

10

30

0.4

20

0.6

30

0

0.2

0.4

10

20

30

0

0.2

0.4

40

0.8

40

0.6

40

0.6

50

1

50

1.2

60

0.8

60

0.8

−1

1.2

70

1

1.4

70

1

50

60

80

1.4

80

1.2

90

1.4

Figure 2: The spectrum of eAn [f ] and that of (ecn [f ] ) eAn [f ] (a) n = 256, (b) n = 1024 or (c) n = 4096. An [f ] is generated by f (θ) = 34 θ cos(θ).

22

Example 2: Table 2 (a) and (b) show the numerical results using CG and GMRES for the normal equations matrices of eAn [f ] with An [f ] generated by f (θ) = 2θ cos(θ) + θi, respectively. Again, we observe that the preconditioners are efficient for speeding up the rate of convergence. Table 2: Numbers of iterations with (a) CG for (eAn [f ] )∗ eAn [f ] and (b) GMRES for eAn [f ] with An [f ] generated by f (θ) = 2θ cos(θ) + θi.

n 128 (a) 256 512 1024

In 14219 78645 >100000 >100000

n 128 (b) 256 512 1024

In 128 248 477 891

Preconditioner 75 96 145 110

Preconditioner 21 23 25 27

23

Example 3: We next consider the matrix sine functions. Table 3 (a) shows the numerical results using MINRES for sin An [f ] with An [f ] generated by θ2 + 1013 ). As the matrix in this case is symmetric negative definite, f (θ) = −( 2π we also show numerical results using CG as a comparison in Table 3 (b). Table 3: Numbers of iterations with (a) MINRES and (b) CG for sin An [f ] with An [f ] θ2 generated by f (θ) = −( 2π + 1013 ).

n 128 (a) 256 512 1024 n 128 (b) 256 512 1024

In 138 227 238 243 In 138 235 253 255

24

| sin cn [f ]| 15 14 11 10 sin cn [f ] 16 14 11 10

Example 4: Table 4 shows the numerical results for (sin An [f ])∗ sin An [f ] θ2 i + 1013 ). Since the normalised matrices with An [f ] generated by f (θ) = −( 2π are highly ill-conditioned, CG without preconditioner requires large numbers of iteration to get the solutions to the desired tolerance. However, the numbers of iterations are reduced significantly with our proposed preconditioner. Table 4: Numbers of iterations with (a) CG for (sin An [f ])∗ sin An [f ] and (b) GMRES for θ2 sin An [f ] with An [f ] generated by f (θ) = −( 2π i + 1013 ).

n 128 (a) 256 512 1024 n 128 (b) 256 512 1024

In 1094 3238 4844 10152 In 128 255 384 463

Preconditioner 31 27 22 16 Preconditioner 16 17 13 10

25

Example 5: Lastly, we consider the matrix cosine functions. Table 5 and 6 show the numerical results for symmetric matrix cos An [f ] with An [f ] generated by f (θ) = ( π2 − 1014 ) cos (θ2 ) − π4 and for (cos An [f ])∗ cos An [f ] with An [f ] generated by f (θ) = ( π2 − 1014 ) cos (θ2 ) + πθ i, respectively. In Figure 3 (a) and (b), we also show the spectrum of the matrices before and after applying the preconditioner | cos An [f ]|. In the zoom-in spectrum shown in Figure 3 (c), we observe that the eigenvalues of the preconditioned matrix are mostly ±1. We conclude that our proposed preconditioners appear effective for these systems defined by matrix cosine functions of Toeplitz matrices. Table 5: Numbers of iterations with MINRES for cos An [f ] with An [f ] generated by f (θ) = ( π2 − 1014 ) cos (θ2 ) − π4 .

n 128 256 512 1024

| cos cn [f ]| 30 36 38 42

In 77 139 261 506

Table 6: Numbers of iterations with (a) CG for (cos An [f ])∗ cos An [f ] and (b) GMRES for cos An [f ] with An [f ] generated by f (θ) = ( π2 − 1014 ) cos (θ2 ) + πθ i.

n 128 (a) 256 512 1024

In 191 435 973 2092

Preconditioner 21 22 22 23

n 128 (b) 256 512 1024

In 126 249 492 972

Preconditioner 16 16 17 17

26

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

(a) -6

-4

-2

0

2

4

6

8

(b) -1.5

-1

-0.5

0

0.5

1

1.5

(c) −1

Figure 3: The spectrum of (a) cos An [f ] and that of (b) | cos cn [f ]| cos An [f ]. (c) The zoom-in spectrum of (b). An [f ] is generated by f (θ) = ( π2 − 1014 ) cos (θ2 ) − π4 and n = 512.

References [1] O. Axelsson and G. Lindskog. On the rate of convergence of the preconditioned conjugate gradient method. Numerische Mathematik, 48:499– 524, 1986. [2] D. Bini, S. Dendievel, G. Latouche, and B. Meini. Computing the exponential of large block-triangular block-Toeplitz matrices encountered in fluid queues. Linear Algebra and its Applications, 502(Supplement C):387 – 419, 2016. Structured Matrices: Theory and Applications. [3] E. Brigham. FFT: schnelle Fourier-Transformation. Einf¨ uhrung in die Nachrichtentechnik. [Introduction to Information Technology]. R. Oldenbourg Verlag, Munich, 1982. Translated from the English by Seyed Ali Azizi. [4] R. Chan and X. Jin. An introduction to iterative Toeplitz solvers, volume 5 of Fundamentals of Algorithms. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2007. [5] R. Chan and M. Ng. Conjugate gradient methods for Toeplitz systems. SIAM Rev., 38(3):427–482, 1996. [6] R. Chan, D. Potts, and G. Steidl. Preconditioners for nondefinite Hermitian Toeplitz systems. SIAM Journal on Matrix Analysis and Applications, 22(3):647–665, 2001.

27

[7] R. Chan and G. Strang. Toeplitz equations by conjugate gradients with circulant preconditioner. SIAM Journal on Scientific and Statistical Computing, 10(1):104–119, 1989. [8] R. Chan and M. Yeung. Circulant preconditioners for complex Toeplitz matrices. SIAM Journal on Numerical Analysis, 30(4):1193–1207, 1993. [9] T. Chan. An optimal circulant preconditioner for Toeplitz systems. SIAM Journal on Scientific and Statistical Computing, 9(4):766–771, 1988. [10] P. Davis. Circulant matrices. John Wiley & Sons, New York-ChichesterBrisbane, 1979. A Wiley-Interscience Publication, Pure and Applied Mathematics. [11] F. Di Benedetto. Analysis of preconditioning techniques for illconditioned Toeplitz matrices. SIAM Journal on Scientific Computing, 16(3):682–697, 1995. [12] D. Duffy. Finite Difference Methods in Financial Engineering: A Partial Differential Equation Approach. The Wiley Finance Series. Wiley, 2013. [13] N. Higham. Functions of matrices. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008. Theory and computation. [14] T. Huckle. Circulant and skewcirculant matrices for solving Toeplitz matrix problems. SIAM Journal on Matrix Analysis and Applications, 13(3):767–777, 1992. [15] T. Huckle, S. Serra Capizzano, and C. Tablino-Possio. Preconditioning strategies for non-Hermitian Toeplitz linear systems. Numer. Linear Algebra Appl., 12(2-3):211–220, 2005. [16] X. Jin, Z. Zhao, and S. Tam. Optimal preconditioners for functions of matrices. Linear Algebra and its Applications, 457:224 – 243, 2014. [17] D. Kressner and R. Luce. Fast computation of the matrix exponential for a Toeplitz matrix. ArXiv e-prints, July 2016. [18] S. Lee, H. Pang, and H. Sun. Shift-invert Arnoldi approximation to the Toeplitz matrix exponential. SIAM Journal on Scientific Computing, 32(2):774–792, 2010. 28

[19] E. McDonald, S. Hon, J. Pestana, and A. Wathen. Preconditioning for Nonsymmetry and Time-Dependence, pages 81–91. Springer International Publishing, Cham, 2017. [20] M. Ng. Iterative methods for Toeplitz systems. Numerical Mathematics and Scientific Computation. Oxford University Press, New York, 2004. [21] M. Ng and J. Pan. Approximate inverse circulant-plus-diagonal preconditioners for Toeplitz-plus-diagonal matrices. SIAM Journal on Scientific Computing, 32(3):1442–1464, 2010. [22] J. Olkin. Linear and Nonlinear Deconvolution Problems (Optimization). PhD thesis, Rice University, Houston, TX, 1986. [23] C. Paige and M. Saunders. Solution of sparse indefinite systems of linear equations. SIAM journal on numerical analysis, 12(4):617–629, 1975. [24] J. Pestana and A. Wathen. A preconditioned MINRES method for nonsymmetric Toeplitz matrices. SIAM Journal on Matrix Analysis and Applications, 36(1):273–288, 2015. [25] D. Potts and G. Steidl. Optimal trigonometric preconditioners for nonsymmetric Toeplitz systems. Linear Algebra and its Applications, 281(1):265 – 292, 1998. [26] M. Powell. Approximation theory and methods. Cambridge University Press, Cambridge-New York, 1981. [27] Y. Saad and M. Schultz. GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 7(3):856–869, 1986. [28] E. Sachs and A. Strauss. Efficient solution of a partial integro-differential equation in finance. Applied Numerical Mathematics, 58(11):1687 – 1703, 2008. [29] S. Serra Capizzano. Superlinear PCG methods for symmetric Toeplitz systems. Math. Comput., 68:793–803, 1999. [30] G. Strang. A proposal for Toeplitz matrix calculations. Stud. Appl. Math., 74(2):171–176, 1986. 29

[31] E. Tyrtyshnikov. Optimal and superoptimal circulant preconditioners. SIAM Journal on Matrix Analysis and Applications, 13(2):459–473, 1992. [32] A. Wathen. Preconditioning. Acta Numerica, 24:329–376, 5 2015.

30