Exponential Integrators for Semilinear Problems Borislav V. Minchev
Thesis submitted for the degree of Philosophiae Doctor (PhD) at
Department of Informatics University of Bergen August 2004
EXPONENTIAL INTEGRATORS FOR SEMILINEAR PROBLEMS
By Borislav V. Minchev
SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF PHILOSOPHIAE DOCTOR AT UNIVERSITY OF BERGEN THORMHLENSGATE 55,
5020 BERGEN
AUGUST 2004
c Copyright by Borislav V. Minchev, 2004 °
UNIVERSITY OF BERGEN
Date: August 2004 Author:
Borislav V. Minchev
Title:
Exponential Integrators for Semilinear Problems
Department:
Informatics
Degree: Ph.D.
Convocation: October
Year: 2004
Permission is herewith granted to University of Bergen to circulate and to have copied for non-commercial purposes, at its discretion, the above title upon the request of individuals or institutions.
Signature of Author
THE AUTHOR RESERVES OTHER PUBLICATION RIGHTS, AND NEITHER THE THESIS NOR EXTENSIVE EXTRACTS FROM IT MAY BE PRINTED OR OTHERWISE REPRODUCED WITHOUT THE AUTHOR’S WRITTEN PERMISSION. THE AUTHOR ATTESTS THAT PERMISSION HAS BEEN OBTAINED FOR THE USE OF ANY COPYRIGHTED MATERIAL APPEARING IN THIS THESIS (OTHER THAN BRIEF EXCERPTS REQUIRING ONLY PROPER ACKNOWLEDGEMENT IN SCHOLARLY WRITING) AND THAT ALL SUCH USE IS CLEARLY ACKNOWLEDGED.
ii
To My Family.
iii
iv
Table of Contents Table of Contents
v
List of Tables
vii
List of Figures
viii
Abstract
ix
Acknowledgements
x
Introduction and motivation
1
1 Exponential Integrators 1.1 Exponential linear multistep methods . . . . . . . . 1.1.1 Integrating Factor methods . . . . . . . . . . 1.1.2 Exponential Time Differencing methods . . . 1.2 Exponential Runge–Kutta methods . . . . . . . . . . 1.2.1 Formulation of the methods . . . . . . . . . . 1.2.2 Nonstiff order conditions . . . . . . . . . . . . 1.2.3 IF RK and ETD RK as special cases . . . . . 1.2.4 More examples of ETD RK methods . . . . . 1.2.5 Exponential RK methods for parabolic PDEs 1.3 General linear methods and exponential integrators . 1.3.1 Formulation of the methods . . . . . . . . . . 1.3.2 Construction of practical GLMs . . . . . . . . 1.3.3 Exponential general linear methods . . . . . .
. . . . . . . . . . . . .
3 3 4 5 8 9 11 21 26 28 36 36 40 42
2 Exponential Integrators and Lie Group Methods 2.1 Background theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Lie group integrators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44 44 48
v
. . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
2.3 2.4
The choice of action . . . . . . . . . . . . . . . . . . Lie group integrators for semilinear problems . . . . 2.4.1 Crouch–Grossman based methods . . . . . . 2.4.2 RKMK based methods . . . . . . . . . . . . . 2.4.3 Methods based on commutator free Lie group
. . . . . . . . . . . . . . . . . . . . . . . . methods .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
52 55 56 56 64
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
67 67 68 70 73 75 78
Conclusions Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83 83 83 84
Bibliography
86
Article 1: Lie group integrators with nonautonomous frozen vector fields. Submitted to J. Appl. Num. Math. . . . . . . . . . . . . . . . . . . . . . . . . . .
91 91
3 Implementation Issues and Numerical Experiments 3.1 Decomposition methods . . . . . . . . 3.1.1 Block Schur–Parlett algorithm 3.1.2 Tridiagonal reduction . . . . . 3.2 Krylov subspace approximations . . . 3.3 Cauchy integral approach . . . . . . . 3.4 Numerical experiments . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
Article 2: Some algorithms for solving special tridiagonal block Teoplitz linear systems. 107 J. Comp. and Appl. Math., vol.156(1) (2003), 179-200. . . . . . . . . . . . . . . 107 Article 3: A method for solving Hermitian pentadiagonal block circulant cystems of linear equations. 133 With I. Ivanov. Springer Lect. Notes Comput. Sci., 2907 (2004), 481-488. . . . . 133
vi
List of Tables 1.1
Coefficients of ETD Adams–Bashforth methods. . . . . . . . . . . . . . . .
8
1.2
Coefficients of ETD Adams–Moulton methods. . . . . . . . . . . . . . . . .
8
1.3
General formulation of an IF Runge–Kutta method. . . . . . . . . . . . . .
9
1.4
The number of rooted trees in 2T∗ for all orders up to ten. . . . . . . . . .
12
1.5
Standard decomposition for all trees up to order 3 and their corresponding differential equations. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
1.6
Relation between elementary differentials and elementary weights. . . . . .
21
1.7
Coloured trees as linear combinations of black trees for the IF RK methods.
25
1.8
A fourth order ETD Runge–Kutta method. . . . . . . . . . . . . . . . . . .
26
1.9
Third order method of Cox–Matthews. . . . . . . . . . . . . . . . . . . . . .
27
1.10 Fourth order method of Cox–Matthews. . . . . . . . . . . . . . . . . . . . .
27
1.11 Fourth order method of Krogstad. . . . . . . . . . . . . . . . . . . . . . . .
27
1.12 Third order ETD2RK3 method. . . . . . . . . . . . . . . . . . . . . . . . . .
28
1.13 Third order ETD2CF3 method. . . . . . . . . . . . . . . . . . . . . . . . . .
28
1.14 Stiff order conditions for explicit exponential Runge–Kutta methods for α = 0. 34 1.15 Fourth order method of Hochbruck–Ostermann with five stages. . . . . . . .
35
2.1
Actions and homeomorphisms on the manifold. . . . . . . . . . . . . . . . .
49
2.2
Third order Crouch–Grossman method. . . . . . . . . . . . . . . . . . . . .
56
2.3
Fourth order RKMK method with exact Exp. . . . . . . . . . . . . . . . . .
58
2.4
Fourth order GIF1/RK method . . . . . . . . . . . . . . . . . . . . . . . . .
59
2.5
Fourth order GIF2/RK method . . . . . . . . . . . . . . . . . . . . . . . . .
61
2.6
Fourth order CF method. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65
vii
List of Figures 1.1
Domain of validity for condition (1.2.18) (shaded). . . . . . . . . . . . . . .
30
1.2
The contour Γ from Definition 1.2.7. . . . . . . . . . . . . . . . . . . . . . .
31
3.1
Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Kuramoto-Sivashinsky equation . . . . . . . . . . . . . . . . . .
3.2
Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Allen-Cahn equation . . . . . . . . . . . . . . . . . . . . . . . .
3.3
80 81
Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Korteweg de Vries equation . . . . . . . . . . . . . . . . . . . . .
viii
81
Abstract In the present work, exponential integrators for time integration of semilinear problems are studied. These integrators, as there name suggests, use the exponential and often functions which are closely related to the exponential function inside the numerical method. Three main classes of exponential integrators, exponential linear multistep (multivalue), exponential Runge–Kutta (multistage) and exponential general linear methods, are discussed. A general formulation of exponential integrators, which includes, as special cases, all known methods, is proposed. The nonstiff order theory for exponential multistage methods, along with a non-recursive rule for generating each order condition from its corresponding rooted tree, is derived. The natural connection between exponential integrators and Lie group methods with affine algebra action is also studied. A new approach for deriving Generalized Integrating Factor Runge–Kutta methods, which allows the nonlinear part of the problem to be approximated by trigonometric polynomials, is proposed. The crucial role of the algebra action in the overall performance of any Lie group method is discussed. A new algebra action arising from the solutions of differential equations with nonautonomous frozen vector fields is proposed. The corresponding exponential integrators based on this action are derived. Different methods for numerically stable computation of the most commonly used functions which appear in the format of an exponential integrator are considered. A generalization of the method based on the tridiagonal reduction is proposed. The new approach allows to compute all functions included in the format of an exponential integrator in the case when the arguments are symmetric (Hermitian) matrices. Some practical issues regarding variable step size implementations as well as the main advantages and disadvantages of the considered numerical techniques are discussed. New effective methods and their modifications for solving special three and five diagonal block systems of linear equations, based on a modified LU factorization are proposed. For illustrating the theoretical results, several numerical experiments are presented.
ix
Acknowledgements I would like to thank Prof. Hans Munthe-Kaas, my supervisor, for his constant support during this research. I am also thankful to Ass. Prof. Ivan Ivanov for his guidance through the early years of my work. Dr. William Wright has expressed his interest to my work at a crucial stage. The time spend together discussing many of the ideas presented in this thesis, have provided me with better understanding and with the precious second point of view. I am also grateful for his valuable comments and suggestions made while reading the manuscript of this thesis. During my participation in the Special Year in Geometric Integration, Oslo, 2002- 2003, I had the opportunity to meet many outstanding people from the international GI community. I am also grateful to all the members of the Bergen–Cambridge–Trondheim research groups. They all have played an important role in my professional live. I would like to thank the Norwegian Research Council, Meltzer Høyskolefond and the Department of Informatics, University of Bergen for their financial support during the last three years. Without it my research stays at Center for Advanced Studies, Norwegian Academy of Sciences and Section de math´ematiques, Universit´e de Gen`eve will not be possible. Finally, but actually on the first place, I would like to thank to my family, my parents, and specially to my wife Stela, for their love and for always believing in me. Bergen, Norway
Borislav V. Minchev
August, 2004
x
Introduction and motivation
Many practical problems arising in the real life applications can be modeled by time dependent partial differential equations (PDEs). In most cases it is extremely difficult, if not impossible, to obtain exact solutions of these problems, therefore numerical methods are used to provide accurate approximations. An often used technique in the construction of numerical integrators for PDEs, is first to semidiscretize the equation in space. There exists many efficient and accurate methods for spatial discretization. If the original vector field can be represented like a sum of two part, linear and nonlinear, the spatial discretization leads to the following semilinear initial value problem u0 = Lu + N (u, t),
u(t0 ) = u0 ,
where u : R → Rd , L ∈ Rd×d , N : Rd → Rd and d is a discretization parameter equal to the number of spatial grid points. Many interesting equations can be brought exactly in this form. Examples are Allen-Cahn, Burgers, Cahn-Hilliard, Kuramoto-Sivashinsky, NavierStokes, Swift-Hohenberg, nonlinear Scr¨ odinger, convection-diffusion equations. Typically, after the spatial discretization, the linear part of the problem will be stiff and the nonlinear part will be nonstiff. Whenever equations model several processes with vastly different rates of evolution, that is the processes proceed at significantly different time scales, one should also expect stiffness. Implicit methods, which overcome the problem of stiffness are regarded as to expensive for implementation. On other hand explicit methods require such small time steps that they become inefficient. Therefore, it is a very important and challenging task to construct effective numerical integrators for solving stiff problems. The first exponential integrators were introduced in 1960 as an alternative approach to overcome the phenomenon of stiffness. The main idea behind these methods is to integrate exactly the linear part of the problem, which is primarily responsible for the stiffness, and then to use an appropriate approximation of the nonlinear part. Thus the exponential function, and often functions which are closely related to the exponential function, appear in the format of the methods. This was exactly the reason why, until recently, these methods have not been regarded as practical. The latest achievements in the field of computing approximations to the product of a matrix exponential function and some related functions 1
with a vector, have provided a new interest in the construction and implementation of exponential integrators. The aim of this thesis is to study different classes of exponential integrators for time integration of semilinear problems and to propose an unified framework in which this methods can be analyzed. The thesis is organized as follows: In Chapter 1 we present the philosophy behind exponential integrators applied to semilinear problems and discuss the three main classes of exponential integrators: exponential linear multistep (multivalue), exponential Runge– Kutta (multistage) and exponential general linear methods. Next, in Chapter 2 we study the connection between exponential integrators and Lie group methods based on the affine algebra action. Finally, in Chapter 3 we discuss different techniques for a fast and numerically stable computation of the most commonly used functions which enter in the format of the methods, and present some numerical experiments.
2
Chapter 1
Exponential Integrators In the first chapter of this thesis we present the philosophy behind exponential integrators applied to semilinear problems and discuss how the three main classes of numerical integrators, linear multistep (multivalue), Runge–Kutta (multistage) and general linear methods, are extended to the exponential setting. In the presentation we assume the reader has basic knowledge of numerical methods for ordinary differential equations and functional analysis as can be found in the monographs [8, 24, 25, 70]. All numerical schemes presented in this chapter are in accordance with the notations introduced in Section 1.3. The chapter is organized as follows: First, in Section 1.1 we consider exponential linear multistep methods based on Integrating Factor (IF) and Exponential Time Differencing (ETD) approach. Next, in Section 1.2 we present the theory of exponential Runge–Kutta methods for both nonstiff and stiff cases. Finally, in Section 1.3 we explain how the idea of general linear methods can also be extended to the exponential setting.
1.1
Exponential linear multistep methods
Our field of interest is the following semi-discretized semilinear initial value problem u0 = Lu + N (u, t),
u(t0 ) = u0 ,
(1.1.1)
where u : R → Rd , L ∈ Rd×d , N : Rd → Rd and d is a discretization parameter equal to the number of spatial grid points. Several interesting problems can be brought to this form by discretizing in space. Examples are Allen-Cahn, Burgers, Cahn-Hilliard, KuramotoSivashinsky, Navier-Stokes, Swift-Hohenberg, nonlinear Scr¨odinger equations. Typically the linear part of the problem will be stiff and the nonlinear part will be nonstiff. There exist two principal approaches in the derivation of exponential linear multistep methods for solving (1.1.1). These will be introduced in the next two subsections. 3
1.1.1
Integrating Factor methods
The idea behind Integrating Factor (IF) methods is an old one and goes back to the work of Lawson [37]. He proposes to ameliorate the effect of the stiff linear part of equation (1.1.1) by using change of variables (also known as Lawson transformation), v(t) = e−tL u(t). The initial value problem (1.1.1) written in the new variable is v 0 (t) = e−tL N (etL v(t), t) = g(v, t),
v(t0 ) = v0 ,
(1.1.2)
where v0 = e−t0 L u0 . The same result can be obtained by premultiplying the original problem (1.1.1) by the so called integrating factor, e−tL , that is e−tL u0 = e−tL Lu + e−tL N (u, t), (e−tL u)0 = e−tL N (u, t).
(1.1.3)
To see way the formulation (1.1.2) is preferable to the original formulation, we calculate the Jacobian of the new problem. Since ∂N tL ∂g = e−tL e , ∂v ∂u and e−tL = (etL )−1 , it follows that the eigenvalues of ∂g/∂v are those of ∂N/∂u. Therefore we should expect that numerical methods applied to (1.1.2) will not suffer from the same stability restrictions like if they are used directly on the problem (1.1.1). The idea now is to apply any numerical integrator (in the case of Lawson, a Runge–Kutta method) on the transformed equation and then to transform back the result into the original variable. Using Euler method, we obtain the following numerical scheme un = ehL un−1 + ehL hNn−1 , where h represents the stepsize of the method and Nn−1 = N (un−1 , tn−1 ). This is known as IF Euler method. The integrating factor implicit Euler method is un = ehL un−1 + ehL hNn . This approach can be easily extended to the class of linear multistep Adams methods. In general, k-step IF Adams methods are defined as un = ehL un−1 +
k X
βi eihL hNn−i ,
i=0
where βi are the coefficients of the Adams method and Nn−i = N (un−i , tn−i ) for i = 0, 1, 2, . . . , k. Similarly IF methods based on backward differentiation formulae (BDF) methods are defined as k X un = αi eihL un−i + β0 hNn , i=1
4
where β0 and αi are the coefficients of the underlying BDF method. As an example, the second order IF Adams–Bashforth method considered in [14] is 3 1 un = ehL un−1 + ehL hNn−1 − e2hL hNn−2 . 2 2
1.1.2
Exponential Time Differencing methods
According to our knowledge, the first paper on Exponential Time Differencing (ETD) methods is the paper of Certaine [13], where an order two and order three methods, reducing to second and third order Adams–Moulton methods, are proposed. Arbitrarily high order Astable ETD methods, reducing to the Adams–Bashforth methods when L = 0, are derived in [53]. Since then, this methods has been rediscovered several times in different fields and under different names. The main idea behind the ETD method is to construct, via the variation of constants formulae, integrators of multistep type. To derive the variation of constants formulae we integrate the equation (1.1.3) between tn−1 and tn = tn−1 + h. We obtain u(tn−1 + h) = e
hL
un−1 + e
= ehL un−1 +
tn L
Z
h
Z
tn−1 +h
e−τ L N (u(τ ), τ )dτ
tn−1
e(h−τ )L N (u(tn−1 + τ ), tn−1 + τ )dτ.
(1.1.4)
0
Note that only exact calculations are used in the derivation of (1.1.4). Therefore (1.1.4) is the exact solution of (1.1.1) with initial condition u(tn−1 ) = un−1 . The approach now is to replace the nonlinear term N by a Newton interpolation polynomial and then solve the resulting integral exactly. The simplest case is when we approximate N (u(t n−1 + τ ), tn−1 + τ ) by a constant Nn−1 . This gives un = ehL un−1 + = e
hL
Z
h
e(h−τ )L Nn−1 dτ
0 [1]
un−1 + φ (hL)hNn−1 ,
where φ[1] (z) is φ[1] (z) =
ez − 1 . z
The above method is known as the exponential time differencing Euler method as it reduces to the classical Euler method when L = 0. The exponential time differencing implicit Euler method is defined as un = ehL un−1 + φ[1] (hL)hNn . In general, we obtain the following explicit form of ETD Adams-Bashforth methods by approximating the nonlinear term in (1.1.4) by higher order polynomial using information 5
from the past (see [14, 53]) un = ehL un−1 + h
k−1 X
gi (hL)∇i Nn−1 ,
(1.1.5)
i=0
where ∇i Nn−1 denotes the i-th backward difference, ∇0 Nn−1 = Nn−1 ,
∇i+1 Nn−1 = ∇i Nn−1 − ∇i Nn−2 ,
for i = 0, 1, 2, . . . ,
and the function gi (z) is given by the recurrence relations zg0 (z) = ez − 1,
zgi+1 (z) + 1 = gi (z) + 21 gi−1 (z) + 13 gi−2 (z) + · · · +
1 i+1 g0 (z).
Similarly, it is also possible to construct ETD Adams–Moulton methods. Next we give an other derivation of both explicit and implicit exponential Adams methods, as proposed in [4]. It is based on the following particular representation of the exact solution. Lemma 1.1.1. The exact solution of the initial value problem u0 = Lu + N (u, t),
u(tn−1 ) = un−1 ,
can be expressed in the form u(tn−1 + h) = ehL un−1 +
∞ X
(i)
hi+1 φ[i+1] (hL)Nn−1 ,
i=0
where (i) Nn−1
and φ[i] (z) are recursively defined as [0]
z
φ (z) = e ,
φ
[i+1]
¯ di ¯¯ = i¯ N (u(t), t), dt t=tn−1 φ[i] (z) − (z) = z
1 i!
,
for i = 0, 1, 2, . . . .
(1.1.6)
Proof. Expand u(tn−1 + h) in Taylor series around the point tn−1 to obtain u(tn−1 + h) = un−1 +
∞ X hk k=1
where (k) un−1
k!
(k)
un−1 ,
¯ dk ¯¯ u(t). = k¯ dt t=tn−1
Because of the given differential equation, we have the following relation (k) un−1
k
= L un−1 +
k−1 X
(i)
Lk−1−i Nn−1 ,
i=0
6
for k = 1, 2, . . . .
(1.1.7)
(k)
Substituting un−1 into (1.1.7), we obtain u(tn−1 + h) =
∞ X hk k=0
k!
k
L un−1 +
k−1 k ∞ X X h k=1 i=0
= ehL un−1 +
k!
(i)
Lk−1−i Nn−1
∞ X k X hk+1 k−i (i) L Nn−1 . (k + 1)! k=0 i=0
Changing the order of the summation in the second term we obtain ! Ã∞ ∞ X ∞ k X X X hk+1 hk+1 k−i (i) (i) L Nn−1 = Lk−i Nn−1 (k + 1)! (k + 1)! i=0 k=0 i=0 k=i ! à ∞ ∞ k X X (hL) (i) Nn−1 = hi+1 (hL)−(i+1) k! =
i=0 ∞ X
k=i+1
(i)
hi+1 φ[i+1] (hL)Nn−1 .
(1.1.8)
i=0
Lemma 1.1.1 gives an alternative formulation of the exact solution and motivates the following ansatz for its numerical approximation un = ehL un−1 + h
k X
βl Nn−l ,
l=0
where β0 , β1 , . . . , βk are coefficients to be computed in order to obtain the desired order. This is done by expanding the nonlinear terms in Taylor series around tn−1 . Since à k ! k ∞ X X hi+1 X (i) h βl Nn−l = βl (1 − l)i Nn−1 , (1.1.9) i! l=0
i=0
l=0
the order conditions for the ETD Adams methods are obtained by comparing (1.1.8) and (1.1.9), resulting in 1 i!
k X l=j
βl (1 − l)i = φ[i+1] (hL),
for i = 0, 1, 2, . . . , k − j.
For j = 0 we obtain the ETD Adams–Moulton methods, whereas for j = 1 we recover the ETD Adams–Bashforth methods (1.1.5). In Tables 1.1 and 1.2 we give the the first few coefficients for the ETD Adams–Bashforth and ETD Adams–Moulton methods respectively. Finally, we mention that it is not possible to construct ETD BDF methods (see [46]). Next, we explain how the IF and ETD approach can be further extended to the class of exponential multistage methods. 7
k
β1
β2
β3
β4
1
φ[1]
0
0
0
2
φ[1] + φ[2]
−φ[2]
0
0
1 [2] [3] 2φ + φ 3 [2] [3] [4] 2 φ + 4φ + 3φ
0
3 4
+ 23 φ[2] + φ[3] 11 [2] [3] 6 φ + 2φ +
−2(φ[2]
φ[1] φ[1]
+
φ[4]
−3φ[2]
−
+
φ[3] )
5φ[3]
−
3φ[4]
X
where X = − 31 φ[2] − φ[3] − φ[4] Table 1.1: Coefficients of ETD Adams–Bashforth methods. k
β0
0
φ[1]
1
φ[2]
2
1 [2] [3] 2φ + φ 1 [2] [3] [4] 3φ + φ + φ
3
β1 0 φ[1] φ[1] φ[1]
+
1 [2] 2φ
β2
β3
0
0
−
φ[2]
0
0
2φ[3]
−
2φ[3]
− 12 φ[2] + φ[3] −φ[2] + φ[3] + 3φ[4]
−
−
3φ[4]
0 1 [2] 6φ
− φ[4]
Table 1.2: Coefficients of ETD Adams–Moulton methods.
1.2
Exponential Runge–Kutta methods
As already mentioned in the previous section, the first exponential Runge–Kutta method was constructed by Lawson in [37]. In fact, once we have transformed the original problem (1.1.1) into the form (1.1.2), it is easy to derive an Integrating Factor Runge–Kutta (IF RK) method. In analogy to the multistep case, the idea now is to apply an arbitrary s-stage Runge–Kutta method to the transformed equation (1.1.2) and then to transform the result back into the original variable. If A = (αij ), b = (βi ) and c = (ci ) are the coefficients of the underlying multistage method then the general formulation of an s-stage IF Runge–Kutta method, written in accordance with the notations introduced in Section 1.3, is given in Table 1.3. This approach requires that the c vector has nondecreasing coefficients, which is usually the case for the most commonly used numerical schemes. Probably the first successful attempt to construct Exponential Time Differencing Runge– Kutta (ETD RK) methods is due to the work of Friedli [20]. Later Strehmel and Weiner [61] constructed the adaptive Runge–Kutta methods and studied their B-convergence [62]. This later class of methods is an extension of the W-method of Streihaug and Wolfbrandt [60], but is also very closely related to the methods of Friedli. The only difference between these methods and the ETD Runge–Kutta methods, presented in Section 1.2.3, is that the later methods compute the ETD φ[i] functions (1.1.6) exactly, while the W-methods and the adaptive Runge–Kutta methods generally use Pad´e approximations to these functions. 8
0 0 0 .. .
0 0 0 .. .
ec1 hL ec2 hL ec3 hL
αss−1 e(cs −cs−1 )hL
0
ecs hL
βs−1 e(1−cs−1 )hL
βs e(1−cs )hL
ehL
0 0 ··· c hL 2 α21 e 0 ··· α31 ec3 hL α32 e(c3 −c2 )hL · · · .. .. . .
α ecs hL s1 β1 ehL
αs2 e(cs −c2 )hL · · · β2 e(1−c2 )hL
···
Table 1.3: General formulation of an IF Runge–Kutta method. The fist modern exponential Runge–Kutta methods proposed in the literature are the methods of Hochbruck and Lubich [27]. In that paper, the authors constructed Rosenbrocklike exponential integrators which use the first ETD function φ[1] . Another class of methods known as Runge–Kutta Munthe-Kaas (RKMK) methods was first proposed in [48]. These transform the original differential equation to a new differential equation which evolves on a Lie algebra, and then take advantage of the fact that the Lie algebra is a linear space. In [50], in order to construct integrators for the differential equation (1.1.1), Munthe-Kaas proposed to combine the use of RKMK methods with the affine action (see Section 2.3). The affine action was also used in [12] to construct integrators based on the Commutator Free (CF) Lie group methods. As we shall see in Chapter 2, all of these methods can be regarded as “pure” exponential integrators which use other than the traditional IF and ETD φ[i] functions (see Section 1.2.3). Motivated by this observation, in the next section we give a slightly more general formulation of the exponential Runge–Kutta methods proposed by Friedli, which allows us to include as special cases all multistage methods mentioned above.
1.2.1
Formulation of the methods
For simplicity of the presentation, we choose to represent the initial value problem (1.1.1) in autonomous form u0 = Lu + N (u(t)),
u(t0 ) = u0 .
(1.2.1)
The aim is to construct a class of exponential integrators which overcome the stiffness in the differential equation (1.2.1) by using a set of precomputed functions along with evaluations of the nonlinear part of the differential equation. The precomputed functions, denoted by φ[l] , can however be a large overhead depending on the dimensionality of the problem, the spatial discretization of the differential equation and the structure of the matrix L. Thus, such methods are likely to be competitive, when the φ[l] functions can be evaluated efficiently. Let for l ∈ N and for all λ ∈ R the operators φ[l] (λ) : Rd×d → Rd×d be analytic, map the spectrum of L into a bounded region in C and can be computed exactly or up to arbitrary 9
high order cheaply, then φ[l] (λ)(hL) =
X
[l]
φj (λ)(hL)j .
(1.2.2)
j≥0
φ[l]
The most common choices of are those associated with the IF RK and ETD RK methods. The exact representation of φ[l] for these special cases can be found in Section 1.2.3. However, we mention that other choices are also possible (see Chapter 2). The exact structure of φ[l] , which leads to methods is still unclear. If h represents the stepsize and Ui denotes the internal stage approximation of the exact solution for i = 1, 2, . . . , s then the computations performed in step number n are related by the equations Ui =
s X m X
[l]
αij φ[l] (ci )(hL) hN (Uj ) + eci hL un−1 ,
j=1 l=1
un =
s X m X
(1.2.3) [l] βj
[l]
φ (1)(hL) hN (Uj ) + e
hL
un−1 ,
j=1 l=1
where m puts a limit on the number of φ[l] functions which can be computed. For the [l] method to be explicit αij = 0 for all j ≥ i. To keep the number of computations of the φ[l] [l]
functions down we choose αij = 0 for all l ≥ i, this ensures that m ≤ s, and the number of φ[l] functions used to compute the internal stage approximations increases as the number of stages increase. There is a certain balance between the initial computational cost of evaluating the φ[l] functions and the benefit they provide from there availability. The best combination is likely to be problem dependent. However, we believe that initially this is a reasonable choice for locating competitive methods. It is also recognized that the term m X
[l]
αij φ[l] (λ)(hL),
l=1
could be simplified due to the general nature of the φ[l] functions. However, this simplification is not made, since it may be more difficult to evaluate the resulting φ[l] functions. The fourth order commutator free Lie group method (see Subsection 2.4.3) is an example. It is also possible to represent the computations of the method in a matrix notation. Introduce the set of constant strictly lower triangular s × s matrices α [l] , with coefficients [l] αij , the first l rows of α[l] are chosen to be zero, this is equivalent to the requirements placed [l]
on αij already. The strictly block lower triangular ds × ds matrix A which describes the internal stage computations, is A=
m X l=1
Φ[l] (c)(hL) (α[l] ⊗ Id ),
where ⊗ is the Kronecker product, Id is the d × d identity matrix and the block diagonal ds × ds matrix Φ[l] , is given by ³ ´ Φ[l] (c)(hL) = diag φ[l] (c1 )(hL), . . . , φ[l] (cs )(hL) . 10
The d×ds matrix bT is defined in a similar fashion to A, except there are now no restrictions T on the vectors β [l] as there was on the matrices α[l] , therefore bT =
m X i=1
T
φ[l] (λ)(hL)(β [l] ⊗ Id ).
We are now in a position to formulate the method. For ease of notation, we introduce the vectors U , N (U ) and echL , as
U =
U1 U2 .. . Us
,
N (U1 )
N (U2 ) N (U ) = .. . N (Us )
,
e
chL
ec1 hL
ec2 hL = . .. ecs hL
.
If h represents the stepsize then the computation performed in step number n are related by the equations = AhN (U ) + echL un−1 ,
U
(1.2.4)
un = bT hN (U ) + ehL un−1 .
Alternatively the computations performed in step number n can be represented in a more Runge–Kutta type formulation as follows c
α[1]
α[2]
T β [1]
T β [2]
···
α[m]
···
T β [m]
,
where each element in row number i of α[l] is multiplied by φ[l] (ci )(hL) and where each T element in β [l] is multiplied by φ[l] (1)(hL). The resulting matrices are then added in a component by component sense to give the matrices A and bT . We next derive the nonstiff order theory for the exponential Runge–Kutta methods (1.2.4). It turns out that the order conditions for this methods are the same as the order conditions for the adaptive Runge–Kutta methods developed in [5]. An advantage of our approach is that it provides a non-recursive rule for generating each order condition from its corresponding rooted tree. In addition it is easy to generalize the same approach in the case of exponential general linear methods.
1.2.2
Nonstiff order conditions
As for many numerical methods the tools used here to develop the nonstiff order conditions are the rooted trees and the B-series. For those not familiar with these concepts we suggest the monographs [8, 23] for a complete treatment. Let 2T∗ denote the set of all bi-coloured 11
(black and white) rooted trees with the requirement that the valency of the white nodes is always one. This means that any white node will have at most one upward branch leaving that node, which leads to a node which can be coloured either white or black. The fact that the white node has valency one is because the first term on the right hand side of (1.2.1) is linear. All the bi-coloured tees in the set 2T∗ we call 2T∗ -trees. As is the case for many numerical methods the number of conditions necessary to obtain a certain order is exactly equivalent to the number of certain rooted trees up to the required order. It is therefore, necessary to have a formula for the number of trees of each order in the set 2T ∗ . Let θn be the number of rooted trees in 2T∗ with n vertices. Using Algorithm 1.2.1 from Subsection 1.2.2.1, it is possible count the number of rooted trees in 2T ∗ for each n ∈ N. In Table 1.4 we list the number of rooted trees for each order up to ten.
Θ=
n
1
2
3
4
5
6
7
8
9
10
θn Pn
2
4
11
34
117
421
1589
6162
24507
99268
2
6
17
51
168
589
2178
8340
32847
132115
i
θn
Table 1.4: The number of rooted trees in 2T∗ for all orders up to ten. We now define several operations on trees which are required for constructing the order conditions. Let τ represent an arbitrary rooted tree in 2T∗ . The order of the tree τ , denoted by |τ |, is defined as the number of vertices in the tree τ . Each tree τ can be decomposed as τ = [tp(τ ); τ1 , . . . , τ` ], where tp(τ ) = { , } represents the colour of the root node and τ1 , . . . , τ` is the forest remaining after the root node has been removed. Let ∅ represent the empty set which remains if the root of or is removed. Clearly = [tp( ); ∅] and = [tp( ); ∅]. The symmetry σ is defined in the same way as for Runge–Kutta methods, σ( ) = σ( ) = 1 and σ(τ ) = σ(τ1 ) . . . σ(τ` )µ1 !µ2 ! . . . , where the integers µ1 , µ2 , . . . count the equal trees among τ1 . . . τ` ∈ 2T∗ . This implies that ¡ ¢ ¡ ¢ σ 6= σ because the colouring of the nodes are different. The density γ of the tree is the same as for the single coloured trees and it is defined as the product over all vertices of the order of the subtree rooted at that vertex. A one to one correspondence between the rooted bi-coloured trees and the elementary differentials means that the later can be defined recursively as a function of the former, as follows Lu, N (u), F (τ )(u) = LF (τ1 )(u), N (`) (u)(F (τ )(u), . . . , F (τ ))(u)), 1 ` 12
if τ = , if τ = , if tp(τ ) = , if tp(τ ) = .
(1.2.5)
Before we continue with expressing the exact and the numerical solution in therms of the above defined elementary differentials we need to insure that they can be regarded as independent. In order to do this, it is convenient to introduce an order relation in the set 2T∗ . In the next subsection we present a constructive way how to generate and order the elements in 2T∗ . 1.2.2.1
Construction of 2T∗ -trees
When using applications like Maple or Matlab, to implement any formulas involving trees, it is crucial to have a compact way of generating all the trees up to some order in a recursive manner. It is also important to be able to identify all the elements in 2T∗ with the set of positive integers. This require an order relation in 2T∗ . The idea is to represent each 2T∗ tree as a pair of two 2T∗ -trees of lower order. Here we follow the approach based on the standard decomposition of N -trees proposed in [52]. First we need to define the product between two 2T∗ -trees. Let τ = [tp(τ ); τ1 , . . . , τk ] and π are two 2T∗ -trees. Definition 1.2.1. [52, Definition 5] The product τ · π ∈ 2T∗ is defined as follows: [tp(τ ); ∅] · π = [tp(τ ); π],
[tp(τ ); τ1 , . . . , τk ] · π = [tp(τ ); τ1 , . . . , τk , π].
Obviously tp(τ · π) = tp(τ ) and (τ · π1 ) · π2 = (τ · π2 ) · π1 for τ, π1 , π2 ∈ 2T∗ . Now we define an order relation in the set 2T∗ in therms of the standard decomposition of trees. Definition 1.2.2. [52, Definition 6] Given τ1 , τ2 ∈ 2T∗ , then τ1 < τ2 if one of the four conditions is fulfilled: • τ1 =
and τ2 = ,
• |τ1 | < |τ2 |, • |τ1 | = |τ2 | and dec1 (τ1 ) < dec1 (τ2 ), • |τ1 | = |τ2 |, dec1 (τ1 ) = dec1 (τ2 ) and dec2 (τ1 ) < dec2 (τ2 ). The standard decomposition is given by Definition 1.2.3. [52, Definition 6] The standard decomposition dec(τ ) of a 2T ∗ -tree τ of order ≥ 2 is the pair dec(τ ) = (dec1 (τ ), dec2 (τ )) ∈ 2T∗ × 2T∗ such that τ = dec1 (τ ) · dec2 (τ ) and dec2 (τ ) is maximal. Clearly the following two relations hold |τ | = |dec1 (τ )| + |dec2 (τ )|, 13
tp(τ ) = tp(dec1 (τ )).
The above definitions give us one to one correspondence between the rooted bi-coloured 2T∗ -trees and the set of all positive integers, together with a convenient way of representing each tree in a unique way as a pointer of two positive integers. All 2T∗ -trees have an additional restriction to the form of the first trees dec1 (τ ) in the standard decomposition. The fact that the valency of the white nodes is always one implies that the only tree with white root in dec1 (τ ) could be . With this in mind together with the fact that it is sufficient to store for each p the first rooted tree first(p) of order p we propose the following algorithm, based on the algorithm given in [52], for computing the standard decomposition dec(τ ) = (dec1 (τ ), dec2 (τ )) for all 2T∗ -trees of order up to certain order pmax . Algotithm 1.2.1.
dec1 (1) = 1; dec1 (2) = 2; tp(1) = 1; r(1) = 1; dec2 (1) = 0; dec2 (2) = 0; tp(2) = 2; r(2) = 1; τ = 2; for p = 2, . . . , pmax do % Index of the 1st tree of order p first(p) = τ + 1; % Construct all the 2T ∗ -trees τ (dec(τ ) = (τ1 , τ2 )) of order p for q = 1, . . . , p − 1 do % All τ1 of order q for τ1 = first(q), . . . , first(q + 1) − 1 do l = max(first(p − q), dec2 (τ1 )); % All the τ2 ≥ dec2 (τ1 ) of order p − q for τ2 = l, . . . , first(p − q + 1) − 1 do if [tp(τ1 ) = 1] or [(tp(τ1 ) = 2) and (r(τ1 ) = 1)] then τ = τ + 1; r(τ ) = p; tp(τ ) = tp(τ1 ); dec1 (τ ) = τ1 ; dec2 (τ ) = τ2 ; end end end end end
Note that r(τ ) = |τ | and that the last value of τ in Algorithm 1.2.1 is exactly equal to Θ (see Table 1.4). Thus Algorithm 1.2.1 provides us a way for computing the number of 2T∗ -trees up to an arbitrary order pmax . The values of dec(τ ) = (dec1 (τ ), dec2 (τ )) for all 2T∗ -trees of order up to three are given in Table 1.5. Now we are in a position to discuss the independence of the elementary differentials given by (1.2.5). 14
1.2.2.2
Independence of the elementary differentials
The independence of the elementary differentials insures tha the exact and the numerical solution can be uniquely expand in B-series. In this subsection we introduce a set of differential equations, similar to the set proposed in [8, Chapter 3], for which any finite number of elementary differentials evaluate to independent vectors. Let U is the set of all 2T∗ -trees up to order p and Θ is the number of the elements in U. Assume that an order relation in U is introduced and that all the pure white trees are numbered with δ1 , . . . , δp . If the standard decomposition, described in Subsection 1.2.2.1, is used the first three δς are 2, 6 and 14. Let any tree τi ∈ U be decomposed as k 1 2 ] τi = [tp(τi ); τim , τim , . . . , τim 1 2 k
for i = 1, 2, . . . , Θ,
(1.2.6)
where all the equal trees are collected together and the power notation to indicate the number of repetitions is used. This implies that all the trees τij for j = 1, 2, . . . , k are distinct. Now for every tree τi ∈ U we consider the following differential equations u0i
k Y 1 mj = νuf (i1 ) + (1 − ν) u , mj ! g(ij )
for i = 1, 2, . . . , Θ,
(1.2.7)
j=1
where uf (i1 )
u , δp = ui , 1
and
if τi1 = ∅,
else,
0, ν= 1,
,
ug(ij )
1, = uδς+1 , u ij ,
if τij = ∅, if ij = δς , else,
if tp(τi ) = , if tp(τi ) = .
The initial values are supposed to be ui (t0 ) = 0 for i = 1, . . . , Θ; i 6= δ1 and uδ1 (t0 ) = 1. If e1 , e2 , . . . , eΘ denote the natural basis in RΘ then the resulting Θ dimensional system of differential equations obey the following property: Theorem 1.2.1. The values of the elementary differentials for the differential equations (1.2.7), evaluated at the initial value are given by eδ1 , F (τi )(u(t0 )) = e , δς+1 ei ,
if i = δp , if i = δς , else,
for i = 1, 2, . . . , Θ.
Proof. The Proof follows by induction on the number of the elements Θ in the set U and from the definition of the differential equations (1.2.7). 15
i
τi
dec1 (τi )
dec2 (τi )
1
1
0
2
2
0
3
1
4
u0 = Lu + N [ ; ∅]
u01 = 0 + 1, u02 = u14 + 0,
1
[ ; ∅]
[ ; ]
u03 = 0 + u1 ,
1
2
[ ; ]
u04 = 0 + u6 ,
5
2
1
[ ; ]
u05 = u1 + 0,
6
2
2
u06 = u2 + 0,
7
1
3
[ ; ] h i ;
8
1
4
;
u08 = 0 + u4 ,
9
1
5
10
1
6
11
2
3
12
2
4
13
2
5
14
2
6
15
3
1
16
3
2
17
4
2
h h
h h h h h
£
; ; ; ; ; ; ;
i i i i i i i
2
¤
[ ; , ] ¤ £ ; 2
u07 = 0 + u3 ,
u09 = 0 + u5 , u010 = 0 + u14 , u011 = u3 + 0, u012 = u4 + 0, u013 = u5 + 0, u014 = u6 + 0, u015 = 0 + 12 u21 , u016 = 0 + u1 u6 , u017 = 0 + 12 u26 .
Table 1.5: Standard decomposition for all trees up to order 3 and their corresponding differential equations. The independence of the elementary differentials follows from the independence of the natural basis e1 , e2 , . . . , eΘ . In the case when p = 3 (Θ = 17) and the order relation defined in Subsection 1.2.2.1 is introduced, the standard decomposition for all 2T ∗ -trees, together with their recursive definitions (1.2.6) and the corresponding differential equations (1.2.7) 16
are given in Table 1.5. 1.2.2.3
Expansions of the exact and the numerical solutions
To compare the exact solution and the numerical solution as Taylor series expansions it is convenient to use B-series. For an elementary weight function a : 2T∗ → R the B-series is defined as X a(τ ) F (τ )(u). B(a(τ ), u) = a(∅)u + h|τ | σ(τ ) ∗ τ ∈2T
The exact solution of (1.2.1) can be represented by the following B-series u(t + h) = B(γ(τ )−1 , u). Note that this is exactly the same as for Runge–Kutta methods. This is due to the definition of γ. We are now interested in finding the elementary weight function which describes the operations of the numerical method.
Before we do this, it is convenient to substitute the expansions for the functions φ [l] defined in (1.2.2) into the computations of the method (1.2.3) and simplify, which gives Ui = un =
s X X
j=1 l≥0 s X X
[l]
aij (hL)l hN (Uj ) + eci hL un−1 , [l]
bj (hL)l hN (Uj ) + ehL un−1 ,
j=1 l≥0
where the coefficients of the method are now [l]
aij
=
m X [k] [k] αij φl (ci ), k=1
[l] bj
m X [k] [k] = βj φl (1).
(1.2.8)
k=1
To obtain B-series expansions of the numerical solution we need the following three results. The first result shows that the substitution of a B-series into the nonlinear part of the differential equation is again a B-series. The second result shows that a B-series operated on by powers of hL, is again a B-series. The final result shows that a B-series operated on by a power series of hL, is again a B-series. Lemma 1.2.2. Let a : 2T∗ → R be a mapping satisfying a(∅) = 1, then hN (B(a(τ ), u)) = B(a0 (τ ), u), 17
where the derivative of the elementary weight function satisfies, a0 (∅) = 0 and a0 (τ ) =
½
if τ = [ ; τ1 ], if τ = [ ; τ1 , . . . , τ` ].
0, a(τ1 ) . . . a(τ` ),
Proof. We refer to [8, 23] for very similar proofs. However, we mention that the elementary weight function depends on the colour of the trees root because the B-series is substituted into N which is represented by a black node. Therefore the elementary weight will be non-zero only for trees which have a black root node. Lemma 1.2.3. Let a : 2T∗ → R be a mapping, then (hL)l B(a(τ ), u) = B((Ll a)(τ ), u), where the elementary weight function satisfies, (Ll a)(∅) = 0, and l
(L a)(τ ) =
½
(Ll−1 a)(τ1 ), 0,
if τ = [ ; τ1 ], if τ = [ ; τ1 , . . . , τ` ].
Proof. Consider first the case when l = 1, which gives X
hL B(a(τ ), u) = hLu +
h|τ |+1
τ ∈2T∗
=
X
h|τ |
τ ∈2T∗
a(τ ) LF (τ )(u) σ(τ )
(La)(τ ) F (τ )(u), σ(τ )
where the elementary weight function must be zero for all trees with a black root node. The general l follows by recursively applying the result for l = 1. Corollary 1.2.4. Let ψx (z) be a power series ψx (z) =
X
x[l] z l ,
l≥0
and let a : 2T∗ → R be a mapping, then ψx (hL)B(a(τ ), u) = B((ψx (L)a)(τ ), u) where the elementary weight function satisfies, (ψx (L)a)(∅) = x[0] a(∅), and (ψx (L)a)(τ ) =
X l≥0
18
x[l] (Ll a)(τ ).
Proof. This corollary is proved by repeated uses of Lemma 1.2.3, as follows X ψx (hL)B(a(τ ), u) = x[l] (hL)l B(a(τ ), u) l≥0
X
=
x[l] B(Ll a(τ ), u)
l≥0
= B
X l≥0
x[l] Ll a(τ ), u .
We now have everything we need to represent the numerical method using B-series. Using Corollary 1.2.4, it follows that ehL un−1 = B((eL 1)(τ ), un−1 ), since un−1 = B(1(τ ), un−1 ), where the elementary weight function 1(τ ) is non-zero only for the empty set. Also from Lemma 1.2.2 and Corollary 1.2.4, we see that X [l] aij (hL)l hN (Uj ) = ψaij (hL)B(ξj0 (τ ), un−1 ) l≥0
= B((ψaij (L)ξj0 )(τ ), un−1 ).
Putting this all together and translating in terms of trees gives the generating functions ξi (τ ) =
s X
(ψaij (L)ξj0 )(τ ) + (eci L 1)(τ ),
j=1
α(τ ) =
s X
(ψbj (L)ξj0 )(τ ) + (eL 1)(τ ).
j=1
To represent these elementary weight functions in matrix form we introduce certain ma[k] trices. Let for l = 0, 1, 2, . . . and k = 1, . . . , m, φl , be the s × s diagonal matrix defined by ´ ³ [k]
[k]
[k]
φl (c) = diag φl (c1 ), . . . , φl (cs ) . T
Also for l = 0, 1, 2, . . . the matrices A[l] and the vectors b[l] to be defined by A[l] =
m X [k] φl (c)α[k] , k=1
[l]T
b
m X T [k] = φl (1)β [k] . k=1
19
(1.2.9)
Note that (1.2.9) are the matrix versions of (1.2.8). The matrix representation of the elementary weight functions interpreted in the natural way are therefore ξ(τ ) = (ψA (L)ξ 0 )(τ ) + (ecL 1)(τ ), α(τ ) = (ψbT (L)ξ 0 )(τ ) + (eL 1)(τ ). Given that we have B-series expansions for both the exact and the numerical solutions, we can now define order in the same way as for Runge–Kutta methods. Definition 1.2.4. An exponential Runge–Kutta method with elementary weight function a : 2T∗ → R, has order p, if for all τ ∈ 2T∗ such that |τ | ≤ p, a(τ ) =
1 . γ(τ )
This shows that it is not possible in general to obtain order by simply using a Runge– Kutta method for the nonlinear part N of the problem. There are coupling conditions between the nonlinear and linear parts of the problem despite the fact that the linear part T has been solved exactly. Note that if L = 0, then A[l] = 0 and b[l] = 0 for all l = 1, 2, . . .. Thus, all order conditions simply reduce to the order conditions corresponding to the black T trees. In this case A = A[0] ⊗ Id and bT = b[0] ⊗ Id . Therefore, the method (1.2.4) is equivalent to a Runge–Kutta method for the nonlinear part N . This method is known as the underlying Runge–Kutta method. It is clear that Definition 1.2.4 is automatically satisfied for all purely white trees. We now define the matrices 1 C j+1 , C [j] = (j + 1)! where C = diag(c1 , . . . , cs ). This strange choice of index is so that the non-recursive rule is straight forward. Now for all remaining trees the elementary weight function a of the numerical solution can equivalently be computed using the following rule: T
• Attach b[j] to the root black node. • Attach A[j] to all remaining nonterminal black nodes. • Attach A[j] e to all terminal black nodes. • Attach C [j] e to all terminal white nodes. • Attach I to all remaining white nodes. The value j is the number of white nodes directly below the corresponding node. Now for each tree multiply from the root to the leaf as in the case for Runge–Kutta methods, then multiply these expressions in a component by component sense. Table 1.6 gives the elementary differentials for all non-pure white trees of order three and less and the corresponding order, density and elementary weight function of the tree. 20
τ
|τ |
γ(τ )
1
1
N
2
2
N 0N
2
2
N 0L
2
2
LN
3
6
N 0N 0N
3
6
N 0N 0L
3
6
N 0 LN
3
6
N 0 LL
3
6
LN 0 N
3
6
LN 0 L
3
6
LLN
3
3
N 00 (N, N )
3
3
N 00 (N, L)
3
3
N 00 (L, L)
F (τ )
α(τ ) P
[0] i bi P [0] [0] ij bi aij
P
P
P P P 1 2
P
P
P
P
P
T
b[0] e T
b[0] A[0] e
[0] i bi ci
b[0] C [0] e
[1] i bi
b[1] e
[0] [0] [0] ijk bi aij ajk
b[0] A[0] A[0] e
[0] [0] ij bi aij cj
b[0] A[0] C [0] e
[0] [1] ij bi aij
b[0] A[1] e
P
P
α(τ )
T
T
T
T
T
[0] 2 i bi ci
b[0] C [1] e
[1] [0] ij bi aij
b[1] A[0] e
[1] ij bi ci
b[1] C [0] e
[2] i bi
b[2] e
[0] [0] [0] ijk bi aij aik
b[0] (A[0] e)(A[0] e)
[0] [0] ij bi aij ci
b[0] (A[0] e)(C [0] e)
[0] 2 i bi ci
b[0] (C [0] e)(C [0] e)
T
T
T
T
T
T
T
Table 1.6: Relation between elementary differentials and elementary weights.
1.2.3
IF RK and ETD RK as special cases
In this subsection we show that the IF RK and ETD RK methods, introduced in the beginning of Section 1.2, are special cases of the exponential Runge–Kutta methods (1.2.4). Therefore, the presented nonstiff order theory governs these special cases. For each of these special cases we first need to identify the structure of the φ[l] functions. One possible approach is to use the variation of constants formulae (1.1.4). It is preferable to replace the nonlinear part N with an approximation, which will make it possible to solve the integral 21
in (1.1.4) exactly. Consider approximations of the nonlinear term N which leads to the IF RK methods. Let N (u(tn−1 + τ ), tn−1 + τ ) ≈ δi eL(tn−1 +τ ) , where δi is a constant chosen in such a way that the approximation matches N (u(tn−1 + τ ), tn−1 + τ ) for τ = ci h. In this case for l = 1, . . . , s we get φ[l] (λ)(hL) = e(λ−cl )hL . (1.2.10) Every IF Runge–Kutta method (see Table 1.3) can be represented in the form (1.2.4) with φ[l] functions given by (1.2.10) and with a special choice of the coefficient matrices α [l] and T the coefficient vectors β [l] . This choice reduces the set of all order conditions to a set which consists only of the order conditions corresponding to the black trees. In the proof of this fact we will need the following lemma. Lemma 1.2.5. Let t ∈ R\{0, −1, −2, ...}, then for j = 0, 1, 2, . . . j X k=0
(−1)k 1 1 = . k!(j − k)! (k + t) t(t + 1) · · · (t + j)
Proof. We prove this statement by induction on j. For j = 0 the equality is obviously true since 1t = 1t . Assume now that the statement is true for some j. Consider (t + j + 1)
j+1 X k=0
1 (−1)k k!(j + 1 − k)! (k + t) =
=
=
j+1 X k=0
(−1)k (t + k) + (j + 1 − k) k!(j + 1 − k)! (t + k) j+1 X
j+1 X
k!(j + 1 − k)!
k=0
(−1)k 1 1 = . k!(j − k)! (k + t) t(t + 1) · · · (t + j)
k=0 j X
(−1)k
+
k=0
(−1)k (j + 1 − k) k!(j + 1 − k)! (k + t)
T
The following theorem defines the structure of the matrices α[l] and the vectors β [l] for the IF RK methods. With this structure of the coefficients, to achieve certain order, it is sufficient to satisfy only the black trees. This implies that the transformed differential equation (1.1.2) is solved using a Runge–Kutta method. Theorem 1.2.6. Let all the non-zero coefficients of an exponential Runge–Kutta method (1.2.4), with φ[l] functions given by (1.2.10) be located in column number l of the matrix T α[l] and in position number l of the vector β [l] for l = 1, 2, . . . , s. The method has order p iff all order conditions corresponding to the black trees are satisfied. Proof. From Definition 1.2.4, it follows directly that if the exponential Runge–Kutta method has order p then all order conditions corresponding to the black trees are satisfied. Let us assume that all the order conditions corresponding to the black trees are satisfied. We need 22
to prove that all the remaining order conditions are also satisfied. From the definition of the φ[l] functions (1.2.10), it follows that for j = 0, 1, 2, . . . , [l]
φj (1) =
(1 − cl )j , j!
[l]
φj (c) =
1 diag((c1 − cl )j , . . . , (cs − cl )j ). j!
(1.2.11)
Since all order conditions corresponding to the black trees involve only the coefficients A [0] , T b[0] and c, we need to express every other order condition in terms of these coefficients. T Having in mind the special structure of the matrices α[l] and the vectors β [l] , after substituting (1.2.11) into (1.2.9), we obtain for j = 1, 2, . . ., A
[j]
[j]T
b
=
=
j X
(−1)k j−k k C [0] A[0] C [0] , k!(j − k)!
k=0
(−1)k [0]T [0]k b C . k!(j − k)!
k=0 j X
(1.2.12)
From the fact that all order conditions corresponding to the black trees are satisfied, it T follows that A[0] , b[0] and c form a Runge–Kutta method. Therefore, C [0] e = A[0] e, C [0] ζ = (A[0] e)(ζ), k C [0] ζ = (A[0] e) . . . (A[0] e)(ζ),
(1.2.13)
where ζ is an arbitrary vector and the multiplications in the second and third expressions are in a component by component sense. Now we are in a position to define a procedure which transforms every coloured tree τ into a linear combination of black trees of order at most |τ |. Each tree τ can be decomposed as τ = (τb , τj , τt ), where τb is a coloured tree on the bottom with less number of white nodes than τ ; τj is tall white tree with j ≥ 1 white nodes and τt is black tree on the top. First applying formula (1.2.12) and then (1.2.13), for the order condition corresponding to a tree τ , we obtain the following three representations in terms of black trees or trees with less white vertices. If τt = ∅ then τ reduces to 1 = . τ= j! τb τ b
If τb = ∅ then τ reduces to τt τ= where δk =
(−1)k k!(j−k)!
= δ0
τt
+ · · · + δk
τt
+ · · · + δj
τt
,
for k = 0, 1, 2, . . . , j. In the general case when τ{t,b} 6= ∅, then τ can be 23
represented as τt
τt
τt τ=
+ · · · + δk
= δ0
τt + · · · + δj
.
(1.2.14)
τb τb τb τb For each of the trees in the linear combination we apply the same procedure. Thus, after a finite number of steps all the trees in the combination will be black. From (1.2.9) it is clear that the order of every single black tree cannot exceed the order of coloured tree. See Table 1.7 for the representations of all trees of order three and less. To complete the proof we need to show that a(τ ) = 1/γ(τ ) for all coloured trees τ , where |τ | ≤ p. We prove this by induction on the number of steps θ in the transformation process. Let θ = 1. P Every coloured tree τ has representation τ = jk=0 δk τk , where all τk are black trees. If γ(τt ) = x1 |τt |x2 then a(τk ) = γ(τ1k ) = x1 (|τt 1|+k)x2 and by Lemma 1.2.5 for t = |τt | it follows that j X
a(τ ) =
k=0 j X
=
k=0 j X
=
k=0
(−1)k a(τk ) k!(j − k)! 1 (−1)k k!(j − k)! γ(τk ) 1 (−1)k k!(j − k)! x1 (|τt | + k)x2
1 1 = . x1 |τt |(|τt | + 1) · · · (|τt | + j)x2 γ(τ )
=
Assume that a(τ ) = 1/γ(τ ) for all coloured trees τ with θ steps in the transformation process. Let τ be a tree with θ + 1 steps in the transformation process. From (1.2.14) it P follows that τ = jk=0 δk τk , where τk are coloured trees with θ steps in the transformation process and hence a(τk ) = 1/γ(τk ). If γ(τt ) = x1 |τt |x2 then a(τk ) = γ(τ1k ) = x1 (|τt 1|+k)x2 and by Lemma 1.2.5 for t = |τt | it again follows that a(τ ) = 1/γ(τ ). The following result shows that the non-zero coefficients of an IF RK method are those of the underlying Runge–Kutta method. Corollary 1.2.7. Under the assumptions of Theorem 1.2.6 it follows that all non-zero coefficients of an exponential Runge–Kutta method (1.2.4) are exactly equal to the coefficients of the underlying Runge–Kutta method. [l]
[l]
Proof. Since φ0 (1) = 1 and φ0 (c) = I, it follows from (1.2.9), that A[0] = α[1] + α[2] + · · · + α[s−1] ,
b[0]
T
T
T
T
= β [1] + β [2] + · · · + β [s] .
T
The result now follows from the special structure of α[l] and β [l] . 24
Table 1.7 gives all the trees up to order 3 with at least one white node in terms of linear combinations of trees with only black nodes. This represents how the order conditions for the IF RK methods are automatically satisfied by the order conditions of the underlying Runge–Kutta method. τ
τ =
=
=
= −
= =
=
1 2
=
=
− − 1 2
−
+
1 2
−
=
Table 1.7: Coloured trees as linear combinations of black trees for the IF RK methods. We now consider approximations of the nonlinear term N which leads to the ETD RK methods. Let N (u(tn−1 +τ ), tn−1 +τ ) ≈ pn−1 (τ ), where pn−1 (τ ) is interpolation polynomial of degree s − 1 that matches N (u(tn−1 + τ ), tn−1 + τ ) at the points τ = c1 h, c2 h, . . . , cs h. This approximation leads exactly to the φ[i] functions (1.1.6) from Lemma 1.1.1. We call this functions ETD φ[i] functions. The explicit form of the first few of them is λ2 λ3 λ hL + (hL)2 + (hL)3 + · · · , 2! 3! 4! 2 1 λ λ λ3 φ[2] (λ)(hL) = φ[2] (λhL) = Im + (hL) + (hL)2 + (hL)3 + · · · , 2! 3! 4! 5! 2 1 λ λ λ3 φ[3] (λ)(hL) = φ[3] (λhL) = Im + (hL) + (hL)2 + (hL)3 + · · · . 3! 4! 5! 6! φ[1] (λ)(hL) = φ[1] (λhL) = 1Im +
(1.2.15)
As an example of an ETD Runge–Kutta method, in Table 1.8, we write out an exponential integrator, which reduces to the classical fourth order Runge–Kutta method (1.3.5) for L = 0. The coefficients of the method are chosen in such a way that all nonstiff order conditions up to order four as well as half of the conditions of order five are satisfied. As we mentioned before, other choices for the φ[l] functions (1.2.2), rather than the IF (1.2.10) and the ETD (1.1.6) functions, are also possible. In Chapter 2 we give examples of functions which originate from the framework of Lie group methods. The question how to find the best set of functions (1.2.2) is open and needs further investigation. It seems that the most commonly used functions in the literature are the ETD φ[i] functions. In the next subsection we present some third and fourth order ETD RK methods, which recently have been studied by several authors. 25
0 1 [1] 2φ
0 0
6 [2] 21 [1] 50 φ − 25 φ 19 [1] 9 [2] [3] 20 φ − 10 φ − 3φ
2 [1] 6 [2] 25 φ + 25 φ 108 [3] 21 [2] 5 φ − 5 φ
b1 (hL)
b2 (hL)
0 0
0 0
0 1 [1] 20 φ
33 [2] 10 φ
−
0 +
123 [3] 5 φ
b3 (hL)
0 b4 (hL)
I
1 e 2 hL 1 hL e2 , hL e
ehL
where 31 [1] 17 [2] φ − φ + 6φ[3] − 4φ[4] , 30 5 1 [1] 1 [2] b2 (hL) = − φ + φ − 4φ[3] + 12φ[4] , 10 5 1 [1] 23 [2] φ + φ − 8φ[3] − 4φ[4] , b3 (hL) = 30 5 1 [1] 7 [2] b4 (hL) = φ − φ + 6φ[3] − 4φ[4] . 30 5
b1 (hL) =
Table 1.8: A fourth order ETD Runge–Kutta method.
1.2.4
More examples of ETD RK methods
All methods presented in this subsection are based on in some sense “mysterious” observations, which do not explicitly involve the order theory from Subsection 1.2.2 or any other order theory. We include this methods as examples of exponential integrators, which during the last few years have been studied by several authors [14, 29, 30, 35, 36]. Thus these methods have obtained an important place on their own. We try as much as reasonably possible, for each method, to present the authors main motivation used in the derivation process. In the presentation of each method we use the notations introduced in Section 1.3. We mention also that all the methods satisfy the classical (nonstiff) order conditions form Subsection 1.2.2 up to the order which their authors predict, but most of them suffer from order reduction in the stiff case. We will comment more on this in Subsection 1.2.5.2. In Tables 1.9 and 1.10 we give the two multistage methods proposed in [14]. When the linear part L is equal to zero, they reduce to the classical third and fourth order Runge–Kutta method respectively. Regarding the derivation of the fourth order method (Table 1.10), in [14] is written: “The computer algebra package Maple was used to confirm that this method is indeed fourth order”. A surprising connection between the fourth order method of Cox–Matthews (Table 1.10) and the fourth order commutator free Lie group method (Table 2.6), based on the affine algebra action (see Section 2.3), was found in [36, Article 5]. Note that the internal stages 26
0 1 [1] 2φ −φ[1]
0 0 2φ[1]
I
0 0 0
1 hL 2
e ehL
φ[1] − 3φ[2] + 4φ[3] 4φ[2] − 8φ[3] −φ[2] + 4φ[3]
ehL
Table 1.9: Third order method of Cox–Matthews.
0 1 [1] 2φ 1 2φ
0 ¡ ¢¡ [1] hL 2
e
0 0
hL 2
−I
¢
φ[1] − 3φ[2] + 4φ[3]
1 [1] 2φ
0 2φ[2] − 4φ[3]
φ
0 0
0 0
0 ¡ ¢ [1] hL
0
2
2φ[2] − 4φ[3]
0 −φ[2] + 4φ[3]
I
1 e 2 hL 1 e 2 hL hL e hL e
Table 1.10: Fourth order method of Cox–Matthews. of this two methods are the same. It is shown in [36, Article 5] that the main step of Cox– Matthews method can be reproduced based on the techniques of continuous Runge–Kutta methods [24, Chapter II.6]. Motivated from the same idea, but also applied to the internal stages of the method, a new fourth order method is derived in [36, Article 5]. This method, which is also based on the classical fourth order Runge–Kutta method (1.3.5), is given in Table 1.11.
0 1 [1] φ 2
0 0
0 0
0 0
1 [1] [2] 2φ − φ φ[1] − 2φ[2]
φ[2]
0
0
0
2φ[2]
0
φ[1] − 3φ[2] + 4φ[3]
2φ[2] − 4φ[3]
2φ[2] − 4φ[3]
−φ[2] + 4φ[3]
I
1 e 2 hL 1 e 2 hL ehL
ehL
Table 1.11: Fourth order method of Krogstad. Similar techniques are used in the construction of ETD2RK3 (Table 1.12) and ETD2CF3 (Table 1.13) methods proposed in [45]. The first of this methods also reduces, for L = 0, to 27
0 1 [1] 2φ [1] φ − 4φ[2]
0 0 4φ[2]
0 0 0
φ[1] − 3φ[2] + 4φ[3] 4φ[2] − 8φ[3] −φ[2] + 4φ[3]
I 1 hL 2
e ehL ehL
Table 1.12: Third order ETD2RK3 method.
0 1 [1] φ 3 2 [1] φ − 43 φ[2] 3
0 0 4 [2] 3φ
0 0 0
φ[1] − 29 φ[2] + 9φ[3] 6φ[2] − 18φ[3] − 32 φ[2] + 9φ[3]
I 1
e 3 hL 2 e 3 hL ehL
Table 1.13: Third order ETD2CF3 method. the classical third order Runge–Kutta method. It is simply a modification of the third order method of Cox–Matthews, which adapts the continuous idea for the internal stages as well. The second method is based on the third order commutator free Lie group method proposed in [12]. However, instead of using the affine algebra action (see Section 2.3), the internal stages of ETD2CF3 are derived based on the algebra action proposed in [Article 1, p.91]. Splitting of the main step is again avoided by using the continuous technique suggested in [36, Article 5].
1.2.5
Exponential RK methods for parabolic PDEs
The order theory presented in Section 1.2.2 highly relays on the assumption that L is bounded linear operator on a d-dimensional Euclidian space Rd . In this case the meaning of the exponential operator etL is given by the well known formula e
tL
=
∞ k k X t L k=0
k!
,
t ∈ R.
The related ETD φ[i] functions are also defined in a similar way (see 1.2.15). To analyze the order of an exponential integrator applied to a Partial Differential Equation (PDE), we need a framework, which allows us to define the exponential and the related functions in the case when L is an unbounded linear operator. In the sequel, we follow the presentation in [29, 30] and restrict our considerations to the framework of sectorial operators and analytic semigroups. It is well known that parabolic problems like reaction-diffusion equations, incompressible Navier-Stokes equations in two and three space dimensions as well 28
as displacement of a shock fit into this framework (see [26, Chapter 3], [42, Section 7.3] and [54, Lecture 5]). The key idea, used in [29, 30], is to analyze exponential integrators of Runge–Kutta type for parabolic PDEs based on the observation that any parabolic PDE can be viewed as an abstract Ordinary Differential Equation (ODE). We illustrate this concept in the next example. Consider the following one dimensional reaction-diffusion equations µ ¶ ∂U ∂ ∂U (x, t) = a(x) (x, t) + ϕ(x, t, U (x, t)), ∂t ∂x ∂x (1.2.16) U (x, 0) = U0 (x) 0 < x < 1, U (0, t) = U (1, t) = 0 t > 0. We define the functions u(t) = [x → U (x, t)] ∈ L2 (0, 1), u0 = [x → U (x, 0)] ∈ L2 (0, 1),
N (u, t) = [x → ϕ(x, t, u(x))] ∈ L2 (0, 1)
and the linear operator
·
∂ Lu = x → ∂x
µ
∂U (x, t) a(x) ∂x
¶¸
.
Thus, the equation (1.2.16) is equivalent to the following abstract semilinear problem on the Banach space L2 (0, 1) u0 (t) = Lu + N (u, t),
u(t0 ) = u0 ,
(1.2.17)
where now the linear operator L is unbounded. An appropriate setting where the exponential and the related operators have a precise meaning, for unbounded linear operator L in a Banach space X, is the framework of sectorial operators and analytic semigroups. We next introduce the main concepts and some error bounds which are crucial for the convergence analysis presented in the recent papers [29, 30]. 1.2.5.1
The framework
Let (X, || · ||X ) be a real or complex Banach space and let L be a linear operator in X with dense domain D(L) = {x ∈ X : Lx ∈ X} equipped with the graph norm ||x||D(L) = ||Lx||X . In order to define the exponential operator etL we need the notion of sectorial operators. Definition 1.2.5. ([26, Definition 1.3.1], [42, Definition 2.0.1]) A densely defined and closed linear operator L in X is called sectorial if there are constants a ∈ R, π/2 < ϕ < π and M > 1 such that the following resolvent condition is satisfied ||(λI − L)−1 ||X←X ≤ 29
M |λ − a|
(1.2.18)
Figure 1.1: Domain of validity for condition (1.2.18) (shaded). on the sector {λ ∈ C : λ 6= a, |arg(λ − a)| < ϕ}. The sector from Definition 1.2.5 is plotted on Figure 1.1. Next we give the definition of an analytic semigroup. Definition 1.2.6. ([26, Definition 1.3.3], [42, Section 2]) The family (S(t))t≥0 of linear operators on X forms a semigroup if the relations S(0) = I and S(s)S(t) = S(s + t),
s, t ≥ 0,
are satisfied. The semigroup is called analytic if the map t → S(t) is analytic. etL
Assuming that the linear operator L is sectorial, we can define the exponential operator by the means of the Cauchy integral formula.
Definition 1.2.7. ([26, Definition 1.3.4], [42, Definition 2.0.2]) Let L be a sectorial operator in X, and, for some radius r > 0 and an angle π/2 < η < ϕ, let the counterclockwise oriented contour Γ consists of the half lines γ+ and γ− given by γ± = a + {λ ∈ C : |λ| ≥ r, arg(λ) = ±η},
¡ ¢ and the arc γr = a + {λ ∈ C : |λ| = r, arg(λ) ≤ η}, see Figure 1.2. Then, the family etL t≥0 defined through Z 1 eλt (λI − L)−1 dλ, t > 0, e0L = I, (1.2.19) etL = 2πi Γ is an analytic semigroup generated by L in X.
The operator etL : X → X is linear and bounded in X for each t ≥ 0. This follows directly from the resolvent bound (1.2.18). For the rest of the presentation in this section 30
Figure 1.2: The contour Γ from Definition 1.2.7. we assume that the linear operator L is sectorial and thus the operator e tL is well-defined and bounded in X. When the parameter a is negative, form Definition 1.2.5, follows that the operator −L has a bounded inverse. Motivated by the definition of the gamma function, for any α > 0, we can define fractional powers of −L through the following expression Z ∞ Z ∞ tL α−1 −α e t dt = (−L) e−x xα−1 dx = (−L)−α Γ(α). 0
0
Therefore (−L)
−α
1 = Γ(α)
Z
∞
etL tα−1 dt.
0
The fractional powers of −L, for any α > 0, are now defined by ¡ ¢ ¡ ¢−1 (−L)α : D (−L)α → X : x → (−L)α x := (−L)−α x, ¡ ¢ where D (−L)α = range ((−L)−α ). e = ωI − L, with ω > a. Fractional powers of If a > 0, we consider the shifted operator L e L are defined in the similar way. From the Cauchy’s integral formula (1.2.19) and the resolvent bound (1.2.18) the following uniform stability bounds are derived in [26, 42, 54] ||etL ||X←X ≤ C0 eat ,
e α tL
ωt −α
||L e ||X←X ≤ Cα,ω e t
,
for 0 ≤ t ≤ T , for 0 < t ≤ T,
ω>a.
(1.2.20)
We next discuss the main assumptions which the nonlinearity N has to satisfy. As in the ODE case, the minimal requirement on N is to be Lipschitz-continuous in X. However, 31
for abstract ODEs, the above condition is hardly satisfied in practice. For example, if X = L2 (0, 1) and N (u, t) = u2 , it is not even guaranteed that u2 ∈ L2 (0, 1). Therefore, we need to consider N like an operator defined on a smaller space V , such that D(L) ⊂ V ⊂ X. Thus, the main assumption imposed on the nonlinearity N : V × [0, T ] → X is to be locally Lipschitz-continuous in a strip along the exact solution u. Therefore, there exist a real number L(R, T ), such that ||N (v, t) − N (w, t)||X ≤ L||v − w||V for all t ∈ [0, T ] and max(||v − u(t)||V , ||w − u(t)||V ) ≤ R, where R is the radius of the strip. e α ), 0 ≤ In [29, 30], the space V is chosen to be the fractional power space Xα = D(L α e v||X . The value of α depends α < 1. Thus V is linear Banach space with norm ||v||V = ||L from the underlying space X and form the type and dimensionality of the problem. It can be determined by the following Lemma, which relies on the classical Sobolev embedding theorem (see[54]). Lemma 1.2.8. Let 1 ≤ p < ∞ and Ω ⊂ Rd be an open and bounded set with smooth boundary. Let L be sectorial in Lp (Ω) with domain D(L) ⊂ W m,p (Ω) for some m ≥ 1. Then for 0 ≤ α ≤ 1 ¯ ⊂ L∞ , V = Xα ⊂ C ν (Ω)
0 ≤ ν < mα −
d p
with continuous embeddings. From Lemma 1.2.8, it follows that for second-order parabolic problems on L 2 (Ω) i.e. m = 2 and p = 2, V ⊂ L∞ for d/4 < α < 1. Thus it is clear that for three dimensional problems, we have to work with stronger norms than in the one and two dimensional cases. We next observe that the definition of the space V = Xα is related with the stability bounds (1.2.20) for the exponential and the related operators. To illustrate this connection, we represent the exact solution of (1.2.17) by the variation of constants formulae u(t) = etL u0 + |{z} V
Z
t 0
)L e|(t−τ ), τ ) dτ. {z } |N (u(τ {z } V ←X
(1.2.21)
X
Since u(t) ∈ V , it is clear that we need to consider etL like an operator from X to V . From (1.2.20) and the definition of the operator norm, for 0 < t ≤ T , we have ||etL ||V ←X = sup w∈X w6=0
e α etL w||X ||L ||etL w||V = sup ||w||X ||w||X w∈X w6=0
e α etL ||X←X ||w||X ||L ≤ sup ≤ Ct−α . ||w||X w∈X w6=0
32
(1.2.22)
Let us now define the operators φ[i] for i = 1, 2, . . . by the formula Z t τ i−1 dτ. φ[i] (tL) = t−i e(t−τ )L (i − 1)! 0
(1.2.23)
Note that φ[i] are bounded operators in X and that when L is a matrix, the above formulas reproduce exactly the ETD φ[i] functions given in (1.1.6). The stability estimate (1.2.22) can be easily extended to all operators φ [i] Z t τ i−1 [i] −i dτ ||φ (tL)||V ←X ≤ t ||e(t−τ )L ||V ←X (i − 1)! 0 (1.2.24) Z t −i −α i−1 −α ≤ Ct (t − τ ) τ dτ ≤ Ct . 0
The last assumption which we impose on the problem (1.2.17) is that it possesses a sufficiently smooth solution u : [0, T ] → V with derivatives in V . In addition we suppose that N : V × [0, T ] → X is sufficiently often Fr´echet differentiable in a strip along the exact solution. The above assumption, insures that the composition map z : [0, T ] → X : t → z(t) = N (u(t), t) is a smooth map and therefore z admits Taylor series expansion. In the next two subsections, we consider respectively explicit and implicit exponential integrators for solving semilinear parabolic PDEs. 1.2.5.2
Explicit methods
Under the assumptions given in the previous subsection, the convergence properties of the explicit exponential Runge–Kutta methods presented in Subsection 1.2.1 are studied in [29]. The main idea behind the analysis presented there is to derive bounds for the error recursion between the exact solution represented by the variation of constants formulae (1.2.21) and the numerical solution given by the scheme (1.2.4). Because of the α-dependence in (1.2.22) and (1.2.24), the error bounds depend from the size of the parameter α. Therefore, the number of order conditions for a method to be of a given order p also depends form α. The simplest possible case is when α = 0 (V = X). In Table 1.14 we list the stiff order conditions derived in [29], which guarantee that an s-stage explicit exponential Runge–Kutta method (1.2.4) is of order p (for p ≤ 4), in the case when α = 0 and the ETD φ[i] functions (1.2.23) are used. The operators J and K, which appear in conditions number 5,7,8 and 9 are arbitrary bounded operators in X. The order conditions from Table 1.14 allow us to analyze the stiff order of the explicit exponential Runge–Kutta methods presented so far. It is shown in [29], that the third order method of Cox–Matthews [14] as well as the method ETD2RK3 (see Subsection 1.2.4) suffer from order reduction down to order two in the worst case. The method ETD2CF3 given in Subsection 1.2.4 has a full stiff order three, in the case when α = 0. Similarly, the fourth 33
No.
order
1
1
order condition s X
bi (hL) = φ[1] (hL)
i=1
2
s X
2
bi (hL)ci = φ[2] (hL)
i=2
3 4
i−1 X
2
aij (hL) = ci φ[1] (ci hL)
j=1 s X
3
bi (hL)c2i = 2φ[3] (hL)
i=2
5
s X
3
³
bi (hL)J φ
i=2
6
s X
4
[2]
(ci hL)c2i
−
i−1 X j=2
´ aij (hL)cj = 0
bi (hL)c3i = 6φ[4] (hL)
i=2
7
4
s X i=2
8
4
s X
9
4
i−1 X j=2
s X i=2
´ 1X aij (hL)c2j = 0 2 i−1
bi (hL)J φ[3] (ci hL)c3i −
bi (hL)J
i=2
³
³
j=2
aij (hL)J φ[2] (cj hL)c2j −
³ bi (hL)ci K φ[2] (ci hL)c2i −
i−1 X j=2
j−1 X k=2
´ ajk (hL)ck = 0
´ aij (hL)cj = 0
Table 1.14: Stiff order conditions for explicit exponential Runge–Kutta methods for α = 0. order method of Cox–Matthews [14] has only stiff order two and the fourth order method of Krogstad (see Subsection 1.2.4) has a stiff order three in the worst case. For each of the above methods, higher order (up to the classical nonstiff order) is possible, if additional smoothness conditions of the problem are satisfied. In general, it can be shown that it is not possible to construct stiff fourth order exponential Runge–Kutta method with only four stages [29]. In Table 1.15 we present the fourth order, five stages method of Hochbruck– Ostermann [29], which satisfies all the order conditions from Table 1.14 except the condition number 7. It is satisfied only in a weak form, that is with bi (0) instead of bi (hL). All exponential integrators which we have considered so far were based on the explicit idea. In some sense, it seems unnecessary to consider implicit exponential integrator, since the hope is to overcome the stiffness by using the exponential and the related functions φ [i] in the format of the method. However, we should keep in mind that for implicit exponential 34
1 [1] 2φ
0 0
0 0
0 0
0 0
e
1 [1] [2] 2φ − φ φ[1] − 2φ[2]
φ[2]
0
0
0
e 2 hL
0
0
ehL
0
e 2 hL
4φ[2] − 8φ[3]
ehL
− 41 φ[2] − a52
¡ hL ¢ 2
φ[1] − 3φ[2] + 4φ[3]
where a52
I
0 1 [1] 2φ
¡ hL ¢ 2
= 12 φ[2]
¡ hL ¢ 2
φ[2] ¡ hL ¢
a52
2
0
φ[2] ¡ hL ¢
a52
2
0
1 [2] 4φ
− a52
¡ hL ¢ 2
−φ[2] + 4φ[3]
− φ[3] (hL) + 14 φ[2] (hL) − 12 φ[3]
¡ hL ¢
1 hL 2 1
1
,
2
Table 1.15: Fourth order method of Hochbruck–Ostermann with five stages. integrators, we can solve the arising nonlinear equations by using fixed-point iteration. What we gain by using implicit multistage integrators is the ability to increase the stage order of the method. Motivated from this observation, a special class of implicit exponential integrators of collocation type, for solving parabolic problems, is studied in [30]. Next, we consider this class of methods. 1.2.5.3
Implicit collocation methods
The main idea behind the exponential collocation integrators [30], is to replace the function N (u, t) in the variation of constants formulae (1.1.4) Z h hL u(tn−1 + h) = e un−1 + e(h−τ )L N (u(tn−1 + τ ), tn−1 + τ )dτ 0
by polynomial approximation pn−1 (τ ) of collocation type. Let c1 , c2 , . . . , cs be non-confluent collocation nodes in the interval [0, 1] and Ui ≈ u(tn−1 + ci h). If pn−1 (τ ) is the unique collocation polynomial of degree s − 1, which satisfies the conditions pn−1 (ci h) = N (Ui , tn−1 + ci h) for i = 1, 2, . . . s, then we can define the numerical solution un of (1.2.17) at time tn = tn−1 + h by the formulas Z h hL un = e un−1 + e(h−τ )L pn−1 (τ )dτ, 0 (1.2.25) Z ch i
Ui = eci hL un−1 +
e(ci h−τ )L pn−1 (τ )dτ.
0
Since pn−1 (τ ) is a polynomial of τ with coefficients that are linear combinations of the values N (Ui , tn−1 + ci h), calculating exactly the integrals form (1.2.25), we obtain Z ci h s X e(ci h−τ )L pn−1 (τ )dτ = h aij (ci hL)N (Uj , tn−1 + cj h), 0
j=1
35
Z
h
e
(h−τ )L
pn−1 (τ )dτ = h
0
s X
bi (hL)N (Ui , tn−1 + ci h),
i=1
where the coefficients aij and bi are linear combinations of the first s, φ[i] functions (1.2.23). Thus, it is clear that the general format of an exponential integrator of collocation type is just a special case of the exponential Runge–Kutta methods introduced in Subsection 1.2.1. The advantage of the above approach is that it does not requires any order theory because of the high stage order. Therefore the construction of new integrators is easy, since there is no need to solve complicate order conditions. Of course all this comes on the price of working with a very restrictive class of methods. The main advantage of the exponential integrators of collocation type comes from the fact that they are implicit, which allows to build up the stage order of the method, and at the same time the nonlinear equations which arise in the format of the method can be simply solved using fixed-point iterations. In [30], the convergence properties of the exponential integrators of collocation type, applied to semilinear parabolic PDEs, are analyzed. It is shown there, that the methods converge at least with their stage order. Higher and even fractional order of convergence is possible if additional temporal and spatial regularity are required. For problems with periodic boundary conditions classical (nonstiff) order can be obtained. The above result suggests that the ideal integrators for solving stiff problems will be explicit exponential integrators with higher stage order. Thus, we naturally arrive to the idea of using general linear methods [8, 69] like underlying methods in the construction of exponential integrators. In the next section we introduce the main concept behind general linear methods and show how they can be extended to the exponential setting.
1.3
General linear methods and exponential integrators
In the previous two sections we have considered exponential integrators based on the two main classes for solving ordinary differential equations: linear multivalued (Section 1.1) and multistage (Section 1.2) methods. General linear methods (GLMs) were introduced in [7] as a unifying framework for the traditional methods to study the properties of consistency, stability and convergence. The extremely general nature of this methods allows new methods with clear advantages over the traditional methods to be constructed. We next present the main idea behind general linear methods and introduce some notations, which are used in the representation of every exponential integrator included in this thesis.
1.3.1
Formulation of the methods
For simplicity of the presentation, we again represent the original semilinear problem (1.1.1) in the following autonomous form u0 = Lu + N (u(t)) = f (u(t)),
u(t0 ) = u0 , 36
f (u(t)) : Rd → Rd .
(1.3.1)
Following [8, Chapter 5], for the numerical solution of (1.3.1), we consider a method in which a collection of vectors forms the input at the beginning of a step, and a similar collection is passed on as output from the current step and as input into the next step. Thus the method is a multivalue method. In addition, as for Runge–Kutta methods, it is assumed that during the computations that constitute the step of the method, s approximations to the solution at points near the current time step are evaluated. Therefore the method is also a multistage method. Methods which are both multistage and multivalued are known as general linear methods. Assume that at the beginning of step number n, r quantities [n−1]
u1
[n−1]
, u2
, . . . , u[n−1] , r
are available from approximations computed in the previous step number n − 1. The corresponding quantities evaluated during the step number n are denoted by [n]
[n]
u1 , u2 , . . . , u[n] r . Let the internal stage values of step number n are denoted by U1 , U2 , . . . , U s and the derivatives evaluated at this step are denoted by f (U1 ), f (U2 ), . . . , f (Us ). If h represents the stepsize, then the quantities imported into and evaluated in step number n are related by the equations Ui = [n] ui
=
s X
j=1 s X j=1
aij hf (U j) + bij hf (U j) +
r X
j=1 r X
[n−1]
dij uj
,
i = 1, 2, . . . s, (1.3.2)
[n−1] vij uj ,
i = 1, 2, . . . r,
j=1
where A = (aij ), B = (bij ), D = (dij ), V = (vij ) are coefficients of the method. Introducing the vector notations [n] [n−1] u1 u1 f (U1 ) U1 [n] [n−1] u u f (U2 ) U2 2 2 [n] [n−1] , u = , u = U = . , f (U ) = . , .. .. .. .. . . [n] [n−1] ur ur f (Us ) Us allows us to rewrite the method (1.3.2) in the following more compact form U A ⊗ I d D ⊗ Id hf (U ) = , [n] [n−1] u B ⊗ Id V ⊗ Id u 37
where ⊗ is the Kronecker product and Id is the d × d identity matrix. Usually, in order to simplify the notations, the Kronecker product with Id is omitted. Thus the coefficients of every general linear method, given by the matrices A, B, D, V , can be written together as the following partitioned (s + r) × (s + r) matrix
M =
A D B V
.
(1.3.3)
Note that the matrix A from (1.3.3) is similar to the A matrix in the Runge–Kutta methods and determines the implementation cost of the method. The quantities u[n] , which are passed from step to step, can have a very general nature. Common choices in linear multistep methods, are approximations to the solution and the derivatives at various previous points, backward difference approximations or approximations to the Nordsieck vector. Like for Runge–Kutta methods, the stage values U of GLMs are approximations to the solution at points near the current time step. That is Ui ≈ u(tn + ci h), where usually 0 ≤ ci ≤ 1, for i = 1, 2, . . . , s. The vector c = [c1 , c2 , · · · , cs ]T , is called the vector of abscissae.
Examples of GLMs Next, we give few examples showing how some well know traditional methods can be alternatively represented like general linear methods. Let us first consider k-step linear multistep methods of Adams type un = un−1 + h
k X
βi f (un−i ).
(1.3.4)
i=0
Natural way of rewriting (1.3.4) as GLM is to choose the number of the internal stages to be only one, corresponding to approximation of u(tn ), and the k + 1 quantities passed from step to step to be of the form
u[n−1]
un−1
hf (un−1 ) = hf (un−2 ) .. . hf (un−k ) 38
.
Thus, (1.3.4) can be represented in the following general linear form
U1
un hf (U1 ) hf (un−1 ) .. . hf (un−k−1 )
=
β0 1 β 1 · · · β0 1 β 1 · · · 1
0
0
0 .. .
0 .. .
1 .. .
0
0
0
··· ··· ···
βk−1 βk
hf (U1 )
βk−1 βk un−1 0 0 hf (un−1 ) 0 0 hf (un−2 ) .. .. .. . . . 1 0 hf (un−k )
.
Similarly, backward differentiation formulae (BDF) methods
un =
k X
αi un−i + hβ0 f (un ),
i=1
are represented as
U1
u n un−1 un−2 .. . un−k−1
=
β 0 α1 α2 · · · β 0 α1 α2 · · · 0
1
0
0 .. .
0 .. .
1 .. .
0
0
0
··· ··· ···
αk−1 αk
αk−1 αk 0 0 0 0 .. .. . . 1 0
hf (U1 )
un−1 un−2 . un−3 .. . un−k
As far as Runge–Kutta methods are concern, it is usually convenient to represent them as general linear methods which pass only one quantity from step to step. That is an approximation to u(tn−1 ). In this way, we can identify the matrices A and B form (1.3.3) with the A matrix and the row vector bT of the Runge–Kutta method respectively. Choosing D = e and V = 1, we can easily see that the classical fourth order Runge–Kutta method 0 1 2 1 2
1 2
0
1 2
1
0
0
1
1 6
1 3
1 3
39
, 1 6
(1.3.5)
can be written as
U1
0
U 1 2 2 U3 = 0 U4 0 1 un 6
0
0
0
0
1 2
0
0
1
1 3
1 3
0 1
0 1 0 1 0 1 1 6 1
hf (U1 )
hf (U2 ) hf (U3 ) . hf (U4 ) un−1
We mention also that it is not always appropriate to represent a Runge–Kutta method like a general liner method with r = 1. Example is the Lobatto IIIA method 0 1 2 1 2
5 24 1 6 1 6
1 3 2 3 2 3
1 − 24 1 6 1 6
.
It is a method which has the so called FSAL property. This means that the First stage of the current step is the Same As the Last stage of the previous step. This special property allows us to eliminate the first stage of the method by passing one more quantity from step to step. Thus the method can be represented in the following general linear form 1 5 1 hf (U1 ) U1 3 − 24 1 24 1 1 U 2 hf (U ) 1 2 2 3 6 6 = . 1 1 un 2 1 u n−1 3 6 6 0 1 0 0 hf (U2 ) hf (un−1 )
1.3.2
Construction of practical GLMs
We first introduce the concepts of pre-consistency, consistency, stability and convergence of general linear methods. The pre-consistency and consistency conditions for a GLM insure that the method can solve exactly the trivial differential equations u0 = 0 and u0 = 1. If the pre-consistency vector u ¯ and the consistency vector v¯ are determined by u[n−1] = u ¯u(tn−1 ) + v¯hu0 (tn−1 ) + O(h2 ), u[n] = u ¯u(tn ) + v¯hu0 (tn ) + O(h2 ),
then the pre-consistency conditions are D¯ u = e,
Vu ¯=u ¯,
and the consistency conditions are Ae + D¯ v = c,
Be + V v¯ = u ¯ + v¯, 40
where e = [1, 1, · · · , 1]T ∈ Rs . Stability of GLM concerns the boundedness of the numerical solution when the method is applied to the trivial differential equation u0 = 0. Thus the general linear method M , defined by (1.3.3), is stable if there exist a constant C such that for all n = 1, 2, . . ., ||V n || ≤ C. The method M is convergent if there exist a non-zero vector u ¯ ∈ Rr such that if u[0] = u ¯u(t0 ) + O(h), then u[n] = u ¯u(t0 + nh) + O(h) for all n, given that nh is bounded. The fundamental result that consistency and stability are necessary and sufficient conditions for convergence of a general linear method is proven in [7]. The order of accuracy of a general linear method is defined relative to a starting procedure S. It is used to produce the initial vector u[0] from the given initial value u0 . In general the starting procedure can be represented by ¯ = S11 hf (U ¯ ) + S12 u0 , U ¯ ) + S22 u0 , u[0] = S21 hf (U ¯ and f (U ¯ ) are where the vectors U ¯ = U
¯1 U ¯2 U .. . ¯s¯ U
,
¯) = f (U
¯1 ) f (U ¯2 ) f (U .. . ¯s¯) f (U
,
¯1 ), f (U ¯2 ), . . . , f (U ¯s¯) are the derivatives at the internal stages U ¯1 , U ¯2 , . . . , U ¯s¯. Note and f (U that the number of stages s¯, required to compute the r components of u[0] , can be different from the number of the stages s used in the method. From the pre-consistency conditions follows, that S12 = e and S22 = u ¯. Similarly to (1.3.3), the starting procedure can be also represented by a partitioned (¯ s + r) × (¯ s + 1) matrix S11 S12 . S= S21 S22
If E represents the shift operator, that shifts the exact solution from tn−1 to tn , then the general linear method M has order of accuracy p if M ◦ S − S ◦ E = O(hp+1 ), where M ◦ S denotes the combined effect of applying the starting procedure S followed by a step of the method M . The meaning of S ◦ E is defined in a similar way. Until recently, very few general linear methods which are significantly different from the traditional methods have been developed. This is mainly due to the very general structure of these methods, which results in a complicated order theory. A reasonable way to advance in the construction of new competitive methods is to introduce some initial assumptions on the structure of the methods. In this way the originally large class of GLMs is limited to a smaller subclass where practical methods are likely to exist. Like we saw in Subsection 1.2.5.3, it is important to base the construction of exponential integrators on methods which have higher stage order. Thus a desirable property of the 41
underlying method is to have stage order equal to the overall order p of the method. Imposing this assumption simplifies the order construction of GLMs and provides asymptotically correct error estimates, which are useful in variable stepsize, variable order implementations. To overcome difficulties with changing the stepsize, it is also convenient to require the quantities passed from step to step to be approximations of the Nordsieck vector i.e.
u[n]
u(tn )
≈
hu0 (tn ) .. . hp (p) p! u (tn )
,
with r = p+1. Under the above assumptions the order conditions for general linear methods become much simpler. It can be shown (see [8, Chapter 5], [69, Chapter 3]) that necessary conditions for a method to have stage order and order p are D = C − ACK,
V = E − BCK,
where the matrices C ∈ Rs×p+1 , K ∈ Rp+1×p+1 and E = exp(K) are given by C=
·
cp−1 c2 ··· e c 2! (p − 1)!
cp p!
¸
,
K=
£
¤ 0 e1 e2 · · · e(p−1) ep .
Other important property which we would like a GLM to have is that its stability region should be identical to the stability region of the corresponding Runge–Kutta methods. Sufficient conditions for a general linear method to have Runge–Kutta stability are BA ≡ XB,
BD ≡ XV − V X, where ≡ means two matrices are equivalent except for their first rows, X is a doubly companion matrix and the spectrum of V is {0, 1} (see [69, Chapter 3]). General linear methods satisfying the above conditions are known to have a property called Inherent Runge–Kutta Stability (IRKS). Recently practical general linear methods with IRKS have been constructed in [9, 10, 69].
1.3.3
Exponential general linear methods
Let us now consider how general linear methods can also be extended to the exponential setting. Motivated from (1.2.3) and (1.3.2), for the solution of the semilinear problem 42
(1.1.1), we consider the following unified format of exponential integrators Ui = [n] ui
=
m s X X
j=1 l=1 m s X X
[l]
αij φ[l] (ci )(hL) hN (Uj ) + [l] βij
[l]
φ (1)(hL) hN (Uj ) +
m r X X
j=1 l=1 m r X X
[n−1]
[l]
δij φ[l] (ci )(hL) uj
[l] νij
, (1.3.6)
[l]
φ (1)(hL)
[n−1] . uj
j=1 l=1
j=1 l=1
Similarly, to the traditional GLMs, we can represent (1.3.6) in the following matrix form U A(φ) D(φ) hN (U ) = , [n] u B(φ) V (φ) u[n−1]
where each of the coefficient matrices A(φ), B(φ), D(φ), V (φ) has entries which are linear combinations of the φ[l] functions. Therefore, the matrix A(φ) D(φ) , M (φ) = (1.3.7) B(φ) V (φ)
has also entries which are linear combinations of the φ[l] functions. All exponential integrators presented in this thesis are given exactly in the form (1.3.7). In addition the following convention is used: The argument of φ[l] is always dropped when it is evaluated at the corresponding abscissae value. According to our knowledge, the first exponential integrators which fit in the format (1.3.6), were constructed in [36, Article 5]. We discuss these methods in Subsection 2.4.2. However, there are several interesting properties regarding the Generalized Integrating Factor Runge–Kutta (GIF/RK) methods of Krogstad which we would like to point out here. First of all they are exponential general linear methods which pass the quantities un , hNn−1 , hNn−2 , . . . from step to step. Secondly they are examples of methods, where other than the traditional IF and ETD φ[i] functions are used (see Table 2.4 and Table 2.5). The improved accuracy of GIF/RK methods is due to their higher stage order (see [46]). However, as it was pointed out in [36, Article 5], it seems that the accuracy comes at the price of stability. Thus, extending the idea of general linear methods with IRKS property to the exponential settings, might be a reasonable way to overcome this difficulty. Finally, we mark that the generalized integrating factor Runge–Kutta methods are also Lie group methods on the manifold. We consider the framework of Lie group methods and how it is related with the construction of exponential integrators in the next chapter.
43
Chapter 2
Exponential Integrators and Lie Group Methods In this chapter we discuss special types of exponential integrators, which arise from the framework of Lie group methods on manifolds. The main idea behind Lie group methods and Geometric Integration in general, is to construct numerical integrators which preserve certain qualitatively properties of the exact flow of a differential equation. A classical example is a differential equation evolving on a sphere. In many cases it is important to construct a numerical integrator which produces an approximation to the exact solution which also belongs to the sphere. When the differential equation evolves on a linear space, it is easy to construct an integrator which stays on the linear space. The challenge now is how to define the basic motions on the manifold in such a way that they provide a good approximation to the flow of the original vector field. In this case the theory of the Lie group integrators provides us, through the freedom in choice of a group action, a suitable framework to work with. All the exponential integrators presented in this chapter are motivated from the above observation. The chapter is organized as follows: we survey the basic theory involved in Section 2.1. In Section 2.2 we present different Lie group integrators and comment on their numerical implementation. Next, in Section 2.3 we discuss the importance of the Lie group action, and present different ways how to define it (see also [Article 1, p.91]). Finally in Section 2.4, based on the main Lie group methods given in Section 2.2, we derive the corresponding exponential integrators for solving the semilinear problem (1.1.1).
2.1
Background theory
Most of the theory presented in this section follows the expositions in [32, 36, 50] and gives the necessary basic background needed to construct Lie group methods which evolve on a manifold. We refer to the monographs [1, 44] for further information concerning manifolds, Lie groups and Lie algebras and to the monograph [23] for numerical treatment 44
and introduction to geometric integration in general. We start with introducing the concept of a manifold acted upon by Lie group, which provides us with abstract definition of the domain where the differential equation evolves. Let u0 = f (u(t)), u(t0 ) = u0 , (2.1.1) be a differential equation defined on d-dimensional topological space M which in a small neighborhood of any point “looks like” Rd but globally, typically has a different geometry. We will call such a space M a d-dimensional manifold. Any d-dimensional manifold can be represented as a d-dimensional surface embedded in RN for some N ≥ d. In other words M is defined as M = {x ∈ RN : g(x) = 0}, where g : RN → RN −d is differentiable and g 0 (x) has a full rank for x ∈ M. We note that this is a very concrete definition of a manifold, but is however sufficient for our purposes. In fact all the exponential integrators presented in Section 2.4 simply evolve on a manifold M ≡ Rd . A more general definition of a manifold, based on coordinate charts, can also be given (see [1, 36, 50]). We now give a definition of a Lie group, its tangent space, and discuss different maps related with them. Definition 2.1.1. A Lie group is a differential manifold G equipped with a group product ? : G × G → G satisfying
g ? (k ? l) = (g ? k) ? l
∀ g, k, l ∈ G,
(associativity)
∃ e ∈ G such that e ? g = g ? e = g,
(identity element)
∀ g ∈ G ∃ g −1 ∈ G such that g −1 g = e, the maps(g, k) → g ? k and g →
g −1
(inverse) (smoothness)
are smooth functions.
Definition 2.1.2. A Lie algebra is a vector space g equipped with a bilinear bracket [·, ·] : g × g → g satisfying [Θ1 , Θ2 ] = −[Θ2 , Θ1 ],
(skew-symmetry)
[Θ1 , [Θ2 , Θ3 ]] + [Θ2 , [Θ3 , Θ1 ]] + [Θ3 , [Θ1 , Θ2 ]] = 0,
(Jacobi identity)
for all Θ1 , Θ2 , Θ3 ∈ g.
b ∈ g then the map If we fix the first argument in the bracket [·, ·] to be the point Θ b adΘ b : g → g given by adΘ b (Θ) = [Θ, Θ] is linear. Powers of adΘ b are defined recursively as ad0Θ b (Θ) = Θ,
b ad1Θ b (Θ) = [Θ, Θ],
k−1 b b adkΘ b (ad b Θ) = [Θ, [. . . , [Θ, Θ]]], b (Θ) = adΘ Θ
45
for k > 1.
Let us denote the set of all right invariant vector fields on G by X(G). It can be shown that X(G) has a structure of a Lie algebra with a bracket given by the (minus) Jacobi bracket (see [36, 50]). Let Te G be the tangent space of G at the identity element e. We define the product ¯ : Te G × G → T G by ¯ d ¯¯ Θ ¯ g = ¯ γ(t) ? g, dt t=0
where γ(t) is a smooth curve in G such that γ(0) = e and γ 0 (0) = Θ. If we fix the first argument Θ ∈ Te G in the product Θ ¯ g, we get a vector field fΘ (·) = Θ ¯ (·) on G. In this way the map Θ → fΘ is a homomorphism between Te G and X(G). This automatically implies that the set Te G also has the structure of a Lie algebra. From now on we use the symbol g to denote the Lie algebra Te G and say that g is a Lie algebra of a Lie group G. The bracket in this case is defined by ¯ ∂ 2 ¯¯ [Θ1 , Θ2 ] = γ1 (s) ? γ2 (t) ? γ1 (s)−1 , ∂s∂t ¯t=s=0
where γ1 (s) and γ2 (t) are smooth curves in G such that γ1 (0) = γ2 (0) = e and γ10 (0) = Θ1 , γ20 (0) = Θ2 . In the case of a matrix Lie algebra g, the bracket [Θ1 , Θ2 ] is simply the matrix commutator Θ1 Θ2 − Θ2 Θ1 . Now we define the exponential map which gives an important connection between a Lie group and its Lie algebra. It is a local diffeomorphism (for finite dimensions) from a neighborhood of 0 ∈ g onto a neighborhood of e ∈ G. Definition 2.1.3. Let G be a Lie group and g its Lie algebra. The exponential map
Exp : g → G is defined as Exp(Θ) = γ(1), where γ(t) ∈ G satisfies the differential equation γ(t)0 = fΘ (γ(t)),
γ(0) = e.
In the case of a matrix Lie algebra g, the exponential map is simply the matrix exponential ∞ X 1 k Θ Θ . e = k! k=0
Definition 2.1.4. The differential of the exponential map is defined as the right trivialized tangent of the exponential map, that is, a map dExp : g×g → g defined through the relation ¯ d ¯¯ b b b + tΘ). dExp(Θ, Θ) Exp(Θ) = ¯ Exp(Θ dt t=0
b ∈ g then the map dExp b : g → g given by dExp b (Θ) = If we fix the first argument Θ Θ Θ b Θ) is linear and satisfies the following theorem. dExp(Θ, 46
Theorem 2.1.1. [50, Theorem 3] The differential of the exponential map and its inverse are given by the following formulas dExpΘ b = dExp−1 = b Θ
¯ ∞ X 1 ez − 1 ¯¯ adkΘ = b, ¯ z (k + 1)! z=adΘ b k=0 ¯ ∞ X Bk k z ¯¯ = ad b , ¯ z e − 1 z=ad b k! Θ Θ
k=0
1 1 where the coefficients Bk are the Bernoulli numbers {1, − 21 , 16 , 0, − 30 , 0, 42 , . . .}.
A crucial point in the theory of Lie group integrator is to define the basic motions on the manifold M. Typically they are given by the Lie group G and its action · on M. All numerical schemes presented in Section 2.2, advance by following flows defined with respect to the basic motions on M in such a way that, it is guaranteed that we stay on the manifold. Definition 2.1.5. An action of a Lie group G on a manifold M is a smooth map
· : G × M → M satisfying
e · p = p ∀ p ∈ M, g · (k · p) = (g ? k) · p ∀ g, k ∈ G, p ∈ M. We say that the group action is transitive if for any two points p1 , p2 ∈ M, there exists at least one element g ∈ G such that g · p1 = p2 . This allows us to move from any point p1 ∈ M to any other point p2 ∈ M, by letting the element g ∈ G, act on the first point p1 . A manifold M which is acted upon by a Lie group via a transitive action is called a homogeneous space. Via the exponential map, every group action naturally defines an algebra action ∗ : g × M → M, by the identity Θ ∗ p = Exp(Θ) · p.
(2.1.2)
Note that the algebra action is not uniquely determined by the group action. Every diffeomorphism Ψ : g → G, (2.1.3) such that Ψ(0) = e and Ψ0 (0) = I, where I is the identity of the algebra, defines an algebra action by the formula Θ ∗ p = Ψ(Θ) · p. Let X(M) denote the set of all vector fields on M. There is a one to one correspondence between the elements in the Lie algebra g and the space X(M). This can be shown in the same way as the correspondence between g and X(G). If we define the product ~ : g × M → T M by ¯ d¯ Θ ~ p = ¯¯ γ(t) · p, dt t=0 47
where γ(t) is a smooth curve in G such that γ(0) = e and γ 0 (0) = Θ, then for a fix Θ ∈ g the product Θ ~ p gives a vector field FΘ (·) = Θ ~ (·) on M. In this way, the map Θ → FΘ is an algebra homomorphism between g and X(M). The vector field FΘ is called a frozen vector field. The following relation between the frozen vector field and the exponential map holds. Theorem 2.1.2. For every Θ ∈ g and every p ∈ M ¯ d ¯¯ FΘ (p) = Θ ~ p = ¯ Exp(tΘ) · p. dt t=0
Proof. By definition
Θ~p = =
¯ ¯ d ¯¯ d ¯¯ γ(t) · p = ¯ (e + tΘ + O(t2 )) · p dt ¯t=0 dt t=0 ¯ ¯ ¯ d¯ d ¯¯ tΘ ∗ p = ¯ Exp(tΘ) · p. dt ¯t=0 dt t=0
The last Theorem is still valid if we replace the exponential map with the diffeomorphism Ψ : g → G defined above. This shows that the algebra homomorphism between g and X(M) is independent of the choice of Ψ. We will use this fact later in the construction of our exponential integrators. Finally, we give the following fundamental result, which is a starting point for every Lie group integrator. Theorem 2.1.3. [36] Every differential equation evolving on a homogeneous space M can
always be written as
u0 (t) = F (u) ~ u,
u(t0 ) = u0 ,
(2.1.4)
where F : M → g. Before we continue with the presentation of the main Lie group integrators, we summarize in Table 2.1 all the actions and homeomorphisms which has been defined so far. We note that in the literature authors often use the symbol “·” to denote each of the operations given in Table 2.1 and let its meaning to be determinate by the arguments. However, we find this approach a bit confusing, so we have chosen different notations for each operation.
2.2
Lie group integrators
All algorithms presented in this section are stated in terms of actions and reduce to traditional Runge–Kutta methods when the manifold M is the vector space Rd . The numerical 48
notation ? · ∗ ¯ ~
diagram G×G →G G×M→M g×M→M g × G → TG g × M → TM
name group operation group action algebra action homeomorphism between g and X(G) homeomorphism between g and X(M)
Table 2.1: Actions and homeomorphisms on the manifold. schemes advance from the point un to the point un+1 with a time step, of size h. The same results can also be formulated in terms of rigid frames (see [12, 15, 55]). The first Lie group integrators we consider are the methods of Crouch and Grossman [15]. The main idea behind these methods is to compose the flows of frozen vector fields at every stage. An s- stage Crouch and Grossman method is formulated by the following algorithm Algotithm 2.2.1. (Crouch–Grossman method) for i = 1, . . . , s do Ui = Exp(hαis Fs ) · · · Exp(hαi1 F1 ) · un
Fi = F (Ui ) end
un+1 = Exp(hβs Fs ) · · · Exp(hβ1 F1 ) · un The function F gives the generic presentation (2.1.4) of the differential equation. The coefficients αij , βj are parameters of the method. If αij = 0 for i ≤ j the method is explicit and is implicit otherwise. The order of the method is defined like for the traditional Runge– Kutta methods. The parameters αij , βj are chosen to satisfy certain order conditions which, because of the non-commutativity of the basic flows, are different from the order conditions for Runge–Kutta methods. In general one can not hope that an order p Runge–Kutta method will lead to an order p Crouch–Grossman method. A complete order theory for this methods involving ordered rooted trees is developed in [55]. There the authors propose the following explicit method of order three 0 3 4 17 24
3 4 119 216 13 51
17 108 − 23
.
(2.2.1)
24 17
By careful investigation of the order conditions, the authors in [55] found that it is not possible to obtain explicit methods of order four with only four stages. Further, they 49
derive a family of fourth order methods with five stages. Higher order methods of Crouch– Grossman type are derived in [33]. An exponential integrator based on the scheme (2.2.1), for the semilinear problem (1.1.1), is given in Section 2.4. We next consider a generalization of the methods first proposed in [48] and later named as Runge–Kutta Munthe-Kaas (RKMK) methods. In their original formulation these methods achieve only second order on a general Lie group. To construct higher order RKMK methods, a correction function was introduced in [49]. Here we follow the formulation of the methods on homogeneous manifolds given in [50]. The main idea behind these methods is to transform the original equation evolving on a manifold into a corresponding equation on a Lie algebra. Since the Lie algebra is a linear space and every Runge–Kutta method in general preserves linear invariants, one can apply any traditional Runge–Kutta method to the transformed equation. Thus, it is guaranteed that the numerical approximation will again be in the Lie algebra. To obtain the desired approximation on the manifold one just needs to apply the inverse transformation. An sstage RKMK method is formulated by the following algorithm Algotithm 2.2.2. (Runge–Kutta Munthe-Kaas method) for i = 1, . . . , s do P Θi = h sj=1 αij Kj
Fi = F (Ψ(Θi ) · un )
Ki = dΨ−1 (Fi )
end un+1 = Ψ(h
Ps
i=1 βi Ki )
· un
Here F gives the generic presentation (2.1.4), Ψ is a diffeomorphism (2.1.3) and dΨ is defined as in Definition 2.1.4. The coefficients αij , βj are the classical Runge–Kutta coefficients. Note that each of the quantities Θi , Fi , Ki ∈ g for i = 1, . . . , s. The values of the internal stages, which lie on the manifold, can be computed by letting the elements Θi ∈ g, act on the point un ∈ M. In other words Ui = Θi ∗ un , for i = 1, . . . , s. A natural choice for the diffeomorphism Ψ is the Exp map, but other choices are also possible. For example choosing Ψ to be an approximation to the Exp map can lead to a significant reduction in the overall computational cost of the algorithm. In Section 2.4 we present different choices of Ψ and relate them with some of the exponential integrators given in [36, Articles 4 and 5]. The main challenge in the implementation of RKMK methods comes form the necessity of computing the dΨ−1 map. This in general can be a very costly task, especially in the case when Ψ ≡ Exp map. Thus, the approach proposed in [50] is to replace dExp −1 with its truncated approximation of order higher than the order of the method. This results in introducing commutators into the format of the method. The last methods under consideration are the Commutator Free (CF) Lie group methods proposed in [12]. They are based on the idea of operating on the elements in the algebra 50
g, like for RKMK methods, and then composing the corresponding resulting flows of frozen vector fields, in a Crouch–Grossman like manner. In this way it is possible to construct an integrator which does not involve any commutators and still uses less Exp evaluations than in the method of Crouch–Grossman. A general format of an s-stage commutator free Lie group methods is given in the following algorithm Algotithm 2.2.3. (Commutator-free Lie group method) for i = 1, . . . , s do P Ps k F ) · · · Exp(h k Ui = Exp(h sk=1 αiJ k k=1 αi1 Fk ) · un Fi = F (Ui )
end un+1 = Exp(h
Ps
k k=1 βJ Fk ) · · · Exp(h
Ps
k k=1 β1 Fk )
· un
k , β k are paramHere again F gives the generic presentation (2.1.4) and the coefficients α ij j eters of the method. They are determined based on the order theory which can be adapted k = 0 for from the order theory presented in [55]. In general the method is implicit unless α ij i ≤ k then it is explicit. The parameter J counts the number of Exp evaluations for each stage. It is preferable to keep it as low as possible in order to obtain competitive methods. Another way of reducing the computational cost is by reusing the flow calculation. The following explicit fourth order method, based on the classical fourth order Runge–Kutta method (1.3.5), which effectively uses only 5 Exp per step is given in [12]
0 1 2 1 2 1 2 1 2
1 2
0
1 2
1 2 - 12
0
1 4 1 12
1 6 1 6
0 1
0
1 6 1 6
. 1 - 12 1
(2.2.2)
4
We use the symbol } to denote all the substages included in a stage with J > 1. We close this section with the observation that in order to implement Algorithm 2.2.1 and Algorithm 2.2.3 the only two things which we need to know are the generic presentation (2.1.4) of the differential equation and the algebra action (2.1.2) on the manifold M. To implement Algorithm 2.2.2, in addition, we need to know either the dΨ−1 map or how the commutators between two elements in g are defined. This observation allows us to construct Lie group integrators without even knowing what is the exact structure of the Lie group G and how it acts on M. This provides us the freedom to define the Lie algebra g independently from the Lie group G. For example, like a Lie algebra of vector fields on 51
M (see [51]). This approach is very useful for applications concerning PDEs, where the Lie algebra is infinite dimensional and establishing the structure of a Lie group G = Exp(g) is technically more demanding then in the finite dimensional case (see [32]). A way of circumventing this problem is to semidiscretize the PDE in space and thus obtain a finitedimensional system. In the next section we utilize this approach and discuss the choice of action as well as the structure of G, in the case when M ≡ Rd .
2.3
The choice of action
When the differential equation evolves on Rd constructing an integrator which stays in Rd is a trivial task and does not require a special theory. However, using the framework of Lie group methods, one can construct useful methods specially designed to solve the equation (2.1.1). In this section we pay particular attention to the choice of group action · and how it is related to the actual performance of the method. We first consider the case when the Lie group G ≡ Rd and its action upon M is given by the vector addition. In other words g · p = g + p,
for all g ∈ G and p ∈ M.
(2.3.1)
From Definition 2.1.1, it follows that the group operation ? is also given by vector addition and that the identity element e is the zero vector in Rd . The Lie algebra g = Te G can be identified with Rd and the algebra homeomorphism between g and X(G) defined by ¯ ¯ d ¯¯ d ¯¯ Θ ¯ g = ¯ γ(t) ? g = ¯ g + tΘ + O(t2 ) = Θ, dt t=0 dt t=0
is simply the identity map. This implies that the exponential map and the homeomorphism between g and X(M) are also given by the identity map. The generic presentation (2.1.4) of the differential equation is u0 = F (u) ~ u = F (u).
If b ∈ Rd is a fixed element in g then its corresponding frozen vector field Fb (p) = b for all points p on the manifold. The Lie group integrators presented in Section 2.2 use the flow of a frozen vector field, as the basic motions on M. This means that the solution of the differential equation u0 = b, u(t0 ) = u0 , which is given by translation u(t0 + h) = u0 + hb, defines an algebra action hb ∗ u0 . Since translations commute, we obtain the trivial bracket [a, b] = 0, for all a, b ∈ g. Note that via the relation (2.1.2) the above algebra action reproduces the group action (2.3.1). This choice simply reduces all the methods presented in Section 2.2 to the traditional Runge– Kutta methods. Like it was mentioned in the previous section, in order to construct Lie group integrators it is enough to know the algebra action and the generic presentation of the differential 52
equation. In the subsequent, discussions which concern the structure of the Lie group and its action on M are kept with the scope of completeness. A more interesting choice for the algebra action, rather than translations, is to move from the point u0 to another point by following the flow of a linear vector field u0 = Au,
u(t0 ) = u0 ,
where A ∈ Rd×d is a constant matrix. In this case the Lie group G is the general linear group GL(d) = {G ∈ Rd×d : detG 6= 0}, its Lie algebra gl(d) is the set of all d × d matrices and the bracket is given by the matrix commutator [A, B] = AB − BA. The only problem here is that the action is not transitive. This is due to the fact that there is no way to advance from the point u0 = 0. A way to avoid this inconvenience is to consider the action arising from the solution of the following differential equation u0 = Au + b,
u(t0 ) = u0 ,
(2.3.2)
where A ∈ Rd×d and b ∈ Rd are constants. This idea was first introduced in [50] and then further investigated for the heat equation in [12, 39, 63], for stiff PDEs in [36, Articles 4 and 5] and for the Schr¨odinger equation in [3]. The action arising from (2.3.2) is often referred to as the affine action. The Lie group G in this case can be identified with the set of all pairs (G, g) ∈ GL(d) × Rd which we denote by the semidirect product GL(d) o Rd (see [67]). Its action on the manifold is defined by (G, g) · p = Gp + g
for all (G, g) ∈ G and p ∈ M.
From Definition 2.1.1, it follows that the group operation ? is given by (G1 , g1 ) ? (G2 , g2 ) = (G1 G2 , G1 g2 + g1 ) and that the identity element e = (I, o), where I is the d × d identity matrix and o is the zero vector in Rd . The Lie algebra g = Te G is the set of all pairs (A, b) ∈ gl(d) o Rd and the bracket is defined as £ ¤ (A1 , b1 ), (A2 , b2 ) = (A1 A2 − A2 A1 , A1 b2 − A2 b1 ). (2.3.3) ¡ ¢ The algebra homeomorphism between g and X(G) is given by (A, b) → f(A,b) (·, ·) , where ¯ ¡ ¢ d ¯¯ f(A,b) (G, g) = (A, b) ¯ (G, g) = ¯ γ(t) ? (G, g) = (AG, Ag + b). dt t=0
If γ(t) is a smooth curve in G such that γ(0) = e and γ 0 (0) = (A, b), then the solution of the differential equation γ 0 (t) = f(A,b) (γ(t)), γ(0) = e,
is given by γ(t) = (etA , φ[1] (tA)tb), where the function φ[1] is the first ETD function from (1.1.6). This implies that the exponential map is defined as Exp(A, b) = (eA , φ[1] (A)b). 53
(2.3.4)
Therefore, the algebra action is ¡ ¢ h(A, b) ∗ u0 = Exp h(A, b) · u0 = ehA u0 + φ[1] (hA) hb,
(2.3.5)
which is exactly the solution of the differential equation (2.3.2) at the time t0 + h. Finally, the frozen vector field is given by F(A,b) (u) = (A, b) ~ u = Au + b. An important observation concerning the affine action is that the generic presentation (2.1.4) of a differential equation (2.1.1) is not uniquely defined. For example, if we choose F (u) = (0, f (u)) then we will recover exactly the classical settings considered in the beginning of this section. Another possible choice is F (u) = (J, f (u) − Ju), where J is the Jacobian of f at the point u. In this case the RKMK method based on forward Euler is given by un+1 = un + J −1 (ehJ − I)f (u). It is a second order method which is often referred to as the exponential fitted Euler method [28]. When the vector field of the differential equation is given in nonautonomous form, like in (1.1.1), we need to transform it to autonomous form by appending t to the dependent variables. This increases the dimension of the manifold and allow us to include the time variable to the arguments of the function F . Such an approach is very useful, especially if we want to construct time dependent frozen vector fields (see [Article 1, p.91]). A natural choice for F , which reflects the semilinear structure of (1.1.1), is F (u, t) = (L, N (u, t)).
(2.3.6)
Methods based on this generic presentation for the equation (1.1.1) are given in the next section. More complicated differential equations, which hopefully approximate the original vector field in a better way, can also be used to define the basic motions on M. The main requirement is that we should be able to solve them exactly and thus define the algebra action like the flow of some differential equation. A way how to construct Lie group integrators based on the action arising from the solution of the following differential equation u0 = Au + b + ct,
u(t0 ) = u0 ,
(2.3.7)
where A ∈ Rd×d , b ∈ Rd and c ∈ Rd are constants, is proposed in [Article 1, p.91]. There the corresponding exponential integrator for solving the semilinear problem (1.1.1), based on the fourth order commutator free Lie group method (2.2.2), is also derived. The main idea is to rewrite the above equations in autonomous form and then apply the affine action to the transformed equation. This approach can be further generalized by including second and higher order terms of t in the frozen vector field. However, one should keep in mind 54
that there is a certain balance between the cost of finding the exponential of (2.3.7) and the benefit provided form choosing a better approximation. We remark that in any case the corresponding algebra homeomorphism ~ will always be the same as in the case of the affine action. This means that for a given problem by choosing the right algebra action (for example, the tangent to the real flow) one can improve the actual performance of the method and even increase its order like in the case of the exponentially fitted Euler method. More discussions about the role of the algebra action are also included in the next section.
2.4
Lie group integrators for semilinear problems
Here we present different exponential integrators for solving the semilinear problem (1.1.1), which are based on the methods considered in Section 2.2 and used the affine algebra action introduced in Section 2.3. Most of the methods included in this section are derived in [36, Articles 4 and 5]. We give their equivalent formulations, which are in accordance with the presentation followed in this thesis, and discuss the connection between the Generalized Integrating Factor Runge–Kutta (GIF/RK) methods introduced in [36, Articles 5] and the RKMK methods. In addition, we propose a new approach regarding the derivation of GIF/RK methods which is applicable to problems where the nonlinear part of (1.1.1) is better approximated by trigonometric polynomials. Before we consider the first numerical schemes, we make the following assumptions. Let h be a constant step size and u1 , u2 , . . . , un are approximations of the solution of (1.1.1) at the moments tj = t0 + jh for j = 1, . . . , n. We are looking for an approximation to the solution at time tn+1 = t0 + (n + 1)h. In other words we consider the following problem u0 = Lu + N (u, tn + t) = f (u, tn + t),
u(tn ) = un .
(2.4.1)
Note that, in general, we allow our numerical schemes to have incoming data consisting not only from the quantity un but also from some of the quantities available from the previous steps. Thus, the resulting methods are exponential general linear methods introduced in Section 1.3. All methods presented in this section are based on the pure Runge–Kutta idea and use only un as the incoming data at the beginning of step number n + 1. The only exceptions are the GIF/RK methods, which are exponential general linear methods. Other examples of Lie group methods which are also exponential general linear methods are the methods based on the action presented in [Article 1, p.91]. It is rather surprising that with the framework of Lie group integrators, which we saw in Section 2.2 are entirely based on the Runge–Kutta idea, one can get more general methods like general linear methods. This can be explained by the crucial role of the algebra action in the overall performance of the method. In the subsequent work we have adopted the notations from Section 1.3. In addition, we introduce the following convention: Unless it is not specified the arguments in the row 55
number i of the functions φ[j] (from Lemma 1.1.1) and φb[j] (to be defined below) are ci hL, for i = 1, 2, . . . , s and j = 1, 2, . . ..
2.4.1
Crouch–Grossman based methods
Keeping in mind (2.3.5) and (2.3.6) the derivation of an exponential integrator based on Algorithm 2.2.1 is a straight forward process. Our first example is a method which corresponds to the scheme (2.2.1).
0 3 [1] 4φ
0 0
0 0
e
17 hL [1] 119 119 108 φ ( 216 hL) 216 e
17 [1] 17 108 φ ( 108 hL)
0
e 24 hL
hL [1] 13 13 38 51 φ ( 51 hL) 51 e
− 23 e 17 hL φ[1] (− 32 hL)
24 [1] 24 17 φ ( 17 hL)
ehL
24
I 3 hL 4
17
Table 2.2: Third order Crouch–Grossman method.
2.4.2
RKMK based methods
Every method based on Algorithm 2.2.2 requires either the exact dΨ−1 map or its approximation which involves commutators between two elements in g. In the case of the semilinear problem (2.4.1) the structure of the Lie algebra g is simpler than the general structure presented in Section 2.3. In fact, since the linear part L stays constant, the Lie algebra g can be identified with all pairs (λL, b) , where λ ∈ R and b ∈ Rd . This allows us to compute the dΨ−1 map exactly and relatively cheaply. The bracket (2.3.3) in this case is given by £
¤ b bb), (λL, b) = (O, L(λb b − λbb)), (λL,
where O denotes the zero matrix in Rd×d .
RKMK methods with exact Exp map Let us first consider the case when Ψ ≡ Exp map. From (2.3.4), it follows that the exponential map in this case is Exp(λL, b) = (eλL , φ[1] (λL)b). It is easy to check that bk−1 Lk (λb b − λbb)), adk(λL, (λL, b) = (O, λ b b b) 56
for k ≥ 1
and thus the expression for dExp−1 has the form dExp−1 b
b (λL, b) =
(λL,b)
∞ X Bk k=0
k!
adk(λL, (λL, b) b b b)
µ ³ −1 λ ´ b = λL, φ[1] (λL) b − bb + b λ
¶ λb b . b λ
(2.4.2)
b = 0 the dExp−1 is equal to the identity map in g. When λ We know that every RKMK method is equivalent to a traditional Runge–Kutta method applied to the corresponding differential equation in the Lie algebra g, followed by the inverse transformation of the approximate solution back to the manifold (see Section 2.2). In order to find the differential equation in g we choose Θ(t) = (tL, z(t)) to be a curve in g such that Θ(0) = (O, o) and the solution of (2.4.1) is given by u(tn + t) = Exp(Θ(t)) · un = etL un + tφ[1] (tL)z(t).
(2.4.3)
By differentiating (2.4.3) and taking into account (2.4.2) and the generic presentation (2.3.6), we obtain the following corresponding differential equation for z(t) ¡ ¢ −1 z 0 (t) = φ[1] (tL) N (Exp(tL, z) · un , tn + t) − z/t + z/t, z(0) = o. (2.4.4)
Thus we see that a single step of a RKMK method for the equation (2.4.1) is equivalent to one step of a traditional Runge–Kutta (RK) method for the equation (2.4.4), followed by computing u(tn + t) from the approximation z(t) via (2.4.3). In Table 2.3, we give the RKMK method based on the classical fourth order Runge–Kutta method (1.3.5) with exact Exp map. The above approach reminds us to the technique used in the construction of the Integrating Factor Runge–Kutta (IF RK) methods presented in Section 1.2. As a matter of fact there exists a connection between the IF RK and the RKMK methods in a sense that every IF RK method is simply a RKMK method with a special choice for the diffeomorphism Ψ. This fact was first observed in [36, Article 4]. Based on a similar idea a class of methods, which can be reasonably named as Generalized Integrating Factor Runge–Kutta (GIF/RK) methods, was derived in [36, Article 5]. Next we discuss this class of methods and show that they also can be brought into the framework of RKMK methods. RKMK methods with approximations to the Exp map Let us chose the diffeomorphism Ψ to be the following approximation to the Exp map Ψ(λL, b) = (eλL , eλL b).
(2.4.5)
In this case, by direct computations, it is possible to find the following closed form for the dΨ−1 map (see [36, Article 4]) dΨ−1 b
(λL, b) (λL,b b)
b
= (λL, e−λL b).
57
0 1 [1] 2φ
0 0
0 0
1 φ[1] − 1 I 1 0 2 2 2I b[3]2 hL φ ( 2 ) −φb[2] φb[3] ( hL φb[2] 2 ) b1 (hL) b2 (hL) b3 (hL)
I
0 0
e
0
e 2 hL
0
ehL
1 6I
ehL
1 hL 2 1
,
where ³ ´ e z2 + I [2] [1] [1] −1 z b = , φ (z) = φ (z)φ 2 2
ez − z − I −1 φb[3] (z) = φ[1] (z) − I = , I − ez ´ ³ ´ 2 hL 1 [1] 1 [1] b[3] ³ hL ´ 1 [1] ³ b[3] b1 (hL) = φ − φ φ − φ φ − 2I φb[3] , 6 3 2 6 2 ´ ³ hL ´ 1 b[2] 1 b[2] ³ b[3] b2 (hL) = , φ + φ φ − 2I φb[3] 3 6 2 b3 (hL) =
1 b[2] 1 b[2] b[3] φ − φ φ . 3 6
Table 2.3: Fourth order RKMK method with exact Exp. Thus, again, if Θ(t) = (tL, z(t)) is a curve in g such that Θ(0) = (O, o) and u(tn + t) = Ψ(Θ(t)) · un = etL (un + z(t)),
(2.4.6)
then the corresponding differential equation for z(t) is ¡ ¢ z 0 (t) = e−tL N Ψ(tL, z) · un , tn + t ,
z(0) = o.
If we put v(t) = un + z(t) then (2.4.6) is simply the Lawson transformation (see Subsection 1.1.1) and the corresponding differential equation for v(t) has the familiar form v 0 (t) = e−tL N (etL v, tn + t),
v(0) = un .
In this way we see that IF RK methods are simply RKMK methods with Ψ given by (2.4.5). General formulation of an IF RK method is given in Table 1.3. Let us now define the diffeomorphism Ψ to be Ψ(λL, b) = (eλL , eλL (b − λNn ) + λφ[1] (λL)Nn ), 58
where Nn = N (un , tn ). Note that at the beginning of step number n + 1 the value un is already known and therefore we can easily compute Nn . In this case
and by setting
dΨ−1 b
(λL, b) (λL,b b)
b
= (λL, e−λL (b − λNn ) + λNn ),
u(tn + t) = Ψ(Θ(t)) · un = etL (un + z(t) − tNn ) + tφ[1] (tL)Nn ,
(2.4.7)
we obtain the following differential equation for z(t) ¢ ¡ z 0 (t) = e−tL N (Ψ(tL, z) · un , tn + t) − Nn + Nn ,
z(0) = o.
(2.4.8)
If we substitute in this case v(t) = un + z(t) − tNn then (2.4.7) has the form u(tn + t) = etL v(t) + tφ[1] (tL)Nn ,
(2.4.9)
and the corresponding equation for v(t) is ¢ ¡ v 0 (t) = e−tL N (u(tn + t), tn + t) − Nn ,
v(0) = un .
(2.4.10)
Thus, applying any traditional Runge–Kutta method to the equation (2.4.8) or (2.4.10) and then transforming back the result into the manifold by (2.4.7) or (2.4.9) respectively, is a RKMK method which we denote by GIF1/RK. This in fact is the first of the generalized integrating factor Runge–Kutta methods proposed in [36, Article 5]. In Table 2.4, we give the general formulation of a fourth order GIF1/RK method which corresponds to a RK method with coefficients A = (αij ), b = (βi ), c = (ci ).
0 c2 φ[1]
0 0
0 0
0 0
I
ec2 hL ec3 hL c3 φ[1] − α32 e(c3 −c2 )hL α32 e(c3 −c2 )hL 0 0 c φ[1] − α e(c4 −c2 )hL − α e(c4 −c3 )hL α e(c4 −c2 )hL α e(c4 −c3 )hL ec4 hL 0 42 43 42 43 4 b1 (hL) β2 e(1−c2 )hL β3 e(1−c3 )hL β4 e(1−c4 )hL ehL where b1 (hL) = φ[1] − β2 e(1−c2 )hL − β3 e(1−c3 )hL − β4 e(1−c4 )hL .
,
Table 2.4: Fourth order GIF1/RK method If we allow our self to choose an approximation to the Exp map which at the beginning of step number n + 1 uses not only the value Nn , but also and the value of Nn−1 from the 59
end of the previous step with size h, we can recover the second of the generalized integrating factor Runge–Kutta methods from [36, Article 5]. Indeed if ³ λ2 Nn − Nn−1 Nn − Nn−1 ´ Ψ(λL, b) = eλL , eλL (b − λNn − ) + λφ[1] (λL)Nn + λ2 φ[2] (λL) , 2 h h then ´´ ³ ´´ ³ ³ ³ b −λL b Nn − Nn−1 . b Nn − Nn−1 dΨ−1 + λ N − λ (λL, b) = λL, e b − λ N − λ n n b b (λL, b) h h
Substituting
u(tn + t) = Ψ(Θ(t)) · un = etL v(t) + tφ[1] (tL)Nn + t2 φ[2] (tL) with v(t) = un + z(t) − tNn −
t2 Nn −Nn−1 , 2 h
Nn − Nn−1 , h
(2.4.11)
we obtain the following equation for v(t)
¡ Nn − Nn−1 ¢ v 0 (t) = e−tL N (u(tn + t), tn + t) − Nn − t , h
v(0) = un .
Its numerical solution by a traditional RK method, followed by the transformation (2.4.11), gives the second of the GIF/RK methods which we denote by GIF2/RK. In Table 2.5, we give the general formulation of a fourth order GIF2/RK method, which again corresponds to a RK method with coefficients A = (αij ), b = (βi ), c = (ci ). Note that GIF2/RK methods are example of exponential general linear methods (see Section 1.3). This process can be further continued and in general one can show that the GIF/RK methods constructed in [36, Article 5] are also RKMK methods. The prove of this fact is a straight forward extension of the above process. We note that for the semilinear problem (2.4.1) all RKMK methods, based on the affine group action, are simply RK methods applied to the transformed equation after some change of the variables and then the approximate solution is transformed back into the original variables. In this way, it is quite likely for any method based on this idea to be a RKMK method with appropriate choice for the diffeomorphism Ψ. The above presented choices for the diffeomorphism Ψ might looks some how strange and unfounded, but all becomes clear when we observe that the Lawson transformation (see Subsection 1.1.1) together with the transformations (2.4.9) and (2.4.11) fits into the form u(tn + t) = φt,Fb (v(t)),
where φt,Fb is the flow of the differential equation u0 = Fb(u, t),
u(0) = un .
(2.4.12)
(2.4.13)
The vector field Fb (u, t) approximates the original vector field f (u, tn + t) of (2.4.1) around the point un and Fb (un , 0) = f (un , tn ) = Lun + N (un , tn ). The corresponding differential equation for the v variable is easily obtained by differentiating (2.4.12) and it has the form µ ¶−1 ³ ´ ∂u 0 v (t) = f (u, tn + t) − Fb(u, t), , v(0) = un . (2.4.14) ∂v 60
where
0 0 0 0 I c a21 (hL) 2 e hL 0 0 0 a31 (hL) α32 e(c3 −c2 )hL 0 0 ec3 hL a41 (hL) α42 e(c4 −c2 )hL α43 e(c4 −c3 )hL 0 ec4 hL b1 (hL) β2 e(1−c2 )hL β3 e(1−c3 )hL β4 e(1−c4 )hL ehL I 0 0 0 0
0 d22 (hL) d32 (hL) , d42 (hL) v12 (hL) 0
a21 (hL) = c2 φ[1] + c22 φ[2] , a31 (hL) = c3 φ[1] + c23 φ[2] − α32 (1 + c2 ) e(c3 −c2 )hL ,
a41 (hL) = c4 φ[1] + c24 φ[2] − α42 (1 + c2 ) e(c4 −c2 )hL − α43 (1 + c3 ) e(c4 −c3 )hL ,
b1 (hL) = φ[1] + φ[2] − β2 (1 + c2 )e(1−c2 )hL − β3 (1 + c3 )e(1−c3 )hL − β4 (1 + c4 )e(1−c4 )hL ,
d22 (hL) = −c22 φ[2] ,
d32 (hL) = −c23 φ[2] + c2 α32 e(c3 −c2 )hL ,
d42 (hL) = −c24 φ[2] + c2 α42 e(c4 −c2 )hL + c3 α43 e(c4 −c3 )hL ,
v12 (hL) = −φ[2] + c2 β2 e(1−c2 )hL + c3 β3 e(1−c3 )hL + c4 β4 e(1−c4 )hL . Table 2.5: Fourth order GIF2/RK method
Now a single step with any RK method applied to the equation (2.4.14), followed by the transformation (2.4.12), will lead to a new method for the corresponding differential equation (2.4.1). This, in fact, is the original formulation of the generalized integrating factor Runge–Kutta methods given in [36, Article 5]. Note that we do not really need to know what is the differential equation in the z variable. It was introduced in the above presentation with the sole purpose of showing the exact correspondence between GIF/RK and RKMK methods. An important requirement for the vector field Fb , which follows from the formulation of the method, is that it should not only approximate the original vector field, but it should also have a flow φt,Fb which is easy to compute exactly. Similar methods, based on the idea of approximating the flow φt,Fb by some numerical scheme, are derived in [43].
For the semilinear problem (2.4.1), the choice of Fb proposed in [36, Article 5] is Fb(u, t) = Lu + Lk (N, t), where Lk (N, t) is the Lagrange interpolating polynomial of degree k − 1 for the function N (u(tn + t), tn + t) which passes through the k points Nn , Nn−1 , . . . , Nn−k+1 . Thus, the choice Fb(u, t) = Lu reproduces the IF RK methods and by letting Fb(u, t) be the k-th Lagrange interpolating polynomial, we obtain the k-th generalized integrating factor 61
Runge–Kutta method GIFk/RK. The first three polynomials Lk (N, t) for a constant step size h, which lead to GIF1/RK, GIF2/RK and GIF3/RK, are respectively L1 (N, t) = Nn , L2 (N, t) = Nn + t L3 (N, t) = Nn + t
µ
Ã
Nn − Nn−1 h 1 2 Nn−2
¶
,
− 2Nn−1 + 32 Nn h
!
+
t2 2
µ
Nn−2 − 2Nn−1 + Nn h2
The exact flow φt,Fb in this case is given by Lemma 1.1.1 and by observing that the equation (2.4.14) becomes ¡ ¢ v 0 (t) = e−tL N (u(tn + t), tn + t) − Lk (N, t) v(0) = un .
¶
.
∂φt,Fb (v) ∂v
= etL
We pay attention to the fact that in order to implement any of the GIFk/RK methods for k > 1, at the beginning of each step, we need to know the value of N and therefore of u not only at the grid point tn but also at the k − 1 previous grid points. Thus, in general, this methods require a starting procedure (see Subsection 1.3.2), which produces the needed values of u and N , before the integration process begins. Such a procedure can be given for example, by the first few steps of any of the other methods presented in this section. Methods which pass more than one quantity from step to step and still use internal stages approximations are examples of general linear methods (see Section 1.3). Thus, GIFk/RK methods for k > 1, are examples of exponential general linear methods. Another possible choice for the vector field Fb , which also satisfies the main requirements for constructing a GIF/RK methods is Fb (u, t) = Lu + Tk (N, t), where Tk (N, t) is a trigonometric polynomial approximating the function N (u(tn + t), tn + t). This choice of Fb, might be preferable when the nonlinear part N of the original differential equation is periodic and square-integrable. Next we briefly discuss methods based on this idea. GIF/RK methods for semilinear problems with periodic nonlinear part We start by giving the exact formula for the flow of a vector field Fb (u, t) = Lu + Tk (N, t), where k X Tk (N, t) = b + (cα sin(αt) + dα cos(αt)) ; k ∈ N, cα , dα ∈ R. α=1
Lemma 2.4.1. The solution of the differential equation u0 = Fb (u, t) = Lu + b +
at the moment tn + h is given by u(tn + h) = e
hL
[1]
k X
(cα sin(αt) + dα cos(αt)) ,
u(tn ) = un
α=1
un + hbφ (hL) +
k ³ X
α=1
´ [α] cα φsin (tn , h, L) + dα φ[α] cos (tn , h, L) , 62
(2.4.15)
where [α] φsin (tn , h, L)
=
φ[α] cos (tn , h, L) =
¡ ¢ ehL L sin(αtn ) + α cos(αtn ) − L sin(α(tn + h)) − α cos(α(tn + h))I , α 2 I + L2 ¡ ¢ ehL L cos(αtn ) − α sin(αtn ) − L cos(α(tn + h)) + α sin(α(tn + h))I . α 2 I + L2
Here the meaning of the operator in the denominator is simply given by applying its inverse to the numerator. Proof. From the variation of constants formulae, it follows that ! Ã Z tn +h k X (cα sin(ατ ) + dα cos(ατ )) dτ. u(tn + h) = ehL un + e(tn +h)L e−τ L b + tn
α=1
The solution of the first integral gives the second term in (2.4.15). It is the same as in the case when the approximation is based on Lagrange interpolating polynomial. Let us now consider the following integral X=
Z
tn +h
e−τ L sin(ατ )dτ.
tn
Integrating by parts twice leads to an equation for X. Multiplying its solution with e (tn +h)L gives the relation e(tn +h)L
Z
tn +h tn
[α]
e−τ L sin(ατ )dτ = φsin (tn , h, L).
In the same way it can be shown that e
(tn +h)L
Z
tn +h tn
e−τ L cos(ατ )dτ = φ[α] cos (tn , h, L).
From the above Lemma, it follows that the flow φt,Fb of (2.4.13) is simply given by [α]
[α]
(2.4.15) with tn = 0 in the expressions for φsin and φcos . The corresponding differential equation for v(t) is ¡ ¢ v 0 (t) = e−tL N (u(tn + t), tn + t) − Tk (N, t) ,
v(0) = un .
(2.4.16)
Now the construction of the method follows the same pattern as before. For example if −Nn−1 sin t, then we substitute N (u(tn + t), tn + t) ≈ Nn + Nnsin h [1]
u(tn + t) = etL v(t) + tφ[1] (tL)Nn + φsin (0, t, L) 63
Nn − Nn−1 , sin h
[α]
tL
sin t−cos I where φsin (0, t, L) = e −L . Applying a traditional RK method to (2.4.16) and then α2 I+L2 transforming the resut into the u variable leads to a new method which is again given by Table 2.5, but with the following new coefficients
[1] a21 (hL) = c2 φ[1] + φbsin , [1] a31 (hL) = c3 φ[1] + φb − α32 (1 + b c2 ) e(c3 −c2 )hL ,
a41 (hL) = b1 (hL) =
d22 (hL) = d32 (hL) = d42 (hL) = v12 (hL) =
sin [1] c4 φ + φbsin − α42 (1 + b c2 ) e(c4 −c2 )hL − α43 (1 + b c3 ) e(c4 −c3 )hL , [1] φ[1] + φbsin − β2 (1 + b c2 )e(1−c2 )hL − β3 (1 + b c3 )e(1−c3 )hL − β4 (1 + b c4 )e(1−c4 )hL , [1] −φbsin , [1] −φbsin + b c2 α32 e(c3 −c2 )hL , [1] −φbsin + b c2 α42 e(c4 −c2 )hL + b c3 α43 e(c4 −c3 )hL , [1] −φbsin + b c2 β2 e(1−c2 )hL + b c3 β3 e(1−c3 )hL + b c4 β4 e(1−c4 )hL , [1]
sin(c h) [1] [α] 1 where b cj = sin jh , for j = 1, . . . , 4, and φbsin (cj hL) = h sin h φsin (0, cj h, L). Note that in this case we have used again the convention from the beginning of this section. [α] [α] In Lemma 2.4.1 we give a more general formulation of the functions φsin and φcos than the one needed in the format of the method. The reason why is to show how the initial [α] [α] grid point tn enters into the functions φsin and φcos . Thus, it is clear that if one wants to construct methods based on the idea proposed in [Article 1, p.91] with algebra action arising [α] [α] from the solution of the differential equation u0 = Fb(u, t), then the values of φsin and φcos should be recompute at the beginning of each step. This increases the computational cost of such methods and they are likely to be competitive with the other methods presented in this section, only if a variable step size strategy is used (see also Chapter 3). Before we consider exponential integrators based on commutator free Lie group methods, we point out that the idea of generalized integrating factor methods can be further exploit. Instead of applying a Runge–Kutta method to the transformed equation (2.4.14) we can alternatively use a linear multistep method. Thus, it is possible to greatly increase the class of exponential linear multistep methods by constructing Generalized Integrating Factor Linear Multistep (GIF/LM) methods (see [46]). It is worth to mentioning that integrating factor Adams methods, ETD Adams-Bashforth and ETD Adams-Moulton methods, considered in Section 1.1, are all special cases of this general class of methods.
2.4.3
Methods based on commutator free Lie group methods
Finally, we consider methods based on Algorithm 2.2.3. Using the formula (2.3.5) for the algebra action and the definition (2.3.6) of the generic presentation of the differential equation (2.4.1), it is easy to observe that the corresponding exponential integrator is obtained form the underlying commutator free Lie group method by simply multiplying each of the 64
coefficients of the method with the function φ[1] . For example, the method based on the scheme (2.2.2) is given by
0
0
0
0
1 [1] 2φ
0
0
0
e
hL 2
0
1 [1] 2φ
0
0
e
hL 2
I
1 [1] 2φ - 21 φ[1]
0
0
0
e
hL 2
0
φ[1]
0
e
hL 2
1 [1] 4φ 1 [1] - 12 φ
1 [1] 6φ 1 [1] 6φ
1 [1] 6φ 1 [1] 6φ
1 [1] - 12 φ
e
hL 2
1 [1] 4φ
e
hL 2
,
(2.4.17)
where again we use the symbol } to denote all the substages included in a stage with J > 1. It is possible to rewrite (2.4.17) in an equivalent form which does not involve a splitting of the internal stages. In Table 2.6, we give such a presentation which is rather unpractical, since it is more expensive to implement. Note that in (2.4.17) the first substage of the fourth internal stage is the same as the second internal stage. This form of the method makes it particularly suitable for numerical implementation. We have included the other formulation as well in order to show that this method also fits into the framework of the “pure” RK methods.
0 1 [1] 2φ 1 2φ
0 ¡ ¢¡ [1] hL
1 [1] 2φ
2
hL 2
−I ¡ ¢ [1] hL
e
− 13 φ
0 0
2
¢
1 [1] 2φ
0 1 [1] 3φ
φ
0 0
0 0
0 ¢ ¡ [1] hL
0
2
1 [1] 3φ
I
0 − 61 φ[1] + 13 φ[1]
¡ hL ¢ 2
1 e 2 hL 1 e 2 hL hL e hL e
Table 2.6: Fourth order CF method. In conclusion, we briefly discuss accuracy, stability and minimizing the number of the function evaluations, for all of the above presented methods. For stiff semilinear problems (1.1.1) the RKMK methods with exact Exp suffer from a step size restriction. This is due to the small convergence radius of the dExp −1 map. Therefore this class of methods are unpractical for such problems. The GIF/RK methods have several orders in magnitude improvement in accuracy over the standard IF RK methods (see the numerical experiments Section 3.4), but also suffer from stability restrictions, especially for problems with eigenvalues of the linearization of the nonlinear part lying on the 65
imaginary axis (see [36, Article 5]). The most expensive methods are the Crouch–Grossman methods. Thus, combining the lower computational cost with the relatively good stability properties and accuracy for the commutator free Lie group methods, suggests that for stiff problems they are the best exponential integrators arising from the framework of Lie group methods. When the equation (1.1.1) comes from the discretization of a highly-oscillatory problems, for example the nonlinear Schr¨odinger equation, then the RKMK methods does not suffer so dramatically from numerical instability like in the previous case. The best Lie group method in this case seems to be the IF RK methods [3].
66
Chapter 3
Implementation Issues and Numerical Experiments In this chapter we address some practical issues regarding the implementation of the exponential integrators considered in the previous two chapters. We also present results from numerical experiments, with some of the discussed methods, for several numerical examples. The main computational challenge in the implementation of every exponential integrator comes from the necessity of computing the exponential and the related φ[l] functions (1.2.2) by some fast and numerically stable algorithm. A great variety of methods for computing the matrix exponential operator are available in the literature. We refer to [47] and the references within for a comprehensive review of these methods. In this chapter we focus on different numerical techniques for computing the most commonly used ETD φ[i] functions (1.1.6). We also discuss the main advantages and disadvantages of the considered methods as well as some practical issues concerning variable step size implementations. The chapter is organized as follows: We first consider, in Section 3.1, methods based on Schur decomposition followed by higher order rational Chebyshev or Pad´e approximations to the function φ[i] . Next, in Section 3.2 we present a technique, which allows us to approximate the action of each φ[i] function on a given vector by means of a projection process onto a small Krylov subspace. In Section 3.3 we discuss methods based on the Cauchy integral formula and comment its implementation in some special cases. Finally, in Section 3.4 we compare different exponential integrators in several numerical examples.
3.1
Decomposition methods
Let us recall the explicit form of the ETD φ[i] functions, which appear in the formulation of most of the exponential integrators presented so far φ[1] (z) =
ez − 1 , z
φ[i+1] (z) =
φ[i] (z) − z
67
1 i!
,
for
i = 2, 3, . . . .
(3.1.1)
Like we saw in the previous two chapters, other functions can also enter in the format of an exponential integrator. Examples are integrating factor RK methods and methods based on the the Lie group theory. However, in most of the known cases, the new functions are closely related with functions from (3.1.1). Therefore, here we restrict our considerations only to the ETD φ[i] functions. A straightforward implementation of (3.1.1) suffers, for small z, from cancellation errors. An illustration of this phenomena, for the first function φ[1] , is given in [35, Table 2.1]. The cancellation errors are even more severe for the second and the subsequent φ [i] functions. In practice, the arguments of the functions in (3.1.1) are matrices of the form A = γhL, where γ ∈ R, h is the step size and L is the discretized linear operator. The size d of the matrix L depends of the spatial discretization and the dimensionality of the problem. Usually, d is very large and thus we need fast, reliable and numerically stable algorithm for computing the functions (3.1.1). At the heart of all decomposition methods is the similarity transformation A = SBS −1 ,
(3.1.2)
where B is such that φ[i] (B) is easy to compute. Then φ[i] (A) = Sφ[i] (B)S −1 .
(3.1.3)
The choice of the matrices S and B in (3.1.2) usually involves two conflicting tasks: Make B close to diagonal so that φ[i] (B) is easy to compute and make S well conditioned so that errors in evaluating φ[i] (B) are not magnified. Thus, it is natural to consider methods based on the Schur decomposition so that S is unitary and therefore well conditioned. Next we present a general algorithm, first proposed in [16], which employs the block form of the Parlett recurrence and does not impose any restrictions on the argument matrix A.
3.1.1
Block Schur–Parlett algorithm
The first step of the algorithm is to compute the Schur decomposition A = QT Q ∗ , where Q is unitary and T is upper triangular. This can be achieved by the standard QR algorithm [22]. We now need to calculate the matrices F [i] = φ[i] (T ). Assuming that the diagonal [i] elements fkk of F [i] are already known, we can compute the whole F [i] using the scalar Parlett recurrence [56]. This is exactly how the MATLAB 6.5 (R13) built in function funm works. The problem with this approach is that it fails to produce correct results, in floating point arithmetic, when the eigenvalues of T are equal or close to each other. To avoid this problem, the authors in [16] propose to reorder T into a block upper triangular matrix Te. The diagonal blocks Tekk of Te are constructed in such a way that the eigenvalues of each diagonal block are “close” to each other and the distinct diagonal blocks are with “sufficiently distinct” eigenvalues. To define the meaning of “close” and “sufficiently distinct”, the authors introduce a special parameter in the format of the method. Since Te is block upper diagonal, so is Fe[i] = φ[i] (Te) . From the commutativity relation 68
Fe[i] Te = TeFe[i] (see the explicit form of the ETD functions (1.2.15)), it follows the following block form of the Parlett recurrence [i] [i] [i] [i] Tekk Fekl − Fekl Tell = Fekk Tekl − Tekl Fell +
l−1 ³ X
j=k+1
´ [i] [i] Fekj Tejl − Tekj Fejl ,
(3.1.4)
´ ³ [i] [i] where Te = (Tekl ) and Fe[i] = Fekl . If the diagonal blocks Fekk are already evaluated, the above relation allows us to compute the whole matrix Fe[i] by solving the Sylvester equation [i] (3.1.4) with respect to Fe . The imposed requirements for the eigenvalues of Tekk guarantee kl
that the equation (3.1.4) is nonsingular and well conditioned [16]. Thus, what we still need [i] to specify is how to compute the diagonal blocks Fekk . In [16], the authors suggest to use a suitable Taylor series truncation or Pad´e approximation. These are local approximations which are very accurate near the origin, but may suffer in accuracy away form it. In [i] addition, the accuracy of such approaches depends of the norm of Fekk and usually, when the norm is large, they require a scaling and squaring strategy [47]. For the first function φ [1] an analogous formula, similar to the well-known fundamental property of the exponential function, is proposed in [28]. It is based on the equalities φ[1] (2z) = (ez + 1)φ[1] (z)/2 and e2z = ez ez . In general, for i > 1, we do not have a simple generalization of the above [i] formulas and thus an alternative approach for computing Fekk can be to use higher order Chebyshev (uniform) rational approximations [68]. We discuss this type of approximations in detail in Subsection 3.1.2. Next we summarize the overall block Schur–Parlett algorithm for computing φ [i] (A) for a general matrix A and i = 1, 2, . . .. Algotithm 3.1.1. (Block Schur–Parlett algorithm) • Compute the Schur decomposition A = QT Q∗ .
• Reorder T into a block upper triangular matrix Te. • Compute φ[i] (Tekk ) for all diagonal blocks Tekk .
• Find φ[i] (Te) by solving the Silvester equation (3.1.4). • Compute φ[i] (A) = Qφ[i] (Te)Q∗ . The cost of Algorithm 3.1.1 depends from the eigenvalue distribution of A, and it is between 28d3 and d4 /3 flops (see [16]) for each i, where d is the dimensionality of A. The advantages of this method are that it is numerically stable and works without any restrictions on the structure of the matrix A. The main disadvantage is its higher computational cost, which makes it applicable only if integrators with fixed stepsize are used. In this case all φ[i] functions can be computed only once, at the beginning of integration process. Thus, the above method provides a benchmark for the computational cost of a method. 69
In the case when A is real symmetric (or complex Hermitian) a cheaper method, also based on the Schur decomposition, can be constructed. In the next subsection we generalize the approach proposed in [40] for computing the matrix exponential operator and later extended in [41] for computing the first function φ[1] , to a general algorithm for computing all the functions from (3.1.1).
3.1.2
Tridiagonal reduction
A common first step in the computation of the eigenvalues and the eigenvectors of a symmetric matrix A is to use a tridiagonal reduction of the form A = QTQT , where Q is orthogonal and T is symmetric tridiagonal. Such a representation of A can be obtained by using Hauseholder reflections [22]. Thus, according to (3.1.3), the computation of φ [i] (A) requires the value φ[i] (T). In [40, 41] efficient numerical algorithms for computing the exponential and the φ[1] function, based on the above tridiagonal reduction followed by Chebyshev (uniform) rational approximation, are developed. Here we utilize this approach and propose how it can be generalized, so that each of the functions from (3.1.1) can be efficiently computed. We first consider, how to find Chebyshev rational approximations to eT , for a given tridiagonal matrix T. The key idea is to compute the largest eigenvalue λ1 of T (for example by bisection method) and then make use of the following obvious equality eT = eλ1 eT−λ1 I , where I is d×d identity matrix. The eigenvalues of T−λ1 I are always located in the interval (−∞, 0]. This allows us to approximate eT−λ1 I by a rational function Rp (z) = Np (z)/Dp (z) (where Np and Dp are polynomials of z of degree p), which minimizes the maximum error for approximating ez on (−∞, 0]. An approximation to eT is then obtained by eT ≈ eλ1 Rp (T − λ1 I).
(3.1.5)
What we gain in this way is that, regardless of the spectrum of T, we can always use a rational Chebyshev approximation to eT−λ1 I in the interval (−∞, 0], which has the same coefficients. If we choose to represent Rp via its partial fraction expansion (see [21]) Rp (z) =
(p) α0
+
p X j=1
(p)
(p)
αj
(p)
z − θj
,
(3.1.6)
(p)
the coefficients αj and the poles θj of Rp can be computed once and for all. To achieve standard double precision (64-bit floating point numbers), it is sufficient to choose p = 14. Since the coefficients and the poles appears in complex conjugate pairs, for even p, it is enough to add just the first p/2 terms in the sum in (3.1.6), and then double the real part of the result. This in fact leads to significant computational savings, since it halves the number of matrix inversions in the corresponding formula (p)
Rp (T − λ1 I) = α0 I +
p X
(p)
αj
j=1
70
h
(p)
T − (λ1 + θj )I
i−1
.
(3.1.7)
(p)
(p)
The values of αj and θj , for p = 14 and 16, are listed in [41, Table 2]. Here we have chosen a diagonal rational approximation Rp , that is, the numerator Np has the same degree as the denominator Dp . We note however, that alternative strategies (e.g. using L-stable approximations) might also be considered without altering the principle of the method. Before we consider how the approximation (3.1.5) can be used to approximate any of the functions φ[i] form (3.1.1), we mention that, once we have the largest eigenvalue λ1 of T, we can always calculate a Chebyshev rational approximation to φ[i] (z) on the interval (−∞, λ1 ] and then used it to approximate φ[i] (T). However, this approach is rather unpractical since it highly depends on the value λ1 . Therefore we need to recompute the coefficients of the approximation for every different λ1 , which is quite a costly task. An alternative approach for computing φ[i] (T), in the case where the largest eigenvalue of T belongs to the interval (−∞, 0], is to replace eT in the definition of each of the functions φ[i] by its Chebyshev rational approximation (3.1.5). Thus, for example, the first function φ[1] can be approximated by p i−1 h ´ ³ X (p) (p) (p) . αj T − (λ1 + θj )I φ[1] (T) ≈ T−1 eλ1 α0 − 1 I + eλ1 j=1
The above formula should not be used when λ1 is positive. The problem in this case is that for each of the functions φ[i] we can not just shift the argument by λ1 I, since the relation between φ[i] (T) and φ[i] (T − λ1 I) is not that simple as it is for the exponential function. A different approach for approximating φ[1] (T), in the case when λ1 > 0, is proposed in [41]. It is based on the observation that for T I , B= 0 0 the exponential of B is given by
eB =
eT φ[1] (T) 0
I
.
(3.1.8)
Therefore, applying the approximation (3.1.5) with Rp given by (3.1.7) and T = B, we obtain p h i−1 X (p) (p) (p) . (3.1.9) αj B − (λ1 + θj )I eB = eλ1 eB−λ1 I ≈ eλ1 α0 I + j=1
Equating the entries in positions (1,2) of (3.1.8) and (3.1.9) leads to the following approximation for φ[1] (T) φ[1] (T) ≈ eλ1
p X j=1
(p)
αj
(p)
λ1 + θ j
71
h
(p)
T − (λ1 + θj )I
i−1
.
We have found that, for λ1 positive, the same idea can be easily generalized to the case when approximations to φ[i] (T) for i > 1 are needed. To approximate the function φ[2] (T), we take the matrix B to be of the form T 0 I . B= 0 0 0 0 I 0 It is easy to see, for example by direct computation, that its exponential is given by eT φ[2] (T) φ[1] (T) . eB = 0 I 0 0 I I
As before, based on formulas (3.1.5) and (3.1.7), we obtain the following approximation [2]
φ (T) ≈ e
λ1
p X j=1
(p)
³
αj
(p)
λ1 + θ j
i−1 h (p) . ´2 T − (λ1 + θj )I
Approximations for the rest of the functions φ[i] , for λ1 > 0, can be computed by a straight forward generalization of the above process. In the next algorithm we summarize the method based on the tridiagonal reduction for computing any of the functions φ[i] (A) for a symmetric matrix A and i = 1, 2, . . .. Algotithm 3.1.2. (Tridiagonal Reduction) • Calculate a symmetric tridiagonal reduction A = QTQT .
• Find the largest eigenvalue λ1 of T.
• Compute φ[i] (T) by a Chebyshev rational approximation with respect to the value of λ1 . • Calculate φ[i] (A) by φ[i] (A) = Qφ[i] (T)QT .
The cost of the algorithm, for each i, is O( 34 d3 +d+d2 +2d3 ), where d is the dimensionality of A. Therefore, the total number of operations for computing each of the functions φ [i] is 3 [1] O( 10 3 d ). In the case of constant step size, once we have computed the first function φ , we can reduce the work for computing the other φ[i] functions by reusing the tridiagonal decomposition. If the matrix A is symmetric and tridiagonal, the above algorithm requires only O(d 2 ) operations, since its first and last step are not needed. In this case instead of computing all the φ[i] functions once, before the integration begins, and then apply them at every step to a different vectors v, we can repeatedly compute the action of each φ[i] on its corresponding 72
vector v. The total number of operations in this case still does not exceed the work required to compute the matrix-vector product φ[i] times v, especially when d À p . This is because the inverse matrices in the third step of the algorithm are replaced by solutions of linear systems with tridiagonal coefficient matrices. In general, algorithms designed to solve such systems require O(d) operations, which leads to O(pd) flops needed to compute the action φ[i] v. This is to be compared with O(d2 ) operations needed for a matrix-vector product. In [Article 2 p.107], we present several direct methods for solving linear systems of equations, some of which can be easily adopted to the tridiagonal case considered here. We will comment more on this in Section 3.3. When A is tridiagonal, the above approach allows to preserve the total number of operations, needed to implement an exponential integrator, approximately the same even when a variable step size strategy is used. The advantage of the Algorithm 3.1.2 is that it is less expensive than Algorithm 3.1.1. Its disadvantages are that it is applicable only in the case when the matrix A is symmetric (Hermitian) and that it cannot be used with variable step size strategy, unless in the case when A is tridiagonal. We next consider a method which is entirely based on the idea of approximating the action of each of the functions φ[i] on a given state vector v. This approach is useful for implementations employing a variable step size strategy.
3.2
Krylov subspace approximations
Since the mid-eighties, the idea of using the Krylov subspace approximations to the action of the evolution operators has been studied by many authors. A short and definitely incomplete list of publications on this topic includes [17, 18, 21, 27, 28, 57, 58]. The convergence properties of the action of the matrix exponential operator are investigated in [17, 21, 58]. Later in [27] sharper error estimates are derived. It is also shown that, unless a good preconditioner is not available, the Krylov approximation to eA v converges faster than its corresponding approximation to the solution of the linear system (I − A)x = v. An approximation to the action of the first function φ[1] , based on the Krylov subspace approximation techniques, was considered in [58] and later studied in [27]. It was found that it obeys the same error bounds as the approximation to the matrix exponential operator. This provided the initial motivation for developing a Rosenbrock-like exponential integrators [28]. The main idea of the Krylov subspace technique is to approximately project the action of the evolution operator φ[i] (A) on a state vector v ∈ Cd , to a small Krylov subspace Km ≡ span{v, Av, . . . , Am−1 v}. Usually, even for a relatively small m ¿ d, an accurate approximation can be obtained [58]. Thus, the approach is to approximate the action of φ[i] (A) by the action of φ[i] applied to the projection of A on the smaller subspace Km . 73
It is convenient to choose an orthogonal basis Vm = [v1 , v2 , . . . , vm ] of Km . It can be generated by the Arnoldi algorithm, with v1 = v/||v||2 as an initial vector. Algotithm 3.2.1. (Arnoldi) Compute v1 = v/||v||2 . for j = 1, 2, . . . , m do for i = 1, 2, . . . , j do hi,j = (Avj , vi ), end w = Avj −
end
Pj
i=1 hi,j vi ,
hj+1,j = ||w||2 , vj+1 = w/hj+1,j ,
Alternatively, the Lanczos algorithm for generating a biorthogonal basis on the subspace Km , can also be used (see [58]). Let Hm be the m × m upper Hessenberg matrix consisting of the coefficients hi,j . Since the Algorithm 3.2.1 is just a modified Gram-Schmidt process, the following relation holds AVm = Vm Hm + hm+1,m vm+1 eTm ,
(3.2.1)
where ei denotes the ith unit vector in Rm . Using the fact that Vm is orthogonal, from the above relation, it follows that VmT AVm = Hm . Therefore, Hm represents the orthogonal projection of A to the subspace Km , with respect to the basis Vm . Similarly, Vm VmT φ[i] (A)v is the projection of φ[i] (A)v on Km , that is the closest approximation to φ[i] (A)v from Km . If β ≡ ||v||2 then v = βv1 and since v1 = Vm e1 , we have φ[i] (A)v ≈ Vm VmT φ[i] (A)v = βVm VmT φ[i] (A)v1
= βVm VmT φ[i] (A)Vm e1 .
(3.2.2)
From the computational point of view, the above formula is not useful, since it still involves operations with the big matrix A. To avoid this, the idea is to replace the term V mT φ[i] (A)Vm in (3.2.2) by suitable approximation. Note that VmT φ[i] (A)Vm is an m × m matrix. By induction, from (3.2.1), one can prove [58, Lemma 3.1] that pm−1 (A)v = βVm pm−1 (Hm )e1 , for all polynomials pm−1 of degree ≤ m − 1. Therefore, it is natural to approximate VmT φ[i] (A)Vm by φ[i] (Hm ). From (3.2.2) we obtain φ[i] (A)v ≈ βVm φ[i] (Hm )e1 .
(3.2.3)
The approximation (3.2.3) can also be derived from the standard Krylov approximation to the solution of linear systems of equations [57] and the Cauchy integral formula (see [27]). 74
The advantage of using (3.2.3) is that instead of working with the original large matrix A we use its orthogonal approximation Hm , which has much smaller dimension. The action φ[i] (A)v is then computed in O(md) operations by using only matrix-vector multiplications between elements with the original large size d. Thus, when m ¿ d the cost of computing the expression βVm φ[i] (Hm )e1 is usually much less than the cost needed to compute φ[i] (A)v. In addition, when A is Hermitian, it is possible to speed up the whole process by applying the Arnoldi algorithm to a shifted and inverted matrix [66]. It is chosen in such a way that it emphasizes the eigenvalues of importance, so that the algorithm can quickly find them. This approach relies on the fact that efficient preconditioned iterative solver is used to find the action of the inverse matrix. The computation of φ[i] (Hm )e1 requires O(m3 ) operations in general and only O(m2 ) if the matrix Hm is tridiagonal (e.g. A is symmetric). It can be done by using a Chebyshev rational approximation, evaluated by partial fraction expansion (see Subsection 3.1.2). The main computational cost of an exponential integrator, using Krylov subspace approximation technique, comes from the repeated application of Algorithm 3.2.1. At every step we need to construct several bases of Krylov subspaces with respect to the same matrix A and different vectors v. In general Algorithm 3.2.1 requires O(md2 ) operations. We mention also that it assumes exact arithmetic is used. In practice, round off errors and cancellations might cause a loss of the orthogonality between the vectors v i . A significant improvement in the performance can be achieved by using double orthogonalization [38]. In addition, a convergence criterion is needed to determine the value of m that gives a sufficiently accurate approximation. Thus, we conclude that the above approach is preferable in the case when the number of Krylov bases needed can be significantly reduced. Efficient exponential integrators based on this reduced idea are developed in [28]. The advantage of the Krylov subspace approximation technique is that, implementations based on a variable stepsize strategy, do not increase the total computational cost of the integrator.
3.3
Cauchy integral approach
The last approach for computing the functions φ[i] or their action on a given vector v, which we consider, is introduced in [35]. It is based on the Cauchy integral formula φ[i] (A) =
1 2πi
Z
ΓA
φ[i] (λ)(λI − A)−1 dλ,
(3.3.1)
where ΓA is a contour in the complex plane that encloses the eigenvalue of A, and it is also well separated from 0. It is practical to choose the contour ΓA to be a circle centered on the real axis. In this way when A is real, based on the symmetry, one can evaluate the integral only on the upper half of the circle and then double the real part of the result. To approximate the integral in (3.3.1), the authors in [35] propose to use the trapezoid rule, 75
which converges exponentially [65]. Therefore, we obtain the following approximation φ[i] (A) ≈
k 1X λj φ[i] (λj )(λj I − A)−1 , k
(3.3.2)
j=1
where k is the number of the equally spaced points λj along the contour ΓA . Usually, values of k = 32 or k = 64, are sufficient to insure correct computations. For problems with diagonal matrix A it is beneficial to choose the contour Γ A to be, in addition, a circle centered at A. Thus, (3.3.2 ) simply reduces to the mean of φ [i] over the equally spaced points along ΓA (or again just half of them for a real A). When the matrix A is non-diagonal, the cost for computing an approximation to the functions φ [i] increases. This is due to the number of the matrix inverses involved in (3.3.2). In the case of constant stepsize, the total impact on the computational time is still small, since all the φ[i] functions can be evaluated only once before the integration begins. However, in the case of variable step size, direct application of (3.3.2) leads to significant increase in the computational work. To gain more insight into how the formula (3.3.1) can be effectively used, in the case of variable step size, we recall that the matrix A = γhL, where γ ∈ R, h is the step size and L is the discretized linear operator. Note that every time when we need to change the step size h, it is enough to change only the parameter γ. The idea now is to represent φ[i] (γhL) in such a way that the number of the matrix inverses used in (3.3.1), respectively in (3.3.2), is independent of γ. If we can choose a suitable contour Γ, such that for all different values of γ, which appear in the integration process, it encloses the eigenvalues of γhL and γΓ is well separated from 0 then we can compute each of the functions φ[i] by the following formula Z 1 [i] [i] φ[i] (γλ)(λI − hL)−1 dλ. (3.3.3) φ (A) = φ (γhL) = 2πi Γ As before, approximating the integral in (3.3.3) by the trapezoid rule, we get k 1X λj φ[i] (γλj )(λj I − hL)−1 , φ (A) ≈ k [i]
(3.3.4)
j=1
where now λj are the equally spaced points along the contour Γ. The above formula allows to reduce the computational work needed to evaluate the functions φ[i] or their action on a given vector v, in the case when a variable step size is used. The main advantage comes form the fact that the inverse matrices in (3.3.4) no longer depend of γ. However, in any case we have to compute k (or k/2) matrix inverses. In the case when the matrix L arises from a finite difference approximation to a second order partial differential operator, we can benefit from its sparse block structure. Similar to the idea presented in Subsection 3.1.2, in this case, we can also evaluate the action of the function φ[i] on a given vector v. If the number of operations required does not exceed O(d2 ) the cost of a matrix-vector product then we obtain a competitive method. Note that 76
in general the matrix φ[i] (γhL) does not retain the sparse block structure of L. Thus, what we need is an efficient method for solving special sparse block linear systems of equations. When the matrix −L is symmetric (Hermitian) and positive definite the most favourable methods are preconditioned conjugate gradient and multigrid methods [34]. They are based on the idea of approximating the solution by an iterative procedure and usually require O(d) operations. The problem with these methods is that they require a good preconditioner or a rather complex algorithm with considerable overhead to organize the computations. In addition, we note that the matrices in (3.3.4) are symmetric (Hermitian) only if λ j ∈ R. Alternatively, direct methods for solving linear systems of equations can be used. In order to be competitive, such methods are specially design to take advantage from the sparse block structure of the coefficient matrix. In general, the structure of L depends on the dimensionality of the problem, the type of the approximation and the boundary conditions imposed. When L is block tridiagonal, two methods are of practical importance: the Buneman variant of cyclic reduction [6] and the decomposition method based on the fast Fourier transform (FFT) [11]. Both of these methods compute the solution in O(d log 2 n) operations, where n is the block size of L. Combination of the above two methods known as Fourier analysis-cyclic reduction (FACR) is proposed in [31]. The asymptotic operation count for this method is reduced to O(d log 2 log2 n) (see [64]). A restriction for all of these methods is that n should be power of two or a composite of small primes. In [Article 2 p.107], we present a method for solving tridiagonal block Teoplitz linear systems of equations, which does not impose any restrictions on n. The method is based on a modified LU factorization and it is fully applicable to all the matrices in (3.3.4). In [Article 3 p.133], the same idea is generalized to the case when the coefficient matrix has pentadiagonal block circulant structure. The complexity of this methods is O(d 2 ). Significant computational savings can be achieved by using the techniques of solving linear systems with multiple right hand sides. Since the methods are based on the LU decomposition idea, we can factorize the k (k/2) coefficient matrices arising from (3.3.4) once and for all and then use only the back-substitution formulas to find their action on a given vector v. Thus, for k ¿ d we obtain methods which can be implemented with a variable step size, without to increase the total computational work. Before we consider some numerical experiments, we summarize the main advantages and disadvantages of the different methods for computing the functions φ[i] or their actions on a vector, presented in this chapter. • Methods based on Algorithm 3.1.1 are expensive and not suitable for implementations using variable step size. Their main advantage is that they do not place any restrictions on the coefficient matrix. • Methods using Krylov subspace approximation techniques are suitable for implementations involving variable step size, but are applicable only in the case when the number of the needed Krylov bases can be significantly reduced. • Methods based on the Cauchy integral formula are suitable for both constant and 77
variable stepsize implementations. Particularly cheap methods, in the case of variable stepsize, can be obtained if the the coefficient matrix has a sparse block structure. Alternatively, for tridiagonal matrices, Algorithm 3.1.2 can be used.
3.4
Numerical experiments
In this section, we present numerical experiments with some of the exponential integrators studied in the thesis. The examples under consideration are Kuramoto-Sivashinsky, Allen-Cahn and Korteweg de Vries (KdV) equations. Since all of the tested exponential integrators, except the method of Hochbruck–Ostermann given in Table 1.15, are based on the nonstiff order conditions, to avoid possible order reduction, we consider examples where the nonlinear term has sufficient spatial regularity. In general, for applications concerning PDEs, the classical order of convergence is not always obtained (see Subsection 1.2.5). Order reduction, due to the lack of sufficient temporal and spatial smoothness, should be expected. For parabolic problems with periodic boundary conditions, full order of convergence can be observed [29, 30]. In our numerical experiments, for the Kuramoto-Sivashinsky and the KdV equations, we also impose periodic boundary conditions. Thus, the linear term L has a diagonal structure and the computation of the φ[i] functions by the Cauchy integral formula (see Section 3.3) simply reduce to the mean of φ[i] over the contour Γ. For all numerical experiments, a constant step size is used. Computation of the φ [i] functions, in the non-diagonal example (Allen-Cahn equation), is also done by the Cauchy integral approach. In this case, the total computational cost of a method is still small, since all the matrix inverses in (3.3.4) are evaluated only once before the time-stepping begins. Extensive numerical tests between linearly implicit methods [2], split step methods [59] and exponential integrators based on the linear multistep and multistage idea are presented in [14, 35]. There the authors conclude that the Exponential Time Differencing Runge– Kutta (ETD RK) methods, presented in Subsection 1.2.3, consistently out perform all the other methods. Comparison between the stability regions for different exponential integrators based on Lie group methods, applied to semi-discretized stiff problems, is given in [36]. There the author concludes that, for such kind of problems, the RKMK methods (see Section 2.2) suffer from numerical instability. It is due to the small convergence radius of the dExp −1 map. Taking into account the above arguments, we have chosen to implement the following exponential integrators: • ETD RK3(CM) The third order method of Cox–Matthews given in Table 1.9; • ETD2RK3 The third order method from Table 1.12; • ETD2CF3 The third order method from Table 1.13; • ETD RK4(CM) The fourth order method of Cox–Matthews given in Table 1.10; 78
• ETD RK4(Kr) The fourth order method of Krogstad given in Table 1.11; • ETD RK4(Min) The fourth order method from Table 1.8; • ETD RK4(HO) The fourth order method of Hochbruck–Ostermann given in Table 1.15; • IF RK4 The fourth order Integrating Factor Runge–Kutta methd based on the classical fourth order Runge–Kutta method (1.3.5); • GIF1/RK4 The fourth order Generalized Integrating Factor Runge–Kutta method from Table 2.4 with the classical fourth order Runge–Kutta coefficients (1.3.5); • GIF2/RK4 The fourth order Generalized Integrating Factor Runge–Kutta method from Table 2.5 with the classical fourth order Runge–Kutta coefficients (1.3.5); • GIF3/RK4 The fourth order Generalized Integrating Factor Runge–Kutta method with third order polynomial approximation to the nonlinear part N , also with the classical fourth order Runge–Kutta coefficients (1.3.5); • CF4 The fourth order Commutator Free Lie group method given in Table 2.6. For each of the three test problems we compare accuracy against step size. The results are given in different figures for each example.
The Kuramoto-Sivashinsky equation The first example is the Kuramoto-Sivashinsky equation ut = −uux − uxx − uxxxx ,
x ∈ [0, 32π]
with periodic boundary conditions and with initial condition borrowed from [35] ³ x ´´ ³ x ´³ 1 + sin . u(x, 0) = cos 16 16
A 128-point Fourier spectral discretization in space is used. Since the boundary conditions are periodic the transformed equation in the Fourier space can be represented in the form (1.1.1), the linear and nonlinear parts are defined as (Lˆ u)(k) = (k 2 − k 4 )ˆ u(k),
N (ˆ u) = −
ik (F ((F −1 (ˆ u))2 )), 2
where F denotes the discrete Fourier transform. The integration in time is done entirely in the Fourier space until t = 65. The results for the different numerical schemes are plotted in Figure 3.1. 79
Kuramoto−Sivashinsky equation
0
10
Kuramoto−Sivashinsky equation
1
10
0
10
−1
10
−1
Relative error at time = 65
Relative error at time = 65
10 −2
10
−3
10
ETD RK3(CM)
−4
10
ETD2RK3 ETD2CF3
−5
10
ETD RK4(CM)
−2
10
−3
10
−4
10
−5
10
IF RK4
−6
10
GIF1/RK4 −7
ETD RK4(HO)
10
GIF2/RK4
−6
10
ETD RK4(Kr)
GIF3/RK4
−8
10
ETD RK4(Min)
CF4
−7
10
−9
−2
10
−1
10
0
10
10
−2
−1
10
Stepsize h
10
0
10
Stepsize h
Figure 3.1: Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Kuramoto-Sivashinsky equation
The Allen-Cahn equation The second example is the Allen-Cahn equation written in the form ut = εuxx + u − u3 ,
x ∈ [−1, 1],
where ε = 0.01 and with boundary and initial conditions also borrowed form [35] u(−1, t) = −1,
u(1, t) = 1,
u(x, 0) = 0.53x + 0.47 sin(−1.5πx).
After discretization in space based on the Chebyshev grid points we can rewrite the equation in the form (1.1.1), with L = εD 2 , N (u) = u − u3 , where D is the Chebyshev differentiation matrix [65]. Therefore, the matrix L is full. The integration in time is until t = 31. In Figure 3.2, we have plotted the results for the different numerical schemes under consideration.
The Korteweg de Vries equation The final example is the KdV equation considered in [36, Article 5] ut = −uxxx − uux ,
x ∈ [−π, π],
with periodic boundary conditions and with initial condition √ u(x, 0) = 3C/ cosh2 ( Cx/2), 80
Allen−Cahn equation
−2
10
−3
−3
10
10
−4
−4
10
Relative error at time = 31
10
Relative error at time = 31
Allen−Cahn equation
−2
10
−5
10
−6
10
ETD RK3(CM) ETD2RK3
−7
10
ETD2CF3 −8
−6
10
−7
10
IF RK4 −8
ETD RK4(CM)
10
−5
10
GIF1/RK4
10
GIF2/RK4
ETD RK4(HO) −9
10
−9
GIF3/RK4
10
ETD RK4(Kr)
CF4
ETD RK4(Min) −10
−10
10
−2
−1
10
10
0
10
−2
10
−1
10
0
10
Stepsize h
10
Stepsize h
Figure 3.2: Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Allen-Cahn equation where C = 625. The exact solution is 2π/C periodic and is given by u(x, t) = u(x − Ct, 0). We use a 256-point Fourier spectral discretization in space. In this case the matrix L is again diagonal. The integration in time is done for one period. The results for the different numerical schemes are plotted in Figure 3.3.
Korteweg de Vries equation
1
Korteweg de Vries equation
1
10
10
0
10
0
10
−1
10
−1
10
−2
Error at time = 2π/C
Error at time = 2π/C
10
−3
10
−4
10
ETD RK3(CM) −5
10
ETD2RK3 ETD2CF3
−6
10
ETD RK4(CM)
−2
10
−3
10
−4
10
−5
10
IF RK4 GIF1/RK4
−6
10
−7
10
ETD RK4(HO)
GIF2/RK4
ETD RK4(Kr)
−8
10
GIF3/RK4
−7
10
ETD RK4(Min)
CF4
−9
10
−8
−7
10
−6
−5
10
10
10
−4
10
−7
10
Stepsize h
−6
−5
10
10
−4
10
Stepsize h
Figure 3.3: Step size versus relative error for ETD RK (left) and GIF/RK (right) methods for the Korteweg de Vries equation As seen from Figure 3.1–3.3, all methods exhibit the predicted nonstiff order of convergence under the chosen initial and boundary conditions. For all examples, the scheme ETD RK4(Min) is more efficient than the other ETD RK schemes. This can be explained with 81
the fact that it was constructed based on the nonstiff order theory presented in Subsection 1.2.2, so that the error coefficients for the elementary differentials of higher order are minimized. As expected, the ETD RK methods perform better than the IF RK methods even based on the same underlying Runge–Kutta method. This is due to the fact that the local error of IF RK methods is bigger than the local error of ETD RK methods (see [46]). The local error for the GIF/RK methods reduces with increasing the degree of the polynomial approximation to the nonlinear part of the problem. This results in several orders in magnitude improvement in accuracy over the standard IF RK methods. However, as it was pointed from Krogstad in [36, Article 5], the improved accuracy comes to some extent at the price of stability. This fact, in particular, is well illustrated in the numerical experiments for the KdV equation, where the GIF/RK methods fail to produce results for large stepsizes. The reason for this is the reduced stability regions for GIF/RK methods [36, Article 5].
82
Conclusions The aim of this thesis was to study different classes of exponential integrators for time integration of semilinear problems and to propose an unified framework in which this methods can be analyzed. Below, we give a summary of the main topics treated in the thesis.
Summary In Chapter 1 we presented the philosophy behind exponential integrators applied to semilinear problems and discussed the three main classes of exponential integrators: exponential linear multistep (multivalue), exponential Runge–Kutta (multistage) and exponential general linear methods. Next, in Chapter 2 we studied the connection between exponential integrators and Lie group methods based on the affine algebra action. The freedom in the choice of the algebra action, allows all exponential integrators presented in Chapter 2 as well as the methods proposed in [Article 1,91] to be applied to nonautonomous and quasilinear problems of the the form u0 = L(u, t)u + N (u, t),
u(t0 ) = u0 .
The only difference in this case is in the definition of the generic function F , which provides the representation of the differential equation on the manifold. Because of the time dependence of the linear part, the resulting methods require at the beginning of each step to recompute the exponential and the related φ[l] functions included in the format of the method. In Chapter 3 we discussed different algorithms for fast and numerically stable computation of the most commonly used ETD φ[i] functions. Finally, we tested some of the exponential integrators studied in the thesis, on several well known examples. Next, we summarize the main achievements of this thesis.
Contributions We proposed in Subsection 1.2.1 a general class of exponential Runge–Kutta methods specifically designed for time integration of semilinear problems. This class of methods include as special cases all known exponential Runge–Kutta methods. The nonstiff order theory developed in Subsection 1.2.2, reduces to the the order theory for the adaptive Runge–Kutta 83
methods proposed in [5]. An advantage of our approach is that it provides a non-recursive rule for generating each order condition from its corresponding rooted tree. In addition, the same technique can also be applied to derive the nonstiff order conditions for the exponential general linear methods considered in Subsection 1.3.3. We believe that locating good methods based on the nonstiff order conditions, is the first step for developing competitive methods for realistic stiff problems. We also studied the natural connection between exponential integrators and Lie group methods with affine algebra action. In particular, in Subsection 2.4.2, we showed that the Generalized Integrating Factor Runge–Kutta (GIF/RK) methods introduced in [36, Article 5] are examples of RKMK methods. In addition, we proposed a new approach in the derivation of GIF/RK methods, which allows the nonlinear part of the problem (1.1.1) to be approximated by trigonometric polynomials. The choice of the algebra action plays a crucial role in the overall performance of any Lie group method. In [Article 1, p.91], we suggested a way how to construct exponential integrators based on the framework of Lie group methods with algebra action arising from the solutions of differential equations with nonautonomous frozen vector fields. The main computational challenge in the implementation of every exponential integrator comes from the necessity of computing the exponential and the related functions by some fast and numerically stable algorithm. In Subsection 3.1.2, we generalized the tridiagonal reduction approach, proposed in [41], to a general algorithm for computing all ETD φ [i] functions. We also commented on the main advantages and disadvantages of other numerical techniques, which can also be used for computing the ETD φ[i] functions. Usually, a variable stepsize implementation of an exponential integrator requires efficient linear solver. In the case when the linear part of the problem arises from a finite difference approximation, we can benefit from its sparse block structure. In [Article 2 p.107], we presented a method for solving tridiagonal block Teoplitz linear systems of equations. The method is a modification of the LU factorization proposed in [19]. In [Article 3 p.133], the same approach was generalized to the case when the coefficient matrix has pentadiagonal block circulant structure.
Future work Many questions have arisen during the work on this thesis, which we intend to consider in future work. For example, the choice of the φ[l] functions plays a significant role in the actual performance of any exponential integrator. Notice that an ETD method will generally perform better than an IF method, even based on the same underlying method. The possible choices of φ[l] functions which lead to methods are still unclear. Thus, determining the set of functions which provides in some sense optimal methods, is a crucial question which requires further investigation. Developing the nonstiff and respectively the stiff order conditions for exponential general linear methods is a field of ongoing research. Such an order theory, will allow other 84
exponential general linear methods, different form the GIF/RK methods and the methods introduced in [Article 1, p.91], to be constructed. The improved accuracy of GIF/RK methods, is due to their higher stage order (see [46]). The main difficulty with these methods, as Krogstad pointed out in [36, Article 5], is that the improved accuracy comes to some extent at the price of stability. A way to overcome this difficulty is to extend the class of general linear methods with inherent Runge–Kutta stability to the exponential setting. Probably the main question regarding the exponential integrators, which still needs to be answered is: Are these methods fully competitive with the existing numerical techniques for solving stiff problems. The answer of this question requires extensive numerical experiments with various implementations, from fixed stepsize and fixed order to variable stepsize and variable order codes. Thus, it is clear that the choice of reliable and numerically stable algorithms for computing the exponential and the related functions is an important direction for future investigations. Differential algebraic equations (DAEs) are ordinary differential equations subject to algebraic constraints. Many problems which arise in the real applications can be represented by DAEs. Examples are the discretized incompressible Navier–Stokes equations, where the incompressibility condition is the algebraic constraint. Extending the class of exponential integrators to the set of DAEs is an other interesting topic for future research.
85
Bibliography [1] R. Abraham, J. Marsden, and T. Ratiu, Manifolds, Tensor Analysis and Applications, Springer, Second edition, 1988. [2] U. Ascher, S. Ruuth, and B. Wetton, Implicit-explicit methods for time-dependent partial differential equations, SIAM J. Numer. Anal. 32 (1995), 797–823. [3] H. Berland and B. Owren, Fourth order exponential time integrators for the nonlinear Scr¨ odinger equation, private conversation, April, 2004. [4] G. Beylkin, J. Keiser, and L. Vozovoi, A new class of time discretization schemes for the solution of nonlinear PDEs, J. Comput. Phys. 147 (1998), 362–387. [5] J. Bruder, K. Strehmel, and R. Weiner, Partitioned adaptive Runge–Kutta methods for the solution of nonstiff and stiff systems, Numer. Math. 52 (1988), 621–638. [6] O. Buneman, A compact non-iterative Poisson solver, Rep. 294, Stanford University Institute for Plasma Research, 1969. [7] J. C. Butcher, On the convergence of numerical solutions to ordinary differential equations, Math. Comp. 20 (1966), 1–10. [8]
, Numerical methods for ordinary differential equations, John Wiley & Sons, 2003.
[9] J. C. Butcher and Z. Jackiewicz, Construction of general linear methods with rungekutta stability properties, Numer. Algorithms 36 (2004), 63–72. [10] J. C. Butcher and W. Wright, The construction of practical general linear methods, BIT 43 (2003), 695–721. [11] B. Buzbee, G. Golub, and C. Nielson, On direct methods for solving Poisson’s equation, SIAM J. Numer. Anal. 7 (1970), 627–656. [12] E. Celledoni, A. Marthinsen, and B. Owren, Commutator-free Lie group methods, FGCS 19(3) (2003), 341–352. 86
[13] J. Certaine, The solution of ordinary differential equations with large time constant, Math. methods for digital comput. (1960), 128–132. [14] P. M. Cox and P.C. Matthews, Exponential time differencing for stiff systems, J. Comput. Phys. 176 (2002), 430–455. [15] P. Crouch and R. Grossman, Numerical integration of ordinary differential equations on manifolds, J. Nonlinear. Sci. 3 (1993), 1–33. [16] P. Davies and N. Higham, A Schur-Parlett algorithm for computing matrix functions, SIAM J. Matrix Anal. Appl. 25(2) (2003), 464–485. [17] V. L. Druskin and L. A. Knizhnerman, Error bounds in the simple Lanczos procedure for computing functions of symmetric matrices and eigenvalues, Comput. Maths. Math. Phys. 7 (1991), 20–30. [18]
, Krylov subspace approximations of eigenpairs and matrix functions in exact an computer arithmetic, Numer. Lin. Alg. Appl. 2 (1995), 205–217.
[19] S.M. El-Sayed, Study of special matrices and numerical methods for special matrix equations, Ph.D. thesis, Bulgarian Academy of Sciences, 1996, in Bulgarian. [20] A. Friedli, Verallgemeinerte Runge–Kutta verfahren zur l¨ oesung steifer differentialgleichungssysteme, Lect. Notes Math. 631 (1978). [21] E. Gallopoulos and Y. Saad, Efficient solution of parabolic equations by Krylov approximation methods, SIAM J. Sci. Statist. Comput. 13 (1992), 1236–1264. [22] G. Golub and C. Van Loan, Matrix computations, Johns Hopkins University Press, Baltimore, 1996, 3td ed. [23] E. Hairer, C. Lubich, and G. Wanner, Geometric numerical integration, Springer, 2003, Number 31 in Springer Series in Computational Mathematics. [24] E. Hairer, S. Nørsett, and G. Wanner, Solving Ordinary Differential Equations I, Nonstiff Problems, Springer, 1993, Number 8 in Springer Series in Computational Mathematics, 2nd edition. [25] E. Hairer and G. Wanner, Solving Ordinary Differential Equations II, Stiff and Differential–Algebraic Problems, Springer, 1996, Number 14 in Springer Series in Computational Mathematics, 2nd edition. [26] D. Henry, Geometric theory of semilinear parabolic equations, Lecture Notes in Mathematics 840, Springer, Berlin, 1981. [27] M. Hochbruck and C. Lubich, On Krylov subspace approximations to the matrix exponential operator, SIAM J. Numer. Anal. 34 (1997), 1911–1925. 87
[28] M. Hochbruck, C. Lubich, and H. Selhofer, Exponential integrators for large systems of differential equations, SIAM J. Sci. Comput 19(5) (1998), 1552–1574. [29] M. Hochbruck and A. Osterman, Explicit exponential Runge–Kutta methods for semilinear parabolic problems, Submitted to SIAM J. Numer. Anal., 2004. [30]
, Exponential Runge–Kutta methods for parabolic problems, To appear in Appl. Numer. Math., 2004.
[31] R. Hockney, The potential calculation and some applications, Methods of Computational Physics 7 (1969), 136–211. [32] A. Iserles, H. Munthe-Kaas, S.P. Nørsett, and A. Zanna, Lie-group methods, Acta Numerica 9 (2000), 215–365. [33] Z. Jackiewicz, A. Marthinsenand, and B. Owren, Construction of Runge–Kutta methods of Crouch–Grossman type of high order, Adv. Comput. Math. 13(4) (2000), 405–415. [34] C. Johnson, Numerical solution of partial differential equations by the finite element methods, Cambridge University press, 1987. [35] A.-K. Kassam and L.N. Trefethen, Fourth order time stepping for stiff PDEs, To appear in: SIAM J. Sci. Comp., 2003. [36] S. Krogstad, Topics in numerical Lie group integration, Ph.D. thesis, University of Bergen, 2003. [37] J. Lawson, Generalized Runge–Kutta processes for stable systems with large Lipschitz constants, SIAM J. Numer. Anal. 4 (1969), 372–390. [38] R. Lehoucq, D. Sorensen, and C. Yang, ARPACK user’s guide, SIAM, 1998. [39] E. Lodden, Geometric integration of the heat equation, Master’s thesis, University of Bergen, 2000. [40] Y. Y. Lu, Exponentials of symmetric matrices through tridiagonal reductions, Linear Algebra Appl. 279 (1998), 317–324. [41]
, Computing a matrix function for exponential integrators, J. Comp. and Appl. Math. 161(1) (2003), 203–216.
[42] A. Lunardi, Analytic semigroups and optimal regularity in parabolic problems, Birkh¨auser, Basel, 1995. [43] Y. Maday, A. Patera, and E. Rønquist, An operator-integration-factor splitting method for time-dependent problems: application to incompressible fluid flow, J. Sci. Comp. 5(4) (1990), 263–292. 88
[44] J. Marsden and T. Ratiu, Introduction to Mechanics and Symmetry, Springer, Second edition, 1999, Number 17 in Text in Applied Mathematics. [45] B. Minchev, Exponential time differencing and Lie-group methods for stiff problems, Talk given at the International Conference on Scientific Computation and Differential Equations, SciCADE 2003, Trondheim, Norway, June 30 - July 4, 2003. [46] B. Minchev and W. Wright, A review of exponential integrators, in preparation, University of Bergen, 2004. [47] C. Moler and C. Van Loan, Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later, SIAM Review 45(1) (2003), 3–49. [48] H. Munthe-Kaas, Lie-Butcher theory for Runge–Kutta methods, BIT 35 (1995), 572– 587. [49] [50]
, Runge–Kutta methods on Lie groups, BIT 38 (1998), 92–111. , High order Runge–Kutta methods on manifolds, Appl. Num. Math. 29 (1999), 115–127.
[51] H. Munthe-Kaas and W. Wright, A hitchhikers guide to Lie-Butcher theory, in preparation, University of Bergen, 2004. [52] A. Murua, Formal series and numerical integrators, part I: Systems of ODEs and symplectic integrators, Appl. Numer. Math. 29 (1999), 221–251. [53] S. Nørsett, An A-stable modification of the Adams-Bashforth methods, Lect. Notes Math. 109 (1969), 214–219. [54] A. Ostermann, Numerical solutions of abstract ODEs, Lecture notes from Dobbiaco Summer School on Numerical Analysis, 2004. [55] B. Owren and A. Marthinsen, Runge–Kutta methods adapted to manifolds and based on rigid frames, BIT 39(1) (1999), 116–142. [56] B.N. Parlett, A recurrence among the elements of functions of triangular matrices, Linear Algebra Appl. 14 (1976), 117–121. [57] Y. Saad, Krylov subspace methods for solving large unsymmetric linear systems, Math. Comp. 37 (1981), 105–126. [58]
, Analysis of some Krylov subspace approximations to the matrix exponential operator, SIAM J. Numer. Anal. 29(1) (1992), 209–228.
[59] J. Sanz-Serna and M. Calvo, Numerical Hamiltonian problems, Chapman & Hall, London, 1994. 89
[60] T. Steihaug and A. Wolfbrand, An attempt to avoid exact jacobian and nonlinear equations in the numerical solution of stiff differential equations, Math. Comp. 33 (1979), 521–534. [61] K. Strehmel and R. Weiner, Behandlug steifer anfangswetprobleme gew¨ ohnlicher differentialgleichungen mit adaptiven Runge–Kutta methoden, Computing 29 (1982), 153– 165. [62]
, B-convergence results for linearly implicit one step methods, BIT 27 (1987), 264–281.
[63] A. Suslowicz, Application of numerical Lie group integrators to parabolic PDEs, Technical report 219, University of Bergen, 2001. [64] P. Swarztrauber, The methods of cyclic reduction, Fourier analysis and the FACR algorithms for the discrete solution of the Poisson’s equation on a rectangle, SIAM Review 19 (1977), 490–501. [65] L. N. Trefethen, Spectral methods in Matlab, SIAM, 2000. [66] J. van den Eshof and M. Hochbruck, Preconditioning Lanczos approximations to the matrix exponential, tech. rep. Heinrich-Heine Universit¨at D¨ usseldorf, 2004. [67] V. Varadarajan, Lie groups, Lie algebras, and their representation, Springer, New York, 1984. [68] R. S. Varga, On higher order stable implicit methods for solving parabolic differential equations, J. Math. and Phys XL (1961), 220–231. [69] W. Wright, General linear methods with inherent Runge–Kutta stability, Ph.D. thesis, The University of Aucland, New Zealand, 2002. [70] K. Yosida, Functional analysis, Springer, Berlin, 1980, Grundlehren der mathematischen Wissenschaften, vol. 123.
90
Article 1 Lie group integrators with nonautonomous frozen vector fields Submitted to Appl. Numer. Math.
91
92
Lie group integrators with nonautonomous frozen vector fields Borislav V. Minchev Department of Computer Science, University of Bergen, Thormhlensgate 55, N-5020 Bergen, Norway
Abstract Lie group methods for nonautonomous semi-discretized in space, partial differential equations are considered. The choice of frozen vector field and its corresponding algebra action on the manifold for such problems is discussed. A new exponential integrator for semilinear problems, based on commutator free Lie group methods with algebra action arising from the solutions of differential equations with nonautonomous frozen vector fields is derived. The proposed new scheme is then compared with some existing methods in several numerical experiments. Key words: Lie group methods, algebra action, exponential integrators, stiff systems 1991 MSC: 65L06, 65M29, 35G10, 58F39
1
Introduction
Recently, a lot of Lie group integrators for solving semi-discretized partial differential equations (PDEs) has been derived in the literature. The original idea was first introduced in [16] and then further investigated for the heat equation in [4,12,17], stiff PDEs in [10,11,14], convection diffusion problems in [3] and for the Schr¨odinger equation in [1]. What is common between all this methods is that to advance from one point to another they all use an algebra action arising from the solution of an autonomous differential equation, which does not depend explicitly on the time t. In this paper we propose a way how to construct Lie group integrators for nonautonomous problems based on an algebra action arising from the solution Email address:
[email protected] (Borislav V. Minchev). URL: http://www.ii.uib.no/∼borko (Borislav V. Minchev).
Submitted to Applied Numerical Mathematics
of a differential equation which can depend explicitly on t. The idea is a natural extension of the autonomous case. The approach is to rewrite the differential equation in its equivalent autonomous form and then to apply the affine action [16] to the transformed equation. Thus, what we obtain are time dependent frozen vector fields. This provides us some extra freedom in the construction process, which can be used to choose the algebra action to be a better approximation of the flow of the original vector field. The paper is organized as follows: We start in Section 2 with introducing some notation and the basic theory involved. Next we consider the framework for nonautonomous problems and discuss how it is related with the choice of the algebra action. In Section 3 we propose a new time dependent frozen vector field and its corresponding algebra action. In addition, we discuss some further generalizations. In Section 4 we derive a new exponential integrator for semilinear problems based on the fourth order commutator free Lie group method [4]. Finally in Section 5 we compare the proposed new exponential integrator with some existing methods and discuss the advantages of the new approach.
2
Background theory and notations
The framework which we use in this paper is given by Lie groups and their action upon a homogeneous manifold [8,15,16]. We take advantage of the fact that, in order to construct a Lie group integrator, we do not really need to know what the structure of the Lie group is and how it acts on the manifold. It is enough to specify the generic presentation of the differential equation and the algebra action on the manifold (see [14]). For simplicity, we do not include a discussion on the structure of the underlying Lie group and how it acts on a manifold. Let us first consider the following differential equation defined on a d + p dimensional manifold M ≡ Rd+p . y 0 = f(y(t)),
y(t0 ) = y0 .
(2.1)
The very first question in the construction of a Lie group integrator is how to define the basic motions on M. Since Rd+p is a linear space, it is easy to construct integrator which stays on the manifold. The challenge in this case is how to choose the basic motions in such a way that they provide a good approximation to the flow of the original vector field. In this paper we define the basic movements on M to be given by the solution of a simpler differential equation y 0 = FΘ (y), y(t0 ) = y0 , (2.2) which locally approximates (2.1). Thus, the Lie algebra g is generated from 94
the set of all coefficients Θ of the frozen vector fields FΘ and the Lie algebra action ∗ : g×M → M on the manifold is simply given by the solution of (2.2). In other words, if Θ ∈ g, its action upon the point y0 ∈ M, which we denote by hΘ ∗ y0 , is given by the solution of (2.2) at time t0 + h. Every frozen vector field can be represented in the form FΘ (y) = Θ ~ y, where ~ : g × M → T M and according to (2.2), it satisfies ¯
d¯ Θ ~ y = ¯¯¯ tΘ ∗ y. dt t=0
Note that the map Θ → FΘ is an algebra homomorphism between g and the set of all vector fields on M. If the algebra action ∗ is transitive i.e. starting from a point y0 ∈ M we can reach any other point y1 ∈ M by letting some element Θ ∈ g act on y0 , the differential equation (2.1) can be rewritten in the form y 0 = F (y) ~ y, y(t0 ) = y0 , (2.3) where F : M → g. The above formulation is called the generic presentation of the differential equation on the manifold and plays an important role in the theory of the Lie group integrators (see [16]). The choice of the frozen vector field FΘ and its corresponding algebra action, very much depends of the actual structure of f(y). The simplest possible case is when g = {b ∈ Rd+p }, F (y0 ) = f(y0 ) = b and Fb (y) = b ~ y = b then the algebra action on M is given by translations and we recover the traditional integration schemes. In the case when f(y) = L(y)y + N(y), one can define the Lie algebra g = {(A, b) ∈ R(d+p)×(d+p) o Rd+p }, the function F (y0 ) = (L(y0 ), N (y0 )) = (A, b) and the frozen vector field F(A,b) (y) = (A, b) ~ y = Ay + b. This is exactly the affine algebra action proposed in [16]. Note that such a representation of f(y) is always possible, for example by letting L(y) be the Jacobian of f at the point y and N(y) = f(y) − L(y)y. Other choices are also possible see for example [12,17]. In this paper we are interesting in the construction of Lie group methods for the following nonautonomous problem defined on Rd u0 = f (u, t),
u(t0 ) = u0 .
(2.4)
Formally it does not fit in the above presented framework, but by adding the trivial differential equation t0 = 1 to the system (2.4), we can rewrite it in the form (2.1) with p = 1 and
f (u, t) , f= 1
u y = . t
This of course is a very well known idea in the theory of ODEs, however its application to Lie group methods, if done carefully, can lead to some extra 95
freedom which we would like to exploit. Note that now the time variable t goes in the definition of the manifold M and therefore it also appears like one of the arguments of the generic function F , the frozen vector field F and its corresponding algebra action.
From a computational point of view it might look some what unreasonable to increase the dimensionality of the problem, but we keep in mind that the solution of (2.4) is given by the first d components of the solution of (2.1). Thus, the approach is to apply a Lie group method to equation (2.1) and then to restate it in Rd . The simplest nonautonomous case is when the Lie algebra g = {b ∈ Rd+1 } or [0] [0] d equivalently is given ³h i´ g = {(b , λ) : b ∈ R , λ ∈ R} then the generic function ³h i´ u0 [0] by F t0 = (f (u0 , t0 ), 1) = (b , 1), the frozen vector field is F(b[0] ,λ) ut =
h
b[0] λ
h i
i
h
[0]
i
and the algebra action is h(b[0] , λ) ∗ ut00 = ut00+hb . When f (u, t) = +hλ L(u, t)u + N (u, t) then the Lie algebra g = {(A, b) ∈ Rd+1×d+1 o Rd+1 }. It can also be represented like the set of all triples (A, b[0] , λ) closed under linear combinations and commutators between the elements (see section 3), where ³h A ∈i´Rd×d , b[0] ∈ Rd , λ ∈ R. In this case the generic function is defined like F ut00 = (L(u0 , t0 ), N (u0 , t0 ), 1) = (A, b[0] , 1), the frozen vector field ³h i´
u = ht i h(A, b[0] , λ) ∗ ut00 =
is F(A,b[0] ,λ)
h
i
Au+b[0] and its corresponding algebra action is given by h hAλ i [0] e u0 +hb φ[1] (hA) , where ehA denotes the matrix exponent0 +hλ [1]
tial and the function φ
is given in Lemma 2 (see section 3).
If we consider just the first d components of the algebra action: in the first case we simply obtain translations like basic motions on Rd ; in the second case we again recover the affine action. Thus, we conclude that for thes two cases the only difference between autonomous and nonautonomos systems is in the definition of the generic function F , which for nonautonomous systems depends also from the time variable. We remark that in the above two cases the frozen vector field does not really depends of t. This explains the observed similarities between autonomous and nonautonomous systems.
A more interesting situation arises when the function f (u, t) has the form L(u, t)u + N [0] (u, t) + tN [1] (u, t). A natural question now is how to choose the frozen vector field in this case. In the next section, we propose a time dependent frozen vector field and its corresponding algebra action which reflects the structure of f . In addition, we discuss how this idea can be further generalized if second and higher powers of t are included. 96
3
Nonautonomous frozen vector fields
We first consider the case when the vector field of (2.4) has the form f (u, t) = L(u, t)u + N [0] (u, t) + tN [1] (u, t). If we treat the nonlinear part as a single function N = N [0] + tN [1] then we do not gain anything in comparison with the affine case presented in the previous section. A more demanding task is to allow our frozen vector field to be time dependent. It is desirable in this case to approximate the nonlinear part of f (u, t) by a linear polynomial of t. The only way to include t in the definition of the frozen vector field is to append it to the dependent variables. Thus, by adding the trivial differential equation v 0 = 1, v(t0 ) = t0 to the system (2.4), we obtain y 0 = Ly + N,
y(t0 ) = y0 ,
(3.1)
where
L(u, t) N [1] (u, t) L= , 0 0
N [0] (u, t) N= , 1
u y = , v
u0 y0 = . t0
The advantage of rewriting (2.4) in the above form is that now we can easily see how to define the Lie algebra g, the generic function F , the frozen vector field F and its corresponding algebra action (see section 2). The Lie algebra in this case is g = {(A, b) ∈ Rd+1×d+1 o Rd+1 }, with
A A= 0
b[1] , 0
b[0] b= , λ
(3.2)
where A ∈ Rd×d , b[1] , b[0] ∈ Rd and λ ∈ R. Equivalently we can represent g like the quadruplet (A, b[1] , b[0] , λ) closed under linear combinations and commutators. The function F which provides the generic presentation is given by F
Ã"
u0 t0
#!
³
´
= L(u0 , t0 ), N [1] (u0 , t0 ), N [0] (u0 , t0 ), 1 = (A, b[1] , b[0] , 1).
(3.3)
To obtain an explicit form of the frozen vector field we use the following result. Lemma 1 The solution of the differential equation y 0 = Ay + b, y(t0 ) = y0 , at the time t0 + h is given by
u(t0 + h) y(t0 + h) = , t0 + hλ 97
where u(t0 + h) is the solution of u0 = Au + b[0] + t0 (1 − λ)b[1] + tλb[1] ,
u(t0 ) = u0 .
Proof: The proof follows directly by substituting (3.2) in the differential equation y 0 = Ay + b and then solving it with respect to the last variable. 2
Thus, we have obtained the following time dependent frozen vector field F(A,b[1] ,b[0] ,λ)
Ã" #!
u t
" #
"
#
u Au + c0 + tc1 = (A, b , b , λ) ~ = , t λ [1]
[0]
(3.4)
where c0 = b[0] + t0 (1 − λ)b[1] and c1 = λb[1] . Note that ³h fori´λ =h 1i weh havei [0] [1] c0 = b , c1 = b and therefore the generic presentation F ut ~ ut = f (u,t) 1 is satisfied. The last thing which we need to define is the algebra action corresponding to the frozen vector field (3.4). In the next Lemma we give a general formula for the flow of the frozen vector field, which approximates the nonlinear part of f by a polynomial of t of degree p. Lemma 2 The solution of the differential equation 0
u = Au +
p X
t j cj ,
u(t0 ) = u0 ,
j=0
where p ∈ N, A ∈ Rd×d and cj ∈ Rd at the time t0 + h is given by u(t0 + h) = ehA u0 +
p X
hk+1 δk φ[k+1] (hA),
k=0
where δk =
Pp
j−k j! j=k (j−k)! t0 cj ,
φ[1] (z) =
ez −1 z
and φ[k+1] (z) =
φ[k] (z)−φ[k] (0) . z
Proof: From the variation of constants formulae it follows that u(t0 + h)=ehA u0 + ehA =ehA u0 +
Z
p µ X
h
e−τ A
0
δj ehA
j=0
µX p
j=0
1 j!
Z
h
0
¶
(t0 + τ )j cj dτ ¶
e−τ A τ j dτ .
Multiple applications of integration by parts complete the proof. 98
2
Combining the results of Lemma 1 and Lemma 2 leads to the following explicit form for the algebra action corresponding to the vector field (3.4) ·
¸ · hA ¸ u0 e u0 + h(b[0] + t0 b[1] )φ[1] (hA) + h2 λb[1] φ[2] (hA) h(A, b , b , λ) ∗ = (3.5) t0 t0 + hλ [1]
[0]
Once we have defined the generic presentation, the Lie algebra g and its action on M we can use any Lie group method to find the solution of (2.4). The solution of (3.1) is simply given by its first d components. In the case when a Runge-Kutta Munthe-Kaas method with exact Exp map is used the format requires the inverse of the dExp map (see [16]). Computationally it might be very expensive to find exactly the dExp−1 map and thus the approach proposed in [16] is to replace it with polynomial approximation of order higher than the order of the method. This imposes the necessity of using [1] [0] commutators between the elements of g. In this case if Θi = (Ai , bi , bi , λi ) for i = 1, 2 are two elements from g then their commutator is given by ³
[1]
[1]
[0]
[0]
[1]
[1]
´
[Θ1 , Θ2 ] = [A1 , A2 ], A1 b2 − A2 b1 , A1 b2 − A2 b1 + λ2 b1 − λ1 b2 , 0 , where [A1 , A2 ] = A1 A2 − A2 A1 is the matrix commutator. The above approach can be easily generalized when the function f (u, t) = P L(u, t)u + pk=0 tk N [k] (u, t). In this case we append p trivial differential equations corresponding to t, t2 , . . . , tp to the system (2.4) . Thus, the dimension of the manifold is d + p, but we keep in mind that we are only interested in its first d components. The Lie algebra is g = {(A, b[p] , . . . , b[0] , λ) : A ∈ Rd×d , λ ∈ R, b[k] ∈ Rd } and its action upon the manifold is given by Lemma 2. The coefficients cj can be found in the same way as in Lemma 1. For p = 2 they are c0 =b[0] + (1 − λ)t0 b[1] + (1 − λ)2 t20 b[2] , c1 =λb[1] + 2λ(1 − λ)t0 b[2] , c3 =λ2 b[2] . We conclude this section with the observation that based on the same idea, methods with approximations of the nonlinear part of f by trigonometric polynomials can also be derived. In this case, the exact flow of the frozen vector field can be computed in the similar manner (see [14]). 99
4
Exponential integrator for semilinear problems
In this section we derive an exponential integrator based on the frozen vector field (3.4) and its corresponding algebra action (3.5) for the semilinear problem u0 = Lu + N (u, t),
u(t0 ) = u0 ,
(4.1)
where L is a constant linear term and N is a nonlinear term. Such systems often arise after the spatial discretization of certain PDEs. Comparisons between the stability regions for different Lie group methods applied to semi-discretized stiff PDEs is given in [11]. There the author suggests that for this type of problem the best methods are likely to be the commutator free Lie group methods [4]. This provides our motivation in the choice of the Lie group method. Next we give an equivalent formulation of the method proposed in [4]. This formulation allows us to construct methods without knowing what the exact structure of the Lie group acting on the manifold is, or how the Exp map between the Lie algebra and the Lie group is defined. The general format of an s stage commutator free Lie group method advancing from point yn to point yn+1 with a time step of size h is given by the following algorithm. Algorithm 1 (Commutator-free Lie group method) for i = 1, . . . , s do P P k k Ui = (h sk=1 αiJ Fk ) ∗ · · · ∗ (h sk=1 αi1 Fk ) ∗ yn Fi = F (Ui ) end P P un+1 = (h sk=1 βJk Fk ) ∗ · · · ∗ (h sk=1 β1k Fk ) ∗ yn
k Here the function F gives the generic presentation (2.3), the coefficients αij , βjk are parameters of the method and the value J counts the number of flow calculations required at each stage. In [4], the following fourth order method based on the classical fourth order method of Kutta is proposed.
0 1 2 1 2 1 2 1 2
1 2
0
1 2
1 2 - 12
0 0
1 4 1 12
1 6 1 6
0 1 1 6 1 6
(4.2)
1 - 12 1 4
We use the symbol } to denote all the substages included in a stage with 100
J > 1. Note that in (4.2) the frozen vector field corresponding to the second stage is the same as for the first substage of the fourth stage. This reduces the cost of the method. In order to use the frozen vector field (3.4) from the previous section we rewrite the nonlinear part of (4.1) in the form N (u, t) = Nn + t
N (u, t) − Nn = N [0] + t N [1] , t
(4.3)
where Nn = N (un , tn ) is the value of the nonlinear part at the beginning of the step number n. Keeping in mind (3.3) and (3.5), based on (4.2), we have found a new fourth order exponential integrator, which written in the original u variable is given by U1 = un , hL U2 = e 2 un + h 12 φ[1] Nn , U3 = e U4 = e Ub = e
un+1 = e [1]
hL 2 hL 2
hL 2
h
un + h U2 + h un + h
hL b 2 U
where Nj = functions are
+h
h
·
·
³
tn [1] φ 2
´
[1]
+ h4 φ[2] N2
i
1 [1] φ Nn 2
+
1 [1] φ Nn 2
+ tn φ[1] + h2 φ[1] + h2 φ[2] N3
³
³
´µ
,
[1] N2
1 [1] φ Nn 2
+ tn φ[1] + h2 φ[2]
1 [1] φ Nn 2
+ tn φ[1] + h2 φ[1] + h2 φ[2]
³
N (Uj , tn +cj h)−Nn tn +cj h hL . 2
6
´
+
[1]
[1] N3
´µ
i
6
,
−
[1]
N2 6
[1] N4
12
¶¸
(4.4) ,
[1]
+
N3 6
[1]
+
N4 4
¶¸
,
for j = 1, . . . , 4 and the arguments of all the φ[j]
It is possible to rewrite (4.4) in equivalent form which does not involves splitting of the internal stages (see [1,14]). Such a representation is rather useless, since its implementation is more expensive than the one proposed, but it shows that (4.4) is a method based just on the pure Runge–Kutta idea. If we represent the nonlinear part of (4.1), as a polynomial of second degree N (u, t) = Nn + t
Nn − Nn−1 N (u, t) − 2Nn + Nn−1 + t2 , t t2
where Nn and Nn−1 are the values of N at the end of step number n and n − 1 respectively, we obtain a method which fits into the framework of general linear methods [2]. Thus, we see that by just changing the algebra action, any Lie group method based on a pure Runge–Kutta method can result in a more general method. This is a very interesting phenomena which highlights the important role of the algebra action. 101
5
Numerical experiments
In this section we present results from numerical experiments on the KuramotoSivashinsky and Allen-Cahn equations. For both examples we compare the following four methods: • IF4 The fourth order integrating factor method [5,10,18] based on the classical fourth order method of Kutta. • CF4 The fourth order commutator free Lie group method (4.2) with affine algebra action [4]. • CF4A1 The method (4.4) with algebra action given by (3.5). • ETDRK4B The method of Krogstad [10]. Since all of the above methods are based on the nonstiff order conditions, to avoid possible order reduction, we consider examples where the nonlinear term N has sufficient spatial regularity. In general, for applications concerning PDEs, the classical order of convergence is not always obtained. Order reduction, due to the lack of sufficient temporal and spatial smoothness, should be expected. For parabolic problems, full order of convergence can be observerd, if periodic boundary conditions are imposed [6,7]. To avoid problems with numerical instability, the computation of the φ[i] functions, which suffer from cancelation errors when the eigenvalues of the discretized linear operator are close to zero, we use the approach of Kassam and Trefethen [9]. The idea is to evaluate the φ[i] functions by Cauchy’s integral formula 1 Z [i] φ[i] (γhL) = φ (γλ) (λI − hL)−1 dλ, (5.1) 2πi Γ where γ ∈ R. The contour Γ is a closed curve in the complex plane that encloses the eigenvalue of γhL and such that γΓ is well separated from zero. The trapezoidal rule is then used to approximate the integral in (5.1). If the discretized linear operator L is diagonal (Kuramoto-Sivashinsky equation) then the integral reduces simply to the mean of φ[i] over the contour Γ. However, for non-diagonal problems (Allen-Cahn equation), the computations become more expensive and require the computation of several matrix inverses. That is why for such problems it is important for a method to use as few φ function evaluations as possible. In addition, if L has a special sparse structure one can apply effective methods to find it inverse [13,14]. The Kuramoto-Sivashinsky equation The first example is the Kuramoto-Sivashinsky equation ut = −uux − uxx − uxxxx , 102
x ∈ [0, 32π]
Kuramoto−Sivashinsky equation
1
10
0
10
−1
Relative error at time = 65
10
−2
10
−3
10
−4
10
−5
10
IF4 ETDRK4B CF4 CF4A1
−6
10
−7
10
−2
−1
10
0
10 Stepsize h
10
Fig. 1. Step size versus relative error for fourth order methods for the Kuramoto-Sivashinsky equation
with periodic boundary conditions and with the initial condition borrowed from [9] µ ¶µ µ ¶¶ x x u(x, 0) = cos 1 + sin . 16 16 A 128-point Fourier spectral discretization in space is used. Since the boundary conditions are periodic the transformed equation in the Fourier space can be represented in the form (4.1), the linear and nonlinear parts are defined as (Lˆ u)(k) = (k 2 − k 4 )ˆ u(k),
N (ˆ u) = −
ik (F((F−1 (ˆ u))2 )), 2
where F denotes the discrete Fourier transform. The integration in time is done entirely in the Fourier space until t = 65. The results for the four different numerical schemes are plotted in Figure 1.
The Allen-Cahn equation The second example is the Allen-Cahn equation written in the form ut = εuxx + u − u3 ,
x ∈ [−1, 1],
where ε = 0.01 and with boundary and initial conditions also borrowed from [9] u(−1, t) = −1,
u(1, t) = 1,
u(x, 0) = 0.53x + 0.47 sin(−1.5πx). 103
Allen−Cahn equation
−2
10
−3
10
−4
Relative error at time = 31
10
−5
10
−6
10
−7
10
−8
10
IF4 ETDRK4B CF4 CF4A1
−9
10
−10
10
−2
−1
10
10
0
10
Stepsize h
Fig. 2. Step size versus relative error for fourth order methods for the Allen-Cahn equation
After discretization in space based on the Chebyshev grid points we can rewrite the equation in the form (4.1), with L = εD2 ,
N (u) = u − u3 ,
where D is the Chebyshev differentiation matrix [18], this means the matrix L is full. The standard build in MATLAB function inv was used to find the matrix inverse in (5.1). The integration in time is until t = 31. In Figure 2 we have plotted the results for the four different numerical schemes.
For both examples we see that all the methods exhibit the expected fourth order, but the best with respect to the accuracy is the ETDRK4B method. For the Kuramoto-Sivashinsky equation the improvement of using the algebra action (3.5) in CF4A1 is small in comparison with the affine action in CF4. However, for the non-diagonal example, the CF4A1 performs significantly better than CF4 and it is competitive with the ETDRK4B method. This together with the fact that it uses only 1 exponential and 2 φ function evaluations per step (for comparison ETDRK4B uses 2 exponentials and 5 φ function evaluations per step) suggests that CF4A1 is the best method in this case. 104
6
Concluding remarks
In this paper we have introduced the use of time dependent frozen vector fields in the construction of Lie group integrators for nonautonomous problems. This approach, provides extra freedom in the choice of the algebra action and allows us to choose the basic motions on the manifold to be given by the solutions of differential equations, which better approximate the flow of the original vector field. Based on this idea we have derived a new fourth order exponential integrator for semilinear problems with constant linear part. We do not claim that the proposed representation (4.3) of the nonlinear part is optimal. Other choices are worth investigating. However, the results from the numerical experiments suggest that the new method based on (4.3) is efficient in the case when the discretized linear operator is non-diagonal and a variable step size strategy is used. The content of this paper poses many questions which need to be answered. For example what are the stability regions of such methods and to what extend the spatial regularity of the problem effects the overall order of the method. An other important question is, for a given problem, how do we find a good algebra action? The goal is to choose a differential equation which is easier to solve, but still captures the key features of the original one. This is a very challenging task and it is likely to be problem dependent. The option presented here is to approximate the vector field by a higher order polynomial with constant coefficients. However, we should keep in mind that there is a certain balance between the benefit provided by increasing the order of the approximation and the computational cost of its corresponding algebra action.
Acknowledgments The author would like to thank Hans Munthe-Kaas and William Wright for useful discussions throughout the work on this paper. This work has been partially supported by Norwegian Research Council through the GI4PDE project, contract number 142955/431.
References [1] H. Berland and B. Owren, Fourth order exponential time integrators for the nonlinear Scr¨ odinger equation, private conversation, April, 2004. [2] J. C. Butcher, Numerical methods for ordinary differential equations, John Wiley & Sons, 2003. [3] E. Celledoni, Eulerian and Semi-Lagrangian schemes based on commutator free
105
exponential integrators, Preprint submitted to GI Preprint Server, 2003. [4] E. Celledoni, A. Marthinsen, and B. Owren, Commutator-free Lie group methods, FGCS 19(3) (2003), 341–352. [5] B. Fornberg and T.A. Driscoll, A fast spectral algorithm for nonlinear wave equations with linear dispersion, J.Comp. Phys. 155 (1999), 456–467. [6] M. Hochbruck and A. Osterman, Explicit exponential Runge-Kutta methods for semilinear parabolic problems, Submitted to SIAM J. Numer. Anal., 2004. [7]
, Exponential Runge-Kutta methods for parabolic problems, To appear in Appl. Numer. Math., 2004.
[8] A. Iserles, H. Munthe-Kaas, S.P. Nørsett, and A. Zanna, Lie-group methods, Acta Numerica 9 (2000), 215–365. [9] A.-K. Kassam and L.N. Trefethen, Fourth order time stepping for stiff PDEs, SIAM J. Sci. Comp., to appear, 2003. [10] S. Krogstad, Generalized integrating factor methods for stiff PDEs, Preprint submitted to J. Comput. Phys., 2003. [11]
, Topics in numerical Lie group integration, Ph.D. thesis, University of Bergen, 2003.
[12] E. Lodden, Geometric integration of the heat equation, Master’s thesis, University of Bergen, 2000. [13] B. Minchev, Some algorithms for solving special tridiagonal block teoplitz linear systems, J. Comp. and Appl. Math. 156(1) (2003), 179–200. [14]
, Exponential integrators for semilinear problems, Ph.D. thesis, University of Bergen, 2004.
[15] H. Munthe-Kaas, Runge-Kutta methods on Lie groups, BIT 38 (1998), 92–111. [16]
, High order Runge-Kutta methods on manifolds, Appl. Num. Math. 29 (1999), 115–127.
[17] A. Suslowicz, Application of numerical Lie group integrators to parabolic PDEs, Technical report 219, University of Bergen, 2001. [18] L. N. Trefethen, Spectral methods in Matlab, SIAM, 2000.
106
Article 2 Some Algorithms for Solving Special Tridiagonal Block Teoplitz Linear Systems Published in J. Comp. and Appl. Math. 156(1), 179-200, 2003
107
108
Some Algorithms for Solving Special Tridiagonal Block Teoplitz Linear Systems Borislav V. Minchev Department of Computer Science, University of Bergen, N-5020 Bergen, Norway
Abstract This paper is focused on different methods and algorithms for solving tridiagonal block Teoplitz systems of linear equations. We consider the El-Sayed method [4] for such systems and propose several modifications that lead to different algorithms, which we discuss in detail. Our algorithms are then compared with some classical techniques as far as implementation time is concerned, number of operations and storage. Comments and conclusions for computing efficiency of the proposed new algorithms are given. Numerical experiments corroborating the theoretical results are also presented. Key words: linear system, block Teoplitz matrix, matrix equation, Woodbury’s formula 1991 MSC: 65F10
1
Introduction
Many problems arising in practice lead to the solution of linear systems of equations with special coefficient matrices. Tridiagonal block Teoplitz linear systems arise in numerical solution of ordinary and partial differential equations (ODE and PDE), interpolation problems, boundary value problems (BVP), etc. [2,3,7,13]. It is known that these systems have the form M x = f, where Email address:
[email protected] (Borislav V. Minchev). URL: http://www.ii.uib.no/∼borko (Borislav V. Minchev).
Published in J. Comp. and Appl. Math., 156(1), 179-200, 2003
(1)
A B ∗ 0 B A B ∗ B A . . .. M = , ... .. . 0 . . B B∗ A
(2)
is an Hermitian tridiagonal block Teoplitz matrix with block size n. A and B are m × m matrices, x and f are column vectors with size nm . The aim of this paper is to discuss different algorithms for solving (1) and compare them as far as time for implementation, number of operations and storage are concerned. We organize the present paper as fallows: 1. In Section 2 we review LU factorization, Cholesky factorization [8] and adaptation the Cyclic Reduction method [1] corresponding to the special form of M . We modify the method described in [4]. 2. In Section 3 we develop algorithms based on these modifications in order to optimize floating point operations, memory space and implementation. 3. Finally we verify the results in a number of numerical experiments.
2
Methods for solving special block tridiagonal Teoplitz linear system
Let us recall some classical direct methods for solving the linear system (1): LU factorization, Cholesky factorization, Cyclic Reduction [8], as well as one modification of LU factorization described in [4], which is based on the solution of a nonlinear matrix equation. For simplicity’s we introduce the following notation x = {xi }i=1,...,n , f = {fi }i=1,...,n , where xi and fi are blocks with size m × 1.
2.1 Block LU factorization
The matrix (2) admits following LU factorization M = L U, 110
where
A1 B1 A2 B2 . .. L = .. 0 .
0
. Bn−1
F 1 G1 F 2 G2 0 . . .. , U = , . . 0 . Gn−1 An Fn
where by, the matrices Ai , Bi , Fi and Gi satisfy the relations A1 F1 B1 = G1 = Ai Fi Bi = Gi = An Fn
= A B ∗ F1−1 A−1 1 B = A − Bi−1 Gi−1 B ∗ Fi−1 A−1 B i = A − Bn−1 Gn−1
(3) f or i = 2, . . . , n − 1.
The matrices Ai and Fi are lower and upper triangular respectively and are obtained by LU factorization. Thus, solving the linear system (1) is equivalent to solving two simpler systems L y = f,
y = {yi }i=1,...,n
U x = y,
x = {xi }i=1,...,n ,
and whose solutions are y1 = A−1 1 f1 −1 yi = Ai (fi − Bi−1 yi−1 ),
f or i = 2, . . . , n
(4)
and xn = Fn−1 yn xi = Fi−1 (yi − Gi xi+1 ),
f or i = n − 1, . . . , 1
respectively.
2.2 Block Cholesky factorization
When the matrix M is positive definite the following factorization M = L L∗ , 111
(5)
exist, where
A1 B1 A2 B2 . .. L = .. 0 .
0
. Bn−1 An
The matrices Ai , Bi satisfy the relations A1 A∗1 B1 = Ai A∗i Bi = An A∗n
= A B ∗ (A∗1 )−1 ) ∗ = A − Bi−1 Bi−1 B ∗ (A∗i )−1 ∗ = A − Bn−1 Bn−1
.
f or i = 2, . . . , n − 1.
(6)
It is well known that Ai and A∗i are lower and upper triangular respectively and are obtained by Cholesky factorization. In this manner, solving the linear system (1) is again equivalent to solving two simpler systems L y = f, y = {yi }i=1,...,n and L∗ x = y,
x = {xi }i=1,...,n .
The solution of the first system can be found by (4). The solution of the second system satisfies xn = (A∗n )−1 yn xi = (A∗i )−1 (yi − Bi∗ xi+1 ),
f or i = n − 1, . . . , 1.
(7)
2.3 Block Cyclic Reduction
In this section we adapt Bini’s method [1] to the special case when the coefficient matrix is given as in (2). Let us derive explicitly the substitution formulas for computing the block coordinates of the solution x. Recall that Block Cyclic Reduction can be applied only if the block size of M is power of 2 in other words let n = 2p . By performing an even-odd permutation of the block-rows and block columns in (1) we obtain
(0)
D1 L(0)
∗
(0)
L(0) D2
(0)
x+
(0)
x−
112
(0)
f+
(0)
f−
,
(8)
where (0)
(0)
(0)
(0)
(0)
(0)
(0)
(0)
x+ = {x+k }k=1,...,2p−1 , x− = {x−k }k=1,...,2p−1 , f+ = {f+k }k=1,...,2p−1 , f− = {f−k }k=1,...,2p−1 are column vectors, whose elements are blocks of size m × 1, satisfying the relations (0)
(0)
(0)
(0)
f or k = 1 . . . 2p−1 .
x+k = x2k , x−k = x2k−1 , f+k = f2k , f−k = f2k−1 The cells
(0)
(0)
D1 = D 2
A
. 0 , L(0) . = . 0
A
B ∗ B . 0 . .. = . . 0 ∗ B B
are matrices of block size 2p−1 × 2p−1 . We apply one step of block-Gaussian elimination to (8) and obtain
Let
¯ −1 −1 (0) ¯ ∗ (0) (0) (0) ¯ (D2(0) − L(0) D1(0) L(0) ) x(0) = f − L D f+ − − 1 ¯ ¯ ∗ (0) ¯ (0) (0)−1 (0) ¯ x+ = D 1 (f+ − L(0) x− ). (0)−1
(0)
M (1) = D2 − L(0) D1
L(0)
(9)
∗
(0)
x(1) = x−
(0)−1
(0)
f (1) = f− − L(0) D1
(0)
f+ .
Note that now the first equation of (9) has the form M (1) x(1) = f (1) .
(10)
Observe, that the matrix M (1) has the form F (1) B (1) (1)∗ (1) A . B . .. = .. .
M (1)
. . ∗
, (1) B
B (1) A(1)
where F (1) = A − BA−1 B ∗ . Obviously, it is also a block tridiagonal matrix and, except for the north-western corner block F (1) it has a block Teoplitz 113
structure, with block size 2p−1 × 2p−1 . Applying once again an even-odd permutation of the block rows and block columns to (10) , we obtain
(1)
∗
(1)
(1) D2
D1 L(1)
L
(1)
x+
(1) x−
(1)
f+
(1) f−
,
(11)
where (1)
(1)
(1)
(1)
(1)
(1)
(1)
(1)
x+ = {x+k }k=1,...,2p−2 , x− = {x−k }k=1,...,2p−2 , f+ = {f+k }k=1,...,2p−2 , f− = {f−k }k=1,...,2p−2 are column vectors, whose elements are blocks of size m × 1 and satisfy the relations (1)
(1)
(1)
(1)
(1)
(1)
(1)
(1)
f or k = 1, . . . , 2p−2 .
x+k = x2k , x−k = x2k−1 , f+k = f2k , f−k = f2k−1 , Again the cells
(1)
D1
A(1)
= 0
. . .
0
A
L(1)
(1)
, D (1) 2
B (1) (1)∗ B . = .. 0 .
F (1)
= 0
A(1) . .
0
A(1)
,
0
. B
(1)∗
B (1)
are matrices of block size 2p−2 × 2p−2 . We apply again one step of Gaussian elimination to (11) and obtain M (2) x(2) = f (2) , where M (2) is of the same type as M (1) , but with block size 2p−2 × 2p−2 . Proceeding in a similar fashion, we obtain a sequence of linear systems of the form M (j) x(j) = f (j) f or j = 1, . . . , p, where
F (j) B (j) (j)∗ (j) A . B . .. = .. .
M (j)
114
. . ∗
(j) B
B (j) A(j)
is square matrix with block of size 2p−j for j = 1, . . . , p. When j = p the cells M (p) = F (p) . The blocks of the matrix M (j) obey the following relations: −1
B (j) = − B (j−1) A(j−1) B (j−1) , (j) (j−1) (j−1)∗ (j−1)−1 (j−1) A = A −B A B f or j = 1, . . . , p, −1 ∗ −B (j−1) A(j−1) B (j−1) , −1 ∗ F (j) = F (j−1) − B (j−1) A(j−1) B (j−1) ,
(12)
where A(0) = A, B (0) = B, F (0) = A. For the block column vectors f (j) and x(j) , we have (j−1)
f (j) = f− (j)
f +k
(j)
and
(j−1) (j−1) f+ ,
− L(j−1) D1
(j) = f2k ,
f or k = 1, . . . , 2p−j−1 ,
(j)
f−k = f2k−1 (j−1)
= x(j) ,
(j−1)
= D1
(j−1)
= x +k ,
x− x+
x2k
(j−1)
(j−1)−1
(j−1)
(j−1)
x2k−1 = x−k
f or j = 1, . . . , p
(j−1)
(f+
f or j = 0, . . . , p − 1
∗ (j−1) (j−1) −L x− ),
f or j = p, . . . , 1,
(13)
(14)
p−j−2 f or k = 1, . . . , 2 , −1
where f (0) = f, x(p) = F (p) f (p) , x(0) = x. For j=0,. . . , p-1 the cells
(j)
D1
A(j)
= 0
. . .
0
A(j)
, L(j)
B (j) (j)∗ B . = .. . 0
0
. ∗ B (j) B (j)
are square matrices of block size 2p−j−1 . 2.4 A modification of LU factorization In 1990 Rojo [14] proposed a new method for solving symmetric circulant tridiagonal linear systems and in recent years it has been modified to dial with matrices M having a special structure. For instance in [5] it is adapted to the case when the coefficient matrix M is pentadiagonal and strongly diagonally dominant. In [11] M is allowed to be not diagonally dominant. El-Sayed [4] extended Rojo’s method to tridiagonal block matrices. His approach consists 115
in introducing a nonlinear matrix equation to solving the problem (1). The algorithms we propose in this paper are based on [4] and investigate different approaches for solving the nonlinear matrix equation of El-Sayed. We discuss Woodbury’s formula and its numerical implementation. Firstly, let us describe an algorithm for solving parametric linear systems of the form. N y = f, (15) where
X B ∗ B A . 0 . .. ... N = , .. . 0 . . B B∗ A is a block tridiagonal matrix with block size n, X is a parameter block of size m × m, and the vectors y = {yi }i=1,...,n , and f = {fi }i=1,...,n are column vectors consisting of n blocks of size m × 1. The matrix N admits the following LU factorization
I ∗ −1 B X . .. N = LU = 0 .
. B ∗ X −1
X B 0 . . 0 .. , . B 0 I X
where I is the m × m identity matrix. The above factorization exists when the parameter X satisfies the nonlinear matrix equation X + B ∗ X −1 B = A.
(16)
Thus, solving the linear system (15) is equivalent to solving two simpler systems L z = f, z = {zi }i=1,...,n (17) U y = z, y = {yi }i=1,...,n , whose solutions are z1 = f 1 zi = fi − B ∗ X −1 zi−1 , yn = X −1 zn yi = X −1 (zi − Byi+1 ),
i = 2, 3, . . . , n,
(18)
i = n − 1, n − 2, . . . , 1,
respectively. Now, we can find the solution of (1). The matrices M and N are related by relation M = N + E1 V1T , 116
where E1
I 0 = .. , .
V1T =
³
´
A − X 0 ... 0 .
0 Using Woodbury’s formula we have
M −1 = N −1 − N −1 E1 (I + V1T N −1 E1 )−1 V1T N −1 . Therefore, the solution x of (1) is obtained from the vector y as follows x= = = =
M −1 f N −1 f − N −1 E1 (I + V1T N −1 E1 )−1 V1T N −1 f y − N −1 E1 (I + (A − X)E1T N −1 E1 )−1 V1T y y − N −1 E1 (I + (A − X)E1T N −1 E1 )−1 (A − X)y1 .
(19)
El-Sayed proposes following decomposition N = LDV, for computing N −1 E1 and E1T N −1 E1 , where
I P . 0 , .. L= 0 . . P I
I Q . . 0 .. V = , . Q 0 I
D = diag(X, X, . . . , X) are square matrices of block size n, P = B ∗ X −1 and Q = X −1 B. The matrix N −1 becomes N −1 = V −1 D−1 L−1 . If we denote L−1 = (Lij ) and V −1 = (Vij ), then Lij =
(
0 ij j−i j−i (−1) Q i ≤ j,
D−1 = diag(X −1 , X −1 , . . . , X −1 ). Therefore, the blocks (N −1 E1 )i of the vector N −1 E1 satisfy the formulas (N −1 E1 )i = (V −1 D−1 L−1 E1 )i P = ns=i (−1)i+1 Qs−i X −1 P s−1 117
f or i = 1, . . . , n.
(20)
Hence E1T N −1 E1 = (N −1 E1 )1 =
n X
Qs−1 X −1 P s−1 .
(21)
s=1
The coordinates xi for i=1, . . . , n of the vector x in (19) are given by xi = yi − [ ns=i (−1)i+1 Qs−i X −1 P s−1 ] × P −1 × [I + (A − X) ns=1 (−1)i+1 Qs−1 X −1 P s−1 ] (A − X)y1 . P
Obviously formulas (20) and (21) are not convenient to implement directly, because they require a great number of redundant multiplications. For this reason we propose two algorithms for computing the vector N −1 E1 . Algorithm F. Solve m linear systems of type (17) with right-hand sides the corresponding to different columns of E1 . Note that this approach does not take into consideration the special structure of the right-hand sides vectors ( having only a very sparse nonzero block). If the matrices A and B are real, the algorithm costs O(8nm3 ) flops and requires the storage of nm2 real numbers. Algorithm R. The blocks (N −1 E1 )i are recursively computed by the formula (20) using the following algorithm: • Find the cells Yi = (−1)n−i X −1 P n−i , Y1 = X −1 Yi = Yi−1 (−P ),
f or i = 1, . . . , n
by
f or i = 2, . . . , n.
• Compute the blocks (N −1 E1 )i by (N −1 E1 )n = Yn (N −1 E1 )i = Yi − Q(N −1 E1 )i+1 ,
f or i = n − 1 ÷ 1.
Our theoretical investigation shows that the algorithm R: 1. Requires half as many flops as the algorithm F, at the expense of minimal increase of storage memory. If the matrices A and B are real the algorithm costs O(4nm3 ) flops and needs to store (n + 2)m2 real numbers; 2. Takes advantage of the special form of the matrix E1 .
3
Algorithms for solving special block tridiagonal linear systems
In this section we compare the algorithms for solving special block tridiagonal linear systems described in Section 2 and there modifications. 118
3.1 Algorithm LU 1. Find the matrices Ai , Bi , Fi and Gi according to (3). 2. Solve the system L y = f by (4). 3. Solve the system U x = y by (5). end. If the matrices A and B are real, this algorithm requires O( 15n m3 + 3 n 3 m + n3 m3 ) = O( 17n m3 ) flops and the storage of (3n + 1)m2 + 2nm real 3 3 numbers. 3.2 Algorithm CHOL 1. Find the matrices Ai and Bi according to (6). 2. Solve the system L y = f by (4). 3. Solve the system L∗ x = y by (7). end. m3 + If the matrices A and B are real, this algorithm requires O( 11n 3 n 3 m + n3 m3 ) = O( 13n m3 ) flops and the storage of 3n+4 m2 + 3nm real numbers. 3 3 2 2 3.3 Algorithm CR (Cyclic Reduction) 1. Find the matrices A(j) , B (j) and F (j) 2. Compute the vectors f (j) 3. Solve the linear system F
for (p)
x
(p)
for
j = 1, . . . , p by (12).
j = 1, . . . , p by (13). = f (p) .
4. Restore the coordinates of the vector x according to (14). end. If the matrices A and B are real, this algorithm requires O([18p+2]m3 + [4pm3 + 4 ∗ 2p m2 ] + 2m3 + 6 ∗ 2p m2 ) = O([22p + 4]m3 + 10 ∗ 2p m2 ) flops and the storage of (2p + 9)m2 + (8 ∗ 2p − 6)m real numbers, where p = log2 n. Some identical computations are imposed by program realization on formulas (j) (12) and (13). It is clear, that the vectors f− are not used in the next calculations. Based on this consideration and according to the special structure of (j) (j) the cells D1 , D2 and L(j) the next new algorithm - modification of Cyclic Reduction is developed. It needs less flops and less storage. 3.4 Algorithm CRM (Cyclic Reduction-Modification) (j)
1. Find the matrices A(j) , B (j) , F (j) and the vectors f+ by the scheme 1.1 Put 119
A(0) = A, B (0) = B, F (0) = F, f−k = f2k−1 , f or k = 1, . . . , 2p−1 . 1.2 For j = 1, 2, . . . , p − 1
compute
(j−1)−1
W1 = B (j−1) A , (j−1)∗ (j−1)−1 A , W2 = B (j−1)∗ W3 = W 1 B , A(j) = A(j−1) − W2 B (j−1) − W3 , B (j) = − W1 B (j−1) , F (j) = F (j−1) − W3 , (j−1) f 1 = f −1 − W 1 f +1 , (j−1)
(j−1)
fk = f−k − W2 f+k−1 − W1 f+k f −k (j)
,
= f2k−1
f+k = f2k
f or k = 2, . . . , 2p−j , f or k = 1, . . . , 2p−j−1 .
1.3 Find
−1
W1 = B (p−1) A(p−1) , ∗ F (p) = F (p−1) − W1 B (p−1) , (p−1) f 1 = f −1 − W 1 f +1 . 2. Solve the linear system F (p) x1 = f1 . 3. Retrieve the coordinates of the vector x by the scheme x2 = A(p−1)
−1
x+k = A(j−1) x+s = A
−1
(j−1)−1
x2k−1 = xk ,
h
(p−1)
f +1
h
h
(j−1)
f +k
(j−1) f +s
i
∗
− B (p−1) x1 , i
∗ − B (j−1) xk − B (j−1) xk+1
f or k = 1, . . . , 2p−j − 1
−B
i
(j−1)∗
x2k = x+k
end.
p−j
j = p−1, . . . , 1
xs , s = 2 p−j f or k = 2 , . . . , 1
If the matrices A and B are real, this algorithm requires O([12pm3 + 4 ∗ 2p m2 ] + 2m3 + 6 ∗ 2p m2 ) = O([12p + 2]m3 + 10 ∗ 2p m2 ) flops and the storage of (2p + 9)m2 + (4 ∗ 2p − 1)m real numbers, where p = log2 n. For the new algorithms based on the discussion in section 2.4, we must address the problem of solving the nonlinear matrix equation (16). In [6], Engwerda proves that, if A is a positive definite matrix, then solution of (16) is equivalent to the solution of following matrix equation ˜ = I, Z + B˜∗ Z −1 B ˜ = A− 12 BA− 12 , where B
1
1
(22)
Z = A− 2 XA− 2 . Thus, the results proposed in 120
[6,10] can be readily adapted to produce following algorithms for (16). Let ε > 0 be a fixed tolerance. Algorithm EI_γ (Engwerda; Ivanov) ˜ = A− 21 BA− 12 . 1. Find the matrix B 2. Solve Equation (22) by the following algorithm: 2.1 Z0 = γI, ( 21 ≤ γ ≤ 1). 2.2 For k = 1, 2, . . . compute ˜ ∗ Z −1 B, ˜ Zk+1 = I − B k ˜ ∗ Z −1 B ˜ − Ik∞ ≤ ε, then stop if k Zk − Zk+1 k∞ = k Zk + B k 2.3 Zk → Z+ , where Z+ is maximal solution of (22). 1
1
3. Compute the solution X = A 2 Zk+1 A 2 . end. If the matrices A and B are real, this algorithm requires O(8m3 +[2m3 + 2m + 2m3 ]k + 4m3 ) = O([6k + 12]m3 ) flops, where k is number of iterations for solving (22). The algorithm needs to store 7m2 real numbers. 3
Since for each k, Zk are positive definite matrices, can modify the above algorithms using the idea proposed by Zhan [15]. EI_γM (Engwerda; Ivanov - Modification) ˜ = A− 12 BA− 12 . 1. Find the matrix B
Algorithm
2. Solve the equation (22) as fallows: 2.1 Z0 = γI, ( 21 ≤ γ ≤ 1). 2.2 For k = 1, 2, . . . ˜L ˜ ∗, ◦ Compute the Cholesky factorization of Zk , Zk = L ˜ Z˜ = B, ˜ ◦ Solve the triangular matrix equation L ˜ ◦ Compute Zk+1 = I − Z˜ ∗ Z, ˜ ∗ Z −1 B ˜ − Ik∞ ≤ ε, then stop if k Zk − Zk+1 k∞ = k Zk + B k 2.3 Zk → Z+ . 1
1
3. Compute X = A 2 Zk+1 A 2 . end. 3
If the matrices A and B are real, this algorithm requires O(8m3 + [ m3 + 3 4m + 2m3 ]k + 4m3 ) = O([ 11 k + 12]m3 ) flops, where k is number of iterations. 3 3 2 The algorithm stores 17m2 +m real numbers. In case we wish to solve (16) by a direct solver, we can use the following adaption of the algorithm presented in [12]. 121
Algorithm M (Meini) 1. Set X0 = A, A0 = A, B0 = B. 2. For k = 1, 2, . . . compute W = A−1 k Bk , Bk+1 = Bk W, W = Bk∗ W, ∗ Ak+1 = Ak − Bk A−1 k Bk − W,
Xk+1 = Xk − W, if k Xk+1 − Xk k∞ ≤ ε for ε > 0, then stop. 3. Xk → X+ , where X+ is maximal solution of (16). end. If the matrices A and B are real, this algorithm requires O(2m3 + 5 ∗ 2m3 ) = O(12m3 ) flops per iteration and the storage of 7m2 real numbers. In analogy with the algorithm EI_γM we can consider the following modification of Meini’s algorithm. Algorithm MM (Meini - Modification) 1. Set X0 = A, A0 = A, B0 = B. 2. For k = 1, 2, . . . ˜L ˜ ∗, 2.1 Compute the Cholesky factorization of Ak , Ak = L 2.2 Solve the triangular matrix equations ˜ Y˜ = Bk , L ˜ Z˜ = Bk∗ , L 2.3 Compute W = Y˜ ∗ Y˜ , Bk+1 = Z˜ ∗ Y˜ , Ak+1 = Ak − Z˜ ∗ Z˜ − W, Xk+1 = Xk − W, if k Xk − Xk+1 k∞ ≤ ε for ε > 0, then stop. 3. Xk → X+ . end. 3
3
If the matrices A and B are real, this algorithm requires O( m3 + 8m3 +3∗ 3 2 2m3 ) = O( 27m ) flops, per iteration and the storage of 19m2 +m real numbers. 3 122
The algorithms EI_γM and MM require less operations per iteration at the expense of a minimal increase of the memory space. However, their MATLAB implementation shows that they are slower than EI_γ and M respectively. This is due because they call fewer built in MATLAB function. The advantage of the algorithms M and MM is their quadratic convergence, which guarantees fewer iterations to reach the required accuracy. Based on the discussion presented in this section, we propose the following new algorithm for (1). 3.5 Algorithm α(β) 1. Solve the matrix equation (16) by algorithm α, where α ∈ {EI_γ, EI_γM, M, M M }. 2. Find the vector y = N −1 f by formulas (18). 3. Compute the matrix N −1 E1 by algorithm β, where β ∈ {F, R}. 4. Compute the solution x of (1) by formula (19) with successive calculation of the expressions: C = (A − X)(N −1 E1 )1 ; (I + C)−1 ; (A − X)y1 ; z = (I + C)−1 (A − X)y1 ; x = y − N −1 E1 z. end. For real matrices A and B, this algorithm requires O(k ∗oα +8nm2 +oβ +6m3 ) flops and the storage of mα + nm + mβ + (2m2 + m) real numbers, where k is number of iterations for solving the matrix equation (16), oα and mα are respectively the number of operations and memory space required for implementation of the algorithm α. The numbers oβ and mβ are similarly defined.
4
Numerical experiments
In this section we wish to corroborate the discussion of Section 3 by solving (1), with M given as in (2), and exact solution x = (1, 1, . . . , 1)T . In our numerical experiments, M is real, symmetric, with several block size n and several size and structure of the cells A and B. The above algorithms are compared by means of execution times and accuracy of the solution. The codes are written in MATLAB language and computations are done on a PENTIUM computer. The results of the experiments are given in separate 123
Table 1 Flops and memory space Algorithm
flops
memory space
LU CHOL
17n 3 3 m 13n 3 3 m
CR
22m3 log2 n + 10nm2 + O(nm)
2m2 log2 n + 8nm
CRM
12m3 log2 n + 10nm2 + O(nm)
2m2 log2 n + 4nm
EI_γ(F)
(8n + 6Iter)m3 + O(nm2 )
nm2 + (n + 1)m
EI_γ(R)
(4n + 6Iter)m3 + O(nm2 )
nm2 + (n + 1)m
M(R)
(4n + 12Iter)m3 + O(nm2 )
nm2 + (n + 1)m
+ O(nm2 )
3nm2 + 2nm
+ O(nm2 )
3n 2 2 m
+
3n 2 m
tables for each example. The following notation is used: • • • • •
•
•
• •
LU stands for the LU algorithm. CHOL stands for the CHOL algorithm. CR stands for the CR algorithm. CRM stands for the CRM algorithm. EI_1(F) stands for the α(β) algorithm. The matrix equation (16) is solved by algorithm EI_γ with initial guess Z0 = I, then N −1 E1 is computed by algorithm F . EI_1(R) stands for the α(β) algorithm. The matrix equation (16) is solved by algorithm EI_γ with initial guess Z0 = I, then N −1 E1 is computed by algorithm R. EI_ 21 (R) stands for the α(β) algorithm. The matrix equation (16) is solved by algorithm EI_γ with initial guess Z0 = 12 I, then N −1 E1 is computed by algorithm R. M(R) stands for the α(β) algorithm. The matrix equation (16) is solved by algorithm M, then N −1 E1 is computed by algorithm R. n = 2p is the block size of matrix M and m is the size of each block.
For all programs the value of ε is set to ε = 10−14 . • Iter is the smallest number k, for which: kZk − Zk+1 k∞ ≤ ε for algorithm EI_γ, kXk − Xk+1 k∞ ≤ ε for algorithm M . • Err. = kx − x˜k∞ , where x˜ is the computed solution. Table 1 reports the flops and memory space required for each program.
124
From Table 1. we see that algorithm CRM requires less memory space than the others. Its number of operation depends of the relationship between the sizes m and n. If m < 5n/6p then algorithm CRM requires O(10nm2 ) flops and if m > 5n/6p then - O(12m3 log2 n). The algorithms EI_γ(R) and M(R) are more effective than the classical LU and CHOL. Even under the assumption that the number of iterations Iter is considerably less than n, for m < 3 and p ≥ 3 they require less flops than algorithm CRM. In the next examples the cells A and B of the matrix M are chosen in such way that they guarantee the existence of a positive definite solution of Equation (16) [9,12].
Example 1. The cells A and B are chosen like in Example 7.3 from [9] i.e.
1.20 −0.30 0.10 A = −0.30 2.10 0.20 , 0.10 0.20 0.65
0.37 0.13 0.12 B = −0.30 0.34 0.12 . 0.11 −0.17 0.29
˜ 2 = kA− 2 BA− 2 k2 = 0.511. To reach the required accuracy in Here kBk solving Equation (16) Algorithm EI_1 needs 404 iterations while Algorithm M only 10 iterations. In Table 2 we present the execution time (in seconds) and the error, of each algorithm for different values of m and n. 1
1
Example 2. We let the cells of the matrix M to be the matrices of example 5.2 from [12], i.e. A = I, and B = (bi,j ) a symmetric matrix, whose entries are determined as below: by the scheme: 1. Fix a real 0 ≤ α ≤ 1/2. 2. For i = 1, . . . , m For j = i, . . . , m bi,j = 2 ∗ i + j end j. Pi−1 Pm Compute s1 = s2 = j=1 bi,j , j=i bi,j . For j = i, . . . , m 1) bi,j = bi,j (1/2−α−s , bj,i = bi,j s2 end j. end i. The matrix B is symmetric, nonnegative and such that Be = (1/2−α)e, where ˜ 2 = kA− 12 BA− 12 k2 = e is the vector having all it’s entries equal to 1. Thus kBk 1/2 − α. 125
Table 2 Execution time (in seconds) and errors for Example 1 m = 3 Algorithm LU CHOL CR CRM EI_1(F) EI_1(R) M(R) LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
26
n = = 64 Err. 2.0650e-014 1.3545e-014 1.2079e-013 6.1284e-014 1.4211e-014 1.9540e-014 1.2879e-014 n = 28 = 256
time 0.06 0.05 0.05 0.06 0.16 0.11 0.06
n = 210 = 1024 Err. time 3.3640e-014 1.04 5.1292e-014 0.88 3.2019e-013 1.42 1.6398e-013 0.77 8.4155e-014 1.81 8.4155e-014 0.83 5.8620e-014 0.77 12 n = 2 = 4096
2.8089e-014 4.6407e-014 3.0931e-013 1.5998e-013 2.9532e-014 2.9976e-014 3.2863e-014
0.27 0.27 0.33 0.22 0.55 0.28 0.22
3.3640e-014 5.1292e-014 3.2019e-013 1.6398e-013 3.0065e-013 3.0065e-013 2.5757e-013
We consider the following two cases: ˜ 2 = 0.1. To obtain the required accuracy for (16), 1. α = 0.4; then kBk Algorithm EI_1 needs 8 iterations, Algorithm M 4 iterations. ˜ 2 = 1 . In this case, we solve the equation (22) by 2. α = 0; then kBk 2 algorithm EI_γ, with initial guess Z0 = 21 I (γ = 21 ). To reach the required accuracy for (16), Algorithm EI_ 12 needs 9 iterations. The use of Algorithm EI_1 is not recommended, because it needs more than 3 ∗106 iterations. Algorithm M needs of 32 iterations. In Tables 3 and 4 we give execution time (in seconds ) and errors, for α = 0 and α = 0.4 respectively.
Example 3. Let A = circ(20, −8, 1, . . . , 1, −8), ˜ 2 = kA− 21 BA− 21 k2 = be a circulant matrix and B = I. In this case kBk 0.1667. To reach the required accuracy for (16) Algorithm EI_1 needs 10 iterations, Algorithm M - 5 iterations. In Table 5 we give the execution time (in seconds ) and the error of each algorithms for different values of m and n. 126
5.99 4.45 11.32 3.40 9.72 4.34 4.23
Table 3 Execution time (in seconds) and errors for Example 2: (α = 0) m=3 Algorithm LU CHOL CR CRM EI_ 21 (R) M(R)
Err. 8.8818e-015 1.1990e-014 6.3505e-014 4.3521e-014 6.3949e-014 1.7764e-013
time 0.11 0.06 0.05 0.06 0.05 0.05
LU CHOL CR CRM EI_ 21 (R) M(R)
5.4845e-014 1.3767e-013 1.0281e-012 6.8057e-013 1.0658e-012 5.6843e-013
0.22 0.22 0.27 0.16 0.17 0.17
LU CHOL CR CRM EI_ 21 (R) M(R)
3.9635e-013 3.6748e-013 1.6434e-011 1.0729e-011 3.8426e-011 3.0809e-011
1.10 0.88 1.42 0.82 0.71 0.73
LU CHOL CR CRM EI_ 21 (R) M(R)
5.5140e-012 8.9115e-012 2.6242e-010 1.7121e-010 1.0141e-009 6.2664e-010
5.99 4.45 11.73 3.35 4.17 4.23
5
m=5 n = 26 = 64 Err. time 4.3299e-015 0.11 7.7716e-015 0.06 4.9960e-015 0.05 1.6431e-014 0.06 7.1054e-014 0.06 1.5632e-013 0.05 n = 28 = 256 2.7645e-014 0.28 5.1958e-014 0.28 4.8628e-014 0.27 2.4425e-013 0.22 4.2633e-012 0.22 8.5265e-013 0.22 n = 210 = 1024 6.2017e-013 1.26 1.4135e-012 1.04 6.9700e-013 1.44 3.9986e-012 0.83 6.7075e-012 0.75 2.0236e-011 0.77 n = 212 = 4096 4.9095e-012 6.70 4.0730e-012 5.05 1.1125e-011 11.87 6.3752e-011 3.40 9.7543e-010 4.34 4.4201e-010 4.34
m = 10 Err. 1.3989e-014 3.9968e-015 7.6605e-015 2.2204e-015 8.5265e-014 2.8422e-014
time 0.11 0.11 0.11 0.06 0.06 0.06
8.7153e-014 6.4615e-014 9.7033e-014 4.0079e-014 2.5295e-012 7.1054e-013
0.55 0.44 0.33 0.22 0.22 0.22
2.7101e-013 3.4261e-013 1.3676e-012 6.7812e-013 3.6380e-011 8.1968e-011
2.58 2.03 1.48 0.88 1.04 1.05
4.1102e-012 2.1547e-012 2.1823e-011 1.0165e-011 5.6480e-010 7.0941e-010
19.17 11.32 11.98 3.51 5.77 5.76
Conclusions
From the discussion and the results obtained by numerical experiments, we can conclude that: 1. The proposed modifications of formulas (20) and (21) (Algorithm R) lead to a considerable decrease in the number of operations, for computing the block vector N −1 E1 . That explains why the execution time for for Algorithms EI_γ(R) and M(R) is less than for Algorithm EI_γ(F). 2. The adapted algorithms CRM, EI_γ(R) and M(R) essentially take advantage of the special structure of the matrix M , and this makes them more effective than the classical algorithms LU, CHOL, CR as far as 127
Table 4 Execution time (in seconds) and errors for Example 2: (α = 0.4) m=3 Algorithm LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
Err. 4.4409e-016 6.6613e-016 4.4409e-016 4.4409e-016 6.6613e-016 6.6613e-016 4.4409e-016
time 0.05 0.05 0.05 0.05 0.11 0.05 0.05
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
4.4409e-016 6.6613e-016 4.4409e-016 4.4409e-016 6.6613e-016 6.6613e-016 4.4409e-016
0.21 0.22 0.27 0.18 0.44 0.17 0.16
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
4.4409e-016 6.6613e-016 4.4409e-016 4.4409e-016 6.6613e-016 6.6613e-016 4.4409e-016
1.04 0.94 1.43 0.77 1.76 0.76 0.72
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
4.4409e-016 6.6613e-016 4.4409e-016 4.4409e-016 6.6613e-016 6.6613e-016 4.4409e-016
5.99 4.45 11.81 3.35 9.62 4.22 4.23
m=5 n = 26 Err. 4.4409e-016 5.5511e-016 5.5511e-016 5.5511e-016 3.3307e-016 3.3307e-016 5.5511e-016 n = 28 4.4409e-016 5.5511e-016 5.5511e-016 5.5511e-016 3.3307e-016 3.3307e-016 5.5511e-016 n = 210 4.4409e-016 5.5511e-016 5.5511e-016 5.5511e-016 3.3307e-016 3.3307e-016 5.5511e-016 n = 212 4.4409e-016 5.5511e-016 5.5511e-016 5.5511e-016 3.3307e-016 3.3307e-016 5.5511e-016
m = 10 = 64 time Err. 0.11 8.8818e-016 0.06 8.8818e-016 0.05 6.6613e-016 0.05 6.6613e-016 0.17 08.8818e-016 0.06 8.8818e-016 0.05 7.7716e-016 = 256 0.27 8.8818e-016 0.27 8.8818e-016 0.28 6.6613e-016 0.20 6.6613e-016 0.60 8.8818e-016 0.22 8.8818e-016 0.16 7.7716e-016 = 1024 1.27 8.8818e-016 1.05 8.8818e-016 1.42 6.6613e-016 0.88 6.6613e-016 2.52 8.8818e-016 0.83 8.8818e-016 0.77 7.7716e-016 = 4096 6.70 8.8818e-016 5.11 8.8818e-016 12.14 6.6613e-016 3.40 6.6613e-016 13.74 8.8818e-016 4.28 8.8818e-016 4.34 7.7716e-016
the number of operation, memory requirements and execution time are concerned. 3. The Algorithms CRM, EI_γ(R) and M(R) are comparable for accuracy of the computed solution, execution time for required flops (for m ≤ 3 and p ≥ 3). For large values of the m, Algorithm CRM requires the least flops, which, together with the fact that it uses the least memory space, suggest es that it is most suitable when the block size of M is a power of two. 128
time 0.11 0.11 0.06 0.06 0.33 0.06 0.06 0.55 0.44 0.28 0.22 1.16 0.22 0.22 2.59 1.97 1.48 0.83 4.78 1.10 0.99 18.51 10.93 12.31 3.57 24.33 5.99 5.76
Table 5 Execution time (in seconds) and errors for Example 3 m=5 Algorithm LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
Err. 6.6613e-016 1.1102e-015 6.6613e-016 7.7716e-016 3.7748e-015 3.7748e-015 5.5511e-016
time 0.05 0.05 0.06 0.05 0.11 0.05 0.05
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
6.6613e-016 1.1102e-015 6.6613e-016 7.7716e-016 3.7748e-015 3.7748e-015 5.5511e-016
0.33 0.22 0.27 0.17 0.60 0.22 0.16
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
6.6613e-016 1.1102e-015 6.6613e-016 7.7716e-016 3.7748e-015 3.7748e-015 5.5511e-016
1.26 1.05 1.48 0.82 2.58 0.77 0.77
LU CHOL CR CRM EI_1(F) EI_1(R) M(R)
6.6613e-016 1.1102e-015 6.6613e-016 7.7716e-016 3.7748e-015 3.7748e-015 5.5511e-016
6.70 5.05 11.84 3.69 13.89 4.45 4.40
m=7 n = 26 Err. 6.6613e-016 6.6613e-016 1.1102e-015 1.7764e-015 2.4425e-015 2.4425e-015 4.4409e-016 n = 28 6.6613e-016 6.6613e-016 1.1102e-015 1.7764e-015 2.4425e-015 2.4425e-015 4.4409e-016 n = 210 6.6613e-016 6.6613e-016 1.1102e-015 1.7764e-015 2.4425e-015 2.4425e-015 4.4409e-016 n = 212 6.6613e-016 6.6613e-016 1.1102e-015 1.7764e-015 2.4425e-015 2.4425e-015 4.4409e-016
m = 10 = 64 time Err. 0.05 1.3323e-015 0.05 8.8818e-016 0.06 1.7764e-015 0.06 1.5543e-015 0.22 2.8866e-015 0.06 2.8866e-015 0.05 6.6613e-016 = 256 0.38 1.3323e-015 0.33 8.8818e-016 0.28 1.7764e-015 0.22 1.5543e-015 0.83 2.8866e-015 0.22 2.8866e-015 0.17 6.6613e-016 = 1024 1.82 1.3323e-015 1.21 8.8818e-016 1.42 1.7764e-015 0.88 1.5543e-015 3.46 2.8866e-015 0.83 2.8866e-015 0.82 6.6613e-016 = 4096 7.58 1.3323e-015 5.71 8.8818e-016 11.91 1.7764e-015 3.74 1.5543e-015 18.29 2.8866e-015 4.67 2.8866e-015 4.62 6.6613e-016
4. If n is not a power of two and the cells of the matrix M satisfy the conditions for existence of the solution of equation (16), then the use of Algorithms M(R) is recommended instead.
129
time 0.11 0.11 0.06 0.06 0.33 0.06 0.06 0.50 0.44 0.28 0.22 1.15 0.23 0.27 2.59 1.98 1.48 0.88 4.83 1.10 0.99 18.290 11.15 12.58 3.87 24.71 5.93 5.77
Acknowledgments This work was partially supported by Shoumen University under contract N 17/2001 and by Norwegian Research Counsil trought the GI4PDE project, contract no. 142955/431. I wish to thank Ivan Ivanov and Antonella Zanna for their helpful remarks and comments.
References [1] D. Bini, B. Meini, Solving Block Banded Block Teoplitz Systems with Structured Blocks: New Algorithms and Open Problems, in: Largre-Scale Scientific Computations of Engineering and Environmental Problems II, Notes on Numerical Fluid Mechanics, vol.73 Vieweg, 2000, pp. 15-24. [2] B.L. Buzbee, G.H. Golub, C.W. Nielson, On direct methods for solving Poisson’s equations, SIAM J. Numer. Analysis 7 (1970) 627-656. [3] F. Diele, L. Lopez, The use of the factorization of five-diagonal matrices by tridiagonal Teoplitz matrices, Appl. Mtah. Lett. 11 (1998) 61-69. [4] S.M. El-Sayed, Study of special matrices and numerical methods for special matrix equations, Ph.D. Thesis, Sofia, 1996 (in Bulgarian). [5] S.M. El-Sayed, I.G. Ivanov, M.G. Petkov, A new modification of the Rojo Method for solving symmetric circulant five-diagonal systems of linear equations, Computers Math. Applic. 35 (1998) 35-44 . [6] J.C. Engwerda, C.M. Ran Andre, A.L. Rijkeboer, Necessary and Sufficient Conditions for the Existence of a Positive Definite Solution of the Matrix Equation X + A∗ X −1 A = Q, Linear Algebra Appl. 186 (1993) 255-275. [7] G. Fiorentino, S. Serra, Multigrid methods for symmetric positive definite block Teoplitz matrices with nonnegative generating functions, SIAM J. Sci. Computing 17 (1996) 1068-1081. [8] G. Golub, C. Van Loan, Matrix Computation, The John Hopkins University Press, Baltimore, 1989. [9] C.-H. Guo and P. Lancaster, Iterative solution of two matrix equations, The Mathematics of Computation 68 (1999) 1589-1603. [10] I. Ivanov, V. Hasanov, F.Uhlig, Iterative methods for computing a positive definite solution of matrix equations X ± A∗ X −1 A = I, submitted to Mathematics of Computation. [11] I. Ivanov, B. Mintchev, A Method for Solving Special Circulant Pentadiagonal Linear System, in: Large-Scale Scientific Computations of Engineering and Environmental Problems II, Notes on Numerical Fluid Mechanics, vol.73, Vieweg, 2000, pp. 144-151.
130
[12] B. Meini, Matrix Equations and Structures: Efficient Solution of Special discrete Algebraic Riccati Equations, in: Second Conference on Numerical Analysis and Applications, LNIS 1988, Springer, Berlin, 2000, pp. 578-585. [13] J. Rissanen, Solution of linear equations with Hancel and Teoplitz matrices, Numer. Math. 22 (1974) 361-366. [14] O. Rojo, A new method for solving symmetric circulant tridiagonal systems of linear equations, Computers Math. Applic. 20 (1990) 61-67. [15] X. Zhan, Computing the extremal positive definite solutions of a matrix equation, Siam J. Sci. Comput. 17 (1996) 1167-1174.
131
132
Article 3 A Method for Solving Hermitian Pentadiagonal Block Circulant Systems of Linear Equations Published in LNCS 2907, Springer 481-488, 2004
133
134
A Method for Solving Hermitian Pentadiagonal Block Circulant Systems of Linear Equations Borislav V. Minchev a , Ivan G. Ivanov b a Department
of Computer Science, University of Bergen, Thormhlensgate 55, N-5020 Bergen, Norway
b Faculty
of Economics and Business Administration, Sofia University, Sofia 1113, Bulgaria
Abstract A new effective method and its two modifications for solving Hermitian pentadiagonal block circulant systems of linear equations are proposed. New algorithms based on the proposed method are constructed. Our algorithms are then compared with some classical techniques as far as implementation time is concerned, number of operations and storage. Numerical experiments corroborating the effectiveness of the proposed algorithms are also reported. Key words: linear system, block circulant matrix, matrix equation, Woodbury’s formula 1991 MSC: 65F10
1
Introduction
Linear systems of equations having circulant coefficient matrices appear in many applications. For example, in finite difference approximations to elliptic equations subject to periodic boundary conditions [2,8] and in approximations of periodic functions using splines [1,9]. In case when multidimensional problems are concerned the coefficient matrices of the resulting linear systems are with block circulant structure [7]. Email addresses:
[email protected] (Borislav V. Minchev), i−
[email protected] (Ivan G. Ivanov). URL: http://www.ii.uib.no/∼borko (Borislav V. Minchev).
Published in LSSC, LNCS 2907, Springer, 481-488, 2004
In this paper we propose a new method and its two modifications for solving Hermitian pentadiagonal block circulant systems of linear equations. It is known that these systems have the form W x = f, where
(1)
M N S S∗ N ∗ ∗ S∗ N M N S ∗ S N∗ . . . 0 ∗ S . . . . (2) W = . . . . . 0 . . . N S ∗ ∗ S N M N S N S S∗ N ∗ M is Hermitian pentadiagonal block circulant matrix with block size n. M , N and S are m×m matrices, x = {xi }i=1,...,n , f = {fi }i=1,...,n , are column vectors with block size n, xi and fi , are blocks with size m × 1.
Our goal is to construct a new effective method for solving (1) and then to compare it with some classical techniques. The paper is organized as follows: in Section 2 we present the new method and discuss its two modifications based on different applications of the Woodbury’s formula [4]; in Section 3 we report some numerical experiments corroborating the effectiveness of the proposed algorithms.
2
A Modification of LU factorizations
Adapting the ideas suggested in [6], we construct a new method for solving linear systems with coefficient matrices of the form (2). Our approach is based on the solution of a special nonlinear matrix equation. One can fine the solution of (1) using the following steps: Step 1. Solve the parametric linear system T y = f, where
X Y S ∗ Y Z N . ∗ S N∗ M . . T = . . . 0 . . S∗ 136
(3)
0
S N
. . N∗ M
is pentadiagonal matrix with block size n. It has a block Teoplitz structure except for the north-western corner, y = {yi }i=1,...,n , and f = {fi }i=1,...,n are column vectors with blocks size n, yi and fi are blocks with size m × 1. The matrix T admits the following LU factorization
X Y S 0 Im ∗ −1 Y X . . . . 0 ∗ −1 . . S . . T = LU = , S X 0 . Y . . . X 0 S ∗ X −1 Y ∗ X −1 Im where Im is the identity matrix with size m × m. The above decomposition exists when the parameters X = X ∗ , Y and Z = Z ∗ satisfy the relations ¯ ¯ Z = Y ∗ X −1 Y + X ¯ ¯ ¯ N = Y ∗ X −1 S + Y . ¯ ¯ M = S ∗ X −1 S + Z
(4)
Let us introduce the following notations F =
Ã
X Y Y∗ Z
!
, Q =
Ã
S 0 N S
!
, R =
Ã
M N N∗ M
!
.
(5)
If F is a positive definite solution of the matrix equation F + Q∗ F −1 Q = R
(6)
and X = X ∗ > 0, Z = Z ∗ > 0 then the blocks X, Y and Z satisfy the system (4). Thus, solving the linear system (3) is equivalent to solve two simpler systems L z = f,
z = {zi }i=1,...,n
U y = z,
y = {yi }i=1,...,n .
Step 2. Solve the pentadiagonal block Teoplitz linear system P u = f, 137
(7)
where
M N S ∗ N M N . S∗ N ∗ M . . . . P = . 0
0
S N
. . . . . . . . . S∗ N ∗ M
(8)
is Hermitian pentadiagonal block Teoplitz matrix with block size n, u = {ui }i=1,...,n and f = {fi }i=1,...,n are column vectors with block size n , ui and fi are blocks with size m × 1. The matrices T and P satisfy the relation P = T + J2 Vˆ , where J2 =
Ã
Im 0 0 . . . 0 0 Im 0 . . . 0
!T
,
Vˆ =
Ã
M − X N − Y 0 ... 0 N∗ − Y ∗ M − Z 0 . . . 0
!
are matrices with block size n × 2 and 2 × n respectively. Using the Woodbury’s formula we have h
P −1 = T −1 − T −1 J2 I2m + Vˆ T −1 J2
i−1
Vˆ T −1 ,
(9)
where I2m is the identity matrix with size 2m × 2m. Therefore, the solution u of (7) is given by h
u = P −1 f = y − T −1 J2 I2m + Vˆ T −1 J2
i−1
Vˆ y.
One can find the matrix T −1 J2 by solving 2m linear systems of type (3) with right-hand sides the corresponding two different columns of J2 . This approach does not take into account the very sparse nonzero structure of J2 . For real M , N and S it costs O(20nm3 ) flops and needs to store 2nm2 real numbers. In order to decrease the number of operations needed to compute T −1 J2 , we consider a new approach which is motivated by the ideas suggested in [5]. Let us denote the block columns vectors of J2 with E1 and E2 respectively i.e. E1 =
³
Im 0 0 . . . 0
´T
,
E2 =
³
0 Im 0 . . . 0
Put A = Y ∗ X −1 and B = S ∗ X −1 . The matrix T admits the following decomposition
T = LDL∗ ,
Im A . 0 where L = B . . . . . 0 B A Im 138
´T
.
is a square matrix of block size n and D = diag(X, . . . , X). Let (L−1 )ij be the blocks of the matrix L−1 . We have ³
L
−1
´
ij
=
(
0 Zi−j+1
Z3 = A2 − B, f or j = 4 . . . n.
Z1 = I m , Z2 = −A, i
