Multigrid algorithms for stochastic finite element

Arenberg Doctoral School of Science, Engineering & Technology Faculty of Engineering Department of Computer Science

Multigrid algorithms for stochastic finite element computations

Eveline Rosseel

Dissertation presented in partial fulfillment of the requirements for the degree of Doctor in Engineering

August 2010

Multigrid algorithms for stochastic finite element computations

Eveline Rosseel

Jury: Dissertation presented in partial Prof. Dr. ir.-arch. H. Neuckermans, fulfillment of the requirements for president the degree of Doctor Prof. Dr. ir. S. Vandewalle, in Engineering promotor Dr. ir. G. Samaey Prof. Dr. ir. G. Degrande Prof. Dr. ir. H. De Gersem Prof. Dr. ir. C. W. Oosterlee (CWI, TU Delft) Priv.-Doz. Dr. O. G. Ernst (TU Bergakademie Freiberg) August 2010

© Katholieke Universiteit Leuven – Faculty of Engineering Kasteelpark Arenberg 1 bus 2200, B-3001 Leuven (Belgium) Alle rechten voorbehouden. Niets uit deze uitgave mag worden vermenigvuldigd en/of openbaar gemaakt worden door middel van druk, fotocopie, microfilm, elektronisch of op welke andere wijze ook zonder voorafgaande schriftelijke toestemming van de uitgever. All rights reserved. No part of the publication may be reproduced in any form by print, photoprint, microfilm or any other means without written permission from the publisher. D/2010/7515/71 ISBN 978-94-6018-232-7

Preface During my engineering education, I entered the field of scientific computing which covers the mathematical aspect of simulating physical problems. My master’s thesis on iterative methods for electromagnetic problems inspired me to continue research in this direction. I would like to thank my supervisor Stefan Vandewalle for giving me the chance to pursue research on multigrid methods. He guided me into this domain, introduced me to many people and encouraged me to attend international conferences. His careful readings of our papers and of this text were of great value to me. During the last years of my PhD I had the chance to apply the developed methods to some more physical applications. I thank Herbert De Gersem and Nico Scheerlinck for providing me with their models and for the fruitful discussions on the results. I also thank Herbert for agreeing to be a member of my jury. At conferences, I had the opportunity to meet several people in my research domain, which has led to many interesting discussions and brought me more insight into solving stochastic problems. In particular, I thank Oliver Ernst from TU Freiberg for inviting me to Freiberg and for agreeing to be part of the jury. I also thank Wolfgang Hackbusch for inviting me to the Max Planck Institute in Leipzig and Tanja Clees for the discussions in Cologne and Leuven. I would like to thank Geert Degrande, Giovanni Samaey and Kees Oosterlee for agreeing to be a member of my PhD jury. Herman Neuckermans kindly accepted to be the president of the jury. I thank Dirk Roose and Bart Nicola¨ı for being part of the supervising PhD committee. I gratefully acknowledge the financial support received from the Special Research Fund (BOF) of the K.U.Leuven and the Research Foundation-Flanders (FWO). My research on local Fourier analysis presented in Chapter 4 was a continuation of the work of Bert Seynaeve, whose office I shared during the first months of my PhD. I wish to express my gratitude for the many insights into convergence analysis that I acquired from his work. I deeply regret that he passed away prematurely.

i

ii

My years at the Scientific Computing Research Group would not have been the same without my colleagues, with whom I spent many enjoyable breaks. I was happy to share my office with Tim Boonen, Daan Huybrechs and Samuel Corveleyn. I thank them for the good and friendly atmosphere. Tim helped me a lot with C++ programming and multigrid in the beginning of my PhD. I thank my colleagues Liesbeth Vanherpe, Bart Vandereycken, Yves Frederix, Pieter Ghysels, Joris Vanbiervliet, Elias Jarlebring, Sam Corveleyn for the uncountably many and pleasant Alma-lunches and for sharing PhD concerns. I thank Yves for solving my computer problems and Bart for the helpful linear algebra discussions. Finally, I wish to thank my family and friends. I want to grasp this opportunity to thank them for being there for me and for all the happy moments together. I enjoyed many evenings and weekends together with my friends from high school, Elisabeth, Kathleen, Liesbet, Els C., Els L., Els V.B., Benedicte, Soetkin, Riet and Paquita. Also the (game) evenings with my (computer science) friends, Wim, Joaquin and Veerle, Liesbeth and Thomas, Joris and Iris, Jan and Lore, Yves and Paula, Tine and Wouter, Lukas and Joke, were great fun. I am very grateful for the support and love that I received from my family, and in particular from my parents(-in-law) and sisters. Over the years, my husband Edwin stood by me with good advice and supported me whenever I needed it. It is great to share my life with him. Dankjewel!

Eveline Rosseel August 2010

Abstract Mathematical models of engineering systems and physical processes typically take the form of a partial differential equation (PDE). Variability or uncertainty on coefficients of a PDE can be expressed by introducing random variables, random fields or random processes into the PDE. Recently developed stochastic finite element methods enable the construction of high-order accurate solutions of a stochastic PDE, while reducing the high computational cost of more standard uncertainty quantification methods, such as the Monte Carlo simulation method. Two prominent variants are the stochastic collocation method and the stochastic Galerkin finite element method. The latter transforms a stochastic PDE into a coupled set of deterministic PDEs. This work aims at enhancing the performance of the stochastic Galerkin finite element method by developing efficient iterative solvers for the resulting high-dimensional algebraic systems. A construction and convergence analysis of multigrid based iterative methods for stochastic Galerkin finite element discretizations are presented. By extending multigrid algorithms for deterministic problems to stochastic Galerkin discretizations, solution methods with optimal convergence properties result. That is, a convergence rate independent of the stochastic, time and spatial discretization parameters can be established. Time-(in)dependent, (non)linear problems with multiple random coefficients are addressed. In the time-dependent case, the stochastic Galerkin method is combined with a high-order implicit Runge-Kutta time discretization. The efficiency and robustness of the developed methods are illustrated with extensive numerical experiments, including the solution of a stochastic biological reaction-diffusion application. The stochastic collocation finite element method solves a stochastic PDE by sampling the random coefficients at a well-chosen set of multidimensional collocation points. The last part of this work compares the accuracy and computational cost of the two stochastic finite element variants. In particular, a stochastic ferromagnetic application is considered. This study suggests that the stochastic Galerkin method is preferable in the case of linear stochastic PDEs, while for nonlinear stochastic problems, sparse stochastic collocation methods can be a better option.

iii

Samenvatting Fysische verschijnselen en ingenieurstoepassingen worden vaak gemodelleerd door één of meerdere partiële differentiaalvergelijkingen (PDV). De parameters in die modellen, zoals de materiaaleigenschappen, afmetingen en begincondities, zijn doorgaans slechts bij benadering gekend. Deze onzekerheid kan wiskundig uitgedrukt worden met behulp van toevalsvariabelen, stochastische processen of random velden. Recent werden stochastische eindige-elementenmethoden ontwikkeld voor het oplossen van PDV’s met stochastische coëfficiënten. Deze methoden zijn in staat om heel wat nadelen van klassieke onzekerheidspropagatiemethoden, zoals de Monte Carlo methode, te vermijden. De twee voornaamste varianten zijn de stochastische collocatiemethode en de stochastische Galerkin eindige-elementenmethode. Deze laatste zet een stochastische PDV om in een stelsel van gekoppelde, deterministische vergelijkingen. Aangepaste methoden zijn vereist om de overeenkomstige hoog-dimensionale algebra¨ısche vergelijkingen efficiënt op te lossen. Dit werk stelt een constructie en convergentie-analyse voor van multigridgebaseerde iteratieve methoden voor stochastische Galerkin eindige-elementenberekeningen. Methoden met optimale convergentie-eigenschappen worden ontwikkeld. Dit betekent dat de convergentiesnelheid niet afhangt van de grootte van de stochastische, ruimtelijke of tijdsdiscretisatie. Tijds(on)afhankelijke, (niet-)lineaire PDV’s met meerdere random coëfficiënten worden beschouwd. In geval van tijdsafhankelijke vergelijkingen wordt de stochastische Galerkin methode gecombineerd met een hoge-orde impliciete Runge-Kutta tijdsdiscretisatie. De efficiëntie en betrouwbaarheid van de methoden worden aangetoond met niet-triviale numerieke experimenten, waaronder de uitwerking van een stochastische reactie-diffusie toepassing. De stochastische collocatiemethode bemonstert toevalsvariabelen in een verzameling van meerdimensionale collocatiepunten. In een laatste deel van dit werk worden de rekenkost en nauwkeurigheid van beide stochastische eindige-elementenmethoden vergeleken. Daarbij wordt een stochastische, ferromagnetische toepassing uitgewerkt. Deze studie geeft aan dat de stochastische Galerkin methode te verkiezen is in geval van lineaire, stochastische problemen, terwijl voor niet-lineaire problemen ijle, stochastische collocatiemethoden een betere optie kunnen zijn.

iv

Contents Contents

v

List of symbols

xi

List of acronyms

xvii

1 Introduction

1

1.1

Uncertainty propagation in numerical simulations . . . . . . . . . .

1

1.2

Main research objectives . . . . . . . . . . . . . . . . . . . . . . . .

2

1.3

Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2 Partial differential equations with stochastic coefficients

5

2.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

2.2

Uncertainty quantification . . . . . . . . . . . . . . . . . . . . . . .

6

2.2.1

Epistemic versus aleatoric uncertainty . . . . . . . . . . . .

6

2.2.2

Classification of uncertainty quantification methods . . . .

6

2.2.3

Probabilistic representation . . . . . . . . . . . . . . . . . .

10

Discrete probabilistic framework . . . . . . . . . . . . . . . . . . .

12

2.3.1

Karhunen-Loève expansion . . . . . . . . . . . . . . . . . .

12

2.3.2

Polynomial chaos expansion . . . . . . . . . . . . . . . . . .

14

2.3.3

Extensions of the polynomial chaos expansion . . . . . . . .

18

2.3

v

vi

CONTENTS

2.4

2.5

2.6

Stochastic Galerkin finite element method . . . . . . . . . . . . . .

22

2.4.1

Spectral stochastic discretization . . . . . . . . . . . . . . .

22

2.4.2

Stochastic model problem . . . . . . . . . . . . . . . . . . .

25

2.4.3

More general problems and special cases . . . . . . . . . . .

29

Stochastic Galerkin discretization matrices . . . . . . . . . . . . . .

32

2.5.1

Definition of matrix elements . . . . . . . . . . . . . . . . .

33

2.5.2

Properties of stochastic discretization matrices . . . . . . .

36

2.5.3

Properties of the global system matrix . . . . . . . . . . . .

38

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

3 Iterative solvers for linear stochastic finite element discretizations

41

3.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41

3.2

One-level methods based on matrix splitting . . . . . . . . . . . . .

42

3.2.1

Basic splitting methods . . . . . . . . . . . . . . . . . . . .

42

3.2.2

Splitting of the Ki -matrices . . . . . . . . . . . . . . . . . .

43

3.2.3

Splitting of the Ci -matrices . . . . . . . . . . . . . . . . . .

45

Multigrid methods in the spatial dimension . . . . . . . . . . . . .

46

3.3.1

Introduction to multigrid . . . . . . . . . . . . . . . . . . .

47

3.3.2

Multigrid for stochastic Galerkin discretizations . . . . . . .

50

3.4

Multilevel methods in the stochastic dimension . . . . . . . . . . .

51

3.5

Preconditioned Krylov subspace methods . . . . . . . . . . . . . .

52

3.6

Implementation aspects . . . . . . . . . . . . . . . . . . . . . . . .

54

3.7

Numerical experiments . . . . . . . . . . . . . . . . . . . . . . . . .

55

3.7.1

Steady-state stochastic diffusion problem . . . . . . . . . .

56

3.7.2

Test case: L-shaped domain problem . . . . . . . . . . . . .

64

3.7.3

Test case: random domain problem . . . . . . . . . . . . . .

69

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

70

3.3

3.8

CONTENTS

vii

4 Convergence analysis of multigrid solvers

73

4.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

4.2

Theoretical convergence analysis . . . . . . . . . . . . . . . . . . .

74

4.2.1

One random variable . . . . . . . . . . . . . . . . . . . . . .

74

4.2.2

Extension to multiple random variables . . . . . . . . . . .

76

4.2.3

Discussion of theoretical convergence properties . . . . . . .

77

Local Fourier convergence analysis . . . . . . . . . . . . . . . . . .

79

4.3.1

Model problem . . . . . . . . . . . . . . . . . . . . . . . . .

79

4.3.2

Introduction to smoothing and convergence analysis . . . .

81

Local Fourier analysis of one-level methods . . . . . . . . . . . . .

82

4.4.1

Basic splitting methods . . . . . . . . . . . . . . . . . . . .

82

4.4.2

Splitting of the Ki -matrices . . . . . . . . . . . . . . . . . .

83

4.4.3

Splitting of the Ci -matrices . . . . . . . . . . . . . . . . . .

87

Local Fourier analysis of multigrid methods . . . . . . . . . . . . .

89

4.5.1

Multigrid in the spatial dimension . . . . . . . . . . . . . .

89

4.5.2

Multigrid with Ki -splitting smoother . . . . . . . . . . . . .

90

4.5.3

Some comments on double orthogonal polynomial chaos . .

92

Local Fourier analysis of multilevel methods in the stochastic dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

4.7.1

One-level methods . . . . . . . . . . . . . . . . . . . . . . .

94

4.7.2

Multigrid methods in the spatial dimension . . . . . . . . .

98

4.7.3

Multilevel methods in the stochastic dimension . . . . . . . 102

4.3

4.4

4.5

4.6 4.7

4.8

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5 Time-dependent stochastic partial differential equations

107

5.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.2

Time discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

viii

CONTENTS

5.3

5.4

5.5

5.2.1

Implicit Runge-Kutta methods . . . . . . . . . . . . . . . . 108

5.2.2

Discretization of the model problem . . . . . . . . . . . . . 110

An algebraic multigrid method . . . . . . . . . . . . . . . . . . . . 111 5.3.1

Smoothing operator . . . . . . . . . . . . . . . . . . . . . . 112

5.3.2

Multigrid hierarchy and intergrid transfer operators . . . . 113

Convergence analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 114 5.4.1

Theoretical asymptotic convergence factor . . . . . . . . . . 114

5.4.2

Discussion of the convergence analysis . . . . . . . . . . . . 116

Implementation aspects . . . . . . . . . . . . . . . . . . . . . . . . 117 5.5.1

Matrix formulation and storage . . . . . . . . . . . . . . . . 117

5.5.2

Krylov acceleration . . . . . . . . . . . . . . . . . . . . . . . 118

5.5.3

Block smoothing . . . . . . . . . . . . . . . . . . . . . . . . 118

5.6

Alternative solution approaches . . . . . . . . . . . . . . . . . . . . 119

5.7

Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

5.8

5.7.1

Test case 1: diffusion problem on square domain . . . . . . 120

5.7.2

Test case 2: electric potential problem . . . . . . . . . . . . 124

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6 A stochastic bio-engineering application

127

6.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

6.2

Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

6.3

6.4

6.2.1

Reaction and diffusion on a growing domain . . . . . . . . . 128

6.2.2

Two-particle model . . . . . . . . . . . . . . . . . . . . . . . 131

6.2.3

Stochastic model . . . . . . . . . . . . . . . . . . . . . . . . 132

Stochastic finite element discretization . . . . . . . . . . . . . . . . 132 6.3.1

Nonlinear stochastic partial differential equations . . . . . . 132

6.3.2

Systems of stochastic partial differential equations . . . . . 136

An algebraic multigrid solution method . . . . . . . . . . . . . . . 140

CONTENTS

6.5

6.6

6.7

ix

6.4.1

Multigrid for a time-dependent, nonlinear, stochastic PDE . 141

6.4.2

Multigrid for a system of nonlinear, stochastic PDEs . . . . 141

Implementation aspects . . . . . . . . . . . . . . . . . . . . . . . . 143 6.5.1

Space-time discretization . . . . . . . . . . . . . . . . . . . 143

6.5.2

Matrix formulation and storage . . . . . . . . . . . . . . . . 144

6.5.3

Krylov preconditioning . . . . . . . . . . . . . . . . . . . . . 146

Numerical experiments . . . . . . . . . . . . . . . . . . . . . . . . . 147 6.6.1

One chemical particle . . . . . . . . . . . . . . . . . . . . . 147

6.6.2

Two chemical particles . . . . . . . . . . . . . . . . . . . . . 149

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

7 Comparison of the stochastic Galerkin and collocation method

155

7.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

7.2

The stochastic collocation method . . . . . . . . . . . . . . . . . . 156

7.3

7.4

7.5

7.6

7.2.1

Computation of statistics . . . . . . . . . . . . . . . . . . . 157

7.2.2

Construction of collocation points . . . . . . . . . . . . . . 158

7.2.3

Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Comparison of stochastic Galerkin and collocation methods . . . . 161 7.3.1

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

7.3.2

Numerical illustration . . . . . . . . . . . . . . . . . . . . . 162

Application: ferromagnetic rotating cylinder . . . . . . . . . . . . . 164 7.4.1

Deterministic model . . . . . . . . . . . . . . . . . . . . . . 165

7.4.2

Stochastic model . . . . . . . . . . . . . . . . . . . . . . . . 168

Stochastic Galerkin discretization . . . . . . . . . . . . . . . . . . . 169 7.5.1

Newton linearization . . . . . . . . . . . . . . . . . . . . . . 170

7.5.2

Computational aspects . . . . . . . . . . . . . . . . . . . . . 173

Numerical comparison for ferromagnetic problem . . . . . . . . . . 176 7.6.1

Lognormal random variables and Hermite polynomials . . . 177

x

CONTENTS

7.7

7.8

7.6.2

Uniform random variables and Legendre polynomials . . . . 179

7.6.3

Combination of distributions . . . . . . . . . . . . . . . . . 181

Influence of uncertainty on the torque . . . . . . . . . . . . . . . . 182 7.7.1

Computation of the torque . . . . . . . . . . . . . . . . . . 182

7.7.2

One random variable . . . . . . . . . . . . . . . . . . . . . . 183

7.7.3

Uncertainty on the parameters of the magnetization curve . 185

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

8 Conclusions

189

8.1

Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . 190

8.2

Suggestions for further research . . . . . . . . . . . . . . . . . . . . 191 8.2.1

Multilevel solution methods . . . . . . . . . . . . . . . . . . 191

8.2.2

Fast solvers for nonlinear PDEs . . . . . . . . . . . . . . . . 192

8.2.3

Fast matrix-vector multiplication . . . . . . . . . . . . . . . 192

8.2.4

Alternative representation of lognormal random fields . . . 192

8.2.5

Application to industrial problems . . . . . . . . . . . . . . 193

A Probability and random fields

195

Bibliography

199

Curriculum vitae

215

List of symbols Probability ω Ω F E ξi ξ Γi Γ ̺i ̺ var(·) cov(·, ·) lc σ

sample or random event sample or event space σ-algebra probability measure random variable random vector range of ξi range of ξ probability density function of ξi joint probability density function of ξ variance covariance (function) correlation length standard deviation; σ 2 variance

Geometry d D D ∂D (r, θ, z) x = (x1 , . . . , xd ) x lz ~n ri

number of spatial dimensions d-dimensional spatial domain transformed domain boundary of D polar coordinates Cartesian coordinates transformed spatial variable length normal vector inner radius (of a cylinder)

xi

xii

ro r0 r∞ r(t) α, β

List of symbols

outer radius (of a cylinder) initial radius (of an apple) final, stationary radius (of an apple) radius growth function (of an apple) parameters of growth function r(t)

Functional analysis L2 (Ω) L2 (D) H 1 (D) H01 (D)

Lebesgue space of square-integrable functions on Ω Lebesgue space of square-integrable functions on D Hilbert space defined by the set {v : D → R|∂v ∈ L2 (D)} Hilbert space defined by the set {v ∈ H 1 (D)|v = 0 on ∂D}

Operators h·i ∇·u ∇u δ·,· L tr(·) U

expectation; inner product in L2 (Ω)-space := ∂u/∂x1 + ∂u/∂x2 + ∂u/∂x3 , divergence operator := [∂u/∂x1 ∂u/∂x2 ∂u/∂x3 ]T , gradient operator Kronecker delta partial differential operator trace of a matrix 1D Lagrange interpolation operator

Polynomials ηq P p, pi ψj τj∗ ζj,r ψep,j τp,j Ψq eq Ψ Ψ ℓk Tm

multi-index in NL polynomial chaos order polynomial degree univariate, orthogonal polynomial of degree j normalization coefficient of ψj root of orthogonal polynomial ψj univariate, double orthogonal polynomial normalization coefficient of ψep,j multivariate, orthogonal polynomial multivariate, double orthogonal polynomial vector of polynomials, [Ψ1 . . . ΨQ ]T multivariate Lagrange polynomial Chebyshev polynomial

List of symbols

Tˆm

xiii

scaled and shifted Chebyshev polynomial

Electromagnetism ~ A Az Az ~ B 1 2 Bref , Bref ~ H Hf Hknee J~s Mz V wmagn,co ε σ ν νc ν¯ ¯d νfinal

magnetic vector potential z-component magnetic vector potential discretized magnetic vector potential magnetic flux density reference values for the magnetic flux density

[Tm] [Tm] [Tm] [T] [T]

magnetic field strength discretized magnetic field strength at linearization point magnetic field strength at knee point applied current density z-component of the torque electric scalar potential magnetic co-energy density electric permittivity conductivity reluctivity chord reluctivity differential reluctivity tensor final reluctivity at the fully saturated range

[A/m] [A/m] [A/m] [A/m2 ] [Nm] [V] [J/m3 ] [F/m] [S/m] [A/Tm] [A/Tm] [A/Tm] [m/H]

Discretization parameters ∆t h L N Nc Nmc Q S sirk w

time step mesh/grid size number of random variables number of spatial degrees of freedom number of stochastic collocation points number of Monte Carlo samples number of stochastic degrees of freedom/basis polynomials number of stiffness matrices number of implicit Runge-Kutta stages Smolyak level

xiv

List of symbols

Matrices 0N Q A Airk Ci Gi , Cî , Fi Gpr Gdu IQ Ki M (k) Mν¯d (k)

(k)

∈ RN Q×N Q ∈ RN Q(sirk )×N Q(sirk ) ∈ Rsirk ×sirk ∈ RQ×Q ∈ R2N ×N ∈ RN ×2N ∈ RQ×Q ∈ RN ×N ∈ RN ×N ∈ R2N ×2N

Mu,i , Mv,i ∈ RN ×N (k) Jv2 ,i ,

(k) Juv,i

Pˆ ,Pûv Wconv

∈ RN ×N

linearized competing species reaction matrix linearized Gray-Scott reaction matrix

(2)N Qsirk ×(2)N Qsirk

∈R ∈ RN ×N

A(:, 1) A(1, :)

zero matrix system matrix matrix with implicit Runge-Kutta parameters stochastic Galerkin discretization matrix subblocks of matrix Ci , defined in (2.50) primary curl matrix dual curl matrix identity matrix stiffness matrix mass matrix block diagonal material matrix

permutation matrix convection matrix first column of matrix A first row of matrix A

Vectors b eb b(t) birk , cirk 1sirk c u(t) x

∈ RN Q×1 ∈ RN Qsirk ×1 RQN → RQN ∈ Rsirk ∈ Rsirk ∈ RQ×1 RQN → RQN ∈ RN Q(2sirk )×1

discretized right-hand side discretized (implicit Runge-Kutta) right-hand side time-dependent discretized right-hand side components vectors with implicit Runge-Kutta parameters constant vector with value 1 stochastic discretization vector defined as hΨi time-dependent discretized unknown solution functions discrete solution/implicit Runge-Kutta stage vector

Multigrid A Ak−1 γ

fine grid system matrix coarse grid operator at level k − 1 multigrid cycle index

List of symbols

ν1 , ν2 k Pk−1 Rkk−1 Pkk−1 Rkk−1 Sk T x(n)

xv

number of pre- and postsmoothing steps, respectively prolongation operator between levels k − 1 and k restriction operator between levels k and k − 1 prolongation operator for a discretized deterministic problem restriction operator for a discretized deterministic problem smoothing operator at level k two-grid operator iterative solution obtained after n iterations

Local Fourier convergence analysis e ej,k φ Gh ΛCi Lh eh L M(x) ωSOR, opt ωJAC, opt ρ s σ(·) S Sehj,k θ ΘD uj,k VCi z ∈ RQ

error error vector of random unknowns at grid point (jh, kh) Fourier grid mode finite difference spatial grid eigenvalue matrix of Ci discrete linear differential operator symbol of Lh Q × Q stochastic discretization matrix, see (4.9) optimal SOR Ki -splitting damping parameter optimal Jacobi Ki -splitting damping parameter convergence factor smoothing factor eigenvalue spectrum of a given operator iteration operator Fourier symbol of S Fourier frequency discrete grid of Fourier frequencies solution vector of random unknowns at grid point (jh, kh) eigenvector matrix of Ci vector containing amplitudes of a Fourier mode

Biological reaction-diffusion problem (k)

(k)

δu , δv ǫ1 , ǫ2 κ κ2 κA κf

Newton increments of the unknown functions u and v, respectively positive coefficients of competing species reaction reaction coefficient reaction rate of second Gray-Scott reaction dimensionless reaction rate of first Gray-Scott reaction feed rate of chemical U

xvi

ns R(u) ς1 , ς2 , χ1 , χ2 U, V

List of symbols

number of seeds reaction kinetics positive constants of competing species reaction chemicals

Other symbols a f ϕi gD hN ı λi ωm sn t Tf u uq ~v vi wk ζk

diffusion coefficient forcing term stochastic function in the representation (2.28) of a(x, ω) Dirichlet boundary value Neumann boundary value imaginary unit eigenvalue (of a covariance function) angular velocity finite element nodal basis function time variable final time of time interval (continuous) solution function polynomial chaos coefficient of u mechanical velocity eigenfunction (of a covariance function) cubature weight multidimensional stochastic collocation point

List of acronyms General FEM

finite element method

FFT

fast Fourier transform

gPC

generalized polynomial chaos

IRK

implicit Runge-Kutta

KL

Karhunen-Loève

LFA

local Fourier analysis

ODE

ordinary differential equation

PC

polynomial chaos

PDE

partial differential equation

PDF

probability density function

SCFEM

stochastic collocation finite element method

SFEM

stochastic finite element method

SGFEM

stochastic Galerkin finite element method

xvii

xviii

List of acronyms

Iterative solvers AMG

algebraic multigrid

CG

conjugate gradients

CG-Circ

CG with approximate circulant Ci -splitting preconditioner

CG-GS-Ci

CG preconditioned by one symmetric GS-Ci iteration

CG-GSA-Ci CG with approximate symmetric GS-Ci preconditioner CG-Kron

CG with approximate Kronecker product preconditioner (3.7)

CG-Mean

CG with approximate mean-based preconditioner (3.6)

CG-MGB

CG preconditioned by one MG-Block cycle

CG-MGP

CG preconditioned by one MG-Point cycle

CG-p-GSA

CG preconditioned by one approximate p-GS cycle

GMRES

generalized minimum residual method

GS(LEX)

(lexicographic) Gauss-Seidel

GS-Ci

Gauss-Seidel Ci -splitting iterations (3.5)

GS-RB

red-black Gauss-Seidel

MG-Block

W (1, 1)-multigrid with collective GS smoother (3.3)

MG-Point

W (1, 1)-multigrid (section 3.3.2) with GSLEX smoother (3.2)

ω-Jacobi

weighted Jacobi

p-GS-Ci

V (1, 1)-cycle p-multilevel method (section 3.4) with GS-Ci

SOR

successive overrelaxation

x-GSLEX

line GSLEX in the x1 -direction

Chapter 1

Introduction 1.1

Uncertainty propagation in numerical simulations

Recent advances in computational resources and scientific computing enable the construction and simulation of increasingly realistic mathematical models for physical phenomena and engineering systems. At the same time, there is a growing need for highly accurate simulation results and for the validation and verification of mathematical models. This raises the importance of uncertainty quantification in numerical simulations. The objective is to model uncertainty from the beginning of a simulation and not to compute only a-posteriori error bounds. When modelling a physical system accurately, the uncertainty present in the model needs to be quantified appropriately and a strategy for determining the accuracy and reliability of the simulated outputs is required. Uncertainty quantification covers the identification and quantification of each source of uncertainty and variability in a mathematical model, and yields an assessment of the effect on the simulation results. Two problems can be identified, i.e., the forward problem where uncertainty is propagated through a given mathematical model, and the inverse problem which infers probabilistic information from measured data. The latter is typically dealt with a Bayesian framework [MNR07, GD06] and is not considered in this work. Mathematical models of real life engineering and scientific processes typically take the form of a partial differential equation (PDE). The variability or randomness of certain parameters, which describe for example the material properties, geometric structure, forcing term or boundary conditions, is modelled by introducing random variables, random fields and random processes into the PDE. The formulation and efficient numerical solution of stochastic PDEs have received increasing

1

2

INTRODUCTION

interest in recent years. Examples include applications in computational fluid mechanics [KL06, LWSK07], elasticity problems [NCSM08, FYK07], fluid-structure interactions [XLSK02, WB09, WSB07], porous media flows [MK05, GD98], electromagnetics [AA07, CHL06] and many other fields. While until recently Monte Carlo simulations [Caf98, BTZ04] were the standard tool for solving stochastic PDEs, new stochastic finite element methods [Mat08, Xiu09] are becoming popular since they enable the computation of high-order accurate stochastic solutions with substantially less computational effort than Monte Carlo simulations. Two prominent variants of the stochastic finite element method are the stochastic collocation method [BNT07, MHZ05] and the stochastic Galerkin finite element method [GS03, BTZ05]. Both approaches transform a stochastic PDE into a set of deterministic PDEs. The former samples the stochastic PDE at a set of well-chosen collocation points in order to obtain an exponential convergence rate. The stochastic Galerkin method applies spectral finite element theory to transform a stochastic PDE into a coupled set of deterministic PDEs. The number of deterministic PDEs is generally smaller than for the stochastic collocation method, but the system is more complicated to solve. As a consequence, specialized solution methods are required since solvers for the original deterministic problem cannot be straightforwardly reused.

1.2

Main research objectives

Different ways of enhancing the performance of the stochastic Galerkin method have been investigated in recent years. For example, by carefully choosing the polynomials to include in the random polynomial basis, the size [BS09, BAS10] and complexity [BTZ05] of the stochastic discretization can be reduced. By developing multi-level solution algorithms, the cost of solving the discretized systems can be optimized [LKD+ 03, EF07, SRNV07, PE09]. The main objective of this work is to decrease the computational cost of the stochastic Galerkin finite element method by constructing and analyzing efficient iterative solvers. For many deterministic problems, multigrid algorithms lead to very efficient and robust solvers [TOS01]. This work aims at extending such methods to stochastic problems. The goal is to create solution methods with optimal convergence properties, i.e., a convergence rate that is independent of discretization parameters, as well as with a low computational complexity. Linear, nonlinear, stationary and time-dependent stochastic PDEs are considered, each requiring a different solution approach. The practical applicability of the methods will be demonstrated by numerical experiments on nontrivial uncertainty quantification problems, for example resulting from electromagnetic and bio-engineering applications. The construction of specialized iterative solvers can be avoided by employing

OUTLINE

3

Chapter 2

stochastic Galerkin method model problem and properties

Chapter 3

linear time-independent iterative solvers

Chapter 4

convergence analysis

Chapter 5

time-dependent Chapter 6

nonlinear bio-application

comparison with

stochastic collocation

Chapter 7

electrotechnical application Figure 1.1: Schematic outline.

stochastic collocation techniques instead of stochastic Galerkin methods. These require however more systems to be solved than in the stochastic Galerkin case. Research on improving the efficiency of the stochastic Galerkin method can therefore not do without a performance evaluation of the stochastic Galerkin method with the stochastic collocation approach. The aim is to increase the efficiency of the stochastic Galerkin method by developing specialized iterative solvers and to determine for which cases either the stochastic Galerkin or the stochastic collocation method is most appropriate.

1.3

Outline

The structure of this work and the connections between the chapters are graphically illustrated in Fig. 1.1. An overview of uncertainty quantification methods with emphasis on the stochastic Galerkin finite element method is given in Chapter 2. The properties of the stochastic Galerkin discretization matrices are discussed in depth since these can be employed to optimize and tune iterative solution methods. The detailed description of the stochastic Galerkin discretization of a stochastic diffusion problem is given, which is used as model problem throughout this work. The contents of Chapter 2 mainly originates from a literature study. Some of the properties of stochastic Galerkin discretization matrices discussed in Chapter 2 have been published in [RV10].

4

INTRODUCTION

Chapter 3 details the construction of efficient one-level and multi-level iterative solution methods for stochastic Galerkin discretizations of linear, time-independent stochastic PDEs. Special attention is given to the robustness and computational cost of the solvers, which are verified by extensive numerical experiments. The convergence properties are theoretically analyzed in Chapter 4, where the results of a local Fourier analysis are given. The results of Chapters 3 and 4 were published in [SRNV07, RV10]. Chapter 5 deals with time-dependent stochastic PDEs. The stochastic Galerkin discretization is combined with a high-order implicit Runge-Kutta time discretization. Appropriate multigrid solvers are constructed and analyzed. This chapter largely corresponds to the contents of the article [RBV08]. In Chapter 6, the case of a system of nonlinear, time-dependent stochastic PDEs is studied. To that end, a set of reaction-diffusion equations, which are used to model the conversion from starch to sugar in growing apples, is considered. After a Newton linearization, the stochastic Galerkin finite element method and an implicit Runge-Kutta time discretization are applied. Multigrid solution approaches proposed in previous chapters are extended to the system under consideration. The work of this chapter is not yet published. The above study of iterative solvers for stochastic Galerkin discretizations is completed with a comparison between the stochastic Galerkin and the stochastic collocation method in Chapter 7. In particular, a nonlinear stochastic convectiondiffusion problem is considered, which models the magnetic vector potential in a ferromagnetic cylinder rotating at high speed. The results on the electrotechnical application are published in [RDV10a, RDV10b]. Chapter 8 formulates the conclusions of this work, summarizes the main contributions, and indicates possible future research directions. An appendix on probability distributions and random fields follows this chapter.

Chapter 2

Partial differential equations with stochastic coefficients 2.1

Introduction

Solving PDEs with stochastic coefficients can be a laborious task. The stochastic Galerkin method is a recent technique which aims at efficiently solving stochastic PDEs. This chapter introduces the concept of PDEs with random coefficients and the stochastic Galerkin finite element method. The discretization is detailed for a steady-state diffusion problem with stochastic diffusion coefficient. This model problem enables us to focus on the difficulties of a stochastic discretization since the discretization of the deterministic part is well understood, see, e.g., [ESW05]. This chapter investigates closely the properties of the algebraic systems that result from a stochastic Galerkin discretization. Knowledge of the matrix elements, sparsity pattern and eigenvalue bounds can be employed in order to optimize the implementation and construction of iterative solvers. Properties of the stochastic Galerkin discretization have been published in [RV10]. This chapter is organized a follows. Section 2.2 gives an overview of uncertainty quantification methods and of the probabilistic representation of uncertainty quantification problems. The stochastic Galerkin method makes use of discrete approximations for random fields and processes. These are outlined in Section 2.3. A description of the stochastic Galerkin finite element method, its convergence properties and its application to a model problem are detailed in Section 2.4. In Section 2.5, the properties of the stochastic Galerkin discretization matrices and implications for the global algebraic system are presented.

5

6

2.2 2.2.1

PARTIAL DIFFERENTIAL EQUATIONS WITH STOCHASTIC COEFFICIENTS

Uncertainty quantification Epistemic versus aleatoric uncertainty

When describing sources of uncertainty in a physical model, different types of uncertainty can be encountered. These give rise to a different terminology, and possibly to a different solution strategy. A common classification [DD09, Naj09] distinguishes between epistemic and aleatoric uncertainty. Epistemic uncertainty, also called model uncertainty, is caused by a lack of knowledge in order to fully characterize the parameters in a mathematical model. This source of uncertainty is reducible, i.e., when additional data becomes available, the uncertainty can be diminished or the exact data values can be determined. Geometric uncertainties, controlled by a production process and qualified by manufacturer’s tolerances, are an example of epistemic uncertainties. Also the case of insufficient experiments to determine a parameter or unpredictable changes such as aging of a structure fall into this category. Aleatoric uncertainty is caused by an intrinsic variability of physical quantities. It is therefore irreducible, i.e., the uncertainty is inherent to certain parameters. Examples of aleatoric uncertainty include the wind pressure on a structure, the mechanical properties of many bio-materials, polymeric fluids or composite materials and random geometric roughness. The variability is often described in a probabilistic framework as a random field or random process [Adl81]. Most engineering systems involve both types of uncertainty. A clear distinction is not always possible. Next to distinguishing between reducible and irreducible uncertainty, also a distinction between error and uncertainty is required. The mathematical description of a physical system yields a modelling error. The discretization of this model in turn yields a discretization error. These error terms, including the errors resulting from solving the discretized models, are not considered in uncertainty quantification, but are important for error control and accuracy tests of a numerical scheme. The discretization error is investigated when determining the rate of convergence of a numerical discretization scheme.

2.2.2

Classification of uncertainty quantification methods

The various sources of uncertainty require different measures, and consequently, a variety of uncertainty quantification methods exists. Fig. 2.1 illustrates a common classification of computational approaches [BTZ05]. The first division into worst-case analysis, possibilistic and probabilistic techniques reflects three different uncertainty measures. Uncertainty can be measured by an interval, by a membership function or by a probability density function. The former requires the least

UNCERTAINTY QUANTIFICATION

7

uncertainty quantification

possibilistic techniques

worst-case analysis

fuzzy sets theory

interval methods probabilistic techniques

sampling methods

non-sampling methods

Monte Carlo simulations

perturbation methods

stochastic collocation methods

stochastic Galerkin methods

Figure 2.1: Classification of uncertainty quantification methods.

information. An application of a worst-case scenario analysis is given in [BNT05] where the solution is constructed based on a perturbation of the data around their nominal values. A membership function defines a fuzzy set and is used in possibility analysis [Zad65, Ber99]. The last case uses probability theory to quantify uncertainty. Whether a possibilistic or probabilistic approach is most appropriate depends on the available data and the inherent variability of the uncertainty. A comparison between probabilistic and fuzzy set based methods [MNHC97] shows that uncertainty is better represented by a stochastic description when enough statistical information is available. Otherwise fuzzy theory is to be applied. This work assumes that a full statistical description of the uncertain parameters is available. Hence, we will deal only with probabilistic approaches. Amongst the probabilistic techniques, sampling and non-sampling methods can be distinguished. This leads to a set of non-intrusive and intrusive solvers. The former approach enables a direct reuse of deterministic simulation code, i.e., solvers for the mathematical problem obtained after replacing all uncertain parameters by a fixed deterministic value. The deterministic solver is used without modification as a building block in a sampling approach in order to determine the output statistics. An intrusive solver, on the other hand, is generally more cumbersome to implement and may require major changes to an existing deterministic simulation code. Monte Carlo simulations. The Monte Carlo simulation method [Fis96, Caf98] is probably the most well known, and also most widely used, statistical solution approach for uncertainty quantification problems. The random parameters are sampled a sufficient number of times and for each sample a deterministic problem

8


is solved. From the combined deterministic solutions, the solution statistics can be extracted. The simplicity in application and robustness are the main strengths of this method. Its performance is however limited by a slow convergence of the solution√statistics w.r.t. the number of samples: the mean value typically converges as 1/ Nmc , with Nmc the number of Monte Carlo samples. Different sampling approaches can be applied, e.g., random sampling, deterministic sampling based on equally spaced samples, or stratified sampling, where the random domain is divided into segments and random samples are generated for each segment. The convergence of Monte Carlo can be accelerated by, for example, variance reduction techniques [Caf98], Quasi-Monte Carlo [Nie04], Markov Chain Monte Carlo [GRS98] or Latin Hypercube Sampling [Loh96], which is a special case of stratified sampling. Monte Carlo simulations are analyzed and applied to stochastic PDEs with a spatial finite element discretization in [BTZ04, BTZ05, CDS09]. A sparse Monte Carlo finite element method for computing the mean and higher order moments of the solution of a stochastic PDE is described in [vS06]. Perturbation methods. Perturbation methods approximate moments of the solution by constructing a Taylor series expansion of the solution around its mean value [KH92]. No sampling is required to perform a perturbation analysis. Typically only second-order approximations of the first and second moment of the solution are computed, which requires about twice or three times the amount of work to solve the corresponding deterministic problem. Performing a perturbation analysis is therefore convenient for cases where higher-order stochastic simulation approaches are too expensive [LSK08]. The applicability of the perturbation method is however limited to low-order approximations and small perturbations. Also the probability density function cannot readily be extracted. In the case of random input fields, the perturbation method can be combined with a truncated Karhunen-Loève expansion of the random fields [BC02, ZL04]. Neumann expansion methods. Closely related to perturbation methods are Neumann expansion methods [GS91, SD88]. These methods are based on the computation of a Neumann series expansion of the solution around its mean value. These methods are again restricted to small parameter uncertainties. An analysis of Neumann expansion methods and an improved Neumann scheme are given in [BC02]. Stochastic Galerkin method. The stochastic Galerkin finite element method (SGFEM) aims at constructing a high-order accurate stochastic solution, including information on the probability density function, while reducing the large computational cost of Monte Carlo simulations. The method was originally proposed by Ghanem and Spanos [GS91] as a spectral discretization technique in the random dimension. It converts a stochastic PDE into a coupled system of


9

deterministic PDEs. The stochastic Galerkin method is highly intrusive and requires specialized methods in order to solve the coupled set of deterministic equations efficiently. In [GS91], the spectral random discretization is combined with a finite element spatial discretization, hence the name ‘stochastic finite element method’. During the last decades, many variants of the original stochastic Galerkin finite element method were proposed, which are summarized in recent review papers [Mat08, Xiu09, Naj09, NL09, Ste09]. Recent trends in the development of the stochastic Galerkin method seek to reduce the resulting set of deterministic PDEs, for example by adaptively refining the stochastic discretization [WK09a], by constructing a sparse stochastic representation of the solution [BS09, BAS10], or by applying an approximate Karhunen-Loève expansion for the solution [GSD07, DGRH07, AZ06]. Stochastic collocation method. The stochastic collocation finite element technique (SCFEM) [XH05, Xiu07, BNT07, MHZ05] samples a stochastic PDE at a multidimensional set of collocation points and, subsequently, applies a Lagrange interpolation in the stochastic dimension to construct the probability density function of the solution. Similar to the Monte Carlo method, this is a non-intrusive sampling based method. It can however achieve a fast convergence rate depending on the choice of collocation points [BNT07, BNT10]. Several variants of the stochastic collocation algorithm exist [Xiu09]. The stochastic collocation method can also be developed as a non-intrusive variant of the stochastic Galerkin method. For example, the non-intrusive spectral projection method solves the stochastic Galerkin equations with Monte Carlo simulations [RNGK03] or cubature rules [LRN+ 02, KL06]. Closely related are the non-intrusive stochastic Galerkin method [AZ07], the stochastic projection method [LRN+ 02, LRD+ 04] and the probabilistic collocation method [LWB07]. The stochastic collocation method suffers from a curse of dimensionality. That is, the computational cost of the stochastic collocation method typically grows exponentially with respect to the number of random variables present in the problem. Sparse grid [NTW08b] and anisotropic sparse grid [NTW08a, GZ07] stochastic collocation approaches can alleviate this problem. A similar curse of dimensionality is present in the stochastic Galerkin method, but not in the Monte Carlo method where the number of samples only grows polynomially with respect to the number of random variables [BTZ05]. The stochastic Galerkin and stochastic collocation methods both yield high-order stochastic solutions, but at a different computational and implementation cost. In Chapter 7, an overview of the stochastic collocation method is given, together with a qualitative and quantitative comparison of the stochastic Galerkin and collocation approach.

10


2.2.3

Probabilistic representation

Mathematical models of real life engineering and physical processes often take the form of a stochastic PDE, with coefficients that are random variables, fields or processes. This can be represented as L(x, t, ω; u(x, t, ω)) = f (x, t, ω),

x ∈ D ⊂ Rd , t ∈ [0, Tf ], ω ∈ Ω,

(2.1)

where f is a forcing term, L a partial differential operator and u the unknown solution function. The PDE (2.1) is completed with suitable boundary and initial conditions. The problem is defined on a d-dimensional spatial domain D and a time interval [0, Tf ]. The stochastic nature of the problem is expressed by the variable ω. This variable represents an element of a sample space Ω, which defines, together with a σ-algebra F and a probability measure E, a complete probability space (Ω, F , E). Typically, the uncertainty is expressed by a function of elements of a sample space, i.e., by random variables, random fields or random processes. A solid overview of probability theory can be found in the books [Gri02, Van84, GD04]. Random variable A random variable ξi (ω) is a measurable function which maps elements from a sample space to real values, ξi : Ω → R. Further on, we will write ξi instead of ξi (ω). The expected value or expectation of a random variable, denoted as hξi i, is defined as the integral of the random variable with respect to the probability measure E. When the probability density function ̺i (yi ) of ξi is defined, for yi ∈ Γi the range of ξi , the expected value corresponds to Z Z yi ̺i (yi )dyi . ξi dE = hξi i := Ω

Γi

An overview of classical distribution functions is given in Table A.1. The variance of a random variable, denoted by var(ξi ), is defined as the second central moment, var(ξi ) := h(ξi − hξi i)2 i. A second-order random variable has a finite variance. The standard deviation equals the square root of the variance. Multiple random variables, ξ1 , . . . , ξL , can be grouped together into a random vector ξ = [ξ1 , . . . , ξL ]T , with L ∈ N0 . When all random variables in ξ are independent, the joint probability density function ̺ of ξ corresponds to the product of the marginal probability density functions, ̺(y) =

L Y

i=1

̺i (yi ) for

y = (y1 , . . . , yL ) ∈ Γ, with Γ =

L Y

i=1

Γi .


11

Independent random variables are also uncorrelated, the reverse is generally not true, except, e.g., in the case of Gaussian random variables. Random process, random field and random wave A random process a(t, ω), with t ∈ T = [0, Tf ], is defined by a mapping T × Ω → R. At every time t ∈ T , a random process corresponds to a random variable. Also, each sample of a random process is a real-valued function in time. This concept models stochastic time series, e.g., the evolution of shares at a stock market. Similarly, a random field a(x, ω), with x ∈ D, is defined by the mapping D × Ω → R. A typical application is a stochastic material parameter that represents a property of a heterogeneous mixture of materials. A random wave a(x, t, ω) generalizes the previous concepts and is defined by the mapping D × T × Ω → R. Note that some authors also apply the term random process to denote a random field or wave. In many applications, Gaussian random processes or random fields are encountered. These processes are completely characterized by their mean and covariance function. The covariance function of a random field a(x, ω), denoted by cova (x1 , x2 ), is defined as cova (x1 , x2 ) := h(a(x1 , ω) − ha(x1 , ω)i)(a(x2 , ω) − ha(x2 , ω)i)i

(2.2)

When the covariance function of a(x, ω) is zero for all x1 6= x2 , a(x, ω) is a white noise random field. Otherwise, a(x, ω) characterizes colored noise. The covariance function (2.2) is a symmetric, positive semi-definite function. Thus, a complete set of orthonormal eigenfunctions and eigenvalues exists. The eigenvalues vi (x) and eigenfunctions λi of a covariance function are the solution of a homogeneous Fredholm integral equation of the second kind, which is given by, Z cova (x1 , x2 )vi (x1 )dx1 = λi vi (x2 ). (2.3) D

Generally, the eigenvalue integral problem (2.3) is solved numerically, e.g., with a Galerkin or collocation type procedure [GS03]. An analysis of the convergence of the Galerkin solution of (2.3) is given in [Tod06]. The high computational and memory cost of Galerkin methods can be alleviated by applying a wavelet-Galerkin approach [PHQ02a] or a Fast Multipole Method [FST05, ST06]. Eiermann et al. [EEU07] discuss another efficient solution procedure, which combines the Lanczosbased thick-restart method with hierarchical matrices. The use of hierarchical matrices for solving (2.3) is explored thoroughly by Khoromskij et al. in [KLM09].

12


2.3

Discrete probabilistic framework

Before solving a stochastic PDE, (2.1) is typically rewritten so that the deterministic and stochastic variables are separated. In the case of random processes, fields or waves, series expansions can be constructed in order to achieve such a separation. This section discusses the two most commonly used expansions, a Karhunen-Loève and a polynomial chaos expansion, applied to a random field a(x, ω).

2.3.1

Karhunen-Lo` eve expansion

Theorem 2.3.1 (Karhunen-Loève expansion [Loè77, GS91]). Given a(x, ω), a second-order random field with mean hai(x) and covariance function cova (x1 , x2 ), then a can be expanded as a(x, ω) = hai(x) +

∞ p X λi vi (x)ξi (ω),

(2.4)

i=1

where λi and vi (x) correspond respectively to the eigenvalues and eigenfunctions of the covariance function cova . The random variables ξi are uncorrelated random variables with zero mean and unit variance, hξi i = 0

and

hξi ξj i = δi,j

∀i, j ≥ 1,

and defined by the equation, Z 1 (a(x, ω) − hai(x))vi (x)dx. ξi (ω) := √ λi D The eigenvalues of the covariance function are positive and sorted in decreasing order, i.e., λ1 ≥ λ2 ≥ . . . ≥ 0. The random variables in (2.4) are uncorrelated, but generally not independent. Only the case of Gaussian random fields yields independent random variables, which are standard normally distributed. For smooth weakly non-Gaussian processes, an algorithm to construct the marginal distribution of the random variables ξi is proposed in [PHQ02b]. Truncated Karhunen-Lo` eve expansion. In practical computations, a random field a is discretely represented by a finite number of random variables. To that end, the infinite summation in (2.4) is truncated after L terms: a(L) (x) := hai(x) +

L p X λi vi (x)ξi (ω). i=1

(2.5)

DISCRETE PROBABILISTIC FRAMEWORK

13

The truncated KL-expansion can be used as a tool for dimension reduction [BS09]. By definition, a random field corresponds to an infinite number of random variables, i.e., one for each spatial point. The truncated KL-expansion (2.5) enables one to reduce this infinite dimensionality to a finite number of random dimensions. The convergence of (2.5) to a(x, ω) follows from theorem 2.3.2 which shows that this expansion is mean-square convergent to a. Theorem 2.3.2. [Mercer’s theorem [BTZ04]] The truncated KL expansion (2.5) of a second-order random field a converges uniformly to a, ! ∞ 2 X sup (x) = sup a − a(L) λi vi2 (x) → 0 for L → ∞. (2.6) x∈D

x∈D

i=L+1

Eq. (2.6) indicates that fewer terms in expansion (2.4) are required to represent a random field with a certain accuracy when the eigenvalues λi decay rapidly to zero. The decay of the KL eigenvalues is influenced, e.g., by the correlation length of the random process. The correlation length is a measure of the range over which variations in one spatial region are correlated with those in another region. The smaller the correlation length, the more terms that need to be used in (2.5) in order to get an accurate approximation [SG04]. The KL eigenvalues decay slowly when the correlation length is small in comparison to the size of the spatial domain D. Another characteristic of the random field that influences the eigenvalue decay is smoothness. A smooth covariance function often leads to a fast decay of the eigenvalues in comparison to less smooth covariance models. A thorough study of the convergence of the truncated KL-expansion is presented in [HQP01]. Covariance functions. The covariance function of a has to be known in order to construct a KL-expansion. When only experimental data are available, an approximation to a truncated KL-expansion can be constructed [BLT03]. In many cases [Gha98, ZL04, EF07, BC02], an exponential covariance function is considered, cova (x1 , x2 ) = σ 2 exp

−kx1 − x2 k1 , lc

(2.7)

Pd with lc the correlation length, σ 2 the variance, and 1-norm kxk1 := i=1 |xd |. Analytical expressions for the eigenvalues and eigenfunctions of this covariance function can be constructed, see Eq. (A.1)–(A.2) in appendix A. Fig. 2.2 illustrates the eigenvalue decay of the exponential covariance kernel (2.7), together with some eigenfunctions, for σ 2 = 0.1, lc = 1 and D = [0, 1]2 . Although an exponential covariance function is often used to model a 2D random field, a Bessel covariance function is more suited for 2D problems from a physical point of view [XK02a].

14


0.5 1

−4

10

eigenfunction v6

1

0

10

20

30

index

40

2 0

−2 1

1

0.5

x2

0.5

0 0

1

0.5

x2

0.5

−2 1

1

0.5

0.5

0 0

1

0.5

x2

0

x2

0

x1

0 0

2

x1

2

−2 1

eigenfunction v36

eigenfunction v1

−2

10

eigenfunction v19

eigenvalue

1.5

eigenfunction v3

0

10

0.5

0 0

x1

0 0

x1

2 0

−2 1

1

0.5

x2

x1

0.5

Figure 2.2: Eigenvalues and eigenfunctions of the exponential covariance function (2.7), with σ 2 = 0.1, lc = 1, and D = [0, 1]2 .

2.3.2

Polynomial chaos expansion

Theorem 2.3.3 (Polynomial chaos expansion [GS03]). Given a second-order random field a(x, ω) and a set of orthonormal Gaussian random variables ξ = {ξi }∞ i=1 . The polynomial chaos expansion of a is the infinite expansion with multivariate Hermite polynomials {Ψq }∞ q=1 , a(x, ω) =

∞ X

aq (x)Ψq (ξ),

(2.8)

q=1

where the coefficient aq (x) is the projection of a onto the respective orthogonal polynomial Ψq , i.e., aq (x) :=

ha(x, ω)Ψq (ξ)i . hΨ2q (ξ)i

(2.9)

This polynomial chaos (PC) representation, also called Wiener-Hermite or FourierHermite expansion, is based on the concept of homogeneous chaos introduced by Wiener [Wie38]. The homogeneous chaos of order P is defined as a subspace of a Hilbert space L2 spanned by the set {ξi }∞ i=1 , where Z L2 := L2 (Ω, F , E) = {X : Ω → R measurable, X 2 (ω)dE(ω) < ∞}. (2.10) Ω

This space is equipped with the inner product Z X(ω)Y (ω)dE(ω), X, Y ∈ L2 . hXY i := Ω


15

The homogeneous chaos of order P contains all polynomials in ξ of exactly degree P and orthogonal to all polynomials of lower degree. The polynomial chaos of order P corresponds to the homogeneous chaos of order 0 up to the homogeneous chaos of order P . A finite dimensional polynomial chaos of order P is a subset of the P th polynomial chaos and is function of only L independent random variables, ξ = [ξ1 , . . . , ξL ]T , L < ∞. It yields an orthonormal basis for a finite dimensional subspace of L2 (Ω, F , E). In the case of L random variables and a polynomial order P , the number of basis polynomials equals Q, with Q given by Q=

(L + P )! . L!P !

(2.11)

This enables one to approximate a by a truncated PC expansion, a(x, ω) ≈

Q X

aq (x)Ψq (ξ).

(2.12)

q=1

The convergence of the polynomial chaos expansion follows from the Cameron and Martin theorem, see theorem 2.3.4. Theorem 2.3.4 (Cameron and Martin theorem [CM47]). The Fourier-Hermite series of any (real or complex) element of L2 converges in L2 -sense to this element. Theorem 2.3.4 implies that a polynomial chaos expansion converges to a random field with finite second-order moments. An extension of this theorem to a Charlier polynomial expansion of Poisson processes is given in [Ogu72]. The accuracy of truncated polynomial chaos approximations to non-Gaussian random processes is evaluated experimentally in [FG04]. Remark 2.3.5. In contrast to the KL-expansion which requires colored noise, a Wiener-Hermite expansion is applicable to both colored and white noise. In the latter case, Itˆ o-calculus is used to solve the stochastic PDE (2.1), and the concept of Wick products is required for a correct discretization of (2.1) [MB99, HLRZ06]. In this work, however, we restricted ourselves to the case of colored noise. Polynomial chaos construction The construction of PC expansions (2.8) and (2.12) requires a multidimensional set of Hermite polynomials. Each polynomial Ψq consists of a product of onedimensional (1D) Hermite polynomials [MK05], Ψq (ξ) :=

L Y

i=1

ψηq,i (ξi ).

(2.13)

16


q 1 2 3 4 5

[0 [1 [0 [0 [2

0 0 1 0 0

ηq 0] 0] 0] 1] 0]

Ψq 1 ξ1 ξ2 ξ3 √ 2 2 2 (ξ1 − 1)

q 6 7 8 9 10

[1 [1 [0 [0 [0

1 0 2 1 0

ηq 0] 1] 0] 1] 2]

Ψq ξ1 ξ2 ξ1 ξ3 √ 2 2 (ξ − 1) 2 2 ξ ξ 2 3 √ 2 2 (ξ − 1) 3 2

Table 2.1: Normalized Hermite polynomials Ψq and multi-index ηq corresponding to a 3-dimensional polynomial chaos of order 2.

The multi-index ηq = [ηq,1 . . . ηq,L ] ∈ NL collects the degrees of the 1D factors PL and satisfies i=1 ηq,i ≤ P so that the total polynomial degree is bounded by P . Table 2.1 shows ηq for L = 3 and P = 2. The one-dimensional polynomials ψj (z), with the subscript j denoting the polynomial degree, are characterized by a three-term recurrence relation [Sze67], ψj+1 (z) = (aj z + bj )ψj (z) − cj ψj−1 (z)

∀j ≥ 1,

(2.14)

with aj , bj and cj constants, and aj , cj > 0. For completeness, we also define a0 and b0 so that ψ1 (z) = (a0 z + b0 )ψ0 , ψ0 (z) = ψ0 and c0 = 0. The recurrence relation 2.14 is used to determine properties of the stochastic Galerkin discretization in Section 2.5.1. The polynomials Ψq are normalized w.r.t. the Gaussian probability density function, i.e., Z 1 y2 Ψi (y)Ψj (y) p hΨi (ξ)Ψj (ξ)i = dy = δi,j , (2.15) exp − 2 (2π)L RL

with δi,j the Kronecker delta. As a result of (2.15), the denominator in (2.9) equals one. Fig. 2.3 illustrates the first six normalized 1D Hermite polynomials. Lognormal random field

A lognormal random field is defined as the exponential of a Gaussian field aG , i.e., alog (x, ω) := exp(aG (x, ω)). Lognormal random fields can be discretely represented by a KL-expansion [PHQ02b] or a PC-expansion [Gha99b]. The latter case is more commonly applied [Gha99c, GSD07, Ull08] since analytic expressions for the PC coefficients (2.9) exist, see theorem 2.3.6. Theorem 2.3.6 (PC representation of a lognormal field [Gha99b]). Consider a Gaussian field aG (x, ω) represented by a p KL-expansion (2.4), and define the functions g1 (x) := haG i(x) and gi (x) := λi−1 vi−1 (x) for i > 1. The coefficients (2.9) of the polynomial chaos expansion (2.8) of the lognormal field


17

4 normalized Hermite polynomial

degree 0 degree 1 2

degree 2 degree 3 degree 4

0

degree 5

-2

-4 -2

2

0 yi

Figure 2.3: Normalized one-dimensional Hermite polynomials ψj , j = 1, . . . , 5.

3

a3 (x)

a1 (x)

2.8

0.5

2.6 1

x2

0 1

1

0.5

0.5

0 0

x2

x1

x2

x1

x2

0 0

0

x2

x1

0 0

x1

0.02

1

0.5

0.5

x1

0.04

−0.05 1

1

0.5

0.5

0 0

a15 (x)

a12 (x)

−0.1 1

1

0.5

0.5

0 0

0.05

0

0

−0.5 1

1

0.5

0.1

a9 (x)

0.5

a2 (x)

1

0.5

0 0

0 1

1

0.5

x2

x1

0.5

Figure 2.4: Polynomial chaos coefficients of a second-order PC expansion (2.12) with L = 4, applied to a lognormal field.

a(x, ω) := exp (aG (x, ω)), are given by ! ∞ 1X 2 a1 (x) = exp g1 (x) + g (x) 2 i=2 i aq (x) = a1 (x)

∞ Y

i=2

√

1 ηq,i−1

(gi (x))

ηq,i−1

(2.16)

q > 1,

(2.17)

18


with ηq,i−1 the (i − 1)th element of the multi-index ηq used to define Ψq (2.13). Proof. See Appendix A. Fig. 2.4 illustrates some polynomial chaos coefficients of a second-order PC expansion of a(x, ω) = exp(aG (x, ω)), where aG (x, ω) is a Gaussian field with mean hαi = 1 and exponential covariance function (2.7), with σ 2 = 0.1 and lc = 0.5. The Gaussian field aG is first discretized by a Karhunen-Loève expansion (2.5) truncated at L = 4.

2.3.3

Extensions of the polynomial chaos expansion

The finite-dimensional polynomial chaos expansion (2.12) is used in many stochastic PDE applications, for representing random inputs as well as unknown solution functions. From the variety of probability distributions, computational challenges and smoothness of random fields, a range of alternative expansions emerged. Generalized polynomial chaos expansion The finite-dimensional polynomial chaos expansion (2.12) is extended to a generalized polynomial chaos (gPC) expansion in [XK02b, XK02a, XK03a]. This expansion uses other random variables than Gaussian, as well as other types of classical orthogonal polynomials than Hermite. The random variables are assumed to be independent. The choice of orthogonal polynomials depends on the distribution of the basis random variables. By applying polynomials that are orthogonal w.r.t. the joint probability distribution of ξ, expansion (2.12) converges exponentially to a finite dimensional random field a based on ξ with respect to the polynomial order P . The relation between orthogonal polynomials and random variables is illustrated in Table 2.2, see also [XK02b, SG04]. These polynomials are part of the so-called Askey scheme of hypergeometric orthogonal polynomials [Sch00]; the gPC expansion is also called Wiener-Askey expansion. The construction of a gPC expansion is analogous to (2.13)–(2.15), where the Hermite polynomials are replaced by another type of orthogonal polynomials and where the weighting function in (2.15) is replaced by the appropriate distribution function, according to Table 2.2. For arbitrary probability density functions, a set of orthogonal polynomials (2.13) can be constructed by a Gram-Schmidt orthogonalization [WSB07, WB08b]. The corresponding expansion (2.12) is then called arbitrary chaos expansion. √ √ consider uniformly distributed random variables defined on [− 3, 3], instead of the more commonly used [−1, 1]-interval. These random variables√have √ zero mean and unit variance. Consequently, Legendre polynomials scaled to the interval [− 3, 3] are used in this work. 1 We


ξi Gaussian Uniform1 Beta Gamma

distribution ̺(yi ) 1 √ exp(−yi2 /2) 2π 1 √ 2 3 yiα−1 (1 − yi )β−1 B(α, β) exp (−yi /θ) yik−1 Γ(k) θk

19

Ψq

support Γi

Hermite

R √ √ [− 3, 3]

Legendre Jacobi

(0, 1)

Laguerre

[0, ∞)

Table 2.2: Correspondence between random variables and orthogonal polynomials from the Askey scheme. The notation of Table A.1 is used.

Tensor product polynomials and double orthogonality P∞ Polynomials constructed according to (2.13) with i=1 ηq,i ≤ P are a subset of full tensor product polynomials for which ηq,i ≤ P holds. The set of full tensor product polynomials defined on L random variables and for which the degree of the univariate factors is bounded by pi forms a Hilbert space with Q=

L Y

(1 + pi )L

i=1

basis polynomials. The number of polynomials Q grows exponentially w.r.t. to the number of random variables restricting the applicability to only a small number of random variables. Tensor product polynomials are particularly suited for a straightforward representation of anisotropy in the stochastic dimension [BTZ04, FST05]. Also, they enable one to construct double orthogonal polynomials [BTZ04, BTZ05, SRNV07]. Definition 2.3.7 (Double orthogonality). Given L independent random variables ξ = (ξ1 , . . . , ξL ) and a full tensor product set of L-dimensional polynomials defined e q1 ,...,qL (ξ) = QL ψepi ,qi (ξi ) with 0 ≤ qi ≤ pi . This set is double on ξ, i.e., Ψ i=1 orthogonal if and only if the following properties are fulfilled ∀i ∈ {1, . . . , L}: e k1 ,...,kL (ξ)i = δ(j ,...,j ),(k ,...,k ) e j1 ,...,jL (ξ)Ψ hΨ 1 L 1 L

e k ,...,k (ξ)i = cp ,j δ(j ,...,j ),(k ,...,k ) , e j ,...,j (ξ)Ψ hξi Ψ i i 1 L 1 L 1 L 1 L

(2.18)

(2.19)

with δ(j1 ,...,jL ),(k1 ,...,kL ) the Kronecker delta and cpi ,ji a constant. e q1 ,...,qL are based on univariate double orthogonal The multivariate polynomials Ψ polynomials which can either be constructed analytically [SRNV07] or by solving

20


double orthogonal polynomial

6

e4,0 ψ

e4,1 ψ

4

e4,2 ψ

2

e4,3 ψ

e4,4 ψ

0 -2 -4 -6

-2

0 yi

2

Figure 2.5: Univariate double orthogonal polynomials (2.20) w.r.t. the standard normal probability density function, for degree pi = 4.

L eigenproblems, each of size (1 + pi ) [BTZ04, BTZ05, EU10]. In the former case, the univariate polynomials are exactly of a specified degree pi and are defined as ψepi ,qi (ξi ) := (−1)pi −qi τpi ,qi

pi Y

r=0, r6=qi

(ξi − ζpi +1,r ),

(2.20)

where the values ζpi +1,r , for r = 0, . . . , pi , are the roots of the normalized univariate polynomial ψpi +1 of degree pi + 1, which is orthogonal w.r.t. the probability measure introduced by ξi : ψpi +1 (ξi ) := τp∗i +1

pi Y

(ξi − ζpi +1,r ).

r=0

The polynomial ψpi +1 corresponds to a univariate polynomial used in the construction of a (generalized) polynomial chaos basis (2.13). The positive normalization constants τpi ,qi and τpi +1 are chosen so that hψepi ,qi (ξi )i = 1 and hψpi +1 (ξi )i = 1. In the case of a standard normal random variable ξi , τpi ,qi and τp∗i +1 are equal to p 1/ (pi + 1)!. The roots of an orthogonal polynomial can be found as the eigenvalues of a tridiagonal matrix, see [Gau04, theorem 1.31]. In Fig. 2.5, a set of univariate double orthogonal polynomials is illustrated for pi = 4. This construction of double orthogonal polynomials enables one to quantify the constant cpi ,ji in definition 2.3.7 as cpi ,ji = ζpi +1,ji , see theorem 2.3.8. Theorem 2.3.8 ([SRNV07]). Univariate double orthogonal polynomials ψepi ,qi (ξi ) of degree pi and defined by (2.20) satisfy hξi ψepi ,ki (ξi )ψepi ,qi (ξi )i = ζpi +1,qi δki ,qi .


21

Proof. The result for ki 6= qi follows immediately from (2.20) by taking into account that ψpi +1 is orthogonal to every polynomial in ξi of degree at most pi : +  * pi Y τ τ p ,q p ,k hξi ψepi ,ki ψepi ,qi i = (−1)ki +qi i ∗i i i ψpi +1 ξi (ξi − ζpi +1,r ) = 0. τpi +1 r=0,r6={ki ,qi }

The proof for ki = qi is based on the following identities: hξi ψepi ,ki i − ζpi +1,ki hψep2i ,ki i = h(ξi − ζpi +1,ki )ψep2i ,ki i =

(−1)pi −ki τpi ,ki hψpi +1 ψepi ,ki i = 0. τp∗i +1

Piecewise polynomial expansions The presented polynomial expansions of the form (2.12) all involve a global stochastic approximation of a random field a. Depending on the smoothness properties of a, local approximations can be more appropriate and may require a lower polynomial degree for representing a with a specified accuracy than the global counterpart. Piecewise polynomial approximation spaces are defined in [BTZ04] and are based on a division of the probability space Ω into a finite number of segments. They enable the construction of a finite element discretization of the stochastic space, similar to spatial finite element discretizations [DBT01]. An example is the Wiener-Haar expansion [LKNG04, LNGK04], which is a wavelet based polynomial chaos expansion. When representing stochastic fields which exhibit a localized sharp variation or a discontinuous change, a wavelet decomposition can be more efficient than a truncated spectral expansion. In [PB06] Wiener-Haar expansions are applied to oscillatory stochastic processes. Piecewise polynomial expansions can also be formulated as an extension to gPC expansions and are then called multi-element gPC expansions [WK06b]. As for Wiener-Haar expansions, a multi-element gPC expansion is appropriate for representing singularities in random space. Adaptive multi-element gPC expansions are presented in [WK05, WK09a] and a multi-element arbitrary chaos expansion is explored in [WK06a, WK06c].

22


2.4

Stochastic Galerkin finite element method

2.4.1

Spectral stochastic discretization

The stochastic Galerkin finite element method [GS03, MK05] applies a spectral Galerkin discretization in the stochastic space. This transforms a stochastic PDE into a system of deterministic PDEs. Given that the stochastic PDE (2.1) can be reformulated with L random variables, ξ = (ξ1 , . . . , ξL ), with joint support Γ, the stochastic discretization constructs an orthogonal basis for a finite dimensional subspace of L2 (Γ). This is achieved by discretizing the solution u by a gPC expansion of order P , u(x, t, ω) ≈

Q X

uq (x, t)Ψq (ξ(ω)),

(2.21)

q=1

where the polynomials Ψq can either be global orthogonal polynomials as in (2.12) or piecewise polynomials as discussed in Section 2.3.3. Global polynomial approximations are particularly attractive in those cases where the solution is smoothly dependent on the input parameters [NT09]. Piecewise polynomial expansions model a non-smooth dependence of the solution on the input parameters more accurately, e.g., in the case of nonlinear oscillatory problems [LKNG04]. The stochastic Galerkin method proceeds by applying the stochastic Galerkin condition to (2.1), with u(x, t, ω) replaced by (2.21). That is, orthogonality of the residual w.r.t. the space spanned by the chosen polynomial basis is imposed, * ! + Q X L x, t, ω; uq (x, t)Ψq (ξ(ω)) Ψk (ξ) = hf (x, t, ω)Ψk (ξ)i , (2.22) q=1

for k = 1, . . . , Q. This yields a coupled system of Q deterministic PDEs in the unknowns uq (x, t), as will be detailed in Section 2.4.2 for the case of a stochastic elliptic PDE. A finite element or finite difference discretization can subsequently be applied to the spatial dimension, along with a suitable time discretization. The inner products h·i cannot be evaluated analytically for general nonlinear problems. In that case, sparse cubature rules can be applied [MK05, Kee04]. Computation of solution statistics After having solved the coupled system of deterministic PDEs (2.22), the statistics of the solution can easily be obtained from the (g)PC representation (2.21). In the case of an orthonormal Wiener-Askey polynomial chaos [XK03a], the mean

STOCHASTIC GALERKIN FINITE ELEMENT METHOD

23

hui and variance var(u) correspond respectively to hui(x, t) = u1 (x, t)

and

var(u)(x, t) =

Q X

(uq (x, t))2 ,

(2.23)

q=2

and the covariance function covu is given by covu (x1 , t1 ; x2 , t2 ) =

Q X

uq (x1 , t1 )uq (x2 , t2 ).

q=2

When double orthogonal polynomials (see definition 2.3.7) are applied in (2.21), the above formulae do not hold. In that case, the following theorem applies. Theorem 2.4.1. The mean and variance of the random wave u(x, t, ω) given by expansion (2.21), with Ψq replaced by an L-dimensional double orthogonal polynoe q1 ,...,qL , as given by definition 2.3.7, and with Q = QL (1 + pi ), satisfy mial Ψ i=1 hui(x, t) =

Q X

var(u)(x, t) =

Q X

q=1

q=1

e q i, uq (x, t)hΨ

e q i2 ) − 2 uq (x, t) (1 − hΨ

Q X

2

! e e uq (x, t)ur (x, t)hΨq ihΨr i ,

r=q+1

e q i = hΨ e q1 ,...,qL i and a 1-to-1 mapping between indices q and (q1 , . . . , qL ). with hΨ

Proof. The result for the mean hui follows directly from the definition of h·i. The

e q yields e q1 ,...,qL by Ψ variance var(u) is defined as (u − hui)2 . Denoting Ψ var(u)(x, t) =

=

=

*

* Q X

Q X q=1

Q X q=1

e q (ξ) − uq (x, t)Ψ

q=1

!2 + e uq (x, t)hΨq i !2 +

e q (ξ) − hΨ e q i) uq (x, t)(Ψ

uq (x, t)2

q=1

+2

Q X

Q Q X X

q=1 r=q+1

2 e q (ξ) − hΨ e qi Ψ

D E e q (ξ) − hΨ e q i)(Ψ′ (ξ) − hΨ′ i) uq (x, t)ur (x, t) (Ψ r r

24


=

Q X

2

uq (x, t)

q=1

e 2i (hΨ q

e qi ) − − hΨ 2

Q X

!

e q ihΨ′ i uq (x, t)ur (x, t)hΨ 2 r r=q+1

.

Applying the orthonormality property (2.18) concludes the proof.

e q ,...,q defined as a product of 1D polynoe q ,...,q i, with Ψ It can be shown that hΨ 1 L 1 L e mials ψpi ,qi (2.20), is given by e q1 ,...,qL i = hΨ

L Y

i=1

1

ψepi ,qi (ζpi +1,qi )

.

(2.24)

The probability density function (PDF) of u can be obtained after sampling the random variables in (2.21) by a Monte Carlo procedure or it can directly be estimated by constructing a multidimensional Edgeworth series, as detailed in [GS03]. Convergence For certain classes of problems, the solution has a very regular dependence on the input random variables. For example, the solution of a linear elliptic PDE with stochastic diffusion coefficient and/or forcing term given by a truncated random field expansion is analytic in the input random variables [BTZ04, BNT07, FST05]. This results in a very fast convergence of the stochastic Galerkin method. Babuˇska et al. [BTZ04, BTZ05] analyzed the convergence of the stochastic Galerkin method applied to a model problem. A global and piecewise polynomial approximation to the stochastic Galerkin solution were considered. An exponential convergence rate of the solution w.r.t. the polynomial degree P was demonstrated for the global polynomial approximation. The exponential convergence was verified numerically for gPC approximations to the solution in [XK02a, XK03b]. The a-priori convergence rates provided by Babuˇska et al. [BTZ04] strongly depend on the number of random dimensions. Depending on the decay rate of the KL- or gPC-coefficients for the stochastic coefficients in the stochastic differential operator L, algebraic or exponential convergence rates can be established, independent of the number of random variables [TS07]. Convergence rates in terms of the number of unknown deterministic functions Q are given in [CDS09]. These convergence rates outperform the 21 -exponent of Monte Carlo convergence rates under mild smoothness conditions on the random diffusion coefficients. It has been shown that the accuracy of a gPC stochastic Galerkin solution decreases in the case of long-term time integrations [WK06b]. As time progresses, the gPC order needs to be raised in order to maintain a constant level of accuracy. This effect occurs when long-term integrations cause discontinuities in


25

the stochastic space or when approximating oscillatory solutions with random frequencies [WK05, LSK06]. A similar effect is observed for gPC expansions of discontinuous solutions or solutions with sharp variations. Piecewise polynomial expansions, e.g., a multi-resolution analysis with finite elements [DBT01], a multielement gPC [WK06c] or a multi-wavelet expansion [LKNG04], result in a robust convergence, albeit at the expense of slower rate of convergence, both w.r.t. the polynomial order and w.r.t. the refinement level of the stochastic discretization.

2.4.2

Stochastic model problem

The remainder of this chapter considers a steady-state stochastic diffusion problem, x ∈ D ⊂ Rd , ω ∈ Ω,

(2.25)

u(x, ω) = gD (x, ω),

x ∈ ∂DD , ω ∈ Ω

(2.26)

∂u(x, ω) = hN (x, ω), ∂n

x ∈ ∂DN , ω ∈ Ω.

(2.27)

−∇ · (a(x, ω)∇u(x, ω)) = f (x, ω),

This model problem is studied by many authors [BTZ04, BTZ05, MK05, XK02a, EF07, PE09, RV10]. It represents the diffusion of a substance in a heterogeneous medium [GZ07] and has important applications in, e.g., groundwater flow [MK05]. The spatial domain D is considered to be deterministically described. This restriction is removed hereafter. In Chapters 3 and 4, we will use this equation to study the convergence properties of iterative solvers. In Chapter 5, the timedependent variant of problem (2.25)–(2.27) is studied. Nonlinear reaction-diffusion and convection-diffusion stochastic problems are considered in Chapters 6 and 7. Overview of assumptions In order to ensure the existence and uniqueness of a (weak) solution to (2.25)– (2.27), several assumptions on the random fields a and f are required [BTZ04, MK05, BS09, BNT07, FST05]. Below these assumptions are summarized. The random fields appearing in the stochastic PDE, a(x, ω), f (x, ω), gD (x, ω) and hN (x, ω), are assumed to be second-order random fields, i.e., with finite variance. Moreover, these fields all represent colored noise. This leads to the finite dimensional noise assumption 2.4.2. Assumption 2.4.2 (Finite dimensional noise [BTZ04, BTZ05, BNT07, FST05, MB99, MK05, ST06]). The random parameters a, f , gD and hN in (2.25)–(2.27) depend on a finite number of random variables.

26


This assumption is satisfied, e.g., when a(x, ω) has the form of a truncated KLexpansion as discussed in Section 2.3. More generally, a stochastic coefficient a(x, ω) can be approximated as a(x, ω) ≈ a(x, ξ) =

S X

ai (x)ϕi (ξ),

(2.28)

i=1

where ξ is a vector of L random variables and ϕi (ξ) a function of ξ. The truncation error of the expansion is determined by S. For example, in case of a KL-expansion (2.5), ϕi (ξ) = ξi−1 and S = L + 1 with ξ0 = 1; in case of gPCexpansion (2.12), ϕi (ξ) corresponds to an orthogonal polynomial in ξ, and S equals the total number of orthogonal, L-dimensional polynomials of a certain order Pa , as determined by relation (2.11). Remark 2.4.3. This work considers a small to moderate number of random dimensions. Typically, L will be in the range 1 − 50. The curse of dimensionality of the stochastic Galerkin and collocation methods prevents their applicability to a larger number of random dimensions. Indeed, the number of stochastic unknowns grows exponentially with the number of random variables L. For the stochastic Galerkin, full tensor stochastic collocation and isotropic sparse stochastic collocation method, the value of Q grows respectively as (P + L)! , Q = nL and O(cL nc (log nc )L−1 ), c, L!P ! with nc the number of stochastic collocation points in each stochastic dimension and c a constant. This dimensionality problem is however the subject of ongoing research, see, e.g., the adaptive sparse collocation strategies in [MZ09, FWK08] or the sparse stochastic Galerkin and collocation finite element methods proposed in [BS09, BAS10]. For a large number of random dimensions, e.g., L ≥ 100, the Monte Carlo method is currently still the best choice [FWK08, BTZ05, XH05], because of its favorable scaling with increasing dimension. Q=

Following the Doob-Dynkin lemma [BTZ04], assumption 2.4.2 yields that the solution u can be expressed as a function of ξ, i.e., u(x, ω) = u(x, ξ). The solution can be seen to be parameterized by the random variables. This parameterization of uncertainty [TS07, FST05, BS09] enables one to rewrite (2.25)–(2.27) as a parametric deterministic problem, −∇ · (a(x, y)∇u(x, y)) = f (x, y), u(x, y) = gD (x, y),

x ∈ D ⊂ Rd , y ∈ Γ,

(2.29)

x ∈ ∂DD , y ∈ Γ,

∂u(x, y) = hN (x, y), x ∈ ∂DN , y ∈ Γ, ∂n where y is an L-dimensional parameter and Γ is the support of the joint probability density function of ξ [BTZ04], or, equivalently, it is the image of ξ(ω) for ω ∈ Ω.


27

Assumption 2.4.4 (Positivity and boundedness [BTZ04, PE09, BS09]). The diffusion coefficient a is bounded and strictly positive, i.e., there exist positive constants amin and amax , with 0 < amin < amax such that E{ω ∈ Ω : amin ≤ a(x, ω) ≤ amax , ∀x ∈ D} = 1.

(2.30)

Assumption 2.4.4 is equivalent to the condition that amin ≤ a(x, y) ≤ amax

x ∈ D, y ∈ Γ.

It ensures the continuity and coercivity of the bilinear form in the weak stochastic Galerkin formulation of (2.25). This is essential for applying the Lax-Milgram theorem [BS02, p.62] in order to prove the existence and uniqueness of a solution to (2.25)–(2.27). From assumption 2.4.4, it follows that a stochastic Galerkin finite element discretization of (2.25)–(2.27) results in a positive definite system matrix. Remark 2.4.5. Assumption 2.4.4 excludes Gaussian diffusion coefficients, even though those are often applied in practice. It turns out however that for a sufficiently small variance sensible solutions can be obtained [Fur08, GS03, XLSK02]. In practice, the discretized stochastic Galerkin system matrix remains positive definite if the stochastic discretization parameters are carefully chosen [PE09]. In order to avoid problems with Gaussian fields, a truncated Gaussian distribution based on Jacobi chaos expansions can be applied [WXK04]. Remark 2.4.6. Also, a lognormal diffusion coefficient does not fulfill the bounds in (2.30). The existence and uniqueness of a stochastic Galerkin solution to (2.25)– (2.27) with lognormal diffusion coefficient is however proven in [Git10] by applying a pointwise extension of the Lax-Milgram theorem. When a lognormal diffusion coefficient is discretized by a Hermite expansion, see theorem 2.3.6, a sufficiently large polynomial order must be applied in order to guarantee the well-posedness of the problem and the positive definiteness of the discretized system [MK05]. In particular, it is shown that the polynomial order applied for discretizing a needs to equal twice the order of the expansion for the solution u. Assumption 2.4.7 (Independence). The random variables ξ = {ξ1 , . . . , ξL } are independent. In the case of a KL-expansion of a Gaussian random field, assumption 2.4.7 is typically automatically satisfied. Also in the case of a KL-expansion with non-Gaussian random variables, assumption 2.4.7 will be invoked, in which case it is a true modelling assumption. The independence of the random variables is required in order to define a (generalized) polynomial chaos. In [BNT07], an auxiliary density function is introduced in order to resolve non-independent random variables. This enables one to construct a generalized polynomial chaos approximation [WK09b]. In addition to the above assumptions, sufficient regularity and smoothness of the diffusion coefficient and right-hand side are required in order to ensure the regularity of the solution. Details can be found in [BTZ05, NTW08b].

28


Stochastic Galerkin discretization A stochastic Galerkin finite element discretization restates (2.29) into its weak formulation. Assuming, for notational simplicity that hN (x, y) = 0, we have: find u ∈ H 1 (D) ⊗ L2 (Γ) such that [EF07] ! Z Z S X ai (x)ϕi (y) ∇u(x, y) · ∇v(x, y)̺(y)dxdy (2.31) Γ

D

=

i=1

Z Z Γ

f (x, y)v(x, y)̺(y)dxdy

D

∀v ∈ H01 (D) ⊗ L2 (Γ).

The definition of the Hilbert space L2 (Γ) follows from (2.10); the tensor products of Hilbert spaces H01 (D) ⊗ L2 (Γ) and H 1 (D) ⊗ L2 (Γ) are detailed in [BTZ04]. The variational formulation (2.31) is discretized by a generalized polynomial chaos expansion (2.21) in the y-dimension and a finite element or finite difference method in the x-dimension. A finite element discretization [BS02] typically uses a conforming triangulation of the spatial domain D. A finite-dimensional subspace of H01 (D) is defined as the span {s1 , . . . , sN }, where sn is a nodal basis function and N the total number of spatial degrees of freedom. The solution is then discretized as u(x, y) ≈

Q X N X

uq,n sn (x)Ψq (y),

(2.32)

q=1 n=1

with uq,n ∈ R, q = 1, . . . , Q, n = 1, . . . , N , a discrete set of scalar unknowns. This enables one to rewrite (2.31) in matrix formulation as ! S X (2.33) Ci ⊗ Ki x = b, i=1

with b, x ∈ RN Q×1 respectively the discrete right-hand side and solution vector. The vector x collects the scalars uq,n of (2.32). The stochastic discretization is expressed by the matrices Ci ∈ RQ×Q , whose elements Ci (j, k) are defined as Z ϕi (y)Ψj (y)Ψk (y)̺(y)dy. Ci (j, k) = Γ

This can be written compactly as Ci = hϕi ΨΨT i

with

Ψ = [Ψ1 . . . ΨQ ]T .

(2.34)

The spatial discretization results in a set of stiffness matrices Ki ∈ RN ×N , with Z ai (x)∇sj (x) · ∇sk (x)dx. (2.35) Ki (j, k) = D


29

The case of a deterministic forcing term f (x, y), i.e., f (x, y) = f (x), yields that ˆ b = c ⊗ fˆ, where the vector c ∈ RQ×1 R is defined as c = hΨi and f is the spatial ˆ discretization of f (x), with f (j) = D f (x)sj (x)dx.

Remark 2.4.8. The algebraic system (2.33) is mathematically equivalent to the following system of matrix equations: S X

Ki XCiT = B,

(2.36)

i=1

with the vector of unknowns x and right-hand side b collected in the multivectors X, B ∈ RN ×Q . That is, for matrix X, we have   u1,1 u2,1 . . . uQ−1,1 uQ,1  ..  . .. X =  ... . .  u1,N

...

uQ,N

From implementation point of view [RBV08, BVlV09], formulation (2.36) is more practical than (2.33) since it enables an easy access of all unknowns per nodal point: this corresponds to a row in the matrix X. Such access will be frequently needed by the iterative solvers discussed in Chapter 3.

2.4.3

More general problems and special cases

As derived in the previous section, the stochastic Galerkin finite element discretization of a stochastic diffusion problem results in an algebraic system with Kronecker product structure (2.33). For more general problems of type (2.22), such a formulation might not always be possible. Alternatively, sometimes simplified algebraic systems can be constructed. In this section, an overview of stochastic Galerkin discretizations of several types of stochastic problems is given. Additive noise. In the class of stochastic PDEs, multiplicative and additive noise can be distinguished. The former occurs when stochastic coefficients are present in the differential operator. An example of additive noise corresponds to the case only the forcing term f in (2.1) is stochastic. Additive noise is generally easier to deal with than multiplicative noise. In that case, the stochastic Galerkin discretization yields a block-diagonal global system matrix, with multiple copies of the deterministic discretization matrix on the diagonal [MK05]. This discrete problem can therefore be formulated as an algebraic system with multiple righthand sides, for which block-iterative solvers have been developed in, e.g., [EEOS05]. The existence of a solution to a stochastic Galerkin discretization of a deterministic problem with stochastic loading is theoretically proven in [ST03, DBT01].

30


Random domain. Problems described on domains with rough boundaries or rough surfaces lead to mathematical models defined on a random domain [BLT03]. A Monte Carlo or stochastic projection method typically requires remeshing the domain for various outcomes of the geometry. This time-consuming remeshing step can be avoided by transforming a random domain into a deterministic domain. To that end, a coordinate transformation [LWSK07, XT06, XS07], a numerical mapping [XS07, CHL06] or a fictitious domain approach [CK07, PP07, NCSM08] can be applied. Afterwards, a stochastic PDE defined on a deterministic domain is to be solved. The former two approaches map the random domain onto a reference domain, while the latter approach extends a PDE into a fictitious domain. The boundary conditions on the original stochastic boundary are enforced via Lagrange multipliers leading to a saddle-point formulation. The fictitious domain approach is more easy to apply to complex geometries than coordinate transformation approaches. However, the introduction of Lagrange multipliers yields more complex linear systems. Example 2.4.9, below, illustrates the coordinate transformation approach. Applications of random domain problems can be found in [CF09, GZ07, WB08b, AA07, LSK08, CHL06], which deal respectively with wind engineering problems, convection problems, heat transfer in channel flow, electrostatics, shock scattering problems and electromagnetic problems. Example 2.4.9. Consider a deterministic problem defined on a stochastic domain, −∇2 u(x, ω) = 1 u(x, ω) = 0

∀x ∈ D(ω)

(2.37)

∀x ∈ ∂D(ω),

(2.38)

with D(ω) = {(x1 , x2 ) | 0 ≤ x1 ≤ 5, 0 ≤ x2 ≤ h(x1 , ω) + k}.

(2.39)

This example is based on a similar problem presented in [XT06]. The upper boundary of the domain is perturbed by a zero mean random process, h(x1 , ω). Some samples of the random domain are shown in Fig. 2.6(a) for k = 1 and h(x1 , ω) discretized by an 8-term KL-expansion based on an exponential covariance function (2.7) with σ 2 = 1/300 and lc = 1. The KL-random √ √ variables are assumed to be independent and uniformly distributed on [− 3, 3]. The random domain (2.39) is mapped to the deterministic domain D = {(x1 , x2 ) | 0 ≤ x1 ≤ 5, 0 ≤ x2 ≤ k} by applying the coordinate transformation, x1 = x1

and

x2 =

x2 . k + h(x1 , ω)


31

x2

1 0.5 0 1

0

2

4

3

5

x1

(a) −4

4 x 10

0.1 0.08

3

hui

var(u)

0.06 0.04

1

0.02

0 0

2

2

1

x1

3

4

5

0 0

1

(b)

2

x1

3

4

5

(c)

Figure 2.6: (a) Four realizations of the random domain (2.39). (b)–(c) Mean and variance of the stochastic Galerkin solution of (2.37)–(2.38) at x2 = 0.75. A secondorder Legendre chaos is applied in (2.21).

This transforms the deterministic PDE (2.37)–(2.38) into the following stochastic PDE for the unknown v(x, ω), −α(x1 , ω)

∂2v ∂2v ∂2v ∂v = α(x1 , ω) 2 − β(x, ω) ∂x ∂x − γ(x, ω) ∂x − ∂x1 ∂x22 1 2 2

x ∈ D, (2.40)

v(x, ω) = 0

x ∈ ∂D.

The coefficients α, β and γ are given by α(x1 , ω) = k 2 + 2kh(x1 , ω) + h(x1 , ω)2 , β(x1 , ω) = −2x2 (k + h(x1 , ω)) γ(x1 , ω) = 2x2

∂h(x1 , ω) ∂x1

2

(2.41)

∂h(x1 , ω) , ∂x1

− x2 (k + h(x1 , ω))

(2.42) ∂ 2 h(x1 , ω) . ∂x21

(2.43)

The partial derivatives in (2.42)–(2.43) can easily be evaluated when h(x1 , ω) is expressed by its truncated KL-expansion (2.5). The coefficients α, β and γ (2.41)–

32


(2.43) are then expressed in terms of L random variables ξi and products of the form ξi ξj . These functions can be represented exactly by a second-order chaos PQp (L+2)! expansion, e.g., β(x, ω) = q=1 βq (x)Ψq (ξ1 , . . . , ξL ) with Qp = 2L! . After replacing v(x, ω) by an appropriate gPC expansion (2.21) in (2.40), the stochastic PDE can be solved by the stochastic Galerkin finite element method. Fig. 2.6(b)– (c) illustrates a cross-section of the mean and variance of the solution at x2 = 0.75. Random boundary conditions. Related to random domain problems are problems with random boundary conditions. In that case, the stochastic Galerkin method can be applied directly; no transformations are necessary [LKD+ 03]. A random Dirichlet boundary condition (2.26), for example, yields after projection onto a (g)PC basis a deterministic Dirichlet boundary condition of the form uq (x) = hgD Ψq i

for q = 1, . . . , Q,

with x ∈ ∂DD .

= hN (x, ξ) with x ∈ ∂DN , can be Random Neumann conditions, e.g., ∂u(x,ξ) ∂n treated implicitly when constructing the variational formulation for the stochastic PDE, thereby requiring the projection of hN onto the (g)PC basis functions.

2.5

Stochastic Galerkin discretization matrices

A stochastic Galerkin finite element discretization of a linear stochastic PDE typically results in a high-dimensional algebraic system of the form (2.33). The matrices Ci (2.34), i = 1, . . . , S, fully characterize the discretization of the stochastic part of (2.25). For example, in case of a truncated KL-expansion in (2.28) with L random variables, we have Ci = hξi−1 ΨΨT i

with

i = 1, . . . , S = L + 1.

(2.44)

When the random input (2.28) is discretized by a gPC-expansion of order Pa , defined on L random variables, the matrices Ci correspond to Ci = hΨi ΨΨT i

with

i = 1, . . . , S =

(L + Pa )! . L!Pa !

(2.45)

From the properties of these stochastic discretization matrices Ci , efficient solvers for the discretized algebraic system (2.33) can be constructed. This section explores the definition of the matrix elements of Ci , their sparsity structure and spectral properties. Also, the properties of the global system matrix (2.33) in case of model problem (2.25) are discussed.

STOCHASTIC GALERKIN DISCRETIZATION MATRICES

2.5.1

33

Definition of matrix elements

Explicit formulae for the matrix elements of (2.44) and (2.45) can be constructed from the properties of the polynomials Ψq . These polynomials are orthonormal w.r.t. the probability measure of ξ, i.e., hΨq Ψk i = δq,k ,

(2.46)

and correspond to a product of one-dimensional orthogonal polynomials ψj , as expressed by (2.13). The recurrence relation for ψj is given by (2.14) with recurrence coefficients aj , bj and cj . Case I: Ci -matrices of the form hξi−1 ΨΨT i Theorem 2.5.1 defines the matrix elements of (2.44). Theorem 2.5.1. Consider matrices Ci , defined by (2.44) and based on orthonormal, multivariate polynomials Ψq satisfying (2.46) and (2.13)–(2.14). Matrix C1 corresponds to the identity matrix IQ . For i = 2, . . . , L + 1, we have that the diagonal elements equal Ci (j, j) = −

bηj,i−1 aηj,i−1

j = 1, . . . , Q,

(2.47)

whereas for j, k = 1, . . . , Q with j 6= k, we have Ci (j, k) =

1 aηj,i−1

δηj,i−1 +1,ηk,i−1 + cηj,i−1 δηj,i−1 ,ηk,i−1 +1

L Y

δηj,m ,ηk,m . (2.48)

m=1,m6=i−1

Proof. The (j, k)-th element of Ci is defined as hξi−1 Ψj Ψk i. The formula for C1 follows straightforwardly from (2.46) and ξ0 = 1. For notational convenience, index “i − 1” is replaced by “r” in the remainder of proof. Based on (2.46) and (2.13)–(2.14), the value of hξr Ψj Ψk i is equal to hξr Ψj Ψk i =

L Y

hξr ψηj,m (ξm )ψηk,m (ξm )i

m=1

= hξr ψηj,r (ξr )ψηk,r (ξr )i

L Y

δηj,m ,ηk,m

m=1,m6=r

34


hξr Ψj Ψk i =

1

aηj,r

L Y δηj,m ,ηk,m ψηj,r +1 − bηj,r ψηj,r + cηj,r ψηj,r −1 ψηk,r m=1,m6=r

δηj,r +1,ηk,r − bηj,r δηj,r ,ηk,r + cηj,r δηj,r ,ηk,r +1 = aηj,r

L Y

δηj,m ,ηk,m .

m=1,m6=r

QL For j 6= k, one has bηj,r δηj,r ,ηk,r m=1,m6=r δηj,m ,ηk,m = 0, and hence (2.48). For j = k, (2.47) follows immediately. Theorem 2.5.1 shows that the Ci -matrices have at most 3 nonzeros per row due to the three-term recurrence relation (2.14). Example 2.5.2. Normalized Hermite polynomials, as shown in Fig. 2.3, satisfy p p j + 1ψj+1 (z) = zψj (z) − jψj−1 (z),

with ψ0 (z) = 1 and ψ1 (z) = z. Theorem 2.5.1 produces the same result for hξi−1 ΨΨT i in the case of a Hermite chaos as reported in [PE09, SD00]: for i > 1, hξi−1 Ψj Ψk i =

√

ηk,i−1 δηj,i−1 +1,ηk,i−1 +

L Y √ ηj,i−1 δηk,i−1 +1,ηj,i−1 δηj,m ,ηk,m . m=1,m6=r

Example 2.5.3. For Legendre polynomials, normalized √ √ w.r.t. the probability density function of uniform random variables on [− 3, 3], recurrence (2.14) equals √ √ j+1 j 3 2j + 1 √ zψj (z) − √ ψj+1 (z) = ψj−1 (z), 3 2j + 3 2j − 1 with ψ0 (z) = 1 and ψ1 (z) = z. Evaluating (2.47)–(2.48) yields that for i > 1 hξi−1 Ψj Ψk i =

L Y ηk,i−1 δηj,i−1 +1,ηk,i−1 + ηj,i−1 δηj,i−1 ,ηk,i−1 +1 √ p 3 δηj,m ,ηk,m . (2ηj,i−1 + 1)(2ηk,i−1 + 1) m=1,m6=r

Example 2.5.4. Laguerre polynomials normalized to the gamma distribution (see table A.1), with shape parameter k and scale parameter θ, are characterized by the recurrence relation: p p −1 z + 2j + k ψj (z) − j(j + k − 1)ψj−1 (z). (j + k)(j + 1)ψj+1 (z) = θ


35

The expectation hξi−1 Ψj Ψk i is then according to (2.47)–(2.48) for i > 1 given by q hξi−1 Ψj Ψq i =θ − ηq,i−1 (ηj,i−1 + k)δηj,i−1 +1,ηq,i−1 + (2ηj,i−1 +k)δηj,i−1 ,ηq,i−1 q − ηj,i−1 (ηq,i−1 + k)δηj,i−1 ,ηq,i−1 +1

L Y

δηj,m ,ηq,m .

m=1,m6=i−1

Remark 2.5.5. The matrices Ci are generally not diagonal. When (2.44) holds, double orthogonal polynomials can be constructed so that Ci , i = 1, . . . , L + 1, is a diagonal matrix. The definition and construction of these polynomials have been given in Section 2.3.3. TheQdimension of the Ci -matrices is then no longer deterL mined by (2.11), but Q = i=1 (1 + pi ), with pi the degree of the 1D polynomial factors. Corollary 2.5.6. The diagonal elements of matrices Ci defined by (2.44), with i = 2, . . . , L + 1 and a P th order generalized polynomial chaos, are zero if the coefficients bj in (2.47) equal zero for j = 0, . . . , P .

Corollary 2.5.6 is particularly useful for reducing the computational cost of block iterative solvers for (2.33) which are based on splitting the Ci -matrices into their diagonal part and the remainder, see Section 3.2.3. Example 2.5.7. The diagonal elements of Ci (2.44), with i > 1, are zero in case of Legendre, Hermite or Chebyshev polynomials, but not in case of Jacobi or Laguerre polynomials. This follows directly from the recurrence relations for the respective 1D orthogonal polynomials, see [Gau04]. Case II: Ci -matrices of the form hΨi ΨΨT i The computation of the matrix elements of Ci in (2.45) can be formulated as a linearization-of-products problem [EU10]. This problem [Ask75, Ans05] deals with the construction of coefficients gjkm such that the product of two orthonormal polynomials ψj ψk can be written as ψj (z)ψk (z) =

j+k X

gjkm ψm (z).

(2.49)

m=0

From the orthonormality it follows that gjkm = hψj ψk ψm i. Explicit formulae for such linearization coefficients exist for various types of orthogonal polynomials. Example 2.5.8. Based on the properties of Hermite polynomials [Sze67, p. 390], the expectation of three multivariate normalized Hermite polynomials equals p p p L Y ηi,m ! ηj,m ! ηk,m ! hΨi Ψj Ψk i = , (wm − ηi,m )!(wm − ηj,m )!(wm − ηk,m )! m=1

36


if 2wm = ηi,m + ηj,m + ηk,m is an even integer and wm ≥ ηi,m , wm ≥ ηj,m , wm ≥ ηk,m , otherwise the result is zero. Theorem 2.5.9. The matrices Ci (2.45) are all-zero matrices for i ≥

(L+2P )! L!(2P )! .

Proof. The proof applies the orthogonality properties of Ψi , see [Kee04]. A stochastic Galerkin discretization of (nonlinear) stochastic PDEs may also require the evaluation of high-order products of polynomials, e.g.,hΨi Ψj Ψk Ψqi. Computational techniques for the corresponding integrals are given in [DNP+ 04, KL06].

2.5.2

Properties of stochastic discretization matrices

Sparsity pattern and structure The matrices Ci (2.34) are symmetric since hϕi Ψj Ψk i = hϕi Ψk Ψj i. Each matrix is sparse due to the orthogonality properties of the polynomials Ψq . Every row of Ci defined as (2.44) contains at most 3 nonzero entries, as follows from theorem 2.5.1. Reordering the polynomials Ψq in Ψ enables one to rewrite Ci (2.44) as a tridiagonal matrix [PE09]. In the case Ci is defined by (2.45), it is possible to construct upper bounds for the number of nonzero entries when the weight function ̺ is even [EU10]. This information is useful for an efficient implementation of matrix-vector multiplications. The matrices Ci possess a hierarchical structure [GK96, PG00]. Given a P th order chaos, each Ci ∈ RQ×Q can be written in block form as Cî FiT Ci = , Cî ∈ RQl ×Ql , Fi ∈ RQh ×Ql , Gi ∈ RQh ×Qh , (2.50) Fi Gi −1)! ˆ with Q = Ql + Qh given by (2.11) and Ql = (L+P L!(P −1)! . Submatrix Ci is defined similarly as Ci , and corresponds to a polynomial chaos of order P − 1. By recursion, Cî has a structure similar to that of (2.50). The recursion terminates with Cî ∈ R1×1 , corresponding to a zeroth order polynomial chaos. This hierarchical structure can be exploited in the construction of iterative solvers; see Sections 3.2.3 and 3.4. Corollary 2.5.10 enables one to reduce the computational work.

Corollary 2.5.10. Let matrix Ci be defined by (2.44) and based on a P th order (generalized) polynomial chaos with L random variables. Then, all off-diagonal elements of subblock Gi in (2.50) equal zero.


37

Proof. The off-diagonal elements of Gi are defined by (2.48), with j 6= k and L X i=1

ηj,i =

L X i=1

ηk,i ≡ P.

(2.51)

Consider the product term in the right-hand side of (2.48). This term is either zero or one. The latter case implies that ηj,m = ηk,m , for all m = 1, . . . , L, with m 6= i − 1. From (2.51) it then follows that ηj,i−1 = ηk,i−1 . However, this corresponds to ηj = ηk and thus j = k, which is not possible for an off-diagonal term. Thus, the product term must equal zero. Eigenvalue bounds Knowledge of the spectrum of the Ci -matrices in (2.33) is important for analyzing and constructing iterative solvers for (2.33). Theorem 2.5.11 provides insight in the eigenvalue bounds of Ci -matrices defined by (2.44) [SRNV07, RBV08]. Theorem 2.5.11. Consider a Ci -matrix, defined by (2.44) and based on a (generalized) polynomial chaos of order P , which is constructed as a product of 1D orthogonal polynomials (2.13). The eigenvalues of Ci lie in the interval [ζP +1,0 ζP +1,P ], where ζP +1,0 is the smallest zero and ζP +1,P the largest zero of a 1D orthogonal polynomial of degree P + 1 and of the type applied in (2.13). Proof. A Ci -matrix (2.34) is based on a set of Q generalized polynomial chaos functions Ψ. This set can be extended to an orthonormal set of Q′ = (P + 1)L basis functions, Ψ = [Ψ1 , . . . , Ψq , ΨQ+1 , . . . , ΨQ′ ]T , which spans the same vector e of definition 2.3.7. Hence, space as the double orthogonal polynomial chaos basis Ψ ¯ e an orthogonal matrix Z exists so that Ψ = Z Ψ. As a consequence, the matrix C i can be diagonalized by the matrix Z: T eΨ e T iZ T . eΨ e T Z T i = Zhξi−1 Ψ C i := hξi−1 ΨΨ i = hξi−1 Z Ψ

Thus, the eigenvalues of C i correspond to the diagonal entries of the diagonal eΨ e T i. According to theorem 2.3.8, these values coincide with the matrix hξi−1 Ψ zeros of the univariate orthogonal polynomials of degree P + 1 that are used to construct Ψ. Since the matrix Ci is a principal submatrix of C i , the eigenvalues of C i determine upper and lower bounds for the eigenvalues of Ci . As a consequence of theorem 2.5.11, zeros of univariate orthogonal polynomials influence the spectral properties of the Ci -matrices. These zeros can be calculated as the eigenvalues of a tridiagonal matrix, see [Gau04, theorem 1.31].

38


Remark 2.5.12. Based on properties of the zeros of orthogonal polynomials, one can show that increasing the polynomial order widens the eigenvalue spectrum of Ci . For polynomials with bounded support, the eigenvalue spectrum is also bounded so that, asymptotically, the polynomial chaos order does not influence the spectrum of Ci . This is the case for Legendre and Jacobi polynomials, which are defined on a finite interval. In the unbounded case, the conditioning of Ci matrices deteriorates for an increasing polynomial order. This occurs for example when Hermite polynomials are applied. Ernst et al. [EU10] analyzed the eigenvalues of hΨi ΨΨT i both for the case of tensor product polynomials and a complete polynomial basis Ψ, as applied in (2.13). Eigenvalue bounds can be obtained from quadrature nodes of appropriate Gaussian quadrature rules. In the univariate case with an even weight function ̺ and a low polynomial order P , exact formulae for the eigenvalues of (2.45) have been derived.

2.5.3

Properties of the global system matrix

Sparsity and structure The Kronecker product structure of the global system matrix (2.33) characterizes the sparsity pattern. Fig. 2.7 illustrates this structure; additional illustrations are given in [EEU07, PG00]. When the diffusion coefficient is discretized by an L-term truncated KL-expansion in (2.28), the number of nonzero N × N blocks per block row is at most 2L + 1 [XS09]. This follows directly from theorem 2.5.1, as discussed in Section 2.5.2. Although in Fig. 2.7 the block structure of the system matrix is sparse, it is also possible that this block structure becomes dense, see corollary 2.5.13. )! Corollary 2.5.13 ([Kee04]). Let Ci , i = 1, . . . , S, with S = (L+2P L!(2P )! , be a set of stochastic discretization matrices, each defined by (2.45), with Ψ a set of Ldimensional orthogonal polynomials of order P . Then the sum of all these matrices, PS i=1 Ci , is a dense matrix.

Proof. The proof is based on theorem 2.5.9 and on an extension of the linearizationof-products rule (2.49) to the multivariate case, see [Kee04].

Diagonalization. Except when double orthogonal polynomials (definition 2.3.7) are used, the algebraic system (2.33) is not block diagonal. When a complete polynomial chaos Ψ (2.13) is used, there exists no orthogonal transformation that can diagonalize the different matrices hξi−1 ΨΨT i simultaneously [SRNV07]. This property is proven in [EU10, theorem 7].

CONCLUSIONS

39

0

0

10

10

20

20

30

30

40

40

50

50

0

10

20

30

40

50

(a) L = 5, P = 3, S = L + 1, Ci = hξi−1 ΨΨT i

0

10

20

30

40

50

(b) L = 5, P = 3, S = Q, Ci = hΨi ΨΨT i

Figure 2.7: Sparsity pattern of the system matrix of (2.33), with Q = 56. Each black square has the size of a Ki -matrix, representing the discretization of the deterministic part of a stochastic PDE.

Positive definiteness. PS Theorem 2.5.14 ([XS09]). Let A = i=1 Ci ⊗ Ki ∈ RN Q×N Q be the stochastic Galerkin finite element discretization of (2.25), where the stochastic coefficient satisfies assumption (2.30) and is discretized by expansion (2.28). Then, A is a positive definite matrix. Proof. The proof is based on the positivity assumption 2.4.4, see [XS09, Fur08]. Remark 2.5.15. In order to assure the positive definiteness of A, a polynomial chaos expansion of a lognormal diffusion coefficient (see theorem 2.3.6) must be based on at least a polynomial order 2P , when the solution is discretized by a PC expansion (2.21) of order P [MK05].

2.6

Conclusions

The stochastic Galerkin discretization of a diffusion problem with stochastic diffusion coefficient results, after a combined use of Karhunen-Loève and (generalized) polynomial chaos expansions, in a deterministic algebraic system with Kronecker product structure. The Kronecker product combines matrices resulting from the spatial and stochastic discretization. Properties of the latter are related to the properties of orthogonal polynomials. The hierarchical structure of the stochastic

40


polynomial basis is reflected in the structure of the stochastic discretization matrices. The orthogonality relationship results in sparse stochastic discretization matrices, although the sum of stochastic discretization matrices can be a dense matrix. Eigenvalue bounds of these matrices depend on the zeros of univariate orthogonal polynomials.

Chapter 3

Iterative solvers for linear stochastic finite element discretizations 3.1

Introduction

The linear system generated by SGFEM basically corresponds to the discretization matrix of a large system of coupled deterministic PDEs. The size of this system grows rapidly with the required spatial accuracy and the required accuracy in the stochastic dimension. By developing multi-level solution algorithms the cost of solving the system can be optimized. This chapter presents iterative solvers for stochastic Galerkin finite element discretizations of steady-state stochastic elliptic problems, given by the model problem (2.25)–(2.27). By exploiting the Kronecker product structure of the discretized system (2.33), efficient solvers can be constructed. We categorize the methods into one-level methods, based on matrix splitting approaches; multigrid methods, which perform a coarsening in the spatial dimension; and multilevel methods which make use of the hierarchical structure of the stochastic discretization. The use of preconditioned Krylov methods is also addressed. The different methods are compared according to their computational cost, difficulty of implementation, and convergence properties. Convergence is studied by numerical experiments. The results of this chapter have been published in [RBV08, RV10]. In Chapter 4, the observed convergence properties are theoretically confirmed by a local Fourier analysis. This chapter is organized as follows. Section 3.2 discusses one-level methods based

41

42

ITERATIVE SOLVERS FOR LINEAR STOCHASTIC FINITE ELEMENT DISCRETIZATIONS

on matrix splitting approaches. Section 3.3.2 presents a multigrid method, which applies the previously discussed one-level methods as smoother. Next, a multilevel method based on a coarsening in the stochastic dimension is proposed in Section 3.4. Section 3.5 discusses preconditioned Krylov subspace methods. Some important implementation aspects are reviewed in Section 3.6. These influence the computational cost of the different solvers. This chapter ends by an extensive parameter study and a numerical comparison of the iterative solvers in Section 3.7.

3.2

One-level methods based on matrix splitting

The Kronecker product system (2.33) can be solved iteratively, based on matrix splitting techniques. Applied to a system Ax = b, a matrix splitting method uses the splitting A = A+ + A− to compute an updated solution xnew from xold as A+ xnew = b − A− xold .

(3.1)

The asymptotic convergence rate of iteration (3.1) is given by − log10 (ρ), with the convergence factor ρ defined as the spectral radius of the iteration operator −(A+ )−1A−. The system matrix (2.33) can be split globally or according to its block structure. Global splitting methods are presented in Section 3.2.1. Based on the block structure of (2.33), a splitting of the stiffness matrices Ki , Ki = Ki+ + Ki− , is possible, as will be discussed in Section 3.2.2. Another approach is based on splitting only the Ci -matrices and is dealt with in Section 3.2.3. Remark 3.2.1. Note that the Kronecker product system matrix (2.33) does not need to be stored in memory explicitly when an iterative solver is used. It suffices to store the matrices Ci and Ki separately in sparse matrix format. The storage is proportional to N for each Ki -matrix, and to Q for each Ci -matrix. The latter follows from the orthogonality of the polynomials Ψq , as discussed in Section 2.5.

3.2.1

Basic splitting methods

By decomposing matrix A into its strictly lower triangular part AL , strictly upper triangular part AU , and diagonal AD , a variety of iterative methods can be constructed. Here, the lexicographic Gauss-Seidel (GSLEX) splitting is considered in more detail. Applied to (2.33) and in the notation of (3.1), this corresponds to A+ = AL + AD =

=

S X i=1

(CiL

+

S X i=1

CiD )

(CiL ⊗ Ki + CiD ⊗ KiL ) +

⊗

(KiL

+

KiD )

+

S X i=1

S X i=1

CiD ⊗ KiD

CiL ⊗ KiU ,

(3.2)

ONE-LEVEL METHODS BASED ON MATRIX SPLITTING

43

with Ci = CiL + CiD + CiU and Ki decomposed analogously. Computational cost. Taking the matrix storage detailed in remark 3.2.1 into account, one can see that a GSLEX-iteration requires a number of multiplications and additions that is proportional to N QS. Although implementing Gauss-Seideltype splitting methods is almost trivial once all matrix elements of Ci and Ki are known, (3.2) leads to a highly intrusive solver.

3.2.2

Splitting of the Ki -matrices

When decomposing only the Ki -matrices in (2.33), the following iteration results: S X i=1

Ci ⊗

Ki+ xnew

=b−

S X i=1

Ci ⊗ Ki− xold .

(3.3)

Different Ki -splittings are discussed in [SRNV07]: pointwise lexicographic or redblack Gauss-Seidel, x-line lexicographic Gauss-Seidel, pointwise damped Jacobi and red-black successive overrelaxation (SOR). For example, when Ki+ corresponds to KiL + KiD , the lower triangular part of Ki , iteration (3.3) is a collective Gauss-Seidel method. In that case, one iteration requires the solution of N systems of size Q × Q, each given by S X i=1

Ki (n, n)Ci x[n] = b[n] −

N X

S X

Ki (n, m)Ci x[m]

(n = 1, . . . , N ), (3.4)

m=1,m6=n i=1

with x[n] ∈ RQ×1 being the Q-vector of unknowns associated with spatial node n. Computational and implementation cost: solving the block systems Optimizing the solution time of the local systems (3.4) is of utmost importance for reducing the computational cost of the Ki -splitting iteration (3.3). One possible approach is to factorize these systems in advance so that in every iteration of (3.3) only matrix-vector multiplications or back substitutions are required. However, the storage of N matrix factorizations (one per spatial grid point) may lead to excessive memory requirements for large values of N or Q. Hence, this approach will not be considered further. In experiments with direct solvers for (3.4), the factorization will be done on the fly. Depending on the properties of the local systems, different solution methods can be selected. Fig. 3.1 shows the average computation time of several solution approaches to solve one local system of the form (3.4). The stochastic model problem (2.25) is solved on a square domain D = [0, 1]2 , discretized with a spatial

Gauss elimination 0 10

CG Sparse LU (umfpack)

10

10

-2

Sparse LU (SuperLU)

-4

1

2

10 10 Dimension block system (Q)

10

3

Average solution time (sec.)



44

0 10

10

-2 Gauss elimination CG

10

-4 0

(a) Gaussian field a

LU (umfpack) 50 100 Dimension block system (Q)

150

(b) Lognormal field a

Figure 3.1: Average computation time to solve one local system (3.4).

P 1 2 4

1 2 3 5

2 3 6 15

4 5 15 70

8 9 45 495

L 10 15 11 16 66 136 1001 3876

20 21 231 10626

40 41 861 135751

Table 3.1: The number of random unknowns Q as a function of the number of random variables L and of the total polynomial degree P .

finite element mesh of 1119 nodes and a Hermite polynomial chaos. The diffusion coefficient is either modelled as a truncated KL-expansion of a Gaussian random field, or as a PC expansion of a lognormal field. The considered methods include an LU solver without pivoting, a sparse LU solver (UMFPACK [Dav04] and SuperLU [DEG+ 99]) and a Krylov method. The tests were performed on an Intel Dual Core 2.0 GHz machine with 2GByte RAM. Values for Q as a function of the number of random variables L and the polynomial order P are given in Table 3.1. The local systems (3.4) are symmetric positive-definite, since the global system matrix is also symmetric positive-definite, see theorem 2.5.14. For Ci -matrices defined as (2.44), e.g., in the case of a Gaussian field a discretized by a KLexpansion, the local systems are sparse, so that the conjugate gradients (CG) method is a suitable solver. As illustrated in Fig. 3.1 (a), the CG solver leads to the best performance. No preconditioning is necessary because the systems are well conditioned. When the Ci -matrices are defined by (2.45), the local systems have the same dimension Q, but are less sparse. In the case of a lognormal diffusion coefficient, the systems (3.4) are entirely dense; this follows from corollary 2.5.13. As a consequence, the local system solves are more time consuming. An LU decomposition yields then the smallest computation time, see Fig. 3.1 (b).

ONE-LEVEL METHODS BASED ON MATRIX SPLITTING

(a) Jacobi

(b) Gauss-Seidel (c) Block Jacobi

45

(d) Block GS

(e) Circulant

Figure 3.2: Nonzero structure of the Ci+ -matrices to be used in iteration (3.5).

3.2.3

Splitting of the Ci -matrices

Applying a splitting to the Ci -matrices leads to the iteration S X i=1

S X

Ci+ ⊗ Ki xnew = b −

i=1

Ci− ⊗ Ki xold .

(3.5)

Fig. 3.2 shows five different constructions for Ci+ . The Gauss-Seidel Ci -splitting corresponds to the decoupled solution algorithm proposed in [FYK07]. In case the conditions of corollary 2.5.6 hold, iteration (3.5) with a Jacobi-type Ci -splitting simplifies to IQ ⊗ K1 xnew = b −

S X i=2

Ci− ⊗ Ki xold .

(3.6)

The block Jacobi and block Gauss-Seidel splittings, illustrated in Fig. 3.2 (c)–(d), are based on the hierarchical block structure (2.50) of the Ci -matrices. Note that when corollary 2.5.10 is applicable, the off-diagonal elements of the Gi -subblock of Ci (2.50) are all zero. The block Jacobi and block Gauss-Seidel Ci -splittings coincide then with the simple Jacobi and Gauss-Seidel splittings, respectively, shown in Fig. 3.2 (a) and (b). In case of a circulant Ci -splitting approach, depicted in Fig. 3.2 (e), matrix Ci+ is constructed as the circulant matrix that minimizes the Frobenius norm kCi+ − Ci kF [CNS00, Tyr92]. Another Ci -splitting approach originates from the Kronecker product preconditioner [Ull10] and is determined by C1+

=

S X tr(K T K1 ) i

i=1

tr(K1T K1 )

Ci ,

and Ci+ = 0 ∈ RQ×Q

for i = 2, . . . , S, (3.7)

P where tr(·) denotes the trace of a matrix; tr(KiT K1 ) := N n=1 Ki (n, n)K1 (n, n). The motivation for this splitting will be given in Section 3.5.

46


Computational and implementation cost: solving the block systems The Gauss-Seidel Ci -splitting (Fig. 3.2 (b)) requires in every iteration the solution of Q systems of size N × N . In case of the diffusion model problem (2.25), these systems can be solved with a standard multigrid cycle [TOS01], accelerated by CG. In [GK96, PG00], the Jacobi Ci -splitting iteration (3.6) is combined with an LU-factorization of K1 , which needs to be computed only once during setup. This approach becomes impractical however for large N . The block Gauss-Seidel Ci -splitting (Fig. 3.2 (c)) requires the solution of P + 1 small Kronecker product systems, with P the polynomial order applied in (2.32). In case of a diffusion problem, these systems can be solved with the multigrid method to be discussed in Section 3.3.2. The circulant splitting approach is implemented efficiently by using FFT techniques [CN96]. Applying an FFT transformation decouples (3.5) into Q independent systems of size N × N . Hence the total cost becomes similar to the cost of a Gauss-Seidel Ci -splitting iteration. The construction of circulant Ci+ -matrices can be done during a setup phase, at a negligible cost. Remark 3.2.2. The cost of a single Gauss-Seidel Ci -splitting iteration is equivalent to the cost of performing Q deterministic simulations. Thus, when many iterations are required, this will rapidly lead to a larger number of simulations, than, for example, required in a stochastic collocation approach. However, by solving the (N × N )-blocks inexactly, as will be suggested in Section 3.5, this computational cost issue is no longer present. Ci -splitting methods require a repeated solve of systems corresponding to the discretization of a deterministic PDE, see, e.g., (3.6). These solves can reuse simulation code available for deterministic PDEs in a black-box way. Hence, Ci -splitting methods can reduce the intrusiveness typically present in the stochastic Galerkin method. The basic and Ki -splitting approaches do not possess this property.

3.3

Multigrid methods in the spatial dimension

The convergence analysis (see Chapter 4) of the splitting iterations presented in Section 3.2 will show that the convergence of these methods, except for the Ci splitting iteration (3.5), strongly depends on the spatial discretization, i.e., on the mesh or grid size h. When fine spatial meshes are used, many thousands of iterations may be required in order to obtain the solution x of system (2.33). A couple of decades ago, a new category of solvers was proposed [Bra77], called multigrid or multi-level methods, which aim at resolving the spatial dependency of the convergence of splitting iterations. These methods are based on a hierarchy

0

−0.05

47

0.1 0.05 0

−0.05

−0.1 1 1 0.5

x1

0.5 0 0

(a)

x2

−0.1 1 1 0.5

x1

0.5 0 0

(b)

x2

error on coarse grid

0.1 0.05

error after 4 GS iterations

initial error

MULTIGRID METHODS IN THE SPATIAL DIMENSION

0.1

0

−0.1 1

1

0.5

x1

0.5 0 0

x2

(c)

Figure 3.3: (a) Initial error when solving a deterministic Poisson problem: a combination of high frequency and low frequency components. (b) Error after 4 lexicographic Gauss-Seidel iterations: the high frequency content of the error has largely vanished. (c) Error after 4 GSLEX iterations represented on a coarser grid.

of spatial discretizations, as opposed to the splitting iterations described so far, which are one-level methods.

3.3.1

Introduction to multigrid

When performing a splitting iteration (3.1), the error after m iterations, em = x − x(m) , can be split into two components: a high frequency error and a low frequency error. Frequency is in this context related to the computational domain (expressed in units per metre) and is not to be interpreted as a temporal frequency (in Hz). Classical splitting iterations typically lead to a fast reduction of the high frequency components, but a very slow convergence of the low frequency components [BHM00]. These methods are said to have a smoothing property, hence the name smoothing or relaxation methods. This smoothing effect is illustrated in Fig. 3.3 (a)–(b). After a few Gauss-Seidel iterations applied to a deterministic Poisson problem, the high frequency components are largely vanished, while many additional iterations are required to reduce the low frequency part of the error. Multigrid is based on the idea that a low frequency error has a high-frequency representation on a coarser grid, see Fig. 3.3(c). After performing some smoothing steps on a fine grid, the error mainly has low frequency components which can be reduced by additional smoothing on a coarser grid. By recursion, the error can be smoothed on the next coarser grid, until the grid is small enough to compute directly the solution. Calculations on coarser grids are performed cheaply due to the smaller system sizes. As a result, multigrid is a very efficient solver for elliptic PDE discretizations. It obtains the solution with an optimal computational complexity of O(N ), in the case of the so-called full multigrid approach, or O(N log N ) in the case of the so-called cycling approach.

48


grid k = 3 grid k = 2 grid k = 1 grid k = 0 V -cycle (γ = 1)

W -cycle (γ = 2)

F -cycle

Figure 3.4: Structure of one multigrid cycle for different values of the cycle index γ.

In order to build a coarse representation of error and residual vectors, a restriction operator is needed. At the same time, an appropriate representation of the system matrix on the coarse grid is required. On a coarse grid, an update to the solution is computed by relaxing on the residual equation for the error, Ae = r

with

r = b − Ax,

where r is the residual and e the error, and given the initial guess e = 0. The update is transferred to the fine grid by applying an interpolation or prolongation operator. Typically, a prolongation operator reintroduces high frequency error components. These can be eliminated by applying additional smoothing iterations. Algorithm 1 summarizes the multigrid methodology [TOS01, pp. 47-48]. The smoothing operation is expressed by the operator Sk , with the constants ν1 and ν2 being the number of pre- and postsmoothing steps respectively. The residual is represented on a coarser grid by applying the restriction operator Rkk−1 . A k prolongator operator Pk−1 transfers the result from a coarse grid on level k − 1 to a finer grid on level k. The hierarchy of grids can be traversed in various ways. The recursion scheme is determined by the cycle index γ; for example, the case γ = 1 is called a V -cycle, the case γ = 2 a W -cycle. The structure of a V -, W -, and F -cycle is illustrated in Fig. 3.4. The computational cost of one V -cycle is smaller than the cost of performing one W -cycle; the multigrid convergence speed is however typically lower for V -cycles than for F - or W -cycles. Geometric multigrid Standard (geometric) multigrid considers a set of spatial discretizations with varying grid size. Isotropic problems defined on regular grids enable a straightforward construction of a coarse grid hierarchy by repeatedly doubling the grid size h in each spatial dimension, see Fig. 3.5. In the case of anisotropic problems, semicoarsening can be applied in order to maintain an optimal multigrid convergence rate [TOS01]. Note that anisotropy can also be handled by adapting the smoother.

MULTIGRID METHODS IN THE SPATIAL DIMENSION

49

Algorithm 1 Multigrid iteration for Ax = b. (γ = 1: V -cycle, γ = 2: W -cycle) (1)

(0)

xk = multigrid(xk , Ak , bk , k) (0)

(0)

• presmoothing: x ¯k = Skν1 (xk , Ak , bk ) (0) • restrict residual: ¯bk−1 = Rkk−1 (bk − Ak x¯k ) (0)

• coarse grid correction: solve Ak−1 vˆk−1 = ¯bk−1 (0)

¯ – if k = 0, vˆk−1 = A−1 k−1 bk−1

(0) – if k > 0, vˆk−1 = multigrid(0, Ak−1 , ¯bk−1 , k − 1) (0) (0) vˆ = multigridγ−1 (ˆ v , Ak−1 , ¯bk−1 , k − 1) k−1

k−1

(0)

(0)

(0)

k • prolongate correction and update solution: x ˆk = x¯k + Pk−1 vˆk−1 (1)

(0)

• postsmoothing: xk = Skν2 (ˆ xk , Ak , bk ) 1 0 0 100 0 1100 00 11 00 1 0 1 11 11 0 1 000 1 11 11 00 0 1 0 00 00 0 1 1 011 1 0011 11 00 11 1 0 0 100 0 1100 00 11 00 1 0 1 11 11 0 1 0 1 00 11 00 0 1 1 011 1 0011 11 00 11 0 1 0 00 00 11 1 0 0 100 0 1100 00 11 00 1 0 1 11 11

100 0 1100 00 11 0 00 1 0 0 100 0 1100 00 11 00 1 0 11 11 1 1 11 11 0 1 11 00 11 00 0 0 1 1 11 00 11 00 0 1 00 00 0 1 00 00 011 1 0011 11 00 1 11 0 0 1 011 1 0011 11 00 11 100 0 1100 00 11 0 00 1 0 0 100 0 1100 00 11 00 1 0 11 11 1 1 11 11 0 1 00 11 00 11 0 1 0 1 00 11 00 011 1 0011 11 00 1 11 0 1 1 011 1 0011 11 00 11 0 1 00 00 0 0 00 00 11 100 0 1100 00 11 0 00 1 0 0 100 0 1100 00 11 00 1 0 11 11 1 1 11 11

100 0 1100 00 11 0 00 1 0 0 100 0 11 00 1 0 11 11 1 1 11 0 1 11 00 11 00 0 0 1 1 11 00 0 1 00 00 0 1 00 011 1 0011 11 00 1 11 0 0 1 011 1 00 11 100 0 1100 00 11 0 00 1 0 0 100 0 11 00 1 0 11 11 1 1 11 0 1 00 11 00 11 0 1 0 1 00 011 1 0011 11 00 1 11 0 1 1 011 1 00 11 0 1 00 00 0 0 00 11 100 0 1100 00 11 0 00 1 0 0 100 0 11 00 1 0 11 11 1 1 11

1 0 1 0 1 0

1 0 1 0 1 0

11 00 11 00 11 00

1 0 1 0 1 0

11 00 11 00 11 00

1 0 1 0 1 0

11 00 11 00 11 00

1 0 1 0 1 0

1 00 0 11 00 1 0 11 1 00 0 11

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

1 0 1 0

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1

1 0 00 00 011 1 0011 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11

1 0 00 00 011 1 0011 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1

1 0 00 00 011 1 0011 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11

1 0 00 00 011 1 0011 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11 0 1 00 00 011 1 0011 11 00 11 0 1 00 11 00 11

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1

1 0 00 011 1 00 11 0 1 00 011 1 00 11 0 1 00 11 0 1 00 011 1 00 11 0 1 00 11

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

1 0 11 00 1 00 0 11

1 0 0 1 0 1

1 0 0 1 0 1

11 00 00 11 00 11

1 0 0 1 0 1

11 00 00 11 00 11

1 0 0 1 0 1

11 00 00 11 00 11

1 0 0 1 0 1

1 0 0 1 0 1

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

100 0 1100 00 11 00 0 1 11 11 0 1 00 00 011 1 0011 11 00 11 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 0 1 00 11 00 011 1 0011 11 00 11 0 1 00 00 11

100 0 1100 00 11 00 0 1 11 11 0 1 00 00 011 1 0011 11 00 11 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 0 1 00 11 00 011 1 0011 11 00 11 0 1 00 00 11

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

100 0 1100 00 11 00 0 1 11 11 0 1 00 00 011 1 0011 11 00 11 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 0 1 00 11 00 011 1 0011 11 00 11 0 1 00 00 11

100 0 1100 00 11 00 0 1 11 11 0 1 00 00 011 1 0011 11 00 11 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 0 1 00 11 00 011 1 0011 11 00 11 0 1 00 00 11

1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

100 0 11 00 0 1 11 0 1 00 011 1 00 11 011 1 00 11 0 1 000 1 00 11 0 1 00 011 1 00 11 0 1 00 11

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

1 0 11 00 1 00 0 11

0 1 1 0 0 1

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

0 1 1 0 0 1

1 0 0 1 1 0 1 0 1 0 0 1 0 1 0 1 1 0 1 0

100 0 1100 00 11 00 0 1 11 11 100 0 1100 00 11 00 1 0 11 11 1 0 11 00 11 00 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 100 0 1100 00 11 00 1 0 11 11

100 0 1100 00 11 00 0 1 11 11 100 0 1100 00 11 00 1 0 11 11 1 0 11 00 11 00 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 100 0 1100 00 11 00 1 0 11 11

1 0 0 1 1 0 1 0 1 0 0 1 0 1 0 1 1 0 1 0

100 0 1100 00 11 00 0 1 11 11 100 0 1100 00 11 00 1 0 11 11 1 0 11 00 11 00 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 100 0 1100 00 11 00 1 0 11 11

100 0 1100 00 11 00 0 1 11 11 100 0 1100 00 11 00 1 0 11 11 1 0 11 00 11 00 011 1 0011 11 00 11 0 1 000 1 0000 11 00 11 100 0 1100 00 11 00 1 0 11 11

1 0 0 1 1 0 1 0 1 0 0 1 0 1 0 1 1 0 1 0

100 0 11 00 0 1 11 100 0 11 00 1 0 11 1 0 11 00 011 1 00 11 0 1 000 1 00 11 100 0 11 00 1 0 11

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

1 0 11 00 1 00 0 11

0 1 1 0 0 1

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

00 11 11 00 00 11

0 1 1 0 0 1

0 1 1 0 0 1

1 0 100 0 1100 00 11 0 00 100 0 1100 00 11 0 00 1 0 0 100 0 1100 00 11 0 00 100 0 1100 00 11 0 00 1 0 0 100 0 11 00 1 0 0 1 11 11 1 11 11 1 1 11 11 1 11 11 1 1 11

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

11 00 11 00

1 0 1 0

1 0 11 00 1 00 0 11

1 0 1 0 1 0

1 0 1 0 1 0

1 0 1 0 1 0

1 0 1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0

1 0 1 0 1 0

1 0 1 0 1 0

11 00 11 00 11 00

1 0 1 0

1 0 1 0

11 00 11 00

1 0 1 0

1 0 1 0

11 00 11 00

Figure 3.5: A fine grid and three coarser grids obtained by doubling the grid size in each dimension repeatedly.

To transfer a solution from a fine to a coarse grid, injection and full weighting are typical restriction operators. The former copies the coarse grid values from the corresponding fine grid values, while the latter applies a weighted sum of the fine grid values to construct the coarse grid values. A bilinear interpolation operator can take care of the reverse transfer from coarse to fine meshes. The coarse grid operator, Ak−1 , can either be constructed directly from discretizing the PDE problem on the coarse grid or as a so-called Galerkin coarse grid operator. Algebraic multigrid For problems defined on unstructured grids it can be very difficult to construct a hierarchy of spatial discretizations. In that case, algebraic multigrid (AMG)

50


can be applied [St¨ u01]. Algebraic multigrid tries to mimic geometric multigrid by constructing a hierarchy of problems algebraically. In contrast to geometric multigrid where first coarse grids are fixed, AMG coarsening is driven by the relaxation method. A sequence of levels is constructed so that an algebraically smooth error can be eliminated at the coarser levels. Restriction and prolongation operators are constructed during a setup phase. The coarse level operators are assembled by using the Galerkin principle [TOS01, p. 423], i.e., k Ak−1 = Rkk−1 Ak Pk−1 .

Since unstructured grids and anisotropic problems can be handled by AMG in a transparent way during setup, AMG enables one to construct very robust iterative solvers, see, e.g., [WE06]. In the view of parallel computing, AMG is shown to possess good scalability properties [CFH+ 00]. Typically, AMG is used as preconditioner for a Krylov method. This makes the scheme more robust and often improves the convergence rates significantly.

3.3.2

Multigrid for stochastic Galerkin discretizations

Multigrid algorithms lead to very efficient and robust solvers for many deterministic problems [TOS01] and can be extended to the discretized stochastic system (2.33) [EF07, SRNV07, RBV08, LKD+ 03]. The iterative methods discussed in Section 3.2 can be applied as smoothers in a standard multigrid framework, as given by Algorithm 1. Both a basic (3.2) and Ki -splitting (3.3) smoother lead to a robust multigrid scheme, as confirmed by the convergence analysis that will be discussed in Sections 4.2 and 4.5. The multigrid coarsening is performed only on the spatial discretization so that all (stochastic) unknowns per spatial node are transferred simultaneously to a coarser level. Thus, a restriction operator Rkk−1 that transfers the residual at level k to level k − 1, is defined as a Kronecker product of an identity matrix IQ ∈ RQ×Q and a restriction operator Rkk−1 . The latter originates from a multigrid hierarchy for the deterministic matrix K1 at level k. The size of IQ equals the number of degrees of freedom in the stochastic dimension. k Analogously, the prolongation operator Pk−1 is constructed as the Kronecker prodk uct of IQ and a prolongation operator Pk−1 , which corresponds to a prolongation operator for the deterministic matrix K1 defined at level k − 1. This results in the following formulation of the prolongation and restriction operators, k Pk−1 = IQ ⊗ Pkk−1

and

Rkk−1 = IQ ⊗ Rkk−1 .

(3.8)

Geometric multigrid. Here, Pkk−1 and Rkk−1 correspond to standard spatial intergrid transfer operators, e.g., bilinear interpolation and full weighting. The coarse

MULTILEVEL METHODS IN THE STOCHASTIC DIMENSION

51

grid operator Ak−1 is obtained by discretizing the problem directly at the coarser grids and leads to a matrix of the form, Ak−1 =

S X i=1

Ci ⊗ Ki,k−1 ,

where Ki,k−1 is given by (2.35), evaluated with respect to spatial basis functions defined on mesh k − 1. To quantify the computational cost, we assume that the number of grid points on level k − 1 corresponds to 1/2d times the number of points on level k, with k = 0 the coarsest level and d the number of spatial dimensions. The total cost of a V -cycle with one pre- and postsmoothing step, denoted by V (1, 1), is then well approximated by 1−22−d cs [BHM00], with cs the cost of one smoothing iteration. Algebraic multigrid. A multigrid hierarchy can be constructed from the dominant term in the Kronecker product system (2.33), i.e., from the stiffness matrix of the averaged deterministic problem. This is the PDE that results from the stochastic PDE by replacing all random parameters by their mean value. Such a hierarchy can be derived by using a classical AMG strategy, e.g., Ruge-St¨ uben AMG [RS87], applied to the stiffness matrix K1 . Suppose Pkk−1 in (3.8) is a prolongation operator constructed for matrix K1 at level k − 1. The restriction operator and coarse grid operator are deduced from (3.8) by applying the Galerkin principle: Rkk−1

= IQ ⊗

T Pkk−1

and

Ak−1 =

S X i=1

Ci ⊗ Ki,k−1

(3.9)

where Ki,k−1 is recursively defined as Ki,k−1 = Pkk−1

3.4

T

Ki,k Pkk−1 .

Multilevel methods in the stochastic dimension

When a high-order polynomial chaos is used, the number of degrees of freedom in the random dimension, Q, can become very large. A coarsening in random space may then be appropriate. The hierarchical structure of the matrices Ci (2.50) provides a straightforward coarsening of the random dimension. As a result, a basic hierarchical solver was proposed in [Gha99a, PG00] where a high-order solution is estimated from a low-order approximation. This section explores the possibility of building a multilevel solution approach with a coarsening in the stochastic dimension, analogous to the previous multigrid algorithm.

52


Given a stochastic discretization (2.33) based on a polynomial expansion (2.32) of order P , a set of P + 1 levels can be constructed. At the coarsest level, corresponding to P = 0, the system matrix reduces to K1 . To transfer a vector at level p to the next coarse level p − 1, injection can be used. This results in the following restriction and prolongation operators: IQl p p−1 Rp = IQl 0 ⊗ IN and Pp−1 = ⊗ IN , (3.10) 0

with Ql = (L + p − 1)!/(L!(p − 1)!). Other choices for interpolation operators are possible. Keese [Kee04, p. 79] proposes a prolongation operator based on an approximation of the Schur complement. This choice yields however only a moderate gain in convergence rate, which does not outweigh the associated increase in computational cost.

The multilevel method suggested here proceeds similarly to Algorithm 1, with the transfer operators given by (3.10) and with a Galerkin construction of the coarse grid operator. The smoothing steps are replaced by Ci -splitting iterations (3.5). When ν1 = ν2 = 1 and γ = 1, i.e., in the case of a V (1, 1)-cycle, each iteration e − 1 systems of size (N × N ), with Q e = (L+1+P )! . requires solving 2Q (L+1)!P ! A combined coarsening in the spatial and stochastic dimension can be achieved by solving the (N × N ) PDE systems in the Ci -splitting block smoother and the coarse level system by multigrid. Also, other smoothing methods can be applied instead of the Ci -splitting smoother. Various approaches will be compared in the convergence analysis in Section 4.7.3, see Table 4.15. One possibility is to replace the Ci -splitting relaxation step by one multigrid cycle of the algorithm presented in Section 3.3.2. This will be demonstrated and further illustrated by the numerical experiments in Section 3.7.

3.5

Preconditioned Krylov subspace methods

In practice, the methods presented in the previous sections will typically be used as preconditioners for a Krylov method. The stochastic Galerkin finite element discretization of the model problem (2.25)–(2.27) with a bounded and uniformly coercive diffusion coefficient yields a symmetric, positive definite system (2.33), see theorem 2.5.14, so that CG can be applied. The basic splitting method (3.2) and the Ki -splitting iteration (3.3) perform poorly as preconditioners. Their spectrum does not approximate the spectrum of (2.33) well, as can be verified by the local Fourier analysis in Section 4.4. The Ci -splitting methods are appropriate as preconditioners, depending on the chaos type and order used. The exact Ci -splitting preconditioners can be approximated by solving the

PRECONDITIONED KRYLOV SUBSPACE METHODS

53

Q (N × N )-block systems inexactly. In the numerical experiments in Section 3.7, a standard W (1, 1)-multigrid cycle is used to approximately solve these systems. Mean-based preconditioner. The mean-based preconditioner [GK96, PG00] is constructed as IQ ⊗ K1 and occurs as a special case of Jacobi Ci -splittings (3.6). The K1 -block systems are solved by an incomplete block factorization or a multigrid cycle. A non-intrusive implementation and a straightforward parallelization are the main assets of this preconditioner. In the case of small input variances and a low-order chaos, very good convergence results can be obtained resulting in a low overall computational cost. A severe performance degradation is however possible in the case of stochastic problems with large input variances and a high-order gPC discretization; this follows from the analysis of the preconditioner’s spectrum in [PE09] and is confirmed by the convergence analysis in Chapter 4. Kronecker product preconditioner. In order to improve the convergence properties of the mean-based preconditioner, a preconditioner of the form G ⊗ K1 is proposed in [Ull10]. The matrix G is the solution of the minimization problem

S

X

Ci ⊗ Ki − G ⊗ K1 . min

G∈RQ×Q i=1

F

The solution to this problem is given by the matrix C1+ in (3.7). In [Ull10], the Kronecker product preconditioner is proven to be symmetric, positive definite and its spectral bounds are derived. This preconditioner can lead to substantial computational cost savings in comparison to the mean-based preconditioner, when solving a stochastic diffusion problem (2.25) with a lognormally distributed diffusion coefficient. Applying matrix formulation (2.36) shows that one preconditioning step corresponds to solving XGT = Y

with

K1 Y = B.

That is, Q systems with matrix K1 and N systems with matrix G are to be solved. Gauss-Seidel Ci -splitting preconditioner. The GSLEX Ci -splitting method (3.5) can be extended to a symmetric CG preconditioner. The preconditioner then consists of one symmetric Gauss-Seidel Ci -splitting iteration, i.e., one forward followed by one backward iteration, and is given by ! !−1 S ! S S X X X D L D D L T (Ci + Ci ) ⊗ Ki , (3.11) Ci ⊗ Ki (Ci + (Ci ) ) ⊗ Ki i=1

i=1

CiL +CiD +(CiL )T ,

i=1

as follows from the symmetry of the Ci -matrices. The with Ci = Q (N × N )-block solves in both the forward and backward iteration can each be

54


approximated by one W (1, 1)-AMG cycle iteration. Note that when corollary 2.5.6 holds and with C1 ≡ IQ , one application of the GS-Ci preconditioner (3.11) to a system with current iterate x and right-hand side b can be written as for q = 1, . . . , Q, P P K1 x[q] = b[q] − ( cq Si=2 Ci (q, c) ⊗ Ki )x[c] x[q] = x[q] + xˆ[q] end

x[q] ∈ RN

x[q] ∈ RN

In that case, only block systems with K1 are to be solved, similar to the meanbased preconditioner case. However, each application of the GS Ci -splitting preconditioner requires the solution of 2Q systems with K1 , in contrast to Q systems in the mean-based preconditioning case. Krylov recycling methods. All iterative methods presented previously attempt to solve the coupled algebraic system (2.33) efficiently. However, as pointed out in Section 2.5.3, a double orthogonal polynomial chaos discretization (2.18)–(2.19) e = QL (1 + results in diagonal Ci -matrices so that system (2.33) corresponds to Q i=1 pi ) smaller systems, each of dimension N × N . These systems can be solved individually, e.g., each preconditioned by K1 . Since all the individual systems have a similar system matrix, Krylov recycling techniques such as GCROT and GCRO-DR [EEU07, JC09] can speed up the computations. By analyzing the KLexpansion (2.4) of the diffusion coefficient, a grouping algorithm and a strategy for when to recycle or recompute components of the Krylov method is proposed in [JCL07].

3.6

Implementation aspects

Matrix storage. As pointed out in remark 3.2.1, the matrices Ci and Ki are stored separately in sparse matrix format. Typically, all Ki -matrices have the same sparsity structure; hence, the description of this structure has to be stored just once. The symmetry of Ci can be exploited for optimizing the storage of Ci , as detailed in [PG00], where also an efficient implementation of matrix-vector multiplications is proposed. Storing the Kronecker product blocks separately leads, even for problems with a dense block structure, to a huge saving of memory cost, as illustrated in the following example.

NUMERICAL EXPERIMENTS

55

Example 3.6.1. In Chapter 7, we will consider a problem with 7 random variables, a third order polynomial chaos, 120 random unknowns and 16 384 spatial unknowns. System (2.33) has then dimension 1 966 080 × 1 966 080. When the diffusion coefficient is represented by a polynomial chaos expansion of order 6, e.g., in the case of a lognormal diffusion coefficient (see remark 2.5.15), )! S = (L+2P L!(2P )! = 1716 matrices Ci and Ki need to be stored. Assuming that all matrices are dense, this amounts to about 11.9% of the storage that would be required for the full matrix. Matrix-vector product. A matrix-vector multiplication can easily be evaluated from matrix equation (2.36). A row-by-row storage format of the multivectors X and B results in a cache efficient implementation, as explained in [BVlV09]. Note that by storing the matrices Ci and Ki separately, the cost of a matrix-vector multiplication is slightly higher compared to storing the entire system matrix [PU09]. In the former case, a matrix-vector multiplication requires about O(SN Q) flops, assuming that the number of matrix elements of a Ci - and a Ki -matrix is proportional to Q and N , respectively. The number of matrix elements of the entire system ma)! trix is at most proportional to O(N Q2 ). This situation occurs when S ≥ (L+2P L!(2P )! Ci -matrices are present in (2.36), as follows from theorem 2.5.13. The cost of the matrix-vector multiplication, being proportional to O(N Q2 ), is smaller than Q2P )! L+k O(SN Q) flops, since S ≥ (L+2P k=P +1 k ≪ Q [PU09]. L!(2P )! = Qp Q with Qp = Parallelism. The stochastic Galerkin finite element discretization leads to highdimensional algebraic systems. The solution of such systems may require a parallel implementation, both for memory and computational cost reasons. Most proposed solvers can easily be extended to a parallel version. For example, by partitioning the spatial mesh over the set of available processors, a parallel version of the presented algebraic multigrid method can be implemented, see [HY02]. A parallel implementation of matrix-vector multiplications of (2.33) and a parallel solver are presented in [KM05, GK96].

3.7

Numerical experiments

In this section, the advantages and limitations of iterative solvers for high-dimensional coupled systems of the form (2.33), are compared numerically. Also the convergence properties, that is, the effect of stochastic and deterministic discretization parameters on the convergence rate, are determined. These experiments validate the theoretical convergence analysis that will be presented in Chapter 4.

56


GS-Ci CG-GS-Ci

Gauss-Seidel Ci -splitting iterations (3.5) CG preconditioned by one symmetric GS-Ci iteration

MG-Point MG-Block CG-MGP CG-MGB

W (1, 1)-multigrid (Section 3.3.2) with GSLEX smoother (3.2) W (1, 1)-multigrid (Section 3.3.2) with collective GS smoother (3.3) CG preconditioned by one MG-Point cycle CG preconditioned by one MG-Block cycle

CG-GSA-Ci CG-Mean CG-Kron CG-Circ

CG CG CG CG

p-GS-Ci CG-p-GSA

V (1, 1)-cycles of the p-multilevel method (Section 3.4) with GS-Ci CG preconditioned by one approximate p-GS-Ci cycle

with with with with

approximate approximate approximate approximate

symmetric GS-Ci preconditioner mean-based preconditioner (3.6) Kronecker product preconditioner (3.7) circulant Ci -splitting preconditioner

Table 3.2: Iterative methods and abbreviations.

In all numerical experiments, the iterations proceed until the Euclidean norm of the relative residual, krk/kbk, is smaller than 10−10 . In this section, 2D geometries are considered, which are discretized by an unstructured triangular mesh, generated by Gmsh [GR09]. A piecewise linear finite element discretization is applied. In Chapter 6, 3D spatial models are used. The tested multigrid methods are of the AMG type. The AMG prolongators are built with classical Ruge-St¨ uben AMG [McC87]. Typically, AMG is accelerated by a Krylov method; CG is applied. When Ci -splitting iterations (3.5) are applied inexactly as preconditioner, the (N × N )-block systems are approximately solved by one W (1, 1)-AMG cycle. In the case of Ki -splitting iterations, the (Q × Q)block systems are solved with CG or a sparse LU factorization, as discussed in Section 3.2.2. An overview of the methods used in the numerical experiments is presented in Table 3.2. In the approximate p-multilevel method case, the GS Ci -splitting (N × N )-block systems are approximated by two W (1, 1)-AMG cycles. This section first illustrates the convergence properties of different iterative solvers, applied to a basic stochastic diffusion problem. Next, their performance is compared in the context of two nontrivial problems: a discontinuous diffusion coefficient defined on an L-shaped domain and a random domain problem as in example 2.4.9. The timings reported originate from tests on a quad-core Xeon 5420 CPU with 2.5 GHz and 8GByte of RAM; an Intel compiled C++ code is used.

3.7.1

Steady-state stochastic diffusion problem

Model problem (2.25) is solved on a square domain, D = [0, 1]2 , with three homogeneous zero Dirichlet boundary conditions and one zero Neumann boundary.


name ag (x, ξ) au (x, ξ) alog (x, ξ) = exp(aG )

57

random discretization KL-expansion KL-expansion Hermite PC expansion

distribution ξ standard normal √ √ uniform on [− 3, 3] standard normal

Table 3.3: Configurations of the random field a(x, ω) in (2.25).

0

(a) mesh

0.0218

(b) mean

0.0435

0

5.35e-05

0.000107

(c) variance

Figure 3.6: (a) Finite element mesh of unit square domain D = [0, 1]2 . (b)–(c) Statistics of the stochastic solution to (2.25)–(2.27), with lognormal diffusion coefficient alog (x, ξ). A SGFEM discretization with 10 573 spatial nodes, 5 random variables and a secondorder Hermite chaos was applied.

The finite element mesh is illustrated in Fig. 3.6(a). A deterministic source term f (x, ω) = 1 is considered. The random diffusion coefficient a(x, ω) is either discretized by a KL-expansion or by a gPC expansion. An overview of the tested configurations is given in Table 3.3. In case of a KL-expansion, an exponential covariance function is assumed, given by (2.7) with variance σ 2 = 0.1 and correlation length lc = 1. The mean value of the random fields ag and au equals the constant function 1. When the stochastic discretization is based on uniformly distributed random variables, Legendre polynomials are used in (2.32); in the case of standard normally distributed random variables, we use Hermite polynomials, see also Table 2.2. Fig. 3.6 (b) and (c) show the mean and variance of the solution when the diffusion coefficient is given by alog (x, ξ). A discretization with 10 573 spatial nodes, 5 random variables and a 2nd order Hermite chaos was applied. Choice of cycle type Most presented iterative solvers make use of a multigrid cycle: either to solve Ki block systems in a Ci -splitting iteration or to solve the global system matrix (2.33). The choice of the cycle type, e.g., a V -, F - or W -cycle, and the number of preand postsmoothing steps influence the total computational cost. W -cycles require

58


Q = 21, P = 2, L = 5 Spatial nodes N 10 275 50 720 114 313 257 986 357 393 MG-Block V 93 (48.2) 157 ( 409 ) 203 (1200) 219 (3777) 250 (5972) F 38 (27.0) 51 ( 177 ) 56 ( 447 ) 60 (1391) 64 (2064) W 31 (25.0) 35 ( 135 ) 33 ( 287 ) 36 ( 962 ) 35 (1236) CG-MGB

V F W

25 (13.1) 33 (86.9) 16 (11.5) 19 (66.4) 14 (11.4) 16 (62.1)

38 20 15

( 227 ) ( 161 ) ( 131 )

40 21 16

( 693 ) ( 489 ) ( 429 )

42 22 16

(1007) ( 712 ) ( 567 )

CG-Mean

V F W

39 ( 3.1 ) 49 (28.2) 28 ( 2.6 ) 31 (19.6) 27 ( 2.6 ) 29 (18.9)

56 32 28

( 80.3 ) ( 50.7 ) ( 45.7 )

60 34 29

( 190 ) ( 124 ) ( 109 )

64 36 30

( 281 ) ( 180 ) ( 159 )

CG-p-GSA

F W

13 ( 6.4 ) 23 (71.3) >300 ( n/a ) > 300 ( n/a ) > 300 ( n/a ) 11 ( 5.7 ) 13 (41.9) 13 ( 108 ) 14 ( 269 ) 14 ( 366 )

Table 3.4: Number of iterations (computational time in sec.) required to solve a Legendre chaos discretization of (2.25) with diffusion coefficient au on D = [0, 1]2 .

more work per cycle than V -cycles, but result in a faster convergence rate and also a more robust convergence behavior. F -cycles represent an intermediate case between V - and W -cycles and aim at combining the low cost of V -cycles with the robust convergence of W -cycles. Table 3.4 shows the number of iterations and computational cost when applying V -, W - or F -cycles. In case of Ci -splitting based methods (CG-Mean and CG-p-GSA), the cycle type used to solve the (N ×N )-block systems is shown. No results for the p-multilevel algorithm of Section 3.4 with V -cycle iterations are given since more than 300 iterations are required for convergence. Although typically V -cycles are applied when using multigrid as preconditioner for a Krylov method, W -cycles turn out to result in an overall lower computational cost for the problems considered. The higher cost-per-cycle of a W -cycle is compensated by a decrease of the total number of Krylov iterations, and consequently, by a decrease of the number of matrix-vector multiplications with the Kronecker product system matrix (2.33). The use of W -cycles reflects a mesh-independent convergence rate. In the next experiments, W -cycles will be employed. Influence of spatial discretization size Table 3.5 illustrates the influence of the mesh size on the convergence rate in the case of a Gaussian and a lognormal diffusion coefficient. The convergence of the multigrid algorithms discussed in Section 3.3.2 and of the Ci -splitting based solvers is asymptotically independent of the mesh size. In the former case, this independence follows from the simultaneous coarsening of all random unknowns


59

Q = 21, P = 2, L = 5 Spatial nodes N 10 275 ag MG-Point 40 MG-Block 31 CG-MGP 17 CG-MGB 14 CG-Mean 29 CG-Kron 23 GS-Ci 16 CG-GSA-Ci 17 p-GS-Ci 7 CG-p-GSA 11 alog

MG-Point MG-Block CG-MGP CG-MGB CG-Mean CG-Kron GS-Ci CG-GSA-Ci p-GS-Ci CG-p-GSA

36 31 18 14 27 21 12 14 5 11

20 751 42 33 19 15 29 24 16 17 7 12

50 720 44 35 20 16 31 25 16 18 7 13

114 313 44 33 20 15 31 25 16 18 7 12

257 986 45 36 20 16 31 26 16 19 7 13

38 33 18 15 28 23 12 14 5 12

40 35 19 16 29 24 12 15 5 13

39 33 19 15 29 23 12 15 5 12

41 36 20 16 30 25 12 16 5 13

Table 3.5: Number of iterations required to solve (2.25) discretized by SGFEM and with a lognormal or a Gaussian diffusion coefficient (alog and ag ) on D = [0, 1]2 .

per spatial node, see (3.8). The Ci -splitting based methods converge independently from the mesh size since discretized deterministic problems are solved at once, without a splitting of the spatial unknowns. This property is maintained when approximating the (N × N )-block systems by a W -multigrid cycle. The computational cost corresponding to the results in Table 3.5 is shown in Fig. 3.7. The solution time is shown as a function of the number of spatial nodes. The mean-based preconditioner requires in all cases the least computing time. By increasing the number of spatial nodes, the number of local solves in the Ki splitting block smoother (3.4) increases proportionally. In addition, the multigrid algorithm may introduce extra coarse levels. Overall, the total computational time scales linearly in the Gaussian case, i.e., for Ci -matrices defined by (2.44). For Ci matrices defined by (2.45), on the other hand, the solution time of multigrid with block smoother increases faster than linear since every smoothing step involves solving many small dense systems. The density of the stochastic discretization and the increased number of stiffness matrices yield that all solvers require substantially more computational time than when definition (2.44) holds.


800

MG-Block

600

CG-MGB

solution time (sec.)

CG-MGP CG-Mean 400

CG-GSA-Ci CG-p-GSA

200

0 0

1(21) 2(42) 3(63) spatial unknowns (total) x 105

(a) ag (x, ω); S = L + 1; Ci = hξi−1 ΨΨT i

10000


60

CG-MGB MG-Block CG-MGP GS-Ci

8000 6000

CG-Mean CG-Kron CG-p-GSA

4000 2000 0 0

1(21) 2(42) 3(63) spatial unknowns (total) x 105

(b) alog (x, ω); S =

(L+2P )! ; L!(2P )!

Ci = hΨi ΨΨT i

Figure 3.7: Solution time as a function of the number of spatial unknowns when solving (2.25) with L = 5, a second-order Hermite chaos and Q = 21. Between brackets, the total number of unknowns is given. The iteration counts are shown in Table 3.5.

Influence of the polynomial order Table 3.6 illustrates the effect of the total polynomial degree P on the number of iterations required to solve (2.33). Except for the multigrid algorithm of Section 3.3.2 with collective GS smoother, the convergence rate of the other solvers is sensitive to the polynomial order. For this example, also the number of p-multilevel iterations is independent of the polynomial order. Increasing the polynomial order leads to a decrease of the convergence rate in the case of Ci -splitting based iterations (3.5). This property is related to the widening of the spectrum of Ci matrices with the polynomial order, see theorem 2.5.11, as will be theoretically demonstrated in Sections 4.2.3 and 4.4.3. The Kronecker product preconditioner converges, as expected, faster than the mean-based preconditioner, especially for lognormal input fields. Computational cost. The computational time is presented as a function of the number of random unknowns in Fig. 3.8. The results were obtained by increasing the polynomial order as in Table 3.6 while keeping the number of random variables constant. Consider first the computational cost of multigrid (CG-MGB) in the case that the diffusion coefficient is discretized by a KL-expansion (case au ). Then, only the dimension of the matrices Ci increases when increasing the polynomial order; the number of stiffness matrices S remains the same. This mainly affects the cost of the block solves in a Ki -splitting smoother. The cost of one block solve with CG is determined by the cost of one CG iteration times the required number of CG iterations. The cost of each CG iteration depends on the number of nonzero


61

CG-MGP

500 400 300

CG-GSA-Ci CG-Mean CG-Kron CG-p-GSA

200 100 0 0

1 4 2 3 total number of unknowns

(a) au , N = 20 751

×104

CG-MGB

CG-MGB solution time (sec.)


600

3

5

6 ×10

2.5 2 1.5

CG-GSA-Ci CG-Mean CG-Kron CG-p-GSA

1 0.5 0 0

1 2 0.5 1.5 2.5 total number of unknowns ×106

(b) alog , N = 10 275

Figure 3.8: Total solution time as a function of the total number of unknowns QN when solving (2.25) with 5 random variables for increasing values of the polynomial order (P = 1, 2, 3, 4, 5). The iteration counts are given in Table 3.6.

elements of the matrix, which, with Ci defined by (2.44), is of the order O(Q). The number of CG iterations for one block solve is proportional to the square root of the condition number of (3.4). In practice, this condition number is close to 1 so that the number of CG iterations is more or less independent of the dimension of the systems. Therefore, the cost of one block solve in a Ki -splitting smoother is of order O(Q). The linear increase of the computational time as a function of the number of random unknowns Q is observed in Fig. 3.8 (a). If the number of stiffness matrices is also increased, see Fig. 3.8(b), then the total AMG computing time tends to grow faster than linear. This is the case in the lognormal field example, where the number of stiffness matrices is given by L!(2P )! S = (L+2P )! , as follows from remark 2.5.15. The block systems, with Ci defined by (2.45), are then dense (see theorem 2.5.13) so that the linear growth of the computational time w.r.t. the number of random unknowns is lost. Therefore, lognormal fields lead to higher computing times than problems containing linear expansions of random variables. For both types of random coefficients, i.e., discretized with KL- and PC-expansions, the solution time of the Ci -splitting based methods is substantially lower than the solution time of the multigrid methods. The timings for the mean-based preconditioner, Kronecker product preconditioner and symmetric Gauss-Seidel preconditioner are very comparable. In the uniform case, the mean-based preconditioner requires generally the lowest solution time, in the lognormal case, the Kronecker product preconditioner.

62


Polynomial order P =1 P =2 P =3 P =4 L = 5, N = 20 751 for au and N = 10 275 for alog CG-MGP au 16 18 20 23 alog 16 18 23 ∗

P =5 26 ∗

CG-MGB

au alog

15 14

15 14

15 14

15 14

15 14

GS-Ci

au alog

9 9

14 12

19 15

24 18

29 20

CG-GSA-Ci

au alog

16 14

17 14

18 15

19 16

20 17

CG-Mean

au alog

22 21

28 27

32 33

36 38

40 43

CG-Kron

au alog

19 18

23 21

26 24

30 27

33 30

CG-p-GSA

au alog

10 9

12 11

13 11

13 11

13 11

Table 3.6: Number of iterations required to solve (2.25) discretized by SGFEM, using solvers from Table 3.2. The corresponding computational time is given in Fig. 3.8. The computations ‘∗’ were interrupted due to an excessive computing time (> 15h).

Anisotropic problem. The independence of the multigrid convergence on the polynomial order does not hold in general. For some problems, low polynomial orders already affect the AMG convergence rate. This is illustrated by the problem −

∂ 2 u(x, ω) ∂ 2 u(x, ω) −a(x, ω) =1 2 ∂x1 ∂x22

x = (x1 , x2 ) ∈ D = [0, 1]2 , ω ∈ Ω, (3.12)

which is discretized similarly to model problem (2.25). Table 3.7 shows the AMG convergence behavior for increasing values of the polynomial order. In case of a Hermite chaos, a deteriorating AMG convergence is observed. As expected from the boundedness of Legendre polynomials, the number of iterations remains small in case of a Legendre chaos. As in Table 3.6, the dependency of the Ci -splitting based solvers on the Hermite polynomial degree is clearly noticeable in Table 3.7.

Influence of the number of random variables Increasing the number of random variables typically does not change the convergence rate of the iterative solvers. This property is illustrated in Fig. 3.9. When a random field is expanded by a KL-expansion, the number of random variables retained in the truncated expansion determines the percentage of the variance that is included in the discretized field. Therefore, an increase of the number of ran-


N = 20 751, L = 5 Polynomial order GS-Ci au ag

63

P =1 9 (6.72) 9 (6.72)

P =2 P =3 P =4 13 (35.0) 17 ( 123 ) 22 ( 364 ) 15 (40.3) 24 ( 173 ) 44 ( 719 )

P =5 26 ( 858 ) 189 (4620)

MG-Point

au ag

36 (14.0) 36 (14.0)

39 (62.3) 42 ( 198 ) 45 ( 519 ) 40 (64.0) 47 ( 221 ) 63 ( 733 )

49 (1199) 162 (2788)

MG-Block

au ag

34 (10.2) 34 (10.3)

36 (56.2) 38 ( 191 ) 40 ( 490 ) 37 (38.5) 41 ( 149 ) 51 ( 523 )

42 (1092) 121 (2550)

CG-MGB

au ag

15 (4.54) 15 (4.58)

16 (25.3) 17 (86.8) 16 (16.9) 18 (66.0)

18 ( 223 ) 21 ( 217 )

18 37

( 474 ) ( 807 )

CG-GSA-Ci

au ag

15 (1.58) 15 (1.62)

16 (7.12) 17 (21.9) 16 ( 7.1 ) 18 (22.4)

17 (51.7) 23 (70.1)

18 42

( 110 ) ( 167 )

CG-Kron

au ag

19 (4.74) 19 (4.82)

22 (9.91) 25 (26.6) 23 (10.3) 28 (24.5)

28 (51.9) 30 ( 109 ) 37 (69.5) >300 ( 764 )

Table 3.7: Number of iterations (total solution time in sec.) required to solve the anisotropic problem (3.12) discretized by SGFEM, using solvers from Table 3.2.

dom variables can have a similar influence on the convergence rate as increasing the input variance. This effect vanishes however asymptotically due to the decay of the KL terms, see Mercer’s theorem 2.3.2. The bottom part of Fig. 3.9 shows the solution time as a function of the total number of unknowns. For this model problem, the mean-based preconditioner is overall the fastest method. The solution time of the other methods, for example the multigrid preconditioner with block smoother, shows a similar behavior. Only the timings for multigrid with basic Gauss-Seidel smoother scale badly w.r.t. the total number of unknowns. Influence of the input variance Table 3.8 illustrates the effect of increasing the variance of the stochastic diffusion coefficient on the required number of CG iterations and computational time. The solvers based on Ci -splittings, both in a one-level and in a multilevel setting, perform poorly for problems with a large input variance. The convergence of the Kronecker product preconditioner (3.7) is in the case of a lognormal input field less sensitive to a large variance than the mean-based preconditioner, as reported in [Ull10]. The robustness of the multigrid solver with Ki -splitting smoother is remarkable, but this method may require a large computational cost. When Ci matrices are defined as hξi−1 ΨΨT i (2.44), multigrid with block smoother yields the lowest computational time for problems with a large input variance. For lognormal input fields, with Ci defined by hΨi ΨΨT i (2.45), the Kronecker product preconditioner is the most favorable method for the tested problems.

64


30 number of iterations

number of iterations

30 CG-MGP 20

CG-MGB CG-GSA-Ci CG-Mean

10

p-GS-Ci CG-p-GSA

0 0

10 5 15 20 number of random variables (L)

10

p-GS-Ci CG-p-GSA 10 5 15 20 number of random variables (L)

3

2 CG-MGP CG-MGB

10

CG-GSA-Ci CG-Mean

(b) au (x, ω) – Legendre chaos

CG-GSA-Ci

1

CG-Mean p-GS-Ci

0 10

CG-p-GSA

5 10 10 10 total number of unknowns (QN ) 6

(c) ag (x, ω) – Hermite chaos

7



10

CG-MGB

0 0

(a) ag (x, ω) – Hermite chaos 10

CG-MGP 20

10

2 CG-MGP CG-MGB

10

1

CG-GSA-Ci CG-Mean p-GS-Ci

0 10

CG-p-GSA

7 5 10 10 10 total number of unknowns (QN ) 6

(d) au (x, ω) – Legendre chaos

Figure 3.9: Number of iterations (top) and total solution time (bottom) when solving (2.25) on D = [0, 1]2 , with a Gaussian (ag (x, ω)) or uniform-based (au (x, ω)) stochastic diffusion coefficient. The number of random variables is increased from 1 to 16.

3.7.2

Test case: L-shaped domain problem

The study of the stochastic diffusion problem in Section 3.7.1 revealed the most important convergence properties of the presented iterative solvers. In this section, it is verified whether these properties continue to hold for a somewhat less trivial problem. The stochastic model problem (2.25)–(2.27) is applied on an L-shaped domain D, which is illustrated in Fig. 3.10(a). Dirichlet boundary conditions are imposed on the upper and lower right boundaries, and zero Neumann conditions elsewhere. The diffusion coefficient a(x, ω) is modelled as a piecewise random field, consisting of three parts, a1 (x, ω), a2 (x, ω), and a3 (x, ω), defined, respectively, on the domains D1 , D2 and D3 .


65

N = 20 751, M = 5, Variance au CG-MGP CG-MGB CG-Mean CG-Kron CG-GSA-Ci CG-p-GSA alog

P = 2, Q = 21 σ 2 = 0.01 σ 2 = 0.25 σ 2 = 0.45 σ 2 = 0.75 15 (24.0) 32 ( 51.9 ) > 250 ( n/a ) > 250 ( n/a ) 15 (21.6) 15 ( 23.1 ) 15 (23.2) 16 (26.6) 18 (4.16) 43 ( 10.1 ) 141 (33.7) 209 (50.1) 17 (7.58) 34 ( 15.1 ) 105 (47.1) 213 (95.2) 14 (6.20) 23 ( 10.2 ) 69 (30.3) 130 (57.0) 10 (11.9) 13 ( 15.4 ) 46 (54.5) >250 ( n/a )

CG-MGP CG-MGB CG-Mean CG-Kron CG-GSA-Ci CG-p-GSA

15 15 18 17 13 10

( 535 ) ( 379 ) (53.7) (68.4) (96.3) ( 114 )

40 15 38 29 17 13

( 1436 ) ( 380 ) ( 111 ) ( 109 ) ( 126 ) ( 148 )

>250 15 51 35 20 13

( n/a ) >250 ( n/a ) ( 381 ) 15 ( 316 ) ( 149 ) 68 ( 195 ) ( 127 ) 43 ( 158 ) ( 149 ) 25 ( 189 ) ( 148 ) 17 ( 195 )

Table 3.8: Number of iterations (solution time in sec.) required to solve (2.25) on D = [0, 1]2 with diffusion coefficient au or alog (see Table 3.3), using solvers from Table 3.2.

u = 10 2 Material 3 (D3 ) ∂u ∂n

∂u ∂n

=0

∂u ∂n

=0

=0

Material 2 Material 1 (D2 ) (D1 )

u=0

0 0

∂u ∂n

=0

2

(a) problem domain

0

5

10

(b) mean

0

0.2351

0.4702

(c) variance

Figure 3.10: (a) Geometry of L-shaped domain problem. (b)-(c) Mean and variance of the steady-state solution of (2.25)–(2.27), with a(x, ω) a discontinuous random field discretized by a linear expansion with uniformly distributed random variables. The solution is represented by a second-order Legendre expansion.

KL-expansion with uniformly distributed random variables First, each part of a(x, ω) is discretized by a KL-expansion, aj (x, ω) ≈ aj,1 (x) +

Lj X i=1

aj,i+1 (x)ξj,i (ω) with x ∈ Dj and j ∈ {1, 2, 3}. (3.13)

We assume that all random variables ξj,i , with√i =√1, . . . , Lj and j ∈ {1, 2, 3}, are independent and uniformly distributed on [− 3, 3]. The solution is expanded

66


with Legendre polynomials. The KL-expansions are based on an exponential covariance function (2.7), with variance σ 2 and correlation length lc . From (3.13), it follows that the conditions of corollaries 2.5.6 and 2.5.10 are satisfied. As default configuration, the following parameters are applied: a1,1 (x) = 30, a2,1 (x) = 5, a3,1 (x) = 100, lc,1 = 1, lc,2 = 0.5, lc,3 = 2.5, σ12 = 100, σ22 = 2.25, σ32 = 900, L1 = 3, L2 = 5, and L3 = 2. Fig. 3.10(b) and (c) show the mean and variance of the solution for these settings. Table 3.9 presents the number of iterations required to solve the discretized system by a selection of the methods given in Table 3.2. The left column shows the iteration counts for one particular choice of the discretization parameters, denoted by “default”. In the other columns, one of these parameters is changed in order to illustrate the effect of, respectively, the number of spatial nodes, the polynomial order, the number of random variables, and the coefficient of variation on the convergence. Only the coefficient of variation has a strong impact on the iteration counts. This relatively stable convergence behavior is typical for the use of Legendre polynomials, as already reported in the context of Tables 3.4 and 3.6 and Fig. 3.9. Amongst the Ci -splitting preconditioners, a circulant Ci -splitting and a Kronecker product preconditioner result in similar or higher iteration counts than the mean-based preconditioner, while requiring a larger computational cost. Therefore, these approaches are not very useful for solving this problem. In Fig. 3.11(a) relative solution times are given, where “relative” is with respect to the time required by the fastest solver for the particular problem. This shows that the mean-based preconditioner (CG-Mean) and the approximate GS Ci -splitting preconditioning approach (CG-GS-Ci ) result in the lowest solution times. The multigrid approaches (CG-MGP and CG-MPB) are, especially in case of a large number of random unknowns, much more computationally expensive. For problems with a large input coefficient of variation, the multigrid method with Ki splitting block smoother can become interesting because of its robust convergence.

KL-expansion with gamma distributed random variables Next, the random variables in (3.13) are assumed to be gamma distributed with shape parameter k = 1 and scale parameter θ = 21 . The solution is then expanded with Laguerre polynomials. Table 3.10 presents the number of iterations required to solve the discretized system (2.33). The same parameters are used as in the uniform case. The convergence behavior is similar to the uniform case, except for the influence of the polynomial order. Since Laguerre polynomials are defined on an unbounded interval, the convergence rate of iterative solvers decreases strongly in the case of a large polynomial order. This effect is similar to the dependency of the convergence on the Hermite polynomial order, see Table 3.6 and the theo-


67

default

N = 103 479

method 1.GS-Ci 2.CG-GS-Ci

9 6

3.CG-MGP 4.CG-MGB

σ µ

≈ 0.6

8 5

P =4 Q = 1001 13 7

L = 40 Q = 861 11 6

14 13

15 14

16 13

15 13

54 13

5.CG-Mean 6.CG-Kron 7.CG-Circ 8.CG-GSA-Ci

22 24 22 13

23 24 22 14

27 32 27 14

24 26 24 14

63 82 61 30

9.CG-p-GSA 10.p-GS-Ci

9 4

10 3

9 4

9 4

17 32

89 20

Table 3.9: Number of iterations required to solve (2.25) on an L-shaped domain with a(x, ω) a discontinuous random field discretized by a linear expansion with L uniform random variables. The column of the table labeled “default” corresponds to N = 12 154, P = 2, L = 10, Q = 66, and σµ ≈ 0.3. The other columns use the same discretization, with one of the parameters changed. 15

default relative solution time

relative solution time

15

N = 103 479 P = 4, (1001)

10

L = 40, (861) σ/µ ≈ 0.6

5

1

default N = 103 479 P = 4, (1001)

10

L = 40, (861) σ/µ ≈ 0.6

5

1 1

2

3

4 5 6 method

7

8

9

10

1

2

3

4 5 6 method

7

8

9

10

Figure 3.11: Relative solution times corresponding to the results presented respectively in Table 3.9 for uniformly distributed random variables (left) and Table 3.10 for gamma distributed random variables (right).

retical analysis in Chapter 4. Fig. 3.11 (b) illustrates the relative solution times corresponding to Table 3.10. Similar trends as in Fig. 3.11 (a) are present, however in this case the Gauss-Seidel Ci -splitting preconditioner performs slightly better than the mean-based preconditioner. h- and p-multilevel methods In Section 3.4, a p-multilevel algorithm in the stochastic dimension is presented. Here, we test different combinations of h- and p-multilevel methods, where the

68


default

N = 103 479


5 4

3.CG-MGP 4.CG-MGB

σ µ

≈ 0.6

5 4

P =4 Q = 1001 15 7

L = 40 Q = 861 6 4

13 13

15 14

15 13

13 13

18 13


23 22 21 12

23 22 21 13

46 42 42 15

23 23 22 12

58 55 44 17

9.CG-p-GSA 10.p-GS-Ci

8 2

8 2

8 6

8 3

10 8

10 10

Table 3.10: Number of iterations required to solve (2.25) on an L-shaped domain with a(x, ω) a discontinuous random field discretized by L gamma distributed random variables. The column of the table labeled “default” corresponds to N = 12 154, P = 2, L = 10, Q = 66, and σµ ≈ 0.3. The other columns use the same discretization, with one of these parameters changed.

h-multigrid method corresponds to the method presented in Section 3.3.2. We consider h-multigrid V (1, 1)- and W (1, 1)-cycles with a Gauss-Seidel Ki -splitting smoother (3.3). The multigrid iterations are used stand-alone (MG-Point), as preconditioner for CG (CG-MGB), or in combination with V (1, 1)-cycles of the p-multilevel algorithm. In the latter case, the p-multilevel method is applied as a so-called outer iteration, while one cycle of the h-multigrid method is performed at every p-level, i.e., instead of applying a GS Ci -splitting smoother. This ph-combination is used stand-alone or as preconditioner for CG. The results in Table 3.11 show that the proposed combination of the p-multilevel method and the h-multigrid method with collective GS smoother leads to a high convergence rate, but suffers from a very large time-per-iteration cost. Multigrid W (1, 1)cycles accelerated by CG typically result in the lowest solution times amongst the multigrid approaches. This confirms the results in Table 3.4. Polynomial chaos expansion of a lognormal field Next, we model each part of a(x, ω) as a lognormal field. That is, aj (x, ω) = exp(ag,j (x, ω)), with ag (x, ω) a Gaussian field, discretized by expansion (3.13) with normally distributed random variables and ag1,1 (x) = 3, ag2,1 (x) = 1, ag3,1 (x) = 5, (σ1g )2 = 0.81, (σ2g )2 = 0.09, and (σ3g )2 = 2.25. Each part of a(x, ω) is now discretized by a Hermite expansion (2.8). The polynomial order used for discretizing a(x, ω) equals twice the polynomial order used for the solution. Table 3.12 presents the number of iterations required by different iterative solvers. Only multigrid with block smoother (CG-MGB) shows a robust convergence behavior. This robustness


69

Hermite polynomials P =1 P =3 P =4

Legendre polynomials P =1 P =3 P =4

V(1,1)-cycle stand-alone p-multilevel with CG CG and p-multilevel W(1,1)-cycle

119 46 27 15

(35.1) (35.2) (8.01) (11.5)

119 40 28 14

(1288) (1261) ( 306 ) ( 443 )

119 38 28 13

(5768) 119 (35.1) 119 (1950) (5428) 46 (35.2) 37 (1642) (1370) 27 (8.03) 28 ( 467 ) (1864) 15 (11.6) 13 ( 586 )

MG-Block p-multilevel CG-MGB CG and p-multilevel

27 11 13 7

(11.8) (11.5) (5.73) (7.32)

27 10 13 6

( 433 ) 27 (1941) ( 439 ) 9 (1832) ( 211 ) 13 ( 943 ) ( 264 ) 6 (1224)

27 11 13 7

119 33 28 12

(8263) (6519) (2000) (2426)

(11.8) 27 ( 660 ) 27 (2780) (11.5) 9 ( 568 ) 8 (2298) (5.74) 13 ( 322 ) 13 (1365) (7.35) 6 ( 382 ) 6 (1746)

Table 3.11: Number of iterations required to solve (2.25)–(2.27) on an L-shaped domain by a multigrid based method with Ki -splitting block smoother. The diffusion coefficient a(x, ω) is discretized by a KL-expansion with 10 random variables, either normally or uniformly distributed. The total solution time in seconds is given between brackets. The finite element mesh consists of 12 154 nodes.

is also confirmed by a local Fourier analysis, see the results in Section 4.7.2. The convergence of the other methods suffers from a high polynomial order and a large coefficient of variation. The Kronecker product preconditioner converges slower than the mean-based preconditioner for this test problem. Despite earlier good results for multigrid with basic GS smoother, in this case, this method fails to converge to the required tolerance within acceptable time. As a consequence, no results are shown in Table 3.12 for the method “CG-MGP” since its convergence rate deteriorates after only a few iteration steps. The relative solution times corresponding to the results in Table 3.12 are illustrated in the left panel of Fig. 3.12. Multigrid with block smoother results generally in the lowest solution time due to its robust convergence behavior. CG preconditioned by an approximate GS-Ci -splitting performs also well.

3.7.3

Test case: random domain problem

In Section 2.4.3, the stochastic Galerkin discretization of a random domain problem is detailed. Here, example 2.4.9 is considered and the transformed stochastic PDE (2.40) defined on the deterministic domain D is solved. This results in a Legendre chaos discretization, where the Ci -matrices in (2.33) are defined by (2.45). Table 3.13 and the right panel of Fig. 3.12 illustrate the convergence and efficiency of the solvers of Table 3.2. The use of Legendre polynomials results in a fast con-

70


default

N = 53 156


115 23

P =3 Q = 286 >300 42

L = 20 Q = 231 125 24

σ 1.4/0.5/2.3 439 42

126 24

3.CG-MGB

13

13

13

14

13


131 137 130 37

135 138 133 38

289 > 300 291 66

139 144 141 37

326 368 329 66

8.p-GS-Ci 9.CG-p-GSA

45 27

41 23

148 45

47 30

142 49

Table 3.12: Number of iterations required to solve (2.25)–(2.27) on an L-shaped domain, with a(x, ω) a discontinuous lognormal field. The column of the table labeled “default” corresponds to N = 12 154, P = 2, L = 10, Q = 66, and µσ = 0.3. The other columns show the effect of modifying one of these parameters.

default, (66) default, (45)

N = 53 156, (66) relative solution time

relative solution time

10

P = 3, (286) L = 20, (231) σ = 1.4/0.5/2.3

5

1 1

2

3

6 4 5 method

7

8

(a) L-shaped domain, lognormal

9

N = 106 370 10

P = 5, (1287) L = 16, (153) σ = 0.25, (45)

5

1

2

3

4 5 6 method

7

8

9

10

(b) random domain, uniform

Figure 3.12: Relative solution times corresponding to Table 3.12 (left) and Table 3.13 (right). Between brackets, the number of random unknowns Q is given.

vergence of Ci -splitting based methods, and especially of the p-multilevel solution approach. The multigrid approaches retain their robust convergence, but require more computational time than Ci -splitting based methods.

3.8

Conclusions

A large variety of iterative solution techniques for stochastic Galerkin discretizations can be constructed. Each of them aims at enhancing the performing of the

CONCLUSIONS

71

default

N = 106 370


L = 16 Q = 153 6 4

σ = 0.25

5 4

P =5 Q = 1287 6 4

5 4

3.CG-MGP 4.CG-MGB

14 14

16 16

14 14

15 14

17 14


16 16 19 12

18 18 22 14

18 17 > 100 10

17 16 20 12

21 21 24 15

9.p-GS-Ci 10.CG-p-GSA

2 9

2 10

2 9

3 8

4 16

9 6

Table 3.13: Number of iterations required to solve the random domain problem (2.40). The column of the table labeled “default” corresponds to N = 11 084, P = 2, L = 8, Q = 45, and σ = 0.1. In the other columns, one of these parameters is modified.

stochastic Galerkin finite element method. There exist however large differences in computational cost and convergence properties which necessitated the presented convergence study. The numerical experiments indicate that Ci -splitting based solvers, e.g., a GaussSeidel Ci -splitting preconditioner or a mean-based preconditioner, yield very good convergence results for problems with a low polynomial chaos order and variance. By approximately solving the block systems with one W (1, 1)-AMG cycle, very fast solvers can be created, with a convergence rate independent of the number of random variables and the mesh size. The multilevel algorithm based on GS Ci -splittings performs similarly to one-level GS Ci -splitting iterations, without improving its computing times. Amongst the Ci -splitting approaches, the Kronecker product preconditioner may lead to a higher convergence rate than the mean-based preconditioner in the case of a high-order stochastic discretization and a lognormal diffusion coefficient with a large coefficient of variation; otherwise this approach is more expensive. The GS Ci -splitting preconditioner has a more robust convergence pattern than the mean-based preconditioner and results in comparable computing times. It requires however more implementation effort. We have developed an AMG multigrid solver that outperforms the other methods in robustness and optimal convergence properties. Employing a block Ki -splitting smoother increases its robustness in comparison to a basic smoothing approach. This multigrid method is however in many cases one of the more expensive solvers, both in computing times and implementation cost.

Chapter 4

Convergence analysis of multigrid solvers 4.1

Introduction

This chapter provides theoretical convergence results for the solvers discussed in Chapter 3. The convergence analysis points out the dependence of the convergence rate on the model characteristics, the discretization parameters and the algorithmic components. Local Fourier analysis (LFA) is a commonly applied tool for computing theoretical convergence factors of iterative methods [TOS01, WJ05, CE89]. It requires several additional assumptions on the discretized PDE, such as the infinite grid and constant coefficients assumption, but the obtained convergence information is typically also relevant for the finite grid case. LFA for multigrid entails both a smoothing and a two-grid convergence analysis. Brandt [Bra77] introduced the smoothing analysis, St¨ uben [ST82] presented a two-grid convergence analysis. Multigrid convergence can also be investigated based on the smoothing and approximation property of the two-grid cycle [Hac85, theorem 6.1.7]. In the context of stochastic PDEs, Elman et al. [EF07] prove the convergence of the two-grid correction scheme presented in Section 3.3.2. No quantitative theoretical convergence factors were however obtained. Accurate theoretical multigrid convergence factors are given by LFA in [SRNV07, RV10], see Section 4.5. This chapter first presents a general convergence analysis of the multigrid solution approach for the stochastic Galerkin finite element discretization of the model problem (2.25)–(2.27) in Section 4.2. A spectral analysis is performed based on

73

74

CONVERGENCE ANALYSIS OF MULTIGRID SOLVERS

linear algebra manipulations, similar to the analysis in [BVlV09]. This analysis provides insights into the convergence behavior, also for the algebraic multigrid case. The remainder of the chapter considers a slightly different model problem, which is introduced in Section 4.3 and discretized by a stochastic Galerkin finite difference method, and the geometric multigrid variant. Results of a local Fourier analysis of the iterative solvers from Chapter 3 are given in Sections 4.4, 4.5 and 4.6. The sharpness of the theoretical convergence factors and the convergence properties are illustrated by numerical experiments in Section 4.7. Section 4.8 reviews the main convergence results obtained in this chapter.

4.2

Theoretical convergence analysis

The iterative methods discussed in Chapter 3 can typically be formulated as xnew = Sxold + ˆb. The asymptotic convergence is characterized by the spectral radius of the iteration operator S, denoted by ρ(S). The spectral radius of a matrix is defined as the maximum of all eigenvalues in absolute value and enables one to describe the performance of an iteration operator without depending on any particular matrix norm. The asymptotic convergence factor is given by ρ(S). The corresponding asymptotic convergence rate is defined as − log10 (ρ) and is a measure for the increase in the number of correct decimal places in the solution per iteration. In this section, we analyze the multigrid method presented in Section 3.3.2, with Ki -splitting block smoother (3.3). This analysis does not make any assumptions on the regularity of the spatial grid and is therefore applicable to both geometric and algebraic multigrid methods. The analysis here is entirely based on linear algebra manipulations, following the methodology presented in [VV05, BVlV09] and has been published in [RBV08]. First, the analysis is restricted to the case where the diffusion coefficient a is represented by an expansion with one random variable, e.g., S = 2 and L = 1 in (2.28). In Section 4.2.2, the extension to the general case, S > 2, is discussed.

4.2.1

One random variable

Consider first the Ki -splitting block smoother (3.3) and define e(m) := xexact −x(m) , the error at iteration step m. The error satisfies e(m+1) = SKi e(m) with −1 C1 ⊗ K1− + C2 ⊗ K2− . SKi := − C1 ⊗ K1+ + C2 ⊗ K2+

An appropriate normalization of the random basis functions Ψ1 , . . . , ΨQ yields that C1 ≡ IQ , with IQ an identity matrix. The matrix C2 is a real symmetric matrix

THEORETICAL CONVERGENCE ANALYSIS

75

defined by (2.34) and with eigenvalue decomposition C2 = VC2 ΛC2 VCT2 . Applying the similarity transform VC2 ⊗ IN to SKi shows that the spectrum of SKi , denoted by σ(SKi ), corresponds to (4.1) σ (SKi ) = σ VCT2 ⊗ IN SKi (VC2 ⊗ IN ) −1 IQ ⊗ K1− + ΛC2 ⊗ K2− = σ − IQ ⊗ K1+ + ΛC2 ⊗ K2+ =

Q [

−1 K1− + λq K2− σ − K1+ + λq K2+

Q [

σ SbKi (λq ) ,

q=1

=

q=1

where SbKi is a matrix-valued function defined as SbKi (z) := −(K1+ + zK2+)−1 (K1− + zK2− ).

with λq ∈ σ(C2 )

(4.2)

Thus, the asymptotic convergence factor of a Ki -splitting iteration is given by ρ (SKi ) = max ρ SbKi (λq ) . (4.3) λq ∈σ(C2 )

In order to characterize then the convergence properties of a two-level multigrid cycle, with coarse grid k − 1 and intergrid transfer operators given by (3.9), we define the matrix-valued function Tˆ (z): −1 ν2 T Tb (z) := SbKi (z) IN − Pkk−1 Pkk−1 (K1 + zK2 ) Pkk−1 Pkk−1

T

ν1 (K1 + zK2 ) SbKi (z) ,

where SbKi is defined by (4.2); ν1 and ν2 are the number of pre- and postsmoothing iterations. An analogous derivation as above shows that the asymptotic convergence factor of the two-level cycle can be determined from the spectral radius of the corresponding iteration matrix T as ρ(T ) = max ρ(Tb (λq )). (4.4) λq ∈σ(C2 )

Formulae (4.3) and (4.4) can be intuitively interpreted as follows. The asymptotic convergence factor of multigrid with collective Gauss-Seidel smoother applied to a stationary SGFEM discretization of (2.25)–(2.27), with S = 2, equals the worst case asymptotic convergence factor of multigrid with Gauss-Seidel smoother, applied to a set of deterministic problems of the form: (K1 + λq K2 ) x = b

with λq ∈ σ(C2 ).

(4.5)

76


4.2.2

Extension to multiple random variables

The case S = 2 enables a decoupling of the stochastic and spatial dimensions, by using a similarity transform based on C2 . Hence, the analysis can be reduced to the analysis of smaller problems of the form (4.5). For these problems, sharp convergence factors can be derived with local Fourier analysis [TOS01, WJ05], at least for the geometric multigrid variant on regular meshes. In general, no decoupling between the spatial and random part of the discretization is possible since the matrices Ci cannot be diagonalized simultaneously, see Section 2.5.3. An exception to this occurs when a is discretized by a linear expansion of random variables in (2.28) and double orthogonal polynomials (see definition 2.3.7) are used as basis functions in SGFEM. Indeed, then all matrices Ci are diagonal, see theorem 2.3.8. Denote the double orthogonal random basis e and the corresponding matrices Ci by C ei . A similar analysis as for (4.4) by Ψ shows that the spectral radius of the two-level multigrid iteration matrix Te can be determined as ρ(Te ) =

e1 , . . . , λ eL+1 )), ρ(TbDO (λ max max . . . e eL+1 ) e1 ) e λ1 ∈σ(C λL+1 ∈σ(C

(4.6)

with the matrix-valued function TbDO (z1 , . . . , zL+1 ) being defined as ν2 T TbDO (z1 , . . . , zL+1 ) := SbDO,Ki (z1 , . . . , zL+1 ) IN − Pkk−1 Pkk−1 L+1 X i=1

zi Ki

!

Pkk−1

!−1

Pkk−1

T

L+1 X i=1

!

ν1 zi Ki  SbDO,Ki (z1 , . . . , zL+1 ) ,

and the matrix-valued function SbDO,Ki (z1 , . . . , zL+1 ) :=

L+1 X i=1

!−1

zi Ki+

L+1 X i=1

!

zi Ki−

.

As pointed out in the proof of theorem 2.5.11, a relation between the eigenvalues of ei exists. On the basis the matrices Ci and the diagonal elements of the matrices C of that relation, the AMG convergence properties in case of a double orthogonal random basis can be shown to be similar to the case of a generalized polynomial chaos basis. Moreover, also when the double orthogonal basis is not used, we can argue that the analysis of the case S = 2 is likely to provide valuable insights for the general case S > 2. The first terms of system (2.33), i.e., C1 ⊗ K1 + C2 ⊗ K2 , represent the mean behavior of the stochastic PDE and the main stochastic variation. This follows from the stochastic discretization of the random coefficient as a truncation of a series of terms of decreasing importance, see Section 2.3. The sum involving the matrices C3 , . . . , CS can be seen as a perturbation of the system matrix. A more thorough (geometric) multigrid analysis for the general stationary case is given in Section 4.6.

THEORETICAL CONVERGENCE ANALYSIS

4.2.3

77

Discussion of theoretical convergence properties

The convergence analysis of the previous section shows that the matrices K1 and K2 as well as the eigenvalues of C2 determine the convergence of AMG with block smoother, see (4.4). In this section, the AMG convergence behavior with respect to the stochastic discretization is discussed. The conclusions agree with the numerical results in Section 3.7 and the theoretical analysis of the geometric multigrid variant in [EF07, SRNV07], see Section 4.6. Polynomial chaos type and order The influence of the polynomial chaos type and polynomial order on the convergence factor (4.4) can be derived from the properties of the zeros of the corresponding 1D orthogonal polynomials, as follows from theorem 2.5.11 for the case a is approximated by a KL-expansion. For example, in the case√of Legendre √ polynomials, the eigenvalues of the matrix C2 take values between − 3 and 3. Thus, AMG convergence is asymptotically independent of the Legendre polynomial order. In the case of Hermite or Laguerre polynomials, the eigenvalues can become arbitrarily large and their range expands for increasing polynomial order. This effect is illustrated in Fig. 4.1(a). Whether the large eigenvalue range affects the AMG convergence or not depends on the particular PDE problem. Consider, e.g., model problem (2.25) with a a Gaussian variable with mean 1 and variance σ 2 , i.e., a := 1 + σξ1 . Then, K1 is a discretized Laplacian and K2 = σK1 . According to (4.5), the AMG convergence corresponds to the worst convergence of multigrid applied to a set of problems of the form: (1 + λq σ)K1 x = b

with λq ∈ σ(C2 ) and C2 = hξ1 ΨΨT i.

The multiplicative factor 1 + λσ can be shifted to the right-hand side. Hence, the AMG convergence is independent of the variance and the polynomial order. However, when a is modelled as a random field a(x, ω), and S = 2, system (4.5) e = b, with K e being a discretized diffusion problem with diffusion corresponds to Kx coefficient e a(x) := a1 (x) + λq a2 (x). When e a(x) violates the ellipticity conditions e is based, (cf. assumption 2.4.4 in Section 2.4.2) of the bilinear form upon which K e.g., due to a large negative λq , the AMG convergence degrades severely and eventually divergence is possible. This typically occurs only for a large (Hermite) polynomial order. For other problems, the polynomial order can have an even more serious impact on the AMG convergence. For example, consider the following problem with a zero-mean Gaussian random variable σξ1 with finite variance σ 2 : 2 ∂2 ∂ u = b. (4.7) + (1 + σξ ) − 1 ∂x21 ∂x22 The stiffness matrix K1 now corresponds to a discrete Laplace operator, while the second stiffness matrix K2 contains only contributions from ∂ 2 u/∂x22 . The AMG

78


2 Eigenvalues C2

Eigenvalues C2

6 4 2 0 -2 -4 -6 0

2

4

6

Hermite order P

8

10

1 0 -1 -2 0

5

10

15

20

Number of random dimensions L

Figure 4.1: Effect of the polynomial order and the number of random variables on the eigenvalues of C2 (2.44) in the case of a Hermite chaos: (a) fixed number of random variables (L = 4) and increasing P ; (b) fixed order (P = 2) and increasing value of L.

convergence rate equals the worst multigrid convergence rate for deterministic problems of the form: 2 ∂2 ∂ u=b with λq ∈ σ(C2 ), C2 = hξ1 ΨΨT i. + (1 + λ σ) − q ∂x21 ∂x22 Increasing the polynomial order broadens the range of λq and consequently increases the anisotropy of the problem. This results in a decreased AMG convergence. Eventually, for a sufficiently large Hermite order, the problem will lose ellipticity and the AMG method will diverge. The numerical results presented in Table 3.7 demonstrate this possible AMG performance degradation. Number of random variables Since the convergence analysis is restricted to S = 2, it does not provide direct information about the influence of the number of random variables L. However, the eigenvalue bounds of the matrices Ci (2.44) do not depend on the number of ei equal zeros random variables. This follows from the fact that the eigenvalues of C of 1D orthogonal polynomials, see Section 2.5.2, and thus are independent of the number of random variables. This property suggests an independence of the AMG convergence on the number of random variables. Fig. 4.1(b) shows the eigenvalues of C2 as a function of the number of random variables. The same eigenvalues can be found for C3 , . . . , CL+1 (2.44), while all eigenvalues of C1 equal 1. Only in case e a quantitative analysis of L > 1 of a double orthogonal polynomial chaos basis Ψ, is possible. Then, the independence of the AMG convergence on the number of random variables can be demonstrated theoretically. Remark 4.2.1. The conclusions of this general multigrid convergence analysis correspond to convergence properties of the geometric multigrid variant obtained by a

LOCAL FOURIER CONVERGENCE ANALYSIS

79

local Fourier analysis, see Section 4.5 and [SRNV07]. The smoothing and approximation properties of a two-grid cycle for (2.25) in [EF07] yield that the multigrid convergence is asymptotically independent of the spatial and stochastic discretization parameters, i.e., the mesh size h, the number of random variables L and the polynomial order P . The convergence analysis presented in this section confirms these results, but indicates also for which type of problems the independence of the convergence rate on the polynomial order no longer holds.

4.3

Local Fourier convergence analysis

In order to analyze the convergence properties of the iterative methods presented in Chapter 3, a local Fourier (LFA) or local mode analysis [Bra77, WJ05] is conducted. In contrast to the convergence analysis in Section 4.2, LFA considers only geometric multigrid, but it produces quantitative asymptotic convergence factors. These convergence factors do not hold in the AMG case, but provide insight into the general behavior and properties of the iterative solvers. LFA considers linear discrete operators with constant coefficients defined on an infinite grid. A general discrete operator with non-constant coefficients can be analyzed through local linearization and replacement by an operator with constant coefficients.

4.3.1

Model problem

Discretization of the model problem We apply a LFA analysis to the following two-dimensional model problem: −

∂ 2 u(x, ω) ∂ 2 u(x, ω) − a(x, ω) = f (x) ∂x21 ∂x22

ω ∈ Ω,

(4.8)

A finite-dimensional version of this problem is also tested in Section 3.7.1, see (3.12), in order to numerically check any P -dependence of the iterative solvers. Here, an infinite grid is assumed in order to eliminate the effect of the boundary conditions. The stochastic Galerkin finite element method described in Section 2.4 is applied to (4.8). A stochastic discretization of the stochastic coefficient a approximates a by an expansion with a finite number of random variables, see (2.28) in Section 2.4.2. After representing the solution with a polynomial chaos expansion (2.21) and applying a stochastic Galerkin projection, this results in the semi-discretized equation, −hΨΨT i

∂ 2 u(x) ∂ 2 u(x)

− a(x, ξ)ΨΨT = f (x)hΨi, 2 ∂x1 ∂x22

80


where the gPC coefficient functions

uq (x) in (2.21) are grouped in a column vector u(x). The real symmetric matrix a(x, ξ)ΨΨT is independent of the solution and will be denoted as M(x), M(x) := ha(x, ξ)ΨΨT i =

S X i=1

S X

ai (x)Ci . ai (x) ϕi (ξ)ΨΨT =

(4.9)

i=1

Using the orthonormality of Ψ, the above equation can further be simplified to −

∂ 2 u(x) ∂ 2 u(x) − M(x) = f (x)hΨi. 2 ∂x1 ∂x22

(4.10)

Conventional spatial discretization methods can be applied to reduce this coupled system of deterministic PDEs to an algebraic system. Here, we apply a standard five-point finite difference scheme on a rectangular grid Gh = {(jh, kh)}j,k∈Z with grid spacing h in x1 - and x2 -directions. The value f (jh, kh), the random variable a(jh, kh, ω) and the corresponding matrix M(jh, kh) will be denoted as fj,k , aj,k and Mj,k . The discrete approximation to u(x) evaluated at point (jh, kh) is denoted as uj,k . Applying the spatial discretization to the system (4.10) leads to − (uj−1,k − 2uj,k + uj+1,k ) − Mj,k (uj,k−1 − 2uj,k + uj,k+1 ) = h2 fj,k hΨi. (4.11) When the equations are collected over all grid points, a linear system results, Lh uh = fh .

(4.12)

The dimension of Lh equals the number of spatial grid points multiplied by Q. Local Fourier representation In the case of a variable coefficient problem, a LFA is performed by freezing the random field a to its value aj,k in the considered grid point (jh, kh) [WJ05]. For problem (4.8), this corresponds to replacing the random field a(x, ω) by a random variable a(ω). As such, Mj,k can be considered to be a fixed known matrix, which will be denoted as M. Equation (4.11) can be represented by a linear discrete operator Lh , see (4.12). This operator is invariant to Fourier grid mode components of the form φj,k (θ, z) := exp(ı(jθx1 + kθx2 ))z,

(4.13)

with z = [1 . . . 1]T ∈ RQ×1 , θ = (θx1 , θx2 ) ∈ [−π, π)2 , and ı the imaginary unit. This gives e h (θ)φj,k (θ, z), Lh φj,k (θ, z) = L

LOCAL FOURIER CONVERGENCE ANALYSIS

81

e h (θ) called the symbol of Lh , with L

e h (θ) := 1 (IQ (exp(−ıθx1 ) − 2 + exp(ıθx1 ))+ L h2 M(exp(−ıθx2 ) − 2 + exp(ıθx2 )) 4 =− 2 h

θx 2 θx 1 2 2 IQ + sin M . sin 2 2

(4.14)

If z in (4.13) is selected to be one of the eigenvectors zq of M with corresponding e h (θ)φj,k (θ, z) simplifies to eigenvalue λq , equality Lh φj,k (θ, z) = L e h (θ, λq )φj,k (θ, zq ) Lh φj,k (θ, zq ) = L

with

e h (θ, λq ) := − 4 L h2

θx 2 θx 1 + sin2 λq . sin2 2 2

(4.15)

(4.16)

Hence, the Fourier mode φj,k (θ, zq ) is an eigenfunction of the (frozen) discrete differential operator.

4.3.2

Introduction to smoothing and convergence analysis

The error after m iterations of an iterative method can be decomposed into a sum of exponential Fourier grid modes of the form (4.13). By substituting the Fourier modes into an error iteration, the spectrum of the iteration operator is investigated. For many stationary iterative methods, e.g., Jacobi, Gauss-Seidel or SOR iterations, it holds that the Fourier modes are eigenfunctions of the corresponding iteration operator Sh . The corresponding eigenvalues are called the amplification factor or Fourier symbol of the iteration operator, denoted by Seh (θ), and determine the asymptotic convergence factor. For a variable coefficient problem with sufficiently smooth variation of the coefficients ai [Hac85, section 8.2.2], the asymptotic convergence factor is defined as ρ :=

max

max

x=(jh,kh)∈Gh θ∈[−π,π)2

fh (θ)), ρ(S

(4.17)

with ρ(Seh (θ)) the spectral radius of Seh (θ). The spectral radius ρ(Seh (θ)) is dependent on the spatial position (jh, kh) due to the choice of the grid point at which Mj,k is frozen.

Remark 4.3.1. For practical computations at finite grids with Dirichlet boundary conditions, definition (4.17) must be modified since the eigenvalues and eigenfunctions of the iteration operator at finite grids may no longer correspond to those

82


computed at infinite grids. Reasonable estimates can however be obtained from the infinite grid eigenvalues Seh (θ) by sampling θ-values from a finite grid and excluding the zero frequencies (θx1 = 0 or θx2 = 0) from the analysis [WJ05]. Based on these heuristics, the definition of the asymptotic convergence factor (4.17) applies to the finite domain θ ∈ ΘD [Wes92, p.111], ΘD := {θ|θ = (θx1 , θx2 ) with θd = π

md 1 , 1 ≤ md ≤ nd − 1, nd = (d = x1 , x2 )}. nd h (4.18)

Multigrid methods employ a smoothing step to eliminate high frequency error components, followed by a coarsening step to eliminate low frequency error components, i.e., the smooth part of the error, on a coarser grid, see Section 3.3.1. The convergence properties of multigrid depend on the quality of the relaxation method. The smoothing factor assesses the quality of a smoothing method and can be obtained by LFA. It is a tool for quantifying the reduction rate of the so-called oscillatory Fourier modes [TOS01], i.e., the modes that are visible only on the fine mesh. Fig. 4.2 [WJ05] illustrates the low and high frequencies in the case of a standard coarsening. The smoothing factor s is defined for a problem with variable coefficients as s :=

max

max

x=(jh,kh)∈Gh θ∈[−π,π)2 / [− π , π )2 2 2

ρ(Seh (θ)).

(4.19)

In order to investigate the convergence of a multigrid method, typically a two-grid analysis is performed. An example of an extension to three-grid analysis is given in [WO01], which is particularly useful in the case of a non-standard coarsening.

4.4

Local Fourier analysis of one-level methods

4.4.1

Basic splitting methods

The basic Gauss-Seidel splitting iteration (3.2) is characterized by the iteration −1 operator SGS,h = −(L+ (L− GS,h ) GS,h ), with corresponding symbols e + (θ) :=(exp(−ıθx1 ) − 2)IQ + (exp(−ıθx2 ) − 2)(ML + MD ) L GS,h

(4.20)

+ exp(ıθx2 )ML

e − (θ) := exp(ıθx1 )IQ + exp(ıθx2 )MD − 4 sin2 L GS,h

θx 2 2

MU ,

(4.21)

LOCAL FOURIER ANALYSIS OF ONE-LEVEL METHODS

83

θx 2 π π 2

0 − π2 −π −π

− π2

0

π 2

π θx 1

Figure 4.2: Low (interior white region) and high frequencies (shaded region) for standard coarsening.

where M = ML + MD + MU is decomposed into its strictly lower triangular part, diagonal and strictly upper triangular part. The asymptotic convergence factor can be computed from (4.17), with SeGS,h (θ) defined by (4.20)–(4.21). The largest eigenvalue in modulus is obtained for θ ∈ ΘD (4.18) at (θx1 , θx2 ) = (πh, πh). A similar convergence analysis can be applied to other splittings. For example, in case of a damped Jacobi-splitting with damping factor ωJAC , the operator −1 − SJAC,h = −(L+ LJAC,h is characterized by the symbols JAC,h ) e+ L JAC,h (θ) := −

2 ωJAC

(IQ + MD ),

e− L JAC,h (θ) :=2 cos(θx1 )IQ + 2

1 ωJAC

MD − M +

1 ωJAC

− 1 IQ

+ 2 cos(θx2 )M .

The largest eigenvalue in modulus of SeJAC,h (θ) is again obtained at (πh, πh).

4.4.2

Splitting of the Ki -matrices

A collective GS iteration (3.3) applied to (4.11) yields the following error iteration, new new old old 2(IQ + M)enew j,k − ej−1,k − Mej,k−1 = ej+1,k + Mej,k ,

enew j,k

new

(4.22)

with = uj,k − u and corresponding iteration operator SKi ,h . Applying (4.22) to mode (4.13) shows that the convergence depends on the spectrum of M,

84


denoted as σ(M), i.e., on the random structure of the model problem. Let zq be an eigenvector of M with corresponding eigenvalue λq , and set z = zq in (4.13). Then, we immediately find that SKi ,h φj,k (θ, zq ) = SeKi ,h (θ, λq )φj,k (θ, zq ) with SeKi ,h (θ, λq ) :=

exp(ıθx1 ) + exp(ıθx2 )λq . (2 − exp(−ıθx1 )) + (2 − exp(−ıθx2 )) λq

(4.23)

We can thus decompose the iteration error into a sum of independent components of the form φj,k (θ, zq ), which are eigenvectors of the collective GSLEX iteration operator, with SeKi ,h (θ, λq ) the corresponding eigenvalues. The distribution of these eigenvalues for several values of λq is shown in Fig. 4.3(a)–(c). The eigenvalues lie inside the unit circle for every λq > 0 and every θ ∈ [−π, π)2 . Fig. 4.3(d)–(e) compares the collective GSLEX eigenvalues in case of a stochastic problem to a related deterministic problem, which applies the mean of a(ω) as deterministic coefficient. We observe that each eigenvalue in the deterministic case is split into Q eigenvalues in the stochastic case. This can be explained intuitively by following a similar methodology as in (4.1). We can rewrite the collective GSLEX iteration operator in Kronecker product notation as − −1 + (IQ ⊗ L− SKi ,h := −(IQ ⊗ L+ x1 x1 + M ⊗ Lx2 x2 ), x1 x1 + M ⊗ L x2 x2 )

(4.24)

− + − with the matrix splittings Lx1 x1 = L+ x1 x1 + Lx1 x1 and Lx2 x2 = Lx2 x2 + Lx2 x2 ; Lx1 x1 2 2 is the discretized differential operator −∂ u/∂x1 . Applying a similarity transform based on the eigenvalue-eigenvector decomposition of M to (4.24) leads to

σ(SKi ) =

Q [

q=1

− −1 + (L− σ (L+ x1 x1 + λq Lx2 x2 ) . x1 x1 + λq Lx2 x2 )

In as far as the eigenvalues λq of M approximate the mean value a1 of the random variable a(ω), each eigenvalue of the deterministic problem will be approximated by Q eigenvalues of the stochastic problem. The smoothing factor of a Ki -splitting GS iteration can be determined by replacing Seh (θ) in (4.19) with SeKi ,h (θ, λq ) (4.23) and taking the maximum over all eigenvalues λq of M. A similar replacement is used compute the asymptotic convergence factor. Analytical expressions for these quantities can be deduced from (4.23). The LFA spectral radius is given by ρ = 1 since the maximum in (4.17) is reached for θ = (0, 0). This implies that collective GSLEX iterations do not converge at infinite grids. For practical situations however at finite grids, collective GSLEX does converge. Considering remark 4.3.1, the largest eigenvalue in modulus then exp(ıπh) ; and thus occurs at (θx1 , θx2 ) = (πh, πh) with SeKi ,h ((πh, πh), λq ) = 2−exp(−ıπh) the asymptotic convergence factor is given by ρ = 1 − π 2 h2 + O(h4 ),


85

1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0

0

0

−0.2

−0.2

−0.2

−0.4

−0.4

−0.4

−0.6

−0.6

−0.6

−0.8

−0.8

−0.8

−1

−1 −1

−0.5

0

0.5

1

−1 −1

(a) λq = 1.0

−0.5

0

0.5

1

−1

−0.5

(b) λq = 0.6

0.5

1

(c) λq = 0.2

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

0

−1 −1

−0.5

0

0.5

1

(d) a = 1.0 (deterministic)

−1

−0.5

0

0.5

1

(e) Gaussian variable a(ω), a1 = 1, σ = 0.1, second-order Hermite PC

Figure 4.3: (a)–(c) Distribution of the eigenvalues of the collective GSLEX iteration operator (3.3), for several values of λq : all eigenvalues lie inside the unit circle, between the drop-shaped curve and the inner curve (which reduces to a point for λq = 1). The eigenvalues corresponding to the oscillatory modes lie at the left of the 3-shaped curve, those corresponding to the smooth modes at the right. (d)–(e) Eigenvalues for a deterministic and a simple stochastic problem (400 equidistant samples of θ ∈ [−π, π)2 ).

which is independent of λq . The smoothing factor and spectral radius of collective GSLEX are shown as a function of λq in Fig. 4.6. For a grid with different grid spacing in the two directions, (hx1 , hx2 ), the eigenvalues of the GSLEX Ki -splitting iteration operator become SeKi ,hx1 ,hx2 (θ, λq ) =

h2x2

h2x2 exp(ıθx1 ) + h2x1 exp(ıθx2 )λq . (2 − exp(−ıθx1 )) + h2x1 (2 − exp(−ıθx2 )) λq

In this case, the modulus of the largest eigenvalue does depend on λq . The spectral 1 + λq radius is given by ρ = max 1 − π 2 2 h2 h2 + O(h4x1 h4x2 ). λq ∈M hx2 + h2x1 λq x1 x2

86


Ki -splitting GSLEX

Eigenvalues exp(ıθx1 )+exp(ıθx2 )λq

(2−exp(−ıθx1 ))+(2−exp(−ıθx2 ))λq

x-line GSLEX

ω-Jacobi

cos(θx1 )+cos(θx2 )λq 1+λq

if λq > 1  1  (1+λ 2 q)

2

exp(ıθx2 )λq

(2(1−cos(θx1 )))+(2−exp(−ıθx2 ))λq 1−ω+ω

β)

√ γ(λ2q +1)+4λq γ √ γ(5λ2q +8λq +5)−4λ3q γ

(γ = λ2q − 2λq + 5) 0 or

β(5λq +8λq +5)−4

if 0 < λq ≤ 1 r

(β = 5λ2q − 2λq + 1)

GS-RB

Smoothing factor r √ β(λ2q +1)+4λ2q β √ 2

cos(θx1 )+cos(θx2 )λq 1+λq

      

λ2q (1+λq )2 √ 5 0 5 λq 2+λq 2+λq 2+3λq 1+2λq 3+2λq

0 < λq ≤ 1 λq > 1 < λq ≤

λq >

√ 1+ 5 2

√ 1+ 5 2

0 < λq ≤ 1 λq > 1

Table 4.1: Eigenvalues and smoothing factor for Ki -splitting iterative methods applied to the (frozen) discrete differential operator (4.11) (GSLEX: lexicographic Gauss-Seidel, GS-RB: collective red-black Gauss-Seidel, x-line GSLEX: line GSLEX in the x1 -direction and ω-Jacobi: weighted Jacobi).

LFA can also be performed for other classical block splitting methods. In Table 4.1, we consider Ki -splitting iterations either based on a red-black Gauss-Seidel splitting, a x-line lexicographic Gauss-Seidel scheme, or a pointwise ω-Jacobi scheme, also known as damped or weighted Jacobi. The table shows for each method the formula for the eigenvalues as a function of θ and λq , and the smoothing factor as a function of λq . The analysis of red-black Gauss-Seidel is somewhat more elaborate than the analysis of lexicographic Gauss-Seidel since the red-black updating scheme results in an intermixing of the high frequency and low frequency Fourier modes. This necessitates a generalized definition of the smoothing factor in which the smoothing factor depends on the number of fine-grid relaxation sweeps, see [TOS01, Yav95]. In Table 4.1, the results for a single smoothing step per cycle are given. The analysis of red-black Gauss-Seidel Ki -splitting iterations can straightforwardly be extended to the analysis of red-black successive overrelaxation (SOR) Ki -splitting iterations using the results in [Yav96, Th. 2.1]. If sGS-RB represents the smoothing factor of red-black Gauss-Seidel, then a good approximation to the optimal SOR damping parameter and the corresponding smoothing factor are given by ωSOR, opt and sSOR, opt : ωSOR, opt =

1+

2 √ 1 − sGS-RB

and

sSOR, opt =

1 + sGS-RB √ . 2(1 + 1 − sGS-RB )2


87

1

Smoothing factor

0.8

0.6 GSLEX-Ki 0.4

GS-RB-Ki

0.2

x-GSLEX-Ki

SOR-RB-Ki

ω-Jacobi-Ki 0 0

1

2

4

3 λq

5

Figure 4.4: Smoothing factors for Ki -splitting methods applied to the (frozen) discrete differential operator (4.11) (GSLEX: lexicographic GS, GS-RB: pointwise red-black Gauss-Seidel, SOR-RB: pointwise red-black successive overrelaxation, x-GSLEX: line GSLEX in the x1 -direction and ω-Jacobi: weighted Jacobi).

The smoothing factor for ω-Jacobi is based on an optimal value for the parameter ω that minimizes the smoothing factor. Some calculations show that the optimal value of ω is the following function of λq , ωJAC,opt,λq =

2 + 2λq , 2 + 3λq

if 0 < λq ≤ 1,

and ωJAC,opt,λq =

2 + 2λq 3 + 2λq

if λq > 1.

As illustrated in Table 4.1, for λq = 1, the smoothing factors coincide with the well-known smoothing factors for the central discretization of the two-dimensional Laplacian. A summary of the results is presented in Fig. 4.4, that shows the smoothing properties of the discussed iterative methods as a function of λq .

4.4.3

Splitting of the Ci -matrices

The Ci -splitting (3.5) applied to (4.11) yields the following error iteration, new new new new + enew enew j−1,k − 2ej,k + ej+1,k + M j,k−1 − 2ej,k + ej,k+1 old old = −M− eold j,k−1 − 2ej,k + ej,k+1 .

88


This corresponds to an iteration operator SCi with symbol SeCi ,h (θ) given by

with

−1 γx1 + IQ + M M− , γx2 θd , (d = x1 or x2 ). γd := − 4 sin2 2

SeCi ,h (θ) := −

(4.25)

The largest eigenvalue in modulus of SeCi ,h (θ) occurs for the finite grid ΘD (4.18) at (θx1 , θx2 ) = (πh, π(1 − h)). Using a Taylor expansion around h = 0, we find the following formula for the γx1 /γx2 notation in (4.25): γx1 π 2 h2 . ≈ γx2 4 − π 2 h2 Thus, the convergence is asymptotically independent of h, but depends on the stochastic discretization via M.

The spectral radius (4.17), with Seh (θ) given by (4.25), can be simplified in case of a Jacobi Ci -splitting (3.6). Consider M+ = diag(M) and a(x, ω) discretized by P (1) (i) a truncated KL-expansion so that aj,k (ω) ≈ aj,k + L+1 i=2 aj,k ξi−1 . The matrices P (1) L+1 (i) M+ and M− then correspond respectively to aj,k IQ and i=2 aj,k Ci , with Ci defined by (2.44). Together with (4.25) and (4.17), this yields ! L+1 X (i) γx2 max ρJAC−Ci = max ρ aj,k Ci , (1) x=(jh,kh)∈Gh θ∈[−π,π)2 γx1 + aj,k γx2 i=2 ≤

max

max

x=(jh,kh)∈Gh θ∈[−π,π)2

γx2 γx1 +

L+1 X

(1) aj,k γx2 i=2

(i)

aj,k ρ(Ci ).

The spectral radius ρ(Ci ) can be bounded in terms of the roots of the univariate orthogonal polynomials ψj in (2.13), see theorem 2.5.11. This shows that the bounds on ρ(Ci ) and ρJAC−Ci depend asymptotically only on the polynomial order P , and not on the number of random variables L. The latter follows from the (i) monotone decay of KL-coefficients aj,k , i = 1, . . . , L + 1. Remark 4.4.1. In [PE09], the eigenvalue spectrum of the mean-based preconditioner is analyzed. This shows that the eigenvalues are independent of the mesh size h, but depend on the polynomial order P and the variance σ 2 . The LFA results confirm these properties. An analysis of the eigenvalue spectrum of the Kronecker product preconditioner (3.7) is presented in [Ull10]. It is shown there that its eigenvalue bounds depend on the stochastic discretization parameters, which is also confirmed by our LFA results.

LOCAL FOURIER ANALYSIS OF MULTIGRID METHODS

4.5

89

Local Fourier analysis of multigrid methods

4.5.1

Multigrid in the spatial dimension

To determine the action of a two-grid operator on the Fourier grid modes (4.13), the Fourier space is divided into subspaces spanned by four harmonics, H(θ, z) := 2 span[φ(θ1 , z) φ(θ2 , z) φ(θ3 , z) φ(θ4 , z)], for a given (θx1 , θx2 ) ∈ − π2 , π2 with θ1 = (θx1 , θx2 ),

θ2 = (θx1 , θx2 − sign(θx2 )π),

θ3 = (θx1 − sign(θx1 )π, θx2 ),

θ4 = (θx1 − sign(θx1 )π, θx2 − sign(θx2 )π).

These spaces are invariant under the fine and coarse grid discrete differential operators, and under certain smoothing operators. The action of a smoothing operator on an element of such a space can be described by regrouping Seh (θ), as defined in Section 4.4, into a (4Q × 4Q) block diagonal matrix Sbh (θ): Sbh (θ) := diag(Seh (θ1 ), Seh (θ2 ), Seh (θ3 ), Seh (θ4 )).

e h (θ) defined by (4.14), we can represent the action of Lh by With L b h (θ) := diag(L e h (θ1 ), L e h (θ2 ), L e h (θ3 ), L e h (θ4 )). L

On the coarse grid, L2h is constructed by discretizing (4.8) with a standard fivepoint finite difference scheme on a rectangular grid with grid spacing 2h. Its action can be represented by b 2h (θ) := − 1 sin2 (θx1 )IQ + sin2 (θx2 )M . L 2 h

The prolongation operator maps the mode φ(2θ, z) onto H(θ, z) [TOS01]. It is h e h (θ) ⊗ IQ , with, in the case of bilinear interpolation, characterized by Pb2h (θ) = P 2h h e P2h (θ) given by   (1 + cos(θx1 ))(1 + cos(θx2 )) 1  (1 + cos(θx1 ))(1 − cos(θx2 ))  h e2h . P (θ) :=  (4.26) 4  (1 − cos(θx1 ))(1 + cos(θx2 ))  (1 − cos(θx1 ))(1 − cos(θx2 )) Using standard coarsening, the restriction operator maps the space H(θ, z) onto the single mode φ(θ1 , z). The corresponding Fourier representation is given by b 2h (θ) = (Pbh (θ))T . R h 2h

To conclude, the action of the two-grid operator, applied to the differential operator (4.11) on the space H(θ, z) is characterized by −1 h ν2 2h b b b b b b I4Q − P2h (θ) L2h (θ) Th (θ) := (Sh (θ)) Rh (θ)Lh (θ) (Sbh (θ))ν1 , (4.27)

90


with ν1 and ν2 the number of presmoothing, respectively postsmoothing steps, and I4Q ∈ R4Q×4Q an identity matrix. Under the assumption that the variation of the coefficients ai is sufficiently smooth [Hac85, Section 8.2.2], the asymptotic convergence factor of the two-grid scheme is defined as ρT G = max max 2 ρ Tbh (θ) . (4.28) x=(jh,kh)∈Gh θ∈[− π , π ) 2 2

4.5.2

Multigrid with Ki -splitting smoother

By choosing z in (4.13) equal to an eigenvector zq of M with corresponding eigenvalue λq , the action of a two-grid operator with a Ki -splitting block smoother (3.3) can be simplified from a representation with a (4Q × 4Q)-matrix in (4.27) to a representation with a (4 × 4)-matrix. This stems from the fact that φ(θ, zq ) is an eigenfunction of the differential operator Lh , see (4.15), and of the smoothing operator SKi ,h , see (4.23). It is also an eigenfunction of the operator L2h on grid G2h with eigenvalue e 2h (θ, λq ) := − 1 sin2 (θx1 ) + sin2 (θx2 ) λq . L 2 h

The action of GSLEX-Ki -splitting iterations, denoted as SbKi ,h (θ, λq ), is given by SbKi ,h (θ, λq ) := diag SeKi ,h (θ1 , λq ), SeKi ,h (θ2 , λq ), SeKi ,h (θ3 , λq ), SeKi ,h (θ4 , λq ) ,

with SeKi ,h (θ, λq ) defined by (4.23). The Fourier symbol of Lh , based on (4.16), simplifies to b h (θ, λq ) := diag L e h (θ1 , λq ), L e h (θ2 , λq ), L e h (θ3 , λq ), L e h (θ4 , λq ) . L

Together with the symbol for the prolongation operator defined in (4.26) and the corresponding symbol for the restriction operator, the action of the two-grid operator with block Ki -splitting smoother is characterized by ν2 −1 h e 2h (2θ, λq ) Teh (θ, λq ) := SbKi ,h (θ, λq ) I4 − Pe2h (θ) L (4.29) ν1 b h (θ, λq ) SbKi ,h (θ, λq ) e 2h (θ)L R , h

(4.30)

where ν1 , ν2 are the number of presmoothing, respectively postsmoothing steps and I4 corresponds to a (4 × 4) identity matrix. When the two-grid operator contains spatially varying coefficients, the asymptotic convergence factor of the two-grid operator with Ki -splitting block smoother can be approximated by the maximum

LOCAL FOURIER ANALYSIS OF MULTIGRID METHODS

91

0.1

0.1

0.05

0.05

0

0

−0.05

−0.05

−0.1

−0.1 −0.1

−0.05

0

0.05

0.1

−0.1

−0.05

0

0.05

0.1

(a) a = 1.0 (deterministic) (b) a(ω) Gaussian, a1 = 1, σ = 0.1, 2nd order PC

Figure 4.5: Eigenvalues of the two-grid iteration matrix with Ki -splitting block smoother for a deterministic and a stochastic problem (400 equidistant samples of (θx1 , θx2 ) ∈ [− π2 , π2 )2 ).

of the convergence factors, each corresponding to the differential operator with frozen coefficients at a fixed grid point (jh, kh), for all grid points (jh, kh) [WJ05], eh (θ, λq ) , ρT G = max max max ρ T (4.31) π π 2 x=(jh,kh)∈Gh λq ∈σ(Mj,k ) θ∈[− 2 , 2 )

if the variation of the coefficients is sufficiently smooth [Hac85, Section 8.2.2].

Fig. 4.5 compares the eigenvalues of the two-grid iteration matrix for a stochastic problem and a related deterministic problem. Similar conclusions hold as for Fig. 4.3 in Section 4.4.2. Each of the eigenvalues of the deterministic problem is split into Q eigenvalues for the stochastic problem. In Fig. 4.6, the convergence properties of one-level and two-level iterative methods with GSLEX Ki -splitting block smoother are illustrated as a function of λq . The smoothing properties of one-level GSLEX Ki -splitting and the convergence factors of the two-grid methods show a strong dependence on λq . Especially small values of λq result in a very poor multigrid performance. These small values occur for example when a high-order polynomial chaos basis is used or when the variance of the random field increases. Also other random field statistics, such as the correlation length, influence the spectrum of M, but less severe than the influence of the variance. Increasing the number of Karhunen-Loève terms has only a small influence on the range of eigenvalues λq ; asymptotically this influence disappears. Numerical results illustrate the convergence properties of the iterative methods in Section 4.7.2.

92


Smoothing factor

1

Convergence factor

GSLEX-Ki (h = 2−5 ) 0.8

Two-grid(1,1) Two-grid(2,1)

0.6

Two-grid(2,2)

0.4

0.2

0 0

1

2

3 λq

4

5

Figure 4.6: Convergence factors for a one-level iterative method and a two-grid cycle with GSLEX Ki -splitting block smoother.

4.5.3

Some comments on double orthogonal polynomial chaos

In case the stochastic coefficient a in (4.8) is approximated by a truncated KLexpansion (2.4), a SGFEM discretization with double orthogonal polynomials, see definition 2.3.7, results in a number of deterministic PDEs of the form ! L X ∂2u eq1 ,...,qL ∂ 2u eq1 ,...,qL e q1 ,...,qL i, a (x)ζ − a (x) + = f (x)hΨ − i+1 p +1,q 1 i i 2 ∂x21 ∂x 2 i=1

e q ,...,q i the qi th zero of the e q ,...,q Ψ e given by (2.24) and ζp +1,q = hξi Ψ with hΨi 1 L 1 L i i normalized 1D orthogonal polynomial ψpi +1 , see Section 2.3.3. The deterministic PL coefficient functions, which are given by a1 (x) + i=1 ai+1 (x)ζpi +1,qi , can be collected into a diagonal matrix Λ(x). The set of decoupled PDEs can be written as a system of discretized equations of the form (4.11), with Mj,k replaced by Λj,k . For analysis purposes, it may again be required to freeze the coefficient to a local (q) value, denoted by λj,k . Obviously, all results of Section 4.5.2 continue to hold for (q)

the decoupling approach, when M and λq are replaced by Λ and λj,k respectively. A multigrid convergence analysis of this decoupled approach yields information on the multigrid convergence in the case of a generalized polynomial chaos basis. By applying the same methodology as in the proof of theorem 2.5.11, it can easily be shown that the eigenvalues [λmin , λmax ] of Mj,k must be contained in the range of (min) (max) eigenvalues [λj,k , λj,k ] of Λj,k . Hence, a convergence analysis of the multigrid

LOCAL FOURIER ANALYSIS OF MULTILEVEL METHODS IN THE STOCHASTIC DIMENSION

93

method for the decoupling approach using polynomials of degree P in ξi , provides us immediately with upper bounds for the multigrid convergence of the general approach, using any set of polynomials with degrees in ξi lower than or equal to P . The analysis of the decoupling approach is a straightforward exercise, once the zeros ζP +1,qi have been computed.

4.6

Local Fourier analysis of multilevel methods in the stochastic dimension

In this section, we apply a LFA to the two-level variant of the p-multilevel algorithm presented in Section 3.4, using a polynomial chaos of order p and p − 1. The action of this algorithm on the Fourier mode (4.13) leads to the result Tp φj,k (θ, z) := Tep (θ) φj,k (θ, z), with Tep (θ) defined as p IQl p−1 −1 ν2 e e e e IQl 0 Lh (θ) (Sep (θ))ν1 . (4.32) IQ − (Lh (θ)) Tp (θ) = (Sp (θ)) 0

e p−1 (θ) and L e p (θ) correspond to (4.14), where the index p refers The symbols L h h to the polynomial order used to construct M. The symbol Sep (θ) represents the action of the iterative method applied at every level of the multilevel algorithm. For example, in the case of a Ci -splitting method, Sep (θ) is given by (4.25). The asymptotic convergence factor is then determined by the spectral radius of (4.32), ρT p =

4.7

max

max

x∈(jh,kh)∈Gh θ∈[−π,π)2

ρ(Tep (θ)).

Numerical results

This section presents some test results that illustrate the convergence behavior and sharpness of the local Fourier mode analysis for the single-grid and multigrid methods discussed in the earlier sections. As a test case, model problem (4.8) is considered with zero forcing term on a unit square domain with homogeneous Dirichlet boundary conditions. We test three configurations for a: ag (x, ω), au (x, ω), and alog (x, ω). The random field ag (x, ω) models a Gaussian field that is discretized by a KL-expansion based on an exponential covariance function (2.7). The random field au (x, ω) is constructed by a similar KL-expansion, but with uniformly √ √ distributed random variables on the interval [− 3, 3]. The solution u(x, ω) is then discretized by Legendre polynomials, while ag (x, ω) leads to a Hermite expansion of the solution. The lognormal random field alog (x, ω) is discretized by a Hermite expansion, as described in Section 2.3.2. For the problem to be welldefined [MK05], the expansion order must equal twice the order of the Hermite

94


expansion for the solution. The numerical results will report the order P used to represent the solution and the statistics, i.e., variance and correlation length, of the underlying Gaussian field ag (x, ω).

4.7.1

One-level methods

Basic splitting methods The GSLEX convergence properties are illustrated in Table 4.2. The table shows the theoretical and numerically observed convergence factors when solving (4.8) with GSLEX iterations (3.2) in case of a Gaussian and a lognormal field. To illustrate the 1 − O(h2 ) behavior of the convergence factors, the table displays the convergence factors subtracted from unity. In the first block of rows, the convergence is presented as a function of the mesh spacing h. The total number of discrete unknowns can be computed as the product of the number of internal grid points (1/h − 1)2 multiplied by the number of random basis functions, Q. For example, the result for the h = 2−7 case corresponds to a system constructed with second-order polynomials in 4 random variables ξi , this leads to Q = 15 and a total of 241 935 unknowns. When the order P of the random polynomials is varied (third block row of Table 4.2), the number of random basis functions Q increases as follows: Q = 5, 35, 70, 126 corresponding to P = 1, 3, 4, 5 respectively. Remark 4.7.1. The results for the lognormal field, shown between brackets, behave similarly to those for the Gaussian field. Hence, the convergence of GSLEX is not influenced by the definition of Ci , (2.44) or (2.45). Note that in case of a lognormal field, there is no problem when the variance increases, since the positivity of a(x, ω) is guaranteed. Remark 4.7.2. From M appearing in (4.20) and (4.21), a dependency of the convergence on the stochastic discretization is expected. Table 4.2 shows however that this effect is very limited. Remark 4.7.3. The convergence of GSLEX is strongly dependent on h: this prevents a direct use of GSLEX in practical computations.

Ki -splitting iterations We first consider collective GSLEX, ω-Jacobi, GS-RB and x-line GSLEX Ki splitting iterations (3.3). Tables 4.3 and 4.4 display the convergence and smoothing factors obtained by LFA and numerically observed convergence factors in case a is modelled as a Gaussian random variable avar or as the Gaussian field ag . Results for an isotropic and an anisotropic problem are given. For the results in Table 4.3, Q = 3 random unknowns are present, for the results in Table 4.4, Q = 15. Although the theoretical convergence factors hold only under the infinite grid LFA

NUMERICAL RESULTS

103 (1 − ρtheo ) 103 (1 − ρnum ) 103 (1 − ρtheo ) 103 (1 − ρnum ) 103 (1 − ρtheo ) 103 (1 − ρnum ) 103 (1 − ρtheo ) 103 (1 − ρnum ) 103 (1 − ρtheo ) 103 (1 − ρnum )

95

h = 2−4 30.65 (28.87) 32.74 L=1 8.102 (7.545) 8.276 P =1 8.625 (8.270) 9.096 σ = 0.1 8.741 (8.439) 9.093 lc = 0.25 8.336 (7.955) 8.896

h = 2−5 7.988 (7.470) 8.247 L=3 8.016 (7.514) 8.262 P =3 7.464 (6.867) 7.827 σ = 0.4 6.481 (5.757) 6.847 lc = 0.5 8.124 (7.665) 8.476

h = 2−6 2.019 (1.887) 2.347 L=4 7.988 (7.470) 8.247 P =4 7.009 (6.380) 7.375 σ = 0.6 4.971 (4.336) 5.395 lc = 2.5 7.907 (7.330) 8.079

h = 2−7 0.5060 (0.4731) 0.6329 L=5 7.967 (7.426) 8.230 P =5 6.602 (5.971) 7.052 σ = 0.7 ρtheo > 1 (3.738) ρnum > 1 lc = 5 7.881 (7.277) 8.017

Table 4.2: Theoretical (ρtheo ) and numerical convergence factors (ρnum ) for GSLEX (3.2) applied to (4.8), with Gaussian field ag (x, ω). Between brackets, results for the lognormal field alog (x, ω) are shown. The discretization uses by default h = 2−5 , L = 4, P = 2, σ = 0.2, lc = 1. In each row of the table, one of these parameters is changed.

assumption, we observe a good correspondence between the numerical and theoretical values. Remark 4.7.4. The results illustrate the independence of the asymptotic convergence factors of the Ki -splitting iterative methods on the distribution of the random parameter a. As in the deterministic case, the line Gauss-Seidel method converges faster than the pointwise schemes for the anisotropic problem. Remark 4.7.5. The smoothing factors, which are indicative of the obtainable multigrid convergence factor, show a clear dependence on the statistics of the random parameter for the pointwise iterative methods. Also here, the advantage of using a line relaxation method for the anisotropic problem is obvious. Next, the convergence properties of Ki -splitting GSLEX iterations are investigated more thoroughly, see Table 4.5. The 1 − O(h2 ) behavior of the convergence factors is illustrated by displaying the convergence factors subtracted from unity. The same discretization parameters are used as in Table 4.2. Remark 4.7.6. The theory in Section 4.4.2 states that the asymptotic convergence of GSLEX Ki -splitting iterations is independent of the eigenstructure of M when an equal grid spacing is used in the x1 - and x2 -dimension. The numerical results confirm the independence of the convergence rate on the polynomial order, on the number of random dimensions, and on the variance and correlation length of the random field. The convergence rate only depends on the grid spacing.

96


Ki -splitting

avar : a1 = 1, σ = 0.25 ρtheo stheo ρnum

avar : a1 = 0.2, σ = 0.05 ρtheo stheo ρnum

ω-Jacobi GSLEX GS-RB x-line GSLEX

0.988 0.964 0.964 0.935

0.983 0.964 0.964 0.827

0.600 0.551 0.407 0.447

0.985 0.962 0.962 0.937

0.800 0.817 0.807 0.447

0.982 0.962 0.962 0.845

Table 4.3: Theoretical convergence factor (ρtheo ), smoothing factor (stheo ), and numerically observed convergence factor (ρnum ) for the one-level Ki -splitting iterative solution of (4.8) with random variable avar (h = 2−4 , 2nd order Hermite PC basis).

Ki -splitting

ag : a1 (x) = 1, σ = 0.25 ρtheo stheo ρnum

ag : a1 (x) = 0.2, σ = 0.05 ρtheo stheo ρnum

ω-Jacobi GSLEX GS-RB x-line GSLEX

0.986 0.964 0.964 0.933

0.982 0.964 0.964 0.821

0.684 0.544 0.390 0.447

0.984 0.962 0.962 0.935

0.898 0.808 0.797 0.447

0.982 0.962 0.962 0.838

Table 4.4: Theoretical convergence factors (ρtheo ), smoothing factor (stheo ), and numerically observed convergence factors (ρnum ) for the one-level Ki -splitting iterative solution of (4.8) (h = 2−4 , 2nd order Hermite PC basis, L = 4, lc = 1).

Ci -splitting iterations Table 4.6 shows the theoretical convergence factors of Ci -splitting iterations applied to (4.8), for various combinations of the spatial and stochastic discretization parameters. Also, the effect of the type of polynomial chaos on the convergence is demonstrated: Legendre polynomials result overall in lower convergence factors than Hermite polynomials. Remark 4.7.7. Table 4.6 shows the independence of the convergence on the grid spacing h, and the dependence on the number of random variables, the polynomial order and the variance. The dependence on the polynomial order is worse in case of Hermite polynomials than in case of Legendre polynomials. This property is related to the infinite range of Hermite polynomials, compared to the boundedness √ √ of Legendre polynomials to the interval [− 3, 3], as explained in Section 4.2.3. Table 4.7 shows convergence factors for Gauss-Seidel and block Gauss-Seidel Ci splitting iterations. Numerical convergence factors are given to illustrate the accuracy of the analysis. Remark 4.7.8. The more expensive block GS Ci -splitting iterations do not result in a faster convergence than Gauss-Seidel Ci -splitting iterations. The same result holds when comparing Jacobi Ci -splittings to block Jacobi Ci -splittings.

NUMERICAL RESULTS

97

1 − ρtheo = 10 × 1 − ρnum = 10−3 × −3

1 − ρtheo = 10−3 × 1 − ρnum = 10−3 × 1 − ρtheo = 10−3 × 1 − ρnum = 10−3 × 1 − ρtheo = 10−3 × 1 − ρnum = 10−3 × 1 − ρtheo = 10−3 × 1 − ρnum = 10−3 ×

h = 2−3 124.4 126.0

h = 2−4 36.35 38.06

h = 2−5 9.494 9.607

h = 2−6 2.400 2.407

h = 2−7 0.6018 0.6023

L=1 9.494 9.607

L=2 9.494 9.607

L=3 9.494 9.607

L=4 9.494 9.607

L=5 9.494 9.607

P =1 9.494 9.607

P =2 9.494 9.607

P =3 9.494 9.607

P =4 9.494 9.607

P =5 9.494 9.607

σ = 0.1 9.494 9.607

σ = 0.2 9.494 9.607

σ = 0.4 9.494 9.607

σ = 0.6 9.494 9.607

σ = 0.7 9.494 ρnum > 1

lc = 0.25 9.494 9.607

lc = 0.5 9.494 9.607

lc = 1 9.494 9.607

lc = 2.5 9.494 9.607

lc = 5 9.494 9.607

Table 4.5: Theoretical (ρtheo ) and numerically observed convergence factors (ρnum ) for lexicographic Gauss-Seidel Ki -splitting iterations applied to (4.8): difference from 1 is given. Unless specified differently, the following default configuration is used: grid spacing h = 2−5 , 2nd order Hermite PC basis, ag with L = 4, σ = 0.2 and lc = 1. ag (x, ω) – Hermite chaos

au (x, ω) – Legendre chaos

h ×

2 9.930

2 10.09

2 10.13

2 10.14

2 8.116

2−5 8.243

2−6 8.275

2−7 8.284

1 8.624

5 10.38

8 10.65

10 10.87

1 5.174

5 8.461

8 9.354

10 9.433

P ρtheo = 10−2 ×

1 3.363

3 18.33

4 27.45

5 37.17

1 3.363

3 12.78

4 16.54

5 19.52

0.1 2.522

0.4 40.36

0.6 90.80

0.7 >1

0.1 2.061

0.4 32.97

0.6 74.19

0.7 >1

0.25 5.966

0.5 8.359

2.5 11.21

5 11.58

0.25 5.247

0.5 6.901

2.5 8.643

5 8.365

ρtheo = 10

−2

L ρtheo = 10−2 ×

σ ρtheo = 10−2 ×

lc ρtheo = 10−2 ×

−4

−5

−6

−7

−4

Table 4.6: Effect of the type of polynomials on the theoretical (ρtheo ) convergence factors for GSLEX Ci -splitting iteration (3.5) applied to (4.11). The discretization uses h = 2−5 , L = 4, P = 2, σ = 0.2, lc = 1. In each row, one of these parameters is changed.

Table 4.8 shows that the circulant Ci -approximation leads to worse convergence factors than the Gauss-Seidel Ci -splitting (see Table 4.7).

98


Gauss-Seidel-Ci h ρtheo = 10−2 × ρnum = 10−2 × L ρtheo = 10−2 × ρnum = 10−2 × P ρtheo = 10−2 × ρnum = 10−2 × σ ρtheo = 10−2 × ρnum = 10−2 × lc ρtheo = 10−2 × ρnum = 10−2 ×

2 7.070 6.695 1 7.128 6.749 1 2.802 2.659 0.1 1.855 1.681 0.25 3.182 2.619 −4

2 7.112 6.656 5 7.058 6.448 3 11.80 11.22 0.4 24.32 23.49 0.5 5.279 4.935 −5

2 7.123 6.734 8 7.058 6.475 4 16.52 15.58 0.6 43.81 43.03 2.5 8.543 8.110 −6

Block GS-Ci 2 7.125 6.763 10 7.007 6.591 5 21.12 20.31 0.7 52.64 51.71 5 9.085 8.720 −7

2 8.205 7.793 1 7.128 6.734 1 3.263 3.051 0.1 2.170 2.015 0.25 5.017 4.636 −4

2−5 8.261 7.853 5 8.479 8.089 3 13.64 12.91 0.4 27.59 26.36 0.5 6.921 6.392

2−6 8.272 7.857 8 8.688 8.426 4 19.01 18.21 0.6 48.33 47.03 2.5 9.114 8.745

2−7 8.276 7.885 10 8.854 8.575 5 24.19 23.33 0.7 57.30 56.33 5 9.394 9.135

Table 4.7: Effect of using the GS versus block GS Ci -splitting strategy on the theoretical (ρtheo ) and numerical (ρnum ) convergence factors for problem (4.11), with lognormal field alog (x, ω). The discretization uses h = 2−5 , L = 4, P = 2, σ = 0.2, Lc = 1. In each row, one of these parameters is changed. Grid spacing ρtheo = 10−1 × Random variables ρtheo = 10−1 ×

h = 2−4 3.226 L=1 1.778

h = 2−5 3.251 L=5 3.432

h = 2−6 3.253 L=8 3.543

h = 2−7 3.253 L = 10 3.607

Order PC-basis ρtheo = 10−1 ×

P =1 1.968

P =3 4.644

P =4 6.102

P =5 7.363

Standard deviation ρtheo = 10−1 ×

σ = 0.1 1.553

σ = 0.4 7.057

σ = 0.6 >1

σ = 0.7 >1

Correlation length ρtheo = 10−1 ×

lc = 0.25 2.475

lc = 0.5 2.955

lc = 2.5 3.381

lc = 5 3.414

Table 4.8: Effect of using the circulant Ci -splitting iteration (3.5) on the theoretical (ρtheo ) convergence factors for problem (4.11). The random field alog (x, ω) is discretized by a Hermite expansion. The discretization uses by default h = 2−5 , L = 4, P = 2, σ = 0.2, lc = 1. Each row corresponds to changing one of these parameters.

4.7.2

Multigrid methods in the spatial dimension

Ki -splitting smoother First, we consider multigrid with Ki -splitting block smoother, which was analyzed in Section 4.5.2. Table 4.9 presents some test results in order to verify the con-

NUMERICAL RESULTS

99

Grid spacing

h = 2−4

ρnum Random variables ρtheo ρtheo ρnum Polynomial order ρtheo ρtheo ρnum Variance σ 2 ρtheo ρtheo ρnum Correlation length ρtheo ρtheo ρnum

0.105 L=1 0.122 0.122 0.113 P =1 0.120 0.123 0.113 σ = 0.05 0.119 0.119 0.112 lc = 0.25 0.120 0.250 0.122

h = 2−5 h = 2−6 h = 2−7 ρtheo = 0.123; ρtheo = 0.164 0.111 0.112 0.113 L=2 L=3 L=4 0.122 0.123 0.123 0.128 0.138 0.164 0.113 0.113 0.113 P =2 P =3 P =4 0.123 0.129 0.140 0.164 0.339 0.765 0.113 0.117 0.151 σ = 0.1 σ = 0.2 σ = 0.3 0.119 0.123 0.133 0.122 0.164 0.496 0.112 0.113 0.135 lc = 0.5 lc = 1 lc = 2.5 0.120 0.120 0.120 0.229 0.224 0.161 0.112 0.112 0.112

h = 2−8 0.113 L=5 0.123 0.198 0.113 P =5 0.183 >1 0.187 σ = 0.4 0.200 >1 0.202 lc = 5 0.120 0.134 0.112

Table 4.9: Numerical (ρnum ) and theoretical convergence factors (ρtheo and ρtheo ) for the two-grid cycle T G(2, 1) with GSLEX Ki -splitting block smoother. Unless specified differently, the following configuration is used: h = 2−7 , 2nd order Hermite PC, KLexpansion of ag with L = 4, σ = 0.2 and lc = 1. The tests on the effect of the correlation length are based on L = 20 and P = 1.

vergence analysis of the two-grid method. Two types of theoretical convergence factors are shown. The first, ρtheo , is based on equation (4.31). The second, ρtheo , is an upper bound for ρtheo based on the connection between the general and the decoupling approach discussed in Section 4.5.3 and calculated by using the roots ζP +1,qi . Also numerically observed results are presented. The eigenvalue ranges for the coefficient matrices, M and Λ, are given in Table 4.10. The results for the various correlation lengths in the last block row of Tables 4.9 and 4.10 are based on a 20-term KL-expansion instead of on a 4-term expansion since decreasing the correlation length broadens the KL-eigenvalue spectrum and thus requires more KL-terms to represent the random field. Remark 4.7.9. For small problems, the upper bounds ρtheo are quite sharp. For larger systems their quality deteriorates, especially for a high polynomial order or a larger number of random dimensions. This is due to the fact that the double ore contains many more basis functions than Ψ, when the polynomial thogonal basis Ψ order or the number of random dimensions is sufficiently high. Remark 4.7.10. The convergence analysis in Section 4.5.2 states that the multigrid convergence is independent of the grid spacing, but not of the eigenvalue distri-

100


σ(Mj,k ) σ(Λj,k )

L=1 [0.706,1.294] [0.706,1.294]

L=2 [0.694,1.306] [0.585,1.415]

L=3 [0.687,1.313] [0.489,1.511]

L=4 [0.682,1.318] [0.422,1.578]

L=5 [0.677,1.323] [0.366,1.634]


P =1 [0.816,1.184] [0.666,1.334]

P =2 [0.682,1.318] [0.422,1.578]

P =3 [0.571,1.429] [0.221,1.779]

P =4 [0.475,1.525] [0.047,1.953]

P =5 [0.389,1.611] [-0.109,2.109]


σ = 0.05 [0.920,1.080] [0.856,1.144]

σ = 0.1 [0.841,1.159] [0.711,1.289]

σ = 0.2 [0.682,1.318] [0.422,1.578]

σ = 0.3 [0.522,1.478] [0.133,1.867]

σ = 0.4 [0.363,1.637] [-1.558,2.156]


lc = 0.25 [0.814,1.186] [0.301,1.699]

lc = 0.5 [0.809,1.191] [0.325,1.676]

lc = 1 [0.806,1.194] [0.331,1.669]

lc = 2.5 [0.802,1.198] [0.428,1.572]

lc = 5 [0.801,1.199] [0.516,1.484]

Table 4.10: Eigenvalue ranges for the coefficient matrices M and Λ. Unless specified differently, the following configuration is used: 2nd order Hermite PC, KL-expansion of ag with L = 4, σ = 0.2 and lc = 1. The tests on the effect of the correlation length are based on L = 20 and P = 1.

bution of M. The mesh-independent convergence is illustrated by the first block row of Table 4.9. As for the λq -dependence, we observe that the range of eigenvalues λq is in particular sensitive to the order of the random polynomial basis and the variance of the random field, see Table 4.10. It is to a lesser extent dependent on the correlation length. Asymptotically the range of eigenvalues λq becomes independent of the number of KL-terms. This explains the observed independence of the convergence factors on the number of random variables. Fig. 4.7 illustrates the performance of the true multi-grid method with a block Ki -splitting smoother (lexicographic and red-black Gauss-Seidel, red-black successive overrelaxation), and with an x-line Gauss-Seidel Ki -splitting smoother, for an isotropic and an anisotropic problem. As in the deterministic PDE case, line relaxation methods can deal effectively with (certain types of) anisotropy when standard coarsening is used. Further, a red-black SOR method with suitable damping factor turns out to be an efficient smoother both for isotropic and anisotropic problems. This confirms with the behavior of the SOR smoothing factor as shown in Fig. 4.4. The practical multigrid convergence behavior is demonstrated in Table 4.11. The problem setup results in a linear system with about 106 unknowns (h = 2−8 , Q = 15). The multigrid convergence rate of the W -cycles is as good as the twogrid performance. Additional results on the numerical multigrid performance are given in Section 3.7.

NUMERICAL RESULTS

101

0

0

10

GSLEX-Ki GS-RB-Ki x-line GSLEX

kr (m) k2 /kr (0) k2

kr (m) k2 /kr (0) k2

10

SOR-RB-Ki

−5

10

x-line GSLEX −5

SOR-RB-Ki

10

−10

10

−10

10

−15

10

GSLEX-Ki GS-RB-Ki

−15

5

10 Iterations

(a) ag : a1 (x) = 1, σ = 0.25

15

10

10

20 Iterations

30

(b) ag : a1 (x) = 0.2, σ = 0.05

Figure 4.7: Residual norms for a V(2,1) multigrid cycle with Ki -splitting smoother: either GSLEX Ki -splitting (GSLEX), red-black Gauss-Seidel (GS-RB), lexicographic xline Gauss-Seidel (x-line GS-LEX) or red-black successive overrelaxation (SOR-RB) Ki splitting (h = 2−4 , 2nd order Hermite PC, KL-expansion of ag with L = 4 and lc = 1).

TG-cycle V-cycle W-cycle

Number of smoothing steps (1,1) (2,1) (2,2) 0.117 0.051 0.039 0.145 0.074 0.055 0.118 0.051 0.039

Table 4.11: Numerical convergence factors for multigrid applied to (4.8). A red-black Gauss-Seidel Ki -splitting smoother is used. (h = 2−8 , 2nd order Hermite PC, KLexpansion of ag with L = 4, σ = 0.2 and lc = 1).

Basic smoother Table 4.12 illustrates the multigrid convergence in case of a standard Gauss-Seidel smoother (3.2). The numerical results demonstrate the accuracy of the analysis. For comparison, the theoretical convergence factors of multigrid with collective Gauss-Seidel smoother (3.3) are also given. Remark 4.7.11. The theoretical convergence factors for both smoothers are very similar. The more expensive Ki -splitting smoother does not yield significant faster convergence. Moreover, the convergence factors are even higher in case of a lognormal field. Remark 4.7.12. The convergence is independent of the spatial and stochastic discretization parameters. There is only a small dependency on the polynomial order, because of the asymmetry of the differential operator (4.8), as predicted by the LFA. Multigrid methods with basic or collective Gauss-Seidel smoother have an optimal convergence behavior.

102


ρ∗theo ρtheo ρnum ρ∗theo ρtheo ρnum ρ∗theo ρtheo ρnum ρ∗theo ρtheo ρnum

h = 2−4 0.123 (0.287) 0.186 (0.231) 0.174 L=1 0.122 (0.280) 0.175 (0.230) 0.151 P =1 0.120 (0.250) 0.131 (0.224) 0.125 σ = 0.1 0.119 (0.239) 0.127 (0.218) 0.117

h = 2−5 0.123 (0.287) 0.186 (0.231) 0.174 L=5 0.123 (0.290) 0.188 (0.231) 0.172 P =3 0.129 (0.320) 0.247 (0.237) 0.226 σ = 0.4 0.200 (0.398) 0.423 (0.281) 0.396

h = 2−6 0.123 (0.287) 0.186 (0.231) 0.175 L=8 0.123 (0.290) 0.191 (0.231) 0.174 P =4 0.140 (0.349) 0.315 (0.244) 0.289 σ = 0.6 0.770 (0.519) 0.902 (0.367) 0.852

h = 2−7 0.123 (0.287) 0.186 (0.231) 0.175 L = 10 0.123 (0.291) 0.192 (0.231) 0.179 P =5 0.183 (0.375) 0.391 (0.250) 0.360 σ = 0.7 >1 (0.580) >1 (0.421) >1

Table 4.12: Theoretical (ρtheo ) (4.28) and numerical (ρnum ) convergence factors for the two-grid iteration TG(2,1) with GSLEX smoother (3.2) applied to problem (4.11), with Gaussian field ag (x, ω) or lognormal field alog (x, ω) (between brackets). Theoretical (ρ∗theo ) convergence factors for the two-grid iteration with a collective GS (3.3) smoother. The discretization uses by default h = 2−5 , L = 4, P = 2, σ = 0.2, Lc = 1. In each row, one of these parameters is changed.

Ci -splitting smoother For completeness, also the convergence of multigrid with Ci -splitting smoother is analyzed. Table 4.13 presents convergence factors for the two-grid iteration (4.28) with a Gauss-Seidel Ci -splitting iteration as smoother (3.5). The Ci -splitting smoother can result in very low convergence factors, but becomes very expensive in case of a large number of spatial and random unknowns, see also remark 3.2.2. Remark 4.7.13. Despite the high convergence rate of multigrid with Ci -splitting smoother, the convergence rate decreases substantially for a large polynomial order or variance. The multigrid convergence is similar to the convergence of the smoother, see Table 4.6.

4.7.3

Multilevel methods in the stochastic dimension

Table 4.14 illustrates the convergence properties of a p-multilevel iteration, combined with a Gauss-Seidel Ci -splitting operator. The numerical convergence factors illustrate that the convergence results for the p-multilevel algorithm are well predicted by the analysis for the two-level case.

CONCLUSIONS

Grid spacing ρtheo = 10−3 × ρnum = 10−3 × Random variables ρtheo = 10−3 × ρnum = 10−3 × Order PC-basis ρtheo = 10−3 × ρnum = 10−3 × Standard deviation ρtheo = 10−3 × ρnum = 10−3 × Correlation length ρtheo = 10−3 × ρnum = 10−3 ×

103

h = 2−4 4.587 3.669 L=1 4.239 3.616 P =1 0.8779 0.7372 σ = 0.1 0.3108 0.2457 lc = 0.25 0.8842 0.8083

h = 2−5 4.587 3.817 L=5 4.939 4.067 P =3 11.71 9.802 σ = 0.4 54.49 49.93 lc = 0.5 2.776 2.229

h = 2−6 4.587 3.781 L=8 4.939 4.123 P =4 22.16 19.29 σ = 0.6 180.3 169.2 lc = 2.5 6.298 5.497

h = 2−7 4.587 3.864 L = 10 5.130 4.351 P =5 35.61 32.09 σ = 0.7 262.6 245.0 lc = 5 7.014 6.318

Table 4.13: Theoretical (ρtheo ) and numerical (ρnum ) convergence factors for the twogrid iteration (4.28) TG(2,1) with a Gauss-Seidel Ci -splitting iteration as smoother (3.5). The random field a(x, ω) is modelled as alog (x, ω). The discretization uses by default h = 2−5 , L = 4, P = 2, σ = 0.2, lc = 1. In each row, one of these parameters is changed.

Remark 4.7.14. The p-multilevel algorithm combined with a Ci -splitting iteration has the same convergence properties as a Ci -splitting iteration (3.5) stand-alone: the convergence is independent of h, but depends on P and σ. Comparing Table 4.14 to Table 4.6 shows the similar convergence behavior. Table 4.15 shows the theoretical convergence factors of the two-level algorithm, combined with various iterative solvers at each p-level. Remark 4.7.15. Since the convergence behavior of the p-multilevel algorithm of Section 3.4 strongly depends on the method used in each relaxation step, we observe in Table 4.15 that only Ci -splitting iterations lead to satisfying convergence results. Note that these results deteriorate for large polynomial chaos orders.

4.8

Conclusions

We have extended and applied local Fourier analysis techniques to study the performance of iterative solvers for PDEs involving random coefficients. To this end, the special structure of the discrete algebraic systems arising from the spectral polynomial expansion approach has been exploited. The LFA shows that the convergence depends crucially on the eigenstructure of a certain matrix, which characterizes the random structure of the PDE. This analysis explains many of the features that are observed in computational experiments.

104


10−3 ×

alog (x, ω) – Hermite chaos

h ρtheo ρnum,TL ρnum,ML L ρtheo ρnum,TL ρnum,ML P ρtheo ρnum,TL ρnum,ML σ ρtheo ρnum,TL ρnum,ML lc ρtheo ρnum,TL ρnum,ML

2 3.551 2.803 2.814 1 3.034 2.615 2.650 1 0.9000 1.851 1.851 0.1 0.2390 0.1818 0.1803 0.25 1.034 0.5651 0.6020 −4

2 3.556 2.871 2.904 5 3.889 3.033 3.063 3 7.753 6.323 6.560 0.4 43.39 36.47 37.99 0.5 2.308 1.745 1.679 −5

2 3.556 2.915 2.939 8 3.923 3.084 3.100 4 13.32 10.81 11.38 0.6 147.9 131.2 140.9 2.5 4.667 4.110 4.137 −6

2 3.556 2.939 2.965 10 4.138 3.066 3.101 5 20.09 16.08 18.08 0.7 218.2 193.8 215.7 5 5.117 4.604 4.663 −7

au (x, ω) – Legendre chaos 2 2.826 2.148 2.203 1 0.5853 0.4831 0.5035 1 1.141 3.476 3.476 0.1 0.1671 0.1305 0.1312 0.25 1.203 0.9526 0.9642 −4

2−5 2.839 2.276 2.327 5 3.028 2.605 2.673 3 4.496 3.491 3.625 0.4 57.51 48.21 51.04 0.5 1.963 1.604 1.631

2−6 2.843 2.517 2.566 8 3.883 3.161 3.272 4 5.776 4.521 4.769 0.6 431.4 365.8 386.4 2.5 3.054 2.440 2.510

2−7 2.844 2.527 2.589 10 3.958 3.351 3.436 5 6.679 5.130 5.488 0.7 >1 886.4 895.5 5 2.779 2.295 2.359

Table 4.14: Theoretical (ρtheo ) and numerically observed (ρnum ) convergence factors for the two-level (TL) and multilevel (ML) V -cycle variant of the p-multilevel algorithm with GS Ci -splitting inner iterations applied to (4.11), with ν1 = ν2 = 1. The discretization uses by default h = 2−5 , L = 4, P = 2, σ = 0.2 and Lc = 1. In each row, one of these parameters is changed.

L

P

basic GS (3.2)

GS Ki -splitting (3.3)

GS Ci -splitting (3.5)

4 4 8

2 4 2

0.9811 0.9811 0.9811

0.9811 0.9811 0.9811

4.918 · 10−3 26.86 · 10−3 7.283 · 10−3

Table 4.15: Theoretical convergence factors for the two-level algorithm described in Section 3.4, using PC orders P and P − 1, applied to problem (4.11) with Gaussian field ag (σ 2 = 0.04 and lc = 1). A GSLEX iteration (3.2), a GSLEX Ki -splitting iteration (3.3) or a GS Ci -splitting iteration (3.5) is used at the smoothing steps; ν1 = ν2 = 1.

CONCLUSIONS

105

Of the one-level methods, only Ci -splitting methods yield good convergence rates, although these deteriorate for large values of the polynomial chaos order. The other one-level methods are to be used in a multigrid context. In that case, both a basic smoother and a Ki -splitting smoother result in comparable convergence properties. The convergence of multilevel methods that apply a coarsening in the stochastic dimension is very similar to the convergence of the Ci -splitting methods on which the multilevel algorithm is based.

Chapter 5

Time-dependent stochastic partial differential equations 5.1

Introduction

In this chapter, we develop efficient multigrid solvers for the stochastic Galerkin discretization of the time-dependent stochastic diffusion problem ∂u(x, t, ω) − ∇ · (a(x, t, ω)∇u(x, t, ω)) = f (x, t, ω), ∂t

(5.1)

with x ∈ D, ω ∈ Ω, t ∈ [0, Tf ]. Model problem (5.1) is completed with suitable boundary and initial conditions, which may also contain random coefficients. Theoretical aspects of the stochastic Galerkin discretization of (5.1) are studied in [NT09]. In those results, no time discretization is introduced. In [LKD+ 03, XS09], the stochastic Galerkin discretization of (5.1) is combined with respectively an implicit backward Euler time discretization, or a mixed explicit/implicit time discretization. In the former case, it is possible to apply directly the multigrid solution approach presented in Section 3.3.2 to solve the discretized system at each time step. In the latter case, the expensive solution of a high-dimensional algebraic system at each time step can be avoided. In this chapter, the stochastic Galerkin finite element method is combined with a high-order time discretization and the solution of the corresponding algebraic systems is addressed. The results of this chapter have been published in [RBV08]. Model problem (5.1) is discretized according to the ‘method of lines’, i.e., the PDE is first discretized in space and in the stochastic dimension, after which a system of ordinary differential equations (ODEs) results. The spatial and stochas107

108

TIME-DEPENDENT STOCHASTIC PARTIAL DIFFERENTIAL EQUATIONS

tic discretization assume that analogous conditions for the coefficient a and the forcing field f hold as for the corresponding steady-state problem (2.25), see Section 2.4.2. This enables one to prove the uniqueness and existence of the stochastic Galerkin weak solution of (5.1) [NT09]. Moreover, it is possible to prove that the solution of (5.1) is analytic w.r.t. the random coefficients [NT09], which is an important requirement for a fast convergence of the stochastic Galerkin solution, see also the discussion on the SGFEM convergence in Section 2.4.1. An appropriate choice of stochastic basis functions results in an exponential convergence rate of the time-dependent stochastic Galerkin solution w.r.t. the applied polynomial order, and an algebraic convergence w.r.t. the total number of stochastic unknowns, i.e., Q [NT09]. This chapter first introduces a high-order time discretization scheme and applies this to the stochastic Galerkin discretization of (5.1). Section 5.3 presents an algebraic multigrid method. The theoretical AMG convergence properties are summarized in Section 5.4 and implementation issues are discussed in Section 5.5. Section 5.6 suggests some alternative Ci -splitting based solution approaches. In Section 5.7, numerical experiments are given to illustrate the AMG convergence behavior and its efficiency in comparison to alternative solution approaches.

5.2

Time discretization

5.2.1

Implicit Runge-Kutta methods

An implicit Runge-Kutta time (IRK) discretization is an important tool for the treatment of stiff differential equations [BW96]. These can, e.g., result from discretizing a diffusion problem (5.1). An IRK method enables the construction of a stable high-order time discretization. Consider a system of ODEs of the form du = f (t, u), with u(t0 ) = u0 ∈ RN . dt An IRK method computes an approximation um+1 to the solution u(tm+1 ) at time tm+1 from an approximation um at time tm . To this end, it introduces a number of auxiliary variables xj , j = 1, . . . , sirk , called stage values or stage vectors, at times tm + cj ∆t with time step ∆t = tm+1 − tm . The IRK procedure corresponds to the following set of equations: um+1 = um + ∆t

sirk X

bj f (tm + cj ∆t, xj )

sirk X

aij f (tm + cj ∆t, xj ),

(5.2)

j=1

xi = um + ∆t

j=1

i = 1, . . . , sirk .

(5.3)

TIME DISCRETIZATION

109

5 12 3 4 3 4

1 3

1

1

1

1 order 1

1 − 12 1 4 1 4

√ 4− 6 10 √ 4+ 6 10

1

√ 88−7 6 360 √ 296+169 6 1800 √ 16− 6 36 √ 16− 6 36

order 3

√ 296−169 6 1800 √ 88+7 6 360 √ 16+ 6 36 √ 16+ 6 36

√ −2+3 6 225√ −2−3 6 225 1 9 1 9

order 5

Table 5.1: Butcher tableaus of Radau IIA IRK methods.

Equation (5.2) expresses um+1 as an update to um in terms of the stage values {xj }j=1,...,sirk . Equation (5.3) describes the system of equations to be solved to compute the stage values. The IRK method is fully characterized by the parameters Airk = [aij ], birk = [b1 . . . bsirk ]T and cirk = [c1 . . . csirk ]T . These parameters are commonly presented in a Butcher tableau as follows: c1 .. .

a11 .. .

csirk

asirk 1 b1

. . . a1sirk .. .. cirk . . = . . . asirk sirk ... bsirk

Airk

(5.4)

bTirk

Table 5.1 illustrates some Butcher tableaus for the popular class of Radau IIA IRK methods [HW02, p.74]. Other well-known IRK methods include Gauss and Lobatto methods, see [HW02, But03] for an overview. Equations (5.2) and (5.3) are often rewritten in terms of the stage value increments ∆xj := xj − um : um+1 = um + [∆x1 . . . ∆xsirk ]A−T irk birk ∆xi = ∆t

sirk X

aij f (tm + cj ∆t, um + ∆xj ),

(5.5) i = 1, . . . , sirk .

(5.6)

j=1

Remark 5.2.1. The case aij = 0 for i ≤ j in (5.4) corresponds to an explicit Runge-Kutta method (ERK). If Airk is lower triangular with at least one non-zero diagonal element, we have a diagonally implicit Runge-Kutta (DIRK) method. If, in addition, all diagonal elements are non-zero and identical, this is called a singly diagonally implicit Runge-Kutta (SDIRK) method [HNW00, p. 205]. The general case of a dense matrix Airk yields a fully implicit Runge-Kutta method. Typically, fully implicit methods have a higher order of accuracy than (S)DIRK methods for the same number of stages. Remark 5.2.2. The backward or implicit Euler method is equivalent to a Radau IIA IRK method of order 1, see Table 5.1. This method is always stable and can be written as um+1 = um + ∆tf (tm+1 , um+1 ).

110


5.2.2

Discretization of the model problem

The spatial and stochastic discretization of model problem (5.1) proceeds analogously to the description in Section (2.4.2), but in this case the polynomial chaos coefficients uq,n in (2.32) are time-dependent functions instead of scalars. Assuming a time-independent stochastic diffusion coefficient a(x, ω) in (5.1), this yields the following ODE-system: S

C1 ⊗ M

du(t) X Ci ⊗ Ki u(t) = b(t), + dt i=1

(5.7)

where b(t), u(t) ∈ RQN ×1 contain respectively the discretized right-hand side and solution vector. Each component of u(t) corresponds to a uq,n (t)-function in (2.32). In (5.7), M ∈ RN ×N is the mass matrix, whose elements are defined as Z sj (x)sk (x)dx, M (j, k) = D

with sj (x) a finite element nodal basis function. Matrix Ki is a stiffness matrix defined by (2.35) and based on diffusion coefficient ai (x), with ai (x) a component of expansion (2.28). The matrices Ci ∈ RQ×Q represent the stochastic discretization and are given by (2.34).

System (5.7) is discretized by an IRK method, i.e., discretization scheme (5.5)– (5.6) is applied. Denote the approximation at time tm+1 to the solution u(tm+1 ) by um+1 . The stage vector increments will be denoted simply as xj , j = 1, . . . , sirk , and are grouped together into a long vector x ∈ RN Qsirk ×1 , where the increments are numbered first along the random dimension, then along the spatial dimension and finally according to the IRK stages. When the coefficient a(x, ω) is timeindependent, the algebraic system (5.6) corresponding to problem (5.7) becomes ! S X (5.8) Ci ⊗ Ki ⊗ Airk x = eb, C1 ⊗ M ⊗ Is + ∆t irk

i=1

where Isirk ∈ R is an identity matrix and eb a known vector depending on um and on the right-hand side of (5.7), sirk ×sirk





  b(tm + c1 ∆t) S   X .. T  eb := ∆t  Ci ⊗ Ki ⊗ Airk [um ⊗ 1sirk ] , IN Q ⊗ Airk Pˆ  − . i=1 b(tm + csirk ∆t)

(5.9)

with 1sirk = [1 . . . 1]T ∈ Rsirk . The matrix Pˆ T is such that it permutes the rows of the vector it multiplies so that all variables are grouped in the same order as the unknowns x.

AN ALGEBRAIC MULTIGRID METHOD

111

Remark 5.2.3. In case of a time-dependent stochastic coefficient a(x, t, ω), each of the elements of the stiffness matrices Ki is time-dependent. According to (5.6), every stiffness matrix Ki (t) is to be evaluated at sirk time positions t = tm + cj ∆t, j = 1, . . . , sirk . This leads to a total of (Ssirk ) stiffness matrices at each time step. Applying (5.6) yields the following system for the stage vector increments:

C1 ⊗ M ⊗ Isirk + ∆t S X i=1

" S X i=1

Ci ⊗ Ki (tm + c1 ∆t) ⊗ Airk (:, 1) . . .

# ! Ci ⊗ Ki (tm + csirk ∆t) ⊗ Airk (:, sirk ) Pˆ x = B.

(5.10)

Matrix Pˆ is a (N Qsirk × N Qsirk ) permutation matrix. It permutes the columns of the matrix that it is multiplied with so that consecutive IRK stages are grouped together in blocks of sirk columns. In the remainder of this chapter, a multigrid solver and a convergence analysis is presented for time-independent a(x, ω). The extension to the general case of a(x, t, ω) is straightforward. Remark 5.2.4. In (5.8) the unknowns are ordered block-wise. The vector x consists of Q consecutive blocks, with each block corresponding to the unknowns associated with a random mode. These blocks can further be subdivided into N blocks, where each one contains the IRK unknowns per spatial node. Similar to the discussion in [RS87] on unknown-based and point-wise ordering of variables, the unknowns in (5.8) can be reordered per spatial point. This yields the system ! S X (5.11) Ki ⊗ Ci ⊗ Airk xˆ = ˆb, M ⊗ C1 ⊗ Is + ∆t irk

i=1

with ˆb being the reordered version of eb (5.9). The vector x ˆ contains N blocks, with each block corresponding to the Qsirk unknowns related to a spatial point. This point-based ordering is more convenient to illustrate the block operations of the point-based AMG method presented in Section 5.3.

5.3

An algebraic multigrid method

Solving fully implicit Runge-Kutta discretizations poses even for deterministic problems major computational challenges [HW02, p. 118]. Recently however, efficient multigrid solvers and preconditioners have been proposed for deterministic IRK discretizations [VV05, BVlV09, MNS07]. The AMG solver proposed in [BVlV09] achieves optimal convergence properties w.r.t. the spatial and time discretization parameters. Here, this AMG solver is extended to coupled stochastic Galerkin discretizations.

112


An AMG solution method for (5.8) can be constructed similarly to the multigrid method presented in Section 3.3.2. The AMG components are built so that all unknowns per spatial node are updated together. The standard multigrid algorithm applies with a Kronecker product structure for the smoothing, prolongation and restriction operators.

5.3.1

Smoothing operator

We suggest to use a block lexicographic Gauss-Seidel smoothing method, i.e., an extension of either the basic GS-splitting (3.2) or the Ki -splitting smoother (3.3). In the former case, one smoothing step consists of a loop over all spatial nodes and random unknowns, in which all IRK unknowns per spatial and random unknown are updated simultaneously. Hence, every iteration involves a sequence of N Q linear systems to be solved. The local system at node n and random unknown q corresponds to the sirk × sirk system: M (n, n)C1 (q, q)Isirk + ∆t

S X

Ki (n, n)Ci (q, q)Airk

i=1

−

XX

m6=n

−

M (n, m)C1 (q, t)Isirk + ∆t

t

X

S X

!

x[n,q] = eb[n,q]

Ki (n, m)Ci (q, t)Airk

i=1

M (n, n)C1 (q, t)Isirk + ∆t

S X

Ki (n, n)Ci (q, t)Airk

i=1

t6=q

!

!

x[m,t]

x[n,t] , (5.12)

with x[n,q] ∈ Rsirk ×1 the unknowns associated with node n and random function Ψq . When corollary 2.5.6 holds and C1 ≡ IQ , iteration (5.12) simplifies to (M (n, n)Isirk + ∆tK1 (n, n)Airk ) x[n,q] = eb[n,q]

−

X

M (n, m)C1 (q, q)Isirk x[m,q] + ∆t

S X X t

m6=n

− ∆t

S X X t6=q

Ki (n, m)Ci (q, t)Airk

i=1

i=2

Ki (n, n)Ci (q, t)Airk

!

!

!

x[m,t]

x[n,t] ,

These sirk × sirk systems are typically dense and very small, e.g., 3 × 3, and can be solved directly, see the discussion in [BVlV09].

AN ALGEBRAIC MULTIGRID METHOD

113

In the second case, one smoothing step consists of a loop over all spatial nodes, in which all random and IRK unknowns per node are updated simultaneously. Hence, every iteration involves a sequence of N linear systems to be solved. The local system at node n corresponds to the Qsirk × Qsirk system: M (n, n)C1 ⊗ Isirk + ∆t = eb[n] −

X

m6=n

S X i=1

Ki (n, n)Ci ⊗ Airk

M (n, m)C1 ⊗ Isirk + ∆t

S X i=1

!

x[n] !

Ki (n, m)Ci ⊗ Airk x[m] , (5.13)

with x[n] ∈ RQsirk ×1 being the unknowns associated with node n. The block GaussSeidel Ki -splitting can also be expressed as a linear iteration based on a matrix splitting of the stiffness matrices Ki and the mass matrix M , M = M + + M − :

+

C1 ⊗ M ⊗ Isirk + ∆t = eb −

S X i=1

C1 ⊗ M

Ci ⊗

−

Ki+

⊗ Airk

⊗ Isirk + ∆t

!

S X i=1

xnew

Ci ⊗

Ki−

⊗ Airk

!

xold . (5.14)

Here, Ki+ and M + are the lower triangular parts of Ki and M , respectively. Remark 5.3.1. The block Gauss-Seidel Ki -splitting method (5.14) entails in every iteration the inversion of a block triangular system. The triangular shape of these systems can be visualized by reordering the unknowns according to (5.11). The block Gauss-Seidel (5.14) system matrix can then be formulated as   S X irk Ki (1, 1)Ci 0  M (1, 1)IQsirk +∆t    i=1     . ..  ,   S S   X X  irk irk  M (N, 1)IQsirk +∆t

Ki (N, 1)Ci

...

M (N, N )IQsirk +∆t

i=1

Ki (N, N )Ci

i=1

with Ciirk = Ci ⊗ Airk and C1 replaced by IQ .

5.3.2

Multigrid hierarchy and intergrid transfer operators

As for the steady-state problems discussed in Section 3.3.2, the multigrid hierarchy is derived from the dominant term in (5.8), i.e., from the stiffness matrix of

114


the averaged deterministic problem. Such a hierarchy can be obtained by using a classical AMG strategy applied to the stiffness matrix K1 . Denote by Pkk−1 a prolongation operator constructed for K1 at level k, then Kronecker product prolongation and restriction operators for (5.8) are built as T k Pk−1 = IQ ⊗ Pkk−1 ⊗ Isirk and Rkk−1 = IQ ⊗ Pkk−1 ⊗ Isirk . (5.15)

Note that all random and IRK unknowns associated with a particular spatial node are prolongated and restricted in a decoupled way. Only the spatial domain is coarsened; the stochastic and time discretization is kept unaltered throughout this multigrid hierarchy.

The coarse grid operator at level k − 1, denoted as Ak−1 , is deduced from (5.15) by using the Galerkin principle. This yields Ak−1 = C1 ⊗Pkk−1 Mk Pkk−1

T

⊗Isirk +∆t

S X i=1

Ci ⊗Pkk−1 Ki,k Pkk−1

T

⊗Airk , (5.16)

with Mk and Ki,k the mass and stiffness matrices at multigrid level k, respectively.

5.4

Convergence analysis

Using Fourier analysis [Bra77, TOS01], valuable insights in the convergence behavior of geometric multigrid methods can be obtained. A local Fourier analysis of geometric multigrid for stochastic, stationary PDEs is given in Section 4.5. This analysis cannot be directly applied to AMG methods. Instead, the methodology from [VV05, BVlV09] is followed. A similar multigrid convergence analysis for the case of stationary stochastic problems is detailed in Section 4.2. We first consider AMG with block GS Ki -splitting smoother (5.14) applied to a stochastic discretization (5.8) with S = 2. This also gives insight in the general case, i.e., S > 2, as discussed in Section 4.2.2. When the block GS smoother (5.12) is used, the convergence analysis proceeds analogously, as will be pointed out in remark 5.4.1.

5.4.1

Theoretical asymptotic convergence factor

Convergence analysis of smoother. The error iteration of the block Gauss-Seidel Ki -splitting smoother (5.14) for C1 = IQ and S = 2 is given by (IQ ⊗ M + ⊗ Isirk + ∆t(IQ ⊗ K1+ + C2 ⊗ K2+ ) ⊗ Airk )enew = −(IQ ⊗ M − ⊗ Isirk + ∆t(IQ ⊗ K1− + C2 ⊗ K2− ) ⊗ Airk )eold . (5.17)

CONVERGENCE ANALYSIS

115

Denote the corresponding iteration matrix by SKi ,irk . This matrix can be decoupled by applying the similarity transform VC2 ⊗ IN ⊗ Virk , with Virk resulting from −1 the eigenvalue decomposition Airk = Virk Λirk Virk and VC2 from C2 = VC2 ΛC2 VCT2 . This enables one to express the spectrum of SKi ,irk as σ(SKi ,irk ) =

Q s[ irk [

r=1 q=1

σ SbKi ,irk (λq , ∆tλirk r ) ,

λirk r ∈ σ(Airk ), λq ∈ σ(C2 ),

with SbKi ,irk being the matrix-valued function defined as SbKi ,irk (r, z) := − M + + zK1+ + zrK2+

−1

M − + zK1− + zrK2− .

(5.18)

Thus, the asymptotic convergence factor of lexicographic block Ki -splitting GaussSeidel (5.14) applied to system (5.8) with S = 2 corresponds to ρ(SKi ,irk ) = max max ρ SbKi ,irk (λq , ∆tλirk ) . (5.19) λirk ∈σ(Airk ) λq ∈σ(C2 )

Remark 5.4.1. In the case of the block Gauss-Seidel smoother (5.12), a similar analysis based on the similarity transform IQN ⊗ Virk shows that the asymptotic convergence factor of (5.12) applied to system (5.8) with S = 2 is given by ρ(Sirk ) = max ρ Sbirk (∆tλirk ) , (5.20) λirk ∈σ(Airk )

with Sbirk being the matrix-valued function defined as Sbirk (z) := − IQ ⊗ M L + M D + z(K1L + K1D )

+(C2L + C2D ) ⊗ z(K2L + K2D ) + C2L ⊗ zK2U

−1

IQ ⊗ (M U + zK1U ) + C2D ⊗ zK2U + C2U ⊗ zK2

In this case, the iteration matrix cannot be decoupled along the random dimension. Two-grid convergence analysis. The analysis of the two-level multigrid cycle proceeds similarly. It is based on the matrix-valued function TbKi ,irk defined as ν2 T TbKi ,irk (r, z) := SbKi ,irk (r, z) IN − Pkk−1 Pkk−1 (M + zK1 + zrK2 ) Pkk−1

−1

Pkk−1

T

ν1 (M + zK1 + zrK2 ) SbKi ,irk (r, z) ,

116


with SbKi ,irk (r, z) given by (5.18), and ν1 and ν2 respectively being the number of pre- and postsmoothing steps. Using this matrix function, the asymptotic convergence factor of the two-level multigrid cycle becomes ρ (TKi ,irk ) = max max ρ TbKi ,irk (λq , ∆tλirk ) . (5.21) λirk ∈σ(Airk ) λq ∈σ(C2 )

Similar to the asymptotic convergence factor (4.5) in the stationary case, this value corresponds to the worst case asymptotic convergence factor of multigrid applied to the set of deterministic problems: (M + ∆tλirk K1 + ∆tλirk λq K2 ) x = b

with λirk r ∈ σ(Airk ), λq ∈ σ(C2 ).

These deterministic systems can be derived from backward Euler discretizations with scaled time step ∆tλirk of ODE systems: M

dx + (K1 + λq K2 )x = b. dt

(5.22)

General discretizations with S > 2. In the case of multiple random variables, i.e., S > 2 in (5.8), the iteration operator according to the error equation (5.17) can no longer be decoupled along the stochastic dimension. When a(x, ω) is represented by a linear expansion of random variables, a double orthogonal polynomial chaos, see definition 2.3.7, succeeds in decoupling the iteration matrices SKi ,irk and TKi ,irk . In general, we can argue that the convergence analysis of the case S = 2 provides valuable insights for the general case S > 2, as discussed in Section 4.2.2.

5.4.2

Discussion of the convergence analysis

The convergence analysis of the previous section shows that the matrices K1 and K2 , as well as the eigenvalues of C2 and Airk , determine the AMG convergence, see equations (5.19) and (5.21). The behavior of the eigenvalues of C2 was investigated in Section 4.2.3. This shows that in the case of the isotropic model problem (5.1) the multigrid convergence rate is asymptotically independent of the polynomial order, the polynomial type and the number of random variables. When solving anisotropic PDEs, see for example (4.7), discretized with a large polynomial chaos order, the multigrid convergence rate can deteriorate. This occurs particularly in the case of polynomials with an infinite support, e.g., Hermite polynomials. The AMG convergence behavior with respect to the IRK discretization, i.e., the influence of the eigenvalues of Airk , is detailed for deterministic PDEs in [BVlV09]. Fig. 5.1 illustrates the eigenvalue distribution of Airk for Radau IIA IRK methods (see Table 5.1). When increasing the number of IRK stages, the eigenvalues of

IMPLEMENTATION ASPECTS

117

imaginary part eigenvalues

sirk = 1, 1st order sirk = 2, 3rd order sirk = 3, 5th order

0.2

sirk = 4, 7th order

0.1

sirk = 5, 9th order

0 -0.1 -0.2 0

0.2

0.4 0.6 real part eigenvalues

0.8

1

Figure 5.1: Eigenvalue distribution of Airk for the Radau IIA IRK methods in Table 5.1.

Airk become more clustered around zero. The convergence analysis shows that the Airk eigenvalues act as a scaling factor for the time step of a backward Euler discretization of a related ODE system (5.22). For smaller time steps, the discretized backward Euler system (5.22) becomes more diagonally dominant, resulting in a faster multigrid convergence [TOS01, p.65]. Hence, the multigrid convergence rate will slightly improve for a large number of IRK stages. Asymptotically, the multigrid convergence is independent of the number of IRK stages.

5.5


As also pointed out for the solution of a steady-state diffusion problem in Chapter 3, the effectiveness of an AMG solver depends strongly on the efficiency of its implementation. In this section, an overview is given of some implementation issues that reduce the computation time and memory usage.

5.5.1

Matrix formulation and storage

Reordering the unknowns shows that the Kronecker product formulation (5.8) is mathematically equivalent to the following matrix system: M X(C1 ⊗ Isirk ) + ∆t

S X i=1

e Ki X(Ci ⊗ ATirk ) = B,

(5.23)

where the unknowns x are collected into the multivector X ∈ RN ×Qsirk . Note that the N rows of X equal the N blocks of the unknown vector xˆ in (5.11). This

118


matrix representation enables an easy access of all unknowns per nodal point: they correspond to a row in the matrix X. Such access is frequently needed for the block smoothing, the matrix-vector multiplication in the residual computation, and the block restriction and prolongation. Note also that storing these multivectors in a row-by-row storage format leads to a cache efficient implementation. With one memory access, a whole set of values can be retrieved from memory that will be used in the subsequent operations. As discussed in Section 3.6, the entire system of dimension N Qsirk × N Qsirk is never stored or constructed explicitly. Only the storage of one mass matrix M , S stiffness matrices Ki and S matrices Ci is required; these matrices can be stored in sparse matrix format.

5.5.2

Krylov acceleration

Typically, AMG is used as a preconditioner for a Krylov method. This makes the scheme more robust and often significantly improves the convergence rates. Because of the non-symmetry of matrix Airk , we use BiCGStab [van92] or one of the GMRES variants such as GMRESR [VV94]. The matrix-vector multiplication needed for Krylov methods can be implemented in a cache efficient way by using the row-by-row storage format suggested above. As explained in [BVlV09], the matrix-vector product Y = AX of a sparse matrix A ∈ RN ×N and a multivector X ∈ RN ×Qsirk is implemented as a sequence of three nested loops, where the inner loop runs over the columns of the multivectors instead of over their rows. This results in an optimal reuse of the cache since the data access patters of X and Y match their storage layout.

5.5.3

Block smoothing

A large part of the computation time is spent in the smoothing steps. At each smoothing iteration (5.14), N systems of size Qsirk × Qsirk have to be solved. Similar to the discussion in Section 3.2.2, various solution approaches for these block systems can be compared. Fig. 5.2 shows the average computation time of several solution approaches to solve one local system (5.13). The model problem (5.1) is solved on a unit square domain, where the diffusion coefficient a(x, ω) is represented by a linear expansion with uniformly distributed random variables. This yields a Legendre chaos representation for the solution in (2.21). Radau IIA IRK methods are applied. The considered iterative solvers include an LU solver without pivoting, a sparse LU solver (UMFPACK [Dav04] and SuperLU [DEG+ 99]) and (unpreconditioned) BiCGStab. The tests were performed on an Intel Dual Core 2.0 GHz processor with 2GByte RAM.


ALTERNATIVE SOLUTION APPROACHES

10

0

119

Gauss elimination BiCGStab Sparse LU (umfpack)

10

10

-2

Sparse LU (SuperLU)

-4

2 1 10 10 Dimension block system (Qsirk )

10

3

Figure 5.2: Average computation time to solve one local system (5.13).

Solving the local systems with SuperLU yields generally the lowest solution times. Note that the Airk -matrices, which are dense matrices, see Table 5.1, increase the density of the local systems (5.13) in comparison to the local systems (3.4) for the corresponding steady-state problems. These Airk -matrices are unsymmetric so that BiCGStab is applied as Krylov solver. The local matrices have clustered, complex eigenvalues and a condition number typically of order O(10). The results in Fig. 5.2 correspond to Ci -matrices defined as hξi−1 ΨΨT i. In the case of a lognormal diffusion field, the Ci -matrices are given by hΨi ΨΨT i so that the local systems (5.13) become entirely dense. This results in a substantially higher computational cost of the Ki -splitting block smoother.

5.6

Alternative solution approaches

Iterative solvers for stochastic Galerkin finite element discretizations of a steadystate diffusion problem (2.25) can be extended to solvers for SGFEM-IRK discretizations of the time-dependent diffusion problem (5.1). A multigrid solution method based on a spatial coarsening is already presented in Section 5.3. In this section, a variant on Gauss-Seidel Ci -splitting iterations (3.5) and the mean-based preconditioner from Section 3.5 is introduced. A mean-based preconditioner for SFEM-IRK discretizations corresponds to IQ ⊗ (M ⊗ Isirk + ∆tK1 ⊗ Airk ),

(5.24)

where each of the Q block systems can also be approximated by an AMG cycle, as detailed in [BVlV09].

120


A Gauss-Seidel Ci -splitting iteration applied to (5.8) is given by

C1+

⊗ M ⊗ Isirk + ∆t

S X i=1

eb −

Ci+

Ci+

⊗ Ki ⊗ Airk

!

xnew =

C1− ⊗ M ⊗ Isirk + ∆t

S X i=1

Ci− ⊗ Ki ⊗ Airk

!

xold ,

where corresponds to the lower triangular part of Ci . For each of the Q stochastic unknowns, a local N sirk × N sirk system has to be solved: M ⊗Isirk +∆t

S X i=1

Ci (q, q)Ki ⊗Airk x[q] = b[q] −∆t

S XX c6=q i=2

Ci (q, c)Ki ⊗Airk x[q] x[c] ,

with x[q] ∈ RN sirk ×1 the unknowns associated with stochastic basis function Ψq and matrix C1 replaced by IQ . These systems can be solved by applying the AMG solution approach for deterministic IRK discretizations presented in [BVlV09]. Since the Ci -splitting preconditioners do not approximate Airk , the convergence behavior w.r.t. the time discretization is inherited from the AMG solver [BVlV09] for the (N sirk × N sirk )-block system. That is, the convergence behavior is similar to the convergence of a Ci -splitting iteration applied to a suitably scaled backward Euler discretization of the same problem (5.1). The influence of spatial and stochastic discretization parameters on the convergence rate can be deduced from the convergence analysis of the corresponding steady-state problem, see the numerical results in Section 3.7 and the convergence analysis in Section 4.4.3.

5.7

Numerical results

This section illustrates the convergence properties of the AMG method presented in Section 5.3 and compares this method to the Ci -splitting preconditioners proposed in Section 5.6. In all numerical tests, the iterations proceed until the Euclidean norm of the relative residual, krk/kbk, is smaller than 10−10 . The tests were performed on a quad-core Xeon 5420 CPU with 2.5 GHz and 8GByte of RAM.

5.7.1

Test case 1: diffusion problem on square domain

In this section, the time-dependent version of the test problem in Section 3.7.1 is considered. This corresponds to solving (5.1) on a unit square domain D = [0, 1]2 with three homogeneous Dirichlet boundaries, one zero Neumann boundary and

NUMERICAL RESULTS

Spatial nodes N 20 751 50 720 (sirk × sirk )-block smoother (5.12) stand-alone F (1,1) ag 31 34 au 29 32 BICGSTAB-V (1,1) ag 21 25 au 20 25 BICGSTAB-F (1,1) ag 17 19 au 17 18 BICGSTAB-W (1,1) ag 17 19 au 17 18 (Qsirk × Qsirk )-block smoother (5.14) stand-alone F (1,1) ag 23 26 au 23 26 BICGSTAB-V (1,1) ag 20 25 au 20 25 BICGSTAB-F (1,1) ag 13 16 au 13 16 BICGSTAB-W (1,1) ag 13 15 au 13 15

121

114 313

257 986

357 393

34 32 28 28 19 19 19 18

37 35 33 33 20 20 20 19

38 36 34 33 21 20 21 19

27 27 26 27 16 16 15 15

30 30 33 33 16 17 18 18

31 31 34 35 17 17 16 16

Table 5.2: Multigrid iteration counts for solving one time step of (5.1) on D = [0, 1]2 . The configuration of the stochastic diffusion coefficient is given in Table 3.3. Applying a second-order chaos based on 5 random variables results in Q = 21. A three-stage (sirk = 3) Radau IIA IRK time discretization is applied with time step ∆t = 0.01.

deterministic forcing term f (x) = 1. A deterministic initial solution u(x, t = 0, ω) = 0 is applied. Choice of multigrid type. Table 5.2 shows multigrid iteration counts when solving (5.1) with different cycle types, i.e., V , W , or F -cycles. The corresponding solution times are presented in Fig. 5.3. The stochastic diffusion coefficient is only spatially dependent, not time-dependent, and its configuration is given in Table 3.3. A selection of spatial discretization sizes is tested in order to illustrate the asymptotic independence of (W -cycle) multigrid on the spatial discretization size. Regardless of the type of block smoother, BiCGStab preconditioned by a multigrid F -cycle yields generally the lowest solution times. Note also that multigrid with Ki -splitting block smoother (5.14) is computationally more expensive than multigrid with (sirk × sirk ) block smoother (5.12). The same conclusions apply in case of the random field alog . An example is given in Table 5.3. BiCGStab preconditioned by a multigrid F -cycle based on the (sirk × sirk )-block smoother (5.12) results in the lowest solution time.

122


smoother (5.12), F(1,1)-AMG


6000

smoother (5.14), F(1,1)-AMG 4000

smoother (5.12), V(1,1)-BiCGStab-AMG smoother (5.14), V(1,1)-BiCGStab-AMG

2000

smoother (5.12), F(1,1)-BiCGStab-AMG smoother (5.14), F(1,1)-BiCGStab-AMG

0 0

2 0.5 1 1.5 number of unknowns (N Qs)× 10 7

smoother (5.12), W(1,1)-BiCGStab-AMG smoother (5.14), W(1,1)-BiCGStab-AMG

Figure 5.3: Computing time in seconds for solving one time step of (5.1) with Gaussian diffusion coefficient ag on a unit square domain, discretized with a second-order Hermite expansion and a Radau IIA IRK time discretization with sirk = 3 and ∆t = 0.01. The multigrid iteration counts are given in Table 5.2.

V (1, 1)-cycle (sirk × sirk )-block smoother (5.12) stand-alone 58 ( 7407 ) with BiCGStab 28 ( 3588 ) (Qsirk × Qsirk )-block smoother (5.14) stand-alone 56 ( 9645 ) with BiCGStab 30 ( 5195 )

F (1, 1)-cycle

W (1, 1)-cycle

34 ( 5977 ) 18 ( 3172 )

34 ( 6757 ) 17 ( 3397 )

29 ( 6359 ) 16 ( 3519 )

28 ( 6891 ) 15 ( 3703 )

Table 5.3: Multigrid iteration counts and total solution time1 in sec. between brackets for solving one time step of (5.1) with lognormal diffusion coefficient alog on D = [0, 1]2 . Applying a second-order Hermite chaos based on 5 random variables results in Q = 21. A Radau IIA IRK time discretization is used with sirk = 3 and ∆t = 0.01. Combined with 50 720 spatial unknowns, the problem contains 3.195 · 106 unknowns.

Influence of time discretization. The influence of the number of IRK stages on the AMG convergence is illustrated in Table 5.4. The results are presented for a number of stages sirk , increasing from 1 up to 6. The first corresponds to a firstorder method, while the later leads to a time integration scheme of order 11. The convergence analysis predicts a slightly increased AMG convergence rate when the number of IRK stages is increased, see the discussion in Section 5.4.2. This effect is visible in the numerical results. Comparison to other solvers. Section 5.6 describes Ci -splitting based methods for solving stochastic Galerkin discretizations of time-dependent problems. For steady-state problems, such Ci -splitting based methods can yield substantial com1 These

RAM.

experiments were performed on a 2.8Ghz quad-core Xeon 5560 CPU with 24GB of

NUMERICAL RESULTS

123

Time discretization order IRK stages sirk (sirk × sirk )-block smoother stand-alone with BiCGStab

1 1

3 2

5 3

7 4

9 5

11 6

33 19

31 18

31 17

31 17

30 17

30 17

(Qsirk × Qsirk )-block smoother stand-alone with BiCGStab

26 15

24 14

23 13

23 13

23 13

23 13

Table 5.4: The number of iterations per time step required to solve (5.1) with Gaussian diffusion coefficient ag on D = [0, 1]2 , using F (1, 1)-AMG cycles as stand-alone solver, or as preconditioner for BiCGStab. The discretization is based on a finite element mesh with 20 751 nodes, a second-order Hermite chaos with L = 5, corresponding to Q = 21, and a Radau IIA IRK scheme with ∆t = 0.01.

BiCGStab with default N = 257 986 P =5 L = 20 s=5 ∆t = 0.01

multigrid (5.12) 18 ( 58.2 ) 24 ( 998 ) 22 (1004) 20 (1783) 19 ( 148 ) 17 ( 54.2 )

multigrid (5.14) 16 ( 91.5 ) 20 ( 1439 ) 16 (23207) 16 (17026) 16 ( 292 ) 14 ( 79.7 )

mean-based Ci -splitting 31 ( 37.2 ) breakdown 46 ( 633 ) 34 ( 688 ) 29 ( 88.2 ) breakdown

Gauss-Seidel Ci -splitting 23 ( 55.9 ) 24 ( 775 ) 40 ( 1259 ) 30 ( 1811 ) 24 ( 143 ) breakdown

Table 5.5: Number of iterations (solution time in sec.) required to solve one time step of (5.1) with diffusion coefficient au (x, ω) on D = [0, 1]2 . A Legendre chaos is used for the stochastic discretization and a Radau IIA IRK time discretization. Multigrid applies either the (sirk × sirk ) block smoother (5.12) or the (Qsirk × Qsirk ) block smoother (5.14) in a F(1,1)-cycle configuration. The default configuration corresponds to N = 20 751, L = 5, P = 2, sirk = 2, ∆t = 0.1. In each row, one of these parameters is modified.

putational time savings compared to the multigrid solvers for small, regular problems, see, e.g., the results in Fig. 3.7. In order to check whether such differences in computational time also appear for time-dependent problems, Table 5.5 presents the number of iteration steps and computational time required to solve a time-dependent model problem with a selection of iterative solvers. The results show that multigrid with (sirk × sirk ) block smoother (5.12) requires a comparable amount of solution time as a Gauss-Seidel Ci -splitting preconditioning. The mean-based preconditioner is slightly cheaper, but the difference in computational time is not as pronounced as for the steady-state problems. The Ci -splitting preconditioners suffer in some cases from a BiCGStab breakdown.

124


V1

V2

Vdielectric

V3

εtape εiso εdielectric

boundary conditions [V] Vdielectric 0 V1 0.087 V2 0.819 V3 −0.906 electric permittivity [F/m] √ εdielectric ε0 (1 + √3/10ξ1 ) εiso 24ε0 (1 + √3/30ξ2 ) εtape 6ε0 (1 + 3/20ξ3 )

Figure 5.4: Configuration of a 2D stochastic transient potential problem.√The √ random variables ξ1 , ξ2 , ξ3 are independent and each uniformly distributed on [− 3, 3]. The permittivity of the free space is given by ε0 = 8.85 · 10−12 F/m.

5.7.2

Test case 2: electric potential problem

As a second test case, we consider a transient potential equation: α

∂V (x, t, ω) − ∇ · (ε(x, ω)∇V (x, t, ω)) = 0. ∂t

(5.25)

Here, V denotes the electric potential and ε the electric permittivity. Equation (5.25) represents a pseudo-time stepping equation for the electric potential, with parameter α = 1 F/(m3 s). No charge density is present. The domain, the boundary conditions and the setup for ε are presented in Fig. 5.4. The model represents a three-phase cable, with four constant potentials along the outer and inner boundaries. The permittivity is expressed as a piecewise constant random field, corresponding to the different material regions of the cable. The stochastic PDE (5.25) models the effect of deviations in permittivity on the resulting electric potential as a function of space and time. Fig. 5.5 shows the mean value and variance of the electric potential at several instances in time. Applying AMG results in similar convergence properties as for the first test problem. An illustration of the convergence history as a function of the iteration index is given in Fig. 5.6. Observe that the use of GMRESR [VV94] results in a more robust convergence behavior than BiCGStab. This is typically also the case for deterministic parabolic PDEs. To limit the memory requirements of GMRESR, the method is restarted every five iterations.

NUMERICAL RESULTS

125

Mean at t = 0.1 -0.0436

-0.906

0.819

Variance at t = 0.1 0.000529 0.00106

0

Mean at t = 0.5 -0.0436

-0.906

0

0.819

Variance at t = 0.5 0.000934 0.00187

-0.906

0

Mean at t = 2 -0.0436

0.819

Variance at t = 2 0.000999

0.002

Figure 5.5: Mean and variance of the solution of Eq. (5.25). The configuration of Fig. 5.4 is used with a three-stage Radau IIA IRK discretization and time step 0.05. The stochastic discretization is based on a second-order Legendre chaos. The electric potential is zero initially.

BiCGStab – AMG (5.12)

relative residual

GMRESR – AMG (5.12) BiCGStab – AMG (5.14) GMRESR – AMG (5.14)

−5

10

−10

10

BiCGStab – mean prec.

0

10

GMRESR – mean prec. BiCGStab – GS-Ci

relative residual

0

10

GMRESR – GS-Ci −5

10

−10

10

−15

10

0

10 20 iterations

30

0

10 20 iterations

30

Figure 5.6: Residual norms as a function of the number of iterations when solving equation (5.25) on the domain illustrated in Fig. 5.4, discretized with 139 632 nodes. A three-stage Radau IIA IRK discretization with time step ∆t = 0.1 is used together with a second-order Legendre chaos resulting in a total of 4.2 × 106 unknowns.

126

5.8


Conclusions

We have constructed and analyzed an AMG method for stochastic Galerkin finite element discretizations of time-dependent, stochastic PDEs. This work extends previous research on multigrid for stochastic finite element problems [LKD+ 03, EF07] towards unstructured finite element meshes and high-order time discretizations. The presented AMG method has very favorable convergence properties with respect to the spatial, random and time discretization. We also constructed alternative preconditioning approaches based on Ci -splitting iterations, which have less optimal convergence properties but may require less computational time in practice.

Chapter 6

A stochastic bio-engineering application 6.1

Introduction

In order to optimize fruit production and storage, a good knowledge of the growth and ripening process is essential. This process can be modelled by a set of timedependent reaction-diffusion problems which simulate, for example, the conversion of starch into sugars in growing fruit. The accuracy of the simulation results depends on the accuracy of a large set of parameters that model the chemical composition and shape of the fruit under consideration. Many of these parameters are typically inherently variable; hence a stochastic simulation is needed. Laboratory tests indicate that the concentration of starch in growing apples is characterized by complex patterns. This spatio-temporal starch patterning is believed to contain crucial information on the quality of apples during development, maturation and ripening. In order to simulate such patterns, a set of time-dependent nonlinear reaction-diffusion problems is to be solved. This chapter introduces the stochastic Galerkin discretization of systems of nonlinear, stochastic PDEs. Uncertainty typically plays an important role in the simulation of nonlinear dynamical systems. It is widely known that the behavior of a nonlinear dynamical system is highly sensitive to small variations [WB09]. This encouraged the study of nonlinear, stochastic PDEs, see for example [SWLB09, WB08c, BBCF07]. Several issues may however arise when solving nonlinear, stochastic PDEs with a stochastic Galerkin finite element method. First, the evaluation of the stochastic Galerkin projection step (2.22) may require numerical integration [MK05]. The

127

128

A STOCHASTIC BIO-ENGINEERING APPLICATION

problems considered here however only involve polynomial reaction terms, which can be directly represented by a polynomial chaos expansion [Mat08]. Secondly, the presence of bifurcations or a loss of smoothness with respect to the random data limits the accuracy of a global spectral stochastic representation of the solution. A discontinuity in probability space can lead to a stochastic Gibbs phenomenon resulting in unphysical oscillations of the solution [WLB09]. This problem can be resolved by applying multi-element stochastic approximation schemes, for example a multi-wavelet stochastic discretization [LKNG04, LNGK04] or the unsteady adaptive stochastic finite element approach introduced in [WB08d]. In this work, we will consider a range of uncertain input parameters so that no bifurcations are present. This justifies the use of global polynomial approximations. In a next step, the method could be extended to multi-element gPC approximations in order to investigate the stochastic bifurcation behavior. This chapter is organized as follows. In Section 6.2, the stochastic model equations are introduced. Section 6.3 describes the stochastic Galerkin finite element discretization and focusses on the difficulties that arise from discretizing a system of nonlinear PDEs. In Section 6.4, an algebraic multigrid solution method is proposed. Implementation issues are discussed in Section 6.5. The performance of the multigrid algorithm is numerically verified in Section 6.6, and the statistics of the concentration of chemical particles in a growing apple are given.

6.2

Model description

We consider the reaction and diffusion of chemical particles in a growing domain D with boundary ∂D. The 3D domain represents an apple; the 3D geometry, a horizontal and vertical cut, and the wire frame model are illustrated in Fig. 6.1. The model consists of four different subdomains. The small black spots represent the seeds, typically 5 to 10 seeds are present in the apple. The seeds are positioned inside the ovary of the apple. The ovary is part of the core or pith of receptacle. The large remaining part of the apple up to the skin of the apple is the cortex of receptacle, i.e., the fleshy tissue or apple flesh. These four different regions are characterized by different material properties. The boundary of the domain corresponds to the skin of the apple.

6.2.1

Reaction and diffusion on a growing domain

Consider the concentration u(x, t) of a chemical particle, whose reaction kinetics are generally described by a polynomial or rational function in u, denoted by R(u).

MODEL DESCRIPTION

129

Y Z X

Figure 6.1: Computer model of an apple; vertical and horizontal cut of the apple model.

On a fixed domain, the standard reaction-diffusion equation for u is given by ∂u(x, t) = ∇ · (a(x)∇u(x, t)) + R(u(x, t)), ∂t

(6.1)

where a = a(x) is the diffusion coefficient. Appropriate boundary and initial conditions complete the model. On a growing domain D(t), two new terms are to be introduced in (6.1). The first one expresses the transport, i.e., drift, of the chemical particles within the volume as the volume moves due to growth. This transport is represented by a convection term. The second one models the dilution due to the local volume increase. The dilution term models the decrease of the local concentration as the containing volume increases. We consider uniform domain growth. The essential characteristic of uniform growth is that any two points of the volume move apart with a relative velocity which depends only on the separation and is independent of the spatial position, i.e., which is a function of time only. As a result, the evolution of the boundary of an apple can be described by r0 r(t), with r0 the initial radius of the domain and r(t) a growth function.

130


In order to describe the reaction-diffusion equation in a fixed domain D, a reference coordinate system for the growing domain is chosen. The origin of the coordinate system corresponds to a reference point which remains fixed while the domain grows. For the apple domain of Fig. 6.1, this reference point coincides with the center of the apple. The coordinates (x, t) of the fixed domain correspond then to the following scaling, x ,t , (x, t) → (x, t) = r0 r(t) with (x, t) the coordinate system in which the growing domain is represented. In case of uniform domain growth, the convection term due to the drift of particles vanishes. Representing problem (6.1) in the reference domain D and taking the additional dilution term into account yields a transformed reaction-diffusion equation for x ∈ D, ′ 1 ∂u(x, t) 3r (t) u(x, t), (6.2) ∇ · (a(x)∇u(x, = 2 t)) + R(u(x, t)) − 2 ∂t r0 r(t) r(t) where r′ (t) corresponds to

dr (t). dt

Growth function for apples. Experiments have shown that the mass of apples follows a logistic growth curve with maximum growth rate α. Assuming a constant apple density, this logistic trend also applies to the volume of an apple. Based on the relationship between the volume and radius of a spherical object, the evolution of the boundary, r0 r(t), of an apple can be deduced. The growth function r(t) is then given by r(t) =

1 ((1 −

β 3 ) exp(−αt)

1

+ β3) 3

,

(6.3)

0 , r0 the initial apple radius, i.e., radius at time t = 0, and r∞ the with β = rr∞ final, stationary, apple radius.

Reaction kinetics. The reaction R(u) is modelled with reaction coefficient κ as R(u) = −κu(x, t)(1 − u(x, t)).

(6.4)

Boundary conditions. The chemical particles modelled by u(x, t) cannot leave the apple. This no-flux boundary yields the Neumann boundary condition: ~n · ∇u(x, t) = 0

∀x ∈ ∂D.

MODEL DESCRIPTION

Initial condition. At time t = 0, a particle is released from the seeds: 1/ns x ∈ Dseeds u(x, 0) = , x∈ / Dseeds 0

131

(6.5)

with ns the number of seeds present in the apple.

6.2.2

Two-particle model

An extended model of the reaction kinetics in an apple considers the concentration of two different types of particles, i.e., u(x, t) and v(x, t). Transforming (6.2) into a two-particle model yields for x ∈ D,  ∂u  1  (x, t) = r2 r(t)  2 ∇ · (au (x)∇u(x, t) + Ru (u(x, t), v(x, t))  0  ∂t   3 dr  − r(t) dt u(x, t) . (6.6) ∂v  1  t) = ∇ · (a (x)∇v(x, t)) + R (u(x, t), v(x, t)) (x, 2  v v 2 r0 r(t)  ∂t    3 dr  − r(t) dt v(x, t)

The Gray-Scott model represents the following cubic auto-catalytic chemical reaction between chemicals U and V : U + 2V → 3V V → W, where W is an inert product. These two reactions occur at different rates throughout the volume according to the relative concentrations at each point. The reaction kinetics Ru and Rv respectively correspond to Ru (u(x, t), v(x, t)) = −κA u(x, t)v(x, t)2 + κf (1 − u(x, t))

(6.7)

Rv (u(x, t), v(x, t)) = κA u(x, t)v(x, t)2 − κ2 v(x, t),

(6.8)

where κf corresponds to the feed rate of chemical U , κ2 is the reaction rate of the second reaction and κA a dimensionless rate constant of the first reaction. We will also consider a competing species reaction model, which has a quadratic instead of a cubic nonlinearity. This reaction is characterized by: Ru (u(x, t), v(x, t)) = ǫ1 u(x, t) − ς1 u(x, t)2 − χ1 u(x, t)v(x, t)

(6.9)

Rv (u(x, t), v(x, t)) = ǫ2 v(x, t) − ς2 v(x, t)2 − χ2 u(x, t)v(x, t),

(6.10)

where ǫ1 , ǫ2 , ς1 , ς2 , χ1 and χ2 are positive reaction constants. Remark 6.2.1. The competing species reaction is considered in order to validate the developed AMG solver on different reaction problems, with a different number of random variables.

132

6.2.3


Stochastic model

The reaction-diffusion models (6.2) and (6.6) depend on several parameters, i.e., material parameters describing the diffusion process, reaction parameters and parameters modelling the growth of the domain. Some of these parameters are not known exactly or can vary depending on the position, e.g., due to the heterogeneity of the tissue. We model this uncertainty by introducing random variables into our model. For example, a stochastic diffusion coefficient a(x, ξ) may contain four random variables, ξ1 , . . . , ξ4 :  x ∈ D1   a1 ξ1  a2 ξ2 x ∈ D2 , (6.11) a(x, ξ) = x ∈ D3 a ξ  3 3   x ∈ D4 a4 ξ4

where the domains D1 , D2 , D 3 and D4 respectively correspond to the seeds, ovary, core and fleshy tissue parts of domain D. All random variables are assumed to be independent and are grouped together in a random vector ξ. The joint probability density of ξ is denoted by ̺(y), with y ∈ Γ and Γ the support of ̺. For the one-particle model (6.2), the stochastic model becomes, 1 3r′ (t) ∂u ξ)∇u(x, t, ξ))+R(u(x, t, ξ))− ∇·(a(x, (x, t, ξ) = 2 u(x, t, ξ), (6.12) ∂t r0 r(t)2 r(t) with R(u(x, t, ξ)) = κ(1 + ξκ )u(x, t, ξ)(1 − u(x, t, ξ)). The initial condition and boundary conditions are assumed to be deterministic. The two-particle model (6.6) can analogously be extended to a set of stochastic partial differential equations.

6.3

Stochastic finite element discretization

This section describes the stochastic Galerkin finite element discretization of the nonlinear stochastic PDE (6.12), and of the stochastic variant of the two-particle model (6.6). First, we focus on the discretization of the nonlinear reaction term; the SGFEM discretization of the time-dependent diffusion part corresponds to the SGFEM-IRK discretization presented in Section 5.2.2, where the diffusion operator is now time-dependent due to the growth r(t) of the domain. Next, we consider the stochastic Galerkin discretization of a system of stochastic PDEs.

6.3.1

Nonlinear stochastic partial differential equations

The nonlinear stochastic PDE (6.12) can be linearized first, i.e., by applying Newton’s method, after which a stochastic Galerkin finite element discretization of

STOCHASTIC FINITE ELEMENT DISCRETIZATION

133

the Jacobian can be constructed, see [LSK06] for an example. Alternatively, the construction of the Jacobian matrix can be avoided by applying a quasi-Newton strategy as in [MK05]. We apply a Newton linearization since a Jacobian can easily be constructed for polynomial nonlinearities as in (6.4) or (6.7)–(6.8). Newton linearization Newton’s method starts from an initial guess u(0) and iteratively updates this approximation by solving a linearized problem at every Newton step. In order to construct a linearized model for (6.12), we start from the weak formulation of (6.12). That is, find u ∈ L2 (0, Tf ; H 1 (D)) ⊗ L2 (Γ) such that u = u0 at t = 0 and Z Z Z Z 3r′ (t) ∂u(x, t, y) u(x, t, y)v(x, y)̺(y)dxdy v(x, y)̺(y)dxdy + ∂t r(t) Γ D Γ D Z Z 1 a(x, y)∇u(x, t, y)∇v(x, y)̺(y)dxdy + 2 r0 r(t)2 Γ D Z Z κ(1 + yκ )u(x, t, y)v(x, y)̺(y)dxdy − D

Γ

+

ZZ

κ(1 + yκ )u(x, t, y)2 v(x, y)̺(y)dxdy = 0 ∀v ∈ H01 (D) ⊗ L2 (Γ).

(6.13)

Γ D

R R R R From now on, the integral Γ D ·̺(y)dxdy will be denoted shortly Γ D ·. The Hilbert space L2 (0, Tf ; H 1 (D)), with [0, Tf ] the time interval on which (6.12) holds, is defined in [NT09]. The left-hand side of (6.13) is the nonlinear residual associ(k) ated with the weak formulation and is denoted by F (u). For u = u(k) + δu in (k) (6.13), we have that the corrections δu ∈ L2 (0, Tf ; H01 (D)) ⊗ L2 (Γ) satisfy Z Z

Z Z (k) 1 ∂δu (x, t, y) a(x, y)∇δu(k) (x, t, y)∇v(x, y) v(x, y) + 2 ∂t r0 r(t)2 Γ D Γ D Z Z ′ 3r (t) + − κ(1 + yκ ) δu(k) (x, t, y)v(x, y) r(t) Γ D Z Z κ(1 + yκ )2δu(k) (x, t, y)u(k) (x, t, y)v(x, y) + (6.14) Γ

+

Z Z

D

2 κ(1 + yκ ) δu(k) (x, t, y) v(x, y) = −F (u(k) )

Γ D

∀v ∈ H01 (D) ⊗ L2 (Γ).

Following the approach in [ESW05, p.325], we obtain the linearized problem by (k) dropping the quadratic term (δu )2 in equation (6.14). This yields: find δu(k) ∈

134


L2 (0, Tf ; H01 (D)) ⊗ L2 (Γ) such that ∀v ∈ H01 (D) ⊗ L2 (Γ) Z Z

Z Z (k) ∂δu (x, t, y) 1 a(x, y)∇δu(k) (x, t, y)∇v(x, y) v(x, y) + 2 ∂t r0 r(t)2 Γ D Γ D Z Z ′ 3r (t) − κ(1 + yκ ) δu(k) (x, t, y)v(x, y) + (6.15) r(t) Γ D Z Z 2κ(1 + yκ )δu(k) (x, t, y)u(k) (x, t, y)v(x, y) = −F (u(k) ). + D

Γ

The solution to (6.15) is called the Newton correction. At Newton step k + 1, (k) the current approximation u(k) to the solution is updated with δu in order to (k) compute a new approximation u(k+1) , i.e., u(k+1) = u(k) + δu . Stochastic Galerkin finite element discretization (k)

At every Newton step, a solution δu to (6.15) needs to be computed. To that end, a stochastic Galerkin finite element discretization, combined with an implicit Runge-Kutta time discretization is applied to (6.15). The construction of the algebraic system proceeds analogously to the discussion in Section 5.2.2. Here, we detail the discretization of the last term in the left-hand side of (6.15). After ap(k) proximating the solution δu and the current approximation u(k) by a generalized polynomial chaos expansion, given by (2.21), we have Z Z

2κ(1 + yκ )

= 2κ

Q Q X X

Γ

D

Q X q=1

i=1 q=1

!

Q X

(k) δu,q (x, t)Ψq (y)

(hΨq Ψi Ψj i + hξκ Ψq Ψi Ψj i)

i=1

Z

D

!

(k) ui (x, t)Ψi (y)

ve(x)Ψj (y)

(k)

(k) (x, t)e v (x)dx, (6.16) ui (x, t)δu,q

(k)

(k)

where the trial function v(x, y) in (6.13) is chosen equal to ve(x)Ψj (y); δu,q and ui (k) represent the gPC coefficientsR of δu and u(k) , respectively. The scalar hξκ Ψq Ψi Ψj i corresponds to the integral Γ yκ Ψq (y)Ψi (y)Ψj (y)̺(y)dy. The evaluation of the product term hΨq Ψi Ψj i has been detailed in Section 2.5.1. Higher-order product terms like hξκ Ψq Ψi Ψj i can be evaluated by applying multiple linearizationof-products rules. Alternatively, a pseudo-spectral computation of higher-order product terms can be applied, see [DNP+ 04, TLNE09]. Next, a spatial discretization with finite element nodal basis functions sn (x) is applied to (6.16). This enables one to rewrite the last term in the left-hand side


135

of (6.15) in matrix notation as 2κ

Q X i=1

(k)

bi ) ⊗ M (t)δ (k) (t), (C i + C u u,i

(6.17)

ci = hξκ Ψi ΨΨT i ∈ RQ×Q . The (N Q × 1)-vector δu(k) (t) with C i = hΨi ΨΨT i, C (k) (k) contains the discretized Newton update δu (x, t, y); each component of δu (t) (k) (k) corresponds to a time-dependent δu,q,n (t)-function, as in (2.32). In (6.17), Mu,i (t) is a time-dependent (N × N )-matrix, whose elements are defined as Z (k) (k) m, n ∈ {1, . . . , N }. (6.18) ui (x, t)sm (x)sn (x)dx Mu,i (t)(m, n) := D

The time discretization of (6.17) is performed by an implicit Runge-Kutta method. This introduces at every time tm , sirk additional unknowns, called stage vector increments. Hence, at every time step m and Newton step k, a system of N Qsirk unknowns results. We group the unknowns together in a long vector x, where the (k) stage vector increments of δu (tm ) are numbered first along the random dimension, then along the spatial dimension and finally according to the IRK stages. Taking remark 5.2.3 into account, we have that matrix (6.17) is represented in the implicit Runge-Kutta equation (5.6) for the IRK stage vector increments by

2κ∆t

"Q X i=1

bi ) ⊗ M (k)(c1 ) ⊗ Airk (:, 1) (C i + C u,i ...

Q X i=1

bi ) ⊗ (C i + C

(k)(cs ) Mu,i irk

#

⊗ Airk (:, sirk ) Pˆ x,

(6.19)

where Pˆ ∈ RN Qsirk ×N Qsirk permutes the columns of the matrix that it is multiplied with so that consecutive IRK stages are grouped together in blocks of sirk columns. (k)(c ) Matrix Mu,i j is defined by (6.18), evaluated for stage vector increment xi,j (x), corresponding to random unknown i and IRK stage j, at t = tz + cj ∆t: Z (k)(cj ) (k) (6.20) (ui (x, tz + cj ∆t) + xi,j (x))sm (x)sn (x)dx, Mu,i (m, n) := D

(k)

with m, n ∈ {1, . . . , N }. The replacement of ui follows directly from the IRK equation (5.6).

(k)

in (6.18) by ui

+ xi,j in (6.20)

Linear algebraic system The linearized system (6.15) is thus discretized by a combination of a spatial finite element discretization, an implicit Runge-Kutta time discretization and a

136


stochastic Galerkin discretization. In every Newton step k and every time step m, a system of N Qsirk unknowns needs to be solved. Using the notation of (5.10), this system is given by h ∆t X C ⊗ K ⊗ i i r02 S

C1 ⊗ M ⊗ Isirk + +3∆tC1 ⊗ M ⊗

h

...

i=1

...

i=1

r ′ (tm +c1 ∆t) r(tm +c1 ∆t) Airk (:, 1)

− κ∆t(C1 + Cκ ) ⊗ M ⊗ Airk + 2κ∆t Q X

Airk (:,1) r(tm +c1 ∆t)2

bi ) ⊗ (C i + C

(k)(cs ) Mu,i irk

...

" Q X i=1

Airk (:,sirk ) r(tm +csirk ∆t)2

r ′ (tm +csirk ∆t) r(tm +csirk ∆t) Airk (:, sirk )

i

i

bi ) ⊗ M (k)(c1 ) ⊗Airk (:, 1) (C i + C u,i

# ! ⊗ Airk (:, sirk ) Pˆ x = b.

(6.21)

Matrix C1 equals a (Q × Q)-identity matrix, when the functions Ψq are normalized. The matrix Ci ∈ RQ×Q depends on the discrete representation (2.28) of the random field a(x, ξ), see (6.11), and is given by hϕi (ξ)ΨΨT i. Matrix Cκ ∈ RQ×Q is defined as hξκ ΨΨT i. The first term in (6.21) is the discretization of the timederivative. The second term represents the discretization of the diffusion operator. The third term corresponds to the discretization of the deterministic dilution term that models the domain growth. The remaining terms represent the linearized reaction term. The vectors x, b ∈ RN Qsirk ×1 denote respectively the unknown IRK stage vector increments and the discretized right-hand side. The Newton update (k) δu at time tm+1 and Newton step k + 1 is obtained by evaluating (5.5).

6.3.2

Systems of stochastic partial differential equations

Applying a stochastic Galerkin finite element discretization to a system of stochastic PDEs does not introduce additional difficulties in comparison to a SGFEM discretization of a single stochastic PDE. The weak formulation (6.13) can be straightforwardly extended to a weak formulation of the two-particle model (6.6). An analogous linearization procedure results then in a linearized system, which, after a SGFEM-IRK discretization, can be formulated as a system of algebraic equations. We will assume that the stochastic diffusion coefficients au (x, ξ) and av (x, ξ) are discretized by an expansion of the form (2.28), with respectively Su and Sv terms and with ϕu,1 (ξ) = ϕv,1 (ξ) = 1. These expansions correspond to a set of stiffness matrices, Ku,i and Kv,i ∈ RN ×N .


137

Competing species model The competing species reaction (6.9)–(6.10) contains quadratic nonlinear terms, similar to the one-particle reaction model (6.4). The coefficients ǫ1 and ǫ2 are perturbed by random variables, i.e., ǫ1 (1 + ξǫ1 ) and ǫ2 (1 + ξǫ2 ). The other coefficients, ζ1 , ζ2 , χ1 and χ2 are deterministic. A straightforward extension of (6.21) to the two-particle model with competing species reaction yields the algebraic system (I2 ⊗ C1 ⊗ M ⊗ Isirk "P Su

∆t + 2 r0 ⊗

i=1

h

Ci ⊗ Ku,i 0 PSvN Q 0N Q C1 ⊗ Kv,1 + i=2 Ci+Su −1 ⊗ Kv,i

Airk (:,1) r(tm +c1 ∆t)2

+ 3∆tI2 ⊗ C1 ⊗ M ⊗


... h

i

r ′ (tm +c1 ∆t) r(tm +c1 ∆t) Airk (:, 1)

...

#!

i

r ′ (tm +csirk ∆t) r(tm +csirk ∆t) Airk (:, sirk )

0Q ǫ1 (C1 + Cǫ1 ) − ∆t ⊗ M ⊗ Airk 0Q ǫ2 (C1 + Cǫ2 )    (k)(c ) (k)(c ) (k)(c ) Q X C i ⊗ χ1 Mu,1 1 C i ⊗ 2ζ1 Mu,i 1 + χ1 Mv,1 1   + ∆t (k)(c ) (k)(c ) (k)(c ) C i ⊗ χ2 Mv,1 1 C i ⊗ 2ζ2 Mv,i 1 + χ2 Mu,1 1 i=1 ⊗ Airk (:, 1) . . .  

Q X i=1

(k)(csirk )

C i ⊗ 2ζ1 Mu,i

(k)(csirk )

+ χ1 Mv,1

(k)(csirk )

C i ⊗ χ2 Mv,1

⊗Airk (:, sirk )] Pûv x = b.

(k)(csirk )

C i ⊗ χ1 Mu,1



 (k)(cs ) (k)(cs ) C i ⊗ 2ζ2 Mv,i irk + χ2 Mu,1 irk (6.22)

Matrices 0N Q ∈ RN Q×N Q and 0Q ∈ RQ×Q are zero matrices and I2 ∈ R2×2 is an identity matrix. The vector x ∈ R2N Qsirk ×1 contains the stage vector increments for the Newton updates to the solutions u and v successively. Matrix Pûv ∈ R2N Qsirk ×2N Qsirk permutes the columns of the matrix that it is multiplied with so that consecutive IRK stage are grouped together in blocks of sirk columns. Matrix Cǫ1 ∈ RQ×Q is defined as hξǫ1 ΨΨT i; an analogous definition holds for matrix Cǫ2 . Matrix C i ∈ RQ×Q is given by hΨi ΨΨT i. The linearized quadratic terms in (6.9)– (k)(c ) (k)(c ) (6.10) are expressed by the (N × N ) matrices Mu,i j and Mv,i j . The elements

138


of these matrices are defined, analogously to (6.20) for t = tz + cj ∆t, as Z (k)(c ) (k) ui (x, tz + cj ∆t) + xu,i,j (x) sm (x)sn (x)dx Mu,i j (m, n) :=

(6.23)

D

(k)(cj )

Mv,i

(m, n) :=

Z D

(k) vi (x, tz + cj ∆t) + xv,i,j (x) sm (x)sn (x)dx, (6.24)

where xu,i,j is the jth IRK stage vector increment corresponding to Newton update (k) δu and random function Ψi . The system (6.22) is to be solved for the stage vector increments of the Newton updates at every time tm+1 and Newton step k + 1. Gray-Scott model The Gray-Scott model is determined by (6.7)–(6.8), where the parameters κA , κf and κ2 are deterministic; any uncertainty on the reaction comes from the uncertainty on the solution functions u and v. Since the Gray-Scott model contains a cubic nonlinearity, expressed by κA uv 2 , in contrast to the previously discussed quadratic nonlinearities, a new linearized model is required. We extend the weak formulation (6.13) to the weak formulation of the two-particle model with GrayScott reaction terms. This enables one to define the nonlinear residuals Fu (u, v) (k) (k) and Fv (u, v). Next, a weak formulation for the corrections δu and δv , with u = (k) (k) u(k) + δu and v = v (k) + δv , is constructed. A linear model is then obtained by (k) (k) dropping the high-order terms in δu and δv from previous formulation. Here, we point out this mechanism for the term −κA uv 2 in (6.7). In the weak formulation, this term is represented by the integral Z Z κA u(x, t, y)v(x, t, y)2 w(x, y) w ∈ H01 (D) ⊗ L2 (Γ). Γ

D

Note that the trial functions are now denoted by w(x, y) in order to avoid confusion (k) (k) with the unknown function v(x, t, y). With u = u(k) + δu and v = v (k) + δv , the product uv 2 can be expanded as 2 2 2 (u(k) + δu(k) ) v (k) + δv(k) = u(k) v (k) + 2u(k) v (k) δv(k) + u(k) δv(k)

2 2 + δu(k) v (k) + 2δu(k) v (k) δv(k) + δu(k) δv(k) .

The first term in this expansion contributes to the nonlinear residual, as in the (k) (k) right-hand side of (6.14). After dropping the high-order terms in δu and δv , the


139

linearized model for −κA uv 2 is given by Z Z 2 κA v (k) (x, t, y) δu(k) (x, t, y)w(x, y) Γ

+

(6.25)

D

Z Z Γ

D

κA 2u(k) (x, t, y)v (k) (x, t, y)δv(k) (x, t, y)w(x, y).

A SGFEM discretization based on a finite generalized polynomial chaos of order P applied to the random fields in (6.25) results for w(x, y) = w(x)Ψ e l (y) in Z Q Q X Q X X (k) (k) (k) hΨi Ψj Ψl Ψq i κA (x, t)w(x)dx e (6.26) vi (x, t)vj (x, t)δu,q D

i=1 j=1 q=1

+2

Z

D

(k) (k) (k) ui (x, t)vj (x, t)δv,q (x, t)w(x)dx e (k)

(k)

(k)

,

(k)

(k)

where vj , ui , δu,q and δv,q are the gPC coefficient functions of v (k) , u(k) , δu and (k)

δv , respectively. The multiple summations in (6.26) can be simplified by first 2 representing the random fields z (k) (x, t, y) := v (k) (x, t, y) and c(k) (x, t, y) := u(k) (x, t, y)v (k) (x, t, y) in (6.25) with a generalized PC expansion (2.8), i.e., ∞ 2 X z (k) (x, t, y) := v (k) (x, t, y) = (6.27) zr(k) (x, t)Ψr (y), r=1

c(k) (x, t, y) := u(k) (x, t, y)v (k) (x, t, y) =

∞ X

cr(k) (x, t)Ψr (y).

(6.28)

r=1

(k)

The functions zr

zr(k) (x, t) :=

(k)

and cr Q Q X X

vi (x, t)vj (x, t)hΨi Ψj Ψr i/hΨ2r i,

Q Q X X

ui (x, t)vj (x, t)hΨi Ψj Ψr i/hΨ2r i,

i=1 j=1

c(k) r (x, t) :=

are respectively defined as

i=1 j=1

(k)

(k)

(6.29)

(k)

(k)

(6.30)

with hΨ2r i = 1 when the polynomials Ψr are orthonormal. The SGFEM discretization of (6.25) then becomes Z Q Z X X (k) zr(k) (x, t)δu,q κA hΨr Ψl Ψq i (x, t)w(x)dx e (6.31) r=1 q=1

+2

Z

D

D

(k) t)δ (x, t) w(x)dx e . c(k) (x, v,q r

140


Due to the orthogonality of the polynomials Ψq , the gPC expansions (6.27)–(6.28) )! can be truncated after Z = (L+2P L!(2P )! terms, with L the number of random variables present in the problem. After a finite element spatial discretization, (6.31) can be written in matrix notation. The matrix representation of the linearized (κA uv 2 ) and (−κA uv 2 )-terms in (6.7)–(6.8) yields a (2N Q × 2N Q) time-dependent matrix, " # Z (k) (k) X C r ⊗ Jv2 ,r (t) C r ⊗ 2Juv,r (t) κA , (6.32) (k) (k) −C r ⊗ Jv2 ,r (t) −C r ⊗ 2Juv,r (t) r=1 with Z =

(L+2P )! L!(2P )!

and C r = hΨr ΨΨT i ∈ RQ×Q . The elements of the (N × N ) (k)

(k)

time-dependent matrices Jv2 ,r and Juv,r are respectively defined as (k)

Jv2 ,r (t)(m, n) := (k) Juv,r (t)(m, n) (k)

:=

Z

D

Z

D

zr(k) (x, t)sm (x)sn (x)dx

m, n ∈ 1, . . . , N

(6.33)

c(k) r (x, t)sm (x)sn (x)dx

m, n ∈ 1, . . . , N ,

(6.34)

(k)

with zr and cr given by (6.29)–(6.30). Next, an implicit Runge-Kutta discretization is applied to (6.32). Hereafter, the analogue of the algebraic system (6.22) for the Gray-Scott reaction can be formulated. Remark 6.3.1. The stochastic Galerkin finite element discretization of the lin)! earized Gray-Scott reaction leads to an expansion (6.32) with Z = (L+2P L!(2P )! terms, due to the cubic nonlinearity. The discretization of the competing-species reaction with quadratic nonlinearity results in an expansion with Q terms, see (6.22). Consequently, the Gray-Scott model requires a larger computational and memory cost than the competing species model.

6.4

An algebraic multigrid solution method

In every Newton iteration, a large algebraic system needs to be solved. This system is typically ill conditioned due to a poor conditioning of the stiffness matrices Ki , Ku,i and Kv,i . The study of time-dependent stochastic diffusion problems in Chapter 5 shows that efficient and robust multigrid algorithms can be constructed for algebraic systems resulting from a combined SGFEM and IRK discretization. In this section, these multigrid algorithms are extended to systems of the form (6.21) and the analogue for discretized systems of stochastic PDEs, e.g., given by (6.22). Only the algebraic variant of multigrid is considered since the geometry of the problem results in a highly unstructured spatial finite element discretization.

AN ALGEBRAIC MULTIGRID SOLUTION METHOD

6.4.1

141

Multigrid for a time-dependent, nonlinear, stochastic PDE

First, the case of one physical unknown function u(x, t, ξ) is considered. We apply a Newton-multigrid method. That is, after a global linearization, linear systems of the form (6.21) are solved with multigrid. Alternatively, a so-called ‘full approximation scheme’ could be constructed [TOS01], where multigrid is directly applied to a discretized nonlinear PDE, without first linearizing the problem. The structure of the linearized algebraic system (6.21) is very similar to the SGFEM-IRK discretization (5.10) of a time-dependent stochastic diffusion problem. The multigrid method presented in Section 5.3 can therefore almost instantly be applied to (6.21). Smoothing operator. Both the (sirk × sirk )- and (Qsirk × Qsirk )-block smoother of Section 5.3.1 can be extended to system (6.21). In the former case, this corresponds to solving a sequence of N Q systems of size (sirk × sirk ) at every smoothing iteration. In the latter case, a collective Gauss-Seidel iteration is applied, for which N systems of size (Qsirk × Qsirk ) have to be solved at every smoothing step. Multigrid hierarchy and transfer operators. As before, in Chapter 5, the AMG multigrid hierarchy is based on an AMG hierarchy constructed for the stiffness matrix K1 . The prolongation, restriction and coarse grid operator have a Kronecker product structure and are defined by (5.15)–(5.16). As a result, the same multigrid hierarchy can be reused for all time- and Newton steps.

6.4.2

Multigrid for a system of nonlinear, stochastic PDEs

Applying AMG to a system of PDEs typically requires important modifications to the construction of the multigrid hierarchy and operators in order to maintain the fast convergence rate of AMG for scalar PDEs [TOS01]. One usually discerns three different strategies for extending AMG to systems of PDEs: the so-called variable-based, unknown-based and point-based AMG approaches. Each of these strategies is based on a particular grouping of the unknowns. In the variable-based approach, no grouping is used and AMG for scalar PDEs is applied unchanged to a discretized system of PDEs. This approach typically does not work very efficiently. The unknown-based approach considers a grouping according to the different PDE unknowns. This approach is mainly useful for applications where the diagonal blocks of the algebraic system corresponding to an unknown solution function are close to being M -matrices. It requires a weak coupling between the different unknown solution functions. The third approach, point-based AMG, reorders the algebraic system according to the spatial grid points. This approach requires that

142


all unknown functions are discretized on the same spatial grid. For an extensive discussion on these various AMG strategies, we refer to [Cle04]. The stochastic variant of the system of PDEs (6.6) contains two unknown functions, u(x, t, ξ) and v(x, t, ξ), discretized on the same finite element grid. Both the unknown-based and point-based AMG approaches are therefore possible. Smoothing operator. We will apply a point-oriented smoother. This approach is known to be effective for deterministic reaction-diffusion models [Cle04]. For the stochastic two-particle model, this corresponds to a (2Qsirk × 2Qsirk )-block Gauss-Seidel smoother. At every collective Gauss-Seidel iteration, N systems of size (2Qsirk × 2Qsirk ) need to be solved, one for each spatial node n. We will compare the performance of this smoother to the performance of a (2sirk × 2sirk )block Gauss-Seidel smoother, which is defined as an extension of the (sirk × sirk )smoother in Section 6.4.1 to the two-particle case. Multigrid hierarchy and transfer operators. Assuming that the stochastic diffusion coefficients au (x, ξ) and av (x, ξ) are (piecewise) constant stochastic functions, the mean stiffness matrices Ku,1 and Kv,1 are equal up to a constant factor. This motivates the use of a multigrid hierarchy for Ku,1 as basis for a multigrid hierarchy for the discretized stochastic two-particle problem. Let P be the AMG prolongation operator derived for Ku,i , then one can construct the prolongation P and restriction operator R as P = I2Q ⊗ P ⊗ Isirk

R = I2Q ⊗ PT ⊗ Isirk .

and

(6.35)

The Galerkin coarse grid operator AH for the Gray-Scott reaction model equals i h′ r ′ (tm +csirk ∆t)Airk (:,sirk ) +c1 ∆t)Airk (:,1) AH = I2Q ⊗ PM PT ⊗ Isirk +3∆t r (tmr(t . . . +c ∆t) r(t +c ∆t) m 1 m s irk

∆t + 2 r0 ⊗

h

"P Su

i=1

Airk (:,1) r(tm +c1 ∆t)2

+ κA ∆t Z X r=1

...

0

C1 ⊗ PKv,1 PT +


" Z " (k)(c ) X C r ⊗ PJv2 ,r 1 PT

"

−C r ⊗ PJv2 ,r

(k)(csirk ) T

C r ⊗ PJv2 ,r

P

(k)(csirk ) T

−C r ⊗ PJv2 ,r

P

P

i

P Sv

i=2

Ci+Su −1 ⊗ PKv,i PT

#

κf C1 0 ⊗ PM PT ⊗ Airk + ∆t 0 −κ2 C1

(k)(c1 ) T

(k)(c1 ) T

r=1

...

Ci ⊗ PKu,i PT 0

C r ⊗ PJuv,r

P

(k)(c1 ) T

−C r ⊗ PJuv,r

P

(k)(csirk ) T

C r ⊗ PJuv,r

P

(k)(csirk ) T

−C r ⊗ PJuv,r

P

#

##

The result for the competing species model follows analogously.

Pûv .


143

concentration at t = 0.1 day 0.902

0.951

concentration at t = 0.1 day 1

(a) vertical cut of the apple

0.88

0.94

1

(b) horizontal cut of the apple

Figure 6.2: Deterministic initial value for u in the two-particle model (6.6).

6.5


6.5.1

Space-time discretization

Initial condition. A discontinuous initial condition, as in (6.5), is prone to introduce numerical errors: negative values for the concentration u(x, t) may arise due to Gibbs phenomena when solving the time-dependent problem (6.2). This is avoided by considering a smooth approximation to (6.5) at time tinit > 0, u(x, tinit , ξ) =

1 nseeds

nX seeds c=1

erfc(kx − xseedc k2 ) √ , 2 a ˆtinit

where a ˆ is a constant representing the mean diffusion coefficient, nseeds equals the number of seeds, and xseedc corresponds to the coordinates of the center of seed number c. In the two-particle case, the same initial condition is used for u and v(x, tinit , ξ) = 1 − u(x, tinit , ξ). An illustration of the initial value for a ˆ = 0.04 and tinit = 0.1 is given in Fig. 6.2. Finite element mesh. In order to obtain accurate simulation results, the finite element mesh is locally refined near the seeds, where the reaction is initiated. An illustration of the mesh is given in Fig. 6.3(a). Note that the quality of the finite element mesh, e.g., the aspect ratio of the tetrahedra, has an impact on the AMG convergence rate. By optimizing the quality of the 3D tetrahedra in GMSH [GR09], improved AMG convergence rates result. Fig. 6.3(b) shows the residual decay of

144


10

10

N = 40217 N = 43393

5

relative residual

10

0

10

−5

10

−10

10

0

(a)

10

20

30

AMG-BiCGStab iterations

40

(b)

Figure 6.3: (a) Finite element triangularization (N = 24779). (b) Residual decay as a function of the number of BiCGStab-AMG iterations when solving (6.21), in case of a second-order Hermite-Legendre chaos and a 2-stage IRK time discretization (Qsirk = 42).

AMG with collective Gauss-Seidel smoother applied to (6.21) for two finite element meshes. The shape and distribution of the tetrahedra of the mesh corresponding to N = 43393 are optimized. This results in a better AMG convergence rate.

6.5.2

Matrix formulation and storage

Multivectors. The discretization of the stochastic two-particle model (6.6) can be rewritten as a system of matrix equations, similar to (5.23) for the discretization of a time-dependent stochastic diffusion problem. The unknown vector x and righthand side b correspond respectively to the multivectors X, B ∈ RN ×2Qsirk . This storage format provides an easy access of the unknowns per spatial point: they are given by a row of X. The 2Qsirk unknowns per row are ordered first per physical unknown, then per stochastic unknown and finally per IRK stage. This leads to an easy distinction between the physical unknown functions u(x, t, ξ) and v(x, t, ξ). System matrix. In order to reduce the memory cost, the algebraic system matrix, see, e.g., (6.21), is never constructed explicitly, as also discussed in Section 5.5.1. Only the separate building blocks are stored in sparse matrix format. As a consequence, for every matrix-vector multiplication or smoothing iteration, the system matrix needs to be assembled on the fly. This increases the computational cost, especially for problems with many individual matrices as in case of a discretized Gray-Scott reaction (6.32).


145

(k)(cj )

Linearized reaction matrices. The matrices Mu,i (k)(c ) Jv2 ,i j

(k)(cj )

, Mv,i

in (6.20), (6.23)–

(k)(c ) Juv,i j

(6.24), and and in (6.33)–(6.34), that appear in the Newton system matrix, are also needed in order to evaluate the nonlinear residual, F (u) in (6.15), or Fu (u, v) and Fv (u, v). The construction of the Newton system therefore comes ‘for free’ once the nonlinear reaction terms are evaluated. Computation of Gray-Scott Jacobian matrices. The computation of Gray-Scott linearized reaction matrices of the form (6.33)–(6.34) can become very expensive. This cost is reduced by employing known information about the precise sparsity (k) structure. Consider, for example, the matrix Juv,r (t) (6.34), evaluated at time tˆ (k)

with cr

defined by (6.30). The (m, n)-th element of this matrix equals

(k) ˆ (t)(m, n) Juv,r

Z Q Q X X (k) (k) hΨi Ψj Ψr i ui (x, tˆ)vj (x, tˆ)sm (x)sn (x)dx. (6.36) := i=1 j=1

D

From the sparsity structure of matrix C r := hΨr ΨΨT i, see Section 2.5, it follows that many scalars hΨi Ψj Ψr i in (6.36) are zero. Denote by nz(i) the number of nonzero elements in row i of matrix C r , with corresponding vector of column indices cni . This enables one to rewrite the double summation in (6.36) as Z Q nz(i) X X (k) (k) (k) ˆ hΨi Ψcni (j) Ψr i ui (x, tˆ)vcni (j) (x, tˆ)sm (x)sn (x)dx. (6.37) Juv,r (t)(m, n) = D

i=1 j=1

(k)

(k)

After representing both ui (x, tˆ) and vcni (j) (x, tˆ) in (6.37) with a spatial finite element discretization, we have that (k) ˆ Juv,r (t)(m, n) =

Q nz(i) X X i=1 j=1

Z

hΨi Ψcni (j) Ψr i

N X N X

(k) (k)

ui,l vcni (j),w

(6.38)

l=1 w=1

sl (x)sw (x)sm (x)sn (x)dx,

D

(k)

(k)

where ui,l and vcni (j),w are the coefficients of the finite element representation of (k)

(k)

the functions ui and vcni (j) , corresponding to node l and w, respectively. The cost of evaluating the double summation over N in (6.38) becomes excessively large for practical values of N . Since the piecewise linear nodal basis functions sn (x) are nonzero on only a small part of the domain D, this information can be used to reduce the computational cost. The sparsity pattern of, e.g., the mass matrix M reveals which nodal basis functions, sl and sw , do not overlap. Therefore, we

146


can rewrite (6.38) as (k) ˆ Juv,r (t)(m, n)

=

Q nz(i) X X i=1 j=1

Z

D

M

hΨi Ψcni (j) Ψr i

N nzX(l) X l=1

(k) (k)

ui,l vcni (j),w

w=1

sl (x)scnM (w) (x)sm (x)sn (x)dx, l

where nz M (l) is the number of nonzero elements of row l in matrix M , and cnM l contains the corresponding indices of the nodal basis functions. Remark 6.5.1. The above construction procedure for the linearized reaction matrices is not needed in case of the competing species reaction. In that case, the (k) (k) matrices Mu,i and Mv,i (6.23)–(6.24) result from linearizing a quadratic nonlinearity. As a consequence, they are each based on only one polynomial chaos coefficient (k) (k) function, ui or vi , instead of on a product of two functions.

6.5.3

Krylov preconditioning

The algebraic multigrid methods developed in Section 6.4 can be used stand-alone or as preconditioner for a Krylov method (BiCGStab). The computational cost of these methods is dominated by the cost of the matrix-vector multiplication and the smoothing operations, which can become very expensive due to the storage format, as discussed in Section 6.5.2. Cheaper preconditioners with comparable convergence properties can however be constructed. Their efficiency will be demonstrated with numerical results in Section 6.6. A. Multigrid for stochastic diffusion problems. In the case of reaction-diffusion problems with a dominant diffusion term, multigrid for linear, time-dependent, stochastic diffusion problems can be used as preconditioner for the linearized algebraic systems. The multigrid preconditioner is constructed similar to the multigrid method described in Section 6.4. It considers as algebraic system the systems (6.21) and (6.22) with all matrices due to reaction removed. The residual computation and smoothing operator therefore do not need to loop over the reaction matrices. This results in a lower computational and memory cost than the AMG method of Section 6.4. B. Multigrid with modified smoother. Alternatively, the multigrid method from Section 6.4 with modified smoothing operator is used as preconditioner. The high computational cost of the collective smoothing operator is reduced by omitting any matrices from the (linearized) reaction of the problem in the smoother. That is,


147

a block smoother for a SGFEM-IRK discretization of a time-dependent diffusion problem is used as relaxation method. Note that this multigrid preconditioner is not convergent when used stand-alone.

6.6

Numerical experiments

The parameters of the growth function (6.3) are set to α = 0.048, r0 = 3.5 cm and r∞ = 5 cm. In all tests, about 2 to 5 Newton steps were sufficient to compute the nonlinear solution up to machine level accuracy. The simulations were performed on a 2.66 GHz Intel Xeon X5550 8-core machine with 32 GByte RAM. The (Qsirk × Qsirk ) or (2Qsirk × 2Qsirk )-blocks of the collective Gauss-Seidel smoothers are solved with SuperLU [DEG+ 99].

6.6.1

One chemical particle

The stochastic one-particle model (6.12) is considered with five random variables: four lognormal random variables to model the stochastic diffusion coefficient (6.11) and one uniformly distributed random variable that models the reaction coefficient κ. This corresponds to a mixed Legendre-Hermite chaos stochastic discretization. The stochastic diffusion coefficient a (6.11) is based on the following set of parameters: a1 = 0.03, a2 = 0.032, a3 = 0.035 and a4 = 0.04. The lognormal random variables are based on zero-mean Gaussian random variables with standard deviation 0.1. The reaction coefficient κ is uniformly distributed on [0.225, 0.275]. The time step ∆t is gradually increased from 0.01 day at t = tinit = 0.05 day, to 0.02 at t = 0.1 day, 0.2 at t = 1 day, 0.2 at t = 5 days and to 0.5 at t = 10 days. The stochastic solution of this one-particle model (6.12) is illustrated in Fig. 6.4, which shows a cut of the apple after 72 hours and the evolution of the solution statistics in time. After 85 days, the steady-state solution is reached, which is deterministic and equal to one on the entire domain. Choice of multigrid method. Table 6.1 presents the average number of AMG iterations and timing results when solving the algebraic system (6.21) with BiCGStab preconditioned by AMG V(3,2)-cycles. The multigrid method from Section 6.4 and the two alternative preconditioners from Section 6.5.3 are applied. Results for stand-alone multigrid are also given. The performance of the (sirk × sirk )-block and the (Qsirk × Qsirk )-block GaussSeidel smoother proposed in Section 6.4.1 is comparable, both in iterations counts and computational time. The two alternative AMG preconditioning approaches effectively reduce the computational time of the multigrid solution method, while requiring a similar number of iterations as the original AMG preconditioner.

148


standard deviation after 3 days 0

7.74e-05

standard deviation after 3 days 0.000155

0

(a) vertical cut

3

pdf at time = 45 and x = [−1 0 0] pdf at time = 45 and x = [−0.46 0 0] pdf at time = 45 and x = [−0.36 0 0]

1

2.5 2 1.5 1 0.5 0 0

0.2

0.4 0.6 concentration

0.00016

(b) horizontal cut

mean / standard deviation

3.5

8e-05

0.8

(c) probability density function

1

0.8 0.6

mean at x = [−1 0 0] mean at x = [−0.46 0 0] mean at x = [−0.37 0 0] standard deviation at x = [−1 0 0] standard deviation at x = [−0.46 0 0] standard deviation at x = [−0.36 0 0]

0.4 0.2 0 0

20

40

60 time [day]

80

100

(d) mean and standard deviation

Figure 6.4: Solution statistics of the stochastic one-particle model (6.12). (a)–(b) Standard deviation after 72 hours. (c) Probability distribution after 45 days in the point (−1, 0, 0) on the skin and the points (−0.46, 0, 0) and (−0.36, 0, 0), respectively. (d) Evolution in time of the mean and standard deviation. (N = 11 799, 5 random variables, second-order Legendre-Hermite chaos, Q = 21, 2-stage IRK discretization)

AMG Convergence properties. Fig. 6.5 illustrates the influence of the discretization parameters on the AMG convergence behavior. AMG with (Qsirk × Qsirk )block Gauss-Seidel smoother is applied as BiCGStab preconditioner to (6.21). Results for the second alternative preconditioner (B) from Section 6.5.3 are given. For different choices of the number of spatial nodes, chaos order and IRK stages, a similar convergence behavior is observed. This suggests that the optimal convergence properties of the multigrid method with block smoother developed in Chapter 5 continue to hold for the nonlinear stochastic reaction-diffusion problem (6.12). Similar results are obtained with the other discussed AMG preconditioning approaches.


149

N = 43393 iter time AMG stand-alone (sirk × sirk )-block Gauss-Seidel smoother (Qsirk × Qsirk )-block Gauss-Seidel smoother AMG preconditioner for BiCGStab (sirk × sirk )-block Gauss-Seidel smoother (Qsirk × Qsirk )-block Gauss-Seidel smoother AMG preconditioner [A] (sirk × sirk )-block Gauss-Seidel smoother (Qsirk × Qsirk )-block Gauss-Seidel smoother AMG preconditioner [B] (sirk × sirk )-block Gauss-Seidel smoother (Qsirk × Qsirk )-block Gauss-Seidel smoother

23.8 23.8

1401 1661

15.8 15.5

1069 975

15.8 16

532 407

15.8 16

670 440

Table 6.1: Average number of V (3, 2)-AMG iterations, stand-alone or as preconditioner for BiCGStab, and solution time (in seconds) required to solve one time step of the algebraic system (6.21) so that ||r||/||b|| < 10−14 . The stochastic problem (6.12) contains 5 random variables and is discretized with a second-order Hermite-Legendre chaos, combined with a 2-stage IRK time discretization for ∆t = 0.1 day. N = 24 799, sirk = 2, Qsirk = 12

10

10

N = 24 799, sirk = 2, Qsirk = 112 5

N = 24 799, sirk = 3, Qsirk = 63

relative residual

10

N = 43 393, sirk = 2, Qsirk = 42 0

10

N = 43 393, sirk = 2, Qsirk = 112

−5

10

−10

10

−15

10

0

4

8

12

16

20

AMG-BiCGStab iterations

Figure 6.5: Relative residual as a function of the number of AMG-BiCGStab iterations when solving one time step of (6.21) with ∆t = 0.01 day. A W(1,1)-AMG cycle with (Qsirk × Qsirk )-block Gauss-Seidel smoother is applied as preconditioner.

6.6.2

Two chemical particles

Gray-Scott model. Here, we consider a stochastic variant of the two-particle model (6.6) with Gray-Scott reaction (6.7)–(6.8) and containing four random vari-

150


ables: the stochastic diffusion coefficients au (x, ξ) and av (x, ξ) are piecewise constant functions (6.11) and each contain two lognormally distributed random variables, based on zero-mean Gaussian variables with standard deviation 0.1. The random variables model the uncertainty on the diffusion coefficient of the core (domain D3 ) and of the fleshy tissue (domain D4 ). The other parameters are deterministic: au,1 = 0.03, au,2 = 0.032, au,3 = 0.035, au,4 = 0.04, av,1 = 0.01, av,2 = 0.012, av,3 = 0.015, av,4 = 0.02, κA = 50, κf = 2 and κ2 = 3. Fig. 6.6 shows the time evolution of the mean and standard deviation of the concentrations u and v. After about 7 days, steady-state is reached, where both u and v are constant functions with values respectively 0.1 and 0.6. Similar to the one-particle case, no uncertainty is present in steady-state. Competing species model. We solve the stochastic variant of the two-particle model (6.6) with competing species reaction model (6.9)–(6.10) and 10 random variables: 8 lognormally distributed random variables perturbing the four parts of the stochastic diffusion coefficients au (x) and av (x), and 2 uniformly distributed random variables modelling the uncertainty on ǫ1 and ǫ2 in (6.9)–(6.10). The same parameters for the diffusion coefficients are used as for the Gray-Scott tests; the parameters of the competing species model are given by ǫ1 uniformly distributed on [0.9, 1.1], ǫ2 uniformly distributed on [0.675, 0.825], ς1 = 1, ς2 = 1, χ1 = 1 and χ2 = 0.5. Fig. 6.8 shows the evolution of the statistics of u and v in time. In contrast to the simulation results presented in Fig. 6.4 and 6.6, the standard deviations of u and v are not equal to zero at steady-state. AMG convergence properties The performance of the AMG method applied to the stochastic two-particle model is illustrated in Fig. 6.7. W(1,1)-cycles of AMG with a (2Qsirk × 2Qsirk )-block Gauss-Seidel smoother are applied as preconditioner for BiCGStab. When changing one of the discretization parameters, only small differences in the convergence behavior are observed. Also, the AMG performance is very similar for the competing species and Gray-Scott reaction model. In order to limit the computational cost, Fig. 6.7 shows the results for the alternative AMG preconditioner (B) presented in Section 6.5.3. In the case of a competing species reaction, the alternative AMG preconditioner (A) presented in Section 6.5.3 results in a poor BiCGStab convergence. The alternative AMG preconditioner (B) yields a fast convergence rate and reduces the computational time of the AMG method proposed in Section 6.4 by almost a factor 2. This reduction in solution time is illustrated in Table 6.2, where the average number of AMG cycles is shown for a stochastic problem with either 8 or 10 random variables. In the former case, the competing species reaction coefficients ǫ1

CONCLUSIONS

151

mean u at t = 1.5 day 0.00667

standard deviation u at t=1.5 day

0.506

1.01

0

(a) horizontal cut of mean 1.2

0.2

0.6 0.4

standard deviation u at x = [−1 0 0] standard deviation u at x = [−0.46 0 0] standard deviation u at x = [−0.36 0 0] standard deviation v at x = [−1 0 0] standard deviation v at x = [−0.36 0 0]

0.15 standard deviation

0.8 mean

0.0265

(b) vertical cut of standard deviation

mean u at x = [−1 0 0] mean u at x = [−0.46 0 0] mean u at x = [−0.37 0 0] mean v at x = [−0.46 0 0]

1

0.0133

0.1

0.05

0.2 0 0

2

4

time [day]

6

(c) mean of u and v

8

10

0 0

2

4

time [day]

6

8

10

(d) standard deviation of u and v

Figure 6.6: Solution of the stochastic two-particle model (6.6) with Gray-Scott reaction terms (6.7)–(6.8) and four lognormally distributed random variables. The problem is discretized with a spatial finite element mesh with 11 799 nodes, a second-order Hermite chaos and a two-stage IRK time discretization.

and ǫ2 are deterministic. In the latter case, the same parameter configuration as for Fig. 6.8 is used.

6.7

Conclusions

We presented a simulation procedure to compute the concentration of chemical particles in a growing apple. This simulation can be used to model the transition of starch into sugar. A stochastic Galerkin finite element approach is applied to quantify the impact of uncertainty on the simulation outcomes. This method

152


−5

N = 24477, P = 2, sirk = 1

N = 11799, P = 2, sirk = 2

N = 43393, P = 2, sirk = 1

N = 43393, P = 2, sirk = 2 N = 24477, P = 3, sirk = 2

−10

10

N = 11799, P = 3, sirk = 1

−5

10

N = 24477, P = 1, sirk = 3

−10

10

−15

−15

0

N = 11799, P = 2, sirk = 1

N = 24477, P = 2, sirk = 2

N = 24477, P = 2, sirk = 3

10

10

0 10

N = 24477, P = 2, sirk = 1

relative residual

relative residual

0 10

10 5 15 20 AMG-BiCGStab iterations

25

(a) Gray-Scott reaction, 4 random variables

10

0

10 5 15 20 AMG-BiCGStab iterations

25

(b) Competing species, 10 random variables

Figure 6.7: Decay of the relative residual as a function of the number of AMGBiCGStab iterations when solving one time and Newton step of the stochastic variant of the two-particle model (6.6). As preconditioner, a W(1,1)-cycle of AMG with (2Qsirk × 2Qsirk )-block Gauss-Seidel smoother is used.

random variables AMG of Section 6.4 8 lognormal variables 8 lognormal and 2 uniform ξi ’s AMG method B of Section 6.5.3 8 lognormal variables 8 lognormal and 2 uniform ξi ’s

iterations

time (sec.)

13.5 14

3144 6194

16 16.5

1796 3649

Table 6.2: Average number of V(3,2)-AMG cycles as preconditioner for BiCGStab and solution time (in seconds) required to solve one time and Newton step of a stochastic two-particle model (6.6), with competing species reaction (6.9)-(6.10). A collective GaussSeidel smoother is applied. The discretization is based on N = 24 799 nodes, a secondorder Hermite-(Legendre) chaos and a 2-stage IRK time discretization corresponding to either 160 or 240 unknowns per spatial point, depending on L = 8 or L = 10.

is combined with a Newton linearization of the nonlinear reaction terms and an implicit Runge-Kutta method for the time discretization. The stochastic Galerkin method can straightforwardly be applied since only polynomial nonlinearities are present. In the case of a cubic nonlinearity, the linearized system contains a product of random fields so that the construction and storage of the algebraic system matrix is more involved and computationally expensive than in the case of a quadratic nonlinearity. An algebraic multigrid solver is presented to solve the algebraic linearized systems. This solver is shown to possess very good convergence properties. A point-based

CONCLUSIONS

153

smoothing approach is key to its performance. The computational cost can be reduced by applying an AMG method based on an approximation to the system matrix as preconditioner for BiCGStab. The stochastic model provides insight into the impact of uncertainty on the distribution of chemical particles in fruit. In order to set up more realistic simulations, additional experiments are needed to determine an accurate reaction model and parameters. An extension to a multi-element stochastic polynomial basis is required so that a bifurcation analysis can be performed.

154


mean u at t = 3 days 0.943

mean v at t = 18 days

0.957

0.972

0.0188

(a) horizontal cut of mean u

0.0913

(b) vertical cut of mean v

standard deviation u at t = 12 days 0.0575

0.0662

0.164

standard deviation v at t = 5 days 0.0748

1.19e-05

(c) vertical cut of standard deviation u

0.00323

0.00645

(d) horizontal cut of standard deviation v

1

0.2

0.8 standard deviation

0.15

mean

0.6 0.4 mean u at x = [−1 0 0] mean u at x = [−0.46 0 0] mean u at x = [−0.37 0 0] mean v at x = [−1 0 0] mean v at x = [−0.46 0 0] mean v at x = [−0.37 0 0]

0.2 0 −0.2 0

20

40 60 time [day]

(e) mean u and v

80

0.1 u at x = [−1 0 0] u at x = [−0.46 0 0] u at x = [−0.36 0 0] v at x = [−1 0 0] v at x = [−0.46 0 0] v at x = [−0.36 0 0]

0.05

100

0 0

20

40 60 time [day]

80

100

(f) standard deviation u and v

Figure 6.8: Solution of the stochastic two-particle model (6.6) with competing species reaction (6.9)–(6.10) and 10 random variables. A spatial discretization with 11 799 nodes, a 2nd order Hermite-Legendre chaos and an implicit Euler time discretization are applied.

Chapter 7

Comparison of the stochastic Galerkin and collocation method 7.1

Introduction

In recent years, both the stochastic Galerkin and the stochastic collocation method have been intensively studied. In order to increase the accuracy and temper the curse of dimensionality, a range of variants and adaptive solution approaches have been proposed. The question of which method, stochastic Galerkin or stochastic collocation, yields the most accurate solution in the lowest solution time, remains still open. This chapter aims at bringing insight into the cost of performing a stochastic Galerkin or collocation simulation by comparing the efficiency and accuracy of both stochastic solution methods numerically. Comparative studies on the performance of both stochastic methods are rare and mostly limited to linear PDEs [Bie10, MHZ05, Xiu09, BNTT09]. Here, the stochastic solution of a nonlinear stochastic convection-diffusion equation is considered. This model studies the uncertainty on the torque yielded by a ferromagnetic cylinder rotating at high speeds. The results of this chapter have been published in [RDV10a, RDV10b]. This chapter is organized as follows. In Section 7.2, the stochastic collocation method is described in more detail. A discussion on the stochastic Galerkin method versus the stochastic collocation approach is initiated in Section 7.3 and illustrated by a small numerical example. The mathematical model of the ferromagnetic cylinder problem and its stochastic discretization are introduced in Section 7.4.

155

156

COMPARISON OF THE STOCHASTIC GALERKIN AND COLLOCATION METHOD

Applying the stochastic Galerkin method to the ferromagnetic problem requires substantial changes to the deterministic solver routines. Its computational aspects are detailed in Section 7.5. Section 7.6 presents a comparison of the accuracy and computational cost of the stochastic collocation and stochastic Galerkin method applied to the ferromagnetic model. In Section 7.7, the propagation of uncertainty on material parameters towards uncertainty on the torque is illustrated.

7.2

The stochastic collocation method

Non-intrusive stochastic finite element methods avoid the construction and solution of a stochastic Galerkin system (2.22). For example, by replacing the integrals of the stochastic Galerkin projection by quadrature formulas, a pseudo-spectral Galerkin method can be constructed. This is equivalent to performing a collocation step instead of a Galerkin projection [BNT07]. In particular when the input data depend nonlinearly on the random variables and possibly have unbounded second moments, applying a stochastic Galerkin projection can be troublesome. This is not an issue for stochastic collocation methods. The stochastic collocation finite element method (SCFEM) [BNT07, XH05] represents the solution of a stochastic PDE (2.1) discretely by an expansion with multivariate Lagrange polynomials, ℓk (ξ), u(x, t, ω) ≈

Nc X

uk (x, t, ζk )ℓk (ξ).

(7.1)

k=1

The Lagrange polynomials ℓk (ξ) are interpolatory polynomials, defined by a set of multidimensional collocation points, {ζ1 , . . . , ζNc }. Each collocation point ζk consists of L components, ζk = (ζk,1 , . . . , ζk,L ), according to the L random variables ξ present in the problem. The set of polynomials {ℓ1 (ξ), . . . , ℓNc (ξ)} belongs to the space of tensor product polynomials with degree at most P = (p1 , . . . , pL ). For example, in the case Qof a full tensor product construction of the polynomials ℓk , we have that Nc = L i=1 (pi + 1).

Remark 7.2.1. Instead of an expansion with Lagrange polynomials, also an expansion with multivariate orthogonal polynomials can be employed [Xiu07]. The solution is then represented by the same type of expansion, called generalized polynomial chaos (gPC) expansion, as in the stochastic Galerkin method, see (2.21). This type of expansion enables an easy calculation of the solution statistics. It introduces however an aliasing error [Xiu09] since the stochastic collocation method computes only an approximation to the gPC expansion of the solution.

THE STOCHASTIC COLLOCATION METHOD

157

The stochastic collocation approach proceeds by requiring that the residual, i.e., L(x, t, ω; u(x, t, ω)) − f (x, t, ω), vanishes at each collocation point: L(x, t, ζk ; u(x, t, ζk )) = f (x, t, ζk )

for k = 1, . . . , Nc .

(7.2)

From the properties of Lagrange interpolation, this is equivalent to solving a set of Nc decoupled deterministic PDEs, each of type (2.1), with ξ replaced by the appropriate collocation point ζk . Note that also for nonlinear stochastic PDEs, the stochastic collocation method leads to a set of uncoupled (nonlinear) deterministic problems. This is a clear advantage over the stochastic Galerkin finite element method. By construction, the stochastic collocation solution interpolates the true solution at the collocation points. Remark 7.2.2. The Monte Carlo simulation method can be written as a collocation procedure [XK03a] where the test basis is δ(ξ − ωk ) instead of ℓk (ξ), with δ(·) the Dirac delta function and ωk an isolated random event. The stochastic collocation method improves the convergence of Monte Carlo by carefully selecting the collocation points.

7.2.1

Computation of statistics

After solving the deterministic PDEs (7.2), the mean of the solution, denoted by hui, is obtained by evaluating the expected value of (7.1): hui(x, t) =

Nc X

k=1

u(x, t, ζk )hℓk i

with

hℓk i =

Z

ℓk (y)̺(y)dy,

Γ

where ̺(y) is the joint probability density function of the random vector ξ and y ∈ Γ, the support of ̺(y). In a similar way other moments of the solution can be computed, for example the variance corresponds to Z u(x, t, y)2 − (hui(x, t))2 ̺(y)dy. var(u)(x, t) = Γ

Note that an explicit evaluation of the Lagrange polynomials for computing the expected values hℓk i can be avoided by choosing the set of collocation points to be a cubature point set [Xiu09, XH05]. In that case, the integrals hℓk i reduce to the cubature weights w1 , . . . , wNc . The expected value and the variance can be written respectively as [LWB07] hui(x, t) =

var(u)(x, t) =

Nc X

u(x, t, ζk )wk

(7.3)

Nc X

u2 (x, t, ζk )wk − (hui(x, t))2 .

(7.4)

k=1

k=1

158


7.2.2

Construction of collocation points

The main advantage of the stochastic collocation method is that it leads to a set of decoupled deterministic PDEs. Simulation code for deterministic PDEs can immediately be reused and parallelization of the algorithm is straightforward. However, to obtain a high accuracy and at the same time a limited computational complexity, the choice of the set of collocation points turns out to be crucial. The multidimensional collocation points are based on one-dimensional collocation points which are typically constructed as Clenshaw-Curtis points or Gauss points [GW69], i.e., zeros of polynomials that are orthogonal w.r.t. the probability density function of a random variable [NTW08b]. A multidimensional set is then obtained by either considering a tensor product grid or a sparse grid of the one-dimensional points. Another possibility for constructing multidimensional collocation points is to apply Stroud’s cubature of degrees 2 and 3 [XH05, DLZZ08]. This approach enables a cheap construction of low-order approximations to the solution’s statistics, especially in the case of many random variables. It is however not suited for constructing a high-order accurate solution. Tensor product grid Given a 1D set of collocation points y i = {y1i , . . . , ypi i +1 }, a 1D Lagrange interpolation operator is defined for a smooth function f (y) as i

U (f )(y) =

pX i +1

f (yji )lji (y)

with

lji (y)

j=1

=

pY i +1

m=1,m6=j

i y − ym , i yji − ym

(7.5)

where lji (y) is a Lagrange polynomial of degree pi . A tensor product formula combines sequences of 1D interpolation operators into a multidimensional operator: piL +1

pi1 +1

(U This with This L or

i1

iL

⊗ . . . ⊗ U )(f )(y) =

X

j1 =1

...

X

jL =1

f (yji11 , . . . , yjiLL )(lji11 (y) ⊗ . . . ⊗ ljiLL (y)).

multidimensional operator defines the Lagrange polynomials ℓk (ξ) in (7.1), QL y replaced by ξ. It requires Nc = i=1 (pi + 1) deterministic simulations. amount becomes rapidly impractical for a large number of random variables polynomial degree pi .

Remark 7.2.3. A full tensor product stochastic collocation solution of the stochastic diffusion problem (2.25) with a diffusion coefficient approximated by a KLexpansion, coincides with a stochastic Galerkin solution represented by a double orthogonal polynomial expansion [BNT07]. The multidimensional collocation points are based on Gauss points, which correspond to the constants cpi ,ji in (2.19).

159

1

1

0.5

0.5

0

−0.5

ξ3

1 0.5

ξ3

ξ3

THE STOCHASTIC COLLOCATION METHOD

0

−0.5

−1 1

−1 1

−1 1

1 0

ξ2

−1 −1

1

0

0

ξ1

ξ2

(a) full tensor

0

−0.5

0 −1 −1

ξ1

(b) sparse Gauss-Legendre

1 0

ξ2

0 −1 −1

ξ1

(c) sparse Clenshaw-Curtis

Figure 7.1: Three-dimensional collocation points: (a) full tensor product grid using Gauss-Legendre abscissa with p1 = p2 = p3 = 5; (b) isotropic Smolyak sparse grid of level 5 using Gauss-Legendre abscissa; (c) isotropic Smolyak sparse grid of level 5 using Clenshaw-Curtis nodes.

Sparse grids A sparse grid construction of stochastic collocation points aims at reducing the number of points needed for a full tensor product grid, while maintaining the approximation quality up to a logarithmic factor. The density of three-dimensional sparse grids in comparison to a full tensor product grid is illustrated in Fig. 7.1. Gauss-Legendre abscissa are zeros of Legendre polynomials; extrema of Chebyshev polynomials determine Clenshaw-Curtis points. Sparse grids correspond to a subset of full tensor product grids. Different algorithms for constructing sparse grids are possible, as detailed in [BG04]. Smolyak’s algorithm [Smo63, BNR00] is typically used. It introduces an integer w, called level, and defines an L-dimensional interpolation operator H(w, L) as X L−1 (−1)w−|i| · (7.6) · (U i1 ⊗ . . . ⊗ U iL ), H(w, L) = w − |i| w−L+1≤|i|≤w

with |i| = i1 + . . . + iL , i ∈ NL and U im defined by (7.5). To compute H(w, L)(f ), one only needs to know function values evaluated at the sparse grid, [ (y i1 × . . . × y iL ). w−L+1≤|i|≤w

i When Clenshaw-Curtis points are used, the abscissa ym are given by the extrema of Chebyshev polynomials, i.e., π(m − 1) i m = 1, . . . , pi + 1. ym = − cos pi − 1

160


By choosing the number of collocation points, pi + 1, in each level according to the relation pi = 2i for i > 0 and p0 = 0, the Clenshaw-Curtis collocation points corresponding to various Smolyak levels are nested [GG98, NTW08b]. As a consequence, fewer deterministic simulations are needed in comparison to a nonnested sparse grid construction with Gauss points, for a fixed Smolyak level w. In order to increase the efficiency of sparse grids, anisotropic Smolyak [NTW08a] or adaptively refined [GZ07] sparse grids can be employed. Both a-priori and aposteriori procedures can be used to determine the important random dimensions which require a more accurate stochastic approximation. Analogously to multi-element stochastic Galerkin methods (see Section 2.3.3), multi-element sparse grid collocation methods can be developed [MZ09, FWK08]. These can accurately represent discontinuities in the stochastic space or random oscillations [LSK08]. Typically a multi-element approach is combined with an adaptive refinement strategy in order to detect discontinuity regions and then refine the collocation points in these regions. Such a multi-element sparse stochastic collocation approach ensures a more efficient adaptive selection of collocation points in comparison to anisotropic sparse grids.

7.2.3

Convergence

The convergence properties of the stochastic collocation finite element method depend on the construction of the collocation points. When the stochastic collocation method is applied to a stochastic diffusion problem of type (2.25) using a full tensor product grid of Gauss points, Babuˇska et al. [BNT07] proved that the probability error converges exponentially with respect to the degree of the Lagrange polynomials. This means that a full tensor product collocation method has the same convergence rate as a polynomial chaos based stochastic Galerkin method albeit that the number of degrees of freedom grows faster in the full tensor product case than for a complete polynomial chaos basis. The convergence requirements for the full tensor product stochastic collocation method are however less restrictive than for the stochastic Galerkin method: the exponential convergence also holds for unbounded and dependent random variables. Sparse collocation grids are becoming very popular since they alleviate the curse of dimensionality of full tensor product grids while maintaining a high level of accuracy. In [NTW08b], the theoretical convergence of the sparse stochastic collocation method is compared to the full tensor product case. A sub-exponential convergence of the probability error with respect to the number of collocation points in the asymptotic regime is proven, along with an algebraic convergence in the pre-asymptotic regime. Similar convergence characteristics also apply to the anisotropic sparse grids introduced in [NTW08a].

COMPARISON OF STOCHASTIC GALERKIN AND COLLOCATION METHODS

161

Note that the spatial and random unknowns can be sparsified simultaneously, e.g., by combining sparse Smolyak collocation with a hierarchic wavelet spatial discretization. The resulting sparse composite stochastic collocation method [Bie10] converges exponentially w.r.t. the number of collocation points. For small Smolyak levels, it has an algebraic convergence rate. The Smolyak sparse grids are then based on Gauss points. Similar results apply for a Hoeffding ANOVA construction of collocation points instead of Smolyak sparse grids [BS09].

7.3

7.3.1

Comparison of stochastic Galerkin and collocation methods Discussion

In choosing between a stochastic Galerkin or a stochastic collocation approach, the main argument for the stochastic collocation approach lies in its non-intrusiveness. The implementation effort of a stochastic collocation method basically corresponds to performing a Monte Carlo simulation. As a result, both implementing and parallelizing a stochastic collocation method becomes almost trivial. When using a sparse collocation grid, efficient implementations of Smolyak’s algorithm are available, see, e.g., [Pet01]. The implementation of a stochastic Galerkin method involves the construction and solution of a fully coupled system of Q deterministic equations. Some iterative solvers, such as the mean-based preconditioner (see Section 3.5), decrease the intrusiveness, but the same ease of implementation as for a stochastic collocation method cannot be attained. Especially for non-polynomial nonlinearities several issues arise, see Section 7.5. Both stochastic solution approaches succeed in constructing high-order accurate stochastic solutions and computing probability density functions, regardless of the input variance. The stochastic Galerkin method typically converges faster than the sparse stochastic collocation method, in terms of the number of deterministic PDEs to be solved [BS09]. These deterministic PDEs result from a stochastic Galerkin projection or a stochastic collocation step applied to a stochastic PDE. Therefore, the trade off between solving a relatively smaller, but fully coupled system of deterministic PDEs versus solving many deterministic PDEs determines which method is most efficient. A good choice of iterative solver for the highdimensional algebraic systems resulting from a stochastic Galerkin discretization obviously influences the computational cost of the stochastic Galerkin method. In this chapter, the stochastic Galerkin and collocation methods are compared w.r.t. their convergence rate, i.e., how many deterministic PDEs need to be solved in order to achieve a certain level of accuracy, and computational cost.

162


Next to the efficiency of a stochastic solver, the robustness is an important quality. The stochastic Galerkin method is founded on a firm mathematical theory [BTZ04, BTZ05] ensuring a robust and stable convergence behavior. This yields, e.g., more flexibility in the adaptive selection of gPC basis functions [BS09, TLNE09] in comparison to the adaptive construction of a sparse stochastic collocation method.

7.3.2

Numerical illustration

In this section, a short comparison of the stochastic collocation and stochastic Galerkin method is given by applying both methods to the stochastic diffusion test case introduced in Section 3.7.2. First, a linear stochastic diffusion coefficient is tested, secondly a nonlinear one. The computations were performed on a quadcore Xeon 5420 CPU with 2.5 GHz and 8GByte of RAM. Remark 7.3.1. In order to compare the accuracy of the stochastic collocation and Galerkin finite element methods, an “exact” stochastic solution is needed. However, since for most problems no analytic solution can be calculated, a high-order stochastic solution is used as reference solution uref . This approach is widely applied in the literature [EB09, NT09, WK05, XH05]. The errors on the mean and variance of u are then respectively calculated as emean =

khui − huref ik2 , khuref ik2

evar =

kvar(u) − var(uref )k2 . kvar(uref )k2

(7.7)

Linear problem. The stochastic diffusion model problem (2.25)–(2.27) is solved on an L-shaped domain, see Fig. 3.10 (for the experiments in this chapter, we set the Dirichlet boundary condition to u = 1 instead of u = 10). The stochastic diffusion coefficient is given by expansion (3.13) with the same default parameters as in Section 3.7.2 and represented by a total of 8 uniformly distributed random variables with L1 = 2, L2 = 3 and L3 = 3. For the error computations, the sparse Gauss-Legendre stochastic collocation solution with Smolyak level 5 is used as reference. Fig. 7.2 illustrates the convergence of the mean and standard deviation as a function of either the polynomial chaos order, the polynomial degree to construct 1D collocation points or the Smolyak level. These three measures, polynomial order, degree and Smolyak level, are not equal to each other, but each one is a measure for the accuracy of the applied stochastic discretization scheme. As confirmed theoretically in [BNT07], an exponential convergence rate is visible for the stochastic Galerkin and stochastic collocation method. The sparse grid stochastic collocation method based on Clenshaw-Curtis points converges more slowly and less regularly. The full tensor product collocation method becomes rapidly impractical for a large polynomial degree, for example applying 4 collocation points in each random dimension results in 48 = 65 536 deterministic simulations. In order

relative error mean

full tensor stochastic collocation sparse Gauss stochastic collocation

0 10

sparse Clenshaw-Curtis collocation stochastic Galerkin method

10

10

−5

−10

6 2 4 0 8 polynomial order/degree/Smolyak level

relative error standard deviation

COMPARISON OF STOCHASTIC GALERKIN AND COLLOCATION METHODS

163

full tensor stochastic collocation sparse Gauss stochastic collocation 0 10

10

10

10


−2

−4

−6

6 2 4 0 polynomial order/degree/Smolyak level

full tensor stochastic collocation sparse Gauss stochastic collocation sparse Clenshaw-Curtis collocation

0 10 10 10 10

stochastic Galerkin method

−2

−4

−6

10

1

3 4 2 10 10 10 number of random unknowns



Figure 7.2: Relative error of the mean (left) and standard deviation (right) of the solution of the stochastic diffusion problem (2.25)–(2.27) solved on an L-shaped domain. full tensor stochastic collocation sparse Gauss stochastic collocation 0 10 10 10 10


−2

−4

−6

10

3 2 10 10 solution time (sec.)

1

4 10

Figure 7.3: Relative error of the standard deviation of the stochastic solution of the stochastic diffusion problem (2.25)–(2.27) solved on an L-shaped domain.

to compare the computational cost, Fig. 7.3 shows the relative error of the standard deviation as a function of the total number of stochastic degrees of freedom, and as a function of the total computational time. The stochastic Galerkin method outperforms the other methods as it constructs the most accurate solution in the least solution time. Note that the stochastic Galerkin systems are solved with the mean-based preconditioner, while the deterministic simulations in the stochastic collocation case are solved with AMG, accelerated by CG. Nonlinear problem. The effect of a polynomial nonlinearity on the performance of the stochastic Galerkin and the stochastic collocation method is illustrated by


10

0

full tensor stochastic collocation

relative error mean

sparse Gauss stochastic collocation stochastic Galerkin method 10

−5


164

10

10

10

10

−10

10

−15

4 1 2 3 polynomial degree/order/Smolyak level

0

full tensor stochastic collocation sparse Gauss stochastic collocation stochastic Galerkin method

−5

−10

4 1 2 3 polynomial degree/order/Smolyak level

Figure 7.4: Relative error of the mean (left) and standard deviation (right) of the stochastic solution of the stochastic diffusion problem (2.25)–(2.27) with nonlinear Gaussian diffusion coefficient (7.8) solved on an L-shaped domain.

replacing the linear stochastic diffusion coefficient (3.13) by  x ∈ D1  30 +10ξ1 +u2 (x, ξ) a(ξ1 , ξ2 , ξ3 , u(x, ξ)) = 5 +1.5ξ2 +u2 (x, ξ) x ∈ D2  100 +30ξ3 +u2 (x, ξ) x ∈ D3

(7.8)

Additional to these 3 random variables, a random variable is introduced in the right-hand side and the Dirichlet boundary condition: f (x, ω) = 1 + 0.1ξrhs and u(x, ω) = 1 + 0.1ξD , ∀x ∈ ∂DDirichlet. All random variables are assumed to be independent and standard normally distributed. The stochastic Galerkin method can easily deal with polynomial nonlinearities such as u2 [Mat08]. The stochastic collocation points are based on zeros of Hermite polynomials, i.e., Gauss-Hermite abscissa. Fig. 7.4 illustrates the convergence w.r.t. the polynomial degree, polynomial chaos order or Smolyak level. A sparse stochastic collocation solution with Smolyak level 4 is used as reference for the error computation. The convergence of the stochastic Galerkin and full tensor stochastic collocation method is also for this nonlinear problem very similar. The stochastic Galerkin method is more efficient than the full tensor product stochastic collocation method, as shown in Fig. 7.5. The Smolyak sparse grid stochastic collocation method requires for this example the least computational time to achieve a certain high-order accuracy.

7.4

Application: ferromagnetic rotating cylinder

We apply the stochastic collocation and Galerkin methods to construct high-order solutions of a nonlinear stochastic PDE representing the magnetic vector potential in a ferromagnetic rotating cylinder. This model can be used for design-

PSfrag

10 10 10 10

−4

full tensor stochastic collocation

full tensor stochastic collocation sparse Gauss stochastic collocation


−6

−8

−10

0

165

sparse Gauss stochastic collocation relative error mean

relative error mean

APPLICATION: FERROMAGNETIC ROTATING CYLINDER

10 10 10 10

1000 500 number of random unknowns

−4


−6

−8

−10

0

6000 2000 4000 computational time (sec.)

Figure 7.5: Relative error of the mean solution of the stochastic diffusion problem (2.25)–(2.27) on an L-shaped domain with nonlinear Gaussian coefficient (7.8).

ing solid-rotor induction machines in various machining tools and in magnetic brakes. At high speeds, when the surface layer of a ferromagnetic rotor gets fully saturated, solid-rotor induction machines and magnetic brakes produce a higher torque [DG09]. As a consequence, designing solid-rotor devices with high-speed conductive parts requires to take ferromagnetic saturation effects into account. These nonlinear material properties can typically not be quantified exactly. A reliable design needs to deal with uncertainty. This uncertainty is expressed by introducing stochastic variables into the mathematical model, which will take the form of a nonlinear stochastic partial differential equation.

7.4.1

Deterministic model

The magnetic field in a rotating cylinder can be described by an Eulerian formulation of the eddy-current model [DG09], ~ ~ − σ~v × ∇ × A ~ + σ ∂ A = J~s . ∇ × (ν∇ × A) ∂t

(7.9)

~ expresses the magnetic vector potential, which is related to the magnetic In (7.9), A ~ as B ~ = ∇ × A. ~ J~s is the applied current density, ν the reluctivity, flux density B σ the conductivity and ~v the mechanical velocity. This equation is applied to a 2D model of a ferromagnetic, hollow cylinder modelling the rotor of a cylindrical magnetic brake, as depicted in Fig. 7.6(a). The inner radius of the cylinder equals ri = 0.020m, the outer radius ro = 0.050m and the length lz = 1m. Given that the current is perpendicular to the plane and that the magnetic flux density lies in the plane, (7.9) can be rewritten in a cylindrical

166


ωm

ν, σ

2

ri ro

~ (T ) |B|

1.5

1

0.5

0 0

(a)

1

2

3

~ (A/m) |H|

4

5 5

x 10

(b)

~ Figure 7.6: (a) Geometry of the model problem with spectral-element grid. (b) |B|~ |H|-characteristic of the ferromagnetic material.

coordinate system (r, θ, z) as ∂Az ∂Az 1 ∂ ν ∂Az ∂Az 1 ∂ νr − + σωm +σ = Js,z , − r ∂r ∂r r ∂θ r ∂θ ∂θ ∂t

(7.10)

~ = (0, 0, Az ) and J~s = (0, 0, Js,z ). The cylinder rotates with an angular with A velocity ωm , which is for our problem in the range between 0 and 300 rad/s. The magnetic material behavior is assumed to be isotropic and nonlinear, i.e., ~ = ν(|B|) ~ B, ~ where the reluctivity ν depends on the magnitude of the magnetic H ~ = (Br , Bθ , 0), and H ~ = (Hr , Hθ , 0) represents the magnetic field flux density B ~ ~ strength. The deterministic |B|-|H|-characteristic is illustrated in Fig. 7.6(b). The conductivity σ is assumed to be constant. The cylinder is placed in a vertical DC ˆy . A non-magnetic steel is used as shaft. As a magnetic field with Bx = 0, By = −B consequence, the inner region carries a negligible flux. These operating conditions ˆy r0 cos(θ). are applied by the boundary conditions Az (ri , θ) = 0 and Az (ro , θ) = B The magnetic brake is operated without electrical excitation; Js,z = 0. The applied magnetic field is steady-state, allowing to omit the time-dependent term. Hence, a convection-diffusion PDE results: 1 ∂ ∂Az ∂Az 1 ∂ ν ∂Az − νr − + σωm = 0. (7.11) r ∂r ∂r r ∂θ r ∂θ ∂θ This rotor model can be embedded in a full machine model as described in [DG10]. Spectral discretization. Applying a spectral spatial discretization to (7.11) offers many advantages [DG09]. The numerical approximation converges faster than in the case of a finite element method and no stabilization techniques, such as upwinding, are needed. To apply a spectral discretization, the magnetic vector potential is discretized by using Chebyshev polynomials Tˆm (r) in the r-direction

APPLICATION: FERROMAGNETIC ROTATING CYLINDER

167

and harmonic functions e−ıλθ in the θ-direction, X X Az (r, θ) = Az,m,λ Tˆm (r)e−ıλθ ,

(7.12)

m∈M λ∈Λ

where M and Λ denote respectively the set of the orders of the Chebyshev polynomials and the harmonic functions, and ı is the imaginary unit. The functions Tˆm (r) correspond to Chebyshev polynomials Tm (s) with s ∈ [−1, 1], shifted and scaled to r ∈ [ri , ro ], i.e., 1 r ˆ Tm (r) = Tm . (7.13) ln em rm √ Here, rm = ri ro is the geometric mean radius p of the hollow cylinder with inner radius ri and outer radius ro , and em = ln ro /ri is a form factor. The spectral spatial discretization method proceeds by collocating the error at the nodes of a tensor product grid, combining Chebyshev points in the r-direction and equidistantly distributed points in the θ-direction, see Fig. 7.6 (a). Linearization. The reluctivity ν is linearized with Newton’s method. The Jacobian of the spectral discretization of (7.11) is expressed in terms of the differential reluctivity tensor ν¯ ¯d [DG09]. This tensor, which is function of r and θ, equals ! dν 1 0 B r Br Bθ . (7.14) +2 ν¯d = νc 2 Bθ 0 1 ~ d|B|

~ and The chord reluctivity νc (|B|)

d ~ ~ 2 ν(|B|) d|B|

have to be evaluated from the mate-

rial characteristic for each spatial collocation point (rp , θp ) according to the local ~ of the magnetic flux density, which follows from the current apmagnitude |B| proximation to the solution Az . At Newton step k + 1, the following linear system needs to be solved: (k) (k) = −Gdu Hf . (7.15) Gdu Mν¯d Gpr + Wconv A(k+1) z

In (7.15), the matrices Gpr ∈ RN ×2N , Gdu ∈ R2N ×N and Wconv ∈ RN ×N represent respectively the primary curl matrix, the dual curl matrix and the convection ma(k) trix, with N the number of spatial degrees of freedom. The matrix Mν¯d ∈ R2N ×2N is a 2-by-2 block matrix, composed of four diagonal blocks that respectively cor(k) (k) (k) (k) respond to ν¯d,rr , ν¯d,rθ , ν¯d,θr and ν¯d,θθ , i.e., the four components of the reluctivity (k) tensor ν¯ ¯d (7.14). Each entry of the right-hand side Hf ∈ R2N ×1 represents the magnetic field strength at the crossing point of the tangent linethat linearizes (k) (k) (k) (k) (k) ~ ~ the |B|-|H|-curve in the operation point Bp : Hf,p = νc,p − ν¯d,p Bp . Further details on the linearization can be found in [DG09]. In [DG09], also a comparison of the solution resulting from a linear and a nonlinear reluctivity is given in order to determine the effect of the nonlinearity induced by ν.

168


Motional eddy current density. Because of the rotation of the cylinder, a current density is induced, which is equal to Jz = −σωm

∂Az . ∂θ

(7.16)

Torque calculation. The simulation is set up to compute the torque as a function of the mechanical velocity. Using the Lorentz-force method, the torque around the rotor axis is computed from the θ-component of the force density fθ as Z ro Z 2π M z = lz fθ r2 dθdr. (7.17) ri

0

For an incompressible, nonlinear, isotropic material, the θ-component of the force density is given by [WM68] fθ = Jz Br +

~ ~ ∂|H| 1 ∂wmagn,co |B| − , r ∂θ r ∂θ

with

wmagn,co =

Z

0

~ H

~ · dH, ~ (7.18) B

~ |B| ~ and where r is the radius and wmagn,co the magnetic co-energy. Note that |H|, wmagn,co are scalar fields. The first term in (7.18) is the Lorentz force whereas the other terms constitute the reluctance force. If the material distribution is homogeneous and invariant in the direction of motion, the reluctance force vanishes. Here, the nonlinearity introduces a heterogeneity in the permeability. The torque calculation by volume integration combined with a spectral-element method results in highly accurate torque values. The alternative Maxwell stress approach is known to be less accurate and is therefore not considered here.

7.4.2

Stochastic model

Random variables ξ1 , ξ2 , ξ3 are introduced in (7.11) to represent the variability of the material parameters and the imposed boundary conditions. This yields 1 ∂ ∂Az ∂Az 1 ∂ ξ1 ν ∂Az − ξ1 νr − + ξ2 σωm = 0, (7.19) r ∂r ∂r r ∂θ r ∂θ ∂θ for r ∈]ri , ro [ satisfying (7.13) and θ ∈ [0, 2π[. Periodic boundary conditions are applied in the θ-direction, while for r = ri and r = ro Dirichlet boundary conditions hold, Az (ri , θ, ξ) = 0

and

ˆy ro cos(θ). Az (ro , θ, ξ) = ξ3 B

(7.20)

In (7.19)-(7.20), ξ is a random vector with joint probability density function ̺(y), y ∈ Γ, collecting the random variables present in the model. Besides ξ1 , ξ2 and

STOCHASTIC GALERKIN DISCRETIZATION

169

~ H|-description. ~ ξ3 , additional random variables are introduced in the |B|-| The nonlinear material characteristic is approximated by a tanh magnetization curve, see Fig. 7.6. As adjustable parameters, we use the magnetic field strength Hknee at a so-called knee point, the reluctivity νfinal at the fully saturated range and two 1 2 reference values Bref and Bref for the magnetic flux densities: ~ = C(|B|)(ξ ~ ~ |H| 4 Hknee + ξ5 νfinal |B|)  ! ~ 1 | B| ~ = tanh with C(|B|) + tanh 1 2 ξ6 Bref

(7.21) ~ |B| 2 ξ7 Bref

!30 

.

These parameters are randomized by respectively ξ4 , ξ5 , ξ6 and ξ7 . Note that this ~ H|-characteristics, ~ stochastic model can easily be extended to other |B|-| in which case other parameters can be perturbed in a similar way. Since the reluctivity ν is nonlinear, it depends on the entire ξ-vector. All random variables are assumed to be independent. The stochastic collocation solution of (7.19)–(7.20) proceeds according to the description in Section 7.2, with the operator L given by the nonlinear convectiondiffusion operator in (7.19) and u replaced by Az . The stochastic Galerkin method needs to be tuned in order to represent the stochastic nonlinearity with a polynomial chaos expansion. This procedure is explained in the next section. Remark 7.4.1. Both the stochastic Galerkin and collocation method employ a global polynomial approximation to represent Az as a function of the random input variables. This can only be justified when the behavior of the solution does not abruptly change with respect to the random input parameters. Otherwise, local polynomial approximations could be a solution, e.g., a multi-element generalized chaos or multi-wavelet expansion as discussed in [BTZ05, WK05, LNGK04] for the stochastic Galerkin case or a multi-element probabilistic collocation approach as presented in [FWK08, MZ09, WB08a]. The derivation of the discretized deterministic systems remains the same in the case of a multi-element polynomial approximation. Only the stochastic Galerkin discretization matrices and the position of the stochastic collocation points will change. For the relatively small uncertainty on the input parameters considered here, a smooth behavior of the solution is observed, thus enabling the use of a global polynomial approximation.

7.5

Stochastic Galerkin discretization

The stochastic Galerkin method proceeds as described in Section 2.4.1. First, a gPC expansion of the stochastic solution Az is constructed as in (2.21). Next, a Galerkin projection is applied to the stochastic residual, see (2.22). For general nonlinearities, the integral in (2.22) cannot be calculated analytically [Mat08]. It

170


can either be numerically approximated or the problem can be linearized first before applying the Galerkin condition. In the former case, linearization can be avoided by applying a quasi-Newton method [MK05]. The latter approach enables one to apply standard stochastic Galerkin techniques for linear stochastic PDEs to the linearized problem. Since a Newton linearization is already available for the deterministic problem, the second approach is considered by extending the Newton linearization to the stochastic Galerkin case.

7.5.1

Newton linearization

Differential reluctivity tensor The differential reluctivity tensor ν¯d , defined in (7.14), becomes for the stochastic model (7.19) a random field that depends on r, θ and ξ. Based on the gPC representation of Az , ν¯d can be reformulated by a similar expansion: ν¯ ¯d (r, θ, ξ) =

∞ X

¯(q) (r, θ)Ψq (ξ). ν¯ d

(7.22)

q=1

Note that expansion (7.22) will be truncated after q = Qν = (L + 2P )!/L!(2P )! terms since the high-order terms cancel out after applying the stochastic Galerkin projection, due to the orthogonality properties of the polynomials Ψq . This corresponds to using a gPC expansion of order 2P , with P the polynomial order for approximating the solution in (2.21). (q) The coefficient functions ν¯d are calculated based on definition (7.14), after repre~ by a gPC expansion: senting B

~ = (Br , Bθ , 0) ≈ B

Q X

(q)

(Br(q) , Bθ , 0) Ψq (ξ),

(q)

with

Br(q) =

(7.23)

q=1

1 ∂Az r ∂θ

(q)

and Bθ

(q)

=−

∂Az . ∂r

Equation (7.23) immediately follows from the gPC expansion of Az and the relation ~ = ∇ × A. ~ The stochastic differential reluctivity tensor ν¯¯d is thus given by B ν¯ ¯d (r, θ, ξ) = νc (r, θ, ξ)

2

Q Q X X i=1 j=1

"

1 0 0 1 (i)

Br (i) Bθ

+

#

dν(r, θ, ξ) ~ 2 d|B|

!

h

(j) Br

(j) Bθ

i

Ψi Ψj . (7.24)


171

This tensor is to be computed for each spatial collocation point (rp , θp ) and evaluated for the current approximation to the solution Az . In order to represent (7.24) by expansion (7.22), a separate gPC representation for the two terms in the right-hand side of (7.24) is constructed. Consider first a gPC representation for the chord reluctivity νc (r, θ, ξ), νc (r, θ, ξ) =

∞ X

νc (r, θ)(q) Ψq (ξ)

q=1

with

νc (r, θ)(q) = hνc (r, θ, ξ)Ψq i/hΨ2q i. (7.25)

The orthonormality of the polynomials Ψq yields that hΨ2q (ξ)i ≡ 1. The chord ~ B|, ~ where the magnitude of B ~ is given by reluctivity is defined as νc = |H|/| v u Q Q uX X (i) (j) (i) (j) ~ |B| = t Br Br + Bθ Bθ Ψi Ψj , (7.26) i=1 j=1

~ depends on the definition of the magnetias follows from (7.23), and where |H| ~ cannot be exactly represented by a gPC expansion zation curve (7.21). Since |B| – due to the square root in (7.26) – the gPC coefficients νc (r, θ)(q) need to be calculated approximately by using numerical integration methods. Therefore, no − → separate gPC expansion is constructed for | B |, but (7.26) is applied directly to evaluate νc in the numerical integration procedure. The numerator hνc (r, θ, ξ)Ψq i in (7.25) corresponds to an L-dimensional integral. It can be approximated by using sparse cubature rules [Kee04, MK05]: Z Z νc (r, θ, z)Ψq (z)̺(z)dz ... hνc (r, θ, ξ)Ψq (ξ)i = ΓL

Γ1

≈

Nc X

wk νc (r, θ, ζk )Ψq (ζk )ˆ ̺(ζk ),

(7.27)

k=1

where {ζ1 , . . . , ζNc } is a set of Nc integration points with corresponding weights w1 , . . . , wNc . The construction of the integration points is similar to the sparse grid construction described in Section 7.2.2 based on Gauss points. The scaling factor ρˆ(ζk ) takes the difference between the weighting function used to compute the Gauss points and the probability density function ̺(z) into account. For example, in the case of L random variables ξi , uniformly distributed on [−1, 1], and Gauss-Legendre cubature points, ̺ˆ = 1/2L . Next, a gPC representation for the second term in (7.24) is constructed. A small ~ 2 is equal to derivation shows that the derivative dν/d|B| ~ ~ − νc B| d|H|/d| dν = , ~ 2 ~ 2 d|B| 2|B|

172


~ ~ can be determined from the derivative of the magnetization curve where d|H|/d| B| ~ As before, dν/d|B| ~ 2 cannot exactly be represented by a gPC expansion. w.r.t. |B|. ~ 2 , instead an Therefore, no separate gPC expansion is constructed for dν/d|B| approximate gPC expansion is numerically computed for the entire second term in (7.24). The gPC coefficients are calculated by sparse high-dimensional cubature rules similar to (7.27). Linearized system The gPC representation of ν¯d (7.22) together with a spectral spatial discretization applied to (7.19) results in the following linearized system at Newton step k + 1: ∞ X i=1

(k) Ψi Gdu M ¯(i) Gpr ν

∞ X

=−

+ (1 + ξ2 )Wconv

d

i=1

!

Q X

A(q)(k+1) Ψq z

q=1

!

! Q ! X (k) (k) (q)(k) Ψi Gdu M (i) − M ¯(i) Gpr Ψq , Az νc

ν ¯d

q=1

(k)

where M ¯(i) ∈ R2N ×2N is composed of 4 diagonal (N × N )-matrices containing the ν ¯d

(k)(i) (k)(i) (k)(i) (k)(i) (k)(i) components of the reluctivity tensor ν¯d , given by ν¯d,rr , ν¯d,rθ , ν¯d,θr , ν¯d,θθ , on the diagonal. The matrices Gdu , Gpr and Wconv result from discretizing the spatial differential operators, see (7.15). The computation of the right-hand side relies on gPC expansion (7.25) for the chord reluctivity νc .

Galerkin condition The stochastic Galerkin method proceeds by imposing orthogonality of the residual w.r.t. the polynomial chaos. This yields a linear algebraic system of N Q equations: ! Qν X (k) (7.28) Ci ⊗ Gdu M ¯(i) Gpr + (IQ + Cξ2 ) ⊗ Wconv A(k+1) z ν ¯d

i=1

=−

Qν X i=1

(k) (k) Ci ⊗ Gdu M (i) − M ¯(i) Gpr Az(k) , νc

ν ¯d

(1)(k) A(k) . . . A(Q)(k) ]T . z = [Az z

The matrix IQ = hΨΨT i ∈ RQ×Q is an identity matrix, as follows from the orthonormality of the polynomials Ψq . The matrices Ci , Cξ2 ∈ RQ×Q characterize the stochastics of the problem and are respectively defined by Ci = hΨi ΨΨT i and Cξ2 = hξ2 ΨΨT i. Properties of these matrices are given in Section 2.5. Each


173

PQν individual matrix is a sparse matrix, however the sum, i=1 Ci , is a full matrix [Kee04]. Due to the orthogonality of the polynomials, all matrices Ci with )! ¯¯d (7.22) and index i > (L+2P L!(2P )! are identically zero so that the gPC expansions of ν )! νc (7.25) can be truncated after Qν = (L+2P L!(2P )! terms. This explains the range of the summations in (7.28). The gPC expansions (7.22) and (7.25) can therefore be limited to a gPC order 2P , with P the order used for approximating the solution.

7.5.2

Computational aspects

Existence of a solution and convergence Newton’s method applied to the deterministic problem (7.11) converges generally within 10 to 20 iterations. A similar convergence is observed for the stochastic nonlinear problem. This property does only hold however when the introduction of random variables into the model does not violate the necessary conditions so that a solution to (7.19) exists. These conditions include that the stochastic coefficient ν(r, θ, ξ; Az ) is bounded and strictly positive [BTZ04], i.e., that there exist positive constants α and β such that E(0 < α ≤ ν(r, θ, ξ; Az ) ≤ β < ∞) = 1 ∀(r, θ) ∈ [ri , ro ] × [0, 2π[, where E is the probability measure used to describe the probability space on which the random variables ξ are defined. In practice, we observed that the convergence of the Newton’s iteration (7.28) can deteriorate in the presence of roundoff and approximation errors. Especially the computation of the gPC representations for νc (7.25) and ν¯¯d (7.22) is a potential source of roundoff errors: the numerical integration has to be accurate enough. The Smolyak cubature formulas applied in (7.22) and (7.25) integrate multivariate polynomials exactly if their total polynomial degree is at most 2w + 1 [MK05, NR96], with w the Smolyak level as defined in (7.6). Beside these errors it is also important to note that certain parameter variations do not yield convergent Newton iterations. This issue originates from the description of the magnetization curve: also for the deterministic problem not all parameter combinations lead to 1 convergent Newton iterations. Therefore, for some parameters, for example Bref , only small variances of the corresponding random variables are allowed. Initial guess At each Newton iteration (7.28), a large algebraic system needs to be solved. A good initial guess can therefore reduce the computational time significantly. The

174


all-zero initial guess: total time = 8187 sec. P = 4, Q = 126 13 Newton steps hierarchically constructed initial guess: total time = 1984 sec. P = 4, Q = 126 2 Newton steps intermediate Newton steps for p < P : 13 steps for p = 0, Q = 1 6 steps for p = 1, Q = 6 3 steps for p = 2, Q = 21 3 steps for p = 3, Q = 56 Table 7.1: Number of Newton steps and total computational time when solving the stochastic problem (7.19) starting from an all-zero initial guess for Az or an iteratively refined initial guess.

hierarchical structure of the polynomial chaos yields a straightforward procedure for determining a good initial guess: • set Az = [ ] • for p from 0 to P : – initial guess Aˆz = [ATz 0T ]T ∈ RN Q×1 , N Qp ×1 , an all-zero vector, Qp = with Q = (L+p)! L!p! and 0 ∈ R

(L+p−1)! (L−1)!p!

– solve the nonlinear stochastic problem (7.19) with initial guess Aˆz for Az represented by a gPC expansion of order p Note that the first iteration, with polynomial order 0, corresponds to solving a deterministic problem. The next example illustrates the possible reduction in computational time by using the above initialization procedure. Example 7.5.1. Problem (7.19) is solved with 5 uniformly distributed random variables: ξ1 on [0.97, 1.03], ξ2 and ξ3 on [0.99, 1.01], ξ5 and ξ6 on [0.98, 1.02]. ˆy ro = 1 Tm. The The boundary condition (7.20) is imposed with magnitude B 7 conductivity σ is equal to 10 S/m. The spatial discretization, represented in Fig. 7.6, uses 16 degrees of freedom in both the r- and θ-dimension, corresponding to N = 256 spatial unknowns. The angular velocity ωm equals 291 rad/s. In Table 7.1, a comparison of the number of Newton steps and the total computational time is given between using an all-zero initial guess or an initial guess created by the described procedure. The computations were performed on a 2.00 GHz Intel Dual Core processor T7300 with 2.0 GByte RAM.


175

Solving the linearized systems A spectral spatial discretization of the linearized deterministic problem (7.10) leads to small, but almost dense systems as in (7.15). The systems are non-symmetric and ill conditioned. To solve these systems, direct solution methods are generally most appropriate [CQHZ06]. The stochastic Galerkin method results after linearization in a Kronecker product system matrix (7.28) containing such dense blocks. For a small number of spatial and random unknowns, direct solution methods can be applied. The number of random unknowns however rapidly increases with increasing polynomial order. For increasing system sizes, the computational cost and memory requirements of direct solution methods become prohibitively large. In that case, iterative solution schemes are required for solving (7.28). In Chapter 3, multigrid and block splitting iterative solvers for SFEM discretizations with the same Kronecker product structure as (7.28) are presented. For spectral spatial discretizations, multigrid methods can be designed [ZWH84]. However, the improvement in convergence rate compared to single grid schemes is rather modest, especially for variable-coefficient problems [CQHZ06]. Therefore, only Ci splitting methods are considered here. These methods split only the stochastic discretization matrices, i.e., the Ci and Cξ2 matrices in (7.28), so that in every iteration, several (N × N )-systems need to be inverted. The block systems can be factorized once at the beginning of the block splitting iteration, so that in every iteration only triangular solves of the factorized system have to be carried out. The convergence rate of block splitting methods typically decreases with increasing polynomial order or input variance. In practice however, depending on the chaos type used, low computing times can be obtained. We apply these splitting methods as preconditioner for GMRES. In Table 7.2, the performance of some block splitting based preconditioners is illustrated. The mean-based preconditioner, the Kronecker product preconditioner and a symmetric block Gauss-Seidel splitting are considered. The angular velocity ωm is set to 151 rad/s. The spatial discretization uses 16 degrees of freedom in the r- and θ-dimension, corresponding to N = 256 spatial unknowns. The stochastic model contains 6 random variables (ξ1 , ξ3 , ξ4 , ξ5 , ξ6 and ξ7 ), either lognormally (configuration A) or uniformly (configuration B) distributed. The former corresponds to a Hermite expansion of the solution, the latter to an expansion with Legendre polynomials. A second-order and a third-order chaos is considered, corresponding to Q = 28 and Q = 84 random unknowns, respectively. In configuration (A) the lognormal distribution of the random variables is based on a Gaussian distribution with mean µg = 0 and standard deviation σg = 0.05. In configuration (B) the random variables are uniformly distributed on [0.95, 1.05]. The linear systems are solved to a relative accuracy equal to 10−12 . Table 7.2 presents the average number of iterations and solution time needed for solving the linearized systems, when performing several Newton iterations until convergence of

176


Number of iterations lognormal (A) uniform (B) chaos order 2 mean-based Kronecker product symmetric GS chaos order 3 mean-based Kronecker product symmetric GS

Solution time (seconds) (A) (B)

41.8 40 16.4

23.5 23 11.8

18.0 17.3 183

9.9 10.2 149

69 65 22.6

29.7 30.7 14

344 362 3958

176 189 2724

Table 7.2: Average number of iterations and solution time for solving one linearized system (7.28) with GMRES, preconditioned by a mean-based preconditioner, a Kronecker product preconditioner and a symmetric Gauss-Seidel (GS) Ci -splitting preconditioner.

the nonlinear solution. The initial guess for the Newton iterations was constructed with the hierarchical refinement procedure described above. The tests indicate that the mean-based preconditioner yields the best performance. In [Ull10] it was reported that the convergence of the Kronecker product preconditioner is less sensitive to large variations of the random variables than the convergence of the mean-based preconditioner. Therefore, such a preconditioner might perform better than the mean-based preconditioner when the random variables have a large variance. Because in the mean-based and Kronecker product preconditioner case, all block systems use the same system matrix, these methods result in a lower computational cost compared to the block Gauss-Seidel splitting method. The block system matrix needs to be factorized only once and the factorized blocks are then used to apply the preconditioner. The mean-based preconditioner is used in subsequent numerical experiments.

7.6

Numerical comparison for ferromagnetic problem

In this section, the computational cost and accuracy of the stochastic Galerkin and collocation methods applied to the stochastic ferromagnetic cylinder problem are numerically compared. This section focuses on solving the magnetic vector potential from (7.19). The mechanical velocity ωm remains fixed and was set to ωm = 151 rad/s in the experiments. In the next section, the mechanical velocity is varied and the statistics of the torque are computed. As the torque computations are directly related to the magnetic vector potential, its accuracy obviously follows

NUMERICAL COMPARISON FOR FERROMAGNETIC PROBLEM

177

the accuracy of the latter. The second set of experiments is used to reveal the influence of uncertainty on the magnetic material parameters onto the torque. The Newton iterations for both stochastic are performed until the relative

new solvers

init

Az − Aold norm of the Newton increments, k/kA , is smaller than 10−12 z z 2

new

old −2 and the absolute norm Az − Az 2 < 10 . The linear systems within a Newton step are solved until the relative residual is smaller than 10−12 . When a larger tolerance is used for the linear systems – as is often done in a NewtonKrylov strategy – errors tend to accumulate and disturb the Newton convergence, especially in the stochastic Galerkin case. A spatial discretization based on 16 degrees of freedom in the r- and θ-dimension is used, which corresponds to N = 256 spatial unknowns. The mean values of the parameters of the magnetization curve (7.21) equal Hknee = 5π −1 · 104 A/m, 1 2 νfinal = 2.5π −1 · 107 m/H, Bref = 10 T and Bref = 0.7 T. The computations are performed on a 2.2 GHz Dual Core Opteron processor with 4 GByte RAM.

7.6.1

Lognormal random variables and Hermite polynomials

In the first set of experiments, the stochastic model (7.19) is perturbed by 4 independent, lognormally distributed random variables, ξ = {ξ1 , ξ4 , ξ5 , ξ7 }, all based on a Gaussian variable with µg = 0 and σg = 0.08. The stochastic Galerkin solution and the lognormal random variables are approximated by a Hermite polynomial expansion with standard normal random variables. The stochastic collocation method constructs 4D collocation points from a full tensor product or a Smolyak sparse grid based on the zeros of Hermite polynomials. The error computation in (7.7) applies a stochastic collocation solution corresponding to a full tensor product grid of 4096 collocation points, based on the zeros of 7th degree Hermite polynomials, as reference solution. Convergence and accuracy. In Fig. 7.7, the error of the mean and variance of Az is presented as a function of respectively the polynomial order in the stochastic Galerkin case, the polynomial degree pi in the dense stochastic collocation case, or the Smolyak level w in the sparse stochastic collocation case. The same polynomial degrees pi , i = 1, . . . , L, are used for all random variables. We observe that the convergence rate of the stochastic Galerkin and full tensor product stochastic collocation solutions is fairly similar. Although the stochastic discretization is based on scaled Hermite polynomials, according to the normal distribution of the random variables used to approximate the lognormal random variables, neither of the methods attain an exponential convergence rate w.r.t. to the polynomial degree or polynomial order. In [BNT07], the stochastic collocation method applied to a linear diffusion problem with lognormal diffusion coefficient yielded an exponential

178


−3

−4

−1

stochastic Galerkin full tensor collocation sparse Gauss collocation

10

−5

10

−6

10

4 1 2 3 5 polynomial order/degree/Smolyak level

10 relative error variance

relative error mean

10

−2

10

−3

10

−4

stochastic Galerkin full tensor collocation sparse Gauss collocation −5 10 4 1 2 3 5 polynomial order/degree/Smolyak level 10

Figure 7.7: Error of the mean and variance of Az as a function of respectively the polynomial order in the stochastic Galerkin case, the polynomial degree pi in the full tensor product stochastic collocation case, or the Smolyak level w in the sparse stochastic collocation case. Problem (7.19) is solved with ξ1 , ξ4 , ξ5 and ξ7 modelled as independent, lognormally distributed random variables.

convergence of the error w.r.t. polynomial degree. These results apparently do not extend to this nonlinear convection-diffusion problem. A lack of exponential decay of the stochastic collocation error for nonlinear problems was also observed in [BBCF07]. There, it was suggested that the total error was dominated by the discretization error in the physical space. Computational cost. While Fig. 7.7 illustrates the convergence rate, it does not give information on the computational cost of the various methods. The computational cost is related to the number of stochastic unknowns, i.e., the number of deterministic PDEs to be solved. The number of stochastic unknowns that corresponds to a polynomial order P , i.e., Q (2.11), is different from the number of unknowns Nc that results from applying the same value as Smolyak level or polynomial degree when constructing stochastic collocation points. More importantly, the computational cost of the stochastic Galerkin method for a certain number of stochastic unknowns is generally substantially higher than the cost of the stochastic collocation method for the same number of unknowns due to the coupling of the stochastic unknowns in the Galerkin case. In order to illustrate the computational cost, Fig. 7.8 displays the decay of the error as a function of either the number of stochastic unknowns or the total computational time. For this problem, the stochastic Galerkin method is more expensive than the stochastic collocation method to attain a similar accuracy. Remark 7.6.1. It is remarkable that a sparse grid construction of stochastic collocation points leads for a same number of random unknowns to a substantially less accurate solution than a full tensor product collocation point grid. It is possible that the number of stochastic dimensions, i.e., only 4, is not large enough for


−3

−3

10


relative error mean

relative error mean

10

−4

10

−5

10

−6

10

179


−4

10

−5

10

−6

1

2

3

10 10 10 number of random unknowns

10

1

10

2

3

4

10 10 10 total computational time (sec.)

Figure 7.8: Error of the mean of Az as a function of either the number of random unknowns (left) or the total computational time (right). Problem (7.19) is solved with ξ1 , ξ4 , ξ5 and ξ7 lognormally distributed.

demonstrating the effectiveness of the sparse collocation method over the dense collocation method. Also note that the convergence rate of the sparse stochastic collocation solution apparently slows down for the higher Smolyak levels. This might be an artefact of using an insufficiently accurate reference solution possibly together with a spatial discretization error limit, as also occurred in [EB09, FYK07]. The reference solution was chosen experimentally as the best possible reference out of the set of obtained solutions.

7.6.2

Uniform random variables and Legendre polynomials

The stochastic model (7.19)-(7.20) is considered with 7 random variables, uniformly distributed on [0.97, 1.03]. This leads in the stochastic Galerkin case to a Legendre polynomial expansion of the solution. The stochastic collocation points are constructed in a sparse Smolyak or a full tensor product grid framework, based on 1D Clenshaw-Curtis or Gauss-Legendre points. In the full tensor product grid case, the 1D grids correspond to Gauss-Legendre points. Fig. 7.9 illustrates the mean and variance of the z-component Az of the magnetic vector potential. A stochastic collocation solution based on Nc = 67 = 279 936 stochastic GaussLegendre collocation points is used as reference for the error computations. Convergence and accuracy. In Fig. 7.10, the error of the mean and variance of Az is presented as a function of either the polynomial order, or the degree of the polynomials to construct the Gauss points in the stochastic collocation case, or the Smolyak level. An exponential decay of the error as a function of the polynomial degree or the polynomial chaos order is observed. This is consistent with the exponential convergence of the stochastic Galerkin and collocation solution reported

180


variance (T 2 m2 )

mean (T m)

0.05

0

-0.05 0.05

−7 ×10 8 6 4 2

0 0.05 0.05

0 0

0.05 -0.05 y (m)

-0.05

0

y (m)

x (m)

0 -0.05 -0.05

x (m)

Figure 7.9: Mean (left) and variance (right) of Az , the stochastic Galerkin solution of (7.19)-(7.20) with 7 random variables, all uniformly distributed on [0.97, 1.03]. The solution is based on a third-order Legendre chaos. −2

relative error mean

−6

10

stochastic Galerkin full tensor collocation sparse Gauss collocation sparse Clenshaw-Curtis

−8

10

−10

10

−12

10

4 1 2 3 5 6 polynomial order/degree/Smolyak level

10 relative error variance

−4

10

−4

10


−6

10

−8

10

−10

10

4 1 2 3 5 6 polynomial order/degree/Smolyak level

Figure 7.10: Error of the mean and variance of Az as a function of either the polynomial order, or the polynomial degree, or the Smolyak level, depending on the type of stochastic discretization. Problem (7.19) contains 7 independent and uniformly distributed random variables on [0.97, 1.03].

in [BNT07, XK02a] which results from the correspondence between the weighting function of Legendre polynomials and the uniform probability density function. Computational cost. The computational cost is illustrated in Fig. 7.11 which displays the error decay as a function of either the number of stochastic unknowns or the total computational time. The stochastic Galerkin method clearly results in the most accurate solution for a fixed number of random unknowns. With respect to the total solution time however, the stochastic Galerkin method does no longer perform better than the stochastic collocation method due to the large cost of solving the high-dimensional linearized systems.


relative error variance

−5

10

−10

10


0



0

10

181

10

−5

10

−10

2

4

10 10 number of random unknowns

10

1

10

2

3

4

5

10 10 10 10 total computational time (sec.)

−1

10


−3

10

−5

10



Figure 7.11: Error of the variance of Az as a function of the number of stochastic unknowns (left) and of the total solution time (right). Problem (7.19) contains 7 independent and uniformly distributed random variables on [0.97, 1.03].

−1

10


−3

10

−5

10

4 1 2 3 polynomial order/degree/Smolyak level

1 10

4 102 103 10 105 total computational time (sec.)

Figure 7.12: Error of the variance of Az as a function of the polynomial degree, order or Smolyak level (left), and as a function of the total computational time (right). Problem (7.19) is solved with ξ1 , ξ2 , ξ3 ξ5 and ξ6 modelled as independent, uniformly distributed random variables, and ξ4 and ξ7 as lognormally distributed random variables.

7.6.3

Combination of distributions

In a third set of experiments, the stochastic model (7.19) contains 7 random variables: ξ1 , ξ2 , ξ3 ξ5 and ξ6 are uniformly distributed on [0.99, 1.01]; ξ4 and ξ7 are lognormally distributed based on a Gaussian random variable with µg = 0 and σg = 0.05. Both Hermite and Legendre polynomials are used for representing the stochastic Galerkin solution and for calculating the stochastic collocation points. The reference solution is based on a sparse collocation grid with Smolyak level 7 and 163 213 stochastic collocation points. Fig. 7.12 illustrates the relative error of the variance of Az . In contrast to the former example with lognormal random variables in Section 7.6.1, the error of

182


the stochastic collocation and Galerkin solutions decreases exponentially, except for the last result of the sparse collocation solution. The use of Legendre and Hermite polynomials to approximate the solution could explain this convergence rate due to the above-mentioned relationship between these polynomials and the probability density function of the random variables. This example may indicate that the first example with lognormal random variables is not representative for the general convergence behavior of the discussed stochastic solution methods. Secondly, remark that the sparse stochastic collocation method achieves the most accurate solution for this example, although its convergence rate is not completely monotone. This latter effect could be a result of an insufficiently accurate reference solution, as also encountered in Remark 7.6.1.

7.7

Influence of uncertainty on the torque

In this section, we describe how uncertainty on the material properties influences the torque statistics. The mean behavior of the torque corresponds in each case to the solution of the deterministic model. The propagation of the input uncertainty to the uncertainty on the corresponding torque motivates the use of stochastic models for reliably designing machining tools composed of a ferromagnetic rotor. First, the uncertainty is modelled by only one random variable - either ξ1 or ξ2 . Next, we consider the effect of uncertainty on the parameters of the magnetization curve, i.e., ξ4 up to ξ7 .

7.7.1

Computation of the torque

Stochastic collocation method. In the stochastic collocation case, for each collocation point a deterministic torque is computed with formula (7.17). The statistics of the torque can be computed from (7.3)–(7.4). Stochastic Galerkin method. Since the torque (7.17) indirectly depends on Az , a gPC representation for Mz can be constructed as follows: Mz ≈

Qν X i=1

Mz(q) Ψq ,

INFLUENCE OF UNCERTAINTY ON THE TORQUE

(q)

where the coefficients Mz Mz(q) = − lz σωm

+ lz

are given by

Q Q X X i=1 j=1

hξ2 Ψq Ψi Ψi i

Z Q Q X X hΨq Ψi Ψi i i=1 j=1

− lz

Z

ro

ri

Z

0

2π

183

ro ri

Z

Z

0

ro

ri

2π

Z

2π 0

(i)

∂Az Br(j) r2 dθdr ∂θ

~ (j) ~ (i) ∂|H| |B| r2 dθdr r ∂θ

(q)

1 ∂wmagn,co 2 r dθdr, r ∂θ

z ~ with Jz replaced by ξ2 σωm ∂A ∂θ and |B| given by (7.26). This gPC expansion ~ ~ is based on gPC expansions for |B|, |H| and wmagn,co , which can be constructed approximately by using the definition of the magnetization curve (7.21) and sparse cubature rules, as in (7.27). From this representation, statistics of the torque can be computed, in a similar way to (2.23). Alternatively, one could sample the gPC representation of the solution Az , and apply a Monte Carlo approach to determine the statistics of the torque.

7.7.2

One random variable

First, the case of one random variable, ξ1 , is considered. In the left column of Fig. 7.13, the mean, standard deviation and probability density function√of the torque √ is illustrated for the case ξ1 is uniformly distributed on [1 − 0.08 3, 1 + 0.08 3]. The results for the stochastic collocation and stochastic Galerkin method are given. The mean corresponds to the solution of the original deterministic problem. In case of only 4 random unknowns, the stochastic collocation and Galerkin results visually coincide. We observe that the coefficient of variation of the torque – which corresponds to the ratio of the standard deviation and the mean, is of the same order of magnitude as the input coefficient of variation. The right column of Fig. 7.13 shows the results for the case where ξ1 is lognormally distributed, based on a Gaussian variable with standard deviation σg = 0.08 and mean µg = 0. A similar standard deviation of the torque results from using the same input standard deviation in the uniform case as well as for the underlying Gaussian variable in the lognormal case. This suggests that the variance of the torque is mainly determined by the input variances and not by the type of probability density function. Next, we consider the case of uncertainty on the conductivity, as expressed by ξ2 . The left column √ presents the results for ξ2 either uniformly distributed √ of Fig. 7.14 on [1 − 0.08 3, 1 + 0.08 3], or lognormally with µg = 0 and σg = 0.08. The


mean torque (Nm)

0 -2000

stochastic stochastic stochastic stochastic

collocation (2) collocation (4) Galerkin (2) Galerkin (4)

-4000 -6000 -8000

0 mean torque (Nm)

184

0

-6000 -8000

0

100 300 200 mechanical velocity (rad/s) 200 standard deviation (Nm)

200

150

100

50

0 0

0.15

100 200 mechanical velocity (rad/s)

stochastic Galerkin (2) stochastic Galerkin (4)

0.05

0 -4300

stochastic collocation (2)

150

stochastic Galerkin (2) stochastic Galerkin (4)

100

50


300

stochastic collocation (2) stochastic collocation (4)

0.1

100 200 300 mechanical velocity (rad/s)

stochastic collocation (4)

0 0

300

-4280 -4260 torque (Nm)

-4240


standard deviation (Nm)

-4000

-10000

-10000


-2000

0.15



0.1

0.05

0 -4320

-4300

-4280 -4260 torque (Nm)

-4240

Figure 7.13: Statistics of the torque, in the case of one random variable ξ1 in the stochastic model, either uniformly (left column) or lognormally (right column) distributed. The number of random unknowns, Q or Nc , is indicated between brackets. The probability density function corresponds to ωm = 111 rad/s.

CONCLUSIONS

185

range of the uniformly distributed variable is chosen so that its standard deviation equals σg . The mean value of the torque is not shown, as it corresponds to the results in Fig. 7.13. The uncertainty on the torque is even more influenced by ξ2 then by ξ1 , as indicated by the larger standard deviation in Fig. 7.14 compared to Fig. 7.13. The standard deviation of the torque in the uniform and lognormal case are again very similar, when using the same input standard deviation σg .

7.7.3

Uncertainty on the parameters of the magnetization curve

Instead of considering only one random variable ξ1 to model the uncertainty on the reluctivity ν, the uncertainty on the magnetization curve can be expressed by 4 random variables, ξ4 up to ξ7 , each perturbing one of the parameters in (7.21). Since multiple random variables are present, both tensor product and sparse stochastic collocation point grids can be applied. The right part of Fig. 7.14 illustrates the torque statistics. In√the uniform√case, all 4 random variables are uniformly distributed on [1 − 0.08 3, 1 + 0.08 3]; in the lognormal case they correspond to µg = 0 and σg = 0.08. Between brackets, the number of random unknowns is given. The mean value of the torque is not shown, as it corresponds to the solution of the deterministic problem (7.10). Comparing these results to Fig. 7.13, we note that modelling the material uncertainty by one random variable instead of the more accurate representation by 4 random variables, yields already a good approximation to the variability of the torque.

7.8

Conclusions

This chapter compares the stochastic Galerkin and stochastic collocation method with respect to their computational cost, implementation cost and accuracy. In particular, the solution of a nonlinear stochastic model is studied, which represents a solid-rotor magnetic brake as a ferromagnetic cylinder rotating at high speed. Both stochastic solvers succeed in computing high-order stochastic solutions. The stochastic Galerkin method typically computes a more accurate solution than the stochastic collocation method for the same number of random unknowns. The stochastic collocation method requires less implementation effort than the stochastic Galerkin approach and enables one to reuse deterministic simulation code. Especially in the case of a nonlinear PDE, the re-implementation of a nonlinear solver routine can be quite hard for the stochastic Galerkin method. Concerning the computational cost, numerical experiments illustrate that for nonlinear problems more computational time is required by the stochastic Galerkin approach to reach the same level of accuracy as a sparse grid stochastic collocation method based on Gauss nodes. For linear problems however, the stochastic

stochastic COMPARISON OF THE STOCHASTIC GALERKIN AND COLLOCATION METHOD



1000 800


600 400 200 0 0

tensor stochastic collocation (16) sparse collocation (9, Gauss) sparse coll. (9, Clenshaw-Curtis) stochastic Galerkin (5)

300 standard deviation (Nm)

186

200

100

0 0

100 300 200 mechanical velocity (rad/s)


(a) uniform

800


(b) uniform 300




1000

600 400 200 0 0

200 150 100 50 0 0

300


250

tensor stochastic collocation (256) sparse stochastic collocation (9) sparse stochastic collocation (57) stochastic Galerkin (5) stochastic Galerkin (35)

x 10

1

−3



0.5

0 -6000

-4000 -5000 torque (Nm)

(e) pdf – uniform


300

(d) lognormal

-3000



(c) lognormal 1.5

300

0.025 0.02

tensor stochastic collocation (81) tensor stochastic collocation (256) stochastic Galerkin (15) stochastic Galerkin (35)

0.015 0.01 0.005 0 −4400

-4300 -4200 torque (Nm)

-4100

(f) pdf – lognormal

Figure 7.14: Statistics of the torque, in the case that the stochastic model contains uniformly or lognormally distributed random variables: the left part (a,c,e) corresponds to one random variable ξ2 , the right part (b,d,f) to 4 random variables ξ4 , ξ5 , ξ6 and ξ7 . The number of random unknowns, Q or Nc , is indicated between brackets. The probability density function corresponds to ω = 111 rad/s.

CONCLUSIONS

187

Galerkin finite element method can outperform the collocation approach, when efficient iterative solvers are used. Such efficient solvers could not be constructed for the nonlinear ferromagnetic example since the spectral spatial discretization resulted in dense systems. These observations suggest that the stochastic Galerkin method may be preferable over the sparse stochastic collocation method for solving linear problems, as also noted in [BS09], but that nonlinear stochastic PDEs might easier be solved with sparse grid stochastic collocation techniques. New adaptive and sparse stochastic Galerkin and collocation techniques are however still being developed in order to reduce the curse of dimensionality. Therefore, these conclusions may change in the future.

Chapter 8

Conclusions This chapter presents the main results of this work and gives an overview of possible future research directions. The main contributions can be summarized as follows: • derivation of analytical formulae to construct stochastic Galerkin discretization matrices; • construction of multigrid solution methods for algebraic systems resulting from stationary and time-dependent stochastic PDEs discretized by the stochastic Galerkin finite element method; • general construction of so-called Ci -splitting based iterative solvers, both in a one-level and multilevel framework, stand-alone or as preconditioner; • local Fourier convergence analysis of iterative solvers for stochastic Galerkin finite element discretizations and numerical comparative study; • convergence analysis of (algebraic) multigrid methods for algebraic systems resulting from time-dependent, stochastic diffusion problems, discretized by the stochastic Galerkin finite element method combined with an implicit Runge-Kutta time discretization; • solution of a nonlinear, time-dependent, stochastic reaction-diffusion problem modelling the concentration of chemical particles in an apple; • development of a stochastic Galerkin and a stochastic collocation solution of a stochastic ferromagnetic application; • comparison of the stochastic Galerkin and collocation finite element methods.

189

190

8.1

CONCLUSIONS

Summary and conclusions

The stochastic Galerkin finite element method provides a powerful tool for computing high-order stochastic solutions of uncertainty quantification problems. It relies on a solid theoretical foundation which ensures its convergence and robustness properties. Over the years, the stochastic Galerkin finite element method has evolved from a new mathematical framework towards a practical and fast alternative to the Monte Carlo simulation method. This can be seen from the growing literature on stochastic Galerkin applications and the integration of stochastic Galerkin solvers into state-of-the-art mathematical libraries, such as the recently released Stokhos package in Trilinos [SHD04]. Since the stochastic Galerkin finite element method transforms a stochastic PDE into a large coupled system of deterministic PDEs, its main computational challenge corresponds to solving the resulting high-dimensional algebraic system. One approach is to decouple the set of deterministic PDEs, which leads to stochastic collocation finite element methods, another approach is to develop special iterative solvers for the algebraic systems. This work presented a range of iterative solvers for the high-dimensional algebraic systems resulting from stochastic Galerkin finite element discretizations. It was shown that multigrid methods for deterministic PDEs can be extended to deal with stochastic PDE discretizations. This enables one to build highly efficient solvers with so-called optimal convergence properties, i.e., the multigrid convergence rate is independent of the spatial, stochastic and, possibly, time discretization parameters. Time-(in)dependent, (non)linear stochastic problems with multiple random coefficients were considered. In the time-dependent case, the stochastic Galerkin method is combined with a high-order implicit Runge-Kutta time discretization. Both the differential operator, right-hand side, boundary conditions and domain description can contain random variables. For smooth stochastic problems with small input variances, cheap Krylov preconditioners such as a mean-based preconditioning approach are the best solver choice. These preconditioners possess less optimal convergence properties than the proposed multigrid solvers, but can result in highly efficient iterative solution methods. The robustness of the developed multigrid methods guarantees however a fast convergence behavior in the case of less trivial stochastic PDE problems. We applied a local Fourier analysis to iterative solvers for stochastic Galerkin discretizations. This confirmed the above convergence properties theoretically. Properties of orthogonal polynomials on which the stochastic Galerkin discretization is based, turn out to have an important impact on the convergence properties of the iterative solvers. Indeed, the spectrum of stochastic Galerkin discretization matrices is determined by the zeros of these orthogonal polynomials. This implies for example that the use of Legendre polynomials, which are defined on a finite inter-

SUGGESTIONS FOR FURTHER RESEARCH

191

val, typically results in better convergence properties than Hermite polynomials, which have an infinite support. Since the stochastic Galerkin and collocation finite element methods both enable an efficient computation of high-order stochastic solutions, it is important to know for what type of problems which method is most appropriate. When solving linear stochastic PDEs, very efficient implementations of the stochastic Galerkin method are possible, suggesting that this method is preferable. The situation changes in the case of a nonlinear stochastic PDE, depending on whether polynomial or non-polynomial nonlinearities are present. In the polynomial nonlinear case, the stochastic Galerkin method can still easily be applied with only small modifications and efficient solvers can be designed. The computational cost may however already grow substantially, as observed for the cubic stochastic Gray-Scott reaction problem discussed in Section 6.6. Non-polynomial nonlinearities typically require numerical approximations to the stochastic Galerkin projection, as illustrated for the ferromagnetic rotor problem. As a result, sparse stochastic collocation methods could be a better approach than the stochastic Galerkin method in the case of nonlinear stochastic PDEs.

8.2 8.2.1

Suggestions for further research Multilevel solution methods

The multigrid solution methods presented so far essentially only coarsen in the spatial dimension. When solving problems with many spatial and random unknowns, both a coarsening in the spatial and stochastic dimension may be required. A multilevel solution approach that coarsens in the stochastic dimension was proposed, but it lacked the optimal convergence properties of the spatial multigrid solver and is therefore of only limited practical use. In case of a large number of random unknowns, multilevel solvers in the stochastic dimension could be appropriate for solving the coarse system in a multigrid method or the block systems of a collective Gauss-Seidel smoother. For a small to moderate number of stochastic degrees of freedom, these blocks are currently solved with a Krylov or sparse direct solver. The proposed multilevel method only considered discretizations with a global stochastic polynomial basis. The multilevel hierarchy was therefore based on lowering the order of the stochastic polynomials. In the case of multi-element stochastic polynomial discretizations, both the stochastic elements and the polynomial order can be coarsened. The effect of coarsening the stochastic elements on the convergence properties of a multilevel solution method could be investigated. An alternative iterative solution approach could be based on leaving terms out of the stochastic expansion used to represent the input random fields. These

192

CONCLUSIONS

expansions typically contain many terms which only slightly contribute to the stochastic solution. When solving the problem with only a few expansion terms, the cost of matrix-vector multiplications and smoothing operations is reduced in case that the Kronecker product blocks of the algebraic system matrix are stored separately in sparse matrix format. This enables one to compute an approximate stochastic solution, which can be used as starting guess to a more accurate full computation. This type of coarsening could serve as a basis to construct another type of multilevel solution methods.

8.2.2

Fast solvers for nonlinear PDEs

The nonlinear, stochastic PDEs studied in this work could easily be linearized by Newton’s method. After linearization, an appropriate (multigrid) iterative solver was applied to the resulting algebraic systems. Alternatively, so-called ‘full approximation schemes’ could be extended to discretized nonlinear, stochastic problems. These schemes apply a multigrid framework, but avoid the global linearization step in contrast to Newton-multigrid methods. For deterministic problems, very efficient nonlinear solvers can be constructed in this way. An extension to stochastic Galerkin discretizations could lower the cost of the stochastic Galerkin method in the case of nonlinear PDEs.

8.2.3

Fast matrix-vector multiplication

The algebraic Kronecker product system matrix resulting from a SGFEM discretization is typically not stored entirely in memory in order to reduce the memory requirements, but only the individual matrices from the stochastic and spatial discretization are stored separately in sparse matrix format. As also pointed out in [PU09], this increases the cost of performing a matrix-vector multiplication. For some problems, the number of individual stiffness matrices can be very large, leading to a very expensive matrix-vector multiplication. The construction of a fast matrix-vector multiplication for stochastic Galerkin discretizations would be very useful. A possible solution can be to use low-rank approximation techniques.

8.2.4

Alternative representation of lognormal random fields

When a stochastic diffusion coefficient is modelled as a lognormal random field, the stochastic Galerkin discretization results in a block dense algebraic system due to a polynomial chaos representation of the lognormal field, see remark 2.5.15 and corollary 2.5.13. This leads to a high computational and memory cost of problems with lognormal fields. Alternatively, lognormal random fields can be discretely

SUGGESTIONS FOR FURTHER RESEARCH

193

represented by a Karhunen-Loève expansion [PHQ02b]. Since such a representation is made of a linear expansion of random variables, the computational cost of the stochastic Galerkin discretization may substantially be reduced. The random variables in that expansion are however not independent and their probability density function does not correspond to an orthogonal polynomial from the Wiener-Askey scheme. Therefore, an arbitrary polynomial chaos will need to be constructed, adapted to the Karhunen-Loève random variables of a lognormal random field. An efficient implementation of this alternative representation of a lognormal field is then required. Also, a comparison of the computational cost of this approach with the standard polynomial chaos representation is indispensable.

8.2.5

Application to industrial problems

Although the stochastic Galerkin and collocation methods have already been applied to some more realistic engineering problems, e.g., the uncertainty quantification of the aerodynamic behavior of a bridge or the dynamic modelling of supersonic flow past an airplane wedge, these applications typically did not apply an efficient iterative solver for the algebraic system but used for example a mean-based preconditioning approach. Extending and adapting multigrid solution methods to more realistic stochastic applications will enhance the performance of the stochastic Galerkin method. The intrusive character of the stochastic Galerkin finite element method might be difficult to deal with in industrial applications. The implementation effort required to apply a stochastic Galerkin method must therefore be compensated by a gain in computational cost and accuracy over Monte Carlo or stochastic collocation methods. This necessitates an extended comparison between stochastic Galerkin and collocation methods.

Appendix A

Probability and random fields Probability distributions. An overview of commonly used probability density functions is given in Table A.1. The support of a probability density function ̺i (yi ) is denoted by Γi . A standard normal random variable is a Gaussian random variable with µ = 0 and σ = 1, i.e., ξi ∼ N (0, 1). Correlated random variables. The correlation of two random variables ξ1 and ξ2 is defined as the ratio of the covariance of these two random variables to the product of their standard deviations, i.e., corr1,2 =

cov1,2 , std1 std2

where the covariance is defined as the expectation of the central cross-product: cov1,2 = h(ξ1 − hξ1 i)(ξ2 − hξ2 i)i. Covariance function. Some covariance kernels enable an analytical solution of the eigenvalue integral problem (2.3). Consider an exponential covariance function, given by (2.7) and defined on a square 2D domain D = [0, 1]2 . Since the covariance kernel is separable, its eigenvalues and eigenfunctions can be written as products of those of two corresponding 1D problems. An analytical solution procedure for these 1D eigenproblems is detailed in [GS03], where a second-order differential equation is solved with appropriate boundary conditions. Combining these results,

195

196

PROBABILITY AND RANDOM FIELDS

̺i (yi )

Γi

Gaussian

1 (yi − µi )2 √ (−∞, ∞) exp − 2σ 2 2πσ 2 Lognormal (= exp(ξG ) with ξG ∼ N (µ, σ) ) ! 2 (ln yi − µ) 1 √ exp − [0, ∞) 2σ 2 yi σ 2π

mean hξi i

variance h(ξi − hξi i)2 i

µ

σ2

σ2 2 )

exp(µ +

(exp(σ 2 ) − 1) · exp(2µ + σ 2 )

Uniform 1 b−a

[a, b]

Gamma (shape parameter k, scale parameter θ) exp (−yi /θ) [0, ∞) yik−1 Γ(k) θk Beta yiα−1 (1 − yi )β−1 (0, 1) B(α, β)

1 (a + b) 2

1 (b − a)2 12

kθ

kθ2

α α+β

αβ (α +

β)2 (α

+ β + 1)

Table A.1: Overview of classical probability distributions.

the following eigenvalues and eigenfunctions are obtained for the 2D case [ZL04]: 4lc2 σ 2 λi = 2 2 (1) (2) +1 lc2 wj +1 lc2 wk

(A.1)

(1)

(1)

(1)

(2)

vi (x) = vj (x1 )vk (x2 )

(1)

vj (x1 ) =

with

lc wj cos(wj1 x1 ) + sin(wj x1 ) r , (1)

l2c wj

2

2

+1

+ lc

(A.2)

(2)

(1)

and vk (x2 ) defined analogously to vj (x1 ). The indices j and k map to the index (1)

i such that the eigenvalues λi are ordered non-increasingly. The scalars wj (2) wk

in (A.1)–(A.2) are two positive roots of the characteristic equation (lc2 w2 − 1) sin(w) = 2lc w cos(w).

and


197

Lognormal random fields. Consider a lognormal random field alog (x, ω) defined as exp(ag )(x, ω), with ag (x, ω) a Gaussian field. The mean µl and variance σl2 of the lognormal field alog (x, ω) can be calculated as µl = exp(µg + σg2 /2)

and

σl2 = µ2l (exp(σg2 ) − 1)),

with µg and σg2 the mean and variance of ag (x, ω). Proof of theorem (2.3.6) (adapted from [Gha99b]). The polynomial chaos expansion coefficients aq (x) of the lognormal field ! ∞ X gi (x)ξi−1 a(x, ω) = exp(aG (x)) = exp g1 (x) + i=2

are defined by (2.9). For q = 1, with Ψ1 = 1, this yields !+ * ∞ X gi (x)ξi−1 a1 (x) = haΨ1 i = exp g1 (x) + i=2

= exp(g1 (x))

∞ Y

i=2

hexp(gi (x)ξi−1 )i

! ∞ 1X 2 = exp g1 (x) + g (x) , 2 i=2 i where the Gaussian probability density function was used to evaluate h·i. The proof of the second part of the theorem, for q ≥ 1, starts again from the definition of aq (x) (2.9) and applies the definition of the multivariate polynomials Ψq (2.13), + *∞ Y exp(gi (x)ξi−1 )Ψq (ξ) aq (x) = exp(g1 (x) i=2

= exp(g1 (x))

∞ Y

exp(gi (x)ξi−1 )ψηq,i−1 (ξi−1 )

i=2

= exp(g1 (x))

∞ Y

i=2

exp

1 2 1 2 g (x) − gi (x) 2 i 2

exp(gi (x)ξi−1 )ψηq,i−1 (ξi−1 )

= a1 (x)

(yi−1 − gi (x))2 exp − ψηq,i−1 (yi−1 )dyi−1 2 −∞

∞ Z Y

i=2

∞

198


aq (x) = a1 (x)

∞ Y

ψηq,i−1 (ξi−1 − gi (x))

i=2

= a1 (x)

∞ Y

i=2

1 η (gi (x)) q,i−1 . √ ηq,i−1

The last equality uses the orthogonality property and normalization constant of the Hermite polynomials (2.15).

Bibliography [AA07]

N. Agarwal and N. R. Aluru. A stochastic Lagrangian approach for geometrical uncertainties in electrostatics. J. Comput. Phys., 226:156– 179, 2007.

[Adl81]

R. J. Adler. The geometry of random fields. Wiley series in probability and mathematical statistics. Chichester, Wiley, UK, 1981.

[Ans05]

M. Anshelevich. Linearization coefficients for orthogonal polynomials using stochastic processes. The Annals of Probability, 33(1):114–136, 2005.

[Ask75]

R. Askey. Orthogonal polynomials and special functions. Regional Conference Series in Applied Mathematics. SIAM, Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, USA, 1975.

[AZ06]

S. Acharjee and N. Zabaras. A concurrent model reduction approach on spatial and random domains for the solution of stochastic PDEs. Int. J. Numer. Meth. Eng., 66:1934–1954, 2006.

[AZ07]

S. Acharjee and N. Zabaras. A non-intrusive stochastic Galerkin approach for modeling uncertainty propagation in deformation processes. Computers and structures, 85:244–254, 2007.

[BAS10]

M. Bieri, R. Andreev, and C. Schwab. Sparse tensor discretization of elliptic sPDEs. SIAM J. Sci. Comput., 31(6):4281–4304, 2010.

[BBCF07] J. Baroth, P. Bressolette, C. Chauvière, and M. Fogli. An efficient SFE method using Lagrange polynomials: application to nonlinear mechanical problems with uncertain parameters. Comput. Methods Appl. Mech. Engrg., 196:4419–4429, 2007. [BC02]

I. Babuˇska and P. Chatzipantelidis. On solving elliptic stochastic partial differential equations. Comput. Methods Appl. Mech. Engrg., 191:4093–4122, 2002.

199

200

BIBLIOGRAPHY

[Ber99]

A. Bernardini. Whys and Hows in Uncertainty Modelling Probability Fuzziness and Optimization, volume 380 of CISM Course and Lectures, chapter What are the random fuzzy sets and how to use them for uncertainty modeling in engineering system, pages 63–126. SpringerVerlag, Wien, Austria, 1999.

[BG04]

H.-J. Bungartz and M. Griebel. Sparse grids. Acta Numerica, 13:147– 269, 2004.

[BHM00]

W. L. Briggs, V. E. Henson, and S. F. McCormick. A multigrid tutorial. SIAM, Philadelphia, USA, second edition, 2000.

[Bie10]

M. Bieri. A sparse composite collocation finite element method for elliptic sPDEs. SIAM J. Numer. Anal., 2010.

[BLT03]

I. Babuˇska, K.-M. Liu, and R. Tempone. Solving stochastic partial differential equations based on the experimental data. Math. Mod. Meth. Appl. S., 13(3):415–444, 2003.

[BNR00]

V. Barthelmann, E. Novak, and K. Ritter. High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math., 12:273–288, 2000.

[BNT05]

I. Babuˇska, F. Nobile, and R. Tempone. Worst case scenario analysis for elliptic problems with uncertainty. Numer. Math., 101:185–219, 2005.

[BNT07]

I. Babuˇska, F. Nobile, and R. Tempone. A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Numer. Anal., 45:1005–1034, 2007.

[BNT10]

I. Babuˇska, F. Nobile, and R. Tempone. A stochastic collocation method for elliptic partial differential equations with random input data. SIAM Rev., 52(2):317–355, 2010.

[BNTT09] J. Bäck, F. Nobile, L. Tamellini, and R. Tempone. Stochastic galerkin and collocation methods for PDEs with random coefficients: a numerical comparison. ICES report 09-33, Institute for Computational Engineering and Sciences, University of Texas, Austin, USA, 2009. [Bra77]

A. Brandt. Multi-level adaptive solutions to boundary-value problems. Math. Comput., 31:333–390, 1977.

[BS02]

S. C. Brenner and L. R. Scott. The mathematical theory of finite element methods. Springer-Verlag, New York, USA, 2002.

[BS09]

M. Bieri and C. Schwab. Sparse high order FEM for elliptic sPDEs. Comput. Methods Appl. Mech. Engrg., 198(13-14):1149–1170, 2009.

BIBLIOGRAPHY

201

[BTZ04]

I. Babuˇska, R. Tempone, and G.E. Zouraris. Galerkin finite element approximations of stochastic elliptic partial differential equations. SIAM J. Numer. Anal., 42:800–825, 2004.

[BTZ05]

I. Babuˇska, R. Tempone, and G. E. Zouraris. Solving elliptic boundary value problems with uncertain coefficients by the finite element method: the stochastic formulation. Comput. Methods Appl. Mech. Engrg., 194:1251–1294, 2005.

[But03]

J.C. Butcher. Numerical methods for ordinary differential equations. Wiley, Chichester, UK, 2003.

[BVlV09]

T. Boonen, J. Van lent, and S. Vandewalle. Algebraic multigrid for fully implicit Runge-Kutta discretizations of the time-dependent divgrad and curl-curl equations. Appl. Numer. Math., 59:507–521, 2009.

[BW96]

J. C. Butcher and G. Wanner. Runge-Kutta methods: some historical notes. Appl. Numer. Math., 22:113–151, 1996.

[Caf98]

R. E. Caflisch. Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7:1–49, 1998.

[CDS09]

A. Cohen, R. DeVore, and C. Schwab. Convergence rates of best Nterm Galerkin approximations for a class of elliptic sPDEs. Technical Report 2009-02, Seminar f¨ ur Angewandte Mathematik, ETH Z¨ urich, Switzerland, 2009.

[CE89]

T. F. Chan and H. C. Elman. Fourier analysis of iterative methods for elliptic problems. SIAM Rev., 31(1):20–49, 1989.

[CF09]

C. Canuto and D. Fransos. Numerical solution of partial differential equations in random domains: an application to wind engineering. Commun. Comput. Phys., 5(2–4):515–531, 2009.

[CFH+ 00] A. J. Cleary, R. D. Falgout, V. E. Henson, J. E. Jones, T. A. Manteuffel, S. F. McCormick, G. N. Miranda, and J. W. Ruge. Robustness and scalability of algebraic multigrid. SIAM J. Sci. Comput., 21(5):1886– 1908, 2000. [CHL06]

C. Chauvière, J. S. Hesthaven, and L. Lurati. Computational modeling of uncertainty in time-domain electromagnetics. SIAM J. Sci. Comput., 28(2):751–775, 2006.

[CK07]

C. Canuto and T. Kozubek. A fictitious domain approach to the numerical solution of PDEs in stochastic domains. Numer. Math., 107:257– 293, 2007.

202

BIBLIOGRAPHY

[Cle04]

T. Clees. AMG strategies for PDE systems with applications in industrial semiconductor simulation. PhD thesis, University of Cologne, Cologne, Germany, 2004.

[CM47]

R. H. Cameron and W. T. Martin. The orthogonal development of non-linear functionals in series of Fourier-Hermite functionals. Ann. Math., 48(2):385–392, 1947.

[CN96]

R. H. Chan and M. K. Ng. Conjugate gradient methods for Toeplitz systems. SIAM Rev., 38(3):427–482, 1996.

[CNS00]

R. H. Chan, W.-F. Ng, and H.-W. Sun. Fast construction of optimal circulant preconditioners for matrices from the fast dense matrix method. BIT, 40(1):24–40, 2000.

[CQHZ06] C. Canuto, A. Quarteroni, M. Y. Hussaini, and T. A. Zang. Spectral Methods: Fundamentals in Single Domains. Springer, Berlin, Germany, 2006. [Dav04]

T. A. Davis. Algorithm 832: UMFPACK V4.3—an unsymmetricpattern multifrontal method. ACM Trans. Math. Softw., 30(2):196– 199, 2004.

[DBT01]

M. K. Deb, I. M. Babuˇska, and J. Tinsley Oden. Solution of stochastic partial differential equations using Galerkin finite element techniques. Comput. Methods Appl. Mech. Engrg., 190:6359–6372, 2001.

[DD09]

A. Der Kiureghian and O. Ditlevsen. Aleatory or epistemic? Does it matter? Structural Safety, 31:105–112, 2009.

[DEG+ 99] J. W. Demmel, S. C. Eisenstat, J. R. Gilbert, X. S. Li, and J. W. H. Liu. A supernodal approach to sparse partial pivoting. SIAM J. Matrix Anal. A., 20(3):720–755, 1999. [DG09]

H. De Gersem. Spectral-element methods for high-speed rotating cylinders. COMPEL, 28(3):730–740, 2009.

[DG10]

H. De Gersem. Combined spectral-element, finite-element discretization for magnetic-brake simulation. IEEE T. Magn., 2010. In press.

[DGRH07] A. Doostan, R. Ghanem, and J. Red-Horse. Stochastic model reduction for chaos representations. Comput. Methods Appl. Engrg., 196:3951–3966, 2007. [DLZZ08]

Y. Ding, T. Li, D. Zhang, and P. Zhang. Adaptive Stroud stochastic collocation method for flow in random porous media via KarhunenLoève expansion. Commun. Comput Phys., 4(4):102–123, 2008.

BIBLIOGRAPHY

203

[DNP+ 04] B. J. Debusschere, H. N. Najm, P. P. Pébay, O. M. Knio, R. G. Ghanem, and O. P. Le Maˆıtre. Numerical challenges in the use of polynomial chaos representations for stochastic processes. SIAM J. Sci. Comput., 26(2):698–719, 2004. [EB09]

M. S. Eldred and J. Burkardt. Comparison of non-intrusive polynomial chaos and stochastic collocation methods for uncertainty quantification. AIAA, (2009-0976), 2009.

[EEOS05] H. Elman, O. G. Ernst, D. P. O’Leary, and M. Stewart. Efficient iterative algorithms for the stochastic finite element method with application to acoustic scattering. Comput. Methods Appl. Mech. Engrg., 194:1037–1055, 2005. [EEU07]

M. Eiermann, O. G. Ernst, and E. Ullmann. Computational aspects of the stochastic finite element method. Comput. Vis. Sci., 10(1):3–15, 2007.

[EF07]

H. Elman and D. Furnival. Solving the stochastic steady-state diffusion problem using multigrid. IMA J. Numer. Anal., 27(4):675–688, 2007.

[ESW05]

H. C. Elman, D. J. Silvester, and A. J. Wathen. Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics. Oxford University Press, UK, 2005.

[EU10]

O. G. Ernst and E. Ullmann. Stochastic Galerkin matrices. SIAM J. Matrix Anal. Appl., 31(4):1848–1872, 2010.

[FG04]

R. V. Field and M. Grigoriu. On the accuracy of the polynomial chaos approximation. Probabilist. Eng. Mech., 19:65–80, 2004.

[Fis96]

G. S. Fishman. Monte Carlo: concepts, algorithms and applications. Springer-Verlag, New York, USA, 1996.

[FST05]

P. Frauenfelder, C. Schwab, and R. A. Todor. Finite elements for elliptic problems with stochastic coefficients. Comput. Methods Appl. Mech. Engrg., 194:205–228, 2005.

[Fur08]

D. G. Furnival. Iterative methods for the stochastic diffusion problem. PhD thesis, University of Maryland, College Park, MD, USA, 2008.

[FWK08]

J. Foo, X. Wan, and G. E. Karniadakis. The multi-element probabilistic collocation method (ME-PCM): error analysis and applications. J. Comput. Phys., 227:9572–9595, 2008.

[FYK07]

J. Foo, Z. Yosibash, and G. E. Karniadakis. Stochastic simulation of riser-sections with uncertain measured pressure loads and/or uncertain material properties. Comput. Methods Appl. Mech. Engrg., 196:4250– 4271, 2007.

204

BIBLIOGRAPHY

[Gau04]

W. Gautschi. Orthogonal Polynomials: Computation and approximation. Oxford University Press, Oxford, UK, 2004.

[GD98]

R. Ghanem and S. Dham. Stochastic finite element analysis for multiphase flow in heterogeneous porous media. Transport Porous Med., 32:239–262, 1998.

[GD04]

R. M. Gray and L. D. Davisson. An introduction to statistical signal processing. Cambridge University Press, Cambridge, UK, 2004.

[GD06]

R. G. Ghanem and A. Doostan. On the construction and analysis of stochastic model: characterization and propagation of the errors associated with limited data. J. Comput. Phys., 217:63–81, 2006.

[GG98]

T. Gerstner and M. Griebel. Numerical integration using sparse grids. Numer. Algorithms, 18:209–232, 1998.

[Gha98]

R. Ghanem. Probabilistic characterization of transport in heterogeneous media. Comput. Methods Appl. Engrg., 158:199–220, 1998.

[Gha99a]

R. Ghanem. Ingredients for a general purpose stochastic finite element implementation. Comput. Methods Appl. Mech. Engrg., 168:19– 34, 1999.

[Gha99b]

R. Ghanem. The nonlinear Gaussian spectrum of log-normal stochastic processes and variables. ASME J. Appl. Mech., 66(4):964–973, 1999.

[Gha99c]

R. Ghanem. Stochastic finite elements with multiple random nonGaussian properties. ASCE J. Eng. Mech., 125:26–40, 1999.

[Git10]

C. J. Gittelson. Stochastic Galerkin discretization of the lognormal isotropic diffusion problem. Math. Mod. Meth. Appl. S., 20(2):237– 263, 2010.

[GK96]

R. G. Ghanem and R. Kruger. Numerical solution of spectral stochastic finite element systems. Comput. Methods Appl. Mech. Engrg., 129:289–303, 1996.

[GR09]

C. Geuzaine and J.-F. Remacle. Gmsh: a three-dimensional finite element mesh generator with built-in pre- and post-processing facilities. Int. J. Numer. Meth. Eng., 79(11):1309–1331, 2009.

[Gri02]

M. Grigoriu. Stochastic calculus: applications in science and engineering. Birkh¨ auser, Boston, USA, 2002.

[GRS98]

W. R. Gilks, S. Richardson, and D. J. Spiegelhalter. Markov chain Monte Carlo in practice. Chapman & Hall CRC Press, Boca Raton, FL, USA, 1998.

BIBLIOGRAPHY

205

[GS91]

R. G. Ghanem and P. D. Spanos. A spectral stochastic finite element formulation for reliability analysis. J. Engrg. Mech. ASCE, 17:2351– 2372, 1991.

[GS03]

R. G. Ghanem and P. D. Spanos. Stochastic finite elements, a spectral approach. Dover, Mineola, New York, 2nd edition, 2003.

[GSD07]

R. G. Ghanem, G. Saad, and A. Doostan. Efficient solution of stochastic systems: application to the embankment dam problem. Structural safety, 29(3):238–251, 2007.

[GW69]

G. H. Golub and J. H. Welsch. Calculation of Gauss quadrature rules. Math. Comput., 23(106):221–230, 1969.

[GZ07]

B. Ganapathysubramanian and N. Zabaras. Sparse grid collocation schemes for stochastic natural convection problems. J. Comput. Phys., 225(1):652–685, 2007.

[Hac85]

W. Hackbusch. Multi-grid methods and applications. Springer, Berlin, Germany, 1985.

[HLRZ06] T. H. Hou, W. Luo, B. Rozovskii, and H.-M. Zhou. Wiener chaos expansions and numerical solutions of randomly forced equations of fluid mechanics. J. Comput. Phys., 216:687–706, 2006. [HNW00]

E. Hairer, S. P. Nørsett, and G. Wanner. Solving ordinary differential equations I: nonstiff problems. Springer, Heidelberg, Germany, 2000.

[HQP01]

S. P. Huang, S. T. Quek, and K. K. Phoon. Convergence study of the truncated Karhunen-Loeve expansion for simulation of stochastic processes. Int. J. Numer. Meth. Eng., 52:1029–1043, 2001.

[HW02]

E. Hairer and G. Wanner. Solving ordinary differential equations II: stiff and differential-algebraic problems. Springer, Berlin, Germany, 2002.

[HY02]

V. E. Henson and U. M. Yang. BoomerAMG: a parallel algebraic multigrid solver and preconditioner. Appl. Numer. Math., 41(1):155– 177, 2002.

[JC09]

C. Jin and X.-C. Cai. A preconditioned recycling GMRES solver for stochastic Helmholtz problems. Commun. Comput. Phys., 6(2):342– 353, 2009.

[JCL07]

C. Jin, X.-C. Cai, and C. Li. Parallel domain decomposition methods for stochastic elliptic equations. SIAM J. Sci. Comput., 29(3):2096– 2114, 2007.

206

BIBLIOGRAPHY

[Kee04]

A. Keese. Numerical solution of systems with stochastic uncertainties. PhD thesis, Technische Universität Braunschweig, Braunschweig, Germany, 2004.

[KH92]

M. Kleiber and T. D. Hien. The stochastic finite element method: basic perturbation technique and computer implementation. Wiley, Chichester, United Kingdom, 1992.

[KL06]

O. M. Knio and O. P. Le Maˆıtre. Uncertainty propagation in CFD using polynomial chaos decomposition. Fluid Dynamics Research, 38:616–640, 2006.

[KLM09]

B. N. Khoromskij, A. Litvinenko, and H. G. Matthies. Application of hierarchical matrices for computing the Karhunen-Loève expansion. Computing, 84(1–2):49–67, 2009.

[KM05]

A. Keese and H. G. Matthies. Hierarchical parallelisation for the solution of stochastic finite element equations. Computers and Structures, 83:1033–1047, 2005.

[LKD+ 03] O. P. Le Maˆıtre, O. M. Knio, B. J. Debusschere, H. N. Najm, and R. G. Ghanem. A multigrid solver for two-dimensional stochastic diffusion equations. Comput. Methods Appl. Mech. Engrg., 192:4723–4744, 2003. [LKNG04] O. P. Le Maˆıtre, O. M. Knio, H. N. Najm, and R. Ghanem. Uncertainty propagation using Wiener-Haar expansions. J. Comput. Phys., 197:28– 57, 2004. [LNGK04] O. P. Le Maˆıtre, H. N. Najm, R. Ghanem, and O. M. Knio. Multiresolution analysis of Wiener-type uncertainty propagation schemes. J. Comput. Phys., 197:502–531, 2004. [Loè77]

M. Loève. Probability theory. Springer, New York, USA, 1977.

[Loh96]

W.-L. Loh. On Latin hypercube sampling. The Annals of Statistics, 24(5):2058–2080, 1996.

[LRD+ 04] O. P. Le Maˆıtre, M. T. Reagan, B. J. Debusschere, H. N. Najm, R. Ghanem, and O. M. Knio. Natural convection in a closed cavity under stochastic non-Boussinesq conditions. SIAM J. Sci. Comput., 26(2):375–394, 2004. [LRN+ 02] O. P. Le Maˆıtre, M. Reagan, H. N. Najm, R. Ghanem, and O. M. Knio. A stochastic projection method for fluid flow: II random process. J. Comput. Phys., 181:9–44, 2002. [LSK06]

G. Lin, C.-H. Su, and G. E. Karniadakis. Predicting shock dynamics in the presence of uncertainties. J. Comput. Phys., 217:260–276, 2006.

BIBLIOGRAPHY

207

[LSK08]

G. Lin, C.-H. Su, and G. E. Karniadakis. Stochastic modeling of random roughness in shock scattering problems: theory and simulations. Comput. Methods Appl. Mech. Engrg., 197:3420–3434, 2008.

[LWB07]

G. J. A. Loeven, J. A. S. Witteveen, and H. Bijl. Probabilistic collocation: an efficient non-intrusive approach for arbitrary distributed parametric uncertainties. In AIAA, Aerospace sciences meeting and exhibit, 2007, Reno, Nevada, number AIAA 2007-317, 2007.

[LWSK07] G. Lin, X. Wan, C.-H. Su, and G. E. Karniadakis. Stochastic computational fluid mechanics. Comput. Sci. Eng., 9(2):21–27, 2007. [Mat08]

H. G. Matthies. Stochastic finite elements: computational approaches to stochastic partial differential equations. Z. Angew. Math. Mech., 88(11):849–873, 2008.

[MB99]

H. G. Matthies and C. Bucher. Finite elements for stochastic media problems. Comput. Methods Appl. Engrg., 168:3–17, 1999.

[McC87]

S. F. McCormick. Multigrid methods. SIAM, Philadelphia, USA, 1987.

[MHZ05]

L. Mathelin, M. Y. Hussaini, and T. A. Zang. Stochastic approaches to uncertainty quantification in CFD simulations. Numerical Algorithms, 38:209–236, 2005.

[MK05]

H. G. Matthies and A. Keese. Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Comput. Methods Appl. Engrg., 194:1295–1331, 2005.

[MNHC97] G. Maglaras, E. Nikolaidis, R. T. Hafka, and H. H. Cudney. Uncertainty: Models and Measures, chapter Analytical-experimental comparison of probabilistic and fuzzy set based methods for designing under uncertainty. Akademie-Verlag, Berlin, Germany, 1997. [MNR07]

Y. M. Marzouk, H. N. Najm, and L. A. Rahn. Stochastic spectral methods for efficient Bayesian solution of inverse problems. J. Comput. Phys., 224:560–586, 2007.

[MNS07]

K.-A. Mardal, T. K. Nilssen, and G. A. Staff. Order-optimal preconditioners for implicit Runge-Kutta schemes applied to parabolic PDEs. SIAM J. Sci. Comput., 29(1):361–375, 2007.

[MZ09]

X. Ma and N. Zabaras. An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations. J. Comput. Phys., 228:3084–3113, 2009.

[Naj09]

H. N. Najm. Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Annu. Rev. Fluid. Mech., 41:35–52, 2009.

208

BIBLIOGRAPHY

[NCSM08] A. Nouy, A. Clément, F. Schoefs, and N. Moës. An extended stochastic finite element method for solving stochastic partial differential equations on random domains. Comput. Methods Appl. Engrg., 197:4663– 4682, 2008. [Nie04]

H. Niederreiter. Monte Carlo and quasi-Monte Carlo methods. Springer, Berlin, Germany, 2004.

[NL09]

A. Nouy and O. P. Le Maˆıtre. Generalized spectral decomposition for stochastic nonlinear problems. J. Comput. Phys., 228:202–235, 2009.

[NR96]

E. Novak and K. Ritter. High dimensional integration of smooth functions over cubes. Numer. Math., 75:79–97, 1996.

[NT09]

F. Nobile and R. Tempone. Analysis and implementation issues for the numerical approximation of parabolic equations with random coefficients. Int. J. Numer. Meth. Eng., 80:979–1006, 2009.

[NTW08a] F. Nobile, R. Tempone, and C. G. Webster. An anisotropic sparse grid stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Num. Anal., 46(5):2411–2442, 2008. [NTW08b] F. Nobile, R. Tempone, and C. G. Webster. A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. Num. Anal., 46:2309–2345, 2008. [Ogu72]

H. Ogura. Orthogonal functionals of the Poisson process. IEEE T. Inform. Theory, 18(4):473–481, 1972.

[PB06]

C. L. Pettit and P. S. Beran. Spectral and multiresolution Wiener expansions of oscillatory stochastic processes. J. Sound Vib., 294:752– 779, 2006.

[PE09]

C. E. Powell and H. C. Elman. Block-diagonal preconditioning for spectral stochastic finite element systems. IMA J. Numer Anal., 29:350– 375, 2009.

[Pet01]

K. Petras. Fast calculation of coefficients in the Smolyak algorithm. Numerical Algorithms, 26:93–109, 2001.

[PG00]

M. F. Pellissetti and R. G. Ghanem. Iterative solution for systems of linear equations arising in the context of stochastic finite elements. Adv. Engrg. Software, 31:607–616, 2000.

[PHQ02a] K. K. Phoon, S. P. Huang, and S. T. Quek. Implementation of Karhunen-Loève expansion for simulation using a wavelet-Galerkin scheme. Probabilist. Eng. Mech., 17:293–303, 2002.

BIBLIOGRAPHY

209

[PHQ02b] K. K. Phoon, S. P. Huang, and S. T. Quek. Simulation of second-order processes using Karhunen-Loève expansion. Computers and Structures, 80:1049–1060, 2002. [PP07]

L. Parussini and V. Pediroda. Fictitious domain with least-squares spectral element method to explore geometric uncertainties by nonintrusive polynomial chaos method. CMES, 22(1):41–63, 2007.

[PU09]

C. E. Powell and E. Ullmann. Preconditioning stochastic Galerkin saddle point systems. Technical Report 2009.88, School of Mathematics, University of Manchester, 2009.

[RBV08]

E. Rosseel, T. Boonen, and S. Vandewalle. Algebraic multigrid for stationary and time-dependent partial differential equations with stochastic coefficients. Numer. Linear Algebra Appl., 15:141–163, 2008.

[RDV10a] E. Rosseel, H. De Gersem, and S. Vandewalle. Nonlinear stochastic Galerkin and collocation methods: application to a ferromagnetic cylinder rotating at high speed. Commun. Comput. Phys., 8(5):947– 975, 2010. [RDV10b] E. Rosseel, H. De Gersem, and S. Vandewalle. Spectral stochastic simulation of a ferromagnetic cylinder rotating at high speed. IEEE T. Magn., 2010. In press. [RNGK03] M. T. Reagan, H. N. Najm, R. Ghanem, and O. M. Knio. Uncertainty quantification in reacting-flow simulations through non-intrusive spectral projection. Combustion and Flame, 132:545–555, 2003. [RS87]

J. W. Ruge and K. St¨ uben. Algebraic multigrid (AMG). In S. F. McCormick, editor, Multigrid Methods, volume 3 of Frontiers in Applied Mathematics, pages 73–130. SIAM, Philadelphia, PA, USA, 1987.

[RV10]

E. Rosseel and S. Vandewalle. Iterative solvers for the stochastic finite element method. SIAM J. Sci. Comput., 32(1):372–397, 2010.

[Sch00]

W. Schoutens. Stochastic processes and orthogonal polynomials. Springer, New York, USA, 2000.

[SD88]

M. Shinozuka and G. Deodatis. Response variability of stochastic finite element systems. J. Eng. Mech., 114:499–519, 1988.

[SD00]

B. Sudret and A. Der Kiureghian. Stochastic finite elements and reliability: a state-of-the-art report. Technical report UCB/SEMM2000/08, University of California, Berkeley, USA, 2000.

[SG04]

C. Soize and R. Ghanem. Physical systems with random uncertainties: chaos representations with arbitary probability measure. SIAM J. Sci. Comput., 26(2):395–410, 2004.

210

BIBLIOGRAPHY

[SHD04]

M. Sala, M. A. Heroux, and D. M. Day. Trilinos Tutorial. Technical Report SAND2004-2189, Sandia National Laboratories, 2004.

[Smo63]

S. A. Smolyak. Quadrature and interpolation formulas for tensor products of certain classes of functions. Soviet Mathematics Dokl., 4:240– 243, 1963.

[SRNV07] B. Seynaeve, E. Rosseel, B. Nicola¨ı, and S. Vandewalle. Fourier mode analysis of multigrid methods for partial differential equations with random coefficients. J. Comput. Phys., 224:132–149, 2007. [ST82]

K. St¨ uben and U. Trottenberg. Multigrid Methods, volume 960 of Lecture Notes in Mathematics, chapter Multigrid methods: Fundamental algorithms, model problem analysis and applications, pages 1–176. Springer-Verlag, Berlin, Germany, 1982.

[ST03]

C. Schwab and R. A. Todor. Sparse finite elements for elliptic problems with stochastic loading. Numer. Math., 95(4):707–734, 2003.

[ST06]

C. Schwab and R. A. Todor. Karhunen-Loève approximation of random fields by generalized fast multipole methods. J. Comput. Phys., 217(1):100–122, 2006.

[Ste09]

G. Stefanou. The stochastic finite element method: past, present and future. Comput. Methods Appl. Mech. Engrg., 198:1031–1051, 2009.

[St¨ u01]

K. St¨ uben. A review of algebraic multigrid. J. Comput. Appl. Math., 128:281–309, 2001.

[SWLB09] S. Sarkar, J. A. S. Witteveen, A. Loeven, and H. Bijl. Effect of uncertainty on the bifurcation behavior of pitching airfoil stall flutter. Journal of Fluids and Structures, 25:301–320, 2009. [Sze67]

G. Szeg¨ o. Orthogonal polynomials. Amer. Math. Soc. Colloq. Publ. XXIII, AMS, Providence, USA, 4th edition, 1967.

[TLNE09] J. Tryoen, O. Le Maˆıtre, M. Ndjinga, and A. Ern. Multi-resolution analysis and upwinding for uncertain nonlinear hyperbolic systems. Technical report, Université Paris Est, CERMICS, 2009. [Tod06]

R. A. Todor. Robust eigenvalue computation for smoothing operators. SIAM J. Numer. Anal., 44(2):865–878, 2006.

[TOS01]

U. Trottenberg, C. W. Oosterlee, and A. Sch¨ uller. Multigrid. Academic Press, San Diego, USA, 2001.

[TS07]

R. A. Todor and C. Schwab. Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients. IMA J. Numer. Anal., 27(2):232–261, 2007.

BIBLIOGRAPHY

211

[Tyr92]

E. E. Tyrtyshnikov. Optimal and superoptimal circulant preconditioners. SIAM J. Matrix Anal. Appl., 13(2):459–473, 1992.

[Ull08]

E. Ullmann. Solution strategies for stochastic finite element discretizations. PhD thesis, Technische Universität Bergakademie Freiberg, Freiberg, Germany, 2008.

[Ull10]

E. Ullmann. A Kronecker product preconditioner for stochastic Galerkin finite element discretizations. SIAM J. Sci. Comput., 32(2):923–946, 2010.

[Van84]

E. Vanmarcke. Random fields: analysis and synthesis. MIT press, Cambridge, MA, USA, 1984.

[van92]

H. A. van der Vorst. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., 13(2):631–644, 1992.

[vS06]

T. von Petersdorff and C. Schwab. Sparse finite element methods for operator equations with stochastic data. Applications of Mathematics, 51(2):145–180, 2006.

[VV94]

H. Van der Vorst and C. Vuik. GMRESR: a family of nested GMRES methods. Numer. Linear Algebr., 1(4):369–386, 1994.

[VV05]

J. Van lent and S. Vandewalle. Multigrid methods for implicit RungeKutta and boundary value method discretizations of parabolic PDEs. SIAM J. Sci. Comput., 27(1):67–92, 2005.

[WB08a]

J. A. S. Witteveen and H. Bijl. An alternative unsteady adaptive stochastic finite element formulation based on interpolation at constant phase. Comput. Methods Appl. Mech. Engrg., 198:578–591, 2008.

[WB08b]

J. A. S. Witteveen and H. Bijl. Efficient quantification of the effect of uncertainties in advection-diffusion problems using polynomial chaos. Numer. Heat Tr. B.-Fund., 53:437–465, 2008.

[WB08c]

J. A. S. Witteveen and H. Bijl. A monomial chaos approach for efficient uncertainty quantification in nonlinear problems. SIAM J. Sci. Comput., 30(3):1296–1317, 2008.

[WB08d]

J. A. S. Witteveen and H. Bijl. An unsteady adaptive stochastic finite element formulation for rigid-body fluid-structure interaction. Computers and Structures, 86:2123–2140, 2008.

[WB09]

J. A. S. Witteveen and H. Bijl. Higher period stochastic bifurcation of nonlinear airfoil fluid-structure interaction. Math. Probl. Eng., 2009:1– 26, 2009.

212

BIBLIOGRAPHY

[WE06]

C.-T. Wu and H. C. Elman. Analysis and comparison of geometric and algebraic multigrid for convection–diffusion equations. SIAM J. Sci. Comput., 28(6):2208–2228, 2006.

[Wes92]

P. Wesseling. An introduction to multigrid methods. Wiley, Chichester, England, 1992.

[Wie38]

N. Wiener. The homogenous chaos. Am. J. Math., 60(4):897–936, 1938.

[WJ05]

R. Wienands and W. Joppich. Practical Fourier Analysis for Multigrid Methods. Chapman & Hall CRC Press, Boca Raton, Florida, USA, 2005.

[WK05]

X. Wan and G. E. Karniadakis. An adaptive multi-element generalized polynomial chaos method for stochastic differential equations. J. Comput. Phys., 209:617–642, 2005.

[WK06a]

X. Wan and G. E. Karniadakis. Beyond Wiener-Askey expansions: handling arbitrary PDF’s. J. Sci. Comput., 27(1–3):455–464, 2006.

[WK06b]

X. Wan and G. E. Karniadakis. Long-term behavior of polynomial chaos in stochastic flow simulations. Comput. Methods Appl. Mech. Engrg., 195(41–43):5582–5596, 2006.

[WK06c]

X. Wan and G. E. Karniadakis. Multi-element generalized polynomial chaos for arbitrary probability measures. SIAM J. Sci. Comput., 28(3):901–928, 2006.

[WK09a]

X. Wan and G. E. Karniadakis. Error control in multi-element generalized polynomial chaos method for elliptic problems with random coefficients. Commun. Comput. Phys., 5(2-4):793–820, 2009.

[WK09b]

X. Wan and G. E. Karniadakis. Solving elliptic problems with nonGaussian spatially-dependent random coefficients. Comput. Methods Appl. Mech. Engrg., 198:1985–1995, 2009.

[WLB09]

J. A. S. Witteveen, A. Loeven, and H. Bijl. An adaptive stochastic finite elements approach based on Newton-Cotes quadrature in simplex elements. Computers & Fluids, 38:1270–1288, 2009.

[WM68]

H. Woodson and J. Melcher. Electromechanical dynamics, volume 2. John Wiley and Sons, New York, USA, 1968.

[WO01]

R. Wienands and C. W. Oosterlee. On three-grid Fourier analysis for multigrid. SIAM J. Sci. Comput., 23(2):651–671, 2001.

BIBLIOGRAPHY

213

[WSB07]

J. Witteveen, S. Sarkar, and Hester Bijl. Modeling physical uncertainties in dynamic stall induced fluid-structure interaction of turbine blades using arbitrary polynomial chaos. Computers and Structures, 85:866–878, 2007.

[WXK04]

X. Wan, D. Xiu, and G. E. Karniadakis. Stochastic solutions for the two-dimensional advection-diffusion equation. SIAM J. Sci. Comput., 26(2):578–590, 2004.

[XH05]

D. Xiu and J. S. Hesthaven. High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput., 27(3):1118–1139, 2005.

[Xiu07]

D. Xiu. Efficient collocational approach for parametric uncertainty analysis. Commun. Comput. Phys., 2(2):293–309, 2007.

[Xiu09]

D. Xiu. Fast numerical methods for stochastic computations: a review. Commun. Comput. Phys., 5(2-4):242–272, 2009.

[XK02a]

D. Xiu and G. E. Karniadakis. Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Comput. Methods Appl. Mech. Engrg., 191:4927–4948, 2002.

[XK02b]

D. Xiu and G. E. Karniadakis. The Wiener–Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput., 24(2):619– 644, 2002.

[XK03a]

D. Xiu and G. E. Karniadakis. Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput. Phys., 187:137–167, 2003.

[XK03b]

D. Xiu and G. E. Karniadakis. A new stochastic approach to transient heat conduction model with uncertainty. Int. J. Heat Mass Tran., 46:4681–4693, 2003.

[XLSK02] D. Xiu, D. Lucor, S.-H. Su, and G. E. Karniadakis. Stochastic modeling of flow structure interactions using generalized polynomial chaos. J. Fluids Eng., 124:51–59, 2002. [XS07]

D. Xiu and J. Shen. An efficient spectral method for acoustic scattering from rough surfaces. Commun. Comput. Phys., 2(1):54–72, 2007.

[XS09]

D. Xiu and J. Shen. Efficient stochastic Galerkin methods for random diffusion equations. J. Comput. Phys., 228:266–281, 2009.

[XT06]

D. Xiu and D. M. Tartakovsky. Numerical methods for differential equations in random domains. SIAM J. Sci. Comput., 28(3):1167– 1185, 2006.

214

BIBLIOGRAPHY

[Yav95]

I. Yavneh. Multigrid smoothing factors for red-black Gauss-Seidel relaxation applied to a class of elliptic operators. SIAM J. Numer. Anal., 32(4):1126–1138, 1995.

[Yav96]

I. Yavneh. On red-black SOR smoothing in multigrid. SIAM J. Sci. Comput., 17(1):180–192, 1996.

[Zad65]

L A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965.

[ZL04]

D. Zhang and Z. Lu. An efficient, high-order perturbation approach for flow in random porous media via Karhunen-Loève and polynomial expansions. J. Comput. Phys., 194:773–794, 2004.

[ZWH84]

T. A. Zang, Y. S. Wong, and M. Y. Hussaini. Spectral multigrid methods for elliptic equations II. J. Comput. Phys., 54:489–507, 1984.

Curriculum vitae Eveline Rosseel ◦ December 6, 1982, Leuven (Belgium)

Higher education 2005-2010 Ph.D. in Engineering, K.U.Leuven, Leuven, Belgium Thesis: Multigrid algorithms for stochastic finite element computations 2000-2005 Engineer in Computer Science (Burgerlijk Ingenieur in de Computerwetenschappen), K.U.Leuven, Leuven, Belgium Thesis: Iterative methods for the mixed formulations of the Maxwell equations

Publications in reviewed journals • E. Rosseel, H. De Gersem, and S. Vandewalle. Spectral stochastic simulation of a ferromagnetic cylinder rotating at high speed. IEEE Transactions on Magnetics, accepted 2010. • E. Rosseel, H. De Gersem, and S. Vandewalle. Nonlinear stochastic Galerkin and collocation simulations: application to a ferromagnetic cylinder rotating at high speed. Comm. Comput. Phys. 8(5), pp. 947–975, 2010. • E. Rosseel, and S. Vandewalle. Iterative solvers for the stochastic finite element method. SIAM J. Sci. Comput. 32, pp. 372–397, 2010. • E. Rosseel, T. Boonen, and S. Vandewalle. Algebraic multigrid for stationary and time-dependent partial differential equations with stochastic coefficients. Numer. Linear Algebra Appl. 15(2-3), pp. 141–163, 2008.

215

216

CURRICULUM VITAE

• G. Deliége, E. Rosseel, and S. Vandewalle. Iterative solvers and stabilisation for mixed electrostatic and magnetostatic formulations. J. Comput. Appl. Math. 215, pp. 348–356, 2008. • B. Seynaeve, E. Rosseel, B. Nicola¨ı, and S. Vandewalle. Fourier mode analysis of multigrid methods for partial differential equations with random coefficients. J. Comput. Phys. 224(1), pp. 132–149, 2007.

Publications in conference proceedings • E. Rosseel, and S. Vandewalle. Iterative solvers for the stochastic finite element method, Proceedings of the Leuven Symposium on Applied Mechanics in Engineering (CDROM) (B. Bergen, M. De Munck, W. Desmet, D. Moens, B. Pluymers, G. Schuëller, D. Vandepitte, eds.), pp. 801–815, 2008. • G. Deliége, E. Rosseel, and S. Vandewalle. Iterative solvers and stabilisation for mixed electrostatic and magnetostatic formulations, Proceedings of the 3rd International Conference on Advanced Computational Methods in Engineering (CDROM) (E. Dicks, J. Vierendeels, L. Vandevelde, L. Dupré, M. Slodicka, R. Van Keer, eds.), pp.1-10, 2005.

Conference presentations • E. Rosseel, N. Scheerlinck, and S. Vandewalle. A multigrid solver for biological reaction-diffusion problems with random coefficients, Fifteenth International Congress on Computational and Applied Mathematics, Leuven, Belgium, July 5–9, 2010. • E. Rosseel, and S. Vandewalle. Iterative solvers for stochastic Galerkin finite element discretizations, SIAM Conference on Applied Linear Algebra, Seaside, California, USA, October 26–29, 2009. • E. Rosseel, and S. Vandewalle. Iterative solvers for the stochastic finite element method, Biennial Conference on Numerical Analysis, Glasgow, Scotland, June 23–26, 2009. • E. Rosseel, and S. Vandewalle. A multigrid solver for the stochastic finite element method, Ninth European Multigrid Conference, Bad Herrenalb, Germany, October 20–23, 2008. • E. Rosseel, and S. Vandewalle. Iterative solvers for the stochastic finite element method, Thirteenth International Congress on Computational and Applied Mathematics, Ghent, Belgium, July 7–11, 2008.

CURRICULUM VITAE

217

• E. Rosseel, and S. Vandewalle. A comparison of iterative solvers for the stochastic finite element method, Tenth Copper Mountain Conference on Iterative Methods, Copper Mountain, Colorado, USA, April 6–11, 2008. • E. Rosseel, and S. Vandewalle. Iterative solvers for the stochastic finite element method, Leuven Symposium on Applied Mechanics in Engineering, Non-Deterministic Modelling workshop, Leuven, Belgium, March 31–April 2, 2008. • E. Rosseel, and S. Vandewalle. Algebraic Multigrid for stationary and timedependent PDEs with stochastic coefficients, Thirteenth Copper Mountain Conference on Multigrid Methods, Copper Mountain, Colorado, USA, March 17–23, 2007. • E. Rosseel, and S. Vandewalle. An algebraic multigrid method for stationary and time-dependent PDEs with stochastic coefficients, Twelfth International Congress on Computational and Applied Mathematics, Leuven, Belgium, July 10–14, 2006.

Seminars • Nonlinear stochastic Galerkin and collocation simulations: application to a ferromagnetic cylinder rotating at high speed, Doctoral Seminar, Katholieke Universiteit Leuven, Belgium, June 8, 2009. • Iterative solvers for the stochastic finite element method, Seminar Numerik, Institut f¨ ur Numerische Mathematik und Optimierung, Bergakademie TU Freiberg, March 4, 2009. • Multigrid for partial differential equations with stochastic coefficients, Oberseminar Numerik / Wissenschaftliches Rechnen, Max Planck Institute for Mathematics in the Sciences, Leipzig, Germany, May 22, 2007. • Algebraic multigrid for stationary and time-dependent PDEs with stochastic coefficients, Doctoral Seminar, Katholieke Universiteit Leuven, Belgium, November 20, 2006.

Teaching 2005-2010 Teaching assistant for Numerical Modelling and Approximation 2005-2010 Teaching assistant for Problem Solving and Engineering Design, part 1

218

CURRICULUM VITAE

2007-2008 Co-supervision of master thesis Uncertainty representation in mathematical models: a fuzzy arithmetic approach, Muna Rabaiah 2008-2009 Co-supervision of master thesis Multigrid methods for boundary control of periodic, parabolic partial differential equations, Dirk Abbeloos

Arenberg Doctoral School of Science, Engineering & Technology Faculty of Engineering Department of Computer Science Scientific Computing Research Group Celestijnenlaan 200A bus 2402, B-3001 Leuven (Belgium)

Multigrid algorithms for stochastic finite element

Multigrid algorithms for stochastic finite element

Suggest Documents

GALERKIN FINITE ELEMENT APPROXIMATIONS FOR STOCHASTIC ...

Stochastic Finite Element Method for Torso ...

ITERATIVE SOLVERS FOR THE STOCHASTIC FINITE ELEMENT

Adaptive error control for multigrid finite element - Springer Link

A Parallel Multigrid Method for the Finite Element Analysis of ...

Finite element algorithms for thermoelastic wear problems

Adaptive implicit-explicit finite element algorithms for

Efficient Finite Element Geometric Multigrid Solvers for Unstructured ...

a multigrid finite element method for the mesoscale

Perturbation Stochastic Finite Element Analysis of Thermoelastic

Stochastic finite element with material uncertainties

FINITE ELEMENT APPROXIMATIONS OF THE STOCHASTIC MEAN

GALERKIN FINITE ELEMENT APPROXIMATIONS OF STOCHASTIC

Stochastic finite element methods in geotechnical engineering

Multigrid finite element methods on semi ... - Semantic Scholar

Parallel Stochastic Dynamic Programming: Finite Element ... - CiteSeerX

STOCHASTIC FINITE ELEMENT ANALYSIS OF GROUNDWATER

multigrid waveform relaxation on spatial finite element ... - SIAM epubs

Fast simplicial finite element algorithms using

Fast Algorithms for Solving High-Order Finite Element Equations for ...

A stochastic finite element procedure for moment and ...

A finite element approximation for the stochastic Maxwell--Landau ...

Sparse finite element methods for operator equations with stochastic ...

Finite Element Methods for the Stochastic Allen-Cahn Equation with ...