Stochastic Galerkin and collocation methods for PDEs with ... - ICMS

Stochastic Galerkin and collocation methods for PDEs with random coefficients: a numerical comparison Fabio Nobile MOX, Department of Mathematics, Politecnico di Milano Acknowledgements: J. B¨ack, L. Tamellini, R. Tempone Uncertainty Quantification Workshop, Edinburgh May 24-28, 2010

Fabio Nobile (MOX, Politecnico di Milano)

Galerkin and Collocation comparison

UQ Workshop Edinburgh, May 24-28, 2010

Outline

1

Elliptic PDE with random coefficients

2

Stochastic Polynomial approximation / Galerkin versus Collocation

3

Numerical comparison





Elliptic PDE with random coefficients Let (Ω, F, P) be a complete probability space. Consider the problem ( − div(a(ω, x)∇u(ω, x)) = f (x) for a.e. x ∈ D, ω ∈ Ω, u(ω, x) = 0 for a.e. x ∈ ∂D, ω ∈ Ω a(ω, x) is a random field parametrized with N independent random variables (finite dimensional noise assumption) a(ω, x) = a(Y1 (ω), . . . , YN (ω), x) Then u(ω, x) = u(Y1 (ω), . . . , YN (ω), x) is a deterministic function of the random vector Y. Remark: u is in general analytic w.r.t. Y. We assume QN that Y has Na joint +probability density function ρ(y) = n=1 ρn (yn ) : Γ → R Fabio Nobile (MOX, Politecnico di Milano)




















Examples Material with inclusions of random conductivity

a(ω, x) = a0 +

1

PN

n=8 Yn (ω) Ωn (x)

with Yn ∼ uniform, lognormal, ...

Random, spatially correlated, material properties a(ω, x) is ∞-dimensional random field (e.g. lognormal), suitably truncated a(ω, x) ≈ amin + e

PN

n=1

Yn (ω)bn (x)

with Yn ∼ N(0, 1), i.i.d. Fabio Nobile (MOX, Politecnico di Milano)




Examples Material with inclusions of random conductivity

a(ω, x) = a0 +

1

PN

n=8 Yn (ω) Ωn (x)

with Yn ∼ uniform, lognormal, ...

Random, spatially correlated, material properties a(ω, x) is ∞-dimensional random field (e.g. lognormal), suitably truncated

random field with Lc=1/4

1 0.9

2.5

0.8

2

0.7 1.5

0.6

a(ω, x) ≈ amin + e

PN

n=1

Yn (ω)bn (x)

0.5

1

0.4

0.5

0.3

0

0.2 −0.5

0.1 0

with Yn ∼ N(0, 1), i.i.d. Fabio Nobile (MOX, Politecnico di Milano)


0

0.2

0.4

0.6

0.8

1

−1



Stochastic polynomial approximation Idea: approximate the response function u(y, ·) by global multivariate polynomials. Since u(y, ·) is analytic, we expect fast convergence. Polynomial space ( PΛw (ΓN ) = span

N Y

) ynpn ,

with p ∈ Λw ,

with Λw ⊂ NN index set

n=1

Approximation: uw (y, x) =


P

p∈Λwup (x)ψp (y),


with {ψp } basis of PΛw




N Y

) ynpn ,

with p ∈ Λw ,


n=1



P







)

N Y

ynpn ,


with p ∈ Λw ,

n=1


P

9

9

8

8

7

7

6

6

5

5

4

4

3

3

2

2

1

1

0



0

0

1

2

3

4

5

6

7

8

9

Total Degree: P n pn ≤ w Fabio Nobile (MOX, Politecnico di Milano)

0

1

2

3

4

5

6

7

8

9

Hyperbolic Cross: Q n (pn + 1) ≤ w + 1 Galerkin and Collocation comparison




)

N Y

ynpn ,


with p ∈ Λw ,

n=1

Approximation: uw (y, x) = 9

P


p∈Λwup (x)ψp (y), 9

TD TD−aniso

8

HC HC−aniso

8

7

7

6

6

5

5

4

4

3

3

2

2

1

1

0

0

0

1

2

3

4

5

6

7

8

Aniso Total Degree: P n αn p n ≤ w Fabio Nobile (MOX, Politecnico di Milano)

9

0

1

2

3

4

5

6

7

8

9

Aniso Hyperbolic Cross: Q αn ≤ w + 1 n (pn + 1) Galerkin and Collocation comparison



Stochastic Galerkin approximation Project the equation onto the subspace VΛw = PΛw (ΓN ) ⊗ H01 (D) [Ghanem-Spanos, Karniadakis et al, Matthies-Keese, Schwab-Todor et al., Knio-Le Maˆıtre et al,Babuska et al.,. . . ]

Find uwSG ∈ VΛw such that ∀v ∈ VΛw Z Z SG f (x)v (y, x) dx E a(y, x)∇uw (y, x) · ∇v (y, x) dx = E D

D

Leads to M coupled deterministic problems with sparse matrix. We solve it by PCG [Ghanem-Pellissetti, Elman-Powell et al., ...] with mean-based preconditioner (block diagonal) Fabio Nobile (MOX, Politecnico di Milano)






D



0

20

40

60

80

100

120

140

160 0

20

40

60

80 100 nz = 885

120

140

160





D



0

20

40

60

80

100

120

140

160 0

20

40

60

80 100 nz = 885

120

140

160



Stochastic Collocation approximation Collocate the equation in a set of points yk ([Tatang 95], [Mathelin-Hussaini ’03], [Hesthaven-Xiu ’05], [Babuˇska -N.-Tempone ’05], [Zabaras et al ’06], [Karniadakis et al ’08] . . . )

Generalized sparse grid construction m(i) 1 Define 1D polynomial interpolants Un on m(i) Gauss knots; 2 3

m(i)

m(i)

m(i−1)

Define ∆n = Un − Un ; N Given an increasing function X g : N → N, define uwSC = Swm,g (u) = ∆1 m(i1 ) ⊗ · · · ⊗ ∆N m(iN ) (u) i∈N N : g (i)≤w

Theorem [Back-N.-Tamellini-Tempone ’10]. Given a Polynomial space PΛ there exist functions m and g , such that Swm,g (u) ∈ PΛ and S m,g is exact in PΛ . Fabio Nobile (MOX, Politecnico di Milano)






m(i)

m(i)

m(i−1)








m(i)

m(i)

m(i−1)






Equivalence table SC: Tensor Product (TP) Total Degree (TD) Hyperbolic Cross (HC) Smolyak (SM)

m, rule (g )

m(i) = i g (i) = maxn (in − 1) ≤ w m(i) = i P g (i) = n (in − 1) ≤ w m(i) = i Q g (i) =( n (in ) ≤ w + 1 2i−1 + 1, i > 1 m(i) = 1, i = 1 g (i) =


P

n (in

− 1) ≤ f (w )

SG:

polynomial space

{p ∈ NN : maxn pn ≤ w } {p ∈ NN : {p ∈ NN :

Q

P

n (pn

n

pn ≤ w }

+ 1) ≤ w + 1}

P {p ∈ NN : n f (pn ) ≤ f (w )} ( 0, for p = 0; 1, for p = 1 f (p) = dlog2 (p)e p ≥ 2




Equivalence table SC: Tensor Product (TP) Total Degree (TD) Hyperbolic Cross (HC)

m, rule (g )

Smolyak (SM)

g (i) =

P

n (in

polynomial space

{p ∈ NN : {p ∈ NN :

Q

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0

−0.2

−0.2

−0.4

−0.4

−0.4

−0.6

−0.6

−0.6

−0.8

−0.8

−0.4

−0.2

0

y1

0.2

0.4

0.6

0.8

1


−1 −1

+ 1) ≤ w + 1}

0

−0.2

−0.6

pn ≤ w }

0.2

y2

y2

1

0.8

0

n

Smolyak Gauss (SM)

1

0.8

−0.8

n (pn

Hyperbolic Cross (HC)

1

0.8

−1 −1

P


− 1) ≤ f (w )

Total Degree (TD)

y2

SG:

{p ∈ NN : maxn pn ≤ w }

m(i) = i g (i) = maxn (in − 1) ≤ w m(i) = i P g (i) = n (in − 1) ≤ w m(i) = i Q g (i) =( n (in ) ≤ w + 1 2i−1 + 1, i > 1 m(i) = 1, i = 1

−0.8

−0.8

−0.6

−0.4

−0.2

0

y1

0.2

0.4

0.6

0.8

1


−1 −1

−0.8

−0.6

−0.4

−0.2

0

y1

0.2

0.4

0.6

0.8

1



Equivalence table SC: Tensor Product (TP) Total Degree (TD) Hyperbolic Cross (HC)

m, rule (g )

SG:

Smolyak (SM)

g (i) =

P

n (in

{p ∈ NN :

Q

P

n (pn

1

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

y2

0

0

0

−0.2

−0.2

−0.2

−0.4

−0.4

−0.4

−0.6

−0.6

−0.6

−0.8

−0.8

−0.6

−0.4

−0.2

0

y1

0.2


0.4

0.6

+ 1) ≤ w + 1}

Aniso Smolyak Gauss (SM)

1

0.8

−0.8

pn ≤ w }

Aniso Hyperbolic Cross (HC)

1

−1 −1

n


0.8

y2

2

{p ∈ NN :

− 1) ≤ f (w )

Aniso Total Degree (TD)

y

polynomial space

{p ∈ NN : maxn pn ≤ w }

m(i) = i g (i) = maxn (in − 1) ≤ w m(i) = i P g (i) = n (in − 1) ≤ w m(i) = i Q g (i) =( n (in ) ≤ w + 1 2i−1 + 1, i > 1 m(i) = 1, i = 1

0.8

1

−1 −1

−0.8

−0.8

−0.6

−0.4

−0.2

0

y1

0.2

0.4

0.6


0.8

1

−1 −1

−0.8

−0.6

−0.4

−0.2

0

y1

0.2

0.4

0.6

0.8

1



Random inclusions – uniform Conductivity coefficient: matrix k=1 8 circular inclusions: k|Ωi ∼ U(0.01, 0.8), i.i.d. forcing term f = 1001F zero boundary conditions quantity of interest ψ(u) =

R F

u

std

mean Computational cost: ] calls to deterministic solver 1 Galerkin : ]stocDoF × ]iterPCG 2 Collocation : ]collocation points Fabio Nobile (MOX, Politecnico di Milano)




Random inclusions – uniform Conductivity coefficient: matrix k=1 8 circular inclusions: k|Ωi ∼ U(0.01, 0.8), i.i.d. forcing term f = 1001F zero boundary conditions quantity of interest ψ(u) =

R F

u

std

mean Computational cost: ] calls to deterministic solver 1 Galerkin : ]stocDoF × ]iterPCG 2 Collocation : ]collocation points Fabio Nobile (MOX, Politecnico di Milano)




Error on E[ψ(u)] versus Computational Cost −2

10

−4

E(ψ1,p) − E(ψ1,ovk)

10

−6

10

−8

10

Collocation

−10

10

MC SG−TP SG−TD SG−HC SG−SM

−12

10

−14

10

0

10

1

10

2

10

3

10

4

10

5

10

6

10

SG: dim−stoc * iter−CG / MC: sample size

Galerkin





Error on E[ψ(u)] versus Computational Cost −2

−2

10

10

−4

−4

10

−6



10

10

−8

10

−10

10

MC SG−TP SG−TD SG−HC SG−SM

−12

10

−14

10

0

10

1

10

−6

10

−8

10

SG−TD MC SC−TP SC−TD SC−HC SC−SM−G SC−SM−CC

−10

10

−12

10

−14

2

10

3

10

4

10

5

10

SG: dim−stoc * iter−CG / MC: sample size

Galerkin


6

10

10

0

10

1

2

10

10

3

10

4

10

5

10

6

10

SC: num−pts / SG: dim−stoc * iter−CG / MC: sample size

Collocation




Error on E[ψ(u)] versus Dim. Polynomial Space −2

10

SG−TD SG−SM SC−TD SC−SM

−4


10

−6

10

−8

10

−10

10

−12

10

−14

10

0

10

1

10

2

3

10

10

4

10

5

10

dim−stoc





On optimal spaces for Stochastic Galerkin Why TD space is the best for Stochastic Galerkin? Answer: P 1 Expand u in Legendre series: u(·, y) = p∈NN cp (·)ψp (y) with {ψp } multivariate orthonormal Legendre polynomials. 2 Since u is analytic, the Fourier coefficients decay exponentially: ! N X Y gn pn kcp kH 1 (D) ∼ e −gn pn = exp − n

n=1

3

In this example gn ≈ g for all n (isotropic problem) Optimal index set Λ of cardinality M: the one corresponding to the M largest Fourier coefficients. X kcp kH 1 (D) ≥ e −gw ⇒ Λ ≡ {p ∈ NN : pn ≤ w } (TD space) n






n=1

3







n=1

3







n=1

3






Random inclusions – lognormal 3

( 1+ uniform) d.f. (1 + l0 + lognormal) d.f.

Zi

k|Ωi = l0 + e , with Zi ∼ N(µ, σ 2 ), i.i.d. l0 , µ, σ so as to have same mean, var. and min. value as uniform test use Hermite polynomials

2.5

2

1.5

1

0.5

0 −0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Optimal space for Stochastic Galerkin In this case the coefficients of the Hermite expansion decay as ! Y X √ √ kcp kH 1 (D) ∼ e −gn pn = exp − gn pn n

n

We introduce the new space square root (SQ) X√ Λ ≡ {p ∈ NN : pn ≤ w } n Fabio Nobile (MOX, Politecnico di Milano)




Random inclusions – lognormal 3

( 1+ uniform) d.f. (1 + l0 + lognormal) d.f.

Zi

k|Ωi = l0 + e , with Zi ∼ N(µ, σ 2 ), i.i.d. l0 , µ, σ so as to have same mean, var. and min. value as uniform test use Hermite polynomials

2.5

2

1.5

1

0.5

0 −0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Optimal space for Stochastic Galerkin In this case the coefficients of the Hermite expansion decay as ! Y X √ √ kcp kH 1 (D) ∼ e −gn pn = exp − gn pn n

n

We introduce the new space square root (SQ) X√ Λ ≡ {p ∈ NN : pn ≤ w } n Fabio Nobile (MOX, Politecnico di Milano)




Convergence plots 0

0

10

10 SG−TP−L SG−TD−L SG−HC−L SG−SM−L SG−SQ−L

10

−4

10

−6

10

−8

10

−10

10

10

−4

10

−6

10

−8

10

−10

10

−12

10

SG−TP−L SG−TD−L SG−HC−L SG−SM−L SG−SQ−L SC−TP−L SC−TD−L SC−HC−L SC−SM−L SC−SQ−L montecarlo

−2

( E(ψ1,p) − E(ψ1,ovk) )/ E(ψ1,ovk)

( E(ψ1,p) − E(ψ1,ovk) )/ E(ψ1,ovk)

−2

−12

0

10

2

4

6

8

10 10 10 10 SG: dim−stoc * iter−CG / SC: nb points/ MC: sample size

10

0

10

2

4

6

8


SG method

SC method

Remarks TD underperforms in this case. SQ is the best space Higher extra-cost between Galerkin and SC due to ill conditioning of the Galerkin matrix. Fabio Nobile (MOX, Politecnico di Milano)




Convergence plots 0

0

10

10 SG−TP−L SG−TD−L SG−HC−L SG−SM−L SG−SQ−L

10

−4

10

−6

10

−8

10

−10

10

10

−4

10

−6

10

−8

10

−10

10

−12

10

SG−TP−L SG−TD−L SG−HC−L SG−SM−L SG−SQ−L SC−TP−L SC−TD−L SC−HC−L SC−SM−L SC−SQ−L montecarlo

−2

( E(ψ1,p) − E(ψ1,ovk) )/ E(ψ1,ovk)

( E(ψ1,p) − E(ψ1,ovk) )/ E(ψ1,ovk)

−2

−12

0

10

2

4

6

8


10

0

10

2

4

6

8


SG method

SC method

Remarks TD underperforms in this case. SQ is the best space Higher extra-cost between Galerkin and SC due to ill conditioning of the Galerkin matrix. Fabio Nobile (MOX, Politecnico di Milano)




Performance of PCG iterations

80

80

70

70

60

60

50

50

iter CG

iter CG

The mean-based preconditioner is less effective for logn variables. PCG requires more iterations than for uniform random variables (for the same variance) [Powell-Ullmann 2009, MIMS tech.report] Conversely, no additional cost for Collocation (]coll.points is the same)

40

30

uniform TD lognormal TD uniform SM lognormal SM

40

30

20

20

10

10

uniform test lognormal test 0 0 10

1

10

2

10

3

10

4

10

5

10

0 0 10

log(dim−stoc)

2

10

3

10

4

10

5

10

log(dim−stoc)

PCG iterations for TD Fabio Nobile (MOX, Politecnico di Milano)

1

10

PCG iterations for SM




Thermal conduction – anisotropic version Conductivity coefficient: matrix k=1 4 circular inclusions: k|Ωi ∼ γi U(0.01, 0.8) forcing term f = 100 zero boundary conditions quantity of interest ψ(u) =

mean Fabio Nobile (MOX, Politecnico di Milano)


R F

u

std UQ Workshop Edinburgh, May 24-28, 2010


Error on E[ψ(u)] versus Computational Cost Approximation in Anisotropic Total Degree polynomial spaces Weights αn estimated theoretically or numerically Galerkin


Galerkin versus Collocation




Error on E[ψ(u)] versus Computational Cost Approximation in Anisotropic Total Degree polynomial spaces Weights αn estimated theoretically or numerically 0

10

−1

10

−2

E(ψ1)

10

−3

10


−4

10

MC SG−isospaces SG−1−7−11−15 SG−1−2−3−4 SG−1−2.5−4−5.5(exp) SG−1−3.5−5.5−7.5(th)

−5

10

−6

10

0

10

2

4

10 10 SG: dim−stoc * iter−CG / MC: sample size

6

10

Galerkin





Error on E[ψ(u)] versus Computational Cost Approximation in Anisotropic Total Degree polynomial spaces Weights αn estimated theoretically or numerically 0

0

10

10

−1

−1

10

10

−2

−2

10 E(ψ1)

E(ψ1)

10

−3

10

−4

−4

10

10 MC SG−isospaces SG−1−7−11−15 SG−1−2−3−4 SG−1−2.5−4−5.5(exp) SG−1−3.5−5.5−7.5(th)

−5

10

−6

10

−3

10

0

10

2

MC SG−1−2.5−4−5.5(exp) SG−1−3.5−5.5−7.5(th) SC−1−2.5−4−5.5(exp) SC−1−3.5−5.5−7.5(th)

−5

10

−6

4

10 10 SG: dim−stoc * iter−CG / MC: sample size

Galerkin


6

10

10

0

10

1

2

3

4

5

10 10 10 10 10 SC: num−pts / SG: dim−stoc * iter−CG / MC: sample size





Conclusions It is possible to set Collocation and Galerkin in the same polynomial space Polynomial spaces / sparse grids can be tuned to the problem if we have information on the 1D convergence and weights Galerkin is better if we compare the accuracy of the methods w.r.t. the dimension of the underlying polynomial space (L2 optimality of Galerkin) Good preconditioners are crucial for Galerkin Collocation cost is less affected by the choice of rand. var. Collocation seems to be better in terms of error versus computational cost. However, a better preconditioner for Galerkin may lead to opposite conclusions The picture may change considerably for non-linear problems





References J. B¨ ack and F. Nobile and L. Tamellini and R. Tempone Stochastic Spectral Galerkin and collocation methods for PDEs with random coefficients: a numerical comparison, to appear in Proc. of ICOSAHOM09. LNCSE, Springer. I. Babuˇska, F. Nobile and R. Tempone. A stochastic collocation method for elliptic PDEs with random input data, SIAM Review, 52(2):317–355, 2010 F. Nobile and R. Tempone Analysis and implementation issues for the numerical approximation of parabolic equations with random coefficients, IJNME, 80:979–1006, 2009 F. Nobile, R. Tempone and C. Webster An anisotropic sparse grid stochastic collocation method for PDEs with random input data, SIAM J. Numer. Anal., 46(5):2411–2442, 2008 F. Nobile, R. Tempone and C. Webster A sparse grid stochastic collocation method for PDEs with random input data, SIAM J. Numer. Anal., 46(5), 2309–2345, 2008


