Kernel-based Collocation Methods versus Galerkin Finite ... - Qi Ye

Kernel-based Collocation Methods versus Galerkin Finite Element Methods for Approximating Elliptic Stochastic Partial Differential Equations Gregory E. Fasshauer and Qi Ye∗ Department of Applied Mathematics, Illinois Institute of Technology, Chicago, Illinois 60616 USA, [email protected] and [email protected] Summary. We compare a kernel-based collocation method (meshfree approximation method) with a Galerkin finite element method for solving elliptic stochastic partial differential equations driven by Gaussian noise. The kernel-based collocation solution is a linear combination of reproducing kernels obtained from related differential and boundary operators centered at chosen collocation points. Its random coefficients are obtained by solving a system of linear equations with multiple random right-hand sides. The finite element solution is given as a tensor product of triangular finite elements and Lagrange polynomials defined on a finite-dimensional probability space. Its coefficients are obtained by solving several deterministic finite element problems. For the kernel-based collocation method, we directly simulate the (infinite-dimensional) Gaussian noise at the collocation points. For the finite element method, however, we need to truncate the Gaussian noise into finite-dimensional noises. According to our numerical experiments, the finite element method has the same convergence rate as the kernel-based collocation method provided the Gaussian noise is truncated using a suitable number terms.

Key words: Kernel-based collocation, meshfree approximation, Galerkin finite element, elliptic stochastic partial differential equations, Gaussian fields, reproducing kernel.

1 Introduction Stochastic partial differential equations (SPDEs) form the basis of a recent, fast growing research area with many applications in physics, engineering and finance. However, it is often difficult to obtain an explicit form of the solution. Moreover, current numerical methods usually show limited success for highdimensional problems and in complex domains. In our recent papers [4, 11], ∗

Corresponding Author

2

G. E. Fasshauer and Q. Ye

we use a kernel-based collocation method to approximate the solution of highdimensional SPDE problems. Since parabolic SPDEs can be transformed into elliptic SPDEs using, e.g., an implicit time stepping scheme, solution of the latter represents a particularly important aspect of SPDE problems. In this paper, we compare the use of a kernel-based collocation method [4, 11] (meshfree approximation method) and a Galerkin finite element method [1, 2] to approximate the solution of elliptic SPDEs. For kernel-based collocation, we directly simulate the Gaussian noise at a set of collocation points. For the Galerkin finite element method, on the other hand, we use a truncated Karhunen-Loéve expansion of the Gaussian noise in order to satisfy a finite-dimensional noise condition. For kernel-based collocation the same collocation locations are used to construct the deterministic basis and the random part. For the Galerkin finite element method one needs to separately set up the finite element basis on the spatial domain and the polynomials on the probability space. For a given kernel function, the convergence rate of the collocation solution depends only on the fill distance of the collocation points. The convergence rate of the finite element solution depends on the maximum mesh spacing parameter and the degrees of the polynomials defined on the finite-dimensional probability space. According to our numerical experiments, the truncation length of the Gaussian noise also affects the convergence results of the finite element method. 1.1 Problem Setting Assume that D is a regular open bounded domain in Rd . Let the stochastic process ξ : D × Ωξ → R be Gaussian with mean zero and covariance kernel Φ : D×D → R defined on a probability space (Ωξ , Fξ , Pξ ) (see Definition A.1). We consider an elliptic SPDE driven by the Gaussian noise ξ with Dirichlet boundary conditions ( ∆u = f + ξ, in D, (1) u = 0, on ∂D, where ∆ is the Laplacian and f : D → R is a deterministic function. We can solve the SPDE (1) by either of the following two numerical methods. Kernel-based collocation method (KC): We simulate the Gaussian noise ξ with covariance structure Φ(x, y) at a finite collection of predetermined collocation points XD := {x1 , · · · , xN } ⊂ D,

X∂D := {xN +1 , · · · , xN +M } ⊂ ∂D

and approximate u using a kernel-based collocation method written as u(x) ≈ u ˆ(x) :=

N X k=1

∗

ck ∆2 K(x, xk ) +

M X k=1

∗

cN +k K(x, xN +k ),

x ∈ D,

Kernel-based Collocation vs. Galerkin Finite Element for SPDEs

3

∗

where K is an integral-type kernel associated with a reproducing kernel K (see Equation (5) in Appendix Appendix A.). Here ∆2 means that we differentiate ∗

∗

with respect to the second argument, i.e., ∆2 K(x, xk ) = ∆y K(x, y)|y=xk . T The unknown random coefficients c := (c1 , · · · , cN +M ) are obtained by solving a random system of linear equations (with constant deterministic system matrix and random right-hand side that varies with each realization of the noise). Details are provided in Section 2. Galerkin finite element method (FE): Since the Galerkin finite element method is based on a finite-dimensional noise assumption (see [1, 2]), assuming Φ ∈ L2 (D × D), we truncate the Gaussian noise ξ by a Karhunen-Loéve expansion, i.e., ξ ≈ ξ n :=

n X

√ ζk qk φk ,

and ζk ∼ i.i.d. N (0, 1),

k = 1, · · · , n,

k=1

where qRk and φk are eigenvalues and eigenfunctions of the covariance kernel Φ, i.e., D Φ(x, y)φk (y)dy = qk φk (x). We approximate the original SPDE (1) by another elliptic SPDE driven by the truncated Gaussian noise ξ n ( ∆un = f + ξ n , in D, (2) un = 0, on ∂D. Next we combine the finite element method in spatial domain D := D ∪ ∂D and the collocation in the zeros of suitable tensor product orthogonal polynomials (Gaussian points) in the finite-dimensional probability space. We obtain the approximation as a tensor product of the finite element solutions defined on the spatial domain and the Lagrange polynomials defined on the finitedimensional probability space, i.e., uh,p ≈ un ≈ u, where h is the maximum mesh spacing parameter and p = (p1 , · · · , pn ) is the degree of the Lagrange polynomials. Details are provided in Section 3. Remark 1.1. Because Φ is always positive semi-definite Mercer’s theorem ensures that its eigenvalues q1 ≥ q2 ≥ · · · ≥ 0 and its eigenfunctions {φk }∞ k=1 P∞ form an orthonormal base of L2 (D) so that Φ(x, y) = k=1 qk φk (x)φk (y). P 2 ∞ Therefore E kξ − ξ n kL2 (D) = k=n+1 qk → 0 when n → ∞, and we can accurately represent the infinite-dimensional noise ξ by a (potentially long) truncated Karhunen-Loéve expansion.

2 Kernel-based Collocation Method In the papers [4, 11] we use the Gaussian fields ∆S, S with means ∆µ, µ ∗

∗

and covariance kernels ∆1 ∆2 K, K (see Theorem A.1), respectively, to construct the collocation approximation u ˆ of the solution u of SPDE (1). Here ∗

∗

∆1 ∆2 K(xj , xk ) = ∆x ∆y K(x, y)|x=xj ,y=xk .

4


Because of the order O(∆) = 2, we suppose that the reproducing kernel Hilbert space HK (D) is embedded into the L2 -based Sobolev space Hm (D) where m > 2 + d/2. Remark 2.1. Since we want to interpolate the values of the differential equation at the collocation points, ∆ω(x) needs to be well-defined pointwise for each available solution ω ∈ HK (D) ⊆ Hm (D) ⊂ C2 (D). This requires the Sobolev space Hm (D) to be smooth enough. If we just need a weak solution as for the finite element method, then the order needs to satisfy m ≥ 2 only. Since ξ is Gaussian with a known correlation structure, we can simulate the values of ξ at the collocation points x1 , · · · , xN , i.e., ξ := (ξx1 , · · · , ξxN )T ∼ N (0, Φ),

where Φ := (Φ(xj , xk ))N,N j,k=1 .

M Consequently, we assume that the values {yj }N j=1 and {yN +j }j=1 defined by

yj := f (xj ) + ξxj ,

j = 1, · · · , N,

yN +j := 0,

j = 1, · · · , M,

are known. Moreover we can also obtain the joint probability density function py of the random vector y ξ := (y1 , · · · , yN +M )T . We define the product space ΩKξ := ΩK × Ωξ ,

FKξ := FK ⊗ Fξ ,

Pµξ := Pµ ⊗ Pξ ,

where the probability measure Pµ is defined on (HK (D), B(HK (D))) = (ΩK , FK ) as in Theorem A.1, and the probability space (Ωξ , Fξ , Pξ ) is given in the SPDE (1). We assume that the random variables defined on the original probability spaces are extended to random variables on the new probability space in the natural way: if random variables V1 : ΩK → R and V2 : Ωξ → R are defined on (ΩK , FK , Pµ ) and (Ωξ , Fξ , Pξ ), respectively, then V1 (ω1 , ω2 ) := V1 (ω1 ), V2 (ω1 , ω2 ) := V2 (ω2 ),

for each ω1 ∈ ΩK and ω2 ∈ Ωξ .

Note that in this case the random variables have the same probability distributional properties, and they are independent on (ΩKξ , FKξ , Pµξ ). This implies that the stochastic processes ∆S, S and ξ can be extended to the product space (ΩKξ , FKξ , Pµξ ) while preserving the original probability distributional properties, and that (∆S, S) and ξ are independent. 2.1 Approximation of SPDEs Fix any x ∈ D. Let Ax (v) := {ω1 × ω2 ∈ ΩKξ : ω1 (x) = v} for each v ∈ R, y and APξB := {ω1 × ω2 ∈ ΩKξ : ∆ω1 (x1 ) = y1 (ω2 ), . . . , ω1 (xN +M ) = yN +M (ω2 )}. Using the methods in [4] and Theorem A.1, we obtain y

Pµξ (Ax (v)|APξB ) = Pµξ (Sx = v|S P B = y ξ ) = pµx (v|y ξ ),


5

where pµx (·|·) is the conditional probability density function of the random variable Sx given the random vector S P B := (∆Sx1 , · · · , ∆SxN , SxN +1 , · · · , SxN +M )T . (Here y ξ is viewed as given values.) According to the natural extension rule, pµx is consistent with the formula (6). Then the approximation u ˆ(x) is solved by the maximization problem u ˆ(x) = argmax v∈R

sup

pµx (v|y ξ ).

µ∈HK (D)

If the covariance matrix   ∗ ∗ N,N N,M ∗ (∆ ∆ K(x , x )) , (∆ K(x , x )) 1 2 j k 1 j N +k j,k=1 j,k=1  KP B :=  ∈ R(N +M )×(N +M ) ∗ ∗ M,M (∆2 K(xN +j , xk ))M,N , ( K(x , x )) N +j N +k j,k=1 j,k=1 is nonsingular, then one solution of the above maximum problem has the form u ˆ(x) :=

N X

∗

ck ∆2 K(x, xk ) +

k=1

M X

∗

∗

cN +k K(x, xN +k ) = kP B (x)T KP B −1 y ξ , (3)

k=1 ∗

∗

∗

∗

where kP B (x) := (∆2 K(x, x1 ), · · · , ∆2 K(x, xN ), K(x, xN +1 ), · · · , K(x, xN +M ))T . This means that its random coefficients are obtained from the linear equation ∗

system KP B c = y ξ . The estimator u ˆ also satisfies the interpolation condition, i.e., ∆ˆ u(x1 ) = y1 , . . . , ∆ˆ u(xN ) = yN and u ˆ(xN +1 ) = yN +1 , . . . , u ˆ(xN +M ) = yN +M . It is obvious that u ˆ(·, ω2 ) ∈ HK (D) for each ω2 ∈ Ωξ . Since the random part of u ˆ(x) is only related to y ξ , we can formally rewrite u ˆ(x, ω2 ) as u ˆ(x, y ξ ) and u ˆ(x) can be transferred to a random variable defined on the finite-dimensional probability space (RN +M , B(RN +M ), µy ), where the probability measure µy is defined by µy (dv) := py (v)dv. Moreover, the probability distributional properties of u ˆ(x) do not change when (Ωξ , Fξ , Pξ ) is replaced by (RN +M , B(RN +M ), µy ). Remark 2.2. The random coefficients are obtained solving by system of linear equations that is slightly different from [4]. However the main ideas and techniques are the same as in [4]. For this estimator it is easier to derive error bounds and compare with Galerkin finite element method. A lot more details of the relationship between the two different estimators are provided in [11, Chapter 7]. 2.2 Convergence Analysis We assume that u(·, ω2 ) belongs to HK (D) almost surely for ω2 ∈ Ωξ . Therefore u can be seen as a map from Ωξ into HK (D). So we have u ∈ ΩKξ = ΩK × Ωξ . We fix any x ∈ D and any > 0. Let the subset

6


n Ex := ω1 × ω2 ∈ ΩKξ : |ω1 (x) − u ˆ(x, ω2 )| ≥ , o such that ∆ω1 (x1 ) = y1 (ω2 ), . . . , ω1 (xN +M ) = yN +M (ω2 ) . Because ∆Sx (ω1 , ω2 ) = ∆Sx (ω1 ) = ∆ω1 (x), Sx (ω1 , ω2 ) = Sx (ω1 ) = ω1 (x) and y ξ (ω1 , ω2 ) = y ξ (ω2 ) for each ω1 ∈ ΩK and ω2 ∈ Ωξ (see Theorem A.1) we can deduce that Pµξ (Ex ) = Pµξ |Sx − u ˆ(x)| ≥ such that S P B = y ξ Z Z = pµx (v|v)py (v)dvdv RN +M

|v−ˆ u(x,v)|≥

Z =

erfc RN +M

√ 2σ(x)

py (v)dv = erfc

√ 2σ(x)

,

∗

∗

where the variance of pµx is σ(x)2 = K(x, x) − kP B (x)T KP B −1 kP B (x) (see Equation (6) given in Appendix Appendix A.). The reader may note that the form of the expression for the variance σ(x)2 is analogous to that of the power function [5, 10], and we can therefore use the same techniques as in the proofs from [4, 5, 10, 11] to obtain a formula for the order of σ(x), i.e., m−2−d/2

σ(x) = O(hX

),

where hX = supx∈D minxj ∈XD ∪X∂D kx − xj k2 is the fill distance of X := XD ∪ X∂D . This implies that ! m−2−d/2 hX µ . sup Pξ (Ex ) = O µ∈HK (D) Because |u(x, ω2 ) − u ˆ(x, ω2 )| ≥ if and only if u ∈ Ex we conclude that sup Pµξ ku − u ˆkL∞ (D) ≥ ≤ sup Pµξ (Ex ) → 0, when hX → 0. µ∈HK (D)

µ∈HK (D),x∈D

Therefore we say that the estimator u ˆ converges to the exact solution u of the SPDE (1) in all probabilities Pµξ when hX goes to 0. Sometimes we know only that the solution u ∈ Hm (D). In this case, as long as the reproducing kernel Hilbert space is dense in the Sobolev space Hm (D) with respect to its Sobolev norm, we can still say that u ˆ converges to u in probability.

3 Galerkin Finite Element Method The right hand side of the SPDE (2)


f (x, ζ) := f (x) + ξn

ξxn

= f (x) +

n X

√ ζk qk φk (x),

7

x ∈ D,

k=1

and the random vector ζ := (ζ1 , · · · , ζn )T has the joint standard normal density function ρn (z) :=

n Y

z ∈ Rn ,

ρ(zk ),

k=1

2 1 where ρ(z) := √ e−z /2 . 2π

Therefore we can replace the probability space (Ωξ , Fξ , Pξ ) by a finitedimensional probability space (Rn , B(Rn ), µζ ) such that un and ξ n have the same probability distributional properties on both probability spaces, where the probability measure µζ is defined by µζ (dz) := ρn (z)dz. In the paper [1] the numerical approximation uh,p of the solution un of the SPDE (2) is sought in a finite-dimensional subspace Vh,p based on a tensor product, Vh,p := Hh (D) ⊗ Pp (Rn ), where the following hold: C1) Hh (D) ⊂ H01 (D) is a standard finite element space, which contains continuous piecewise polynomials defined on regular triangulations with a maximum mesh spacing parameter h. C2) Pp (Rn ) := ⊗nk=1 Ppk (R) ⊂ L2,ρn (Rn ) is the span of the tensor product of polynomials with degree at most p = (p1 , · · · , pn ), where Ppk (R) is a space of univariate polynomials of degree pk for each k = 1, . . . , n. Thus the approximation uh,p ∈ Vh,p and uh,p (x) is a random variable defined on the finite-dimensional probability space (Rn , B(Rn ), µζ ). Next we construct the Gaussian points z j := (z1,j1 , · · · , zn,jn )T ,

j ∈ Np := {j ∈ Nn : 1 ≤ jk ≤ pk + 1, k = 1, . . . , n} ,

where zk,1 , . . . , zk,pk +1 are the roots of the Hermite polynomials ηpk +1 of degree pk + 1 for each dimension k = 1, . . . , n. The ηpk +1 are also orthogonal polynomials on the space Ppk (R) with respect to a standard normal weight ρ. Here these Hermite polynomials are only used to set up the Gaussian points for approximating the Gaussian fields. Let a polynomial base of Pp (Rn ) be lj (z) :=

n Y

lk,jk (zk ),

j ∈ Np ,

k=1 k +1 where {lk,j }pj=1 is the Lagrange basis of Ppk (R) for each k = 1, . . . , n, i.e.,

lk,j ∈ Ppk (R),

lk,j (zk,i ) = δij ,

i, j = 1, . . . , pk + 1,

and δij is the Kronecker symbol. For each Gaussian point z j ∈ Np , we compute the finite element solution uh (·, z j ) ∈ Hh (D) of the equation

8


Z −

Z ∇uh (x, z j )∇γ(x)dx =

D

fξn (x, z j )γ(x)dx,

for any γ ∈ Hh (D).

D

The approximation uh,p is the tensor product of the finite element solutions and the Lagrange polynomials, i.e., X uh (x, z j )lj (ζ), x ∈ D. (4) uh,p (x, ζ) := j∈Np

This indicates that uh,p is interpolating at all Gaussian points z j ∈ Np . We assume that un belongs to L2,ρn (Rn ) ⊗ H01 (D). According to [1, Theorem 4.1] and [1, Lemma 4.7] we get the error bound kEρn (un − uh,p )kH1 (D) 0 Z 1/2 n X 1/2 3/2 2 n Eρn |∇u (x) − ∇w(x)| dx + C2 ≤C1 infn pk e−rk pk w∈L2,ρn (R )⊗Hh (D)

D

k=1

with positive constants r1 , . . . , rn and C1 , C2 independent of h and p.

4 Side-by-Side Comparison of Both Methods 4.1 Differences •

•

•

•

Probability spaces: For kernel-based collocation (KC) we transfer the probability space (Ωξ , Fξ , Pξ ) to the tensor product probability space (ΩKξ , FKξ , Pµξ ) such that the Gaussian noise ξ has the same probability distributional properties defined on both probability spaces, while for Galerkin finite elements (FE) we approximate the Gaussian noise ξ by the truncated Gaussian noise ξ n such that limn→∞ E kξ − ξ n kL2 (D) = 0 and ξ n has the same probability distributional properties on the probability space (Ωξ , Fξ , Pξ ) and the finite-dimensional probability space (Rn , B(Rn ), µζ ) Basis functions: The bases of the KC solution u ˆ are the kernel functions ∗ ∗ ∆2 K and K centered at the collocation points XD ⊂ D and X∂D ⊂ ∂D, while the bases of the FE solution uh,p are the tensor products of the triangular finite element bases defined on D and the Lagrange polynomials defined on (Rn , B(Rn ), µζ ). Simulation: For KC we can simulate the Gaussian noise ξ at the collocation points XD because we know its covariance kernel Φ, i.e., ξ = (ξx1 , · · · , ξxN )T ∼ N (0, Φ) and Φ = (Φ(xj , xk ))N,N j,k=1 . For FE we can simulate the random vector ζ = (ζ1 , · · · , ζ2 )T ∼ N (0, In ) in order to introduce random variables on (Rn , B(Rn ), µζ ). Interpolation: In KC ∆ˆ u and u ˆ are interpolating at the collocation points XD ∪ X∂D ⊂ D in the domain space, respectively, while in FE uh,p is interpolating at the Gaussian points Np ⊂ Rn in the probability space.


•

•

∗

9

∗

Function spaces: For KC, u ˆ ∈ span{∆2 K(·, xj ), K(·, xN +k )}N,M j,k=1 ⊗ N +M P1 (R ) ⊂ HK (D) ⊗ L2,py (RN +M ), while for FE we have uh,p ∈ Hh (D) ⊗ Pp (Rn ) ⊂ H01 (D) ⊗ L2,ρn (Rn ). Approximation properties: The KC result u ˆ approximates the solution u of the SPDE (1) and its convergence rate depends on the fill distance hX of the collocation points, while the FE result uh,p approximates the truncated solution un of the SPDE (2) with a convergence rate that depends on the maximum mesh spacing parameter h of the triangulation and the degree p of the Lagrange polynomials.

4.2 Relationship Between the Two Methods Roughly speaking, the random parts of u ˆ and uh,p are simulated by the normal random vectors ξ and ζ, respectively. For the following we assume that Φ is positive definite on D and the dimensions of ξ and ζ are the same, i.e., N = n. We firstly show the relationship between ξ and ζ. Since Φ is positive definite, we have the decomposition Φ = VDVT , where D and V are the eigenvalues and eigenvector matrices of Φ, respectively. Therefore ξ ∼ VD1/2 ζ ∼ N (0, Φ),

ζ ∼ D−1/2 VT ξ ∼ N (0, IN ).

We can also use ξ and ζ to approximate the Gaussian noise ξx for any fixed x ∈ D. Using simple kriging, we let ξˆx := c(x)T ξ = b(x)T Φ−1 ξ ∼ N (0, b(x)T Φ−1 b(x)), where b(x) := (Φ(x, x1 ), · · · , Φ(x, xN ))T . According to [9], 2 E ξx − ξˆx = Φ(x, x) − b(x)T Φ−1 b(x) = O(hkXD ), when Φ ∈ C2k (D × D) and hXD is the fill distance of XD . For the KarhunenLoéve expansion, ξxN

=

N X √

qj φj (x)ζj = φ(x)T Q1/2 ζ ∼ N (0, φ(x)T Qφ(x)),

j=1

where φ(x) := (φ1 (x), · · · , φN (x))T and Q = diag(q1 , · · · , qN ). We also have Z E D

∞ X ξx − ξxN 2 dx = qj . j=N +1

This shows that the kernel-based method and the finite element method, respectively, use a kernel basis and a spectral basis to approximate Gaussian

10


fields. It also shows that we should suitably choose collocation points such that lim hXD = 0 =⇒ lim ξˆ = lim ξ N = ξ. N →∞

N →∞

N →∞

Usually, the smoothness of Φ is related to its eigenvalues in the sense that the order k of continuous differentiability becomes large when the eigenvalues qj P∞ decrease fast, e.g., Φ(x, y) = j=1 (2πj)−2k sin(2πjx) sin(2πjy). Following these discussions, when the eigenvalues of the covariance kernel Φ decrease fast, then the Galerkin finite element method seems to be preferable to the kernel-based collocation method because we can truncate the Gaussian noise ξ at a low dimension. However, when the eigenvalues change slowly, then we may use the kernel-based collocation method because we are able to directly simulate ξ by its covariance structure. 4.3 Competitive Advantages •

•

•

•

•

•

•

The kernel-based collocation method is a meshfree approximation method. It does not require an underlying triangular mesh as the Galerkin finite element method does. For both methods, the (collocation) points can be placed at rather arbitrarily scattered locations which allows for the use of either deterministic or random designs, e.g., Halton or Sobol’ points. The kernel-based collocation method can be applied to a high-dimensional domain D with complex boundary ∂D. We can also generalize it to solve a system of elliptic SPDEs derived by vector Gaussian noises. The collocation method requires the SPDE solution to be smooth enough such that interpolation is well-behaved at each collocation point while the finite element method can solve non-smooth problems. ∗

The interpolation matrix K P B for the collocation method is usually a dense (and sometimes ill-conditioned) matrix. The finite element method, on the other hand, usually gets the solutions by a sparse linear system because its basis consists of local elements. To obtain the truncated Gaussian noise ξ n for the finite element method we need to compute the eigenvalues and eigenfunctions of the covariance kernel Φ. This, however, is usually difficult to do, so one must estimate them. For the collocation method we need not worry about this issue. If the truncated dimension n for the finite element solutions is large, then the degree p of the polynomials has to become correspondingly large in order to satisfy a given error tolerance (see [2]). Once the kernel functions are fixed, the error of the collocation solution only depends on the collocation points. In the finite element method, the dimension of its polynomialQ space defined n on the finite-dimensional probability space is equal to np = k=1 (pk + 1). So we need to compute np deterministic finite element solutions. In the collocation method, we need to simulate the N -dimensional nonstandard


•

•

•

11

normal vector. Therefore, when n N we may choose the finite element method, while vice versa the collocation method may be preferable. For various covariance kernels Φ for the Gaussian noise ξ, the choice of reproducing kernels can affect the kernel-based collocation solution. How to choose the “best” kernels is still an open question. Polynomials are used to construct the approximations for the finite element method, and we only need to determine the appropriate polynomial degree. The paper [1] also discusses any other tensor-product finite dimensional noise. In the papers [4, 11] we only consider Gaussian noises. However, one may generalize the kernel-based collocation idea to other problems with colored noise. The finite element method works for any elliptic SPDE whose differential operator contains stochastic coefficients defined on a finite dimensional probability space. For the collocation method this idea requires further study.

5 Numerical Examples In this section we present a few simple numerical experiments comparing the kernel-based collocation method to the Galerkin finite element method. Let the domain D := (0, 1)2 ⊂ R2 and the covariance kernel of the (finitedimensional) noise be Φ(x, y) :=4π 4 sin(πx1 ) sin(πx2 ) sin(πy1 ) sin(πy2 ) + 16π 4 sin(2πx1 ) sin(2πx2 ) sin(2πy1 ) sin(2πy2 ) so that we are able to demonstrate the effects of a “correct” and “incorrect” truncation dimension for the finite element method. We use the deterministic function f (x) := −2π 2 sin(πx1 ) sin(πx2 ) − 8π 2 sin(2πx1 ) sin(2πx2 ) and the Gaussian noise ξ with the covariance kernel Φ to set up the right hand side of the stochastic Poisson equation with Dirichlet boundary condition as in SPDE (1). Then its solution has the form u(x) := sin(πx1 ) sin(πx2 ) + sin(2πx1 ) sin(2πx2 ) + ζ1 sin(πx1 ) sin(πx2 ) ζ2 sin(2πx1 ) sin(2πx2 ), x = (x1 , x2 ) ∈ D, + 2 where ζ1 , ζ2 are independent standard normal random variables defined on (Ωξ , Fξ , Pξ ), i.e., ζ1 , ζ2 ∼ i.i.d. N (0, 1). For the collocation methods, we use the C4 -Matérn function with shape parameter θ > 0 gθ (r) := (3 + 3θr + θ2 r2 )e−θr ,

r > 0,

12


to construct the reproducing kernel (Sobolev-spline kernel) Kθ (x, y) := gθ (kx − yk2 ),

x, y ∈ D.

According to [6], we can deduce that its reproducing kernel Hilbert space HK (D) is equivalent to the L2 -based Sobolev space H3+1/2 (D) ⊂ C2 (D). Then ∗ R1R1 we can compute the integral-type K θ (x, y) = 0 0 Kθ (x, z)Kθ (y, z)dz1 dz2 . Next we choose Halton points in D and uniform grid points on ∂D as collocation points. Using the kernel-based collocation method, we can set up the approximation u ˆ via formula (3). Since we chose ξx = ζ1 2π 2 sin(πx1 ) sin(πx2 ) + ζ2 4π 2 sin(2πx1 ) sin(2πx2 ), we can let the dimension of the probability space be either n = 1 or n = 2 for the finite element method. The Gaussian points zj,1 , · · · , zj,pj +1 ∈ R are computed as the roots of the Hermite polynomial of degree pj + 1, j = 1, 2. For n = 1 we have ξ 1 = ζ1 2π 2 sin(πx1 ) sin(πx2 ), while n = 2 gives ξ 2 = ξ. We approximate the mean and variance of the arbitrary random variable U by its sample mean and sample variance based on s := 10000 simulated sample paths using the above algorithm, i.e.,  2 s s s 1 X 1X 1X U (ωk ), Var(U ) ≈ U (ωk ) − U (ωj ) . E(U ) ≈ s s s j=1 k=1

k=1

PDF, x1 = 0.52632, x2 = 0.52632

Collocation Points 1

0.5

0.8

0.4

0.3

0.6

0.3

0.4

0.2

0.2

0.1

0

0

0.2

0.4

0.6

0.8

0 −4

1

−2 Empirical

Approximate Mean

0

2

Relative Root−mean−square Error

0.25

4

Theoretical

Approximate Variance

2

1

1 0.5 0

0.2

Mean, θ = 0.9 Variance, θ = 0.9 Mean, θ = 1.9 Variance, θ = 1.9 Mean, θ = 2.9 Variance, θ = 2.9

0.15

0.1

0.05

−1 1

1

0.5

0.5 0 0 0.02 0.04 0.06 Relative Absolute Error

0.08

0 1 0.5

0.5 0 0 0.01 0.02 0.03 0.04 0.05 0.06 Relative Absolute Error

N = 65, M = 28 and θ = 0.9

1

0 0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

fill distance hX

Convergence of Mean and Variance

Figure 1. Kernel-based Collocation Method.

According to the numerical results (see Figures 1, 2 and 3), the approximate probability density functions are well-behaved for both numerical methods. The means and variances of the kernel-based collocation solutions are

Kernel-based Collocation vs. Galerkin Finite Element for SPDEs PDF, x1 = 0.52632, x2 = 0.52632

Triangle Mesh 0.5

0.8

0.4

0.6

0.3

0.4

0.2

0.2

0.1

0

0

0.2

0.4

0.6

0.8

1

0.3

Mean p1 = 1

0 −4

−2

0

Empirical Approximate Mean

2

4

Theoretical


2

Variance p1 = 1

0.25 Relative Root−mean−square Error

1

13

1

1 0.5

Mean p1 = 3 Variance p1 = 3 0.2

Mean p1 = 7 Variance p1 = 7

0.15

0.1

0

0.05 −1 1

0 1 1 0.5

1 0.5

0.5

0.5

0 0

0 0

0.02 0.04 0.06 0.08 Relative Absolute Error

0

0.05

0.1 0.15 0.2 Relative Absolute Error

0 0.04

0.25

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

maximum mesh spacing parameter h

h = 0.15 and p1 = 7


Figure 2. Galerkin Finite Element Methods, n = 1. PDF, x1 = 0.52632, x2 = 0.52632

Triangle Mesh 1

0.5

0.8

0.4

0.6

0.3

0.4

0.2

0.2

0.1

0.3

Mean p1 = 1 p2 = 2

0

0.2

0.4

0.6

0.8

1

0 −4

−2

0

Empirical Approximate Mean

2

Relative Root−mean−square Error

0

4

Theoretical


2

Variance p1 = 1 p2 = 2

0.25

1

1 0.5

Mean p1 = 3 p2 = 7 Variance p1 = 3 p2 = 7 0.2

Mean p1 = 7 p2 = 7 Variance p1 = 7 p2 = 7

0.15

0.1

0

0.05 −1 1

0 1 1 0.5

1 0.5

0.5 0 0

0

0.5 0 0


0


h = 0.15, p1 = 7 and p2 = 3

0.1

0 0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

maximum mesh spacing parameter h


Figure 3. Galerkin Finite Element Methods, n = 2.

smooth estimators, while the means and variances of the finite element solutions are piecewise smooth estimators. If we suitably truncate the Gaussian noise, then the finite element method has the same convergence rate as the kernel-based collocation method. If not, the kernel-based collocation method seems to do better than the finite element method for the variance.

14


Appendix A. Reproducing Kernels and Gaussian Fields Let K : D × D → R be a reproducing kernel and HK (D) be its reproducingkernel Hilbert space (see [10, Definition 10.1]). If K ∈ C(D × D) then its ∗

integral-type kernel K is defined by Z ∗ K(x, y) := K(x, z)K(y, z)dz,

x, y ∈ D.

(5)

D

Remark A.1. As in the proof of [4, Lemma 2.2], in order to “match the spaces”, any other kernel that “dominates” K (in the sense of [8]) could play the role ∗

of the integral-type kernel K. Let Hm (D) be the classical L2 -based Sobolev space of order m ∈ N. The differential operator P : Hm (D) → L2 (D) and the boundary operator B : Hm (D) → L2 (∂D) are linear combinations of the derivatives Dα , α ∈ Nd0 , P with nonconstant coefficients defined on D and ∂D respectively, P i.e., P = |α|≤m cα Dα and B = |α|≤m−1 bα Dα |∂D where cα ∈ C(D) and bα ∈ C(∂D). Their orders are given by O(P ) := m and O(B) := m − 1. Definition A.1 ([3, Definition 3.28]). A stochastic process S : D × Ω → R is said to be Gaussian with mean µ : D → R and covariance kernel Φ : D × D → R on a probability space (Ω, F, P) if, for any pairwise distinct points XD := {x1 , · · · , xN } ⊂ D, the random vector S := (Sx1 , · · · , SxN )T is a multi-normal random variable on (Ω, F, P) with mean µ and covariance matrix Φ, i.e., S ∼ N (µ, Φ), where µ := (µ(x1 ), · · · , µ(xN ))T and Φ := (Φ(xj , xk ))N,N j,k=1 . We view the reproducing-kernel Hilbert space HK (D) as a sample space and its Borel σ-field B(HK (D)) as a σ-algebra to set up the probability spaces. Theorem A.1 ([4, Theorem 3.1]). Suppose that the reproducing kernel Hilbert space HK (D) is embedded into the Sobolev space Hm (D) with m > d/2. Further assume that the differential operator P and the boundary operator B have the orders O(P ) < m − d/2 and O(B) < m − d/2. Given a function µ ∈ HK (D) there exists a probability measure Pµ defined on (ΩK , FK ) = (HK (D), B(HK (D))) such that the stochastic processes P S, BS given by P Sx (ω) = P S(x, ω) := (P ω)(x),

x ∈ D ⊂ Rd ,

BSx (ω) = BS(x, ω) := (Bω)(x),

x ∈ ∂D,

ω ∈ ΩK = HK (D),

ω ∈ ΩK = HK (D), ∗

are Gaussian fields with means P µ, Bµ and covariance kernels P1 P2 K, ∗

B1 B2 K defined on (ΩK , FK , Pµ ), respectively, where

Kernel-based Collocation vs. Galerkin Finite Element for SPDEs ∗

∗

P1 P2 K(x, y) := Pz1 Pz2 K(z 1 , z 2 )|z1 =x,z2 =y , ∗

∗

B1 B2 K(x, y) := Bz1 Bz2 K(z 1 , z 2 )|z1 =x,z2 =y ,

15

x, y ∈ D, x, y ∈ ∂D.

When P := I then we rewrite P S = S which indicates that Sx (ω) = ω(x). Remark A.2. The probability measure Pµ defined in Theorem A.1 can be verified to be Gaussian (see [3, 4, 11]). The papers [6, 7, 11] show what kinds of reproducing kernels satisfy the conditions that their reproducing-kernel Hilbert spaces are embedded into the Sobolev spaces. One typical example is the Sobolev spline kernel computed by the Matérn function. M Given XD := {xj }N j=1 ⊂ D and X∂D := {xN +j }j=1 ⊂ ∂D, [4, Corollary 3.2] shows that the random vector S P B := (P Sx1 , · · · , BSxN +M )T defined on (ΩK , FK , Pµ ) has a multi-normal distribution with mean mµP B and ∗

covariance matrix KP B , i.e., ∗

S P B ∼ N (mµP B , KP B ), where mµP B := (P µ(x1 ), · · · , P µ(xN ), Bµ(xN +1 ), · · · , Bµ(xN +M ))T and   ∗ ∗ N,N N,M ∗ (P P K(x , x )) , (P B K(x , x )) 1 2 j k 1 2 j N +k j,k=1 j,k=1  KP B :=  . ∗ ∗ M,M (B1 P2 K(xN +j , xk ))M,N , (B B K(x , x )) 1 2 N +j N +k j,k=1 j,k=1 Fix any x ∈ D. We can compute the joint probability density function pµX and pµJ of S P B and (Sx , S P B ) defined on (ΩK , FK , Pµ ) respectively. By Bayes’ rule, we can obtain the conditional probability density function pµx of Sx given S P B defined on (ΩK , FK , Pµ ), i.e., for each v ∈ RN +M , pµ (v, v) 1 (v − mµx (v))2 √ exp − = , v ∈ R, (6) pµx (v|v) := Jµ pX (v) 2σ(x)2 σ(x) 2π ∗

∗

where mµx (v) := µ(x) + kP B (x)T KP B † (v − mµP B ), σ(x)2 := K(x, x) − ∗

∗

∗

kP B (x)T KP B † kP B (x) and kP B (x) := (P2 K(x, x1 ), · · · , B2 K(x, xN +M ))T . In particular, given the real observation y := (y1 , · · · , yN +M )T , Sx conditioned on S P B = y has the probability density pµx (·|y).

References 1. I. Babuˇska, F. Nobile and R. Tempone, A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data, SIAM Rev., vol. 52, 2010, pp. 317–355. 2. I. Babuˇska, R. Tempone and G. E. Zouraris, Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations, SIAM J. Numer. Anal., vol. 42, 2004, pp. 800–825.

16


3. A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics, Kluwer Academic Publishers, 2004. 4. I. Cialenco, G. E. Fasshauer and Q. Ye, Approximation of Stochastic Partial Differential Equations by a Kernel-based Collocation Method, Int. J. Comput. Math., Special Issue: Recent Advances on the Numerical Solutions of Stochastic Partial Differential Equations, 2012, to appear. 5. G. E. Fasshauer, Meshfree Approximation Methods with Matlab, World Scientific Publishing Co. Pte. Ltd., 2007. 6. G. E. Fasshauer and Q. Ye, Reproducing Kernels of Generalized Sobolev Spaces via a Green Function Approach with Distributional Operators, Numer. Math., vol. 119, 2011, pp. 585–611. 7. G. E. Fasshauer and Q. Ye, Reproducing Kernels of Sobolev Spaces via a Green Kernel Approach with Differential Operators and Boundary Operators, Adv. Comput. Math., DOI: 10.1007/s10444-011-9264-6. 8. M. N. Lukić and J. H. Beder, Stochastic Processes with Sample Paths in Reproducing Kernel Hilbert Spaces, Trans. Amer. Math. Soc., vol. 353, 2001, pp. 3945– 3969. 9. M. Scheuerer, R. Schaback and M. Schlather, Interpolation of Spatial Data – A Stochastic or A Deterministic Problem?, Data Page of R. Schaback’s Research Group, 2010. 10. H. Wendland, Scattered Data Approximation, Cambridge University Press, 2005. 11. Q. Ye, Analyzing Reproducing Kernel Approximation Methods via A Green Function Approach, Ph.D. thesis, Illinois Institute of Technology, 2012.