A result of Dick 2014 ... Books and survey papers: Dick, Kuo, Sloan 2013, Dick and ... Hinrichs, N., Mario Ullrich 2014: âThe Clenshaw Curtis Smolyak algorithm ...
Some Complexity Results for Numerical Integration Erich Novak Friedrich-Schiller University Jena
MCQMC 2014, Leuven, April 2014
Erich Novak
Complexity Results for Numerical Integration
1 / 36
The problem For a d-variate function f : [0, 1]d → R, approximate the
integral
Sd (f ) =
Z
f (x) dx. [0,1]d
Problem instances: functions f from a function class Fd of d-variate functions; often we have an initial error max |Sd (f )| = 1.
f ∈Fd
Algorithms: deterministic or randomized algorithms using n function values. Erich Novak
Complexity Results for Numerical Integration
2 / 36
The problem
Information complexity: n(ε, Fd ) is the minimal number of function values needed by an optimal algorithm to approximate Sd up to error ε < 1 for all f ∈ Fd . Optimal error bounds: Sometimes more convenient: e(n, Fd ) = inf sup |Sd (f ) − An (f )| An f ∈Fd
= minimal error achievable with n function values. What is known about the numbers n(ε, Fd ) or e(n, Fd )?
Erich Novak
Complexity Results for Numerical Integration
3 / 36
Plan for the talk
Some classical results (≤ 1971) Randomized algorithms Tensor product problems, tractability, weighted spaces Curse of dimensionality for C k functions
Erich Novak
Complexity Results for Numerical Integration
4 / 36
Classical results
Optimality of linear algorithms Optimal order for C k functions Lipschitz functions and the curse of dimensionality
Erich Novak
Complexity Results for Numerical Integration
5 / 36
Optimality of linear algorithms Smolyak 1965 and Bakhvalov 1971 Theorem Assume that Fd is convex and symmetric. Then non-linear or adaptive algorithms cannot be better than linear algorithms An (f ) =
n X
ai f (xi )
i=1
and e(n, Fd ) = inf
x1 ,...,xn
sup
Sd (f ).
f ∈Fd , f (xi )=0
See Hickernell, Plaskota, Wasilkowski and others for different Fd . Erich Novak
Complexity Results for Numerical Integration
6 / 36
Smooth functions Fd = Cdk = {f ∈ C k ([0, 1]d )
kD β f k∞ ≤ 1 for all kβk1 ≤ k}
Theorem (Bakhvalov 1959)
e(n, Cdk ) ≍ n−k/d or: for all ε ∈ (0, 1) and d ∈ N ad,k ε−d/k ≤ n(ε, Cdk ) ≤ bd,k ε−d/k . Lower bound trivial for fixed ε > 0 and large d. Arbitrary Lipschitz domains: N. and Triebel 2006. Fractals: Dereich, M¨ uller-Gronbach 2014. Erich Novak
Complexity Results for Numerical Integration
7 / 36
Lipschitz functions Fd = {f ∈ C ([0, 1]d ) |f (x) − f (y )| ≤ kx − y k∞ }
Theorem (Maung Zho Newn and Sharygin 1971) e(n, Fd ) =
d · n−1/d 2d + 2
for n = md with m ∈ N. Observe that e(2d , Fd ) =
1 e(1, Fd ). 2
Curse of dimension. Similar results: Babenko, Sukharev, Chernaya Erich Novak
Complexity Results for Numerical Integration
8 / 36
Randomized algorithms
Optimal algorithm for Lp , p ≥ 2 Optimal order for C k functions Importance sampling Markov chain Monte Carlo
Erich Novak
Complexity Results for Numerical Integration
9 / 36
Optimal algorithm for Lp , p ≥ 2 Error of An now e(An ) = sup (E(S(f ) − An (f ))2 )1/2 , f ∈Fd
numbers nran (ε, Fd ) and e ran (n, Fd ) defined as before. Let Fd be the unit ball of Lp ([0, 1]d ) with 2 ≤ p ≤ ∞. Theorem (Math´e 1995) e ran (n, Fd ) = and An (f ) =
1√ n+ n
Pn
i=1 f (Xi )
1 √ 1+ n
with i.i.d. Xi is optimal.
Erich Novak
Complexity Results for Numerical Integration
10 / 36
Smooth functions Fd = Cdk = {f ∈ C k ([0, 1]d )
kD β f k∞ ≤ 1 for all kβk1 ≤ k}
Theorem (Bakhvalov 1959)
e ran (n, Cdk ) ≍ n−k/d−1/2 Algorithm: 1) For n = 2m, use m values for a good L2 -approximation fm . 2) Compute integral of f − fm by simple Monte Carlo.
Advantages: Small error, even deterministic. Good error control, unbiased estimator.
Erich Novak
Complexity Results for Numerical Integration
11 / 36
Importance sampling Integration for a RK-Hilbert space Fd ⊂ L1 with positive kernel, R S(f ) = Rd f (x)̺(x) dx. Theorem (Hinrichs 2010)
e ran (n, Fd ) ≤
π 1/2 2
n−1/2 kSk
and this error bound can be achieved with importance sampling. Proof by deep results (Grothendieck) of Banach space theory. Such problems are strongly polynomially tractable. N. and Wo´zniakowski 2011: under some additional assumptions, the algorithm of Hinrichs is optimal. Erich Novak
Complexity Results for Numerical Integration
12 / 36
Markov chain Monte Carlo Want compute E(f ) =
R
G
f (x) dµ(x) but cannot sample from µ.
Use a reversible Markov chain (Xn )n with spectral gap α > 0 and initial distribution ν. Let F = {f ∈ L4 (µ) | kf k4 ≤ 1}. Theorem (Rudolf 2012) e ran (An,n0 , F )2 ≤
2Cν (1 − α)n0 2 + , nα n 2 α2
dν − 1k2 and where Cν = 64k dµ n
An,n0 (f ) =
1X f (Xn0 +i ). n i=1
Erich Novak
Complexity Results for Numerical Integration
13 / 36
Tensor product problems, tractability, weighted spaces
Optimal order for Wpk,mix ([0, 1]d ) Smolyak algorithm Tractability Tractability for unweighted problems QMC for RKHS and weighted spaces Decomposable kernels and lower bounds
Erich Novak
Complexity Results for Numerical Integration
14 / 36
Optimal order for Wpk,mix ([0, 1]d ) 1 < p < ∞ and k, d ∈ N fixed. kD α f kp ≤ 1 for kαk∞ ≤ k. Theorem (upper bound: Frolov 1976, Skriganov 1994, lower bound: Roth 1954, Bykovskii 1985, Temlyakov 1990) e(n, Wpk,mix ([0, 1]d )) ≍ n−k (log n)(d−1)/2 . First step: cubature formulas of the form An (f ) =
Am |detA| X f ad a d m∈Z
for functions with compact support; A does not depend on k. Second step: transformation for the general case. Erich Novak
Complexity Results for Numerical Integration
15 / 36
Smolyak algorithm / sparse grids Theorem Upper bounds for Wpk,mix ([0, 1]d ) and the Smolyak algorithm are almost optimal, but not quite. Smolyak 1963 Temlyakov 1985 . . . Wasilkowski, Wo´zniakowski 1995 N. and Ritter 1996 . . . Bungartz, Griebel 2004 N. and Wo´zniakowski [2010, Chapter 15] Sickel, Tino Ullrich 2011 Hinrichs, N., Mario Ullrich 2014 Dinh D˜ ung, Tino Ullrich 2014 Erich Novak
n−k (log n)(d−1)(k+1/2) for p = 2. Complexity Results for Numerical Integration
16 / 36
Tractability The problem is strongly polynomially tractable iff n(ε, d) ≤ C ε−p
for all ε ∈ (0, 1) , d ∈ N.
The problem is polynomially tractable iff n(ε, d) ≤ C d q ε−p
for all ε ∈ (0, 1) , d ∈ N.
The problem is weakly tractable iff lim
ε−1 +d→∞
ln n(ε, d) = 0. ε−1 + d
Introduced by Wo´zniakowski, 2 papers in 1994. Gnewuch, N., Kritzer, Pillichshammer, Papageorgiou, Petras, Siedlecki . . . Erich Novak
Complexity Results for Numerical Integration
17 / 36
Tractability by smoothness assumptions?
Usually, we cannot obtain tractability even by strong smoothness assumptions. Sometimes: yes. Tractability of star discrepancy A result of Dick 2014
Erich Novak
Complexity Results for Numerical Integration
18 / 36
Star-discrepancy For x1 , . . . , xn ∈ [0, 1]d defined by n X 1 ∗ 1[0,t) (xi ) D∞ (x1 , . . . , xn ) = sup t1 · · · td − n t∈[0,1]d i=1 Sobolev space (or functions with bounded variation) F1 = {f : [0, 1] → R | f (1) = 0, f ′ ∈ L1 },
kf k = kf ′ kL1 and Fd = F1 ⊗ · · · ⊗ F1 . Hlawka-Zaremba-equality yields
∗ D∞ (x1 , . . . , xn ) = sup |Sd (f ) − Qn (f )|, kf k≤1
where Qn (f ) =
1 n
Pn
i=1 f (xi ). Erich Novak
Complexity Results for Numerical Integration
19 / 36
The star-discrepancy is tractable Theorem Heinrich, N., Wasilkowski, Wo´zniakowski 2001: n(ε, Fd ) ≤ C d ε−2 . Hinrichs 2004: n(ε, Fd ) ≥ c d ε−1 for ε ≤ ε0 . Aistleitner 2011: C = 100. Aistleitner, Hofer 2012: More upper bounds. Doerr 2014: ∗ E(D∞ (x1 , . . . , xn ))
Erich Novak
≍
r
d n
for
n ≥ d.
Complexity Results for Numerical Integration
20 / 36
New tractability result of Dick 2014 Let 0 < α ≤ 1 and 1 ≤ p ≤ ∞ and o n P (x)| ≤ 1 . Fα,p,d = f : [0, 1]d → R | k∈Zd |f˜(k)| + sup |f (x+h)−f khkα p
Theorem (Dick 2014)
e(n, Fα,p,d ) ≤ max
d − 1 d α/p √ , α n n
!
for any prime number n. Hence the complexity is at most quadratic in d. n 1 o n 2 o n d o k k k Algorithm: use points xk = , , . . . , . n n n Erich Novak
Complexity Results for Numerical Integration
21 / 36
Weighted Sobolev spaces and QMC Inner product for d = 1 is Z Z 1 f dx hf , g i1,γ =
1
1 g dx + γ 0 0 tensor products for d > 1 and γi .
Z
1
f ′ (x)g ′ (x) dx,
0
Theorem (Sloan, Wo´zniakowski 1998) There exist points x1 , . . . , xn for a QMC rule such that the problem P is strongly polynomially tractable iff γi < ∞.
Compute the mean of the quadratic error of QMC algorithms over all (x1 , . . . , xn ) ∈ [0, 1]nd and obtain ! Z Z 1 K (x, y ) dx dy . K (x, x) dx − n [0,1]2d [0,1]d Erich Novak
Complexity Results for Numerical Integration
22 / 36
Weighted Sobolev spaces and constructions Sloan and Wo´zniakowski 1998 was continued in many directions. General weights, different Hilbert spaces. Lower bounds for arbitrary algorithms: decomposable kernels (N. and Wo´zniakowski 2001, NW2010 Ch. 11). Construction of good QMC methods: Baldeaux, Chen, Cools, Dick, Doerr, Gnewuch, Hickernell, Joe, Kritzer, Kuo, Leobacher, Niederreiter, Nuyens, Owen, Pillichshammer, Reztsov, Skriganov, Sloan, Wang, Wasilkowski, Waterhouse, Wo´zniakowski, and many others. Books and survey papers: Dick, Kuo, Sloan 2013, Dick and Pillichshammer 2010, 2014, N. and Wo´zniakowski 2008, 2010, 2012. Erich Novak
Complexity Results for Numerical Integration
23 / 36
Curse of dimensionality for C k functions
Joint work with Aicke Hinrichs, Mario Ullrich and Henryk Wo´zniakowski 2014
Erich Novak
Complexity Results for Numerical Integration
24 / 36
Result Fd = Cdk = {f ∈ C k ([0, 1]d )
kD β f k∞ ≤ 1 for all kβk1 ≤ k}
Theorem
Cdk suffers from the curse for all k ∈ N.
More quantitative: there is ck > 0 such that n(ε, Cdk ) ≥ ck (1 − ε)d d/(2k+3) for all ε ∈ (0, 1) and d ∈ N. The curse holds even for much smaller classes Fd . Erich Novak
Complexity Results for Numerical Integration
25 / 36
Lipschitz functions
Lip(f ) = supx,y ∈[0,1]d
|f (x)−f (y )| kx−y k2
Theorem Let Fd = {f : [0, 1]d → R | Lip(f ) ≤ Ld }.
Then the curse of dimensionality holds for Fd iff lim inf Ld d→∞
Erich Novak
√
d > 0.
Complexity Results for Numerical Integration
26 / 36
Directional derivatives up to order k Theorem Let Fd,k be the class of all functions f ∈ C k−1 [0, 1]d satisfying kf k∞ ≤ 1, 1 Lip(f ) ≤ √ , d 1 Lip(D ϑ1 . . . D ϑi f ) ≤ d
for all directions ϑ1 , . . . , ϑi ∈ S d−1 and i < k. Then the curse of dimensionality holds for Fd,k .
Erich Novak
Complexity Results for Numerical Integration
27 / 36
For which ε do we have the curse?
Theorem Let Fd,k be the class of all functions f ∈ C k−1 [0, 1]d satisfying 1 1 kf k∞ ≤ 1, Lip(f ) ≤ √ , Lip(D ϑ1 . . . D ϑi f ) ≤ d d
for all directions ϑ1 , . . . , ϑi ∈ S d−1 and i < k. Then 1 n(ε, Fd,k ) ≥ 2(d + 1)
d 8 for d ∈ N, ε ≤ εk . 7
Problem: εk is very small for large k!
Erich Novak
Complexity Results for Numerical Integration
28 / 36
For which ε do we have the curse?
Theorem Let Fd,k be the class of all functions f ∈ C k−1 [0, 1]d satisfying 600 Ck kf k∞ ≤ 1, Lip(f ) ≤ √ , Lip(D ϑ1 . . . D ϑi f ) ≤ d d
for all directions ϑ1 , . . . , ϑi ∈ S d−1 and i < k. Then 1−ε n(ε, Fd,k ) ≥ d +1
d 8 for d ∈ N, ε ≤ 1. 7
Erich Novak
Complexity Results for Numerical Integration
29 / 36
Infinite smoothness Open Problem 2 from N. and Wo´zniakowski 2008 still open: Does the integration problem suffer from the curse of dimensionality for the classes Fd = {f : [0, 1]d → R | kD α f k∞ ≤ 1 = Lα for all α ∈ Nd0 }? Answer “yes” if numbers Lα = 1 are replaced by certain larger numbers. Answer “no” for somewhat smaller spaces: Vyb´ıral 2014: “Approximation and Integration are weakly or quasi-polynomially tractable” Hinrichs, N., Mario Ullrich 2014: “The Clenshaw Curtis Smolyak algorithm is weakly tractable” Erich Novak
Complexity Results for Numerical Integration
30 / 36
Fooling functions fix an algorithm which uses sample points x1 , . . . , xn construct a function f ∈ Fd which vanishes at x1 , . . . , xn if −f ∈ Fd , then
Z
f (x) dx [0,1]d
is a lower bound for the error find fooling functions for any choice of sample points x1 , . . . , xn here: fooling functions vanish on the convex hull K of x1 , . . . , xn Erich Novak
Complexity Results for Numerical Integration
31 / 36
Results of Elekes and Dyer et al. x1 , x2 , . . . , xn ∈ [0, 1]d
K : convex hull of {x1 , x2 , . . . , xn }
Dyer, F¨ uredi, McDiarmid 1990 based on Elekes 1986: Volume(K ) ≤ n(d + 1) b
6 d 7
i am exponentially small !!!
b b
Erich Novak
Complexity Results for Numerical Integration
32 / 36
What we will need is ... x1 , x2 , . . . , xn ∈ [0, 1]d
K : convex hull of {x1 , x2 , . . . , xn } √ For δ = d/100: Volume(Kδ ) ≤ n(d + 1) b
7 d 8
i am still exponentially small !!!
b b
Erich Novak
Complexity Results for Numerical Integration
33 / 36
Initial fooling functions f1 (x) = dist(x, K ) is Lipschitz Z Sd (f1 ) ≥ f1 (x) dx [0,1]d \Kδ
≥ ≥
√
d (1 − λd (Kδ )) 100 √ d ! 7 d 1 − n (d + 1) 100 8
f2 (x) = dist(x, K )2 is (almost) twice differentiable fk (x) = dist(x, K )k is unfortunately not (almost) k-times differentiable Erich Novak
Complexity Results for Numerical Integration
34 / 36
Higher smoothness
instead of dist(x, K )2 use dist(x, Kδ )2 as initial function convolute with indicator functions of balls with small size control the support control the derivatives inductively control the integral inductively in the case of infinite smoothness: control the limit and you are done!
Erich Novak
Complexity Results for Numerical Integration
35 / 36
Summary
Numerical integration is intractable in the worst case setting for classical function spaces, like C k ([0, 1]d ). Remedies: Weighted spaces, problems with a structure Randomized algorithms
Erich Novak
Complexity Results for Numerical Integration
36 / 36
