SAMSI Workshop on Sequential Monte Carlo Methods,. September 2008. ... Mean field particle interpretation of Feynman-Kac measures. P. Del Moral (INRIA ...
Some theoretical aspects of Particle/SMC Methods P. Del Moral
INRIA, Centre Bordeaux-Sud Ouest SAMSI Workshop on Sequential Monte Carlo Methods, September 2008. ,→ ∼ Joint works : D. Crisan, D. Dawson, A. Doucet, J. Jacod, A. Jasra, A. Guionnet, M. Ledoux, L. Miclo, F. Patras, T. Lyons, S. Rubenthaler,... http-references : ,→ Feynman-Kac formulae. Genealogical and interacting particle systems, Springer (2004), + References ,→ DM, Doucet, Jasra. SMC Samplers. JRSS B (2006). P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
1 / 47
Outline
1
Some foundations & Motivating Applications
2
A simple mathematical model
3
Some Feynman-Kac sampling recipes
4
A series of applications
5
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
2 / 47
Summary
1
Some foundations & Motivating Applications Some ”different” particle interpretation models Sequential Monte Carlo & Feynman-Kac models Motivating application areas
2
A simple mathematical model
3
Some Feynman-Kac sampling recipes
4
A series of applications
5
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
3 / 47
Particle Interpretation models Mathematical physics and molecular chemistry (≥ 19500 s) : Particle/microscopic interpretation models, particle absorption, macro-molecular chains, quantum and diffusion Monte Carlo. Environmental studies and biology (≥ 19500 s): Population, gene evolutions, species genealogies, branching/birth and death models. Evolutionary mathematics and engineering sciences (≥ 19700 s): Adaptive stochastic search method, evolutionary learning models, interacting stochastic grids approximations, genetic algorithms. Applied Probability and Bayesian Statistics (≥ 19900 s): Approximating simulation technique (recursive acceptance-rejection model), Sequential Monte Carlo, http-ref : interacting Monte Carlo Markov chains (Andrieu, Bercu, DM, Doucet, Jasra). Pure mathematics (≥ 19600 s for fluid models, ≥ 19900 s for discrete time and interacting jump models): Stochastic linearization tech., mean field particle interpretations of nonlinear PDE and measure valued equations.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
4 / 47
Central idea of particle/SMC in stochastic engineering : Physical and Biological intuitions ∈ Engineering problems [learning, adaptation, optimization,...] Sequential Monte Carlo Particle Filters Genetic Algorithms Evolutionary Population Diffusion Monte Carlo Quantum Monte Carlo Sampling Algorithms
Sampling Prediction Mutation Exploration Free evolutions Walkers motions Transition proposals
Resampling Updating Selection Branching Absorption Reconfiguration Acceptance-rejection
More botanical names : spawning, cloning, pruning, enrichment, go with the winner, and many others. Pure mathematical point of view : = Mean field particle interpretation of Feynman-Kac measures
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
5 / 47
Some application areas of Feynman-Kac formulae Physics : Feynman-Kac-Schroedinger semigroups ∈ nonlinear integro-differential equations (∼ generalized Boltzmann models). Spectral analysis of Schr¨ odinger operators and large matrices with nonnegative entries. Particle evolutions in disordered/absorbing media. Multiplicative Dirichlet problems with boundary conditions. Microscopic and macroscopic interacting particle interpretations. Chemistry and Biology: Self-avoiding walks, macromolecular simulation, directed polymers. Spatial branching and evolutionary population models. Coalescent and Genealogical tree based evolutions.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
6 / 47
Some application areas of Feynman-Kac formulae Rare events analysis: Multisplitting and branching particle models (Restart type methods). Importance sampling and twisted probability measures. Genealogical tree based simulations (default tree sampling models). Advanced Signal processing: Optimal filtering, prediction, smoothing. Open loop optimal control, optimal regulation. Interacting Kalman-Bucy filters. Stochastic and adaptative grid approximation-models Statistics/Probability: Restricted Markov chains (w.r.t terminal values, visiting regions, constraints simulation problems,...) Analysis of Boltzmann-Gibbs type distributions (simulation, partition functions, localization models...). Random search evolutionary algorithms, interacting Metropolis/simulated annealing algo, combinatorial counting. P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
7 / 47
Summary
1
Some foundations & Motivating Applications
2
A simple mathematical model Standard notation A genetic type spatial branching process Genealogical tree approximation measures Limiting Feynman-Kac measures
3
Some Feynman-Kac sampling recipes
4
A series of applications
5
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
8 / 47
Standard notation E measurable space, P(E ) proba. on E , B(E ) bounded meas. functions. Z (µ, f ) ∈ P(E ) × B(E ) −→ µ(f ) = µ(dx) f (x) M(x, dy ) integral operator on E Z M(f )(x) = M(x, dy )f (y ) Z [µM](dy ) = µ(dx)M(x, dy )
(=⇒ [µM](f ) = µ[M(f )] )
Bayes-Boltzmann-Gibbs transformation : G : E → [0, ∞[ with µ(G ) > 0 ΨG (µ)(dx) =
P. Del Moral (INRIA Bordeaux)
1 G (x) µ(dx) µ(G )
INRIA Centre Bordeaux-Sud Ouest, France
9 / 47
Note If
µ = Law(X ) and M(x, dy ) := P(Y ∈ dy | X = x)
Then Expectation operators Z µ(f ) = P(X ∈ dx) f (x) = E(f (X )) Z M(f )(x) = P(Y ∈ dy | |X = x) f (y ) = E(f (Y ) | X = x) Z [µM](dy ) = P(Y ∈ dy | X = x) P(X ∈ dx) = P(Y ∈ dy ) Bayes rule (Y = y fixed observation) : µ(dx) := p(x) dx
and G (x) = p(y | x) ⇓
ΨG (µ)(dx) = P. Del Moral (INRIA Bordeaux)
1 G (x) µ(dx) = p(x | y ) dx µ(G )
INRIA Centre Bordeaux-Sud Ouest, France
10 / 47
Only 3 Ingredients A state space : En with n = time/level index [transitions, paths, excursions,...]. 0 0 0 Xn := (Xn−1 , Xn0 ), X[0,n] , X[t0 n−1 ,tn ] , X[T , ... n−1 ,Tn ]
A Markov Proposal/Exploration/Mutation transition : Mn (xn−1 , dxn ) := P (Xn ∈ dxn | Xn−1 = xn−1 ) A potential/likelihood/fitness/weight function on En : Gn : xn ∈ En −→ Gn (xn ) ∈ [0, ∞[ Running Examples : [Confinement] Xn =Simple random walk (SRW) on En = Z and Gn = 1A . [Filtering] Mn =signal transitions, Gn =Likelihood weight function.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
11 / 47
SMC/Genetic type branching particle model : ξn1 .. . i Updating/Branching ξn −−−−−−−−−−−−−−−−→ .. . N ξn
ξbn1 .. . ξbni .. . ξbnN
Mn+1 1 ξn+1 −−−−−−−−−−−−−−−−− −−→ .. .. . . Prediction/Exploration i ξn+1 −−→ −−−−−−−−−−−−−−−−− . .. .. . N ξn+1 −−−−−−−−−−−−−−−−− −−→
Selection/Branching : (∀n ≥ 0 s.t. n (x 1 , . . . , x N ) × Gn (x i ) ∈ [0, 1]) Acceptance probability: ξbni = ξni Otherwise : ξbni = ξnj
with probability
n (ξn1 , . . . , ξnN ) Gn (ξni )
Gn (ξnj ) with probability PN k k=1 Gn (ξn )
Running examples: [Confinement & Filtering] = [(Gn = 1A ) & (Gn =Likelihood)]. P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
12 / 47
Some remarks : n = 0 =⇒ Simple Mutation-Selection Genetic model. Gn = exp {−Vt ∆t} & n = 1 =⇒ Vt -expo-clocks ⊕ uniform selection Gn ∈ [0, 1] & n = 1 ⇒ Interacting Acceptance-Rejection Sampling. Better fitted individuals acceptance : For
n (x 1 , . . . , x N )Gn (x i ) = Gn (x i )/ sup Gn (x j ) 1≤j≤N
Related branching rules: [DM-Crisan-Lyons MPRF 99, DM 04] (Given ξn = (ξni )i ) Pni := Proportion of offsprings of the individual ξni PN k Unbiasedness property: E Pni = Gn (ξnj )/ k=1 Gn (ξ n ) hP i 2 N i i Local mean error : E f (ξni ) ≤ i=1 Pn − E Pn P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
Cte N
13 / 47
Interacting-Branching proc. ,→ 3 Particle/SMC occupation measures: (N = 3)
•
•
•
/ •=• • ~? ~ ~ ~~ ~ / • @~ /• •=• < @@ xx x @@ x @@ xxx x / •=• /• • /•
Current population ,→
N 1 X δξni ←i-th individual at time n N i=1
Historical/genealogical tree ,→
N 1 X δ(ξi ,ξi ,...,ξi )←i-th ancestral line n,n 0,n 1,n N i=1
Complete genealogical tree ,→
1 N
PN
i=1
δ(ξi ,ξi ,...,ξi ) 0
1
n
⊕ Mean potential values [Success proportions (Gn = 1A )] ,→
N 1 X Gn (ξni ) N i=1
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
14 / 47
Limiting measures (”Test” functions f : En → R) Occupation measures of the Current population ηnN (f ) :=
N 1 X γn (f ) f (ξni ) −→N↑∞ ηn (f ) := N γn (1) i=1
with the Feynman-Kac measures (Xn Markov with transitions Mn ): Y γn (f ) := E fn (Xn ) Gp (Xp ) 0≤p ) ≤ c(n) exp −2 N/c(n) P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
42 / 47
A stochastic perturbation model ⇔ Uniform estimates w.r.t. time Feynman-Kac (nonlinear) dynamical semigroup : ηp
Φp,n (ηp ) := ηn
A local transport formulation (works ∀ approximation scheme ηnN ' ηn !) η0 ⇓ η0N
→
η1 = Φ1 (η0 ) → Φ1 (η0N ) ⇓ η1N
→
η2 = Φ0,2 (η0 ) →
···
→ ηn = Φ0,n (η0 )
→
Φ0,2 (η0N )
→
···
→
Φ0,n (η0N )
→
Φ2 (η1N ) ⇓ η2N
→
···
→
Φ1,n (η1N )
→
···
→
⇓ N ηn−1
Φ2,n (η2N ) .. .
→
N Φn (ηn−1 ) ⇓ ηnN
Key decomposition formula ηnN − ηn
=
n X
N [Φq,n (ηqN ) − Φq,n (Φq (ηq−1 ))] ” ' ”
q=0 P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
n X 1 √ e −λ N q=0
(n−q)
43 / 47
Some crude uniform estimates w.r.t. time Hypothesis : (Time homogeneous models) ∃(m, r ) s.t. for any (x, y )
.
.
M m (x, ) ≥ M m (y , ) and Gn (x) ≤ r Gn (y )
Limiting system stability properties : kΦp,p+nm (η) − Φp,p+nm (µ)ktv ≤ (1 − 2 /r m−1 )n and w.r.t. Csisz´ ar’s H-entropy criteria H(Φp,p+nm (µ), Φp,p+nm (η)) ≤ αH (r m /) (1 − 2 /r m−1 )n H(µ, η) Examples : αH (t) = t (tv norm & Boltzmann entropy), αH (t) = t 1+p (Havrda-Charvat & Kakutani-Hellinger p-integrals, αH (t) = t 3 (L2 -norm),...
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
44 / 47
Some crude uniform estimates w.r.t. time Hypothesis : (Time homogeneous models) ∃(m, r ) s.t. for any (x, y )
.
.
M m (x, ) ≥ M m (y , ) and Gn (x) ≤ r Gn (y )
Lp -mean error bounds √ sup sup n≥0 N≥1
with
p p1 ≤ 2 b(p) m r 2m−1 /3 N E ηnN − ηn (f )
b(2p)2p = (2p)p 2−p
and b(2p+1)2p+1 =
(2p + 1)(p+1) −(p+1/2) p 2 p + 1/2
Uniform concentration estimates : sup P | ηnN − ηn (f )| ≥ δ ≤ 6 exp −N δ 2 5 /(32mr 4m−1 ) n≥0
Extensions to Zolotarev’s seminorms kηnN − ηn kF = supf ∈F |[ηnN − ηn ](f )| P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
45 / 47
Fluctuations and large deviations Central Limit Theorems [Sharp Lp estimates] {http-ref :
1999 2004 : DM, Guionnet, Jacod, Ledoux, Tindel} √ VnN (f ) := N ηnN (f ) − ηn (f ) =⇒ Vn (f ) = Centered Gaussian r.v.
1
Functional Central Limit Theorems. [∀d, ∀(f i )1≤i≤d ] (VnN (f 1 ), . . . , VnN (f d )) =⇒ Vn (f 1 ), . . . , Vn (f d )
2 3 4
Empirical processes Donsker type theorems. Convergence rates Berry Esseen type theorems. Path space models (Complete tree and genealogical tree).
Large deviations principles [Sharp asymptotic expo estimates] 1 log P ηnN 6∈ V(ηn ) N→∞ N lim
Example : V(ηn ) = {µ : |ηnN (f ) − ηn (f )| ≤ } (weak and strong τ -topo). {http-ref 1998 2004 : P. Del Moral (INRIA Bordeaux)
DM, Dawson, Guionnet, Zajic}
INRIA Centre Bordeaux-Sud Ouest, France
46 / 47
Feynman-Kac (nonlinear) semigroup ηp −→ Φp,n (ηp ) := ηn
LOCAL FLUCTUATION THEOREM :
N
Wn :=
“ ”i √ h N N N ηn − Φn ηn−1 ' Wn Centered and Independent Gaussian field
Local transport formulation : η0 ⇓ η0N
→
η1 = Φ1 (η0 )
→
Φ1 (η0N ) ⇓ η1N
→
η2 = Φ0,2 (η0 )
→
···
→
Φ0,n (η0 )
→
Φ0,2 (η0N )
→
···
→
Φ0,n (η0N )
→
Φ2 (η1N ) ⇓ η2N
→
···
→
Φ1,n (η1N )
→
···
→
⇓ N ηn−1
→
Φ2,n (η2N ) . . . N Φn (ηn−1 ) ⇓ ηnN
Key decomposition formula entering the stability of the limiting system:
N
ηn − ηn
n X
=
N
N
[Φq,n (ηq ) − Φq,n (Φq (ηq−1 ))]
q=0 n 1 X N Wq Dq,n ←- First order decomp. Φp,n (η) − Φp,n (µ) ' (η − µ)Dp,n + (η − µ)⊗2 . . . √ N q=0
'
⇒
√ Two lines proof of a Functional CLT :
n h i X N N ηn − ηn ' Wq Dq,n q=0
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
47 / 47
