Adaptive stochastic search method, evolutionary learning models, interacting stochastic grids ... Mean field particle interpretation of Feynman-Kac measures.
A mean field theory of sequential Monte Carlo methods P. Del Moral
INRIA, Centre Bordeaux-Sud Ouest Department of Statistics, Oxford Univ., december 2008.
∼ Series of joint works : F. C´erou, D. Crisan, D. Dawson, A. Doucet, A. Guyader, J. Jacod, A. Jasra, A. Guionnet, M. Ledoux, L. Miclo, F. Patras, T. Lyons, S. Rubenthaler,... A pair of http-references : ,→ Feynman-Kac formulae. Genealogical and interacting particle systems, Springer (2004), + References ,→ DM, Doucet, Jasra. SMC Samplers. JRSS B (2006). P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
1 / 32
Outline
1
Foundations & application areas
2
A simple mathematical model
3
Mean field particle models
4
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
2 / 32
Summary
1
Foundations & application areas Some ”different” particle interpretation models Sequential Monte Carlo & Feynman-Kac models Motivating application areas
2
A simple mathematical model
3
Mean field particle models
4
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
3 / 32
Particle Interpretation models Mathematical physics and molecular chemistry (≥ 19500 s) : Particle/microscopic interpretation models, particle absorption, macro-molecular chains, quantum and diffusion Monte Carlo. Environmental studies and biology (≥ 19500 s): Population, gene evolutions, species genealogies, branching/birth and death models. Evolutionary mathematics and engineering sciences (≥ 19700 s): Adaptive stochastic search method, evolutionary learning models, interacting stochastic grids approximations, genetic algorithms. Applied Probability and Bayesian Statistics (≥ 19900 s): Approximating simulation technique (recursive acceptance-rejection model), Sequential Monte Carlo, new interacting MCMC technology (Andrieu, Bercu, DM, Doucet, Jasra). Pure mathematics (≥ 19600 s for fluid models, ≥ 19900 s for discrete time and interacting jump models): Stochastic linearization tech., mean field particle interpretations of nonlinear PDE and measure valued equations.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
4 / 32
Central idea of particle/SMC/evolutionary sampling : Physical and Biological intuitions ∈ Engineering problems [learning, adaptation, optimization,...] Sequential Monte Carlo Particle Filters Genetic Algorithms Evolutionary Population Diffusion Monte Carlo Quantum Monte Carlo Sampling Algorithms
Sampling Prediction Mutation Exploration Free evolutions Walkers motions Transition proposals
Resampling Updating Selection Branching Absorption Reconfiguration Acceptance-rejection
More botanical names : spawning, cloning, pruning, enrichment, go with the winner, and many others. Pure mathematical point of view : = Mean field particle interpretation of Feynman-Kac measures
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
5 / 32
Some application areas of Feynman-Kac formulae Physics : Feynman-Kac-Schroedinger semigroups ∈ nonlinear integro-differential equations (∼ generalized Boltzmann models). Spectral analysis of Schr¨ odinger operators and large matrices with nonnegative entries. Particle evolutions in disordered/absorbing media. Multiplicative Dirichlet problems with boundary conditions. Microscopic and macroscopic interacting particle interpretations. Chemistry and Biology: Self-avoiding walks, macromolecular simulation, directed polymers. Spatial branching and evolutionary population models. Coalescent and Genealogical tree based evolutions.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
6 / 32
Some application areas of Feynman-Kac formulae Rare events analysis: Multisplitting and branching particle models (Restart type methods). Importance sampling and twisted probability measures. Genealogical tree based simulations (default tree sampling models). Advanced Signal processing: Optimal filtering, prediction, smoothing. Open loop optimal control, optimal regulation. Interacting Kalman-Bucy filters. Stochastic and adaptative grid approximation-models Statistics/Probability: Restricted Markov chains (w.r.t terminal values, visiting regions, constraints simulation problems,...) Analysis of Boltzmann-Gibbs type distributions (simulation, partition functions, localization models...). Random search evolutionary algorithms, interacting Metropolis/simulated annealing algo, combinatorial counting. P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
7 / 32
Summary
1
Foundations & application areas
2
A simple mathematical model Standard notation A genetic type spatial branching process Genealogical tree approximation measures Limiting Feynman-Kac measures
3
Mean field particle models
4
Some theoretical aspects
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
8 / 32
Standard notation E measurable space, P(E ) proba. on E , B(E ) bounded meas. functions. Z (µ, f ) ∈ P(E ) × B(E ) −→ µ(f ) = µ(dx) f (x) M(x, dy ) integral operator on E Z M(f )(x) = M(x, dy )f (y ) Z [µM](dy ) = µ(dx)M(x, dy )
(=⇒ [µM](f ) = µ[M(f )] )
Bayes-Boltzmann-Gibbs transformation : G : E → [0, ∞[ with µ(G ) > 0 ΨG (µ)(dx) =
P. Del Moral (INRIA Bordeaux)
1 G (x) µ(dx) µ(G )
INRIA Centre Bordeaux-Sud Ouest, France
9 / 32
Only 3 Ingredients A state space : En with n = time/level index [transitions, paths, excursions,...]. 0 0 0 Xn := (Xn−1 , Xn0 ), X[0,n] , X[t0 n−1 ,tn ] , X[T , ... n−1 ,Tn ]
A Markov Proposal/Exploration/Mutation transition : Mn (xn−1 , dxn ) := P (Xn ∈ dxn | Xn−1 = xn−1 ) A potential/likelihood/fitness/weight function on En : Gn : xn ∈ En −→ Gn (xn ) ∈ [0, ∞[ Running Examples : [Confinement] Xn =Simple random walk (SRW) on En = Z and Gn = 1A . [Filtering] Mn =signal transitions, Gn =Likelihood weight function.
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
10 / 32
SMC/Genetic type branching particle model : ξn1 .. . i Updating/Branching ξn −−−−−−−−−−−−−−−−→ .. . N ξn
ξbn1 .. . ξbni .. . ξbnN
Mn+1 1 ξn+1 −−−−−−−−−−−−−−−−− −−→ .. .. . . Prediction/Exploration i ξn+1 −−→ −−−−−−−−−−−−−−−−− . .. .. . N ξn+1 −−−−−−−−−−−−−−−−− −−→
Selection/Branching : (∀n ≥ 0 s.t. n (x 1 , . . . , x N ) × Gn (x i ) ∈ [0, 1]) Acceptance probability: ξbni = ξni Otherwise : ξbni = ξnj
with probability
n (ξn1 , . . . , ξnN ) Gn (ξni )
Gn (ξnj ) with probability PN k k=1 Gn (ξn )
Running examples: [Confinement & Filtering] = [(Gn = 1A ) & (Gn =Likelihood)]. P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
11 / 32
Some remarks : n = 0 =⇒ Simple Mutation-Selection Genetic model. Gn = exp {−Vt ∆t} & n = 1 =⇒ Vt -expo-clocks ⊕ uniform selection Gn ∈ [0, 1] & n = 1 ⇒ Interacting Acceptance-Rejection Sampling. Better fitted individuals acceptance : For
n (x 1 , . . . , x N )Gn (x i ) = Gn (x i )/ sup Gn (x j ) 1≤j≤N
Related branching rules: [DM-Crisan-Lyons MPRF 99, DM 04] (Given ξn = (ξni )i ) Pni := Proportion of offsprings of the individual ξni PN k Unbiasedness property: E Pni = Gn (ξnj )/ k=1 Gn (ξ n ) hP i 2 N i i Local mean error : E f (ξni ) ≤ i=1 Pn − E Pn P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
Cte N
12 / 32
Interacting-Branching proc. ,→ 3 Particle/SMC occupation measures: (N = 3)
•
•
•
/ •=• • ~? ~ ~ ~~ ~ / • @~ /• •=• < @@ xx x @@ x @@ xxx x / •=• /• • /•
Current population ,→
N 1 X δξni ←i-th individual at time n N i=1
Historical/genealogical tree ,→
N 1 X δ(ξi ,ξi ,...,ξi )←i-th ancestral line n,n 0,n 1,n N i=1
Complete genealogical tree ,→
1 N
PN
i=1
δ(ξi ,ξi ,...,ξi ) 0
1
n
⊕ Mean potential values [Success proportions (Gn = 1A )] ,→
N 1 X Gn (ξni ) N i=1
P. Del Moral (INRIA Bordeaux)
INRIA Centre Bordeaux-Sud Ouest, France
13 / 32
Limiting measures (”Test” functions f : En → R) Occupation measures of the Current population ηnN (f ) :=
N 1 X γn (f ) f (ξni ) −→N↑∞ ηn (f ) := N γn (1) i=1
with the Feynman-Kac measures (Xn Markov with transitions Mn ): Y γn (f ) := E fn (Xn ) Gp (Xp ) 0≤p
