Adaptive Importance Sampling Strategies - CERMICS

1 downloads 0 Views 159KB Size Report
Bottom line of adaptive methods: add a biasing term, depending on ξ only, and adapt it on-the-fly in order to reach a uniform distribution of ξ(qt). • The resulting ...
Adaptive Importance Sampling Strategies Tony LELIEVRE, Mathias ROUSSET, Gabriel STOLTZ MICMAC project team, INRIA & CERMICS, Ecole des Ponts et Chauss´ees http://cermics.enpc.fr/∼stoltz/

The metastability problem

Convergence of the Adaptive Biasing Force method [6]

• Applications in computational physics and Bayesian statisitics, when some high dimensional probability measure has to be sampled Z • Measure to be sampled µ(dq) = Z −1 e−βV (q) dq with Z = e−βV (q) dq r D 2 dWt, ensemble averages can be approximated by • For an ergodic dynamics such as dqt = −∇V (qt) dt + β trajectorial averages: Z Z T 1 hAi = A(q) µ(dq) = lim A(qt) dt (1) T →+∞ T 0 D • Although the convergence (1) is theoretically ensured, it can be very slow from a numerical viewpoint • Metastability arises from free-energy barriers, which can have either energetic or entropic origins

• Dynamics

 q      dqt = −∇ V − Ft ◦ ξ − β −1 ln(|∇ξ|−2) (qt) |∇ξ|−2(qt) dt + 2β −1|∇ξ|−1(qt) dWt,   Z ξ (t, z) ′  F (z) = E f (X ) ξ(X ) = z = f (q) dψ  t t  t Σ(z)

• Existence and uniqueness of the solution [3]  !  −1  ∇ V − Ft ◦ ξ ψ + β ∇ψ    , ∂tψ = div  −2  |∇ξ|   Z  • Nonlinear PDE on the law of qt: f |∇ξ|−1ψ(t, ·)dσΣ(z)   Σ(z) ′  Z . F (z) =  t   −1ψ(t, ·)dσ  |∇ξ|  Σ(z)  Σ(z)

• Expected longtime limits: Ft(z) → F (z), ψt(q) → ψ∞(q) = e−β(V (q)−F (ξ(q)))

• Proof using entropy estimates. The relative entropy of µ with respect to ν is H(µ|ν) =

Free-energy biased sampling • Consider a function ξ : D → Rm (m ≪ dim(D)) such that ξ(qt) is some slowly evolving degree of freedom (a notion to be precised...) • Marginal equilibrium distribution (m = 1 to simplify)

Σ(z)

e−βV (q) δξ(q)−z (dq) dz = Z −1

Z

e−βV (q)

Σ(z)

dσΣ(z)(dq) |∇ξ(q)|

where the macroscopic and microscopic entropies are respectively Z ξ  ξ Em(t) = EM (t) = H ψ (t, ·) ψ∞ , em(t, z) ψ ξ (t, z) dz

dz = e−βF (z) dz,

M

where Σ(z) = q ∈ D ξ(q) = z is a submanifold of D 

• Conditional equilibrium distribution

−βV (q) |∇ξ(q)|−1 σ e Σ(z)(dq) ξ ν (dq | z) = Z e−βV |∇ξ|−1 dσΣ(z)

• Mean force

∇F (z) =

f (q) ν ξ (dq | z)



∇ξ ∇ξ · ∇V −1 − β div f= 2 |∇ξ| |∇ξ|2

with

Σ(z)



• Application: entropic barrier 8

8

4

4

x coordinate

x coordinate

y coordinate

y coordinate

3.

−4

0

−4

0.

x coordinate

−8

−6

−2

2

6

8

−8 0.0

x coordinate

5000

10000

15000

20000

 ξ ξ em(t, z) = H ν (t, · | z) ν (∞, · | z)

• The overall rate of convergence of the method is limited by the rate of convergence of the projected dynamics, so that the metastability in the ξ direction is removed

• When the potential is biased by the free-energy, V(q) = V (q) − F (ξ(q)), the new marginal distribution is constant = uniform sampling in ξ and the metastability is removed in this direction!

0

dµ dµ. Then ln dν

• The marginal density satisfies a simple diffusion equation ∂tψ ξ = β −1∂zz ψ ξ , therefore EM → 0 • Control of the microscopic entropy when assuming some uniform ergodicity for the dynamics at fixed ξ(q) = z (logarithmic Sobolev inequality)

Σ(z)

Z





E(t) = H(ψ(t, ·) | ψ∞) = EM (t) + Em(t),

−8 0

5000

Time

10000

15000

20000

Time

(1) Potential for which entropic barriers have to be overcome (0 in the region enclosed by the curve, +∞ outside) and (2) associated free energy profile when ξ(x, y) = x. Typical trajectories for a simple Metropolis random walk (3) and a dynamics biased by the free energy (4).

Application to Statistical Physics: Mulitple replica & Selection • Model dimer in a solvent (double-well potential), solvent particles interacting through the purely repulsive potential (truncated Lennard-Jones):       #2 " 12 6 σ σ  (r − σ − w)2 4ε − + ǫ if r ≤ σ, VWCA(r) = Vdimer(r) = h 1 − r r 2  w 0 if r > σ,

• Two energy minima (compact state r = r0 = 21/6σ, stretched state r = r0 + 2w), energy barrier h |q1 − q2| − r0 • Reaction coordinate = dimer bond length: ξ(q) = 2w ξ ∂zz ψt • Implementation with multiple replicas [7] and selection procedure with the fitness function S = c ξ ψt

Adaptive methods: A general framework and consistency results [5] • Bottom line of adaptive methods: add a biasing term, depending on ξ only, and adapt it on-the-fly in order to reach a uniform distribution of ξ(qt).

Bond length

3.0

• The resulting potential is Vt = V − Ft ◦ ξ, and rules to update the bias are needed q • Denote by ψ(t, q) the law of the process dqt = −∇(V − Ft ◦ ξ)(qt) dt + β2 dWt

• General update formula for Adaptive Biasing Potential method [4, 8] using the observed free energy ! Z   dσΣ(z) dFt(z) −1 = Ft Fobs(t, z) , Fobs(t, z) = −β ln ψ(t, ·) dt |∇ξ| Σ(z) • General update formula for Adaptive Biasing Force method [1, 2] using the observed mean force Z   dΓt(z) = Gt Γobs(t, z) − Γt(z) , Γobs(t, z) = f dψ ξ (t, ·|z) dt Σ(z) Possibly, set Γt(z) = Γobs(t, z) • If some equilibrium is reached and the updating functions Ft and Gt are strictly increasing (with Gt(0) = 0), then F∞ = F + c and Γ∞ = ∇F∞ • Issues with the case m > 1: ABF is not a gradient dynamics

References [1] E. Darve and A. Pohorille, Calculating free energies using average force, J. Chem. Phys. 115(20) (2001) 9169–9183 ´nin and C. Chipot, Overcoming free energy barriers using unconstrained molecular dynamics sim[2] J. He ulations, J. Chem. Phys. 121(7) (2004) 2904–2914 `vre and R. Roux, Existence, uniqueness and convergence of a particle approx[3] B. Jourdain, T. Lelie imation for the adaptive biasing force process, In preparation. [4] A. Laio and M. Parinello, Escaping free-energy minima, Proc. Natl. Acad. Sci. U.S.A 99 (2002) 12562-12566. `vre, M. Rousset and G. Stoltz, Computation of free energy profiles with parallel adaptive [5] T. Lelie dynamics, J. Chem. Phys. 126 (2007) 134111 `vre, M. Rousset and G. Stoltz, Long-time convergence of an adaptive biasing force method, [6] T. Lelie Nonlinearity 21 (2008) 1155-1181

0.5

3.5

0.4

2.5

Escape rate

µξ (dz) = Z −1

Z

the total entropy can be decomposed as

Z

2.0

0.3

0.2

1.5 0.1 1.0

0

50

100

150

0.0 0.0

200

0.1

0.2 Time

Time

0.3

0.4

Left: Dynamics with and without bias. Right: Selection procedure with increasing selection strength c.

Application to Bayesian statistics (Monte-Carlo ABF) • Hidalgo stamp problem: the thickness of Ndata = 485 stamps are measured, and the corresponding histogram is approximated by a mixture of N Gaussians. N × [v 3N −1, • Parameters x n = (q1, . . . , qN −1, µ1, . . . , µN , v1, . . . , vN ) ∈ SN −1 × [µ , µ ] , +∞) ⊂ R max min min o with SN −1 = (q1, . . . , qN −1) 0 ≤ qi ≤ 1, q1 + . . . qN −1 ≤ 1 r N  v  X PN −1 vi i 2 qi • The corresponding mixture is f (y | x) = exp − (y − µi) , where qN = 1 − i=1 qi 2π 2 i=1





• The likelihood of observing the data yi, 1 ≤ i ≤ Ndata is

Π(y | x) =

NY data

f (yd | x) ∝ e−βVlikelihood

d=1

• Potential V = Vprior + Vlikelihood such that the probability of a given configuration is proportional to exp(−V ) • A simple Metropolis random-walk is metastable • Use a Monte-Carlo ABF dynamics where ξ(x) = q1 is the reaction coordinate. • Principle of the method = update the average force experienced in the q1 direction and obtain the free-energy bias by integrating the approximated mean force

N=3 N=4 N=5 N=6 N=7

[7] P. Raiteri, A. Laio, F. L. Gervasio, C. Micheletti, and M. Parrinello, Efficient reconstruction of complex free energy landscapes by multiple walkers metadynamics, J. Phys. Chem. B 110(8) (2006) 3533–3539 [8] F. G. Wang and D. P. Landau, Determining the density of states for classical statistical models: A random walk algorithm to produce a flat histogram, Phys. Rev. E 64(5) (2001) 056101

0.05

0.075

0.1 0.125 Thickness

0.15

Left: Histogram of the data, and fit with several gaussian modes. Middle: Biasing potential obtained from a Monte-Carlo ABF dynamics. Right: Evolution of the averages µ1, µ2 and µ3 for a biased dynamics