A general framework for Simulating Random Multi-Phase Media

A general framework for Simulating Random Multi-Phase Media P.S. Koutsourelakis Institute of Engineering Mechanics University of Innsbruck

Baltimore 2005 – p.1/23

Motivation

Soil Profile

Multiphase Fire-resistant

Concrete

Baltimore 2005 – p.2/23

Motivation

Soil Profile

Multiphase Fire-resistant

Concrete

Variability - Uncertainties in Various Length Scales •

Appropriate Probabilistic Model

•

Incorporate them in the analysis

Baltimore 2005 – p.2/23

Modeling Multi-Phase Media Random Field Approach

Baltimore 2005 – p.3/23

Modeling Multi-Phase Media Random Field Approach   1 Binary Fields: I (j) (x) =  0

if x ∈ phase j otherwise

Baltimore 2005 – p.3/23

Modeling Multi-Phase Media Random Field Approach   1 Binary Fields: I (j) (x) =  0 X

if x ∈ phase j otherwise

I (j) (x) = 1 ∀x

j

Baltimore 2005 – p.3/23

Modeling Multi-Phase Media Random Field Approach   1 Binary Fields: I (j) (x) =  0 X

if x ∈ phase j otherwise

I (j) (x) = 1 ∀x

j

Discrete-Valued Fields: I(x) = j

if x ∈ phase j

Baltimore 2005 – p.3/23

Modeling Multi-Phase Media Random Field Approach   1 Binary Fields: I (j) (x) =  0 X

if x ∈ phase j otherwise

I (j) (x) = 1 ∀x

j

• •

Discrete-Valued Fields: I(x) = j I (j) (x) and I(x) are random functions.

if x ∈ phase j

Their probabilistic characteristics are related to the morphological uncertainties. Baltimore 2005 – p.3/23

Incorporation of Uncertainties Monte Carlo Simulations

Baltimore 2005 – p.4/23

Incorporation of Uncertainties Monte Carlo Simulations

•

Generate samples of the medium (i.e. I(x))

•

Solve deterministically for each sample

•

Process statistics of the results

Baltimore 2005 – p.4/23

Incorporation of Uncertainties Monte Carlo Simulations

•

Generate samples of the medium (i.e. I(x))

•

Solve deterministically for each sample

•

Process statistics of the results

Baltimore 2005 – p.4/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics:

Baltimore 2005 – p.5/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics: 1st order - volume fraction of phase j : E[I (j) (x)] ≈

1 K

P

(j) k Ik (x).

Baltimore 2005 – p.5/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics: 1st order - volume fraction of phase j : E[I (j) (x)] ≈

1 K

2nd order -(cross)-correlations : E[I (j) (x1 )I (r) (x2 )] ≈

P 1 K

(j) k Ik (x).

P

(j) (r) k Ik (x1 )Ik (x2 ).

Baltimore 2005 – p.5/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics: 1st order - volume fraction of phase j : E[I (j) (x)] ≈

1 K

2nd order -(cross)-correlations : E[I (j) (x1 )I (r) (x2 )] ≈

P 1 K

(j) k Ik (x).

P

(j) (r) k Ik (x1 )Ik (x2 ).

Higher order - lineal path functions, statistics of Fourier or wavelet transforms of I k .

Baltimore 2005 – p.5/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics: 1st order - volume fraction of phase j : E[I (j) (x)] ≈

1 K

2nd order -(cross)-correlations : E[I (j) (x1 )I (r) (x2 )] ≈

P 1 K

(j) k Ik (x).

P

(j) (r) k Ik (x1 )Ik (x2 ).

Higher order - lineal path functions, statistics of Fourier or wavelet transforms of I k .

In general any function g(I) : E[g(I(x))] ≈

1 K

P

k

gm (Ik (x)).

Baltimore 2005 – p.5/23

Problem Formulation Given a number of samples of the medium {Ik (x)}K k=1 we can extract statistics: 1st order - volume fraction of phase j : E[I (j) (x)] ≈

1 K

2nd order -(cross)-correlations : E[I (j) (x1 )I (r) (x2 )] ≈

P 1 K

(j) k Ik (x).

P

(j) (r) k Ik (x1 )Ik (x2 ).

Higher order - lineal path functions, statistics of Fourier or wavelet transforms of I k .

In general any function g(I) : E[g(I(x))] ≈

1 K

P

k

gm (Ik (x)).

Goal: Digitally generate sample functions of the random field I(x) based on given probabilistic information (i.e. volume fractions, correlation functions etc).

Baltimore 2005 – p.5/23

Problem Formulation Let I denote the values of I(x) at a discrete domain i.e. an N × N lattice.

Baltimore 2005 – p.6/23

Problem Formulation Let I denote the values of I(x) at a discrete domain i.e. an N × N lattice. Consider M features-functions gm for which: E[gm (I)] = µm

Baltimore 2005 – p.6/23

Problem Formulation Let I denote the values of I(x) at a discrete domain i.e. an N × N lattice. Consider M features-functions gm for which: E[gm (I)] = µm

(known)

Baltimore 2005 – p.6/23

Problem Formulation Let I denote the values of I(x) at a discrete domain i.e. an N × N lattice. Consider M features-functions gm for which: E[gm (I)] = µm

(known)

Incomplete Description of I F=



f (I) :

Z

gm (I)f (I)dI = µm



Baltimore 2005 – p.6/23

Problem Formulation Let I denote the values of I(x) at a discrete domain i.e. an N × N lattice. Consider M features-functions gm for which: E[gm (I)] = µm

(known)

Incomplete Description of I F=



f (I) :

Z

gm (I)f (I)dI = µm

•

Real pdf f ∗ (unknown) belongs in F.

•

Are all pdfs in F equivalent?



Baltimore 2005 – p.6/23

Existing Methods Most commonly use maps from Gaussian fields that are able to reproduce at best 2nd order information.

Baltimore 2005 – p.7/23

Existing Methods Most commonly use maps from Gaussian fields that are able to reproduce at best 2nd order information. Deficiencies •

2nd order characteristics give an incomplete description of the uncertainties in the medium.

Baltimore 2005 – p.7/23

Existing Methods Most commonly use maps from Gaussian fields that are able to reproduce at best 2nd order information. Deficiencies •

2nd order characteristics give an incomplete description of the uncertainties in the medium.

(Yeong & Torquato 1998)

Baltimore 2005 – p.7/23

Existing Methods Most commonly use maps from Gaussian fields that are able to reproduce at best 2nd order information. Deficiencies •

2nd order characteristics give an incomplete description of the uncertainties in the medium.

•

All pdfs f ∈ F are considered equivalent for simulation purposes.

Baltimore 2005 – p.7/23

Revised Requirements - Goals •

The simulation method should be able to incorporate as much information as possible.

Baltimore 2005 – p.8/23

Revised Requirements - Goals •

The simulation method should be able to incorporate as much information as possible.

•

A rationale should be developed to select the most appropriate f ∈ F.

Baltimore 2005 – p.8/23

Proposed Framework - Theory Maximum Entropy Principle  R p(I) = arg maxf ∈F − f (I) log f (I)dI

subject to f (I) ∈ F, i.e: Z f (I)dI = 1 and Ef [gm (I)] = µm

∀m

Baltimore 2005 – p.9/23

Proposed Framework - Theory Maximum Entropy Principle  R p(I) = arg maxf ∈F − f (I) log f (I)dI

subject to f (I) ∈ F, i.e: Z f (I)dI = 1 and Ef [gm (I)] = µm •

∀m

Information of any order can be introduced.

Baltimore 2005 – p.9/23

Proposed Framework - Theory Maximum Entropy Principle  R p(I) = arg maxf ∈F − f (I) log f (I)dI

subject to f (I) ∈ F, i.e: Z f (I)dI = 1 and Ef [gm (I)] = µm

∀m

•

Information of any order can be introduced.

•

The fusion of the available information is done in an optimal manner.

Baltimore 2005 – p.9/23

Proposed Framework - Theory Maximum Entropy Principle  R p(I) = arg maxf ∈F − f (I) log f (I)dI

subject to f (I) ∈ F, i.e: Z f (I)dI = 1 and Ef [gm (I)] = µm

∀m

•

Information of any order can be introduced.

•

The fusion of the available information is done in an optimal manner.

(Jaynes 1979): “ Given incomplete information, the distribution of maximum entropy is not only the one that can be realized in the greatest number of ways; for large sample size the overwhelming majority of distributions compatible with our information have entropy very close to the maximum.” Baltimore 2005 – p.9/23

Proposed Framework - Theory Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)}

Baltimore 2005 – p.10/23

Proposed Framework - Theory Maximum Entropy Distribution P 1 exp {− m λm gm (I)} p(I) = Z(λ) R P partition function : Z(λ) = exp {− m λm gm (I)} dI

Baltimore 2005 – p.10/23

Proposed Framework - Theory Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)} The vector λ = (λ1 , λ2 , . . . , λm ) is determined by solving the equations: R Ep [gm (I)] = gm (I)p(I)dI = µm ∀m

Baltimore 2005 – p.10/23

Proposed Framework - Theory Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)} The vector λ = (λ1 , λ2 , . . . , λm ) is determined by solving the equations: R Ep [gm (I)] = gm (I)p(I)dI = µm ∀m Equivalence with Maximum Log-Likelihood Estimation P P L(λ) = k log p(Ik ) = − m λm µm − log Z(λ)

Baltimore 2005 – p.10/23

Proposed Framework - Theory Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)} The vector λ = (λ1 , λ2 , . . . , λm ) is determined by solving the equations: R Ep [gm (I)] = gm (I)p(I)dI = µm ∀m Equivalence with Maximum Log-Likelihood Estimation P P L(λ) = k log p(Ik ) = − m λm µm − log Z(λ)

If gm are linearly independent then L(λ) is concave and the solution vector λ is unique.

Baltimore 2005 – p.10/23

Proposed Framework - Algorithms •

Find λ that maximizes L(λ) = − find p(I).

P

m

λm µm − log Z(λ) in order to

Baltimore 2005 – p.11/23

Proposed Framework - Algorithms P

•

Find λ that maximizes L(λ) = − find p(I).

•

Draw samples from the maximum entropy distribution P 1 exp {− m λm gm (I)}. p(I) = Z(λ)

m

λm µm − log Z(λ) in order to

Baltimore 2005 – p.11/23

Proposed Framework - Algorithms P

•

Find λ that maximizes L(λ) = − find p(I).

•

Draw samples from the maximum entropy distribution P 1 exp {− m λm gm (I)}. p(I) = Z(λ)

m

λm µm − log Z(λ) in order to

These tasks can be carried out simultaneously using Markov Chain Monte Carlo

Baltimore 2005 – p.11/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

m

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X m

λm µm −log Z(λ)

Z(λ) =

R



P



exp − m λm gm (I) dI not known analytically

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R



P



exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

R

1dI is known.

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R





P

exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

•

For λ1 6= λ0 : 1

Z(λ )

= =

Z

e−

P

Z(λ0 )

1 m λm gm (I)

Z

p0 (I)

R

1dI is known.

0

p0 (I)dI = Z(λ )

Z

e

−

P

1 0 m (λm −λm )gm (I) p0 (I)dI

w(I)p0 (I)dI

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R





P

exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

•

For λ1 6= λ0 : 1

Z(λ )

= =

Z

e−

0

P

Z(λ )

1 m λm gm (I)

p0 (I)

Z

R

1dI is known.

0

p0 (I)dI = Z(λ )

Z

e

−

P

1 0 m (λm −λm )gm (I) p0 (I)dI

1 X w(I )p0 (I )dI ≈ Z(λ ) w(I (j) ) N j 0

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R





P

exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

•

For λ1 6= λ0 : 1

Z(λ )

= =

Z

e−

0

P

Z(λ )

1 m λm gm (I)

p0 (I)

Z

R

1dI is known.

0

p0 (I)dI = Z(λ )

Z

e−

P

1 0 m (λm −λm )gm (I)

p0 (I)dI

1 X w(I)p0 (I )dI ≈ Z(λ ) w(I (j) ) N j 0

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R





P

exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

•

For λ1 6= λ0 : 1

Z(λ )

= =

Z

e−

0

P

Z(λ )

1 m λm gm (I)

p0 (I)

Z

R

1dI is known.

0

p0 (I)dI = Z(λ )

Z

e−

P

1 0 m (λm −λm )gm (I)

p0 (I)dI

1 X w(I)p0 (I )dI ≈ Z(λ ) w(I (j) ) N j 0

- Noisy function evaluations

Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) L(λ) = −

X

λm µm −log Z(λ)

Z(λ) =

m

R





P

exp − m λm gm (I) dI not known analytically

Importance Sampling •

For λ0 = 0 (uniform distribution p0 ), Z(0) =

•

For λ1 6= λ0 : 1

Z(λ )

= =

Z

e−

0

P

Z(λ )

1 m λm gm (I)

p0 (I)

Z

R

1dI is known.

0

p0 (I)dI = Z(λ )

Z

e−

P

1 0 m (λm −λm )gm (I)

p0 (I)dI

1 X w(I)p0 (I )dI ≈ Z(λ ) w(I (j) ) N j 0

- Noisy function evaluations - The closer λ1 is to λ0 , the smaller the variance of w(I) and the less the noise. Baltimore 2005 – p.12/23

Proposed Framework - Maximization of L(λ) Conjugate Gradients 1 Iteration i = 0: Set λ0 = 0, u0 = −∇L(λ0 ) and v0 = u0 . 2 Let a∗ = arg maxa L(λi + αvi ). Set λi+1 = λi + a∗ vi . 3 Set ui+1 = −∇L(λi+1 ) and vi+1 = ui+1 + γvi . 4 Set i = i + 1 and goto step 2.

Baltimore 2005 – p.13/23

Proposed Framework - Maximization of L(λ) Conjugate Gradients 1 Iteration i = 0: Set λ0 = 0, u0 = −∇L(λ0 ) and v0 = u0 . 2 Let a∗ = arg maxa L(λi + αvi ). Set λi+1 = λi + a∗ vi . 3 Set ui+1 = −∇L(λi+1 ) and vi+1 = ui+1 + γvi . 4 Set i = i + 1 and goto step 2. Advantages •

Robust for noisy functions

Baltimore 2005 – p.13/23

Proposed Framework - Maximization of L(λ) Conjugate Gradients 1 Iteration i = 0: Set λ0 = 0, u0 = −∇L(λ0 ) and v0 = u0 . 2 Let a∗ = arg maxa L(λi + αvi ). Set λi+1 = λi + a∗ vi . 3 Set ui+1 = −∇L(λi+1 ) and vi+1 = ui+1 + γvi . 4 Set i = i + 1 and goto step 2. Advantages •

Robust for noisy functions

•

Allows for small enough steps that result in reduced noise

Baltimore 2005 – p.13/23

Proposed Framework - Maximization of L(λ) Conjugate Gradients 1 Iteration i = 0: Set λ0 = 0, u0 = −∇L(λ0 ) and v0 = u0 . 2 Let a∗ = arg maxa L(λi + αvi ). Set λi+1 = λi + a∗ vi . 3 Set ui+1 = −∇L(λi+1 ) and vi+1 = ui+1 + γvi . 4 Set i = i + 1 and goto step 2. Advantages •

Robust for noisy functions

•

Allows for small enough steps that result in reduced noise

•

Makes use of only the derivatives of L instead of the curvature matrix

∂ 2 L(λ) ∂λi λj

which

requires more storage and its estimation contains higher error.

Baltimore 2005 – p.13/23

Proposed Framework - Calculation of L(λ) and ∇L(λ) i

L(λ ) = −

P

i m λm µ m

i

− log Z(λ )

∂L(λi ) ∂λm

= −µm + Ep(i) [gm (I)]

Baltimore 2005 – p.14/23

Proposed Framework - Calculation of L(λ) and ∇L(λ) i

L(λ ) = − Z(λi ) =

R

P

i m λm µ m

i

− log Z(λ )

∂L(λi ) ∂λm

= −µm + Ep(i) [gm (I)]

 P exp − m λim gm (I) dI

Baltimore 2005 – p.14/23

Proposed Framework - Calculation of L(λ) and ∇L(λ) i

L(λ ) = − Z(λi )

=

R



P

exp −

i m λm µ m

P

i

− log Z(λ )

i m λm gm (I)



dI

∂L(λi ) ∂λm

p(i) (I)

=

= −µm + E

1 Z(λi )



exp −

P

p(i)

[gm (I)]

i m λm gm (I)



Baltimore 2005 – p.14/23

Proposed Framework - Calculation of L(λ) and ∇L(λ) i

L(λ ) = − Z(λi )

=

R

P



exp −

i m λm µ m

P

i

− log Z(λ )

i m λm gm (I)



dI

∂L(λi ) ∂λm

p(i) (I)

=

= −µm + E

1 Z(λi )



exp −

P

p(i)

[gm (I)]

i m λm gm (I)



Importance Sampling Estimators ˆ Z(λ

i+1

N ˆ i) X Z(λ )= w(I (j) ) N j=1

Epi [gm (I)] ≈

µ ˆi+1 m

N ˆ i+1 ) X Z(λ (j) (j) = w(I )g (I ) m N Z(λi ) j=1

where: •

w(I) = e

−

P

i+1 i m (λm −λm )gm (I)

Baltimore 2005 – p.14/23

Proposed Framework - Calculation of L(λ) and ∇L(λ) i

L(λ ) = − Z(λi )

=

R

P



exp −

i m λm µ m

P

i

− log Z(λ )

i m λm gm (I)



dI

∂L(λi ) ∂λm

p(i) (I)

=

= −µm + E

1 Z(λi )



exp −

P

p(i)

[gm (I)]

i m λm gm (I)



Importance Sampling Estimators ˆ Z(λ

i+1

N ˆ i) X Z(λ )= w(I (j) ) N j=1

Epi [gm (I)] ≈

µ ˆi+1 m

N ˆ i+1 ) X Z(λ (j) (j) = w(I )g (I ) m N Z(λi ) j=1

where: −

P

i+1 i m (λm −λm )gm (I)

•

w(I) = e

•

The samples I (j) are drawn from p(i) (I).

Baltimore 2005 – p.14/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler).

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution.

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution. 2. Draw Ix from the conditional distribution p(i) (Ix /I−x ) where Ix is the value of I (j) at x and I−x denotes the rest of the values of I (j) (at every point y ∈ D\{x}).

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution. 2. Draw Ix from the conditional distribution p(i) (Ix /I−x ) where Ix is the value of I (j) at x and I−x denotes the rest of the values of I (j) (at every point y ∈ D\{x}). (j+1)

3. Set Ix(j+1) = Ix and I−x

(j)

= I−x .

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution. 2. Draw Ix from the conditional distribution p(i) (Ix /I−x ) where Ix is the value of I (j) at x and I−x denotes the rest of the values of I (j) (at every point y ∈ D\{x}). (j+1)

3. Set Ix(j+1) = Ix and I−x

(j)

= I−x .

4. Set j = j + 1. If j ≤ N goto Step 1.

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution. 2. Draw Ix from the conditional distribution p(i) (Ix /I−x ) where Ix is the value of I (j) at x and I−x denotes the rest of the values of I (j) (at every point y ∈ D\{x}). (j+1)

3. Set Ix(j+1) = Ix and I−x

(j)

= I−x .

4. Set j = j + 1. If j ≤ N goto Step 1. Samples {I (j) }N j=1 are correlated but asymptotically distributed according to p(i) (independently of the initial sample I (0) ).

Baltimore 2005 – p.15/23

Simulation of samples I (j) from p(i) (I) Gibbs Sampler (or Metropolis-Hastings sampler). 1. Given I (j) , select randomly a point x ∈ D under the uniform distribution. 2. Draw Ix from the conditional distribution p(i) (Ix /I−x ) where Ix is the value of I (j) at x and I−x denotes the rest of the values of I (j) (at every point y ∈ D\{x}). (j+1)

3. Set Ix(j+1) = Ix and I−x

(j)

= I−x .

4. Set j = j + 1. If j ≤ N goto Step 1. Samples {I (j) }N j=1 are correlated but asymptotically distributed according to p(i) (independently of the initial sample I (0) ). The respective estimators converge asymptotically as N → ∞. Baltimore 2005 – p.15/23

Applications in 1D - Hard Rods

Baltimore 2005 – p.16/23

Applications in 1D - Hard Rods

Baltimore 2005 – p.16/23

Applications in 1D - Hard Rods Simulation based on Autocorrelation: gm (I) = Ii Ii+m

Baltimore 2005 – p.16/23

Applications in 1D - Hard Rods Simulation based on Autocorrelation & Lineal-Path Function: gm (I) = Ii Ii+m

gm (I) = Ii Ii+1 . . . Ii+m

Baltimore 2005 – p.16/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Hard Disks Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.17/23

Applications in 2D - Three-phase medium Simulation based on the Lineal-Path Function:

Baltimore 2005 – p.18/23

Applications in 2D - Three-phase medium Simulation based on the Lineal-Path Function:

128 × 128 pixels Baltimore 2005 – p.18/23

Applications in 2D - Functionally Graded Material Simulation based on the Volume Fraction: gm (I) = Im,j

Baltimore 2005 – p.19/23

Applications in 2D - Functionally Graded Material Simulation based on the Volume Fraction: gm (I) = Im,j

128 × 128 pixels

Baltimore 2005 – p.19/23

Feature Selection Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)}

Baltimore 2005 – p.20/23

Feature Selection Maximum Entropy Distribution P 1 p(I) = Z(λ) exp {− m λm gm (I)} Ep [gm (I)] = µm

Baltimore 2005 – p.20/23

Feature Selection Maximum Entropy Distribution    P  1 p(I) = Z(λ) exp − m λm gm (I)  

Baltimore 2005 – p.20/23

Feature Selection Maximum Entropy Distribution    P  1 p(I) = Z(λ) exp − m λm gm (I)  

We are given a number of samples of the medium {Ik }K k=1 drawn from the actual (unknown) distribution f ∗ (I).

Baltimore 2005 – p.20/23

Feature Selection Maximum Entropy Distribution    P  1 p(I) = Z(λ) exp − m λm gm (I)  

We are given a number of samples of the medium {Ik }K k=1 drawn from the actual (unknown) distribution f ∗ (I). Question Which features-functions gm (I) should be selected so that the maximum entropy distribution is as close as possible to the actual.

Baltimore 2005 – p.20/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I) f ∗ (I) dI

Baltimore 2005 – p.21/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I) f ∗ (I) dI

actual (unknown) pdf

Baltimore 2005 – p.21/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I)

f ∗ (I) dI

max. entropy pdf

Baltimore 2005 – p.21/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I) f ∗ (I) dI

strictly positive (except f ∗ (I) = p(I) ∀I)

Baltimore 2005 – p.21/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I) f ∗ (I) dI

strictly positive (except f ∗ (I) = p(I) ∀I) D(f ∗ , p) = Entropy(p) − Entropy(f ∗ )

Baltimore 2005 – p.21/23

Feature Selection Kullback-Leibler Divergence (distance) ∗

D(f , p) = −

R

f ∗ (I) log

p(I) f ∗ (I) dI

strictly positive (except f ∗ (I) = p(I) ∀I) D(f ∗ , p) = Entropy(p) − Entropy(f ∗ ) Given a number of features {gm }m of which we need to select M , then the optimal choice is the M − sized batch that minimizes (w.r.t. g m ) the entropy of the maximum entropy distribution p.

Baltimore 2005 – p.21/23

Feature Selection - Algorithmic Considerations Performing an optimization over all M − sized batches of features is practically infeasible.

Baltimore 2005 – p.22/23

Feature Selection - Algorithmic Considerations Performing an optimization over all M − sized batches of features is practically infeasible. Greedy-Stepwise Algorithm: If p

(M )

=

1 − Z(λ) e

PM m

λm gm (I)

then:

Baltimore 2005 – p.22/23

Feature Selection - Algorithmic Considerations Performing an optimization over all M − sized batches of features is practically infeasible. Greedy-Stepwise Algorithm: If p

(M )

=

1 − Z(λ) e

PM m

λm gm (I)

then:

gM +1 = arg max δ(gm ) = Entropy(p(M ) ) − Entropy(p(M +1) ) gm

Baltimore 2005 – p.22/23

Feature Selection - Algorithmic Considerations Performing an optimization over all M − sized batches of features is practically infeasible. Greedy-Stepwise Algorithm: If p

(M )

=

1 − Z(λ) e

PM m

λm gm (I)

then:

gM +1 = arg max δ(gm ) = Entropy(p(M ) ) − Entropy(p(M +1) ) gm

For features of the same scale: δ(gm ) ≈k Ep(M ) [gm ] − Ef ∗ [gm ] k

Baltimore 2005 – p.22/23

Conclusions •

Discrete-valued random fields can be used to model random heterogeneous materials.

Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. •

In practice, these are not uniquely defined by the available probabilistic information.

Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. • In practice, these are not uniquely defined by the available probabilistic information. •

Simulation methods should be able to incorporate as much probabilistic information as possible and not be restricted to 2nd order statistics which provide an inaccurate description.

Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. • In practice, these are not uniquely defined by the available probabilistic information. • Simulation methods should be able to incorporate as much probabilistic information

as possible and not be restricted to 2nd order statistics which provide an inaccurate description. •

In the proposed framework, the fusion of the available information is done in an optimal manner by employing the maximum entropy principle.

Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. • In practice, these are not uniquely defined by the available probabilistic information. • Simulation methods should be able to incorporate as much probabilistic information

as possible and not be restricted to 2nd order statistics which provide an inaccurate description. • In the proposed framework, the fusion of the available information is done in an

optimal manner by employing the maximum entropy principle. •

An MCMC-based procedure is used to incorporate the target probabilistic characteristics and draw samples from the maximum entropy distribution.

Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. • In practice, these are not uniquely defined by the available probabilistic information. • Simulation methods should be able to incorporate as much probabilistic information

as possible and not be restricted to 2nd order statistics which provide an inaccurate description. • In the proposed framework, the fusion of the available information is done in an

optimal manner by employing the maximum entropy principle. • An MCMC-based procedure is used to incorporate the target probabilistic

characteristics and draw samples from the maximum entropy distribution. •

The Kullback-Leibler divergence can be used to assess the descriptive power of the probabilistic features selected. Baltimore 2005 – p.23/23

Conclusions • Discrete-valued random fields can be used to model random heterogeneous

materials. • In practice, these are not uniquely defined by the available probabilistic information. • Simulation methods should be able to incorporate as much probabilistic information

as possible and not be restricted to 2nd order statistics which provide an inaccurate description. • In the proposed framework, the fusion of the available information is done in an

optimal manner by employing the maximum entropy principle. • An MCMC-based procedure is used to incorporate the target probabilistic

characteristics and draw samples from the maximum entropy distribution. • The Kullback-Leibler divergence can be used to assess the descriptive power of the

probabilistic features selected. Baltimore 2005 – p.23/23