Overview
Sequential Monte Carlo Methods • Click to edit Master in Rtext styles • • • •
Second level Third level Fourth level Fifth level
Thomas Jakobsen Jeffrey Todd Lins Saxo Bank A/S
• • • • • • •
The • model Click to edit Master text styles An introduction to sequential Monte Carlo methods • Second level Algorithm & implementation • Third level volatility model A factor stochastic Example, artificial 2-factor model • Fourth level Analysis of forex data • Fifth level Conclusions
[email protected],
[email protected]
useR! 2006 Vienna, June 15-17, 2006
www.saxobank.com
useR! 2006 Vienna, June 15-17, 2006
The Model •
Sequential Monte Carlo Methods
Markovian, nonlinear, non-Gaussian state-space model:
x
x
…
xt −1
x
X: Unobserved variables
0 1 • Click to edit Master ttext styles Y: Observations θ: Parameters • ySecond level … y yt y1 0 • Third level Described by p( x ) • Fourth level Prior state distribution p( x | x , Θ) for t ≥ 1 Markovian state space model ( y | x , Θ) level for t ≥ 1 Observational model • pFifth
• Useful whento a (partially observed) state needs to be • Click edit Master text styles tracked or forecasted:
•
0
t −1
t
t
t
•
Observations arrive sequentially and are noisy.
•
Problem statement: – –
Estimate recursively in time the posterior distribution p(x|y,Θ). (”tracking the state”) Additionally: Estimate Θ.
useR! 2006 Vienna, June 15-17, 2006
• Second level • Third level • Fourth level Sequential Monte Carlo methods are algorithms for inference hidden state space models. • Fifthinlevel – Tracking problems (robots, vision, radar etc.) – Time series analysis (economical/financial data etc.) – General online inference
t −1
•
• Also known as particle filters, condensation, sampling importance resampling etc.
useR! 2006 Vienna, June 15-17, 2006
Sequential Monte Carlo Methods • SMC methods: a nonlinear, non-Gaussian • Click to Basically edit Master text styles version of the Kalman filter (but approximate – not closed form) • Second level
• posterior Third level • The at time t-1 is represented by a set of weighted particles. The particles are drawn i.i.d. and • Fourth level recursively updated. • Fifth level
• Next slide: Illustration of update
A bootstrap approach (from [1]) • • • • •
Unweighted measure
Click to edit Master text styles Compute importance Second level weights using info at time t-1 Third level Weighted measure Resampling Fourth level Fifth level Prediction Apprx. of p(x_t|y_1:t-1)
useR! 2006 Vienna, June 15-17, 2006
useR! 2006 Vienna, June 15-17, 2006
Comb. Parameter and State Estimation
Algorithm • Liu & West, Combined Parameter and State Estimation: (auxiliary particle filter with state estimation)
• Often the parameters are knowntext (or obtained • Click to edit Master styles through separate analysis). • •
• Second level However: If parameters are unknown, how to carry out • Third level of x and Θ? combined estimation • Fourth level Liu & West describe a simple approach. • Fifth level
• Click to edit Master text styles 1. For j = 1,..., N : µ = E ( x | x , Θ ) • Second level 2. Sample an integer k ∈ {1,..., N} with probability • Third g ∝ w level p( y | µ , m ) 3. Sample Θ ~ N (⋅ | m , h V ). • Fourth level 4. Sample x ~ p (⋅ | x , Θ ). • Fifth level p( y | x , Θ ) 5. Evaluate weight : w ∝
Input : Monte Carlo sample ( xt( j ) , Θ t( j ) ) and weights wt( j ) , j = 1,..., N . ( j) t +1
( j) t +1
( j) t
( j) t +1
t +1
(k ) t +1
(k ) t
t +1
(k ) t
(k ) t
(k ) t +1
( j) t
( j) t
( j) t
2
t
(k ) t +1
t +1
(k ) t +1 (k ) t +1
(k ) t +1 (k ) t
p( yt +1 | µ , m )
Repeat 2. - 5. until approximation is sufficiently accurate. ( j) m (j) + (1 − a) Θt t = aΘ t
Θt ,Vt : Posterior mean and variance matrix from Θ t( j ) and weights wt( j ) useR! 2006 Vienna, June 15-17, 2006
useR! 2006 Vienna, June 15-17, 2006
R Implementation • To describe the model, the user supplies his own functions as arguments to main SMC function (together with and [hyper]parameters). • YClick to edit Master text styles •
• Second level R language very suitable for implementation, especially • Thirdbecause level of – Vectorization Fourth level – •Built-in statistical functions – The possibility of supplying user-defined functions as arguments • Fifth level – Ease of visualization and interaction
Factor Stochastic Volatility Model • (similar to the model of Liu & West)
• • • • •
t
t
t
t
t
t
t
t
ti t
t1
tk
ti
t −1
t
t
• Quite efficient but still computationally heavy.
U
Innovations variance matrix
ε t ~ N (⋅ | 0, Ψ )
Idiosyncratic noise variances
Ψ = diag(Ψ1 ,..., Ψk )
– For large datasets, a C/C++ optimization is needed (we already developed a faster C# version).
• • • • •
Click to +edit Master text Observatio ns styles y = α + Xf ε Local series level α Second level Factor loadings matrix X ,H ) Factors f ~ N (⋅ | 0level Third Factor variances H = diag(h , Κ , h ) Fourth level h = exp( λ ) λ = µ + Φ( λ − µ ) + γ Log factor variances Fifth level γ ~ N (⋅ | 0, U) Innovations
useR! 2006 Vienna, June 15-17, 2006
useR! 2006 Vienna, June 15-17, 2006
Example, artificial 2-factor model
Example, artificial 2-factor model
Click to edit Master text styles Second level Third level Fourth level Fifth level
useR! 2006 Vienna, June 15-17, 2006
• • • • •
Click to edit Master text styles Second level Third level Fourth level Fifth level
useR! 2006 Vienna, June 15-17, 2006
Example, FX data • Model exchange rates with a factor stochastic volatility model. •
•
• Click to edit Master text styles • Second Per-minute datalevel – EURUSD, GBPUSD, JPYUSD, CHFUSD. • Third level • log Fourth The return level for currency i on day t is given by s • Fifth level y = log( ) ti
ti
FX data, example Spot rates. 434 bank days of data. Index 1.0 at 2004-10-01.
• • • • •
Click to edit Master text styles Second level Third level Fourth level Fifth level
st −1,i
where s is the spot rate in US dollars.
EURUSD GBPUSD JPYUSD CHFUSD
useR! 2006 Vienna, June 15-17, 2006
useR! 2006 Vienna, June 15-17, 2006
FX data, example
FX data, results
Log return, log (s(t)/s(t-1)) (50 data points)
• • • • •
Click to edit Master text styles Second level Third level Fourth level Fifth level
• Click to edit Master text styles • Second level Log returns • Third level • Fourth level • Fifth level Volatility factor 1
EURUSD GBPUSD JPYUSD CHFUSD useR! 2006 Vienna, June 15-17, 2006
Volatility factor 2
useR! 2006 Vienna, June 15-17, 2006
Conclusion • R is flexible and powerful enough for implementing efficient particle filters •
•
• Click to edit Master text styles For large datasets, however, an optimized C/C++ • Second version is really level needed (because of the heavy computational burden). • Third level Combined parameter • Fourth level and state estimation can be useful but also unstable when there are too many parameters – •Alternative: Do separate/offline estimation of parameters (using, Fifth level e.g., full MCMC)
References [1] A.•Doucet, Freitas and N.text Gordon, editors, Click N. todeedit Master styles Sequential Monte Carlo Methods in Practice, Springer, 2001. • Second level
Third level [2] Liu• and West: Combined Parameter and State Estimation in Simulation-Based Filtering, pp. 197-223, in [1].• Fourth level • Fifth level
• Package may be forthcoming
useR! 2006 Vienna, June 15-17, 2006
Questions? • Clickfor to your edit Master text styles • Thanks attention. • Second level • Third level • Emails: »
[email protected] • Fourth level »
[email protected] • Fifth level • Web site: » www.saxobank.com
useR! 2006 Vienna, June 15-17, 2006
useR! 2006 Vienna, June 15-17, 2006
