generic resampling particle filter joint parameter estimation for electricity prices with jump diffusion A
Bahri Uzunoglu Upsala University, Gotland Campus, Department of Energy Technology Cramergatan 3, 621 57 Visby, Sweden Email:
[email protected]
Abstract-In this paper, a particle filter for parameter estima tion of jump diffusion models employed for modeling electricity prices [I], [2], [3] is implemented. A jump-diffusion model [4] is investigated. The jumps have the possibility to give a better explanation of the behavior of electricity prices [5]. Introduction of the jump components however complicates parameter estimation problem by the inclusion of several new parameters [4]. These parameters will describe the jump fre quency and distribution. The jump models are non-Gaussian and this increases the complexity of the models further, [4], [5]. A known filtering technique for these models is particle filter 111, [2], [3]. The performance of generic particle filter to model the jump frequency and distribution parameters has been investigated. In this paper, a preliminary study is conducted. The performance of augmented generic particle filter to model the jump frequency and distribution parameters has been analyzed for a benchmark example employed in the maximum likelihood state estimator solution of [4] and favorable results were obtained. The results are compared with bench-marking closed form solution of 141 in order to once again to highlight the contribution of the paper.
I.
INTRODUCTION
Non-linear filters are important tools for managing the uncertainty in electricity markets. Operational decisions such as unit commitments by better market forecasting can be im proved which directly impact electricity market prices. There has been a series of previous investigation by the researcher which has created the necessary ground work and know-how for this research [1], [2], [3], [6], [7], [4], [5]. A wide spread of research projects continue to focus on improving the accuracy and the robustness of electricity mar ket predictions [4], [5]. The objective is to improve the time horizon and improve their performance. One such tool that is of great importance is a non-linear filtering tools. The non linear filtering or data assimilation research is focused on making the best use of observations using advanced variational and ensemble data assimilation techniques [1], [2], [3], [7]. Kalman filtering techniques have been widely used to esti mate the parameters of the linear Gaussian systems [8], [9]. Unscented and Extended Kalman filter techniques widens the spectrum of estimation tools to that of nonlinear and non gaussian systems [8], [9]. However, these estimation tools have This research was sponsored by grant number 17781 nonlinear fi Iters for electricity markets, forecasting and grid integration.
Dervis Bayazit Federal Home Loan Bank of Atlanta, Financial Risk Modeling 1475 Peachtree Street, N.E., Atlanta GA 30309 Email:
[email protected]
their own limitations [8], [9] and hence they are not preferred when the underlying system is especially non-Gaussian and nonlinear. Another method that can be employed, is expectation maxi mization approach [10],[11]. To implement the EM algorithm for a general nonlinear state-space, the sequential Bayesian Monte Carlo methods that is particle filters can be employed [10]. Particle filter is general method that can be used for a general nonlinear state-space system parameter estimation problems. The growing complexity of energy markets requires the introduction of increasingly sophisticated tools [7], [4], [5]. The analysis of spot market and forward prices are getting more demanding. In order for market participants to use these markets in an efficient way, it is important to employ good mathematical models of these markets [5], [7], [4]. This has proved to be particularly difficult for electricity, where markets are complex, and exhibit a number of unique features, mainly due to the problems involved in storing electricity. The stochastic differential equation models of the price dynamics are going to be the main focus as reviewed in the next section. Using the non-linear filters, based on spot prices, the ob jective of this paper is to successfully estimate parameters for simulated market data for the stochastic differential equation models of the spot price dynamics. The introduction of jumps will have the flexibility to give a better explanation of the behaviour of electricity prices however it brings its own complication on parameter estimation which is addressed by nonlinear filters [1], [2], [3]. II.
THE REL ATIONSHIP BETW EEN SPOT AND FORWARD PRICES AND STOCH ASTIC MODELS
The stochastic models of derivatives have been employed in electricity markets with the main motivation of derivative evaluation concepts. These derivative concepts are derived from financial models for electricity price modelling. Standard financial reasoning of risk premium which is the gain for having a risky investment, forward/future price and spot price have been implemented for electricity markets [12]. Risk premium is the difference between expected spot price and the price of futures or forward price [12], [5], [4]. The relationship between spot price and forward price is different
is reflected by a change of drift in the process. If the phys ical measure P is used instead of risk-neutral measure, the relationship between forward and futures can be defined as
from the financial and commodity markets. Spot price is the best estimate of going rate of electricity at some specific time in the future. [n the future, the forward price is the price that is traded for a price in the future by a trader. The link that defines spot price and future price has been scrutinized by some authors based on the argument that that relation between spot and future prices cannot be established through the no arbitrage argument of convenience yield as a result of non storability of electricity [12], [5], [4]. For simplicity only forward contracts, not swaps or other financial products will be emphasized. Here the interest rate is kept constant so that forward and future prices will coincide which is defined as forward for the rest of this work. Suppose S(t) is a stochastic process defining the price dynamics of the spot and r > 0 is the constant risk-free interest rate. If a forward contract is defined as f (t,T) where the T is delivery date. The premium or payoff from this position will be" S(T) - f (t,T)" at delivery time T. The value of a derivative is given as the present expected value of its pay off [5], [7], [4]. [f the expectation lEQ is taken with respect to a risk-neutral probability Q which means that there is a market free of arbitrage and if the interest rate is taken as constant then for a martingale process which is a sequence of random variables with no knowledge of past events, the forward contract contract under risk neutral measure will be
o
f(t,T)
The above argument formulates mathematically that the forward price is the best prediction of the spot price [12], [5], [4]. This is called rational expectation hypothesis that is equivalent to having a physical measure Q P. In reality, it can expected and argued that the above relationship will not hold. For the rest of this work, parameter estimation on forward data will be not employed. The above arguments which are used to explain the risk premium and context of this work will not be employed for the rest of this work. The spot price data will be used parameter estimation so the argument that spot price explains the market will be used in this work. =
Spot price based models based on stochastic differential equations
A.
Mean reversion is typical feature of electricity markets as a result Black Scholes equation is not suitable for com modities which does not address mean reversion. Schwartz model of commodity markets allows mean reversion process. This model extends to Omstein-Uhlenbeck process. Omstein Uhlenbeck process is one of the basic processes that can employ the mean-reverting functionality that extends Wiener process or Brownian motion to mean reversion [12], [5], [4]. Omstein-Uhlenbeck process for spot prices or log spot have been employed for different markets such as Nordpool electricity market and Alberta electricity market and this process has found many applications in financial time series. Arithmetic Ornstein-Uhlenbeck specifies a continuous time process similar to the below. 1) Mean reversion diffusion process:
(1)
where Ft is the filtration containing all market information up to time t. [f the forward price is adapted to this filtration given by Ft, we will have
f(t,T)
(2)
since the forward price which is set at time t cannot include more information about the market than given by the filtration Ft. With respect to a risk-neutral probability Q, this will lead to well-known connection between a forward contract and underlying spot in a market where the two assets can be traded as defined below
f(t,T)
t
S(t) er(T- ).
(4)
(5) where St is the logarithm of the electricity price at time t. So is the initial condition and � is the mean reversion process. Mean reversion process decides how fast the process go back to the long-term mean level Ct. If the same equation is written in in tegral form, a first order autoregressive time can be discretized as order one AR(1). Herein Wt is a standard Brownian motion with dWt N(0,dt) and dt is infinitesimal time interval [4]. The unknown constants of this stochastic differential equation are 81 [�Ct (J2] of the tuple 81. There has been many extension to this model that move away from this basic model by implementing time dependent implementations of Ct(t) or by implementing deterministic supply function that account for nonlinear price spikes, volatility stochastic formulations or other stochastic processes which will not be investigated here [4]. 2) Price jump processes and spikes: Mean reversion pro cess of Omstein-Uhlenbeck process is not good enough how ever to model price spikes. In earlier investigations several models that combine jump and mean-reversion have been
(3)
rv
The convenience yield is not included which a discount for the interest rates for simplicity of the discussion. This is to keep the focus of work to spot price parameter estimation which is the objective of this study. The above processes belong to semi-martingale processes under the so called risk neutral probability measures [[2], [5], [4]. Semi-martingales are martingales that have a stopping time which localize the stochastic process. The above relationships under change measure to risk neutral process using Girsanov theorem creates a martingale instead of the real process under the physical measure which
=
2
Electricity Price Simulation for Alberta Canada daily
employed for electricity spot price processes. A jump process can defined in the following form
80 70 00
(6)
[ 60
where Pt is a discontinuous, one dimensional standard Poisson process [12], [5], [4]. This will redefine the mean reversion diffusion process with the jump term.
.� 50
1il �
40
20
(7) The arrival rate for this process is w. As the arrival time 1 if there is a jump, dPt 0 for the no jump case. The amplitude distribution of Qt is exponentially distributed with mean f. The sign of the jump Qt is a Bernoulli random variable distribution with parameter 1jJ. The unknown constants of this stochastic differential equation are 82 [w '1jJ ,] of the tuple 82. These jump models can be extended to other models with asymmetric up and down jump models which will not be investigated in this study. Markov regime switching processes are another set of tools to define price jumps and spikes as opposed to jump processes [13]. A Markov process can be defined for several regimes from St,l to St,2. As an example, St,l can be a regime with base dynamics given by Ornstein-Uhlenbeck St,l St and the dynamics of the process will be given by the following stochastic differential equation,
dPt
=
Days
=
Fig. 1.
=
the relation between spot and futures prices. This limitation can be improved if changes in spot prices are implemented to depend on more than one factor. As the number of factors (stochastic/deterministic) increases in a model, it is difficult to obtain analytic solution that can be used for bench-marking purposes. In this study, however the focus is on spot price mean-reverting jump process diffusion model where closed form bench-marking tools are available for parameter estima tion problem that is being studied.
=
'" (0; - St,I) dt + (}dWt·
Daily electricity prices for Alberta simulated.
4) Single-factor Mean-reverting jump diffusion process for spot prices: If jump process and mean-reversion diffusion are implemented together to model electricity prices, the following basic model will be a single-factor Levy process [12], [5], [4]. Levy process is the extension of mean-reverting diffusion process with jump increments for spot prices
(8)
For each regime a separate and independent different under lying price process is modeled where the regime switching is defined by a random variable and because of the Markov property, the current state depends only on immediate past. There is no conclusive results on the advantage of Markov regime switching models in comparison to jump-diffusion models in terms of statistical properties [13]. The parameter estimation of Markov regime switching is less developed since the regime is latent [13]. 3) Single factor and multi factor models: Single factor or multi factors can be used for spot price modelling. Spot price in these models are either itself a Markov process as in in the single factor or is a function of a multidimensional Markov process for multi-factor models. Single factor models have comparatively few parameters. These models are limited in addressing the relation between spot and future prices. In this study only spot price modelling is addressed with single factor models. Future studies will be extended to multi-factor models for comparison [12], [5], [4]. One such multi-factor model is stochastic volatility model. Jumps alone might not cover all the complexity in electricity prices since volatility might show probabilistic features [14], [5]. To capture volatility, various two-factor stochastic volatil ity models are available in the literature [14], [5]. The single factor models have relatively limited parameter space. This limitation can cause loss of accuracy in explaining
(9) This is an Ornstein-Uhlenbeck process driven by independent increment process. The unknown parameters of this stochastic differential equation are 8 ['" 0; (}2 W 1jJ I] ' The question of estimating such models on data is not easy even for one factor model when we try to fit the stochastic model to Spot data [4]. For multi-factor models, these can be even more challenging problem involving highly sophisticated techniques [12], [5], [4]. We will limit our investigations to above single-factor model to concentrate on the parameter estimation problem. The simulation of this model can be seen in the following Figure 1 [12], [5], [4]. =
III. PARTICLE FILTER It is common in science to have measurements of the data of dynamic system. However, even though data is observed this may not be enough to understand the output of the system and its driving forces. In most cases, underlying driving state system is not observed and it has to be extracted from the noisy observations. Particle filtering is an accepted method of extraction of latent state variable from the noisy measurements. 3
In order to analyze a dynamic system it is required to have models for underlying state system and noisy measurements. We will assume that these models are known in their proba bilistic forms: Consider the state sequence {Xk' kEN} .
fk(Xk-l, rlk-d hk(Xk-l, vk-d, fk
In the resampling stage, particles with large weights may be selected many times resulting to particle collapse, [15]. This causes impoverishment of the particle set due to sampling from discrete distribution rather than a continuous one. This implies that sampling at every stage has to be avoided. This leads to the concept of effective sample size,
(10) (11)
-------
Neff=
Rn, x Rnn --+ Rn" and hk : Rn, x Rn" --+ Rn, are functions that may be nonlinear, and {rlk' k E N} and {Vk-l, kEN} are i.i.d state system and measurement noises. The core idea of filtering lies on updating prior samples of the state system in light of a new measurement. This follows from Bayesian filter approach: :
p(XkIZ1k : -d=
/ p(xklxk-dp(Xk-lIZ1k: -ddxk-l'
Algorithm 1 Resampling Algorithm . . . . . N [{xl,;, wLiJ}jN�ll= RESAMPLE[{xL wl,;};�\l Initialize the CDF: C1= 0 fori= 2 to Ns do Construct CDF: Ci= Ci-l+ wl,; end for Start at the bottom of the CDF:i= 1 Draw a starting point: U1 lU[O, N;ll for j 1 to Ns do Move along the CDF: Uj= Ul+ Ns-l(j 1) while Uj > Ci do i=i+l end while Assign sample: x1*= xk Assign weight: w�= Ns-1 j Assign parent:i =i end for
(13)
where the normalizing denominator
p(zkIZ1:k-d=
/ p(zklxk)P(Xklz1k: -ddxk.
A particle filter can be defined in three steps
(19)
i L..,i=l(wk)2
which is a convenient measure of degeneracy. It varies between 1 and Ns . A value close to 1 indicates a sample collapse. On the other hand, if the value is close to Ns, particles in the sample ar�read homogeneously. Resampling has to be performed if Neff is less than an empirically chosen threshold NT. We use Algorithms 1 and 2 as their given form in [15].
(12)
p(zklxk)P(XkIZ1k : -d , P(XkIZl:k)= p(zkIZ1k : -d
1
",N,
(14)
�l
rv
1) Generate Ns random samples, {xl,;_ d from poste rior pdf p(Xk-lIZ1k : -d 1 by 2) Predict state, {xl,;} �l of the system at time k using (10). Practically, this is sampling from prior pdf
=
-
-
p(xklz1k : -d·
3) Update prior sample {xl,;} �l given the measurement Zk. First calculate a weight for the particle by using the likelihood density function and the weight calculated in the previous step, [15]: (15)
Algorithm 2 Generic Particle Filter fori= 1 : Ns do Draw xk from (10). Calculate the weight wl,; using (15) and map it to the
for a given prior sample point and a measurement. Then, normalize each weight so that they sum up to unity, i.e,
i Wk=
Ai Wk . N 2::j=,,1tU�
(16)
particle.
end for
Resample (with replacement) prior particles to obtain a new set of particles {xl,; *} such that
�1
Pr{xl,;*=
xU= w� for alli, j
Calculate normalization factor:
fori= 1 : Ns do
(17)
= 2::[::;\ wk
t
Calculate normalized weights:
end for
This new set of particles is a sample from the posterior pdf p(XkIZ1:k). The proof of the last statement can be found in [16].
Calculate
-------
if Neff
