A full Bayesian approach to generalized maximum ... - Semantic Scholar

Stoch Environ Res Risk Assess (2010) 24:761–770 DOI 10.1007/s00477-009-0362-7

ORIGINAL PAPER

A full Bayesian approach to generalized maximum likelihood estimation of generalized extreme value distribution Seonkyoo Yoon • Woncheol Cho • Jun-Haeng Heo Chul Eung Kim

•

Published online: 24 December 2009 Springer-Verlag 2009

Abstract This study develops a full Bayesian GEV distribution estimation method (BAYBETA), which contains a semi-Bayesian framework of generalized maximum likelihood estimator (GMLE), to make full use of several advantages of the Bayesian approach especially in uncertainty analysis. For the full Bayesian framework, the optimal hyperparameter of beta prior distribution on the shape parameter of the GEV distribution is found as (6.4990, 8.7927) through simulation-based analysis. In a performance comparison analysis, the performances of BAYBETA, which adopts beta(6.4990, 8.7927) as prior density on the shape parameter of the GEV distribution, are almost the same as or slightly better than GML, outperforming MOM, ML, and LM in terms of root mean square error (RMSE) and bias when the shape parameter is negative. Also, a case study of two hydrologic extreme value data shows that the traditional uncertainty analysis using asymptotic approximation of ML and GML has limitations in describing the uncertainty in high upper quantiles, while the proposed full Bayesian estimation method BAYBETA

S. Yoon (&) Water Resources Division, Korea Institute of Construction Technology, Simindae-Ro, Ilsanseo-Gu, Goyang-Si, Kyeonggi-Do, South Korea e-mail: [email protected] W. Cho J.-H. Heo School of Civil and Environmental Engineering, Yonsei University, Seoul 120-749, South Korea C. E. Kim Department of Applied Statistics, Yonsei University, Seoul 120-749, South Korea

provides a consistent and complete description of the uncertainty. Keywords GEV distribution Generalized maximum likelihood estimator Bayesian analysis Beta distribution

1 Introduction The generalized extreme value (GEV) distribution, introduced by Jenkinson (1955), has been frequently and widely applied for the modeling of extreme hydrologic events, such as flood flow (NERC 1975), extreme rainfall depth (Coles and Tawn 1996), coastal water level (Huang et al. 2008), extreme temperature (Siliverstovs et al. 2009), and wind speed (Walshaw 2000). For the estimation of parameters and quantile of the GEV distribution, the maximum likelihood estimation (ML), the L-moments estimation (LM), the probability weighted moments estimation (PWM), and the method of moments (MOM) have been used. Hosking et al. (1985) and Madsen et al. (1997) compared these estimation techniques. Martins and Stedinger (2000) showed that, for a small sample size, ML sometimes absurdly under-estimates shape parameter j and so causes a large bias and RMSE in extreme upper quantile estimators. This is consistent with the results of Hosking et al. (1985) in which unstable behavior of maximum likelihood quantile estimator was reported for the GEV model when the sample size is small. Coles and Dixon (1999) suggested that the small-sample problems can be improved by using a penalized likelihood method. For the same purpose, Martins and Stedinger (2000) proposed generalized maximum likelihood estimation (GML). The generalized maximum likelihood estimators (GMLE) are

123

762

obtained by maximizing the generalized likelihood function derived from the semi-Bayesian framework which adopts beta prior distribution on the shape parameter of the GEV distribution. They showed that the GML quantile estimator outperformed MOM, ML, and LM in the range of -0.35 B j B 0, which is an rightly heavy-tailed case and the most likely range in hydrological practice. Park (2005) introduced a simulation-based systematic way of selecting hyperparameters of the beta prior distribution adopted in the GMLE. He found that the hyperparameter, (6, 9), used in the GML quantile estimator enables the most precise estimation in the range of -0.35 B j B 0.05 which coincides with the most likely range in hydrological practice indicated by Martins and Stedinger (2000). However, even though it employs a Bayesian framework, and consequently attains stability for a small sample size, GML cannot make full use of the several advantages of the Bayesian approach, especially in uncertainty analysis. This is because GML is based on the maximum likelihood method and regards only the shape parameter as a random variable, while regarding other parameters as constant. Coles and Pericchi (2003), Coles et al. (2003), and Coles (2004) argued that, through the analysis of Venezuelan rainfall data of which the 1999 rainfall caused devastating catastrophes, the proper consideration of uncertainty of the extreme value model is an essential and crucial element for anticipating such catastrophes. They demonstrated the suitability of a full Bayesian framework for uncertainty analysis, which considers all parameters as random variables and so enables an implicit allowance for parameter uncertainty. Reis Jr. and Stedinger (2005) pointed out that the conventional ML method has limitations in managing the uncertainty because it resorts to crude asymptotic approximation, and showed that Bayesian methods provide a full and complete description of the uncertainty in parameters and quantiles. To make full use of the Bayesian framework together with the stability of GML for a small sample size, this study develops a full Bayesian estimation method of the GEV distribution (BAYBETA) which contains a semiBayesian framework of GMLE and considers not only the shape parameter but also other parameters (location and scale parameter) as random variables. It can be achieved through the adoption of near-flat priors for location and scale parameters while the beta prior on shape parameter remained. However, it is necessary to reassess the hyperparameter of the beta prior due to the conceptual difference between the BAYBETA and GML methods. To be specific, in a full Bayesian framework, both parameter uncertainty and randomness in future observations are reflected in the estimation by regarding all parameters as random variables, while GML cannot do this (Coles 2001). Furthermore, the

123

Stoch Environ Res Risk Assess (2010) 24:761–770

Bayesian approach found the posterior mean as the parameter estimate that minimizes expected quadratic loss, while the GML parameter estimator corresponds to the posterior mode. This means that the hyperparameter of beta prior adopted in GMLE may not be an optimal value in the full Bayesian framework. Therefore, the objective of this study is to find the optimal hyperparameter for the full Bayesian framework containing the semi-Bayesian framework of GMLE, and to analyze the suitability of the derived full Bayesian framework for hydrologic extreme value modeling, in comparison with the existing estimation techniques. The paper is structured as follows. In Sect. 2, the GEV distribution and its existing estimation techniques including ML and GML, to be compared with the full Bayesian method, are discussed. In Sect. 3, the full Bayesian approach to GMLE and the process of finding the optimal hyperparameter of the beta prior are presented. In Sect. 4, the derived full Bayesian framework is tested in comparison with the existing estimation techniques in view of hydrologic extreme value modeling. Finally, conclusions and recommendations are presented.

2 Generalized extreme value distribution The extreme value distributions such as the Gumbel, Frechet, and Weibull are derived from asymptotic analysis (Fisher and Tippett 1928). Therefore, unlike other distributions such as gamma or log-normal, which are also used to model hydrologic extreme events, the extreme value distributions have theoretical justification for stochastic behavior of extreme values. It is a straightforward process to check that these families can be combined into a single family having a cumulative distribution function as follows (Jenkinson 1955) ( ) ðx nÞ 1=j F ð xÞ ¼ exp 1 j j 6¼ 0 a ð1Þ ð x nÞ j¼0 ¼ exp exp a where {(n, a, j):n [ R, a [ 0, j [ R} and 1 - j(x - n)/ a [ 0. Here, n, a, and j are the location, scale, and shape parameters, respectively. 2.1 Maximum likelihood estimation and asymptotic approximation For given observations x = {x1, x2,…,xn}which are identically distributed from a GEV distribution, the log-likelihood function of the GEV distribution is given as follows (Hosking 1985)


763

h i X 1 1 lnðyi Þ ðyi Þ1=j ln L hjx ¼ n lnðaÞ þ j ð2Þ where h = (n, a, j) and yi = [1 - (j/a)(xi - n)]. The parameters (n, a, j) are estimated by solving the partial derivatives of the log-likelihood function. This is usually conducted by using the Newton-Raphson method. Hosking (1985) suggested a computer algorithm which solves the differential equations by the Newton-Raphson method. The maximum likelihood estimator of quantile xp, for nonexceedance probability p, is obtained as i ^ ah x^p ¼ ^ n þ 1 ð lnð pÞÞj^ ð3Þ ^ j where this quantile estimator is an inversion of the GEV distribution. The maximum likelihood method can easily calculate the standard error and confidence interval of parameters by the asymptotic normality of the maximum likelihood estimator. Also, the standard error and confidence interval of quantiles can be estimated through the delta method which enables the asymptotic normality of the quantile estimator (Coles 2001).

Martins and Stedinger (2000) proposed a generalized maximum likelihood estimator to prevent the estimate of j from being a severe negative value which causes a large bias and variance of extreme upper quantile estimators. They showed that the GML quantile estimator outperformed MOM, ML, and LM for a small sample size in the range of -0.35 B j B 0, which is an rightly heavy-tailed case and the most likely range in hydrological practice. GML estimation is based on the semi-Bayesian framework which adopts beta prior distribution only for the shape parameter of the GEV distribution. The beta prior distribution for j which Martins and Stedinger (2000) suggested is pðjÞ ¼

ð4Þ

where B(u, v) = C(a)C(b)/C(a ? b). The mean and variance of the beta distribution are E(j) = -0.1 and Var(j) = 0.015, respectively. GML estimation is carried out in the same way as ML. The generalized likelihood function which is derived from the semi-Bayesian framework is GLðn; a; jjxÞ ¼ Lðn; a; jjxÞpðjÞ Therefore, the generalized log-likelihood function is

ð6Þ

The Newton-Raphson method can also be used to estimate the parameters maximizing the generalized likelihood function. The confidence intervals of the parameters and quantile are also easily calculated based on the asymptotic normality of the maximum likelihood estimator. Note that the GML parameter estimator corresponds to the posterior mode in terms of Bayesian analysis. Park (2005) introduced a simulation-based systematic way of selecting hyperparameters of beta prior distribution adopted in a GML estimator. He confirmed that the hyperparameter (6, 9) used in the GML quantile estimator enables the most precise estimation in the range of -0.35 B j B 0.05 which coincides with the most likely range in hydrological practice indicated by Martins and Stedinger (2000).

3 Full Bayesian approach to GMLE 3.1 Full Bayesian framework with Markov Chain Monte Carlo (MCMC) method

2.2 Generalized maximum likelihood estimation

ð0:5 þ jÞu1 ð0:5 jÞv1 ; 0:5 j 0:5; Bðu; vÞ u ¼ 6; v ¼ 9

ln½Lðn; a; jjxÞ ¼ n lnðaÞ n X 1 1=j 1 lnðyi Þ ðyi Þ þ j i¼1 þ lnfpðjÞg

ð5Þ

In hydrologic extreme value analysis, it is very important to take uncertainty into account in the extreme quantile estimates because they have inevitably large sampling variability, and having no consideration for such variability can cause devastating catastrophes (Coles and Pericchi 2003). So, in the classical approach, confidence intervals are generally added to estimates of extreme quantiles. Although the ML method provides a straightforward way of doing so, it may not be able to sufficiently account for uncertainty unless the sample size is large because it is based on an asymptotic normality approximation of the maximum likelihood estimator (Coles 2001; Casella and Berger 2001; Katz et al. 2002). Furthermore, there are difficulties of interpretation in such a presentation of results. On the contrary, any approximation that accounts for the uncertainty is unnecessary in the Bayesian inference because the Bayesian framework derives a full posterior distribution of the parameters, which enables the uncertainty in parameter and quantiles to be completely described. Therefore, the Bayesian method has been constantly suggested in hydrologic field studies (Kuczera and Parent 1998; Coles 2004; Coles and Pericchi 2003; Coles et al. 2003; Reis Jr. and Stedinger 2005; Renard et al. 2006a, b; Sisson et al. 2006; Ribatet et al. 2007a, b; Smith and Marshall 2008; Lee and Kim 2008).

123

764


For given observations x = {x1, x2,…, xn}, if the prior distribution is denoted by p(h) and the likelihood function as L(h|x), Bayes’ theorem states that f ðhjxÞ ¼ R

LðhjxÞ pðhÞ LðhjxÞ pðhÞdh

ð7Þ

where the integral term of the denominator is the marginal distribution of x. The posterior distribution is now used to make statements about parameter h = (n, a, j), which is still considered a random variable. For example, the mean of the posterior distribution is used as point estimates of h and the certain probability interval (credible interval) of the posterior distribution as interval estimates of h. For the computation of the posterior distribution from the Bayesian framework, the Markov Chain Monte Carlo (MCMC) method must be used owing to unavoidable difficulties in calculation which are due to the integral in Eq. 7. The Metropolis-Hastings algorithm is used in this study, which is one of the widely used MCMC methods (Gelman et al. 2004). It generates a sample from the posterior distribution without calculating the integral in Eq. 7, so that any desired feature of the posterior distribution can be accurately summarized (Bates and Campbell 2001). Prediction is also easy in the Bayesian approach. If y is a future observation with the GEV distribution f(y|n, a, j), then the posterior predictive density of a future y, given observations x = {x1, x2,…,xn}, is given by Z ð8Þ f ðyjxÞ ¼ f ðyjn; a; kÞpðn; a; jjxÞdndadj Compared to other approaches to prediction, the predictive density has the advantage that it reflects uncertainty in the model through p(n, a, j|y) and uncertainty due to variability in future observations through f(y|n, a, j) (Coles 2001; Beirlant et al. 2004). Unlike the Bayesian approach, the GML method considers only the shape parameter of three parameters as a random variable, so it adopts prior distribution only for itself. This means that GML conceptually differs from the full Bayesian approach which considers all parameters of a model as random variables. Therefore, GML cannot derive posterior distribution and so must depend on asymptotic approximation to describe the uncertainty of the model. This study applies the full Bayesian framework to GMLE in order to unify the benefits of both approaches; stability of GMLE in a small sample size and suitability of Bayesian inference for quantifying the uncertainty. The next section describes how to define the prior distribution for this.

physical intuition, can be included in the model by constructing appropriate prior distribution. In some cases it may be valid to utilize this device to include genuine knowledge about the process under modeling. More commonly, it is difficult to obtain such knowledge, and vague distributions which have large variance are then adopted to reflect this prior ignorance. The Bayesian framework adopting the vague prior distributions can be told that it is essentially formal, but it leads to an inference in which parameter uncertainty is properly formalized and for which inferences on predictions are naturally handled (Coles et al. 2003). Therefore, this study adopts independent zero-mean normal prior distributions on n and log (a) with variances of 104. The beta prior distribution on j in GMLE’s semiBayesian framework remains. All of these can be summarized as follows. 1 n2 fn ðnÞ ¼ pffiffiffiffiffiffi exp 4 ð9Þ 10 2p 102 1 /2 f/ ð/Þ ¼ pffiffiffiffiffiffi exp 4 ; where / ¼ log a ð10Þ 10 2p 102 fj ðjÞ ¼

ð0:5 þ jÞu1 ð0:5 jÞv1 ; Bðu; vÞ

0:5 k 0:5

pðn; /; jÞ ¼ fn ðnÞf/ ð/Þfj ðjÞ

ð11Þ ð12Þ

It was found that outcomes from this framework are unchangeable under changes of variances in the above equations, implying the robustness of the prior specification. The hyperparameter of the beta prior distribution on j in GMLE is (u, v) = (6, 9) which was found by Park (2005) to be best choice for GMLE’s semi-Bayesian framework. However, there is a contrast between the GML estimator and predictive Bayesian values due to the implicit allowance for parameter uncertainty in the full Bayesian estimates. Also, the GML parameter estimator corresponds to the posterior mode, but the posterior mean is found to be the parameter estimate that minimizes expected quadratic loss in the full Bayesian approach. These conceptual differences imply that the hyperparameter (u, v) = (6, 9) of beta prior, although it is the best choice for GMLE’s semi-Bayesian framework, can be inadequate for the full Bayesian framework, and may need to be adjusted. Thus, a simulation-based systematic method was carried out in this study, which is similar to the process suggested by Park (2005), to find the optimal hyperparameter (u, v) of beta prior for the full Bayesian framework. 3.3 Optimal selection of hyperparameters

3.2 Prior distribution setting

3.3.1 Experiment setting

Prior knowledge about the parameters, which may come from other data sets or from a modeler’s experience and

The purpose of this simulation-based experiment is to find the best hyperparameter (u, v) for a full Bayesian

123


765

framework which contains the semi-Bayesian framework of the GML approach. The experiment changes the mean of prior density beta(u, v) while the variance of the prior is fixed at 0.015, which is of beta(6, 9) in GMLE, in order to maintain the degree of informativeness of the beta prior. The range of change of the mean is (-0.130, -0.060). This is summarized in Table 1 with the corresponding value of (u, v). The criterion of comparison, which is suggested by Park (2005), is defined by 8vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi93 2 u Nn X X X