The Role of Jackknife and Bootstrap in. Statistics of Extremes: an Environmental. Application. M. Ivette Gomes. Universidade de Lisboa, Portugal. 1 ...
The Role of Jackknife and Bootstrap in Statistics of Extremes: an Environmental Application M. Ivette Gomes Universidade de Lisboa, Portugal
1
1. OUTLINE • Resampling methodologies have recently revealed to be very fruitful in the field of statistics of extremes. • We mention the importance of – the Generalized Jackknife and – the Bootstrap in the obtention of a reliable semi-parametric estimate of any parameter of extreme or rare events, like a high quantile, the expected shortfall, the return period of a high level or the primary parameter of extreme events, the extreme value index (EVI). • In order to illustrate such topics, we shall consider an application related to minimum-variance reduced-bias (MVRB) estimators of a positive EVI in the field of Environment. 2
2. EXTREME VALUE THEORY (EVT) – A BRIEF INTRODUCTION 2.1. The extreme value index (EVI) • We use the notation γ for the EVI, the shape parameter in the Extreme Value d.f., (
EVγ (x) =
exp(−(1 + γx)−1/γ ), 1 + γx > 0 if γ = 6 0 exp(− exp(−x), x ∈ R if γ = 0,
and we now consider models with a heavy right-tail, i.e. F := 1 − F ∈ RV−1/γ ,
for some
γ > 0,
where the notation RVα stands for the class of regularly-varying functions with an index α, i.e., positive measurable functions g(·) such that ∀x > 0, g(tx)/g(t) → xα, as t → ∞. 3
The extreme value index γ measures essentially the weight of the right tail F = 1 − F : • If γ < 0, the right tail is light, and F has a finite right endpoint (xF := sup{x : F (x) < 1} < +∞); • If γ > 0, the right tail is heavy, of a negative polynomial type, and F has an infinite right endpoint; • If γ = 0, the right tail is of an exponential type. endpoint can then be either finite or infinite.
The right
4
0.1
0.8
g0 (x)
0.6
g1.5 (x)
! (x) g!0.5 (x)
0
g!0.5 (x) 2
0.4
x 3
4
5
0.2
! (x)
g0 (x)
g1.5 (x)
x
0 -3
-2
-1
0
1
2
3
4
5
P.d.f. gγ (x) = dEVγ (x)/dx, for γ = −0.5, γ = 0 and γ = 1.5, together with the √ normal p.d.f., φ(x) = exp(−x2 /2)/ 2π, x ∈ R. 5
2.2. First and higher-order frameworks • If F ∈ RV−1/γ , γ > 0, then [Gnedenko, 1943, AM], F is in the domain of attraction for maxima of a Fr´ echet-type Extreme Value d.f., and we write F ∈ DM(EVγ>0) =: DM+. • In this same context of heavy right-tails [de Haan, 1984], and with the notation U (t) = F ←(1 − 1/t), t ≥ 1, with F ←(y) = inf{x : F (x) ≥ y} the generalized inverse function of the underlying model F , we can further say that F ∈ DM +
⇐⇒
F ∈ RV−1/γ
⇐⇒
U ∈ RVγ ,
the so-called first-order conditions. 6
+ • For consistent semi-parametric EVI-estimation, in DM , we merely need to work with adequate functionals, dependent on an intermediate tuning parameter k, related to the number of top order statistics (OS) involved in the estimation, i.e.
k = kn → ∞
and
kn = o(n), as n → ∞.
• To obtain full information on the non-degenerate asymptotic behaviour of semi-parametric EVI-estimators, we need further assuming a second-order condition, ruling the rate of convergence in the first-order condition, or even a third-order condition. For technical simplicity, we consider a Pareto-type class of models, assuming that, as t → ∞, γ 2ρ U (t) = Ct 1 + A(t)/ρ + o(t ) ,
A(t) =: γβtρ, ρ < 0. 7
3. EVI-ESTIMATORS 3.1. Classical EVI-estimators + , and with the notation (X1:n ≤ · · · ≤ Xn:n) • For models in DM standing for the ascending OS associated with the available sample (X1, . . . , Xn), the classical EVI-estimators are the Hill estimators [Hill, 1975, AS], averages of the log-excesses, Vik := ln Xn−i+1:n − ln Xn−k:n,
1 ≤ i ≤ k < n,
i.e., Hn(k) ≡ H(k) := 1 k
k X
Vik ,
1 ≤ k < n.
i=1
• But these EVI-estimators have often a strong asymptotic bias for moderate up to large k, of the order of A(n/k), and the adequate accommodation of this bias has recently been extensively addressed. 8
3.2. Second-order reduced-bias (SORB) EVI-estimators • We mention the pioneering papers by Peng (1998) [SN], Beirlant, Dierckx, Goegebeur and Matthys (1999) [Extremes], Feuerverger and Hall (1999) [AS], and Gomes, Martins and Neves (2000) [Extremes], among
others. • In these papers, authors are led to SORB EVI-estimators, with asymptotic variances larger than or equal to (γ (1 − ρ)/ρ)2, where ρ(< 0) is the aforementioned “shape” second-order parameter, ruling the rate of convergence of the distribution of the normalized sequence of maximum values towards the limiting non-degenerate random variable, with law EVγ .
9
3.3. MVRB EVI-estimators • Later on, Caeiro, Gomes & Pestana (2005) [Revstat], Gomes, Martins & Neves (2007) [Revstat] and Gomes, de Haan and Henriques-Rodrigues (2008) [JRSS] have been able to reduce the bias without increasing the asymptotic variance, kept at γ 2. • Those estimators, called minimum-variance reduced-bias (MVRB) EVI-estimators, are all based on an adequate “external” consistent estimation of the pair of second-order parameters, (β, ρ) ∈ (R, R−), done through estimators denoted (βˆ, ρˆ), and outperform the classical estimators for all k. • We now consider the simplest class of MVRB EVI-estimators:
ˆ (n/k)ρˆ /(1 − ρˆ) . H(k) ≡ H βˆ,ˆ (k) := H(k) 1 − β ρ 10
3.4. A brief symptotic comparison of classical and MVRB EVIestimators • The Hill estimator reveals usually a high asymptotic bias. Indeed, from the results of de Haan & Peng (1998) [SN], √
d
√
√
k (H(k) − γ ) = Normal0,γ 2 + bH kA(n/k) + op( kA(n/k)), √ √ where the bias bH kA(n/k) = γ β k (n/k)ρ/(1 − ρ) can be very large, moderate or small (i.e. go to ∞, constant or 0) as n → ∞. • This non-null asymptotic bias, together with a rate of conver√ gence of the order of 1/ k, leads to sample paths with a high variance for small k, a high bias for large k, and a very sharp MSE pattern, as a function of k.
11
√ • Under the same conditions as before, k H(k) − γ is asymptotically normal with variance also equal to γ 2 but with a null mean value. Indeed, under the validity of the aforementioned third-order condition related to Pareto-type class of models, we can adequately estimate the vector of second-order parameters, (β, ρ) so that H(k) outperforms H(k) for all k. Indeed, we can write [Caeiro, Gomes & Henriques-Rodrigues, 2009, CSTM] d √ √ 2 √ 2 k H(k) − γ = Normal0,γ 2 + bH kA (n/k) + op( kA (n/k)).
And is it still possible to improve the performance of estimators through the use of computer intensive methods? YES, it is.
12
4. RESAMPLING METHODOLOGIES • The use of resampling methodologies [Efron, 1979, AS] has revealed to be promising in the estimation of the nuisance parameter k, or equivalently, in the estimation of optimal sample fractions (OSF), k/n, as well as in the reduction of bias of any estimator of a parameter of extreme events. • If we ask how to choose the tuning parameter k in the EVIestimation, either through H(k) or through H(k), generally denoted T (k), we usually consider the estimation of k0T := arg min MSE(T (k)), k
where MSE stands for mean square error.
13
4.1. The bootstrap methodology and OSF-estimation • To obtain estimates of k0T , and with bxc standing as usual to the integer part of x, one can use a double-bootstrap method applied to an adequate auxiliary statistic like A(k) := T (k) − T (bk/2c), which tends to the well-known value zero and has an asymptotic behaviour similar to the one of T (k) (see Gomes and Oliveira, 2001, Extremes, among others, for the estimation through H(k) and Gomes, Figueiredo and Neves (2012), Extremes, for the the estimation through MVRB EVI-estimators. • At such optimal levels, we have a non-null asymptotic bias, and if we still want to remove such a bias, we can then make use of the generalized jackknife (GJ) methodology. 14
4.2. The GJ methodology and bias reduction • The main objectives of the Jackknife methodology are: 1. Bias and variance estimation of a certain statistic, only through manipulation of observed data x. 2. The building of estimators with bias and MSE smaller than those of an initial set of estimators. • The Jackknife or the GJ are resampling methodologies, which usually give a positive answer to the question: “May the combination of information improve the quality of estimators of a certain parameter or functional?” • The pioneering EVI reduced-bias estimators are, in a certain sense, generalized jackknife estimators, i.e., affine combinations of well-known estimators of γ. 15
• The generalized jackknife statistic was introduced by Gray and (1) (2) Shucany (1972): Let Tn and Tn be two biased estimators of γ, with similar bias properties, i.e., (i)
Bias(Tn ) = γ + φ(γ)di(n),
i = 1, 2.
Then, and trivially, if q = qn = d1(n)/d2(n) 6= 1, the affine combination
(1)
TnG := Tn
(2)
− qTn
/(1 − q)
is an unbiased estimator of γ.
16
5. GJ-MVRB EVI-ESTIMATION 5.1. A GJ corrected-bias EVI-estimator • Given H, the most natural GJ r.v. is the one associated with the random pair H(k), H(bθkc) , 0 < θ < 1, is H
GJ(q,θ)
(k) :=
H(k)−q H(bθkc) , 0 < θ < 1, 1−q
with A2(n/k) Bias∞ [H(k)] q = qn = = 2 −→ θ2ρ. Bias∞ [H(bθkc)] A (n/bθkc) n/k→∞ It is thus sensible to consider q = θ2ρ, θ = 1/2, and, with ρˆ a consistent estimator of ρ, the GJ-MVRB EVI-estimator, H
GJ
ρ H(k) − H(bk/2c) 22ˆ (k) := . 2ˆ ρ 2 −1
17
• Then, and provided that ρb − ρ = op(1), d √ 2 √ GJ k H (k) − γ = Normal0,σ2 + op( kA (n/k)), GJ
with 2 = γ 2 (1 + 1/(2−2ρ − 1)2 , σGJ
just as proved in Gomes, Martins & Neves, 2013, CSTM. • We have again a trade-off between variance and bias . . . The bias decreases, but the variance increases . . . But we are able to reach a better performance at optimal levels.
18
As an (asymptotic) illustration:
MSE H
!
BIASH2
!
BIASH2
MSE H!
MSE H GJ
VarH GJ
VarH = VarH
! !
0
!
!
0.5
BIASH2 GJ
1
r = k /n
! 19
!
5.2. The GJ EVI-estimation applied to environmental data • The first set of data, already considered in Gomes, Figueiredo & Neves (2012), Extremes, are related to the number of hectares, exceeding 100 ha, burnt during wildfires recorded in Portugal during 21 years (1990-2010). • Most of the wildfires are extinguished within a short period of time, with almost negligible effects. However, some wildfires go out of control, burning hectares of land and causing significant and negative environmental and economical impacts. • The data (a sample of size n = 2627) do not seem to have a significant temporal structure, and we have used it as a whole.
20
• A box-and-whiskers plot of the data provides evidence on the Burnt area over 500ha heaviness of the right tail .
0
10000
30000
50000
Box-and-whiskers plot associated with the burnt areas in Portugal, above 100 ha. (1990-2003). 21
• Let us have a look at the behaviour of the adaptive EVIestimators under consideration for this data set. • The double-bootstrap algorithm depends very weakly on the choice of a subsample size n1 = o(n). For n1 = bn0.955c = 1843, and B=250 bootstrap replications, we have got – ˆ k0H = 157 & the Hill EVI-estimate, H ∗ = 0.73, ∗ – ˆ k0H = 1319 & the MVRB EVI-estimate, H = 0.66, – ˆ k0H
GJ
= 2296 & the GJ-MVRB EVI-estimate, H
GJ ∗
= 0.65,
the values presented in the following figure. Remark 1. Note that bootstrap confidence intervals as well as asymptotic confidence intervals are easily associated with the estimates presented, the smallest size (with a high coverage probability) GJ ∗ . being related with H 22
0.9
H 0.8
H * = 0.73 (0.675, 0.785) 0.7
H * = 0.66 (0.625, 0.695)
H H GJ
0.6
H GJ* = 0.65 (0.622, 0.678)
k
0.5 0
73
1319
2296
2800
EVI-estimates for the burned areas.
23
6. SOME OVERALL CONCLUSIONS 1. The double-bootstrap algorithm, despite of computationally intensive, is quite reliable for the estimation of OSF. 2. The most attractive features of the GJ EVI-estimators are their stable sample paths (for a wide region of k values), close to the target value, and the “bath-tube” M SE patterns. The insensitivity of the mean value (and sample path) to changes in k is indeed the nicest feature of all GJ-estimators.
24
REFERENCES 1. Beirlant, J., Dierckx, G., Goegebeur, Y. and Matthys, G. (1999). Tail index estimation and an exponential regression model. Extremes 2, 177–200. 2. Caeiro, F., Gomes, M.I. and Pestana, D.D. (2005). Direct reduction of bias of the classical Hill estimator. Revstat 3(2), 113–136. 3. Caeiro, F., Gomes, M.I. and Henriques-Rodrigues, L. (2009). Reduced-bias tail index estimators under a third order framework. Communications in Statistics – Theory & Methods 38:7, 1019–1040. 4. Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife Ann. Statist. 7:1, 1–26. 5. Feuerverger, A. and Hall, P. (1999). Estimating a tail exponent by modelling departure from a Pareto distribution. Annals Statistics 27, 760–781. 6. Gnedenko, B.V. (1943). Sur la distribution limite du terme maximum d’une s´ erie al´ eatoire. Ann. Math. 44, 423–453. 7. Gomes, M.I. and Oliveira, O. (2001). The bootstrap methodology in Statistics of Extremes: choice of the optimal sample fraction. Extremes 4:4, 331–358, 2002. 8. Gomes, M.I., Martins, M.J. and Neves, M. (2000). Alternatives to a semi-parametric estimator of parameters of rare events: the Jackknife methodology. Extremes 3(3), 207–229. 9. Gomes, M. I., Martins, M. J. and Neves, M. (2007). Improving second order reduced bias extreme value index estimation. Revstat 5(2), 177–207.
25
10. Gomes, M. I., de Haan, L. and Henriques Rodrigues, L. (2008). Tail index estimation through accommodation of bias in the weighted log-excesses. J. Royal Statistical Society B70, Issue 1, 31–52. 11. Gomes, M.I., Figueiredo, F. and Neves, M.M. (2012). Adaptive estimation of heavy right tails: resampling-based methods in action. Extremes 15, 463–489. 12. Gomes, M.I., Martins, M.J. and Neves, M.M. (2013). Generalised Jackknife-Based Estimators for Univariate Extreme-Value Modelling. Communications in Statistics: Theory and Methods 42:7, 1227–1245. 13. Gray, H.L., and Schucany, W.R. (1972). The Generalized Jackknife Statistic. Marcel Dekker. 14. de Haan, L. (1984). Slow variation and characterization of domains of attraction. In Tiago de Oliveira, ed., Statistical Extremes and Applications, 31–48, D. Reidel, Dordrecht, Holland. 15. de Haan, L. and Peng, L. (1998). Comparison of tail index estimators. Statistica Neerlandica 52, 60–70. 16. Hill, B.M. (1975). A simple general approach to inference about the tail of a distribution. Ann. Statist. 3, 1163–1174. 17. Peng, L. (1998). Asymptotically unbiased estimator for the extreme-value index. Statist. and Probab. Letters 38(2), 107–115.
THAT’s ALL and THANKS . . . 26
