Submitted to the Annales de l’Institut Henri Poincar´ e - Probabilit´ es et Statistiques
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails Propri´et´es asymptotiques uniformes d’un estimateur non-param´etrique de l’indice des valeurs extrˆemes conditionnel Yuri Goegebeura, Armelle Guilloub and Gilles Stupflerc a University
of Southern Denmark, Department of Mathematics & Computer Science, Campusvej 55, 5230 Odense M, Denmark E-mail:
[email protected]
b Universit´ e
c Aix
de Strasbourg & CNRS, IRMA, UMR 7501, 7 rue Ren´ e Descartes, 67084 Strasbourg Cedex, France E-mail:
[email protected]
Marseille Universit´ e, CERGAM, EA 4225, 15-19 all´ ee Claude Forbin, 13628 Aix-en-Provence Cedex 1, France E-mail:
[email protected]
Abstract. We consider a nonparametric regression estimator of conditional tails introduced by Goegebeur, Y., Guillou, A., Schorgen, G. (2013). Nonparametric regression estimation of conditional tails - the random covariate case. It is shown that this estimator is uniformly strongly consistent on compact sets and its rate of convergence is given. R´ esum´ e. Nous consid´erons l’estimateur ` a noyau de l’indice des valeurs extrˆemes conditionnel pr´esent´e dans Goegebeur, Y., Guillou, A., Schorgen, G. (2013). Nonparametric regression estimation of conditional tails - the random covariate case. Nous montrons la consistance uniforme presque sˆ ure de cet estimateur sur les compacts et nous calculons sa vitesse de convergence presque sˆ ure. Keywords: Tail-index, kernel estimation, strong uniform consistency.
1. Introduction Extreme value analysis has attracted considerable attention in many fields of application, such as hydrology, biology and finance, for instance. The main result of extreme value theory asserts that the asymptotic distribution of the – properly rescaled – maximum of a sequence (Y1 , . . . , Yn ) of independent copies of a random variable Y with distribution function F is a distribution having the form −1/γ
Gγ (x) = exp(−(1 + γx)+
) where y+ = max(0, y)
1 imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
2
Y. Goegebeur et al.
for some γ ∈ R, with G0 (x) = exp(−e−x ). The distribution function F is then said to belong to the maximum domain of attraction of Gγ and the parameter γ is called the extreme value index. Many applications in the areas of finance, insurance and geology, to name a few, can be found in the case when γ > 0, where F is a heavy-tailed distribution i.e. the associated survival function F := 1 − F satisfies F (x) = x−1/γ L(x), where γ shall now be referred to as the tail-index and L is a slowly varying function at infinity: namely, L satisfies, for all λ > 0, L(λx)/L(x) → 1 as x goes to infinity. In this case, the parameter γ clearly drives the tail behavior of F ; its estimation is in general a first step of extreme value analysis. For instance, if the idea is to estimate extreme quantiles – namely, quantiles with order αn > 1 − 1/n, where n is the sample size – then one has to extrapolate beyond the available data using an extreme value model which depends on the tail-index. For this reason, the problem of estimating γ has been extensively studied in the literature. Recent overviews on univariate tail-index estimation can be found in the monographs of Beirlant et al. [2] and de Haan and Ferreira [17]. In practice, it is often useful to link the variable of interest Y to a covariate X. In this situation, the tail-index depends on the observed value x of the covariate X and shall be referred to, in the following, as the conditional tail-index. Its estimation has been addressed in the recent extreme value literature, albeit mostly when the covariates are nonrandom. Smith [31] and Davison and Smith [10] considered a parametric regression model while Hall and Tajvidi [19] used a semi-parametric approach to estimate the conditional tail-index. Fully nonparametric methods have been considered using splines (see Chavez-Demoulin and Davison [4]), local polynomials (see Davison and Ramesh [9]), a moving window approach (see Gardes and Girard [12]), or a nearest neighbor approach (see Gardes and Girard [13]), among others. Less attention though has been paid to the random covariate case, despite its practical interest. One can recall the works of Wang and Tsai [33], based on a maximum likelihood approach in the Hall class of distribution functions (see Hall [18]), Daouia et al. [7] who use a fixed number of nonparametric conditional quantile estimators to estimate the conditional tail-index, later generalized in Daouia et al. [6] to a regression context with response distributions belonging to the general max-domain of attraction, and Goegebeur et al. [16] and Gardes and Stupfler [14] who both provide adaptations of Hill’s estimator (Hill [22]), the latter also studying an average of Hill-type statistics to improve the finite sample performance of the method. In this paper, we focus on a nonparametric regression estimator of conditional tails introduced by Goegebeur et al. [16]. The particular structure of this estimator makes it possible to study its uniform properties. Note that uniform properties of estimators of the conditional tail-index are seldom considered in the literature. One can think of the work of Gardes and Stupfler [14], who study the uniform weak consistency of their estimator. Outside the field of conditional tail-index estimation, uniform convergence of the Parzen-Rosenblatt density estimator (Parzen [28] and Rosenblatt [29]) was first considered by Nadaraya [27]. His results were then improved by Silverman [30] and Stute [32], the latter proving a law of the iterated logarithm in this context. Analogous results on kernel regression estimators were obtained by, among others, Mack and Silverman [26], H¨ ardle et al. [20] and Einmahl and Mason [11]. Uniform consistency of isotonized versions of order−α quantile estimators introduced in Aragon et al. [1] was shown in Daouia and Simar [8]. The case of estimators of the left-truncated quantiles is considered in Lemdani et al. [25]. Finally, the uniform strong consistency of a frontier estimator using kernel regression on high order moments was shown in Girard et al. [15]. The paper is organised as follows. Our main results are stated in Section 2. The estimator is shown to be uniformly strongly consistent on compact sets in a semiparametric framework. The rate of convergence is provided when a further condition on the bias is satisfied. The rate of uniform convergence is closely linked to the rate of pointwise convergence in distribution established in Goegebeur et al. [16]. The proofs of the main results are given in Section 3. Auxiliary results are postponed to the Appendix. 2. Main results We assume that the covariate X takes its values in Rd for some d ≥ 1. We shall work in the following semiparametric framework:
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
3
(SP ) X has a probability density function f with support S ⊂ Rd having nonempty interior and the conditional survival function of Y given X = x is such that ∀ x ∈ S, ∀ y ≥ 1, F (y | x) = y −1/γ(x) L(y | x) where γ(x) > 0 and L(· | x) is a slowly varying function at infinity.
The estimator of the conditional tail-index we shall study in this paper is defined as
γn (x) := b
n X i=1
Kh (x − Xi )(log Yi − log ωn,x )+ 1l{Yi >ωn,x } n X i=1
.
(1)
Kh (x − Xi )1l{Yi >ωn,x }
Here Kh (u) := h−d K(u/h) where K is a probability density function on Rd and h := hn is a positive sequence (1) (0) tending to 0 while for all x, (ωn,x ) is a positive sequence tending to infinity. Note that b γn (x) = Tn (x)/Tn (x) where, for all t ≥ 0, n 1X Tn(t) (x) := Kh (x − Xi )(log Yi − log ωn,x )t+ 1l{Yi >ωn,x } . n i=1
The estimator (1) is an element of the family of estimators introduced in Goegebeur et al. [16], which can be seen as an adaptation of the classical Hill estimator of the tail-index for univariate distributions (see Hill [22]). Note that the threshold ωn,x is local, i.e. it depends on the point x where the estimation is to be made, while the bandwidth h is global.
We first wish to state the uniform strong consistency of our estimator on an arbitrary compact subset Ω of Rd contained in the interior of S. To this end, we first assume that for every x ∈ S the slowly varying function L(· | x) appearing in F (· | x) is normalised (see Bingham et al. [3]): (A1 ) For all x ∈ S and y ≥ 1,
L(y | x) = cL (x) exp
Z
1
y
α(v | x) dv v
where cL (x) > 0 and α(· | x) is a function converging to 0 at infinity.
Let k · k be a norm on Rd and for r > 0, let Ωr be the set of those points in Rd whose distance to Ω is not more than r: Ωr = {x ∈ Rd | ∃x′ ∈ Ω, kx − x′ k ≤ r}. Remark that since Ω is contained in the interior of the closed set S, the distance of Ω to the boundary of S must be positive. As a consequence, the set Ωr is contained in S for all r > 0 small enough. We can therefore introduce some classical regularity assumptions: (A2 ) For some r > 0, on Ωr , the functions f and γ are positive H¨ older continuous functions, log cL is a H¨ older continuous function and α(y | ·) is a H¨ older continuous function uniformly in y ≥ 1: for all x, x′ ∈ Ωr , |f (x) − f (x′ )| ≤ |γ(x) − γ(x′ )| ≤
| log cL (x) − log cL (x′ )| ≤ sup |α(y | x) − α(y | x′ )| ≤ y≥1
Mf kx − x′ kηf , Mγ kx − x′ kηγ ,
McL kx − x′ kηcL , Mα kx − x′ kηα .
Let moreover η := ηγ ∧ ηcL ∧ ηα . We introduce the oscillation of x 7→ log ωn,x at a point x ∈ Rd over the ball B(x, ε): ∀ ε > 0, ∆(log ωn,x )(ε) := sup |log ωn,x − log ωn,z | z∈B(x, ε)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
4
Y. Goegebeur et al.
and the quantity α(y | x) := supt≥y |α(t | x)| for all y ≥ 1. Our results are established under the following classical regularity condition on the kernel: (K) K is a probability density function which is H¨ older continuous with H¨ older exponent ηK > 0: for all x, x′ ∈ Rd , |K(x) − K(x′ )| ≤ MK kx − x′ kηK and its support is included in the unit ball B of Rd .
Especially, if (K) holds then K is bounded with compact support. Let s nhd F (ωn,x | x) vn (x) := log n and introduce the hypothesis (C) For some b > 0, it holds that lim sup sup vn (x)∆(log ωn,x )(n−b ) < ∞. n→∞
x∈Ω
Our uniform strong consistency result may now be stated: Theorem 1. Assume that (SP ), (K), (A1 ) and (A2 ) hold and that • inf vn (x) → ∞; x∈Ω
• inf ωn,x → ∞; x∈Ω
• hη sup log ωn,x → 0; x∈Ω
• sup ∆(log ωn,x )(h) → 0; x∈Ω
• sup α(y | x) → 0 as y → ∞. x∈Ω
Assume moreover that condition (C) is satisfied. Then it holds that sup |b γn (x) − γ(x)| → 0
x∈Ω
almost surely as n → ∞.
Note that the hypotheses inf ωn,x → ∞ and sup α(y | x) → 0 as y → ∞ imply the convergence x∈Ω
x∈Ω
sup α(ωn,x | x) → 0
x∈Ω
which shall frequently be used in the proofs of our results. Besides, using the mean value theorem, it holds that |eu − 1| ≤ 2|u| for u ∈ R such that |u| is sufficiently small. As a consequence, using the condition sup ∆(log ωn,x )(h) → 0, this inequality implies that for n large enough
x∈Ω
sup x∈Ω
ωn,x ≤ 2 sup ∆(log ωn,x )(h) → 0. − 1 ωn,z x∈Ω z∈B(x, h) sup
(2)
Finally, the conditions
sup ∆(log ωn,x )(h) → 0 and lim sup sup vn (x)∆(log ωn,x )(n−b ) < ∞
x∈Ω
n→∞
x∈Ω
are satisfied if for instance ωn,x = ng(x) where g : S → R is a positive H¨ older continuous function whose H¨ older exponent is not less than η. In other words, Theorem 1 requires that a continuity property on x 7→ log ωn,x be satisfied. Our second aim is to compute the rate of uniform strong consistency of the estimator (1):
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
Theorem 2. Assume that the conditions of Theorem 1 are satisfied. If moreover lim sup sup vn (x) {α(ωn,x | x) ∨ hηf ∨ hη log ωn,x ∨ ∆(log ωn,x )(h)} < ∞ n→∞
5
(3)
x∈Ω
then it holds that sup vn (x) |b γn (x) − γ(x)| = O (1)
x∈Ω
almost surely as n → ∞.
bn . The terms hηf and hη log ωn,x Let us highlight that condition (3) controls the bias of the estimator γ correspond to the bias which stems from the use of a kernel regression, while the presence of the other terms is due to the particular structure of the semiparametric model (SP ). Besides, as pointed out in Goegebeur et al. [16], the rate of pointwise convergence of b γn (x) to γ(x) is [nhd F (ωn,x | x)]1/2 . Up to the term [log n]1/2 , the rate of uniform convergence of γ bn to γ is therefore the infimum (over Ω) of the rate of pointwise convergence of b γn (x) to γ(x). 3. Proofs of the main results
Before starting the proof of Theorem 1, let us note that assuming that (SP ), (A1 ) and (A2 ) hold then it is easy to show that there exists a positive constant MF such that the function (x, y) 7→ log F (y | x) has the following property: for all x, x′ ∈ Ωr such that kx − x′ k ≤ 1 and y, y ′ ≥ e, 1 ′ ′ log F (y | x) ≤ M kx − x′ kη log y + (4) + α(y ∧ y | x ) | log y − log y ′ |. F γ(x′ ) F (y ′ | x′ )
Moreover, if (A2 ) holds then one may take a positive number r such that the four conditions of the hypothesis hold on Ω2r . Since Ωr is compact, f := supΩr f < ∞ and f := inf Ωr f > 0. As a consequence, the uniform relative oscillation of f over the ball B(x, h) can be controlled as f (z) sup sup − 1 = O (hηf ) → 0. (5) r x∈Ω z∈B(x, h) f (x) Second, γ := supΩr γ < ∞ and γ := inf Ωr γ > 0 and we thus have γ(z) sup sup − 1 = O (hηγ ) → 0. x∈Ωr z∈B(x, h) γ(x)
(6)
Third, we can write for all x, x′ ∈ Ωr and t ≥ 1
α(t | x) ≤ α(t | x′ ) + |α(t | x) − α(t | x′ )| and the roles of x and x′ are symmetric in the above inequality, so that taking the supremum over t ≥ y on both sides yields ∀ y ≥ 1, |α(y | x) − α(y | x′ )| ≤ Mα kx − x′ kηα .
(7)
We may now prove the key result for the proof of Theorem 1, which is a uniform law of large numbers for (t) (t) (1) (0) Tn (x) and Tn (x). In what follows, we let µn (x) := E(Tn (x)). Proposition 1. Assume that the conditions of Theorem 1 are satisfied. Then for every t ∈ {0, 1} it holds that T (t) (x) n sup vn (x) (t) − 1 = O(1) almost surely as n → ∞. µn (x) x∈Ω imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
6
Y. Goegebeur et al.
ardle and Marron [21]: we shall in Proof of Proposition 1. The proof is based on that of Lemma 1 in H¨ fact show complete convergence in the sense of Hsu and Robbins [24]. Since Ω is a compact subset of Rd , we may, for every n ∈ N \ {0}, find a finite subset Ωn of Ω such that: ∀ x ∈ Ω, ∃ χ(x) ∈ Ωn , kx − χ(x)k ≤ n−b
and ∃ c > 0, |Ωn | = O (nc ) ,
where b, which we may take to be not less than 1/d + 1/2ηK , is given by condition (C) and |Ωn | stands for the cardinality of Ωn . Notice that, since nhd → ∞, one has n−b /h → 0, so that one can assume that eventually χ(x) ∈ B(x, h) for all x ∈ Ω. Next, remark that kx − χ(x)k ≤ n−b ≤ h ≤ 1 and that since n−b ≤ h the convergences n−bη sup log ωn,x ≤ hη sup log ωn,x → 0 and sup ∆(log ωn,x )(n−b ) ≤ sup ∆(log ωn,x )(h) → 0 x∈Ω
x∈Ω
x∈Ω
x∈Ω
hold. Consequently, Lemma 1 entails s vn (x) F (ω | x) n,x − 1 → 0. − 1 = sup sup F (ωn,χ(x) | χ(x)) x∈Ω x∈Ω vn (χ(x))
(8)
Pick ε > 0 and an arbitrary sequence of positive numbers (δn ) converging to 0; using together (8) and the triangular inequality thus shows that for n large enough ! T (t) (x) n − 1 > ε ≤ R1,n + R2,n P δn sup vn (x) (t) µn (x) x∈Ω where
! T (t) (z) ε n R1,n := P δn vn (z) (t) − 1 > µn (z) 4 z∈Ωn ! T (t) (x) T (t) (χ(x)) ε n n and R2,n := P δn sup vn (x) (t) . − (t) > µn (x) x∈Ω µn (χ(x)) 2 P P The goal of the proof is now to show that the series n R1,n and n R2,n converge. The result of Proposition 1 shall then be an easy consequence of Borel-Cantelli’s lemma and Lemma 6. X
We start by controlling R1,n . To this end, apply Lemma 3 to get that there exists a positive constant κ such that for n large enough, ! T (t) (z) ε κ 2 nhd F (ωn,z | z) n ≤ 2 exp − ε . ∀ z ∈ Ωn , P δn vn (z) (t) − 1 > µn (z) 4 16 δn2 vn2 (z) Use now the definition of vn (z) to get
R1,n Hence
P
n
κ 2 log n c . = O n exp − ε 16 δn2
R1,n converges.
We now turn to R2,n . Using the triangular inequality gives ε ε R2,n ≤ P δn sup vn (x)S1,n (x) > + P δn sup vn (x)S2,n (x) > =: R3,n + R4,n 4 4 x∈Ω x∈Ω where
n 1 X Kh (x − Xi ) Kh (χ(x) − Xi ) S1,n (x) := − (log Yi − log ωn,χ(x) )t+ 1l{Yi >ωn,χ(x) } , (t) n i=1 µ(t) µn (χ(x)) n (x) n 1 X Kh (x − Xi ) t t Y − log ω ) 1 l − (log Y − log ω ) 1 l S2,n (x) := (log , i n,x i {Y >ω } n,χ(x) {Y >ω } + + i n,x i n,χ(x) n i=1 µ(t) n (x)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
7
P P and it is enough to show that the series n R3,n and n R4,n converge. P To deal with n R3, n use once again the triangular inequality to obtain K (x − X ) K (χ(x) − X ) h h i i (t) − µn (χ(x)) ≤ |Kh (x − Xi ) − Kh (χ(x) − Xi )| (t) µ(t) µn (χ(x)) n (x) µ(t) (χ(x)) n + (t) − 1 Kh (x − Xi ). µn (x)
Using hypothesis (K) and Lemma 4, there exists a positive constant M such that for n large enough: K (x − X ) K (χ(x) − X ) M n−b ηK h h i i −b (t) ≤ ∨ ∆(log ω )(n ) . − ∀ x ∈ Ω, µn (χ(x)) n,x d (t) µ(t) h µn (χ(x)) h n (x) Besides
n
e (t) m n (z) := (t)
1X K2h (z − Xi )(log Yi − log ωn,z )t+ 1l{Yi >ωn,z } n i=1
is the empirical analogue of mn (z) defined before Lemma 4; since the support of the random variable Kh (x − Xi ) is included in B(χ(x), 2h), one has for n large enough (t) −b ηK e n (χ(x)) m n ∨ ∆(log ωn,x )(n−b ) . ∀ x ∈ Ω, vn (x)S1,n (x) ≤ 2d VM vn (x) (t) h µn (χ(x))
(t) e (t) Moreover, since m n (z) is a kernel estimator of mn (z, z) for which the conditions of Lemma 2 are satisfied, we get for n large enough: # " (t) m e (t) m n (z) e n (z) ∀ z ∈ Ωn , δn vn (z) (t) ≤ 2δn vn (z) 1 + (t) − 1 . µn (z) mn (z)
The fact that b ≥ 1/d + 1/2ηK gives
n−b sup vn (z) h z∈Ωn
ηK
η ηK /d √ n−b K 1 → 0. ≤ n ≤ h nhd
Using first this convergence together with hypothesis (C) and (8) and then Lemma 3 entails for n large enough: ! (t) m X log n e n (z) R3,n ≤ P δn vn (z) (t) − 1 > ε = O nc exp −κ′ ε2 2 mn (z) δn z∈Ωn P where κ′ is a positive constant. Hence n R3,n converges. P To control n R4,n first use Lemmas 2(iv) and 4 to get, for n large enough ) ( (t) (t) (t) mn (χ(x)) µn (χ(x)) mn (χ(x)) ≤ 2. = sup sup (t) (t) (t) x∈Ω x∈Ω µn (x) µn (χ(x)) µn (x) Therefore, since the support of the random variable Kh (x − Xi ) is included in B(χ(x), 2h), one has for n large enough and all x ∈ Ω S2,n (x) ≤ 2d+1 VkKk∞ S3,n (x)
where kKk∞ := sup K and B
S3,n (x) :=
n 1 X K2h (χ(x) − Xi ) t t Y − log ω ) 1 l − (log Y − log ω ) 1 l (log . i n,x i {Y >ω } n,χ(x) {Y >ω } + + i n,x i n,χ(x) (t) n i=1 mn (χ(x))
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
8
Y. Goegebeur et al.
We then get R4,n and it is enough to control
P
≤ P δn sup vn (x)S3,n (x) >
n
x∈Ω
ε 2d+3 VkKk∞
=: R5,n
R5,n . We start by considering the case t = 0. In this case, S3,n (x) reduces to
S3,n (x) =
n
1 X K2h (χ(x) − Xi ) 1l{ωn,x ∧ωn,χ(x) ε ≤ 2 exp −κ′′′ ε2 nhd F (ωn,x | x) ∀ x ∈ Ω, P νn (x)
(10)
(11)
where the inequality F (ωn,x /2 | x) ≥ F (ωn,x | x) was used. We conclude by noting that according to (4), F (ωn,x | x) log 2 F (ωn,x | x) lim sup sup log ≤ < ∞. < ∞ ⇒ 0 < lim sup sup γ F (ωn,x /2 | x) n→∞ x∈Ω n→∞ x∈Ω F (ωn,x /2 | x) This property together with (10) entails the convergences δn sup
νn (x)
(0) x∈Ω mn (x)
→ 0 and sup ∆(log ωn,x )(n−b ) x∈Ω
νn (x) (0)
mn (x)
→ 0.
(12)
Reporting (10) along with (12) into (9), recalling condition (C) and using the triangular inequality together with (8) shows that for n large enough, X Vn (x) log n − 1 > ε = O nc exp −κ′′′ ε2 2 R5,n ≤ P δn vn (z) νn (x) δn z∈Ωn P where (11) was used in the last step. As a consequence, n R5,n converges in this case as well. This completes the proof of Proposition 1. With Proposition 1 at hand, we can now prove Theorem 1 and Theorem 2. Proof of Theorem 1. Notice that γn (x) = b
(1)
(1)
(0)
(0)
(1)
(0)
µn (x) Tn (x) µn (x) µn (x) µn (x) Tn (x)
.
Applying Proposition 1 twice yields T (1) (x) µ(0) (x) n n sup (1) − 1 → 0 almost surely as n → ∞. (0) x∈Ω µn (x) Tn (x)
(13)
(14)
Moreover, recalling that γ is continuous and therefore bounded on the compact set Ω, using Lemma 2(i) and (iv) twice entails µ(1) (x) n − γ(x) → 0 as n → ∞. (15) sup (0) x∈Ω µn (x) The result follows by reporting (14) and (15) into (13).
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
10
Y. Goegebeur et al.
Proof of Theorem 2. Note that because nhd → ∞, the hypothesis lim sup sup vn (x)∆(log ωn,x )(h) < ∞ n→∞
x∈Ω
entails condition (C). Besides, Proposition 1 yields µ(0) (x) T (1) (x) n n sup vn (x) (0) − 1 = O(1) and sup vn (x) (1) − 1 = O(1) x∈Ω x∈Ω Tn (x) µn (x)
(16)
for t ∈ {0, 1}, so that using condition (3), µ(1) (x) n − γ(x) = O(1). sup vn (x) (0) µn (x) x∈Ω
(17)
almost surely as n → ∞. Moreover, Lemma 2 (iv) gives (t) µn (x) 1 t sup − γ (x) = O(1) ηf ∨ hη log ω n,x f (x)F (ωn,x | x) x∈Ω α(ωn,x | x) ∨ h
The result follows by reporting (16) and (17) into (13).
Appendix: Auxiliary results and proofs The first lemma of this section is a technical result that gives an upper bound for the oscillation of the log-conditional survival function. Lemma 1. Assume that (SP ), (A1 ) and (A2 ) hold. Let moreover ε := εn , ε′ := ε′n and ε′′ := ε′′n be three positive sequences tending to 0 and assume that • inf ωn,x → ∞ ; x∈Ω
• ε′′η sup log ωn,x → 0 ; x∈Ω
• sup ∆(log ωn,x )(ε′ ) → 0 ; x∈Ω
• sup α(y | x) → 0 as y → ∞. x∈Ω
Then it holds that, for n large enough, F (ωn,z | z ′ ) 2 ∀ (x, x ) ∈ Ω × Ω , ∀(z, z ) ∈ B(x, ε ) × B(x , ε ), log ≤ MF ε′′η log ωn,z + ∆(log ωn,x )(ε′ ). ′ γ F (ωn,x | x ) ′
ε
′
′
′
′′
In particular,
sup sup
sup
x∈Ω x′ ∈Ωε z∈B(x, ε′ )
F (ωn,z | z ′ ) 1 = O(1). − 1 ′′η ′ ′ log ωn,x ∨ ∆(log ωn,x )(ε ) F (ωn,x | x ) z ′ ∈B(x′ , ε′′ ) ε sup
Proof of Lemma 1. Pick (x, x′ ) ∈ Ω × Ωε and (z, z ′ ) ∈ B(x, ε′ ) × B(x′ , ε′′ ). Use (4) to get for n large enough ′ 1 ′ log F (ωn,z | z ) ≤ M kx′ − z ′ kη log ωn,z + + α(ωn,z ∧ ωn,x | x ) |log ωn,x − log ωn,z | . F γ(x′ ) F (ωn,x | x′ ) imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
11
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
For every y ≥ 1, inequality (7) entails sup α(y | x) ≤ sup α(y | x) + Mα εηα .
x∈Ωε
(18)
x∈Ω
Using then (2) with ε′ instead of h, we get inf
inf
x∈Ω z∈B(x, ε′ )
sup sup
sup
x∈Ω x′ ∈Ωε z∈B(x, ε′ )
ωn,z ∧ ωn,x = inf ωn,x (1 + o(1)) → ∞, so that x∈Ω
α(ωn,z ∧ ωn,x | x′ ) → 0.
Especially, since 0 < γ ≤ γ(x′ ), we obtain for n large enough:
F (ωn,z | z ′ ) 2 ≤ MF ε′′η log ωn,z + ∆(log ωn,x )(ε′ ) ∀ (x, x ) ∈ Ω × Ω , ∀(z, z ) ∈ B(x, ε ) × B(x , ε ), log ′ γ F (ωn,x | x ) ′
ε
′
′
′
′′
which is the first part of the result. To prove the second part, note that because sup ∆(log ωn,x )(ε′ ) → 0 it x∈Ω
holds that for n large enough F (ωn,z | z ′ ) 2 ≤ 2MF ε′′η log ωn,x + ∆(log ωn,x )(ε′ ). ∀ (x, x ) ∈ Ω × Ω , ∀(z, z ) ∈ B(x, ε ) × B(x , ε ), log ′ γ F (ωn,x | x ) ′
ε
′
′
′
′′
Consequently
sup sup
sup
x∈Ω x′ ∈Ωε z∈B(x, ε′ )
1 F (ωn,z | z ′ ) sup log = O(1). ′′η log ω ′ F (ωn,x | x′ ) n,x ∨ ∆(log ωn,x )(ε ) z ′ ∈B(x′ , ε′′ ) ε
Using the equivalent eu − 1 = u(1 + o(1)) therefore completes the proof of Lemma 1. The second lemma examines the behavior of the conditional moment t m(t) n (x, z) := E((log Y − log ωn,x )+ 1l{Y >ωn,x } | X = z) (t)
(t)
and that of its smoothed version µn (x) = E(Kh (x − X)mn (x, X)). Let Γ be Euler’s Gamma function: Z +∞ ∀ t > 0, Γ(t) := v t−1 e−v dv. 0
Lemma 2. Assume that (SP ), (A1 ) and (A2 ) hold. Pick t ≥ 0 and assume that K is a bounded probability density function on Rd with support included in B. If moreover • inf ωn,x → ∞ ; x∈Ω
• hη sup log ωn,x → 0 ; x∈Ω
• sup α(y | x) → 0 as y → ∞ x∈Ω
then, as n → ∞, the following estimations hold: (t) 1 mn (x, z) − 1 (i) sup sup = O (1). η t x∈Ω z∈B(x, h) α(ωn,x | x) ∨ h α γ (z)Γ(t + 1)F (ωn,x | z) (t) 1 mn (x, z) (ii) sup sup − 1 = O (1). (t) η x∈Ω z∈B(x,h) α(ωn,x | x) ∨ h log ωn,x mn (x, x) (t) µn (x) 1 − 1 (iii) sup = O (1). η η (t) f ∨ h log ωn,x f (x)mn (x, x) x∈Ω α(ωn,x | x) ∨ h
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
12
Y. Goegebeur et al.
1 (iv) sup η η x∈Ω α(ωn,x | x) ∨ h f ∨ h log ωn,x
(t) µn (x) − 1 = O (1). f (x)γ t (x)Γ(t + 1)F (ωn,x | x) (0)
Proof of Lemma 2. (i) When t = 0, there is nothing to prove, since mn (x, z) = F (ωn,x | z) and Γ(1) = 1. In the case t > 0, an integration by parts yields Z +∞ Z +∞ (log y − log ωn,x )t−1 F (rωn,x | z) (t) dr. F (y | z) dy = t F (ωn,x | z) (log r)t−1 t mn (x, z) = y rF (ωn,x | z) 1 ωn,x From (SP ) and (A1 ), one has ! Z rωn,x F (rωn,x | z) α(v | z) −1/γ(z)−1 −1/γ(z)−1 dv − 1 − r = r exp . rF (ω | z) v ωn,x n,x
(19)
For all y ∈ R, the mean value theorem yields |ey − 1| ≤ |y|e|y| . Meanwhile, Z rωn,x α(v | z) dv ≤ α(ωn,x | z) log r. ωn,x v
Choosing n so large that sup
sup
x∈Ω z∈B(x, h)
α(ωn,x | z) < 1/2γ, (18), (19) and (20) together imply that, for all
x ∈ Ω and z ∈ B(x, h), Z +∞ Z −1/γ(z)−1 ηα t−1 F (rωn,x | z) dr ≤ (α(ωn,x | x) + Mα h ) −r (log r) rF (ω | z) 1
(20)
n,x
+∞
(log r)t r−1/2γ−1 dr
1
which, since the integral on the right-hand side of this inequality converges, gives Z +∞ 1 t−1 F (rωn,x | z) −1/γ(z)−1 sup sup (log r) dr = O (1) −r ηα α(ω | x) ∨ h rF (ωn,x | z) x∈Ω z∈B(x, h) n,x 1
as n → ∞. An elementary change of variables and the well-known equality tΓ(t) = Γ(t + 1) thus entail m(t) (x, z) 1 n t − γ (z)Γ(t + 1) sup sup = O (1) η x∈Ω z∈B(x, h) α(ωn,x | x) ∨ h α F (ωn,x | z) as n → ∞ and (i) is proven.
(ii) Since for all x ∈ Ω, 0 < γ ≤ γ(x) ≤ γ < ∞, applying (i) entails (t) mn (x, z) γ t (x)Γ(t + 1)F (ωn,x | x) 1 − 1 sup sup = O (1) . η (t) t α γ (z)Γ(t + 1)F (ωn,x | z) x∈Ω z∈B(x,h) α(ωn,x | x) ∨ h mn (x, x) Moreover, hypothesis (A2 ) and the mean value theorem yield # t " γ (x) t−1 1 sup tr sup sup |γ(x) − γ(z)| = O(hηγ ). γ t (z) − 1 ≤ γ t γ≤r≤γ x∈Ω z∈B(x,h)
(21)
(22)
Besides, using Lemma 1 gives 1 sup sup η x∈Ω z∈B(x,h) h log ωn,x
F (ωn,x | x) F (ω | z) − 1 = O(1). n,x
(23)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
13
Note finally that since η ≤ ηγ ∧ ηα and inf ωn,x → ∞ one has x∈Ω
hηγ ∨ hηα → 0. η x∈Ω h log ωn,x sup
Using then (22) and (23) together with (21) yields (ii). (iii) Let us remark that for all x ∈ Ω: (t)
µn (x) (t)
f (x) mn (x, x)
=
Z
(t)
K(u)
B
f (x − hu) mn (x, x − hu) du. (t) f (x) mn (x, x)
From (5) and (ii) it follows that 1 sup sup ηf ∨ hη log ω α(ω | x) ∨ h n,x n,x x∈Ω z∈B(x, h) as n → ∞, which yields (iii).
f (z) m(t) (x, z) n − 1 → 0 (t) f (x) mn (x, x)
(iv) This is a straightforward consequence of (i) and (iii). The third lemma is essential to prove Proposition 1. It gives a uniform exponential bound for large deviations (1) (0) of Tn and Tn . Lemma 3. Assume that (SP ), (A1 ) and (A2 ) hold. Assume that K is a bounded probability density function on Rd with support included in B. If moreover • inf ωn,x → ∞ ; x∈Ω
• hη sup log ωn,x → 0 ; x∈Ω
• sup α(y | x) → 0 as y → ∞ x∈Ω
then there exists a positive constant κ such that for all n large enough, one has for t ∈ {0, 1} and every ε > 0 small enough: ! T (t) (x) n − 1 > ε ≤ 2 exp −κε2 nhd F (ωn,x | x) . ∀ x ∈ Ω, P (t) µn (x) Proof of Lemma 3. For every x ∈ Ω: ! T (0) (x) n d (0) P (0) − 1 > ε = P hd Tn(0) (x) − hd µ(0) (x) > εh µ (x) . n n µn (x)
Notice now that if Wn,i (x) := hd Kh (x − Xi )1l{Yi >ωn,x } then n
hd Tn(0) (x) − hd µ(0) n (x) =
1X [Wn,i (x) − E(Wn,i (x))] n i=1
is a mean of bounded, centered, independent and identically distributed random variables. Define (0)
τn (x) :=
ε εkKk∞ hd µn (x) nhd µ(0) (x) and λ (x) := . n n kKk∞ Var(Wn, 1 (x))
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
14
Y. Goegebeur et al.
Bernstein’s inequality (see Hoeffding [23]) yields, for all ε > 0: ! T (0) (x) τn (x)λn (x) n . − 1 > ε ≤ 2 exp − P (0) µn (x) 2(1 + λn (x)/3)
Applying Lemma 2(iii) yields for n large enough: εf τn (x) . ≥ x∈Ω nhd F (ωn,x | x) 2kKk∞
(24)
inf
2 Moreover, since Wn, 1 (x) is bounded by kKk∞ , it follows from the inequality Wn, 1 (x) ≤ kKk∞ Wn, 1 (x) that 2 E(Wn, 1 1 1 (x)) ≤ sup ≤ . (0) d λ (x) ε x∈Ω n x∈Ω εkKk∞ h µn (x)
sup
(25)
Finally, it holds that τn (x)λn (x) ≥ 2(1 + λn (x)/3)
τn (x) inf d x∈Ω nh F (ωn,x | x)
1 inf x∈Ω 2(1/λn (x) + 1/3)
nhd F (ωn,x | x).
Using (24), (25) and the fact that the function t 7→ 1/[2(t + 1/3)] is decreasing on R+ , it is then clear that for all n large enough, if ε > 0 is small enough, there exists a positive constant κ1 that is independent of ε such that ! T (0) (x) n − 1 > ε ≤ 2 exp −κ1 ε2 nhd F (ωn,x | x) . ∀ x ∈ Ω, P (0) µn (x) (1)
We now turn to Tn (x). For every x ∈ Ω, it holds that ! ! (1) T (1) (x) Tn (x) n P (1) − 1 > ε = P −1>ε +P (1) µn (x) µn (x)
(1)
Tn (x) (1)
µn (x)
− 1 < −ε
!
=: u1,n (x) + u2,n (x).
We shall then give a uniform Chernoff-type exponential bound (see Chernoff [5]) for both terms on the right-hand side of the above inequality. We start by considering u1,n (x). Let ϕn (s, x) := E(exp(sKh (x − X)(log Y − log ωn,x )+ 1l{Y >ωn,x } )) be the moment generating function of the random variable Kh (x−X)(log Y −log ωn,x )+ 1l{Y >ωn,x } . Markov’s inequality entails, for every q > 0, ! ! !! (1) q Tn (x) > exp(q[ε + 1]) ≤ exp −q[ε + 1] + n log ϕn ,x . (26) u1,n (x) = P exp q (1) (1) µn (x) nµn (x) Our goal is now to use inequality (26) with a suitable value q ∗ (ε, x) for q. To this end, notice that Z Z ϕn (s, x) = f (z) dz + ψn (sKh (x − z) | x, z) f (z) dz Rd \B(x, h)
B(x, h)
where ψn (s | x, z) := E(exp(s(log Y − log ωn,x )+ 1l{Y >ωn,x } ) | X = z)
is the conditional moment generating function of the random variable (log Y − log ωn,x )+ 1l{Y >ωn,x } given X = z. In particular, since f is a probability density function on Rd , Z [ψn (sKh (x − z) | x, z) − 1] f (z) dz. (27) ϕn (s, x) = 1 + B(x, h)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
15
This equality makes it clear that it is enough to study the behavior of ψn (· | x, z). One has s Y 1l{Y >ωn,x } | X = z . ψn (s | x, z) = 1 − F (ωn,x | z) + E ωn,x From this we deduce that ψn (s | x, z) = 1 + F (ωn,x | z)
Z
+∞
sts
1
F (tωn,x | z) dt. tF (ωn,x | z)
A use of (19) and (20) therefore entails, for all s < 1/γ, ! −1 1 −s ψn (s | x, z) = 1 + sF (ωn,x | z) + Rn (s | x, z) γ(z) where Rn (s | x, z) satisfies, for all δ > 0, if n is large enough, sup
sup
x∈Ω z∈B(x, h)
|Rn (s | x, z)| ≤ sup
x∈Ω z∈B(x, h)
Since by (18) it holds that sup
sup
x∈Ω z∈B(x, h)
sup
sup
sup
s 0:
|Rn (s | x, z)| → 0
(29)
as n → ∞. We shall now derive a suitable value for the parameter q. Given X = x, if the remainder term Rn (1) were identically 0, then one would have mn (x, x) = γ(x)F (ωn,x | x) and thus an optimal value of q would be obtained by minimizing the function " −1 # q q . 1− q 7→ −q[1 + ε] + n log 1 + n nF (ωn,x | x) Straightforward but cumbersome computations lead to the optimal value r 2 4ε 2 − F (ωn,x | x) − 1 − F (ωn,x | x) 2 − F (ωn,x | x) − ε + 1 ⋆ qc,+ (ε) := nF (ωn,x | x) . 2 1 − F (ωn,x | x)
(30)
Since we are mostly interested in what happens in the limit n → ∞ and ε → 0, we may examine the behavior ⋆ of qc,+ (ε) in this case. Using (30), we get the following asymptotic equivalent ∗ qc,+ (ε) = nF (ωn,x | x)
ε . 2(ε + 1)
(1)
∗ Note that since qc,+ (ε)/[nmn (x, x)] = ε/[2γ(x)(ε + 1)] is positive and converges to 0 as ε → 0, the moment (1)
∗ generating function ψn (· | x, x) at qc,+ (ε)/[nmn (x, x)] is well-defined and finite for ε small enough and therefore this choice of q is valid. Back to our original context, taking into account the presence of the covariate X motivates the following value for q: ∗ qn,+ (ε, x) :=
Mε nhd f (x)F (ωn,x | x) ε+1
where M is a positive constant to be chosen later. For ε small enough and for n so large that the quantity (1) ∗ ∗ ϕn qn,+ (ε, x)/(nµn (x)), x is well-defined and finite for all x ∈ Ω, replacing q by qn,+ (ε, x) in the righthand side of (26) gives !! ∗ qn,+ (ε, x) d ,x . (31) ∀ x ∈ Ω, u1,n (x) ≤ exp −M εnh f (x)F (ωn,x | x) + n log ϕn (1) nµn (x) imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
16
Y. Goegebeur et al.
Using the classical inequality log(1 + r) ≤ r for all r > 0 together with (27) and (28), we obtain Z log ϕn (s, x) ≤ [ψn (sKh (x − z) | x, z) − 1] f (z) dz B(x, h)
≤
Z
B(x, h)
sKh (x − z)F (ωn,x | z)
! −1 1 − sKh (x − z) + Rn (sKh (x − z) | x, z) f (z) dz. γ(z)
According to Lemma 2(iv), ∗ qn,+ (ε, x) (1) nµn (x)
=
Mε d εhd F (ωn,x | x) h f (x) [1 + r1,n (x)] =M (1) ε+1 γ(x)(ε + 1) µn (x)
(32)
where r1,n (x) → 0 as n goes to infinity, uniformly in x ∈ Ω. As a consequence, using an elementary Taylor expansion, we get, for all z ∈ B(x, h), " #−1 ∗ qn,+ (ε, x) 1 γ(z) M ε d − K (x − z) h [1 + r1,n (x)]Kh (x − z) = γ(z) 1+ h (1) γ(z) γ(x) ε + 1 nµn (x) γ(z) M ε d h [1 + r1,n (x)]Kh (x − z) +k γ(x) ε + 1 where k(r)/r → 0 as r goes to 0. Letting pn (x, z) :=
γ(z) [1 + r1,n (x)]hd Kh (x − z) γ(x)
and using (6), the uniform convergence of r1,n to 0 and the fact that K is bounded yields pn (x, z) = hd Kh (x − z) + r2,n (x, z) where sup
sup
x∈Ω z∈B(x, h)
|r2,n (x, z)| → 0
as n goes to infinity. Especially, " #−1 ∗ qn,+ (ε, x) 1 Mε d − h K (x − z) + εr (ε, x, z) K (x − z) = γ(z) 1 + h 3,n h (1) γ(z) ε+1 nµn (x)
(33)
where r3,n (ε, x, z) → 0 as ε goes to 0 and n goes to infinity, uniformly in x ∈ Ω and z ∈ B(x, h). Besides, since for every ε0 > 0 q ∗ (ε, x) M ε hd Kh (x − z) n,+ sup sup sup Kh (x − z) − →0 (1) ε+1 γ(x) ε 0, !! q u2,n (x) ≤ exp −q[ε − 1] + n log ϕn − (1) ,x . nµn (x) Recall (27) and use the inequality log(1 − r) ≤ −r for all r ∈ (0, 1) to get Z [ψn (−sKh (x − z) | x, z) − 1] f (z) dz. log ϕn (−s, x) ≤ B(x, h)
We choose q as ∗ qn,− (ε, x) :=
ε nhd f (x)F (ωn,x | x) 4kKk22
which, using the ideas developed to control u1,n (x), yields ∀ x ∈ Ω, u2,n (x) ≤ exp −κ2 ε2 nhd F (ωn,x | x) .
for some constant κ2 > 0. Setting κ = κ1 ∧ κ2 completes the proof of Lemma 3. (t)
The fourth lemma of this section establishes a uniform control of the relative oscillation of x 7→ µn (x). Before stating this result, we let (t) m(t) n (x) := E(K2h (x − X)mn (x, X))
where K := 1lB /V is the uniform kernel on Rd , with V being the volume of the unit ball of Rd ; let further Kh (u) := h−d K(u/h). Lemma 4. Assume that (SP ), (K), (A1 ) and (A2 ) hold. Pick t ∈ {0, 1} and let ε := εn be a sequence of positive real numbers such that ε ≤ h. If moreover imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
18
Y. Goegebeur et al.
• inf ωn,x → ∞ ; x∈Ω
• hη sup log ωn,x → 0 ; x∈Ω
• sup ∆(log ωn,x )(ε) → 0 ; x∈Ω
• sup α(y | x) → 0 as y → ∞ x∈Ω
then sup
sup
x∈Ω z∈B(x, ε)
ηK
[ε/h]
µ(t) (z) 1 n − 1 = O (1) . (t) ∨ ∆(log ωn,x )(ε) µn (x)
Proof of Lemma 4. For all x ∈ Ω and z ∈ B(x, ε), we have (0) ≤ E |Kh (x − X) − Kh (z − X)| 1l{Y >ωn,x } + E Kh (z − X) 1l{Y >ωn,x } − 1l{Y >ωn,z } µn (x) − µ(0) n (z) (0)
(0)
=: R1,n (x, z) + R2,n (x, z)
(35)
and we shall handle both terms in the right-hand side separately. Hypothesis (K) and the inclusion B(z, h) ⊂ B(x, 2h) entail that |Kh (x − X) − Kh (z − X)| ≤ From (36), we get (0)
sup z∈B(x, ε)
MK h ε iηK 1l{X∈B(x, 2h)} . hd h
R1,n (x, z) ≤ 2d MK V m(0) n (x)
h ε iηK h
.
(36)
(37)
Because K is a probability density function on Rd with support included in B, applying Lemma 2(iii) implies that m(0) (x) n sup (0) − 1 → 0 as n → ∞ (38) x∈Ω µn (x)
which, together with (37), yields sup x∈Ω
h ε i−ηK R(0) (x, z) 1,n = O(1). (0) z∈B(x, ε) h µn (x) sup
We now turn to the second term. One has (0) R2,n (x, z) = E Kh (z − X) F (ωn,x | X) − F (ωn,z | X) .
Furthermore, using Lemma 1 with ε′′ = 0 entails F (ωn,z | x′ ) 1 sup sup sup − 1 = O(1). ′ x∈Ω x′ ∈B(x, 2h) z∈B(x, ε) ∆(log ωn,x )(ε) F (ωn,x | x )
(39)
(40)
(41)
Besides, hypothesis (K) and the inclusion B(z, h) ⊂ B(x, 2h) imply that d (0) E(Kh (z − X)m(0) n (x, X)) ≤ 2 MK V mn (x).
(42)
Using the obvious identity F (ωn,z | X) (x, X) − 1 |F (ωn,x | X) − F (ωn,z | X)| = m(0) n F (ω | X) n,x
(43)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
19
and recalling that the support of the random variable Kh (z − X) is contained in B(z, h) ⊂ B(x, 2h), (40) and (41) yield: (0) R2,n (x, z) 1 = O(1), sup sup (0) x∈Ω z∈B(x, ε) ∆(log ωn,x )(ε) mn (x) and (38) entails (0)
sup
sup
x∈Ω z∈B(x, ε)
R2,n (x, z) 1 = O(1). ∆(log ωn,x )(ε) µ(0) n (x)
(44)
Applying (35) together with (39) and (44) gives sup
sup
x∈Ω z∈B(x, ε)
which shows Lemma 4 in this case.
ηK
[ε/h]
µ(0) (z) 1 n − 1 = O(1) (0) ∨ ∆(log ωn,x )(ε) µn (x)
We now turn to the case t = 1. Note that for all real numbers a, b ≥ 1 such that a 6= b one has ∀ y ≥ 1, |(log y − log a)+ 1l{y>a} − (log y − log b)+ 1l{y>b} | ≤ | log b − log a|1l{y>a∧b} . Inequality (45) then implies, for all x ∈ Ω and z ∈ B(x, ε): (1) (z) µn (x) − µ(1) ≤ E |Kh (x − X) − Kh (z − X)| (log Y − log ωn,x )+ 1l{Y >ωn,x } n ωn,x + log E Kh (z − X)1l{Y >ωn,x ∧ωn,z } ωn,z (1)
(1)
=: R1,n (x, z) + R2,n (x, z)
(45)
(46)
and we shall once again take care of both terms in the right-hand side of this inequality. Start by using (36) to get h ε iηK (1) . (47) (x) sup R1,n (x, z) ≤ 2d MK V m(1) n h z∈B(x, ε) (0)
We now use the same idea developed to control R1,n (x, z): applying Lemma 2(iii) entails
which, together with (47), yields sup x∈Ω
m(1) (x) n − 1 → 0 as n → ∞ sup (1) x∈Ω µn (x)
h ε i−ηK R(1) (x, z) 1,n = O(1). (1) z∈B(x, ε) h µn (x) sup
(48)
To control the second term, write (1)
sup z∈B(x, ε)
R2,n (x, z) ≤ ∆(log ωn,x )(ε)
sup z∈B(x, ε)
E Kh (z − X)1l{Y >ωn,x ∧ωn,z } .
Note that since ωn,x ∧ ωn,z is either equal to ωn,x or ωn,z , we can write, for all z ∈ B(x, ε) (0) E Kh (z − X)1l{Y >ωn,x ∧ωn,z } ≤ E Kh (z − X)m(0) n (x, X) ∨ E Kh (z − X)mn (z, X) . imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
20
Y. Goegebeur et al.
Recall now (41) and (43) to obtain, for n large enough, uniformly in x ∈ Ω and z ∈ B(x, ε), E Kh (z − X)1l{Y >ωn,x ∧ωn,z } ≤ 2E Kh (z − X)m(0) n (x, X) .
(49)
Finally, using (42) and (49) yields:
(1)
R2,n (x, z) 1 = O(1), sup sup (0) ∆(log ω )(ε) n,x x∈Ω z∈B(x, ε) mn (x) and (38) entails (1)
R2,n (x, z) 1 sup sup = O(1) (0) ∆(log ω )(ε) n,x x∈Ω z∈B(x, ε) µn (x) so that Lemma 2(iv) gives (1)
sup
sup
x∈Ω z∈B(x, ε)
R2,n (x, z) 1 = O(1). ∆(log ωn,x )(ε) µ(1) n (x)
(50)
Applying (46) together with (48) and (50) implies that sup
sup
x∈Ω z∈B(x, ε)
ηK
[ε/h]
which completes the proof of Lemma 4.
µ(1) (z) 1 n − 1 = O(1) (1) ∨ ∆(log ωn,x )(ε) µn (x) (0)
The fifth lemma of this section provides a uniform control of both the difference of two versions of µn (x) for two families of thresholds that are uniformly asymptotically equivalent and the empirical analogue of this quantity. Lemma 5. Assume that (SP ), (A1 ) and (A2 ) hold. Assume that K is a bounded probability density function on Rd with support included in B and that • inf ωn,x → ∞ ; x∈Ω
• hη sup log ωn,x → 0 ; x∈Ω
• sup α(y | x) → 0 as y → ∞. x∈Ω
For an arbitrary family of positive sequences (ρn,x ) such that sup ρn,x → 0 as n → ∞, let x∈Ω
Then
Mn (x)
:=
and Un (x)
:=
E(Kh (x − X)1l{(1−ρn,x )ωn,x ε ≤ 2 exp −κεnhd F (ωn,x | x) . ∀ x ∈ Ω, P ρn,x Mn (x) imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
21
Proof of Lemma 5. We start by noting that F ((1 − ρn,x )ωn,x | X) F ((1 + ρn,x )ωn,x | X) Mn (x) = E Kh (x − X)ρn,x F (ωn,x | X) . − ρn,x F (ωn,x | X) ρn,x F (ωn,x | X) Use then (SP ) and (A1 ) to get, for an arbitrary z ∈ B(x, h), F ((1 ± ρn,x )ωn,x | z) (1 ± ρn,x )−1/γ(z) exp = ρn,x ρn,x F (ωn,x | z)
Z
(1±ρn,x )ωn,x
ωn,x
! α(v | z) dv . v
(51)
Since 0 < γ ≤ γ(z) and sup ρn,x → 0, a Taylor expansion of the exponential function in a neighborhood of x∈Ω
0 yields (1 ± ρn,x )−1/γ(z) 1 1 = ∓ (1 + r1,±,n (x, z)) ρn,x ρn,x γ(z)
(52)
where r1,+,n (x, z) and r1,−,n (x, z) converge to 0 as n → ∞, uniformly in x ∈ Ω and z ∈ B(x, h). Besides, for all u ∈ (−1, 1), ( Z (1+u)ωn,x α(v | z) α(ωn,x | z)| log(1 + u)| if u > 0 dv ≤ (53) ωn,x v α((1 + u)ωn,x | z)| log(1 + u)| if u < 0 so that, because inf ωn,x → ∞, sup ρn,x → 0 and sup α(y | x) → 0 as y → ∞: x∈Ω
exp
Z
x∈Ω
(1±ρn,x )ωn,x ωn,x
α(v | z) dv v
x∈Ω
!
=1+
Z
(1±ρn,x )ωn,x
ωn,x
α(v | z) dv (1 + r2,±,n (x, z)) v
where r2,+,n (x, z) and r2,−,n (x, z) converge to 0 as n → ∞, uniformly in x ∈ Ω and z ∈ B(x, h). Moreover, (53) yields Z 1 (1±ρn,x )ωn,x α(v | z) sup sup dv → 0 as n → ∞. v x∈Ω z∈B(x, h) ρn,x ωn,x
Plugging this together with (52) into (51) and recalling that 0 < γ ≤ γ(z) entails
γ(z) F ((1 − ρn,x )ωn,x | z) F ((1 + ρn,x )ωn,x | z) → 0 as n → ∞. − 1 − 2 ρn,x F (ωn,x | z) ρn,x F (ωn,x | z) x∈Ω z∈B(x, h) sup
sup
Consequently, Mn (x) − 1 → 0 as n → ∞. sup x∈Ω 2E Kh (x − X)ρn,x F (ωn,x | X)/γ(X)
(54)
Recalling (5) and (6), we get
E K (x − X)F (ω | X)/γ(X) n,x h − 1 sup → 0. (0) x∈Ω µn (x)/γ(x)
It only remains to recall (54) and to apply Lemma 2(iv) to obtain γ(x)Mn (x) sup − 1 → 0. x∈Ω 2f (x)ρn,x F (ωn,x | x)
(55)
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
22
Y. Goegebeur et al.
We proceed by controlling Un (x). For every x ∈ Ω, d Un (x) hd Mn (x) d . − 1 > ε = P h Un (x) − h Mn (x) > ε P ρn,x Mn (x) ρn,x
Notice now that if Zn,i (x) := hd Kh (x − Xi )1l{(1−ρn,x )ωn,x 0, Un (x) τn (x)λn (x) . P ρn,x − 1 > ε ≤ 2 exp − Mn (x) 2(1 + λn (x)/3) Applying (55) yields, for n large enough, inf
x∈Ω
τn (x) d nh F (ωn,x
| x)
≥
εf . γkKk∞
(56)
2 Moreover, since Zn, 1 (x) ≤ kKk∞ Zn, 1 (x), it follows that 2 E(Zn, 1 1 1 (x)) ≤ sup ρn,x ≤ sup ρn,x → 0 d M (x) λ (x) εkKk h ε x∈Ω ∞ n x∈Ω n x∈Ω
sup
(57)
as n → ∞. Finally, it holds that 1 τn (x)λn (x) τn (x) inf nhd F (ωn,x | x). ≥ inf x∈Ω 2(1/λn (x) + 1/3) x∈Ω nhd F (ωn,x | x) 2(1 + λn (x)/3) Using (56) and (57) it is then clear that, for all n large enough, if ε > 0 is small enough, there exists a positive constant κ that is independent of ε such that Un (x) ∀ x ∈ Ω, P ρn,x − 1 > ε ≤ 2 exp −κεnhdF (ωn,x | x) . Mn (x) This completes the proof of Lemma 5.
The final lemma is the last step in the proof of Theorem 2. Lemma 6. Let (Xn ) be a sequence of positive real-valued random variables such that for every positive nonrandom sequence (δn ) converging to 0, the random sequence (δn Xn ) converges to 0 almost surely. Then P lim sup Xn = +∞ = 0 i.e. Xn = O(1) almost surely. n→∞
Proof of Lemma 6. Assume that there exists ε > 0 such that P lim sup Xn = +∞ ≥ ε. Since by n→∞
definition lim sup Xn = lim sup Xp is the limit of a nonincreasing sequence, one has n→∞
n→∞ p≥n
∀ k ∈ N, ∀ n ∈ N, P
[
p≥n
{Xp ≥ k} ≥ ε ⇒ ∀ k ∈ N, ∀ n ∈ N, ∃ n′ ≥ n, P
′
n [
p=n
{Xp ≥ k} ≥ ε/2.
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
Uniform asymptotic properties of a nonparametric regression estimator of conditional tails
23
It is thus easy to build an increasing sequence of integers (Nk ) such that Nk+1 −1 [ ∀ k ≥ 1, P {Xp ≥ k} ≥ ε/2. p=Nk
Let δn = 1/k if Nk ≤ n < Nk+1 . It is clear that (δn ) is a positive sequence which converges to 0. Besides, for all k ∈ N \ {0} it holds that Nk+1 −1 Nk+1 −1 [ [ [ P sup δp Xp ≥ 1 = P {δp Xp ≥ 1} ≥ P {δp Xp ≥ 1} = P {Xp ≥ k} ≥ ε/2. p≥Nk
p≥Nk
p=Nk
p=Nk
Hence (δn Xn ) does not converge almost surely to 0, from which the result follows.
Acknowledgements The authors acknowledge the editor and an anonymous referee for their helpful comments which led to significant enhancements of this article. References [1] Aragon, Y., Daouia, A., Thomas-Agnan, C. (2005) Nonparametric frontier estimation: a conditional quantile-based approach. Econometric Theory 21(2), 358–389. [2] Beirlant, J., Goegebeur, Y., Segers, J., Teugels, J. (2004) Statistics of extremes - Theory and applications, Wiley Series in Probability and Statistics. [3] Bingham, N.H., Goldie, C.M., Teugels, J.L. (1987) Regular Variation, Cambridge, U.K.: Cambridge University Press. [4] Chavez-Demoulin, V., Davison, A.C. (2005) Generalized additive modelling of sample extremes. J. R. Stat. Soc. Ser. C 54, 207–222. [5] Chernoff, H. (1952) A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Statist. 23(4), 493–507. [6] Daouia, A., Gardes, L., Girard, S. (2013) On kernel smoothing for extremal quantile regression, to appear in Bernoulli. [7] Daouia, A., Gardes, L., Girard, S., Lekina, A. (2011) Kernel estimators of extreme level curves. Test 20(2), 311–333. [8] Daouia, A., Simar, L. (2005) Robust nonparametric estimators of monotone boundaries. J. Multivariate Anal. 96, 311–331. [9] Davison, A.C., Ramesh, N.I. (2000) Local likelihood smoothing of sample extremes. J. R. Stat. Soc. Ser. B 62, 191–208. [10] Davison, A.C., Smith, R.L. (1990) Models for exceedances over high thresholds. J. R. Stat. Soc. Ser. B 52, 393–442. [11] Einmahl, U., Mason, D.M. (2000) An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theor. Probab. 13(1), 1–37. [12] Gardes, L., Girard, S. (2008) A moving window approach for nonparametric estimation of the conditional tail index. J. Multivariate Anal. 99, 2368–2388. [13] Gardes, L., Girard, S. (2010) Conditional extremes from heavy-tailed distributions: an application to the estimation of extreme rainfall return levels. Extremes 13, 177–204. [14] Gardes, L., Stupfler, G. (2013) Estimation of the conditional tail index using a smoothed local Hill estimator. Extremes, to appear. Available at http://hal.archives-ouvertes.fr/hal-00739454. [15] Girard, S., Guillou, A., Stupfler, G. (2013) Uniform strong consistency of a frontier estimator using kernel regression on high order moments. ESAIM, to appear. Available at http://hal.archives-ouvertes.fr/hal-00764425. [16] Goegebeur, Y., Guillou, A., Schorgen, A. (2013) Nonparametric regression estimation of conditional tails – the random covariate case, to appear in Statistics. [17] de Haan, L., Ferreira, A. (2006) Extreme value theory: An introduction, Springer. [18] Hall, P. (1982) On some simple estimates of an exponent of regular variation. J. R. Stat. Soc. Ser. B 44, 37–42. [19] Hall, P., Tajvidi, N. (2000) Nonparametric analysis of temporal trend when fitting parametric models to extreme-value data. Statist. Sci. 15, 153–167. [20] H¨ ardle, W., Janssen, P., Serfling, R. (1988) Strong uniform consistency rates for estimators of conditional functionals. Ann. Statist. 16, 1428–1449. [21] H¨ ardle, W., Marron, J.S. (1985) Optimal bandwidth selection in nonparametric regression function estimation. Ann. Statist. 13(4), 1465–1481. [22] Hill, B.M. (1975) A simple general approach to inference about the tail of a distribution. Ann. Statist. 3, 1163–1174. [23] Hoeffding, W. (1963) Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30.
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
24
Y. Goegebeur et al.
[24] Hsu, P.L., Robbins, H. (1947) Complete convergence and the law of large numbers. Proc. Nat. Acad. Sci. U.S.A. 33, 25–31. [25] Lemdani, M., Ould-Said, E., Poulin, N. (2009) Asymptotic properties of a conditional quantile estimator with randomly truncated data. J. Multivariate Anal. 100, 546–559. [26] Mack, Y.P., Silverman, B.W. (1982) Weak and strong uniform consistency of kernel regression estimates. Z. Wahrscheinlichkeitstheorie verw. Gebiete 61, 405–415. [27] Nadaraya, E.A. (1965) On non-parametric estimates of density functions and regression curves. Theory Probab. Appl. 10, 186–190. [28] Parzen, E. (1962) On estimation of a probability density function and mode. Ann. Math. Statist. 33(3), 1065–1076. [29] Rosenblatt, M. (1956) Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27(3), 832–837. [30] Silverman, B.W. (1978) Weak and strong uniform consistency of the kernel estimate of a density and its derivatives. Ann. Statist. 6(1), 177-184. [31] Smith, R. L. (1989) Extreme value analysis of environmental time series: an application to trend detection in ground-level ozone (with discussion). Statist. Sci. 4, 367–393. [32] Stute, W. (1982) A law of the iterated logarithm for kernel density estimators. Ann. Probab. 10, 414–422. [33] Wang, H., Tsai, C.L. (2009) Tail index regression. J. Amer. Statist. Assoc. 104(487), 1233–1240.
imsart-aihp ver. 2013/03/06 file: Hill_loiunif_IHP_revised6.tex date: November 21, 2013
