A central limit theorem for associated variables.pdf

A central limit theorem for associated variables Paulo Eduardo Oliveira

Univ. Coimbra, Dep Matematica Apartado 3008, 3000 Coimbra, Portugal

Abstract Using a coupling technique we prove a Central Limit Theorem for associated random variables supposing only the existence of moments of second order, and assumptions that imply some sort of weak stationarity. Supposing the existence of absolute moments of order 3 and without any stationarity condition, we derive a convergence rate, based on a convenient version of the classical Berry-Esseen inequality.

1 Introduction The rate of convergence on the Central Limit Theorem as been a subject of wide interest and for which there exists an extensive literature. An account of results and references may be found in Hall 6] or Rachev 8]. Many of the results obtained refer to the uniform distance and use the Berry-Esseen inequality as a tool to derive the appropriate bounds. A modi cation of the classical procedure was proposed by Wallace 13] by considering formal expansions of the characteristic functions with respect to some reference distribution function, typically some gaussian distribution function. This procedure, called in the literature as Edgeworth expansion, provide bounds that are asymptotically better than those attainable via the classical Berry-Eseen inequality, although requiring the existence of moments of order 4 or 5, depending on the way the errors are controlled. For independent identically distributed random variables it is known that the best attainable rate based on the Berry-Esseen inequality is of order n;1=2 , when only requiring the existence of third order absolute moments. If there exists moments of order bigger than 3, then this convergence rate may be improved. Some results on the convergence rate where also obtained using distances other than the uniform. A discussion about some distances and their relations may be found in Maejima, Rachev 4]. Dropping the identically distributed assumption, it is still possible to retain essentially the same convergence rate although the normalization required becomes dierent. Our interest is to look at this sort of results replacing the independence by an association assumption. Results for this dependence structure were obtained by Wood 14] with respect to the uniform distance keeping a stationarity assumption, and by Suquet 11] with respect to an weighted L2 distance which metrises convergence in distribution. We will restrict ourselves to bounds based on the Berry-Esseen inequality as these still provide, for reasonable sized samples, better bounds than those obtained via Edgeworth expansions. As mentioned before, these produce rates which are asymptotically better, as the error becomes of order n;3=2 , but the constants involved seem to be so large, that Edgeworth expansion based inequalities only become better than their Berry-Esseen Author supported by grant PRAXIS/2/2.1/MAT/19/94 from JNICT and Instituto de Telecomunica c~oes - Polo de Coimbra

1

counterparts for very large samples. Results by Seoh, Hallin 9] show that for the Wilcoxon signed rank statistic the sample size should be at least 128 098 in order to improve the Berry-Esseen bounds. Besides, once the result for independent variables is established, the corresponding one for associated variables is derived easily reproducing exactly the same arguments as in the proof of our Theorem 3.5 below. The bounds for independent variables being classical, we include here a version of the Berry-Esseen Theorem for sake of completeness. We shall now de ne our framework and make more precise some of the mentioned results. Let X1  : : :  Xn be centered random variables, Sn = X1 + + Xn and s2n = ESn2 . If the variables are independent and identically distributed with nite third order moments, then the classical Berry-Esseen Theorem states that sup Fn (x) N1 (x) x j

;

j 

c 3 n

(1)

p

where Fn is the distribution function of s;1 n Sn , Na the distribution function of a gaussian variable 3 with mean 0 and variance a,  = E X1 and 2 = EX12 . The constant c, as proved by Shiganov 10], may be taken less or equal than 0.7915. variables are not identically distributed the upper Pn Ifthe bound in (1) my be replaced by cs;3 , where j = E Xj 3 (see, for example, Galambos 3]). n j =1 j The main tool for proving these upper bounds is the following inequality proved independently by Berry 1] and Esseen 2]. j

j

j

j

Lemma 1.1 Let F1 and F2 be distribution functions with characteristic functions '1 and '2 , respectively, and assume that F2 is dierentiable with f2 = F20 . Then, for every U > 0,

sup F1 (x) F2 (x) j

;

j 

1 Z U  '1 (t) '2 (t)  dt + 24B    t U ;

;U

(2)

where B = sup jf2 (x)j.

This will be used here to derive a convenient version of the Berry-Esseen Theorem, referring to a dierent normalization of the sum Sn. In what follows Fn denotes the distribution function of n;1=2 Sn and we will suppose that there exists a constant c0 > 0 such that

s2n inf n n



c0 :

(3)

2 A convergence rate for independent variables Following the proof of Theorem 10 in Galambos 3] we derive a Berry-Esseen result, which will be used later when studying the convergence rate for associated random variables. We will denote by 'Z the characteristic function of the random variable Z .

Theorem 2.1 Let X1 : : :  Xn3 be independent centered random variables with nite third order absolute moments j = E jXj j . If (3) holds then

  24 Pj j 96 Pj j sup Fn (x) Nn;1 s2n (x) s2nn1=2 + c0  2snn : x ;



2

p

(4)

 Proof : The proof will consist on deriving convenient bounds for 'Sn (tn;1=2 ) exp( ;

and then apply (2). Suppose rst that

1=2

n P 2(  )1=3 j j

t

 j j 

;



t2 s2n ) 2n

3=2 cP 0n 4 j j :

 2 As 'Xj  is the characteristic function of Xj Yj where Yj has the same distribution as Xj and is ;

independent of Xj , a Taylor expansion gives, for some ( 1 1), 2

;

2  2 2 2 2 3 'Xj ( t ) = 1 j t + t3 E(Xj3=2 Yj )3 1 j t + 4 t 3=2 j n n n 6n 3n !   2 t2 3 p

;

;



exp



j j

;

4 t j n + 3n3=2

j

;

j j

as E(Xj Yj ) = 0 Var(Xj Yj ) = 2j2 = 2EXj2 and E Xj Yj 3 6E Xj EYj2 8j . Now ;

j

;

j



j

;

j



E( Xj + Yj )3 j

j

j

0 1  2 Y 2 n  3 X 2 X t t t 4 t 'Sn ( ) = 'Xj ( ) exp @ j2 + 3n3=2 j A n n n j =1 j j  2 2 2 2!  2 2! p



p



j j

;

j



2E Xj 3 + j

j



t sn + t sn = exp 2t sn n 3n 3n taking account of the upper bound for t and (3). Finally, for t in the stated interval,    !  !  ! 'Sn ( t ) exp( t2s2n ) exp 2t2 s2n + exp t2s2n 2 exp t2s2n :  n 2n  3n 2n 3n 

p

;

Suppose now that t

j j 

;

2(

exp

;



;

;

;



;

Pn1=2j )1=3 . From the Taylor expansion j

3  2 t2 'Xj ( tn ) 1 = 2jn + 6tn3=j2  p

;

;

for some ( 1 1), it follows 2

;

  22 3 'Xj ( t ) 1 j t + t 3=2j n 2n 6n p

;



j j

j j + 48 P P 2 = 3  8(  ) 2



j j

j j



7=48

using Holder's inequality and the fact that the j are non negative. It follows that in the interval t 2(Pn1=2j )1=3 the characteristic function 'Xj (tn;1=2 ) is bounded away from zero. On the other j hand j j 

  4 4 6 2 'Xj ( t ) 12 2 j t2 + 2 t j3 n 4n 36n p

;





j4

j P

t j 4n3=2  (P  )1=3 + 144n3=2 j j j 3

 j j

t 3 j 14437n3=2 

 j j

3

j j

!



from which follows that

3 3 3 2 t2  2 t2 log 'Xj ( tn ) = 2jn + t 3=2j +  37 t 3=2j = 2jn + j t 3=2j 6n 144n 2n where  ( 1 1) and j 3 + 3772 ( 1 1). Thus we nd the expansion p

2

;

j j

;

2



j j

j j

;

;

n 3 X 2 X X t log 'Sn ( n ) = log 'Xj ( tn ) = 2tn j2 + t3=2 j  2n j j =1 j p

p

j j

;

P

from what follows, remembering that s2n = j j2 ,

 0 1    'Sn ( t ) exp( t2 s2n ) exp( t2 s2n ) exp @ t 3 X j A 1   n 2n  2n  2n3=2 j 0 3 1 3 X 2 s2 X t t t n) exp( j exp @ 3=2 j A 3=2 p

;

;







;

j j

;

j j

;

j j

2n 2n

2n

j





j

e1=16 exp( t2 s2n ) t 3 X  : 2 2n n3=2 j j ;

j j

From the two upper bounds derived in each interval for t we deduce that

  'Sn ( t ) exp( t2 s2n ) 16 t 3 X j exp( t2 s2n )  n 2n  3n n3=2 j p

;

;

j j



;

3=2 0n . for every t 4cP j j 3=2 0n To nish the proof just use (2) with U = 4cP j to nd j j 

j

P   16 Pj j Z U 2 2 s2 96 n j j t n sup Fn (x) Nn;1 s2n (x) n3=2 ;U t exp( 3n ) dt + c0  2snn3=2 P P 2 s2 96 j j 16 j j Z 1 2 t n t exp( ) dt + p

;







p

;

3n n3=2 ;1 P P 24 j j 96 j j s2nn1=2 + c0  2snn : ;

c0  2snn p





p

Inequality (4) in the case of independent and identically distributed variables gives the n;1=2 rate as usual.

Corollary 2.2 Let X1  : : :  Xn be independent and identically distributed centered random variables 3 with nite third order absolute moments  = E jX1 j . Then   sup Fn (x) ; Nn;1 s2n (x)  242 1=2 +

 n

where 2 = EX12 .

4

96 :  23 n1=2 p

(5)

3 A CLT for associated variables In this section we will present a Central Limit Theorem and a convergence rate for associated random variables. We include here the de nition and a basic inequality which is essential to our proofs. For a better account on association of random variables we refer the reader to Newman 5].

Denition 3.1 The random variables X1 : : :  Xn , are associated if, for every f g coordinatewise increasing real valued functions dened on IRn ,

Cov(f (X1  : : :  Xn ) g(X1  : : :  Xn )) 0: 

The random variables Xn  n 2 IN, are associated if, for every n 2 IN, the variables X1  : : :  Xn are associated.

A nice tool for proving Central Limit Theorems is the Newman's inequality, which provides a control on the characteristic functions by the covariance structure of the variables.

Theorem 3.2 (Newman 5]) Let X1  : : :  Xn be associated random variables with nite second order moments. Then

  n n n Y X E exp(i X  1  tj Xj ) E exp(itj Xj ) 2 tj tk Cov(Xj  Xk ):   jk=1 ;

j =1



j =1

j

(6)

j

j 6=k

According to this inequality, after some adequate control on the covariances, the variables may be replaced by another ones with the same distributions but independent. This coupling will be the basis of the results we will prove for associated variables. We begin with a Central Limit Theorem for which the proof is an adaptation to a somewhat simpler framework of the technique that was used in the proof of Theorem 9 in Oliveira, Suquet 7].

Theorem 3.3 Let Xn n IN, be associated centered random Pjkvariables. For each k IN, let m be 2

2

the largest integer less or equal to n=k and dene Ykj = i=(j ;1)k+1 Xi  i = 1 : : :  m. Suppose that the following conditions are veried

s2n = lim 1 ES 2 = 2 > 0 lim n!+1 n n!+1 n n m 1X 2 EYkj = ak lim m!+1 m j =1 n Z 1X

> 0 n i=1

8

1=2 g fjXi j>n

and

Xi2 d P

;!

(7)

ak = 2 lim k!+1 k

(8)

0:

(9)

Then n;1=2 Sn converges in distribution to a gaussian random variable with mean 0 and variance 2 .

Proof : The blocks Yk1 : : :  Ykm being increasing functions of the Xn  n IN, are associated, so the method of proof will consist on approximating the sum Sn by the sum of these blocks and 2

5

reason as if these blocks were independent, based on (6). First we shall look at

  'Sn ( t ) 'Smk ( t ) t Var1=2 (Sn Smk ) = n mk 2 mk

2 n X X 1 1 1 4 E(Xi Xj )+ t E(Xi Xj ) + n n p

p

;

 j j

 j j

p

ij =1

; p

 j j

mk

ij =mk+1

mk X n 1 X 1 + n n p

2 t4

;

p

i=1 j =mk+1

31=2 1  E(X X )5 i j

mk

; p



n mk X X k n mk 1 E( X X ) + i j n n mk ij =mk+1 E(Xi Xj )+ nm( mk + n)2 ij =1 ;

p

p

;

31=2

mk X n X k + E(Xi Xj )5 n m( mk + n) i=1 j=mk+1 which converges to zero according to (7). Next, we approach 'Smk by the product of the characteristic functions of the blocks Ykj , using (6). According to (7) and (8), there exists a constant c > 0 such that for m (thus n) large enough,   m n Y   1 X t t t2 E(Y Y ) ct2 (2 ak ): ' ( ) ' ( ) Xj  Smk mk k mk  2 ij =1 mk ki kj j =1 p

p

p

p

p

p

;





;

i6=j

From this point on we may reason as if the blocks Yk1  : : :  Ykm were independent (to be completely accurate we should introduce a new set of random variables with the required properties, but we will not do so to keep the notation simpler). That is, we must now check that these blocks verify a Central Limit Theorem, what we will accomplish by verifying the Lindeberg condition. Put P ak , so m 2 ;1 2 . From (8) and Lemma 4 in Oliveira, Suquet 7], it follows r 2 rn = (mk) j=1 EYkj n k the Lindeberg condition for the blocks Yk1  : : :  Ykm writes m Z X 1 Y 2 dP 0 ;!

j =1

p Ykj j>"rn mkg mk kj

;!

fj

where " > 0 is xed. From Lemma 4 in Utev 12] this integral is less or equal than Z jk m mk Z X 1X o X2 d P 1 X n o X2 d P n

m j=1 i=(j;1)k+1

jXi j> 2"

p ak k

p

m k

i



m j =1

jXi j> 2"k

p ak pmk

i

k

which converges to zero when n + (k xed) according to (9). Summing up the inequalities above we have, for each k xed, that there exits some constant c > 0 such that ;!

1

  2 2 t ; t 2   lim sup 'Sn ( n ) e ct2 (2 akk ):  n!+1 p

;



;

Finally, letting k + and taking account of (8) it follows 'Sn ( ptn ) e; t 2 , which proves the theorem. We may state an immediate corollary for the case of stationary random variables. 2 2

;!

1

;!

6

Corollary 3.4 Let Xn n IN, be associated, weakly stationary centered random variables verifying (7) and (9). Then n and variance 2 .

;1=2

2

Sn converges in distribution to a gaussian random variable with mean 0

Based on the bounds derived in the previous section and in inequality (6) we may prove a convergence rate for the Central Limit Theorem for associated variables. A result on this setting has been proved by Wood 14] who, supposing that the variables are stationary, obtained for n = m k, 3 4 2 2 sup Fn (x) Na (x) 16sn m(2a 3sn ) + 3E3 S1n=2 snm x 9E Sn P Cov(X  X ). The stationarity assumption was dropped by Suquet 11] where a2 = EX12 + 2 1 1 k k=2 considering an weighted L2 distance instead of the supremum distance. j

;

;

j 

j

j

j

j

Theorem 3.5 Let Xn  n IN, be associated centered random variables. For each k IN let m be P jk 2 = EY 2 the largest integer less or equal to n=k. Dene Ykj = i=(j ;1)k+1 Xi  i = 1 : : :  m, kj kj P 2 and kj = E Ykj 3 . Suppose that inf m2IN m1 m  c > 0 . Then, for n = m k , 0 j =1 kj   P P   2 2 m3 X   24 j kj 96 j kj s c 1 n 0 2   sup Fn (x) Nn;1 s2n (x) (P  )2  n m kj  + 1=2 P 2 + 2 )1=2 x  m j kj c0  2m(Pj kj j kj  2

j

2

j



;



;

Proof : Using (2) we have   1 Z U Fn (x) Nn;1 s2n (x)  ;U ;



p

j

(10)

 2 2   'Sn ( ptn ) e; t 2snn    d P +  242sn U : t n   p

;

p

P Note that p1sn Sn = p1m mj=1 Ykj , so the integral may be decomposed in the sum

Z U  E exp(i ptm Pj Ykj ) Qmj=1 E exp(i ptm Ykj )    dt+ I1 + I2 + I3 = t ;U    2 ) Z U  Qmj=1 E exp(i ptm Ykj ) exp( 2tm2 Pj kj  dt+  +  t ;U    t2 s2  Z U  exp( 2tm2 Pj kj ; 2nn  2 ) e  dt: +    t ;U    The third integral is bounded by using the inequality e;t e;s  t s , so   Z   2 2 2 U X X     2  2  t dt = U2  snn m1 kj I3 12  snn m1 kj   :   ;U  ;

;

;

;

;



;

j

;

j j

7

 j

;

;

j

j

The integral I1 is bounded using (6),

I1



  Z m 2 2 U X X  U  sn 1 2  : 1 Cov( Y  Y ) t dt = kj kl 2m jl=1 2  n m j kj  ;U j j

;

j 6=l

3=2 0m where we may choose U = 4cP according to the proof of Theorem 2.1. Finally, to bound I2 we j kj use (4) to nd P P 96 j kj 24 j kj I2 m1=2 P 2 + P 2 )1=2  j kj c0  2m( j kj and we get (10) by summing up these bounds. P 2 c for some constant Note that the conditions of Theorem 3.3 imply both that inf m m1 j kj 0 2 s 1 P 2 n c0 > 0 and that n m j kj 0 according to (7) and (8), respectively. The extra term appearing in (10) when comparing to what appears in the independent case was already present in the stationary version of Wood 14] who also provided an example showing that this term can not be avoided, in general, and is responsible for a possibly arbitrarily slow convergence rate. If we wish to derive an Edgeworth expansion based inequality, we need only to replace the bound of I2 by the convenient one, as the integral I1 remains the same, and I3 is modi ed in a way that does not aect its nal treatment, as in this case instead of Nn;1 s2n (x) we will have this function plus a convenient perturbation. Finally note that Newman's inequality (6) holds in a somewhat more general setting, as association is not really required. Indeed, it is easy to check that the proof of this inequality only requires that the variables are linear positive quadrant dependent (LPQD), thus as this inequality was the main tool to prove our results, they also hold for LPQD random variables. We refer the reader to Newman 5] for the de nition of LPQD and the proof of (6). The upper bounds (4), (5) and (10) derived here seem to be of interest only for large samples. Indeed, even in the independent and identically distributed case the upper bound (5) is, for reasonable values of n, larger than 1, thus rendering the bound useless. Some numerical evaluations suggest that there is a big gap to be lled by optimizing the constants involved. This question was not addressed here. The lack of optimization evidently reects also on the bounds for associated variables, as these are derived by adding a convenient term to the independent version. The following tables, containing simulated values, con rm these comments. The rst table reports simulated results for independent variables with distribution 95 ; 89 + 49 109 compared with the upper bound (5), and to what we nd when the variables are "slightly" associated. By "slightly" associated we mean that the variables are associated but Xi and Xk are in fact independent whenever k i 2, thus leaving us to deal only with the covariances of Xi and Xi+1 . We will denote this by saying that the variables are associated(2). p





;

;!

j

n

100 200 300 500

;

j 

upper bound (5) independent associated(2) 2.015192 0.093087 0.042001 1.424956 0.062520 0.058189 1.163472 0.036359 0.047569 0.901221 0.038086 0.042813

It is interesting to note that the observed upper bounds for associated variables seem to behave not very dierently from the ones observed for independent variables. This seems to be connected 8

to the fact that the variables are only "slightly" associated. If we call the variables associated(m) whenever they are associated and Xi  Xk are in fact independent if k i m, the eect of m is illustrated on the table below for variables constructed from variables with the distribution mentioned above, for the choice n = 100, j

;

j 

m 3 4 5 10 20 associated(m) 0.041783 0.102225 0.069066 0.376153 0.495995 The last values of this table show the eect of the correction term for associated variables appearing in (10). The same eect is observed if we use absolutely continuous variables. The next table reports simulated values for exponentially distributed variables with parameter  and n = 100,

 = 0:25

=1

m independent associated (m) m independent associated (m)

2 3 4 5 10

0.047228 0.064108 0.045430 0.031737 0.029662

0.047049 0.069020 0.099881 0.093499 0.162102

2 3 4 5 10

0.043543 0.037238 0.036112 0.038310 0.026067

0.024927 0.045861 0.059330 0.107544 0.145102

Note that the bound (5) is 10.32186 when  = 0:25 and 4.788111 when  = 1 due to the fact that the absolute third order moments are quite large.

References 1] Berry, A. C., The accuracy of the Gaussian approximation to the sum of independent variables, Trans. Amer. Math. Soc. 49 (1941), 122-136. 2] Esseen, C. G., Fourier analysis of distribution functions. A mathematical study of the LaplaceGaussian law, Acta Math. 77 (1945), 1-125. 3] Galambos, J., Advanced probability theory . Marcel Dekker, 1988. 4] Maejima, M., Rachev, S. T., An ideal metric and the rate of convergence to a self-similar process, Ann. Probab. 15 (1987), 702-727. 5] Newman, C. M., Asymptotic independence and limit theorems for positively and negatively dependent random variables, in Inequalities in Statistics and Probability , IMS Lect. Notes Monographs Series, 5 (1984), 127-140. 6] Hall, P., Rates of convergence in the central limit theorem . Pittman, 1982. 7] Oliveira, P. E., Suquet, Ch., An L2 0 1] invariance principle for LPQD random variables, Port. Mathematica 53 (1996), 367-379. 8] Rachev, S. T., Probability metrics and stability of stochastic models . Wiley, 1991. 9] Sheo, M., Hallin, M., When does Edgeworth expansions beat Berry-Esseen? Numerical evaluations of Edgeworth expansion, J. Statist. Plann. Inference 63 (1997), 19-38. 9

10] Sighanov, I. S., Re nement of the upper bound of a constant in the remainder term of the central limit theorem, in Stability Problems for Stochastic Models (Eds. V. V. Kalashnikov, V. M. Zolotarev), Vsesoyez Nauk Issled, 109-115, 1982. 11] Suquet, Ch., Distances euclidiennes sur les mesures signees et application a des theoremes de Berry-Esseen, Bull. Melg. Math. Soc. 2 (1995), 161-181. 12] Utev, S. A., On the central limit theorem for '-mixing arrays of random variables, Th. Prob. Appl. 35 (1990), 131-139. 13] Wallace, D. L., Asymptotic approximations to distributions, Ann. Math. Statist. 29 (1958), 635-654. 14] Wood, T. E., A Berry-Esseen theorem for associated random variables, Ann. Probab. 11 (1983), 1042-1047.

10