Most Stringent Test for Location Parameter of a ... - Google Sites

Most Stringent Test for Location Parameter of a Random Number from Cauchy Density Atiq-ur-Rehman †

Asad Zaman

Email: [email protected]

E-mail: [email protected]

International Institute of Islamic Economics International Islamic University, Islamabad

International Institute of Islamic Economics International Islamic University, Islamabad

Abstract: We

study

the

test

for

location

parameter

of

a

random number from Cauchy density, focusing on point optimal

tests.

We

compute

critical

develop

values

and

analytical power

technique

curve

of

a

to

point

optimal test. We study the power properties of various point

optimal

tests.

The

problem

turned

out

to

be

different in its nature, in that, the critical value of a

test

determines

the

power

properties

of

test.

We

found that if for given size α and any point θm in alternative space, if the critical value of a point optimal test is 1, the test optimal for that point is the most stringent test.

Keywords: Cauchy density, Power Envelop, Location Parameter, Stringent Test

____________ †

Correspondence Address: F-11, Kuwait Hostel, International Islamic University, old campus, Islamabad, Pakistan. Phone: +92-300-5569022, +92-334-5336263

1.

Introduction Cauchy Distribution (or Lorentz Distribution in terminology of Physicists) has its

applications in Physics, Spectroscopy and in Statistics. It is used to measure the sensitivity of an estimator\test statistics to normality assumptions due to its heavier tails which are extremely unlikely under normality assumptions.

Since the moments are not defined for Cauchy distribution, the tests/estimators based on asymptotic properties are not appropriate while studying the properties of Cauchy distributions. Statistician had discussed some statistical properties of the distribution. The principal focus of studies is on goodness of fit to the Cauchy distribution. The studies of Rublyk (1997), Rublyk (1999) Gurtler & Henze (2000), Rublyk (2001), Rublyk (2003) and Matsui & Takemura (2005) are the examples. Other studies explore parameter estimation, & behavior of Likelihood function etc. See Copas (1975), Besbeas & Morgan (2001) and Lawless (1972) for example.

For an observation X from Cauchy distribution, we explore the tests for location parameter of the distribution with focus on point optimal (Neyman-Pearson) tests & develop method to find out the most stringent test. Lehman (1986) discussed that for location parameter of an observation X from Cauchy density, UMP test does not exist. Obviously the Stringent test is the feasible choice in absence of UMP test if we can find it. We study the power properties of a large number of NP1 tests and develop technique to find out power envelop and power curve of a point optimal test of given size, and hence to find shortcoming of test. By luck of draw, the problem turned to be different in its nature, in that, we are able to construct the power envelope analytically. Similarly we can trace power curves of a point optimal test as well. This made possible the computation of shortcoming of a large number of point optimal tests. The techniques are discussed below. ___________________________________ 1

The terms ‘point optimal test’, ‘Neyman-Pearson test’ and abbreviation ‘NP test’ are used interchangeably in the paper. Furthermore Tθ is mathematical shorthand for NP test optimal for point θ.

2.

The problem Let Ca(θ) denotes the Cauchy distribution, with location parameter θ and unit

scale parameter & X ~ Ca(θ) i.e. 1

f(X|θ)

=

π 1 + ( X − θ )



2

We are interested in testing the Hypothesis H0: θ = 0 versus H1: θ > 0. The Null space Θ0 = (0) and the alternative space Θ1 = R+.

3.

Point Optimal Test & Power Envelope

3.1

Test Statistics for Point Optimal Test & Critical Values

Given the problem, Neyman Pearson Lemma allows us to construct test optimal for a point θ ∈ Θ1. Let L(X,θ ) denote density at X given the location parameter θ than the test statistics is: L( X , 0 , θ )

L( X , θ ) L( X , 0)

For test of size α the critical values can be computed by assuming H0 is true & finding Cα(θ) such that  L( X , θ ) ≥ C ( θ )  α  α  L( X , 0) 

[1]

P

Where P(.) denotes probability & the subscript α refers to size of the test. Now For Cauchy density, L( X , θ )

 π 1 + ( X − θ ) 2 

L( X , 0)

π( 1 + X2)

−1

−1

2

1+ X

1 + (X − θ)

2

Given an observation X, we reject the null if L(X,0,θ) > Cα(θ) and accept otherwise. This procedure maximizes power at θ ∈ Θ1. By changing θ, we get different test statistics optimizing power for that new point. Now, for θ ∈ Θ1, we plot L(X,0,θ) for different values of X. A Typical graph is shown in fig 1. Remark: For the figures labeled ‘typical graph’, there is no significance of scale especially if it is not visible. Just general shape of objective function is described by the graph. Fig 1: A Typical Plot of L(X,0,θ) 3 2 1 0 -15

-10

-5

0

5

10

15

The solid line denotes the values of L(X, 0,θ). The graph cuts the line y =1 (the dotted line) at one point and converges to 1 as X converges to ± ∞

The graph of L(X,0,θ) is increasing in a finite interval and decreasing otherwise. L(X,0,θ) = 1 only at X= θ/2. Little mathematical formulation verifies that for X>θ/2, L(X,0,θ) is greater than 1 & smaller than 1 otherwise. For finite θ, the graph of L(X,0,θ) converges to 1 if X → ± ∞. Below we discuss how to find out the critical value Cα(θ) analytically.

Theorem 1: Given an NP test of size α , optimal for alternative θ ,

Cα ( θ )

2

(8 + 8k + 4θ 2 + 4θ 2 k)2 − 4(4 + 4k + kθ4 + 4kθ2)(4 + 4k) 4 2 2( 4 + 4⋅ k + kθ + 4⋅ kθ )

2

8 + 8k + 4θ + 4θ k +

Where Cα(θ) be defined in [1] above and tan ( π θ ) 2

k

Proof: Using definition of Cα(θ),given size of test = α & assuming H0 is true, Cα(θ) is the value such that  L( X , θ ) ≥ C ( θ )  α  α  L( X , 0) 

[2]

P

To find Cα(θ), solving the equation, 2

1+ X

1 + ( X − θ)

[3a]

C α (θ )

2

For simplicity, denote Cα(θ) by C, we can rewrite [3a] as: 2

X ( 1 − C) + 2CXθ + 1 − C − Cθ

2

0.

[3c]

Supposing l, m being the roots of equation [3b], since l, m specify the range where we reject the Null. So assuming the Null is true, integration of the density function over range (l, m) must be equal to size of test α i.e.

⇒

⌠   ⌡l

m

1

 tan − 1( m) − tan − 1( l)  α

π

⇒

(

1

π 1+ X

m−l 1 + lm

)

2

dx

α

tan ( π α )

[3c]

Now l, m being roots of equation [3b] quadratic in X, therefore: l+m

−2

Cθ 1−C

[3d]

1 − C − Cθ

lm

[3e]

2

1−C

Solving [3c], [3d], and [3e] for C yield: 2

2

8⋅ C + 4⋅ C ⋅ θ − 4 − 4⋅ C

2

(2 − 2⋅ C − Cθ2)2 Assuming

k

[4a]

tan ( π α )

tan ( π θ )

[4b]

2

And solving [4] for C gives: C=

(8 + 8k + 4θ + 4θ k ) + 4(4 + 4 K + kθ 2(4 + 4 K + kθ + 4kθ )

8 + 8k + 4θ 2 + 4θ 2 k ±

2

2

2

4

4

+ 4kθ 2 )(4 + 4k )

2

This equation gives two values of C corresponding to P[L(X,0,θ) >C] = α

and P[L(X,0, θ )1, power of Tθ converges to zero for the large β.

2.

If Cα(θ) ≤1, power of Tθ converges to 1 for the large β.

Proof: Remember we reject the null if L(X,0,θ) > Cα(θ) a) If Cα(θ) >1, than roots of equation [3b] lies on RHS of point X=θ/2. Let l, m be the roots of quadratic equation [3b], than the roots determine boundaries of rejection region. It can be shown unique maxima of L(X,0,θ) lies inside interval (l, m). Therefore the rejection region is bounded by roots l, m. Than for some θ ∈ R+, power of test is just integration of density function f(X|β) on interval (l, m). Fig 2: Rejection region when Cα(θ) >1 Rejection Region

2.5

y = Cα(θ) 2 1.5 1 0.5 0

-5

-2.5

0

Acceptance Region

2.5

5

Value of L(X,0,θ) is large than Cα(θ) for points between root of L(X,0,θ) = Cα(θ) which is the rejection region

Now Let R(T, β) denote power of test T at β & test Tθ is such that Cα(θ) >1, than

(

θ

lim R T , β

)

=

β →∞

=

⌠ lim  β →∞  ⌡l

m

1

lim

π

β →∞

1

π1 + ( x − β )

dx

 ( tan − 1) ( m − β ) − tan − 1( l − β )  .

=0

b) We divide proof of [b] in to two parts i) If Cα(θ) 1, therefore according to theorem 2, the test has zero power for the large alternatives. Whereas, as we will show later, there is a certain class of test for which Cα(θ) 1.

Proof: We can rewrite expression for Cα(θ) derived in theorem 1 as: Cα ( θ )

2

2

(

8 + 8k + 4θ + 4θ k + 4θ ( 1 + k) 2 + θ

(

4

2 4 + 4⋅ k + kθ + 4⋅ kθ

)

)

[6]

2

2

Let θ be so small that it higher powers are negligible than the expression reduces to:

Cα ( θ )

[6b]

8 + 8k + 4θ 2( 1 + k) 2( 4 + 4⋅ k)

Obviously, for θ > 0 the numerator of expression on right hand side of [6b] is greater than denominator. Hence Cα(θ)>1.

The Lemma gives crucial information about the power curve of LMP test, in that, the critical value of LMP is larger than 1 & thus according to theorem 2, have zero power for large alternatives. Therefore, LMP has 100% shortcoming. So LMP should never be used if we don’t have precise information of θ. Empirical results shows that LMP is not good even outside a small neighborhood of the point for which it is optimal. A test of size 5% optimal for θ = 0.5 has only 1% power for θ = 4 and for θ = 15, its power is zero up to 3 decimal places.

Fig 4: Performance of LMP test 0.8

0.4

0.6

0.4

0.2

0.2

0

0 0

2.5

5

0

2.5

5

The solid lines in the two graphs represent the power envelope whereas the dotted lines represent power curves of test optimal for θ = 0.1. Size of test is 5% for the left panel and 1% for the right panel. Immediate decline in power of LMP is obvious in the two panels.

So having no prior information of the true parameter, the choice of LMP test is hazardous.

4.2

Berenblutt & Webb type test In their discussions of tests for autocorrelation, Berenblutt & Webb (1973)

recommend to use another extreme strategy, to maximize power for largest possible alternative. The alternative space we have is R+ ranging from 0 to ∞, therefore we have maximize power for ∞ . One of the result of following Lemma is the proof that; β → ∞ ⇒

Cα(θ) → 0 < 1, and thus (by theorem 2) the test has 100% power for large

alternatives.

Lemma 2: For Cα(θ) defined in [2], following results hold. lim Cα ( θ )

R1

β →∞

0

lim Cα ( θ ) θ

R2

β →∞

R3

lim

β →∞

2 Cα ( θ ) ⋅ θ

0

(

2 + 2⋅ k + 2⋅ ( 1 + k ) ⋅ 2 + θ

)

2

k

Where ‘k’ is defined in [4b] above. Proof: Again for simplicity, denote Cα(θ) by C than for R1 lim Cα ( θ )

β →∞

2

=

lim

θ →∞

2

(

4

2 4 + 4⋅ k + k θ + 4⋅ k θ 2

2

8 + 8⋅ k + 4⋅ θ + 4⋅ θ ⋅ k

=

lim

θ →∞

θ

4

2⋅

+

(4 + 4⋅ k + k θ θ

= .

0

(

8 + 8k + 4θ + 4θ k + 4θ ( 1 + k ) 2 + θ

4

)

2

)

2

(

4⋅ θ ⋅ ( 1 + k ) ⋅ 2 + θ

θ

4

+ 4⋅ k θ

)

2

4

)

2

Similarly for R2 3

3

8θ + 8⋅ kθ + 4⋅ θ + 4⋅ θ ⋅ k lim Cα ( θ ) θ

=

β →∞

θ

lim θ →∞

4

2⋅

(

2

+

4⋅ θ ⋅ ( 1 + k) ⋅ 2 + θ

(4 + 4⋅ k + kθ4 + 4⋅ kθ2) θ

θ

)

2

4

4

= 0 And for R3: 2

2

4

4

8θ + 8⋅ k θ + 4⋅ θ + 4⋅ θ ⋅ k lim Cα ( θ ) θ

2

β →∞

=

θ

lim

θ →∞

4

2⋅

=

+

(4 + 4⋅ k + k θ θ

(

3

4

4⋅ θ ⋅ ( 1 + k ) ⋅ 2 + θ

+ 4⋅ k θ

2

)

θ

)

2

4

4

2 + 2k + 2 1 + k k

Hence the results

Solving equation [3] for X & using results of Lemma 2, one can find out the range for which B&W type test rejects Null. It is observed that power curve of B&W type test is increasing in β. So, obviously the tests of this type are preferable to LMP in that they have smaller shortcoming than LMP.

Practically, if we design test maximizing power for a very large alternative, it has negligible discrimination between Null and an alternative in neighborhood of Null. Therefore this type of test is expected to have smaller power for alternatives close to null. Therefore such test possess heavy shortcoming (But not larger than that of LMP). For different sizes, we computed the shortcoming of B&W type. Figure below shows the plot shortcoming of B&W type test versus its size.

Fig 5: Shortcoming of B&W type tests 1

0.75

0.5

0.25

0 0

0.1

0.2

Size of test represented on X-axis & shortcoming of B&W on Y-axis. Shortcoming of B&W type test is less than 1, but still massive

For size 1%, the test has 96% shortcoming, which is obviously disappointing, and for size 20%, the shortcoming is 40%. That is, we still have a lot of loss in using this type of test. As we will soon show, there is an intermediate class of tests other than B&W type tests, all of which has 100% power for infinitely large alternatives. Is it possible to choose a test in that class which has smaller shortcoming? Next we investigate the same question.

4.3

Intermediate choice of Alternative Cases where both LMP and B&W type tests perform poorly, it is natural to

choose an intermediate alternative to optimize power for. Several strategies are recommended by different writers to choose a suitable intermediate. See Efron (1975), Davies (1969) Fraser et al (1976) & King (1985) for example.

As we discussed, critical value of a test is ‘critical’ in determining the power properties, choosing an intermediate value, we must have to consider the behavior of critical value. The typical plot of Cα(θ) follow the pattern given in fig below.

Fig 6: A typical plot of Cα(θ) 3.5 3 2.5 2 1.5 1 0.5 0 0

5

10

15

20

Cα(θ) is large than 1 for a finite range and than decreases continuously

We see that for test of fixed size, there exist a θm ∈ R+ with Cα(θ) = 1 and Cα(θ) >1

θ ∈ ( 0, θm)

Cα(θ) 1, an NP test optimal in this interval is the test with maximum shortcoming. Whereas for choice of θ ∈ ( θm , ∞) the test is certain to have 100% and power for very large alternative. Now first step in an analysis is to search θm so that we can avoid choosing a test with 100% shortcoming. Luckily, we are able to compute critical value of an NP test analytically as well as we can analytically compute power of an of NP test at any point. This makes computation of exact power envelope, power curve & shortcoming of a test possible. We compute the shortcoming of a large number of NP tests. E.g. for size of test > 5% we compute shortcoming of test optimizing power at each point of the vector (0, 0.5…10000) to choose a feasible alternative which minimizes the shortcoming (The algorithm given appendix). We found that for the test optimal for θm (θm defined above), shortcoming is minimum. Let Tθ denote test optimal for θ and S(Tθ) denote the shortcoming of Tθ . For fixed size the typical plot of Tθ versus S(Tθ) is given in fig below:

Fig 7: Shortcoming of NP Tests of fixed size 3.5 3 2.5 2 1.5 1 0.5 0 0

5

10

15

Dotted line indicates shortcoming of the test, whereas the solid line is for value of Cα(θ). Shortcoming suddenly drops down for Cα(θ) = 1 and then increases monotonically with θ

Tests optimal for any point in ( 0, θm) have 100% shortcoming, which is according to our expectation. Than shortcoming suddenly drops down for Tθm and than increases monotonically with θ and converges to shortcoming of B& W type tests. The pattern of plot remains same for different size of test.

There are some other strategies to choose a feasible alternative, e.g. using the information matrix etc. But in our case, any strategy for choosing a feasible intermediate point optimal test falls into one of the two categories discussed & therefore there is no need to study them separately.

5.

Recommended test

Studying power properties of a large number of point optimal tests, we reach the conclusion that Tθm possesses smallest shortcoming, & hence is the most stringent test. Therefore, given the size of test α, the problem of searching for a stringent test reduces to

search for a point θm in R+ such that Cα(θm) = 1. Having an analytical formula to compute Cα(θ), this is worth spending few moments on computer. In table 1, we tabulate θm for different sizes. It was observed that Tθm has significantly smaller shortcoming than

that of B&W. Below we plot the shortcoming of the two types (Tθm and B&W) of test for size 1-10%. Fig 8: Comparison of Tθm and B&W test 1.00

0.75

0.50

0.25

0.00 0%

2%

4%

6%

8%

10%

12%

Dotted line represent shortcoming of Tθm which is much smaller than shortcoming of B&W type test represented by solid line.

5.1

Other Characteristic of Recommended Test

5.1.1

Effect of size of test on shortcoming of Tθm It can be seen from fig 8 that shortcoming of Tθm increases by a large factor if we

reduce size of test by 1%. Reducing size of test from 5% to 4% increases the shortcoming from 51% to 59%. Hence the arbitrary choice of size of test is risky. It’s better to choose the larger size since it cuts down shortcoming by multiple factor.

5.1.2

Abuse of Tθm, consequences In absence of any prior belief about alternative, Tθm is the only feasible option,

since it possess minimum shortcoming. But suppose we have solid reasons to believe that

the alternative is some β ≠ θm. So, one may think to use an NP test optimal for β to get a larger power. There are two possibilities If β < θm than using Tβ may be beneficial by a large amount in the neighborhood of β but as we had discussed, the test may have zero power if the true parameter is larger than our belief. We are at risk of 100% shortcoming. If β > θm than surprisingly, Tβ is not much beneficial even in the neighborhood of β. We computed the maximum shortcoming of Tθm of different sizes on the range (θm, ∞) which is tabulated below. It turns out that the maximum difference between power envelope & power curve of Tθm is negligible in interval (θm, ∞).

Table 1:

Size of test

θm

SC of Tθm on (θm, ∞)

1%

63.642

0.0009

2%

31.790

0.0018

3%

21.158

0.0027

4%

15.832

0.0036

5%

12.628

0.0045

6%

10.485

0.0054

7%

8.948

0.0063

8%

7.790

0.0072

9%

6.885

0.0081

10%

6.156

0.0091

For test of any size, the maximum advantage we can have to maximize power beyond θm is less than 1%.Hence maximizing for a larger alternative does not have any practical benefit even at the point for which we maximize power.

Figure 9: Abuse of Tθm 100%

100%

75%

75%

50%

50%

25%

25%

0%

0% 0

10

20

30

0

10

20

30

Solid lines in the two graphs represent power envelope and lines with dashed lines represent power curves of Tθm. The Dotted line in left panel represent power curve of an NP test optimal in (0, θm). The test has significant gain over Tθm for a small range, but zero power for large alternatives. In right panel, dotted line denote point wise shortcoming of Tθm which is negligible for range (θm, ∞) Whereas, optimizing beyond θm will always increase shortcoming. Optimal choice in this case is again Tθm.

Solid lines in the two graphs represent power envelope and lines with cross marks represent power curves of Tθm. The Dotted line in left panel represent power curve of an NP test optimal in (0, θm). The test has significant gain over Tθm for a small rang, but zero power for large alternatives. In right panel, dotted line denote point wise shortcoming of Tθm which is negligible for range (θm, ∞). Whereas, optimizing beyond θm will always increase shortcoming. Optimal choice in this case is again Tθm.

Acknowledgment The first author likes to acknowledge the contribution of Higher Education Commission of Pakistan, which is sponsoring him for higher education through indigenous fellowship scheme.

6. Appendix 6.1 Computation of power curve of a point optimal test & power envelope & shortcoming of a test.

Suppose we want to trace power curve of a test of size α which optimizes power at θ ∈ R+. A lot of computational burden is released by the analytic formula in our hands. Even without computing the actual test statistics, we can compute the critical value by the formula derived in theorem1. Putting the critical value in [3b] and solving for X, yield the roots l, m. Theorem 2 allows us to specify the range for which value of actual test statistics will be larger than critical value. That is if critical value ‘Cα(θ) ’ is larger than 1, the rejection region is bounded by the roots i.e. L(X,0,θ) > Cα(θ) if l < X < m Now to compute power of test for some θ ∈ R+, the power of test is probability that random variable X lies in specified range.

(

θ

lim R T , β

β →∞

)

=

=

P [l ≤ X ≤ m ] ⌠   ⌡l

m

1

π1 + ( x − β )

dx

Now once we have specified range for which null should be rejected for Tθ, The locus of R(T, β), gives the power curve. The power curves discussed in paper were traced by computing power at β = 0, 0.5 …10000 for size of test > 5% and at β = 0, 1…50000 for size of test < 5%. Computing R(Tβ , β) yield power envelop for β = 0, 0.5…10000. For any test T, R(Tβ , β) - R(Tθ, β), β = 0, 0.5 …10000 yield vector of point wise shortcoming, and maximum of this vector is the shortcoming of the test.

6.2 References [1]

Berenblutt I. & Webb G.I. (1973) ‘A new test for autocorrelated errors in linear

regression model’ Journal of Royal Statistical Society, series B34

[2]

Copas J.B (1975). ‘On Unimodality of the Likelihood for Cauchy distribution’

Biometrika, 62

[3]

Davies R.B. (1969), ‘Beta-optimal tests and an application to the summary

evaluation of experiments’ Journal of Royal Statistical Society, series B31

[4]

Efron B. ‘Defining curvature of a statistical problem’ Anals of statistics, 3(6)

[5]

Fraser D.A.S. Guttman I. & Styan G.P.H. (1976), ‘Serial correlation and

distribution on a sphere’ Communications in Statistics

[6]

King M.L. (1987) ‘Toward a theory of point optimal testing’ Econometric

Reviews

[7]

Gurtler N. & Henze, N. (2000). ‘Goodness-of-fit tests for Cauchy distribution

based on the empirical characteristic function’ Ann. Inst. Statist. Math, 52

[8]

Lawless J. F. (1972). ‘Conditional confidence interval procedure for location &

scale parameters of the Cauchy and logistic distribution’ Biometrika, 52

[9]

Lehman E.L. (1986). ‘Testing Statistical Hypothesis’, 2nd Edition, John Wiley &

Sons, Inc

[10]

Matsui M. & Takemura A.(2003). ‘Empirical Characteristic Function Approach to

Goodness of Fit Test for Cauchy Distribution with Parameters estimated by MLE or EISE’ CIRJE discussion paper series, University of Tokyo

[11]

Rublyk F. (1999). ‘A goodness-of-fit test for Cauchy distribution’, Proceedings of

the Third International Conference on Mathematical Statistics, Tatra Mountains, Mathematical Publications Bratislava

[12]

Rublyk F. (2001) ‘A quantile goodness-of-fit for Cauchy distribution, based on

extreme order statistics’. Applications of Mathematics

[13]

Zaman A.(1996) ‘Statistical Foundations of Econometric Techniques’, Academic

Press Inc