Statistics, Vol. 39, No. 4, August 2005, 287–301
On the geometry of F , Wald, LR, and LM tests in linear regression models ENIS SINIKSARAN* Department of Econometrics, Faculty of Economics, Istanbul University, Beyazit- Istanbul, Turkey (Received 31 August 2004; in final form 12 May 2005) In this article, we examine F , Wald, LR, and LM test statistics in the linear regression model using vector geometry. These four statistics are expressed as a function of one random variable – the angle between the vectors of unrestricted and restricted residuals. The exact and nominal sampling distributions of this angle are derived to illuminate some facts about the four statistics. Alternatively, we offer that the angle itself can be used as a test statistic. A Mathematica program is also written to carry out the approach. Keywords: Geometry; F -test; Wald; Likelihood ratio; Lagrange multiplier; Projection; Angle; Mathematica
1.
Introduction
As an elegant and powerful tool, geometry serves to clarify and unify the many aspects of statistics. Although this view was recognized by early authors like Fisher, Durbin, and Kendall, geometrical approaches have not been commonly promoted to statistical teaching, research, and consulting in all levels. The reasons for this were stated by some authors. Herr [1] pointed out that one reason could be the telegrammatic style of the pure geometric approach of the early authors mentioned earlier. Bryant [2] indicated the lack of the relevant material in the elementary level literature. Saville and Wood [3] asserted that the fashion of formalism was one of the major reasons for the preeminence of algebraic methods in statistics. In recent years, however, statisticians and econometricians have found geometry increasingly useful. In the last two decades, some books were written in which the word ‘geometry’ appeared in their titles, see, Saville and Wood [4], Murray and Rice [5], Wickens [6], and Marriott and Salmon [7]. In some other books, writers devoted one or two chapters to geometrical approaches, such as Davidson and Mackinnon [8], Draper and Smith [9], and Efron and Tibshirani [10]. Several papers in which it was all done geometrically or geometric ideas appeared as asides were written, including Bring [11], Critchley et al. [12], McCartin [13], and Huang and Draper [14]. Despite the recent growth, we believe that many additional attempts should be made to *Email:
[email protected]
Statistics ISSN 0233-1888 print/ISSN 1029-4910 online © 2005 Taylor & Francis Group Ltd http://www.tandf.co.uk/journals DOI: 10.1080/02331880500178521
288
E. Siniksaran
promote the geometrical approaches to statistics. This article is written to show the merit and the power of geometry in the study of classical tests in the linear regression models. As noted by Bryant [2], the statistical corresponding of two fundamental ideas in geometry should be well understood: projection and angle. Projection gives the best fit and angle measures the goodness of that fit. In this article, we show some known comparisons of F , Wald, LR, and LM tests using projections and angles in a subject-space picture. We also offer that using angle as a test statistic which can be very useful in exploring the relations among the classical tests. See Buse [15], Ramanathan [16], and Davidson and MacKinnon [8] for the geometric comparisons of W , LR, and LM tests using the shape of log-likelihood function. See also Van Garderen [17] for an alternative comparison using differential geometry.
2.
Geometry of unrestricted and restricted models
Consider the linear regression model of the form y = Xβ + ε,
(1)
where y is an n × 1 vector of responses, X is an n × k non-stochastic matrix of regressors and has a full column rank k, β is a k × 1 vector of parameters, and ε is an n × 1 vector of random errors. We assume ε ∼ N (0, σ 2 I). Now we partition X as follows X ≡ [X1 |X2 ], where X1 has size n × (k − r) and X2 has size n × r. The model (1), then, can be rewritten as y = X1 β 1 + X2 β 2 + ε,
(2)
where β 1 and β 2 are subvectors of β with (k − r) and r components, respectively. Assume that we want to test the joint significance of the regressors of submatrix X2 , that is, the null hypothesis H0 : β 2 = 0 against Ha : β 2 = 0. The model excluding β 2 is y = X1 β 1 + ε.
(3)
Let us call equation (2), the unrestricted model and equation (3), the restricted model. Let the unrestricted and restricted least squares parameter estimates be βˆ ≡ [βˆ 1 |βˆ 2 ] and β˜ ≡ [β˜ 1 |0] respectively. ˆyur , ˆyr , eur , and er are the vectors of unrestricted and restricted fitted values and residuals, respectively. Figure 1 is a subject-space picture in which the variables are represented by vectors in Euclidean n-dimensional space, which is denoted by V n . The figure illustrates the least squares estimation and the orthogonal decomposition of response variable for unrestricted and restricted models. As columns of matrix X are independent vectors, they span a k-dimensional subspace. This is the column space of X and generally called estimation space in statistics. Let us denote this subspace of V n by δ(X) which is illustrated by ordinary plane in figure 1. When we fit the unrestricted model in equation (2) by the least squares method, we are essentially choosing a linear combination ˆyur = X1 βˆ 1 + X2 βˆ 2 in δ (X), which is uniquely determined by the perpendicular from y onto δ (X). The difference vector eur = y − ˆyur lies in the orthogonal complement of δ (X). This (n − k) dimensional subspace is called left-null space of X in linear algebra or error space in statistics. The columns of matrices X1 and X2 span (k − r) and r dimensional subspaces of δ (X), respectively. Denote these subspaces by δ(X1 ) and δ(X2 ). For simplicity, in figure 1, these
Geometry of F, Wald, LR, and LM tests
289
Figure 1. The least squares estimation in unrestricted and restricted models.
subspaces are illustrated by line segments with points on each of their ends. For fitting the restricted model in equation (3) by least squares, the vector y is projected onto δ(X1 ). This projection gives the vector of restricted fitted values yˆ r . Notice also that the projection from yˆ ur onto δ(X1 ) gives the vector yˆ r . When we desire to test the null hypothesis H0 : β 2 = 0 versus Ha : β 2 = 0, F , W , LR, and LM tests statistics are commonly used. Under general conditions, these four statistics can be expressed as functions of unrestricted and restricted residuals of least squares estimation [see ref. 18, p. 97, 147–150]: F =
n − k er er − eur eur , r eur eur
(4)
W =
n(er er − eur eur ) , eur eur
(5)
er er , eur eur
(6)
n(er er − eur eur ) . er er
(7)
LR = n log LM =
The scalar products in these equations give the squared lengths in the right-angled triangle formed by the vectors er , eur , and yˆ ur − ˆyr in figure 1. Thus, we can write each test statistic as a function of the angle () between the vectors eur and er : n−k tan2 , r W = n tan2 , F =
(8) (9)
LR = n log sec2 ,
(10)
LM = n sin2 .
(11)
Many facts dealing with these statistics can be illuminated from this viewpoint. For instance, it is known that LM can also be defined as LM = nR 2
290
E. Siniksaran
where R 2 is the squared multiple correlation coefficient from the regression of er on X. This relation can be easily seen in figure 1. The projection from the response variable er onto δ (X) gives the vector ˆyur − ˆyr as the vector of fitted values. Thus, the squared lengths of er and yˆ ur − yˆ r are the total and explained sum of squares of this regression, respectively. The cosine of the angle between the vectors er and yˆ ur − yˆ r is the multiple correlation coefficient. The squared multiple correlation is then R 2 = cos2 (90 − ) = sin2 . Hence, the equation LM = nR 2 is identical to the equation (11). The angular equivalents of the test statistics would also suggest that the angle itself can be used directly as a test statistic. To do this, we need to derive angular sampling distributions under the null hypothesis.
3. Angular equivalent of F -test Solving equation (8) for , we see that = arctan
r F. n−k
(12)
When the random variable F is distributed, the non-central F distribution with r and (n − k) degrees of freedom and δ non-centrality parameter, as a monotonic function of the random variable F , has the following density NF (θ; r, n − k, δ) −δ/2 2e (n − k)1/2(n−k) r r/2 csc θ Hypergeometric1F1 1/2(n − k + r), r/2, (1/2)δ sin2 θ = beta [r/2, 1/2(n − k)]
× sec θ [(n − k) sec2 θ ]−1/2(n−k+r)
(n − k) tan2 θ r
r/2 ,
0 < θ < 90,
(13)
where Hypergeometric1F1 is the Kummer confluent hypergeometric function with three parameters. Under the null hypothesis H0 : β 2 = 0, the non-centrality parameter is zero and the non-central distribution of becomes the following central distribution: 2(n − k)1/2(n−k) r r/2 csc θ sec θ[(n − k) sec2 θ ]−1/2(n−k+r)
r/2 (n − k) tan2 θ/r F (θ; r, n − k) = , beta [r/2, 1/2(n − k)] 0 < θ < 90.
(14)
Figure 2 shows the graphs of density functions of with some parameter values. The ∞critical region for F -test of H0 at significance level α is F > F (r, n − k, α), where α = F (r,n−k,α) f (F ; r, n − k)dF and F (r, n − k, α) is (1 − α) percent upper tail value of F density function f (F ; r, n − k). When we want to use as a test statistic, the critical region
Geometry of F, Wald, LR, and LM tests
Figure 2.
291
Graphs of density function F (θ ; r, n − k).
90 of the same hypothesis at α is > F (r, n − k, α) where α = F (r,n−k,α) F (θ ; r, n − k)dθ and F (r, n − k, α) is (1 − α ) percent upper tail value of density function F (θ; r, n − k). Table A1 in the Appendix A gives selected values of F (r, n − k, α) for α = 0.05. When the observed is greater than the tabulated critical value F (r, n − k, α), H0 is rejected at the level of significance α. We will use symbol θ as the realized value of the random variable .
4. Angular equivalents of exact and asymptotic W , LR, and LM tests The relations between F and three classical statistics W , LR, and LM are well known. These relations can be easily obtained by the equations (4)–(7) or equivalently by equations (8)–(11): nrF , n−k
rF LR = n log 1 + , n−k W =
LM =
(15)
nrF n − k + rF
Each of three tests statistics is the function of F -statistic. So, one can derive their exactsampling distributions under the null hypothesis. In this case, each statistic has a different exact-sampling distribution, hence each test has a different critical value. However, these three exact tests have the same size and the power curve – the correct size and the correct power curve of F -test. Therefore, there is actually no need to derive these distributions. As some writers indicate [ref. 8, p. 450], there is even no need to calculate W , LR, and LM, as no more information is gained than what is already in F . In contrast, it is common practice to use these statistics and make inferences from their asymptotic distribution, which is the central χ 2 distribution with r degrees of freedom. In a finite sample, however, when the critical regions
292
E. Siniksaran
are calculated from this nominal distribution, there may be conflicts in inference among the tests and because of the different critical regions, powers of tests cannot be equal. At this point, it would be worth obtaining the angular equivalents of the asymptotic distributions of three statistics to investigate these issues from the geometric viewpoint. Solving equations (9), (10) and (11) for we have W = arctan , n √ = arccos e−LR/n , (16) LM = arcsin . n We know that W , LR, and LM have the asymptotic χ 2 distribution with r degrees of freedom. As a monotonic function of three statistics, the random variable has the following nominal asymptotic distributions for each statistic: 21−(r/2) e−(1/2) tan θ csc θ sec θ(n tan2 θ)r/2 , gamma (r/2) 2
W (θ; r, n) = LR (θ; r, n) =
0 < θ < 90,
21−(r/2) n[n log(sec2 θ )]1/2(r−2) (sec2 θ)−(n/2) tan θ , gamma (r/2)
21−(r/2) e−1/2n sin θ cot θ (n sin2 θ)r/2 LM (θ; r, n) = , gamma (r/2)
0 < θ < 90,
(17)
2
0 < θ < 90.
Note that if we derived the angular equivalents of the exact distributions of the three statistics using equations (15) and (16), we would obtain the same distribution – the angular exact distribution of F defined in equation (14). Figure 3 shows the graphs of angular nominal
Figure 3. The angular exact and angular nominal distributions of W , LR, and LM.
Geometry of F, Wald, LR, and LM tests
293
distributions of W , LR, and LM defined in equation (17) and the graph of their angular exact distribution defined in equation (14) for some parameter values. In a sense, we see three nominal and one actual sampling distribution of three statistics together as a function of the same random variable. This gives us the opportunity to compare the three statistics and investigate some facts about them. In figure 3, for instance, the famous inequality LM ≤ LR ≤ W is very evident. Sizes of F test are closer to sizes of LM test than to those of W and LR tests. We see also that if we increase r, W and LR overreject more severely (compare figure 3(a) with (b) or figure 3(c) with (d)). As expected, when n is increased, nominal sizes of the tests become closer to exact sizes (compare figure 3(a) with (c) or figure 3(b) with (d)). The critical region for W , LR, and LM tests of H0 at size α is W , LR, LM > ∞ χ 2 (r, α), where α = χ 2 (r,α) f (χ 2 ; r)dχ 2 and χ 2 (r, α) is (1 − α) percent upper tail value of χ 2 density function f (χ 2 ; r). Equivalently, if we want to use as a test statistic, the critical regions of the same hypothesis at α are > W (r, n, α), > LR (r, n, α), 90 90 and > LM (r, n, α), where α = W (r,n,α) W (θ ; r, n)dθ = LR (r,n,α) LR (θ ; r, n)dθ = 90 LM (r,n,α) LM (θ ; r, n) dθ, and W (r, n, α), LR (r, n, α), and LM (r, n, α) are (1 − α) percent upper tail values of density functions in equation (17). Tables A2–A4 in Appendix A give these selected critical values of for α = 0.05. When the observed is greater than the tabulated critical values, H0 is rejected at the level of significance α. Let us make a hypothetical example to illustrate the approach. Let the sample data be
3 8 12 y= 9 , 6 24 21
1 1 1 X= 1 1 1 1
4 13 8 9 13 6 7 13 8 2 19 1 16 1
10 13 9 16 6 11 9
15 16 19 12 , 18 24 30
1 1 1 X1 = 1 1 1 1
4 8 13 7 8 19 16
13 9 6 13 , 2 1 1
10 13 9 X2 = 16 6 11 9
15 16 19 12 . 18 24 30
Assume that we want to test the null hypothesis H0 : β 2 = 0. The regression of y on X gives the unrestricted residuals: eur = [0.744, −1.341, −0.990, 0.547, 0.300, 0.846, −0.105] , and the regression of y on X1 gives the restricted residuals: er = [0.137, −0.169, −3.413, 1.241, −0.032, 0.323, 1.913] . Substituting for eur eur = 4.45, er er = 17.00, n = 7, k = 5, and r = 2 in the equations (4)– (7), we have that F = 2.82, W = 19.75, LR = 9.39, and LM = 5.17. F (2, 2, 0.05) = 19.00 and χ 2 (2, 0.05) = 5.99. So W and LR tests reject the null hypothesis, whereas F and LM fail to reject it at the level of significance α = 0.05. Thus, the sample produces conflicting inferences. To implement the angular equivalents of the tests, we need to calculate the angle θ between the vectors eur and er . This angle is θ = 59.24◦ . From tables A1–A4, we see that F (2, 2, 0.05) = 77.08, W (2, 7, 0.05) = 42.77, LR (2, 7, 0.05) = 49.32, and LM (2, 7, 0.05) = 67.69. θ is greater than W (2, 7, 0.05) and LR (2, 7, 0.05), W and LR tests reject the null hypothesis, whereas as θ is smaller than F (2, 2, 0.05) and
294
E. Siniksaran
LM (2, 7, 0.05), F and LM tests fail to reject H0 . As expected, the inferences are the same as those of the traditional approach. The p-values of the tests can be calculated both from the traditional approach and the angular equivalents as follows: F:
∞
90
f (F ; r, n − k)dF =
2.82
F (θ ; r, n − k)dθ = 0.26164,
77.08
∞
W:
f (χ 2 ; r)dχ 2 =
19.75
∞
LR:
∞
5.17
W (θ ; r, n)dθ = 0.00005,
42.77
f (χ ; r)dχ = 2
90
2
9.39
LM:
90
LR (θ; r, n)dθ = 0.00916,
49.32
f (χ 2 ; r)dχ 2 =
90
LM (θ ; r, n)dθ = 0.07545.
67.69
Evans and Savin [19] presented some results that when the tests are modified by some correction factors and the nominal sizes of the three test statistics are closer to the exact size. The modified test statistics are obtained by replacing n in equations (9)–(11) by (n − k), (n − k + r/2 − 1), and (n − k + r), respectively. Thus the modified versions of the functions in equation (17) can also be obtained by replacing n by (n − k), (n − k + r/2 − 1), and (n − k + r), respectively. In figure 4, we see the modified angular nominal sampling distributions of W , LR, LM, and their exact angular sampling distribution defined in equation (14) for some values of k, r, and n. If we compare it with the figure 3, we clearly see that the modification gives better approximations to the exact distribution. The approximation is almost perfect in the case of LR.
Figure 4. The angular exact and angular modified nominal distributions of W , LR, and LM.
Geometry of F, Wald, LR, and LM tests
295
5. The powers of tests As nominal sizes of three statistics differ from the exact size of F -statistic, their power curve are different from the true power curves of F test. Evans and Savin [19] calculated the powers of the tests for large samples by using the fact that the three statistics are functions of F statistic. These power values of the tests can also be defined by the angular equivalents of the statistics as follows: F : P [N > F (r, n − k, α)],
W : P [N > W (r, n, α)],
LR: P [N > LR (r, n, α)],
LM: P [N > LM (r, n, α)]
where N is a random variable having the density function defined in equation (13) as the angular equivalent of the non-central F distribution with r and (n − k) degrees of freedom and δ non-centrality parameter. The non-centrality parameter δ can be expressed as δ = (n − k)d 2 where d is the difference between the null and the alternative hypothesis. When, for example, r = 2, n = 20, k = 4, d = 0.5, and α = 0.05, the upper tail values are F (2, 16, 0.05) = 33.98, W (2, 20, 0.05) = 28.69, LR (2, 20, 0.05) = 30.58, and LM (2, 20, 0.05) = 33.18. Then the powers are 90 90 N F (θ ; 2, 16, 4)dθ = 0.352, W: NF (θ; 2, 16, 4)dθ = 0.537, F: 33.98
90
LR: 30.58
28.69
NF (θ ; 2, 16, 4)dθ = 0.470,
90
LM:
NF (θ ; 2, 16, 4)dθ = 0.379.
33.18
In figure 5, we see the sizes and powers of the example aforementioned as shaded areas in the angular distributions.
6.
Remark
In the traditional approach, Wald, LR, and LM tests are taught within the context of likelihood estimation. We have to admit that if these tests were not carried out within this context, the learners would miss the basic concepts of the tests. Hence, our geometric approach lending itself easily to pictorial representations should be considered as a supplementary tool for
Figure 5. The power values of F , W , LR, and LM dealing with the angular distributions.
296
E. Siniksaran
clarifying and unifying the concepts of the traditional approach. We believe that an integrated viewpoint can provide new insight into the procedures.
7.
Computations
All computations and graphical work were done using Mathematica 4.0. A computer program as a Mathematica notebook titled ‘GeoTest’is also written to implement the tests geometrically. It can be downloaded from http://www.istanbul.edu.tr/iktisat/econometrics/siniksaran. After entering the data, the parameters in the null hypothesis, and the level of significance, the program computes the angle between the unrestricted and the restricted residuals and compares it with the upper tail values of the four angular distributions in equations (14) and (17). It also gives the results of the traditional approach. A sample output given below shows the results of the hypothetical example given in the article: ** ** ** ** ** F , Wald, LR and LM Tests ** ** ** ** ** Model: y = β 0 + β 1 x1 + β 2 x2 + β 3 x3 + β 4 x4 + ε, Null hypothesis: H0 : β 3 = β 4 = 0 ** * Geometric Process ** * Calculated angle: 59.24 Angular statistics F Wald LR LM
Critical Angles 77.08 42.77 49.32 67.69
p-values 0.26164 0.000051322 0.0091609 0.075451
** * Traditional process ** * Statistics F Wald LR LM
Calculated values 2.822 19.75 9.386 5.169
** * Results of the tests ** * H0 H0 H0 H0
failed to reject by F -Test rejected by Wald-Test rejected by LR-Test failed to reject by LM-Test
Critical values 19.00 5.991 5.991 5.991
p-values 0.26164 0.000051322 0.0091609 0.075451
Appendix A Table A1. n − k/r
1
2
3
4
5
6
7
8
9
10
16
20
30
50
80
120
85.50 71.81 61.44 54.23 48.98 44.97 41.79 39.19 37.02 35.17 27.92 25.01 22.39 20.45 15.86 12.54 10.25
87.13 77.08 68.38 61.78 56.68 52.63 49.32 46.56 44.20 42.17 33.98 30.58 27.49 25.18 19.64 15.58 12.75
87.75 79.44 71.82 65.78 60.97 57.04 53.77 51.00 48.61 46.53 37.93 34.28 30.93 28.39 22.25 17.71 14.51
88.09 80.84 74.00 68.41 63.86 60.09 56.91 54.17 51.80 49.71 40.93 37.13 33.60 30.92 24.34 19.42 15.94
88.31 81.81 75.53 70.32 66.01 62.39 59.30 56.63 54.28 52.21 43.35 39.46 35.81 33.02 26.10 20.88 17.17
88.47 82.52 76.69 71.79 67.69 64.21 61.22 58.61 56.31 54.26 45.40 41.44 37.71 34.83 27.65 22.17 18.25
88.59 83.07 77.61 72.97 69.06 65.71 62.80 60.26 58.00 55.98 47.16 43.17 39.37 36.43 29.02 23.33 19.23
88.69 83.52 78.37 73.95 70.19 66.96 64.14 61.66 59.45 57.47 48.70 44.69 40.85 37.86 30.28 24.39 20.14
88.77 83.89 78.99 74.77 71.16 68.04 65.30 62.88 60.71 58.76 50.07 46.06 42.19 39.16 31.42 25.37 20.97
88.84 84.20 79.53 75.48 72.00 68.97 66.31 63.95 61.83 59.91 51.30 47.29 43.41 40.35 32.48 26.28 21.75
89.09 85.41 81.64 78.31 75.40 72.82 70.51 68.44 66.55 64.82 56.79 52.89 49.01 45.89 37.58 30.76 25.64
89.19 85.90 82.50 79.48 76.82 74.45 72.32 70.39 68.62 67.00 59.34 55.54 51.72 48.61 40.19 33.13 27.73
89.34 86.65 83.85 81.34 79.10 77.09 75.27 73.60 72.06 70.64 63.75 60.24 56.62 53.61 45.18 37.78 31.94
89.49 87.41 85.22 83.24 81.47 79.86 78.39 77.04 75.78 74.60 68.79 65.72 62.48 59.73 51.67 44.15 37.92
89.60 87.95 86.21 84.64 83.21 81.92 80.73 79.63 78.60 77.63 72.78 70.17 67.35 64.92 57.54 50.27 43.93
89.67 88.32 86.90 85.61 84.44 83.37 82.39 81.47 80.62 79.81 75.72 73.49 71.06 68.93 62.30 55.49 49.30
Geometry of F, Wald, LR, and LM tests
1 2 3 4 5 6 7 8 9 10 16 20 25 30 50 80 120
(1 − α) percent upper tail values of the distribution F (θ ; r, n − k) for α = 0.05.
297
298
Table A2.
(1 − α) percent upper tail values of the distribution W (θ ; r, n − k) for α = 0.05.
1
2
3
4
5
6
7
8
9
10
16
20
30
50
80
120
1 2 3 4 5 6 7 8 9 10 16 20 25 30 50 80 120
62.97 54.19 48.53 44.42 41.24 38.67 36.53 34.72 33.16 31.79 26.10 23.67 21.40 19.69 15.49 12.36 10.14
67.78 59.98 54.72 50.75 47.59 44.98 42.77 40.87 39.21 37.74 31.46 28.69 26.08 24.08 19.09 15.31 12.60
70.32 63.17 58.22 54.42 51.34 48.77 46.58 44.66 42.98 41.48 34.95 32.01 29.21 27.04 21.57 17.36 14.32
72.01 65.34 60.65 57.00 54.02 51.51 49.34 47.44 45.76 44.25 37.60 34.56 31.63 29.35 23.54 19.00 15.71
73.27 66.97 62.50 58.99 56.10 53.64 51.51 49.63 47.96 46.46 39.75 36.65 33.64 31.28 25.20 20.41 16.90
74.26 68.27 63.98 60.59 57.78 55.38 53.29 51.44 49.79 48.29 41.58 38.43 35.36 32.94 26.65 21.64 17.95
75.07 69.34 65.21 61.93 59.20 56.85 54.80 52.98 51.34 49.86 43.16 39.99 36.87 34.40 27.94 22.75 18.90
75.75 70.25 66.26 63.07 60.41 58.12 56.10 54.31 52.70 51.23 44.55 41.37 38.22 35.71 29.11 23.76 19.77
76.34 71.03 67.16 64.07 61.47 59.23 57.25 55.49 53.89 52.45 45.80 42.61 39.44 36.91 30.19 24.70 20.58
76.85 71.71 67.96 64.95 62.41 60.21 58.27 56.53 54.96 53.53 46.93 43.73 40.55 38.00 31.18 25.57 21.33
78.97 74.58 71.34 68.69 66.44 64.47 62.71 61.12 59.67 58.34 52.04 48.91 45.72 43.11 35.95 29.83 25.09
79.88 75.84 72.83 70.36 68.25 66.39 64.73 63.22 61.84 60.57 54.48 51.41 48.26 45.66 38.40 32.07 27.10
81.41 77.93 75.33 73.18 71.33 69.68 68.20 66.85 65.61 64.45 58.84 55.94 52.92 50.38 43.10 36.49 31.13
83.06 80.23 78.10 76.32 74.78 73.40 72.15 71.00 69.94 68.95 64.04 61.44 58.68 56.31 49.28 42.57 36.87
84.34 82.02 80.26 78.79 77.51 76.36 75.31 74.35 73.45 72.60 68.38 66.10 63.65 61.51 54.99 48.45 42.66
85.28 83.34 81.86 80.62 79.54 78.56 77.67 76.85 76.08 75.36 71.72 69.73 67.56 65.66 59.71 53.54 47.86
E. Siniksaran
n/r
Table A3.
(1 − α) percent upper tail values of the distribution LR (θ ; r, n − k) for α = 0.05.
1
2
3
4
5
6
7
8
9
10
16
20
30
50
80
120
1 2 3 4 5 6 7 8 9 10 16 20 25 30 50 80 120
81.58 67.50 58.19 51.78 47.08 43.44 40.53 38.14 36.12 34.39 27.52 24.71 22.17 20.28 15.78 12.51 10.22
87.13 77.08 68.38 61.78 56.68 52.63 49.32 46.56 44.20 42.17 33.98 30.58 27.49 25.18 19.64 15.58 12.75
88.85 81.85 74.22 67.88 62.76 58.57 55.09 52.15 49.62 47.42 38.43 34.66 31.21 28.61 22.36 17.76 14.54
89.50 84.65 78.13 72.21 67.22 63.03 59.48 56.45 53.82 51.52 41.98 37.92 34.19 31.38 24.57 19.54 16.00
89.77 86.40 80.91 75.49 70.70 66.58 63.03 59.96 57.27 54.91 44.96 40.69 36.74 33.75 26.47 21.07 17.27
89.89 87.54 82.96 78.04 73.51 69.50 66.00 62.92 60.21 57.80 47.57 43.12 38.98 35.84 28.15 22.43 18.40
89.95 88.30 84.50 80.08 75.82 71.96 68.52 65.47 62.76 60.34 49.89 45.29 40.99 37.72 29.68 23.68 19.43
89.98 88.81 85.67 81.72 77.75 74.06 70.71 67.70 65.01 62.58 51.98 47.26 42.83 39.44 31.09 24.82 20.38
89.99 89.17 86.58 83.07 79.39 75.87 72.62 69.67 67.01 64.59 53.89 49.07 44.53 41.04 32.40 25.89 21.26
89.99 89.41 87.29 84.18 80.78 77.44 74.31 71.43 68.80 66.40 55.64 50.75 46.10 42.52 33.62 26.89 22.10
90.00 89.92 89.28 87.86 85.87 83.58 81.21 78.85 76.58 74.42 63.92 58.79 53.77 49.82 39.76 31.96 26.33
90.00 89.98 89.69 88.87 87.52 85.81 83.91 81.93 79.94 78.00 67.99 62.87 57.75 53.67 43.08 34.74 28.68
90.00 90.00 89.96 89.76 89.28 88.51 87.49 86.28 84.96 83.57 75.25 70.44 65.38 61.18 49.80 40.48 33.56
90.00 90.00 90.00 89.99 89.93 89.79 89.54 89.16 88.65 88.04 83.03 79.34 74.98 71.06 59.39 49.02 40.99
90.00 90.00 90.00 90.00 90.00 89.99 89.96 89.90 89.80 89.65 87.63 85.51 82.51 79.45 68.84 58.06 49.15
90.00 90.00 90.00 90.00 90.00 90.00 90.00 89.99 89.98 89.96 89.41 88.53 86.94 85.01 76.65 66.42 57.11
Geometry of F, Wald, LR, and LM tests
n/r
299
300
Table A4.
(1 − α) percent upper tail values of the distribution LM (θ ; r, n − k) for α = 0.05.
1
2
3
4
5
6
7
8
9
10
16
20
30
50
80
120
1 2 3 4 5 6 7 8 9 10 16 20 25 30 50 80 120
90.00 90.00 90.00 78.52 61.23 53.14 47.80 43.86 40.79 38.30 29.34 25.99 23.08 20.97 16.09 12.66 10.31
90.00 90.00 90.00 90.00 90.00 87.84 67.69 59.93 54.68 50.72 37.73 33.18 29.31 26.54 20.25 15.88 12.91
90.00 90.00 90.00 90.00 90.00 90.00 90.00 81.25 68.72 62.13 44.34 38.69 33.99 30.69 23.29 18.21 14.78
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 76.92 50.36 43.53 38.03 34.22 25.82 20.14 16.33
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 56.29 48.07 41.72 37.41 28.07 21.84 17.68
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 62.51 52.51 45.21 40.38 30.12 23.37 18.90
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 69.66 57.00 48.60 43.22 32.03 24.79 20.02
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 79.89 61.71 51.96 45.97 33.84 26.12 21.07
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 66.89 55.35 48.68 35.57 27.38 22.05
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 73.09 58.84 51.37 37.24 28.58 22.99
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 69.43 46.49 34.98 27.91
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 52.43 38.80 30.77
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 69.34 47.71 37.15
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 66.72 48.59
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 67.13
90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00 90.00
E. Siniksaran
n/r
Geometry of F, Wald, LR, and LM tests
301
References [1] Herr, D.G., 1980, On the history of the use of geometry in the general linear model. The American Statistician, 34, 131–135. [2] Bryant, P., 1984, Geometry, statistics, probability: variation on a common theme. The American Statistician, 38, 38–48. [3] Saville, D.J. and Wood, G.R., 1986,A method for teaching statistics using n-dimensional geometry. The American Statistician, 40, 205–214. [4] Saville, D.J. and Wood, G.R., 1991, Statistical Methods: the Geometric Approach (New York: Springer-Verlag). [5] Murray, M.K. and Rice, J.W., 1993, Differential Geometry and Statistics, Monographs on Statistics and Applied Probability, Vol. 48 (London: Chapman & Hall). [6] Wickens, T.D., 1995, The Geometry of Multivariate Statistics (New Jersey: Lawrence Erlbaum Associates, Inc.). [7] Marriott, P. and Salmon, M., 2000, Applications of Differential Geometry to Econometrics (UK: Cambridge University Press). [8] Davidson, R. and Mackinnon, J.G., 1993, Estimation and Inference in Econometrics (New York: Oxford University Press). [9] Draper, N.R. and Smith, H., 1998, Applied Regression Analysis (US: Wiley Series in Probability & Statistics). [10] Efron, B. and Tibshirani, R.J., 1998, An Introduction to the Bootstrap, Monographs on Statistics and Applied Probability, Vol. 57 (US: Chapman & Hall/CRC). [11] Bring, J., 1996, A geometric approach to compare variables in a regression model. The American Statistician, 50, 57–62. [12] Critchley, F., Marriott, P. and Salmon, M., 2002, On preferred point geometry in statistics. Journal of Statistical Planning and Inference, 102, 229–245. [13] McCartin, B.J., 2002, A geometric characterization of linear regression. Statistics, 42, 647–664. [14] Huang, Y. and Draper, N.R., 2003, Transformations, regression geometry and R2 . Computational Statistics & Data Analysis, 42, 647–664. [15] Buse, A., 1982, The likelihood ratio, Wald, and Lagrange multiplier tests: an expository note. The American Statistician, 36, 153–157. [16] Ramanathan, R., 1993, Statistical Methods in Econometrics (London: Academic Press). [17] Garderen, K.J. Van, 2000, An alternative comparison of classical tests: assessing the effects of curvature. In: P. Marriott and M. Salmon (Eds) Applications of Differential Geometry to Econometrics (UK: Cambridge University Press). [18] Johnston, J. and DiNardo, J., 1997, Econometric Methods (US: McGraw-Hill). [19] Evans, G.B.A. and Savin, N.E., 1982, Conflict among the criteria revisited: the W , LR, and LM tests. Econometrica, 50, 737–748.
