Exact Likelihood Ratio Test for the Parameters of the Linear ...

10.2478/v10048-009-0003-9

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009

Exact Likelihood Ratio Test for the Parameters of the Linear Regression Model with Normal Errors Martina Chvostekov´a, Viktor Witkovsk´y Institute of Measurement Science, Slovak Academy of Sciences, D´ubravsk´a cesta 9, 84104 Bratislava, Slovakia E-mail: [email protected], [email protected] In this paper we present an exact likelihood ratio test (LRT) for testing the simple null hypothesis on all parameters of the linear regression model with normally distributed errors. In particular, we consider the simultaneous test for the regression parameters, β, and the error standard deviation, σ. The critical values of the LRT are presented for small sample sizes and a small number of explanatory variables for usual significance levels, α = 0.1, 0.05, and 0.01. The test is directly applicable for construction of the (1 − α)-confidence region for the parameters (β, σ) and the simultaneous tolerance intervals for future observations in linear regression models. For comparison, the suggested method for construction of the tolerance factors of the symmetric (1 − γ)-content simultaneous (1 − α)-tolerance intervals is illustrated by a simple numerical example. Keywords: linear regression model; exact likelihood ratio test; simultaneous tolerance intervals; confidence-set approach.

1. Introduction

ous tolerance intervals and/or the tolerance factors of the symmetric (1 − γ)-content simultaneous (1 − α)-tolerance intervals n the paper we present an exact likelihood ratio test for is illustrated by a simple numerical example. testing the simple null hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative H1 : (β, σ)  (β0 , σ0 ) on the parame2. Likelihood ratio test of the hypothesis ters β and σ of the linear regression model Y = Xβ + σZ with H0 : (β, σ) = (β0 , σ0 ) normally distributed errors, Z ∼ N(0, In ). Although the derivation of the exact distribution of the Consider the linear regression model Y = Xβ + σZ with norlikelihood ratio test (LRT) statistic under the null hypothesis mally distributed errors, where Y represents the n-dimensional H0 is straightforward, it seems that the result is not available random vector of response variables, X is the n × k matrix of in the standard (statistical) literature on linear regression mod- non-stochastic explanatory variables (for simplicity, here we els nor in the literature on their applications in measurement assume that X is a full-rank matrix), β is a k-dimensional vecscience and metrology. tor of regression parameters, Z is an n-dimensional vector of The test is directly applicable for construction of the standard normal errors, i.e. Z ∼ N(0, In ), and σ is the error (1 − α)-confidence region for the parameters (β, σ) and for standard deviation, σ > 0. construction of the simultaneous tolerance intervals for future Here we consider the LRT for testing the simple null observations in linear regression models. The simultaneous hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative tolerance intervals are important for many measurement pro- H1 : (β, σ)  (β0 , σ0 ). Based on the above assumptions the cedures. The most common application for simultaneous tol- log-likelihood function, denoted as (β, σ | Y = y, X), is given erance intervals is the multiple-use calibration problem; see by e.g. [7], [6], [1]. The tolerance intervals have been recogn n nized and considered in various settings by many authors, see (β, σ | y, X) = − log(2π) − log(σ2 ) 2 2 e.g. [9], [8], [10], [3], [4], [5], [6], and [2]. These simulta(1) 1  neous tolerance intervals are constructed such that with given (y − Xβ) (y − Xβ). − 2σ2 confidence level (1 − α) at least a specified proportion (1 − γ) of the population is contained in the tolerance interval for all The (−2)-multiple of the logarithm of the LRT statistic, say possible values of the predictor variates. All known simulta- λ(y | X) for observed value y of Y, and given X, for testing the neous tolerance intervals in regression are conservative in that null hypothesis H0 : (β, σ) = (β0 , σ0 ) is given by the actual confidence level exceeds the nominal level (1 − α). ⎛ ⎞ ⎜⎜⎜ ⎟⎟ Here we present the method for computing the critical valλ(y | X) = −2 ⎝⎜ sup (β, σ | y, X) − sup (β, σ | y, X)⎟⎠⎟ ues of the LRT as well as the tables of the critical values for (β,σ)∈H0 (β,σ)  selected small sample sizes in the range n = k + 1, . . . , 100 ˆ ML | y, X) = −2 (β0 , σ0 | y, X) − (βˆ ML , σ (2) with k explanatory variables, k = 1, . . . , 10, and usual signifi⎛ 2 ⎞ ˆ ⎟⎟⎟ ⎜⎜ σ 1 cance levels α = 0.1, 0.05, and 0.01. ⎟⎠ − n, = 2 (y − Xβ0 ) (y − Xβ0 ) − n log ⎜⎜⎝ ML The suggested method for construction of the simultaneσ0 σ20

I

1

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009 where βˆ ML = βˆ = (X  X)−1 X  y is the standard least squares estimate (LSE) of β (which is also the MLE of β) and σˆ ML is the maximum likelihood estimate (MLE) of the standard deˆ  (y − X β)/n. ˆ Under given viation σ, i.e. σ ˆ ML = (y − X β) model assumptions, and under the null hypothesis H0 , it is straightforward to derive the distribution of the test statistic λ(Y | X):

k = 1, . . . , 10, selected small sample sizes, n = k +1 : (1) : 40, n = 45 : (5) : 100 and ∞, and for the usual significance levels α = 0.1 (Table 2), α = 0.05 (Table 3), and α = 0.01 (Table 4). Alternatively, the test of the hypothesis H0 : (β, σ) = (β0 , σ0 ) could be based on the test statistic F defined as ˆ  (Y −X β)/(n−k) ˆ F = λ(Y | X)/(kS 2/σ20 ), where S 2 = (Y −X β)  −1  ˆ and β = (X X) X Y. Hence, we get

1 (βˆ − β0 ) X  X(βˆ − β0 ) n − k 1

 F = + (Y − Xβ ) (Y − Xβ ) 0 0 k k S2 σ20  (6) ⎛ ⎞ 2 2 ⎜⎜⎜ (Y − Xβ0 ) MX (Y − Xβ0 ) ⎟⎟⎟ n log (n − k)S /nσ0 + 1 ⎟⎠ − n − , − n log ⎜⎝ k S 2 /σ20 nσ20   ∼ Z  Z − n log Z  MX Z + n log(n) − 1 Note that the leading term in F is the standard F-statistic for     ∼ Z (PX + MX )Z − n log Z MX Z + n log(n) − 1 testing the hypothesis on regression parameters H0 : β = β0  against the alternative H1 : β  β0 . Under null hypothesis ∼ Qk + Qn−k − n log (Qn−k ) + n log(n) − 1 , : (β, σ) = (β0 , σ0 ) we get directly H 0 (3)

λ(Y | X) ∼

where PX = X(X  X)−1 X  , MX = In −PX , Z ∼ N(0, In ), Qk ∼ χ2k and Qn−k ∼ χ2n−k are two independent random variables with chi-square distributions, with k and n − k degrees of freedom, respectively. Note that the distribution of λ(Y | X) does not depend on the particular form of the regression design matrix X but only on the number of observations n and on k = rank(X) – the rank of the matrix X, i.e. λ(Y | X) ∼ λ(Y | n, k). This LRT rejects the null hypothesis H0 : (β, σ) = (β0 , σ0 ) for large values of the observed test statistic λ(y | X), i.e. for the given significance level α ∈ (0, 1) the test rejects the null hypothesis if λ(y | X) > λ1−α , (4)

F ∼

(7)

Then, the test rejects the null hypothesis if



Fobs > F1−α ,

(8)

where Fobs denotes the observed value of the statistic F and

F1−α is the (1 − α)-quantile of the distribution of the random

variable F . Similarly as before, the quantiles F1−α could be evaluated numerically, by inverting the cumulative distribution function of the random variable F , denoted by FF (x):

FF (x) = Pr(F ≤ x)   Q   xkQn−k n−k = Pr Qk ≤ − Qn−k + n log +1 n−k n 

∞    xkqn−k qn−k  − qn−k + n log Fχ2k = +1 n−k n 0

where λ1−α is the (1−α)-quantile of the distribution of the random variable λ(Y | n, k), given in (3). The quantiles λ1−α could be evaluated numerically, by inverting the cumulative distribution function, of the random variable λ(Y | n, k), denoted by FLR (·):

× fχ2n−k (qn−k ) dqn−k .

FLR (x) = Pr(λ(Y | n, k) ≤ x)

 = Pr(Qk ≤ x − Qn−k + n log(Qn−k ) − n log(n) − 1 )

∞  = Fχ2k x − qn−k + n log(qn−k ) − n(log(n) − 1)

(9)

Numerical Algorithm

0

× fχ2n−k (qn−k ) dqn−k ,

n − k n log(Qn−k /n) + 1 Qk /k + − . Qn−k /n − k k k Qn−k /n − k

The Matlab algorithm for computing the quantiles λ1−α and

is available upon request from the authors. F1−α For those interested in the idea of the algorithm, we present here a simplified working version of the Matlab code used for calculation of the quantiles of the distribution of the random variable λ(Y | n, k), which is based on the following functions available in Matlab:

(5)

where Fχ2k (·) denotes the cumulative distribution function of the chi-square distribution with k degrees of freedom, and fχ2n−k (·) denotes the probability density function of the chisquare distribution with n − k degrees of freedom. Notice that since the family of normal distributions meets FZERO Single-variable nonlinear zero finding. X = FZERO the regularity conditions, from the standard asymptotic re(FUN,X0) tries to find a zero of the function FUN near sult about the distribution of the LRT λ(Y | X) we get that X0, if X0 is a scalar. λ1−α → χ2k+1,1−α as n → ∞, where χ2k+1,1−α denotes the (1−α)- QUADGK Numerically evaluate integral by adaptive Gaussquantile of chi-square distribution with k + 1 degrees of freeKronrod quadrature. Q = QUADGK(FUN,A,B) attempts dom. to approximate the integral of scalar-valued function The critical values of the LRT are presented in the enFUN from A to B using high order global adaptive closed tables for different number of explanatory variables, quadrature and default error tolerances. 2

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009 CHI2CDF Chi-square cumulative distribution function. P = CHI2CDF(X,V) returns the chi-square cumulative distribution function with V degrees of freedom at the values in X, (Statistics Toolbox). CHI2PDF Chi-square probability density function (pdf). Y = CHI2PDF(X,V) returns the chi-square pdf with V degrees of freedom at the values in X, (Statistics Toolbox).

two alternative confidence sets. The first is a modification of Wilson’s confidence set, which removes the lower bound imposed on σ, and the second is based on a product set formed from the ellipsoidal confidence set for β and a one-sided confidence interval for σ. They found the tolerance intervals based on the product confidence set to be efficient and easy to compute compared with those constructed from the ellipsoidal and the modified Then, for example, the 0.95-quantile λ1−0.05 of the random ellipsoidal confidence sets. variable λ(Y | n, k), based on the linear regression model with Here we suggest a new procedure for simultaneous (1−α)n = 15 observations and k = 2 regressors, could be calculated tolerance intervals covering at least the (1 − γ)-content about by calling the user defined function LRinv: the true mean x β, for any vector x = (x1 , . . . , xk ) of explanatory variables, based on the LRT (4) and the confidence set >> quantile = LRinv(2,15,0.95) (10), defined by    which gives as a result the value 1−γ (x | Y, X) = inf x β + uγ1 σ ; T1−α (β,σ)∈C1−α (Y | X) >> quantile = 8.6813   (11) sup x β + u1−γ2 σ , The function LRinv could be defined as follows (note that it (β,σ)∈C1−α (Y | X) is calling other auxiliary functions, given bellow): where uγ1 and u1−γ2 are the pre-specified quantiles of the standard normal distribution, and such that γ = γ1 + γ2 , with function quantile = LRinv(k,n,prob) γ ∈ (0, 1). x0 = k + 1; As a special case we get the symmetric tolerance intervals quantile = fzero(@(x)LRcdf(x,n,k)-prob,x0); about the fitted regression function, with γ1 = γ2 = γ/2. No% tice, that immediately we get the following probability statefunction cdf = LRcdf(x,n,k) ment cdf = quadgk(@(q)LRfun(q,n,k,x),0,Inf);   % 1−γ (x | Y, X) ≥ 1 − γ, Pr Pr x β + σZ ∈ T1−α function fun = LRfun(q,n,k,x) (12) fun = chi2cdf(x - n * (log(n)-1) - q ... for all x and Z ∼ N(0, 1), Z ⊥ Y ≥ 1 − α, + n * log(q), k) .* chi2pdf(q, n-k); where Z ∼ N(0, 1) is a standard normal random variable stochastically independent of the random vector Y. Note that 3. Simultaneous tolerance intervals considering the exact F -test (8) could be more appropriate The suggested exact tests, (4) and (8), for testing the simple for constructing the simultaneous tolerance intervals. Traditionally, the symmetric tolerance intervals are prenull hypothesis H0 : (β, σ) = (β0 , σ0 ), could be directly used sented in the form to construct the exact simultaneous confidence region for all 1−γ parameters of the linear regression model. In particular, based T1−α (x | Y, X) = x βˆ ∓ S K(x | 1 − α, 1 − γ, Y, X), (13) on LRT (4), the exact (1 −α)-confidence region for the paramwhere K(x | 1 − α, 1 − γ, Y, X) is the tolerance factor which eters β and σ is given as depends on particular values of predictors x, confidence coefLR (Y | X) = {(β, σ) : λ(Y | X) ≤ λ1−α } . (10) ficient 1 − α, coverage content C1−α

(1 − γ), design matrix X, erˆ  ˆ Moreover, the confidence regions based on the exact tests ror standard deviation S = (Y − X β) (Y − X β)/(n − k), and  −1  could be directly used for constructing the (1 − γ)-content βˆ = (X X) X Y, i.e.

simultaneous (1 − α)-tolerance intervals in linear regression model with normal errors. These intervals are constructed such that, with confidence 1 − α, we can claim that at least a specified proportion, 1 − γ, of the population is contained in the tolerance interval, for all possible values of the predictor variates, see e.g. [1] and [2]. Several authors considered the confidence-set approach for constructing the simultaneous tolerance intervals in linear regression. In [10], Wilson used an ellipsoidal confidence set for the regression coefficients, β, and standard deviation of the residuals, σ, which imposes an unnecessary lower bound on σ, as noted by Limam and Thomas in [5]. They considered

K(x | 1 − α, 1 − γ, Y, X) =    1 ˆ  inf x β + uγ/2 σ = x β− (β,σ)∈C1−α (Y | X) S ⎛ ⎞   ⎟⎟ 1 ⎜⎜⎜   ˆ = ⎝⎜ x β + u1−γ/2 σ − x β⎟⎟⎠ . sup S (β,σ)∈C1−α (Y | X)

(14)

Numerical Algorithm Derivation of the tolerance bounds (11) requires numerical optimization for given x, α, γ, the design matrix X, and the observed vector y of Y. 3

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009 Simultaneous Tolerance Intervals

A simplified version of the Matlab code used for calculation of the lower tolerance bound, given in (11), as well as the tolerance factor, (14), is based on the function:

9000 8000 Speed [miles/hour]

FMINCON Minimizes the function FUN subject to the nonlinear constraints defined in NONLCON (Optimization Toolbox). function [lowerBound, factor] = ... LowerBound(x,y,X,quantileU,quantileLR) x = x(:); newxq = [x;quantileU]; [n,k] = size(X); betaML = X \ y; resid = y - X * betaML; sigmaML = sqrt(resid’ * resid / n); betaSigma0 = [betaML;sigmaML]; fminconOpt = optimset(’MaxFunEvals’,... 500*n, ’Display’, ’off’); [betaSigmaArgMin,lowerBound] = ... fmincon(@(x) fun(x,newxq),betaSigma0,... [],[],[],[],[-Inf;-Inf;1.0e-12],[],... @(x) nonlcon(x,y,X,sigmaML,n,... quantileLR),fminconOpt); factor = (x’ * betaML - lowerBound) / ... (sqrt(n / (n-k)) * sigmaML); % function y = fun(x,newxq) y = newxq’ * x; % function [c,ceq] = ... nonlcon(x,y,X,sigmaML,n,quantileLR) sig2 = x(3)^2; resid = y - X * x(1:2); fun = resid’ * resid / sig2 - ... n * log(sigmaML^2/sig2) - n - quantileLR; c = fun; ceq = [];

7000

Measurements Upper Tolerance Bound Upper Confidence Bound Fitted Regression Line Lower Confidence Bound Lower Tolerance Bound

6000 5000 4000 3000 2000

1.25

1.3 1.35 1.4 Orifice opening [in]

1.45

1.5

Figure 1. The (1 − γ)-content symmetric simultaneous (1 − α)tolerance intervals for the speed-orifice problem calculated at several specified predictors at the significance levels α = 0.05 and for prespecified population content 1 − γ, with γ = 0.05. The data and the explicit values of the tolerance factors for the selected simultaneous tolerance bounds, depicted by the symbol ×, are presented in Table 1.

For given k = 2, n = 15, and the significance levels α = 0.05, 0.01, the critical values of the test (4) and (8), respectively, for testing the null hypothesis H0 : (β, σ) = (β0 , σ0 ), are λ1−0.05 = 8.6813, λ1−0.01 = 12.6160, ∗ ∗ = 8.1578, F1−0.01 = 17.3985. F1−0.05 Compare with the Table 3 and Table 4. The tables of critical

are not presented in this paper. values F1−α The tolerance factors of the (1 − γ)-content symmetric simultaneous (1 − α)-tolerance intervals for the speed-orifice problem calculated at several specified predictors fi , i = 1, . . . , 15, (see the Table 1, columns 3-4, respectively), for different combinations of selected significance levels α = 0.05, α = 0.01, and population content 1 − γ, for γ = 0.05 and γ = 0.01, are given in Table 1, (see the columns 5-8). For illustration, using the function LowerBound for the speed-orifice data with the predictor vector x = (1, 1.3531), with the quantile of standard normal distribution uγ/2 = −1.96 for selected significance levels γ = 0.05 and with the quantile λ1−α = 8.6813 for selected confidence coefficient 1−α = 0.05, we get

An alternative algorithm for the approximate solution based on Monte Carlo simulations, which is especially useful for linear regression models with k > 2, was suggested in [11]. 4. Example

For numerical illustration we consider fifteen hypothetical pairs of values (xi , yi ) selected for the speed-orifice problem, as considered and studied in [3] and [5], the measurement values are given in Table 1. A simple linear regression model Yi = β1 +β2 xi +σZi , with independent errors Zi ∼ N(0, 1), i = 1, . . . , 15 (i.e. n = 15 and >> [lowerBound,factor] = ... k = 2), was considered for the analysis of the speed-orifice LowerBound([1,1.3531],y,X,-1.96,8.6813) measurements. The ML estimates βˆ of the regression param- >> lowerBound = 4.7228e+003 eter β = (β1 , β2 ) and σˆ ML of the standard deviation σ are >> factor = 3.7996  βˆ = (−19041.86, 17929.64) , σˆ ML = 121.50. The upper tolerance bound could be calculated from the tol 1 n 2 Moreover, x¯ = 1n ni=1 xi = 1.3531, S x = n−1 i=1 (xi − x¯) = erance factor, see (13), or directly by using the function 0.0292. LowerBound — by minimizing the function −(x β + u1−γ/2 ). 4

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009

Table 1: Tolerance factors of the (1 − γ)-content symmetric simultaneous (1 − α)-tolerance intervals for the speed-orifice problem calculated at several specified predictors for different combinations of selected significance levels α = 0.05, α = 0.01, and population content 1 − γ, for γ = 0.05 and γ = 0.01. Tolerance Factors Speed [miles/hour] yi 4360 4590 4520 4770 4760 5070 5230 5080 5550 5390 5670 5490 5810 6060 5940

α = 0.05

α = 0.01

Orifice Opening [in]

Predictors [in]

Standardized Predictors

xi

fi

( fi − x¯)/S x

γ = 0.05

γ = 0.01

γ = 0.05

γ = 0.01

1.3100 1.3130 1.3200 1.3220 1.3380 1.3400 1.3470 1.3550 1.3600 1.3640 1.3730 1.3760 1.3840 1.3950 1.4000

1.2362 1.2654 1.2800 1.2947 1.3093 1.3239 1.3385 1.3531 1.3678 1.3824 1.3970 1.4116 1.4262 1.4408 1.4701

-4 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 4

6.1212 5.3466 4.9779 4.6298 4.3139 4.0495 3.8664 3.7996 3.8664 4.0495 4.3139 4.6298 4.9779 5.3466 6.1212

7.0053 6.2590 5.9090 5.5836 5.2946 5.0593 4.9014 4.8451 4.9014 5.0593 5.2946 5.5836 5.9090 6.2590 7.0053

7.5563 6.5510 6.0722 5.6201 5.2095 4.8654 4.6268 4.5396 4.6268 4.8654 5.2095 5.6201 6.0722 6.5510 7.5563

8.5817 7.6125 7.1578 6.7348 6.3585 6.0519 5.8459 5.7723 5.8459 6.0519 6.3585 6.7348 7.1578 7.6125 8.5817

5. Discussion The presented exact likelihood ratio test (4) for testing the null hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative H1 : (β, σ)  (β0 , σ0 ) on the parameters β and σ of the linear regression model Y = Xβ + σZ with normally distributed errors, Z ∼ N(0, In ), is especially useful for construction of the simultaneous confidence region for the regression parameters, see (10), and for the simultaneous tolerance intervals (11), and/or the tolerance factors (14), respectively, for future observations predicted for any value of the predictor vector x. If compared with the (1 − γ)-content symmetric simultaneous (1 − α)-tolerance intervals based on the product-set approach, as proposed by Limam and Thomas in [5], the suggested tolerance intervals given by (11) are slightly broader for the predictor values close to the middle of the fitted regression function, and became narrower for more distant predictors.

[2]

Acknowledgments

[9]

[3] [4]

[5]

[6]

[7] [8]

The research was supported by the grants VEGA 1/0077/09, VEGA 2/7087/27, APVV-SK-AT-0003-09, and APVV- [10] RPEU-0008-06. [11]

References [1] De Gryze, S., Langhans, I. & Vandebroek, M. (2007). Using the correct intervals for prediction: A tutorial on tolerance

5

intervals for ordinary least-squares regression. Chemometrics and Intelligent Laboratory Systems 87, 147–154. Krishnamoorthy, K. & Mathew, T. (2009). Statistical Tolerance Regions: Theory, Applications, and Computation. Wiley. Lieberman, G. J. & Miller, R. G. Jr. (1963). Simultaneous tolerance intervals in regression. Biometrika 50, 155–168. Lieberman, G. J., Miller, R. G. Jr. & Hamilton, A. (1967). Unlimited simultaneous discrimination intervals in regression. Biometrika 54, 133–145. Corrections in Biometrika 58, 687. Limam, M. M. T. & Thomas, D. R. (1988). Simultaneous tolerance intervals for the linear regression model. Journal of the American Statistical Association 83 (403), 801–804. Mee R. W., Eberhardt, K. R. & Reeve, C. P. (1991). Calibration and simultaneous tolerance intervals for regression. Technometrics 33 (2), 211–219. Scheff´e, H. (1973). A statistical theory of calibration. Annals of Statistics 1(1), 1–37. Wallis, W. A. (1951). Tolerance intervals for linear regression. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Berkeley CA: University of California Press, Berkeley. Wilks, S. S. (1942). Statistical prediction with special reference to the problem of tolerance limits. The Annals of Mathematical Statistics 13, 400–409. Wilson, A. L. (1967). An approach to simultaneous tolerance intervals in regression. The Annals of Mathematical Statistics 38(5), 1536–1540. Witkovsk´y, V. & Chvostekov´a M. (2009). Simultaneous tolerance intervals for the linear regression model. In MEASUREMENT 2009. Proceedings of the International Conference on Measurement, Smolenice, May 20-23, 2009.

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009

Table 2: Critical values of the likelihood ratio test (LRT) for testing the null hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative H1 : (β, σ)  (β0 , σ0 ) on parameters of the normal linear regression model with k, k = 1, . . . , 10, explanatory variables, selected small sample sizes n, n = k + 1, . . . , 100, and the significance level α = 0.1.

n/k

1

2

3

4

5

6

7

8

9

10

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 45 50 55 60 65 70 75 80 85 90 95 100 ∞

9.0856 6.8029 6.0549 5.6856 5.4659 5.3203 5.2168 5.1395 5.0795 5.0316 4.9925 4.9599 4.9324 4.9089 4.8885 4.8707 4.8550 4.8410 4.8286 4.8173 4.8072 4.7979 4.7895 4.7818 4.7747 4.7682 4.7621 4.7565 4.7513 4.7464 4.7418 4.7375 4.7335 4.7297 4.7261 4.7228 4.7196 4.7166 4.7137 4.7013 4.6915 4.6835 4.6771 4.6712 4.6664 4.6623 4.6587 4.6555 4.6527 4.6502 4.6479 4.6052

15.1840 10.6953 9.2154 8.4775 8.0346 7.7393 7.5279 7.3695 7.2460 7.1471 7.0662 6.9986 6.9414 6.8924 6.8499 6.8126 6.7798 6.7505 6.7244 6.7008 6.6795 6.6601 6.6424 6.6262 6.6112 6.5974 6.5847 6.5728 6.5617 6.5514 6.5418 6.5327 6.5242 6.5162 6.5087 6.5015 6.4948 6.4884 6.4609 6.4391 6.4214 6.4068 6.3945 6.3840 6.3749 6.3670 6.3600 6.3539 6.3484 6.3434 6.2514

21.6002 14.7996 12.5243 11.3749 10.6781 10.2095 9.8722 9.6176 9.4185 9.2585 9.1271 9.0171 8.9239 8.8437 8.7740 8.7129 8.6589 8.6108 8.5678 8.5289 8.4937 8.4617 8.4324 8.4056 8.3808 8.3580 8.3368 8.3171 8.2987 8.2816 8.2656 8.2505 8.2364 8.2231 8.2105 8.1987 8.1874 8.1392 8.1012 8.0705 8.0451 8.0238 8.0056 7.9900 7.9764 7.9644 7.9538 7.9444 7.9359 7.7794

28.2608 19.0877 15.9760 14.3856 13.4126 12.7532 12.2757 11.9136 11.6292 11.3998 11.2108 11.0524 10.9176 10.8015 10.7005 10.6117 10.5332 10.4631 10.4002 10.3435 10.2921 10.2453 10.2024 10.1630 10.1268 10.0932 10.0621 10.0332 10.0062 9.9810 9.9574 9.9353 9.9145 9.8948 9.8763 9.8588 9.7840 9.7252 9.6778 9.6388 9.6061 9.5784 9.5545 9.5337 9.5155 9.4994 9.4850 9.4722 9.2364

35.1190 23.5341 19.5577 17.5048 16.2386 15.3748 14.7461 14.2670 13.8893 13.5838 13.3313 13.1190 12.9381 12.7820 12.6459 12.5262 12.4200 12.3253 12.2401 12.1633 12.0935 12.0298 11.9716 11.9180 11.8686 11.8229 11.7805 11.7411 11.7043 11.6699 11.6376 11.6074 11.5789 11.5521 11.5267 11.4187 11.3343 11.2664 11.2108 11.1643 11.1248 11.0909 11.0615 11.0357 11.0130 10.9927 10.9746 10.6446

42.1428 28.1187 23.2566 20.7249 19.1522 18.0733 17.2841 16.6803 16.2027 15.8151 15.4940 15.2236 14.9925 14.7928 14.6183 14.4647 14.3283 14.2063 14.0967 13.9976 13.9075 13.8253 13.7500 13.6807 13.6168 13.5576 13.5026 13.4514 13.4037 13.3590 13.3172 13.2778 13.2408 13.2059 13.0576 12.9422 12.8499 12.7743 12.7113 12.6580 12.6123 12.5727 12.5381 12.5075 12.4803 12.4559 12.0170

49.3085 32.8254 27.0617 24.0382 22.1485 20.8455 19.8882 19.1532 18.5701 18.0955 17.7014 17.3687 17.0839 16.8374 16.6218 16.4315 16.2625 16.1112 15.9750 15.8517 15.7396 15.6373 15.5434 15.4569 15.3771 15.3032 15.2345 15.1706 15.1108 15.0549 15.0025 14.9532 14.9068 14.7104 14.5583 14.4371 14.3382 14.2559 14.1865 14.1271 14.0756 14.0307 13.9910 13.9558 13.9244 13.3616

56.5988 37.6412 30.9634 27.4378 25.2223 23.6877 22.5561 21.6843 20.9906 20.4246 19.9537 19.5553 19.2137 18.9176 18.6581 18.4290 18.2251 18.0424 17.8779 17.7288 17.5930 17.4690 17.3552 17.2503 17.1534 17.0636 16.9801 16.9023 16.8296 16.7616 16.6977 16.6377 16.3844 16.1893 16.0345 15.9086 15.8041 15.7161 15.6410 15.5760 15.5193 15.4694 15.4251 15.3855 14.6837

64.0000 42.5556 34.9535 30.9172 28.3688 26.5963 25.2847 24.2713 23.4629 22.8018 22.2505 21.7834 21.3823 21.0339 20.7283 20.4581 20.2174 20.0016 19.8069 19.6304 19.4696 19.3225 19.1874 19.0629 18.9478 18.8411 18.7418 18.6492 18.5627 18.4816 18.4056 18.0857 17.8408 17.6472 17.4903 17.3605 17.2514 17.1584 17.0781 17.0081 16.9466 16.8921 16.8434 15.9872

71.5009 47.5599 39.0252 34.4711 31.5834 29.5677 28.0715 26.9123 25.9853 25.2257 24.5911 24.0524 23.5892 23.1863 22.8325 22.5193 22.2400 21.9893 21.7630 21.5576 21.3704 21.1990 21.0415 20.8962 20.7618 20.6371 20.5211 20.4129 20.3117 20.2168 19.8197 19.5174 19.2795 19.0873 18.9288 18.7958 18.6827 18.5852 18.5004 18.4259 18.3599 18.3012 17.2750

6

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009

Table 3: Critical values of the likelihood ratio test (LRT) for testing the null hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative H1 : (β, σ)  (β0 , σ0 ) on parameters of the normal linear regression model with k, k = 1, . . . , 10, explanatory variables, selected small sample sizes n, n = k + 1, . . . , 100, and the significance level α = 0.05.

n/k

1

2

3

4

5

6

7

8

9

10

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 45 50 55 60 65 70 75 80 85 90 95 100 ∞

11.8545 8.8706 7.8893 7.4046 7.1164 6.9257 6.7901 6.6888 6.6103 6.5477 6.4966 6.4541 6.4182 6.3874 6.3608 6.3375 6.3170 6.2988 6.2825 6.2679 6.2546 6.2426 6.2316 6.2216 6.2123 6.2038 6.1959 6.1885 6.1817 6.1753 6.1694 6.1638 6.1586 6.1536 6.1490 6.1446 6.1404 6.1363 6.1328 6.1167 6.1038 6.0934 6.0847 6.0774 6.0712 6.0658 6.0611 6.0570 6.0533 6.0500 6.0470 5.9915

19.3470 13.4989 11.5844 10.6358 10.0694 9.6927 9.4241 9.2228 9.0663 8.9412 8.8388 8.7536 8.6813 8.6195 8.5659 8.5190 8.4776 8.4408 8.4079 8.3783 8.3515 8.3271 8.3048 8.2844 8.2657 8.2483 8.2323 8.2174 8.2035 8.1906 8.1785 8.1671 8.1564 8.1464 8.1369 8.1280 8.1195 8.1115 8.0770 8.0497 8.0276 8.0092 7.9938 7.9806 7.9693 7.9594 7.9507 7.9429 7.9360 7.9299 7.8147

27.1540 18.3296 15.4138 13.9556 13.0778 12.4904 12.0691 11.7521 11.5047 11.3063 11.1435 11.0075 10.8923 10.7933 10.7074 10.6321 10.5656 10.5064 10.4533 10.4056 10.3623 10.3230 10.2870 10.2540 10.2236 10.1955 10.1695 10.1454 10.1229 10.1019 10.0822 10.0638 10.0464 10.0301 10.0147 10.0002 9.9864 9.9274 9.8809 9.8433 9.8122 9.7861 9.7640 9.7448 9.7282 9.7136 9.7006 9.6891 9.6788 9.4877

35.2052 23.3410 19.3806 17.3814 16.1687 15.3518 14.7630 14.3179 13.9693 13.6888 13.4581 13.2649 13.1008 12.9596 12.8369 12.7292 12.6339 12.5490 12.4728 12.4041 12.3419 12.2852 12.2334 12.1858 12.1420 12.1014 12.0639 12.0290 11.9964 11.9660 11.9376 11.9108 11.8857 11.8621 11.8398 11.8187 11.7285 11.6577 11.6007 11.5537 11.5144 11.4811 11.4523 11.4274 11.4055 11.3861 11.3689 11.3534 11.0705

43.4538 28.5094 23.4748 20.9120 19.3464 18.2858 17.5176 16.9345 16.4763 16.1065 15.8015 15.5456 15.3278 15.1400 14.9765 14.8329 14.7056 14.5920 14.4901 14.3981 14.3146 14.2386 14.1689 14.1050 14.0460 13.9915 13.9409 13.8938 13.8500 13.8089 13.7705 13.7345 13.7005 13.6686 13.6384 13.5098 13.4094 13.3288 13.2627 13.2074 13.1606 13.1204 13.0855 13.0549 13.0279 13.0038 12.9823 12.5916

51.8679 33.8153 27.6847 24.5412 22.6089 21.2931 20.3360 19.6068 19.0319 18.5667 18.1821 17.8587 17.5829 17.3448 17.1371 16.9544 16.7923 16.6475 16.5175 16.3999 16.2932 16.1959 16.1067 16.0247 15.9491 15.8791 15.8142 15.7538 15.6974 15.6447 15.5952 15.5488 15.5052 15.4640 15.2892 15.1533 15.0446 14.9557 14.8816 14.8190 14.7653 14.7187 14.6780 14.6421 14.6102 14.5816 14.0671

60.4240 39.2429 31.9998 28.2622 25.9522 24.3719 23.2179 22.3357 21.6383 21.0724 20.6035 20.2085 19.8711 19.5793 19.3244 19.0998 18.9004 18.7221 18.5618 18.4167 18.2849 18.1646 18.0543 17.9528 17.8592 17.7725 17.6919 17.6170 17.5470 17.4815 17.4201 17.3624 17.3081 17.0783 16.9006 16.7591 16.6437 16.5478 16.4668 16.3975 16.3376 16.2852 16.2391 16.1981 16.1615 15.5073

69.1047 44.7793 36.4110 32.0684 29.3717 27.5192 26.1616 25.1207 24.2955 23.6244 23.0673 22.5971 22.1946 21.8462 21.5414 21.2725 21.0335 20.8196 20.6270 20.4526 20.2941 20.1492 20.0163 19.8940 19.7811 19.6764 19.5792 19.4886 19.4040 19.3248 19.2505 19.1807 18.8863 18.6599 18.4803 18.3343 18.2134 18.1115 18.0244 17.9493 17.8837 17.8259 17.7747 17.7290 16.9190

77.8962 50.4142 40.9101 35.9540 32.8629 30.7317 29.1648 27.9601 27.0028 26.2225 25.5736 25.0249 24.5546 24.1468 23.7897 23.4743 23.1936 22.9421 22.7155 22.5102 22.3234 22.1525 21.9958 21.8513 21.7179 21.5941 21.4791 21.3719 21.2718 21.1780 21.0900 20.7204 20.4377 20.2144 20.0335 19.8840 19.7584 19.6513 19.5590 19.4785 19.4078 19.3451 19.2892 18.3070

86.7873 56.1388 45.4906 39.9134 36.4216 34.0061 32.2250 30.8522 29.7589 28.8659 28.1220 27.4919 26.9512 26.4816 26.0700 25.7060 25.3817 25.0910 24.8288 24.5911 24.3745 24.1764 23.9944 23.8267 23.6717 23.5278 23.3941 23.2693 23.1528 23.0435 22.5867 22.2394 21.9663 21.7459 21.5643 21.4120 21.2824 21.1709 21.0738 20.9886 20.9132 20.8460 19.6751

7

MEASUREMENT SCIENCE REVIEW, Volume 9, Section 1, No. 1, 2009

Table 4: Critical values of the likelihood ratio test (LRT) for testing the null hypothesis H0 : (β, σ) = (β0 , σ0 ) against the alternative H1 : (β, σ)  (β0 , σ0 ) on parameters of the normal linear regression model with k, k = 1, . . . , 10, explanatory variables, selected small sample sizes n, n = k + 1, . . . , 100, and the significance level α = 0.01.

n/k

1

2

3

4

5

6

7

8

9

10

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 45 50 55 60 65 70 75 80 85 90 95 100 ∞

18.2902 13.6860 12.1608 11.4053 10.9553 10.6585 10.4474 10.2898 10.1676 10.0705 9.9911 9.9249 9.8693 9.8217 9.7804 9.7443 9.7126 9.6845 9.6593 9.6366 9.6162 9.5975 9.5805 9.5655 9.5506 9.5376 9.5253 9.5140 9.5033 9.4936 9.4845 9.4758 9.4677 9.4601 9.4528 9.4462 9.4393 9.4337 9.4279 9.4031 9.3833 9.3673 9.3538 9.3427 9.3330 9.3247 9.3175 9.3109 9.3054 9.3003 9.2958 9.2103

29.0049 19.9602 17.0089 15.5571 14.6967 14.1276 13.7236 13.4219 13.1882 13.0017 12.8495 12.7229 12.6160 12.5245 12.4452 12.3760 12.3149 12.2606 12.2122 12.1686 12.1286 12.0933 12.0606 12.0306 12.0031 11.9773 11.9540 11.9323 11.9119 11.8927 11.8753 11.8586 11.8430 11.8283 11.8145 11.8013 11.7889 11.7773 11.7269 11.6869 11.6546 11.6279 11.6054 11.5862 11.5697 11.5552 11.5424 11.5312 11.5213 11.5122 11.3449

40.0323 26.4262 21.9730 19.7741 18.4648 17.5960 16.9772 16.5139 16.1540 15.8663 15.6308 15.4351 15.2692 15.1271 15.0040 14.8962 14.8011 14.7165 14.6409 14.5728 14.5112 14.4552 14.4040 14.3571 14.3140 14.2742 14.2373 14.2031 14.1712 14.1414 14.1136 14.0875 14.0629 14.0398 14.0181 13.9976 13.9779 13.8946 13.8290 13.7760 13.7322 13.6955 13.6643 13.6374 13.6140 13.5935 13.5753 13.5590 13.5445 13.2767

51.3036 33.0704 27.0678 24.0871 22.3031 21.1139 20.2636 19.6250 19.1275 18.7288 18.4020 18.1293 17.8982 17.6998 17.5276 17.3768 17.2436 17.1251 17.0189 16.9233 16.8368 16.7579 16.6859 16.6199 16.5591 16.5029 16.4509 16.4026 16.3576 16.3155 16.2762 16.2393 16.2046 16.1719 16.1411 16.1120 15.9877 15.8903 15.8118 15.7474 15.6935 15.6476 15.6082 15.5740 15.5440 15.5175 15.4939 15.4727 15.0863

62.7725 39.8704 32.2873 28.5000 26.2217 24.6963 23.6016 22.7767 22.1322 21.6145 21.1893 20.8337 20.5319 20.2725 20.0470 19.8493 19.6744 19.5186 19.3790 19.2531 19.1390 19.0352 18.9402 18.8530 18.7728 18.6985 18.6297 18.5658 18.5062 18.4506 18.3984 18.3495 18.3036 18.2603 18.2194 18.0455 17.9098 17.8013 17.7119 17.6375 17.5745 17.5204 17.4735 17.4324 17.3961 17.3639 17.3350 16.8119

74.4068 46.8075 37.6208 33.0087 30.2212 28.3473 26.9976 25.9775 25.1784 24.5350 24.0055 23.5619 23.1847 22.8600 22.5775 22.3294 22.1097 21.9139 21.7382 21.5796 21.4358 21.3048 21.1850 21.0749 20.9734 20.8796 20.7926 20.7116 20.6362 20.5657 20.4996 20.4376 20.3793 20.3244 20.0914 19.9106 19.7662 19.6482 19.5500 19.4670 19.3958 19.3343 19.2804 19.2329 19.1907 19.1529 18.4753

86.1831 53.8658 43.0585 37.6075 34.2989 32.0664 30.4533 29.2306 28.2704 27.4956 26.8567 26.3205 25.8640 25.4704 25.1275 24.8260 24.5588 24.3204 24.1063 23.9129 23.7374 23.5773 23.4308 23.2962 23.1720 23.0572 22.9506 22.8514 22.7589 22.6724 22.5914 22.5153 22.4437 22.1411 21.9075 21.7218 21.5706 21.4450 21.3391 21.2485 21.1702 21.1019 21.0416 20.9882 20.9404 20.0902

98.0839 61.0328 48.5916 42.2904 38.4511 35.8518 33.9680 32.5365 31.4097 30.4986 29.7459 29.1133 28.5738 28.1081 27.7018 27.3443 27.0271 26.7438 26.4892 26.2590 26.0499 25.8592 25.6845 25.5238 25.3755 25.2383 25.1109 24.9923 24.8817 24.7782 24.6811 24.5900 24.2063 23.9117 23.6785 23.4893 23.3326 23.2007 23.0882 22.9911 22.9064 22.8318 22.7658 22.7068 21.6660

110.0954 68.2983 54.2123 47.0516 42.6737 39.7007 37.5402 35.8945 34.5963 33.5447 32.6744 31.9419 31.3163 30.7755 30.3034 29.8874 29.5180 29.1877 28.8906 28.6219 28.3776 28.1546 27.9502 27.7621 27.5885 27.4277 27.2784 27.1393 27.0094 26.8879 26.7740 26.2963 25.9318 25.6444 25.4120 25.2202 25.0591 24.9220 24.8038 24.7009 24.6104 24.5304 24.4589 23.2093

122.2068 75.6532 59.9139 51.8861 46.9629 43.6102 41.1679 39.3033 37.8296 36.6337 35.6425 34.8069 34.0924 33.4741 32.9336 32.4569 32.0333 31.6542 31.3129 31.0040 30.7230 30.4664 30.2309 30.0142 29.8140 29.6285 29.4561 29.2956 29.1456 29.0052 28.4191 27.9747 27.6260 27.3450 27.1137 26.9200 26.7554 26.6138 26.4907 26.3827 26.2871 26.2020 24.7250

8