Comparing Two Approaches in Heteroscedastic Regression Models
arXiv:1810.09497v1 [stat.ME] 22 Oct 2018
A. A. Jafari∗ Department of Statistics, Yazd University, Yazd, Iran
Abstract Recently, a generalized test approach is proposed by Sadooghi-alvandi et al. (2016) and a fiducial approach is proposed by Xu and Li (2018) to test the equality of coefficients in several regression models with unequal variances. In this paper, it is shown that the considered quantities in these approaches are identically distributed and therefore, these approaches are same. Also, this result satisfies for the one-way ANOVA problem.
Keywords: Fiducial approach; Generalized test variable; One-Way ANOVA; Regression.
1
Introduction
Consider the k regression models Y i = X i β i + εi
εi ∼ N 0, σi2 Ini ,
i = 1, . . . , k,
where Xi is ni × p design matrix with rank p, Y i = (Yi1 , . . . , Yini )′ is ni × 1 (ni > p for all i) observation vector, βi = (βi1 , βi2 , . . . , βip )′ is the vector of parameters with dimension of p, εi is the ni ×1 disturbance vector, and Ini is ni × ni identity matrix. Furthermore, all the εi are independent. ˆ = (X ′ Xi )−1 X ′ Y i and S 2 = It is well-known that the unbiased estimations for β i and σi2 are β i i i i −1 ′ ′ ′ Y i Ip − Xi (Xi Xi ) Xi Y i /(ni −p), respectively, such that they are independent. When the variances
σi2 ’s are unknown, an usual test statistic to test
H0 : β 1 = β 2 = · · · = β k ,
(1.1)
is (see Tian et al., 2009; Sadooghi-alvandi et al., 2016; Xu and Li, 2018) Q0 =
k X
ˆ′ Si−2 β i
i=1
ˆ (Xi′ Xi ) β i
−
"
k X
ˆ′ Si−2 β i
#"
(Xi′ Xi )
i=1
k X i=1
Si−2
(Xi′ Xi )
#−1 "
k X i=1
Si−2
ˆ (Xi′ Xi ) β i
#
.
Under null distribution, Q0 can be approximated by the chi-square distribution with p(k−1) degrees of freedom for large sample sizes. However, this approximation does not work well for small samples (see ∗
[email protected]
1
Tian et al., 2009; Sadooghi-alvandi et al., 2016; Xu and Li, 2018). Therefore, some other approaches are proposed to test equality of regression models with unequal variances for example a parametric bootstrap approach by Tian et al. (2009), a generalized approach by (Sadooghi-alvandi et al., 2016) and a fiducial approach by (Xu and Li, 2018). In this paper, we compare these generalized and fiducial approaches. We will see that although these approaches are proposed in different ways and are not the same in appearance but they are identical. In Section 2, the generalized and fiducial approaches are reviewed and compared. In Section 3, one way-ANOVA is discussed as a special case.
2
Comparing two approaches
For testing H0 in (1.1), a generalized approach is proposed by Sadooghi-alvandi et al. (2016) and a fiducial approach is proposed by Xu and Li (2018). In this section, these approaches are reviewed, briefly. Then, it is shown that they are identical. ˆ and S 2 , respectively. For given Consider bi and s2i , i = 1, . . . , k, are the observed values of β i i 2 bi , si , Xu and Li (2018) derived a fiducial quantity to test (1.1) as QF =
k X
t′i ti −
" k X
1
si−1 t′i (Xi′ Xi ) 2
i=1
i=1
#"
k X
s−2 i
#−1 "
(Xi′ Xi )
i=1
k X
1
′ 2 s−1 i (Xi Xi ) ti
i=1
#
,
(2.1)
where ti follows a multivariate student’s t-distribution tp (ni −p, 0, Ip ), i = 1, . . . , k, (see Kotz and Nadarajah, 2004) and they are mutually independent. Therefore, the p-value to test (1.1) is given by P (QF > Q0 ). N N Let H = C Ip where denotes Kronecker product, C = [Ik−1 : 1], and 1 = (1, . . . , 1)′ . Also, h −1 i −1 . Sadooghi-alvandi et al. (2016) defined a consider W ′ W = [HSH ′ ] where S = diag s2i Xj′ Xj generalized test variable to test the hypothesis in (1.1) as −1 (ni − p) s2i QG = Z ′ W H diag H ′ W ′ Z, Xj′ Xj Ui
(2.2)
where Z ∼ N (0, Ip(k−1) ) and Ui ∼ χ2(ni −p) , i = 1, . . . , k, such that Z and Ui ’s are mutually independent.
Therefore, the generalized p-value is given by P (QG > Q0 ). Lemma 2.1. QF and QG are identically distributed. Proof. Consider V i ∼ N (0, Ip ) and Ui ∼ χ2(ni −p) , i = 1, . . . , k, such that V i and Ui ’s are mutually i N h i −p Ip . Based on Sadooghi-alvandi et al. (2016), QG has independent. Also, consider D = diag nU i the same distribution as
Q∗G = V ′ (Ipk − qq ′ ) D (Ipk − qq ′ ) V , where
V =
V1 V2 .. . Vk
,
q=
q1 q2 .. . qk
,
−1/2 k h i X 1/2 ′ s−2 qi = s−1 Xj′ Xj . j i (Xi Xi ) j=1
2
h q i N ni −p Consider P = (Ipk − qq ′ ) and D∗ = diag Ip . Then Ui Q∗G
= V ′ P D∗ D∗ P V = V ′ D∗ P P D∗ V
= V ′ D∗ P D∗ V = V ′ D∗ (Ipk − qq ′ ) D∗ V = V ′ DV − V ′ D∗ qq ′ D∗ V #−1 #" k " k r k X X X ni − p 1 ni − p ′ −2 −1 ′ ′ ′ si (Xi Xi ) = V iV i − s V i (Xi Xi ) 2 Ui Ui i i=1 i=1 i=1 # " k r X ni − p 1 ′ 2 × s−1 i (Xi Xi ) V i U i i=1 =
k X
T ′i T i
i=1
where T i =
3
q
ni −p Ui V i
−
"
k X
1
′ ′ 2 s−1 i T i (Xi Xi )
i=1
#"
k X
#−1 "
′ s−2 i (Xi Xi )
i=1
k X
1
′ 2 s−1 i (Xi Xi ) T i
i=1
#
,
∼ tp (ni − p, 0, Ip ). Therefore, QF and QG are identically distributed.
One Way ANOVA problem
Let Yi1 , Yi2 , . . . , Yini is a random sample from a normal distribution with mean µi and variance σi2 , i = 1, . . . , k. The problem of testing equality of means of these k distributions, i.e. H0∗ : µ1 = · · · = µk , is well-known to one-way ANOVA. It is a special case of H0 in (1.1) with p = 1 and Xi = 1. Therefore, the fiducial quantity in (2.1) becomes to
QF =
k X i=1
t2i −
P k
i=1
Pk
q
ni si ti
ni i=1 s2i
2
,
(3.1)
where ti has a t distribution with ni − 1 degrees of freedom, which is the fiducial quantity introduced by Li et al. (2011). When p = 1 and Xi = 1, the generalized test variable in (2.2) becomes to (ni − p) s2i QG = Z ′ W C diag C ′ W ′ Z, ni U i where W ′ W = [CSC ′ ]
−1
(3.2)
, S = diag s2i /ni , Z ∼ N (0, I(k−1) ) and Ui ∼ χ2(ni −p) , i = 1, . . . , k. This
generalized test variable is introduced by Sadooghi-Alvandi et al. (2012). This generalized test variable is also proposed by Xu and Wang (2008) in another form (see Sadooghi-Alvandi et al., 2012). Based on Lemma, QF in (3.1) and QG in (3.2) are identically distributed. Therefore, the all proposed approaches by Xu and Wang (2008), Li et al. (2011) and Sadooghi-Alvandi et al. (2012) for the one-way ANOVA problem with unequal variances are same. 3
References Kotz, S. and Nadarajah, S. (2004). Multivariate t-Distributions and Their Applications. Cambridge University Press, Cambridge. Li, X., Wang, J., and Liang, H. (2011). Comparison of several means: A fiducial based approach. Computational Statistics & Data Analysis, 55(5):1993–2002. Sadooghi-Alvandi, S. M., Jafari, A. A., and Mardani-Fard, H. A. (2012). One-way ANOVA with unequal variances. Communications in Statistics-Theory and Methods, 41(22):4200–4221. Sadooghi-alvandi, S. M., Jafari, A. A., and Mardani-Fard, H. A. (2016). Comparing several regression models with unequal variances. Communications in Statistics - Simulation and Computation, 45(9):3190–3216. Tian, L., Ma, C., and Vexler, A. (2009). A parametric bootstrap test for comparing heteroscedastic regression models. Communications in Statistics-Simulation and Computation, 38(5):1026–1036. Xu, J. and Li, X. (2018). A fiducial p-value approach for comparing heteroscedastic regression models. Communications in Statistics - Simulation and Computation, 47(2):420–431. Xu, L.-W. and Wang, S.-G. (2008). A new generalized p-value for anova under heteroscedasticity. Statistics & Probability Letters, 78(8):963–969.
4
