null distribution of multiple correlation coefficient under mixture normal ...

IJMMS 30:4 (2002) 249–255 PII. S0161171202012838 http://ijmms.hindawi.com © Hindawi Publishing Corp.

NULL DISTRIBUTION OF MULTIPLE CORRELATION COEFFICIENT UNDER MIXTURE NORMAL MODEL HYDAR ALI and DAYA K. NAGAR Received 14 April 2001

The multiple correlation coefficient is used in a large variety of statistical tests and regression problems. In this article, we derive the null distribution of the square of the sample multiple correlation coefficient, R 2 , when a sample is drawn from a mixture of two multivariate Gaussian populations. The moments of 1 − R 2 and inverse Mellin transform have been used to derive the density of R 2 . 2000 Mathematics Subject Classification: 62H10, 62H15.

1.
Introduction. µ(p×1), and Σ(p×p) > 0 are partitioned as 
 Suppose
that x(p×1),  x1 µ1 σ11 σ
21 , µ = , and Σ = , where x(2) = (x2 , . . . , xp )
and µ(2) = (µ2 , . . . , µp )
x(2) µ(2) σ Σ

x=

21

22

are (p − 1) × 1 and Σ22 is (p − 1) × (p − 1), so that Var(x1 ) = σ11 , Cov(x(2) ) = Σ22 , and σ12 is the (p − 1) × 1 vector of covariances between x1 and x2 , . . . , xp . The multiple ¯1·2···p , is defined as correlation coefficient between x1 and x(2) , denoted by R  ¯1·2···p = R

σ
21 Σ−1 22 σ21 σ11

1/2 .

(1.1)

Let A be the sample sum of squares and
products  matrix formed from N indepena a
dent observations on x. Partition A as A = a 11 A21 , where A22 is (p −1)×(p −1). The 21

22

sample multiple correlation coefficient between x1 and x(2) is defined by  
−1 a21 A22 a21 1/2 . R= a11

(1.2)

It is well known that, when the underlying population is normal, the random matrix A has Wishart distribution with n = N − 1 degrees of freedom and parameter matrix Σ. ¯1·2···p = 0 if and only if x1 is independent of x(2) = (x2 , . . . , xp ). Furthermore, Further, R ¯1·2···p is zero, the distribution when the population multiple correlation coefficient R of R 2 is beta with parameters (1/2)(p − 1) and (1/2)(N − p). In practice, it is often the case that the random variables are not normally distributed. When such is the case, how would the departure from the normality affect the conventional inference procedure? Specifically, one may wonder what would be the sampling distributions of some commonly used statistics? For providing some answers to the above questions, Srivastava and Awan [9] and Tan [11] derived the distribution of the sample sum of squares and products matrix when sampling from a mixture of two multivariate normal distributions. The normal mixture is defined as follows:     (1.3) f (x) = Np µ1 , Σ; x + (1 − )Np µ2 , Σ; x , x ∈ Rp ,

250

H. ALI AND D. K. NAGAR

where Np (µ, Σ; x)

  1 = (2π )−(1/2)p det(Σ)−1/2 exp − (x − µ)
Σ−1 (x − µ) , 2

x ∈ Rp , µ ∈ Rp , Σ > 0, (1.4)

and 1 −  is known as the degree of contamination. This model is very common in medical, biological, and agricultural experiments (Titterington et al. [12]). For results on the distribution theory and robustness studies of certain test statistics when sampling from a mixture normal model, see Srivastava [8], Srivastava and Awan [9, 10], Kabe and Gupta [5], Amey and Gupta [2], and Nagar and Castañeda [7]. Srivastava [8], using certain transformations, derived the null distribution of multiple correlation coefficient when sampling from a mixture of two multivariate normal distributions (see also Gupta and Kabe [3]). Amey [1] integrated the joint density of a11 , a21 , and A22 suitably to derive the density of R 2 and studied its robustness. In this article, we derive the null distribution of R 2 when sampling from a mixture of two multivariate normal distributions. First, we derive the hth null moment of 1−R 2 . Then, by using the inverse Mellin transform, the density of 1 − R 2 is obtained from which the density of R 2 is deduced. Note that R 2 is a function of the elements of sample sum of squares and products matrix A. Therefore, in our derivation, we use the distribution of A when sampling from the above model. Srivastava and Awan [9] and Tan [11] have shown that the density of A, when sampling from (1.3), is a binomial sum of linear noncentral Wishart densities:

   N γ  (1 − )N−γ Wp n, Σ, cγ2 Σ−1 νν
; A , f (A) = γ γ=0 N

(1.5)

where n = N − 1, cγ2 = γ(N − γ)/N, and ν = (µ1 − µ2 ). Here Wp (n, Σ, cγ2 Σ−1 νν
; A) represents the noncentral Wishart density with n degrees of freedom and noncentrality parameter matrix cγ2 Σ−1 νν
defined by     1 1 (p) 1 n; cγ2 Σ−1 AΣ−1 νν
, Kp (n, Σ, ν) etr − Σ−1 A det(A)(1/2)(n−p−1) 0 F1 2 2 4

(1.6)

where     −1  1 1 n det(Σ)(1/2)n etr − cγ2 Σ−1 νν
Kp (n, Σ, ν) = 2(1/2)pn Γp 2 2 and Γm (a) = π (1/4)m(m−1)

(1.7)

m

j=1 Γ (a − (1/2)(j − 1)).

2. Null moments of 1 − R 2 . In this section, we derive moments of 1 − R 2 when 
 ¯ R1·2···p = 0 (or equivalently σ21 = 0). Let Σ0 = σ011 Σ022 and U = 1 − R 2 . Since a11 is scalar, then a
A−1 a21 det(A)  . = U = 1 − R 2 = 1 − 21 22 (2.1) a11 a11 det A22

NULL DISTRIBUTION OF MULTIPLE CORRELATION COEFFICIENT . . .

251

The hth null moment of U is given by

   N γ  (1 − )N−γ Eγ U h , γ γ=0

N   E Uh =

    Eγ U h = Kp n, Σ0 , ν

(2.2)

   −h 1 etr − Σ−1 A a−h 0 11 det A22 2 A>0 (2.3)   1 (p) 1 −1
n; cγ2 Σ−1 × det(A)(1/2)(n−p−1)+h 0 F1 AΣ νν dA. 0 0 2 4

−h by their integral representations, namely Replacing a−h 11 and det(A22 )

∞   1 1 a−h = exp − y y1h−1 dy1 , Re(h) > 0, a 11 1 11 2h Γ (h) 0 2

   −h 1 1 = (p−1)h etr − A22 Y22 det A22 2 Γp−1 (h) Y22 >0 2

h−(1/2)(p−1+1)  dY22 , × det Y22

Re(h) >

1 (p − 2), 2

respectively, in (2.3) and integrating A, the moment expression is rewritten as       2(1/2)np Kp n, Σ0 , ν Γp (1/2)n + h Eγ U h = Γ (h)Γp−1 (h)

∞

h−(1/2)p   −(1/2)n−h × y1h−1 det Y22 det Σ−1 0 +Y 0

(p)



× 1 F1 where Y =

 y1

0  0 Y22

(2.4)

(2.5)

Y22 >0

 −1 −1
 −1 1 1 1 Σ0 νν dy1 dY22 , n + h; n; cγ2 Σ−1 Σ0 + Y 0 2 2 2

(p)

and 1 F1

is the confluent hypergeometric function of matrix argu-

−1 −1 −1
ment (Gupta and Nagar [4]). Since rank(Σ−1 0 (Σ0 + Y ) Σ0 νν ) = 1, the only nonzero −1 −1 −1 −1
−1 −1
−1 characteristic root of the matrix Σ0 (Σ0 + Y ) Σ0 νν is tr((Σ−1 0 + Y ) Σ0 νν Σ0 ) and therefore,   −1 −1
 −1 1 1 (p) 1 n + h; n; cγ2 Σ−1 Σ0 νν Σ0 + Y 1 F1 0 2 2 2   (2.6)  −1 −1
−1  1 1 1 = 1 F1 n + h; n; cγ2 tr Σ−1 + Y Σ νν Σ , 0 0 0 2 2 2

where 1 F1 is the confluent hypergeometric function of scalar argument (see [6]). Substituting (2.6) in (2.5) and expanding 1 F1 in series form, the moment expression simplifies to     ∞  2 t   cγ   2(1/2)np Kp n, Σ0 , ν Γp (1/2)n + h (1/2)n + h t   Eγ U h = Γ (h)Γp−1 (h) 2 (1/2)n t t! t=0 ×

∞ 0

y1h−1

Y22 >0

  −(1/2)n−h h−(1/2)p det Y22 det Σ−1 0 +Y

 −1 −1 t  −1 × ν
Σ−1 Σ0 ν dy1 dY22 , Σ0 + Y 0

(2.7)

252

H. ALI AND D. K. NAGAR

where (a)r = a(a + 1) · · · (a + r − 1) and (a)0 = 1. Noting that Σ0 is a block diagonal matrix, we obtain −1 −1 t 
−1  −1 Σ0 ν ν Σ0 Σ0 + Y   −1  −1 −1 −1 t −1 = ν12 σ11 + ν
2 Σ−1 Σ22 ν2 1 + σ11 y1 22 Σ22 + Y2

=

k+=t

(2.8)

−1 k 
−1  −1 −1 −1  t!  2 −1  ν1 σ11 1 + σ11 y1 ν2 Σ22 Σ22 + Y22 Σ22 ν2 , k!!

 −1    −1     det Σ−1 1 + σ11 y1 det Σ22 det Ip−1 + Σ22 Y22 . 0 + Y = σ11 Now substituting (2.8) in (2.7), we have     2(1/2)np Kp n, Σ0 , ν Γp ((1/2)n + h) Eγ U h = det(Σ0 )(1/2)n+h Γ (h)Γp−1 (h)   2 k ∞  2 t  cγ (1/2)n + h t 1 ν1   × 2 k!! σ11 (1/2)n t t=0 k+=t

∞   −((1/2)n+h+k) × y1h−1 1 + σ11 y1 dy1

×

(2.9)

0

Y22 >0

  h−(1/2)p −(1/2)n−h det Y22 det Ip−1 + Σ22 Y22



 −1 −1 −1  × ν
2 Σ−1 Σ22 ν2 dY22 . 22 Σ22 + Y22 1/2

1/2

Substituting Z = (Ip−1 + Σ22 Y22 Σ22 )−1 , the integral involving Y22 is evaluated as

Y22 >0

  h−(1/2)p −(1/2)n−h 
−1  −1 −1 −1  det Y22 det Ip−1 + Σ22 Y22 Σ22 ν2 dY22 ν2 Σ22 Σ22 + Y22

 −h = det Σ22

det(Z)(1/2)(n−p) 0