On construction of improved estimators in multiple-design ...

Ann. Inst. Statist. Math. Vol. 47, No. 4, 665-674 (1995)

ON CONSTRUCTION OF IMPROVED ESTIMATORS IN MULTIPLE-DESIGN MULTIVARIATE LINEAR MODELS UNDER GENERAL RESTRICTION T. SHIRAISHI 1 AND Y. KONNO 2

1Department of Mathematical Sciences, Yokohama City University, 22-2 Seto, Kanazawa-ku, Yokohama 236, Japan 2 Department of Mathematics and [nformatics, Chiba University, 1-33 Yayoi-cho, Chiba 263, Japan

(Received December 20, 1993; revised January 4, 1995)

Abstract. Consider a set of p equations Yi = Xi~i + ei, i = 1,... ,p, where t h e rows of the r a n d o m error m a t r i x ( e l , . . . ,ep) : n x p are m u t u a l l y indep e n d e n t and identically d i s t r i b u t e d with a p-variate d i s t r i b u t i o n function F ( x ) having null m e a n and finite positive definite variance-covariance m a t r i x ]E. We are m a i n l y interested in an i m p r o v e m e n t upon a feasible generalized least squares e s t i m a t o r ( F G L S E ) for ~ = (~'1,... ,~'p)' when it is a priori suspected t h a t C ( = c0 m a y hold. For this problem, Saleh and Shiraishi (1992, Nonparametric Statistics and Related Topics (ed. A. K. Md. E. Saleh), 269-279, N o r t h - H o l l a n d , A m s t e r d a m ) investigated the p r o p e r t y of e s t i m a t o r s such as the shrinkage estimator. (SE), the positive-rule shrinkage e s t i m a t o r (PSE) in the light of their a s y m p t o t i c d i s t r i b u t i o n a l risks associated with the M a h a l a n o b i s loss function. We consider a general form of e s t i m a t o r s and give a sufficient condition for p r o p o s e d e s t i m a t o r s to improve on F G L S E with respect to their a s y m p t o t i c d i s t r i b u t i o n a l q u a d r a t i c risks ( A D Q R ) . T h e relative merits of these e s t i m a t o r s are s t u d i e d in the light of the A D Q R under local alternatives. It is shown t h a t the SE, the P S E and the K u b o k a w a - t y p e shrinkage e s t i m a t o r (KSE) o u t p e r f o r m the F G L S E a n d t h a t the P S E is the most effective a m o n g the four e s t i m a t o r s considered under C ( = co. It is also observed t h a t the P S E and the K S E fairly improve over the F G L S E . Lastly, the construction of estim a t o r s improved on a generalized least squares e s t i m a t o r is studied, assuming n o r m a l i t y when E is known.

Key words and phrases: Shrinkage estimators, generalized least squares estimators, a s y m p t o t i c d i s t r i b u t i o n , seenfingly u n r e l a t e d regression model.

1.

Introduction Consider p different regression models with cross correlation,

(1.1)

Yi:nxl=Xs~i+ei

and

E[eie}]=aijI~, 665

(i,j=l,...,p)

666

T. SHIRAISHI

AND

Y. K O N N O

w h e r e X i (~. x qi) is a design m a t r i x of full r a n k , {i (qi x 1) is a c o h m m v e c t o r of u n k n o w n p a r a m e t e r , a n d ei (n x 1) is an error t e r m v e c t o r c o n t i n u o u s l y d i s t r i b u t e d w i t h E[ei] = 0. W e also a s s u m e t h a t E = (crij)i4=l ..... p is p o s i t i v e definite. T h e m o d e l (1.1) is referred to as t h e seenfingly u n r e l a t e d regression m o d e l of Zellner (1962). We c a n r e w r i t e t h e m o d e l (1.1) as follows: (1.2)

Y = X~ + e

and

Coy(e) = ~ G~ L,

(xl~

w h e r e Y " ,z.p x 1 = ( Y ( . . . . , Y/,)', ~ : q • 1 = (~'1 . . . . . {'p)', e" ,,p x 1 = ( 4 . . . . . ~,)',

. . . .

]')

X : np x 1 . . . . . 0

.

0

...

and

q =

qi.

,

In this p a p e r , we are p r i m a r i l y i n t e r e s t e d in the e s t i m a t i o n of { w h e n it. is s u s p e c t e d that a general restriction (1.3)

Ho:

C{=

Co

holds, w h e r e C is an r x q m a t r i x of r a n k r a n d c0 is an r - d i m e n s i o n a l c o l u n m vector. T a k i n g a c c o u n t of the f o r m of t h e c o v a r i a n c e m a t r i x in (1.2) a n d utilizing a n y c o n s i s t e n t e s t i m a t o r s E of E , we have a feasible g e n e r a l i z e d least s q u a r e s estimator (FGLSE)

(1.4) For e x a m p l e , we m a y t a k e E,, = n - l [ m a t ( Y

- X~,,)] , [mat( Y - X~,,)],

rl X p

~,1 X p

as a c o n s i s t e n t e s t i m a t o r . Here, we define m a t , , • by mat,, xp(77) = ( g l . . . . . ~1,) for ;7 - , , p x 1 = ( g ; . . . . . 7/'p)' a n d p - d i m e n s i o n a l c o l u m n v e c t o r s Y,, i = 1 . . . . , p. See S r i v a s t a v a a n d Giles (1987) for a r e v i e w of t h e e s t i m a t o r (1.4). F u r t h e r m o r e , t a k i n g t h e r e s t r i c t i o n (1.3) into c o n s i d e r a t i o n , we have a r e s t r i c t e d estinm.tor ( R E ) ,

(1.5)

~,, = ~ , , - ( E ~ , ' # D , ~ ) - ' C ' { C ( E ~ , ~ # D , , )

~C'} ~(C~,,-eo),

w h e r e D,, : q x q = ( D n i j ) i , j = l ..... p w i t h D n i j : qi x qj = X i X I j / r t , a n d tile m a t r i x o p e r a t o r # is defined b y A # B = ( a i d B i d ) i , j = l ..... p for A = ( a i j ) i j = l ..... p a n d q x q m a t r i x B = ( B i j ) i , j = L ..... p so t h a t Bid is qi x qj. ~n m a y be b i a s e d a n d even i n c o n s i s t e n t unless the r e s t r i c t i o n (1.3) holds, while it p e r f o r m s b e t t e r t h a n ~,, w h e n (1.3) holds. So we p r o p o s e a w e i g h t e d c o m b i n a t i o n of ~,, a n d ~ , , b e i n g of t h e f o r m (1.6)

~gn =~r~ q - { I q - 9 ( s

,

CONSTRUCTION OF IMPROVED ESTIMATORS

667

where /2,, = n ( 4 . - ~ ( . ) ' C 'I"-, , t O ( 4 .

~.)

and

Fn = C ( ~ n l : # : D , ~ ) - 1 Q n ( ~ 7 7 1 # D n ) - 1 C ' . Here, s is a test statistic for testing Ho : C { = Co v.s. HA : C { r Co, and Q,, is a consistent estimator of Q which is a weight m a t r i x of full rank associated with a quadratic loss function (1.7)

(4 + - r

+ - ~)

for an estimator 4 + of ~. T h e estimator (1.6) reduces to one considered in Saleh and Shiraishi (1992) when O , = E , 7 1 # D .... We obtain a sufficient condition t h a t the proposed estimator o u t p e r f o r m s the F G L S E (1.1) with respect to their a s y m p t o t i c distributional risks (i.e., the risk by reference to the a s y m p t o t i c distribution of an estimator) associated with the loss function (1.7) under the local alternatives. We construct the shrinkage estimator (SE), the positive-rule shrinkage estimators (PSE), and the K u b o k a w a - t y p e estimator (KSE), which are b e t t e r t h a n the F G L S E . 2.

Asymptotic distributional quadratic risks (ADQR)

First, we introduce the assumption to c o m p u t e the a s y m p t o t i c distributional risks of estimators. Let. (2.1)

47 = { / t ( y ~ - I

@

.~n)X}-lxt(,y.]-I (7__)It,)Y,

which is referred to as the generalized least, squares estimator. ~, covariance m a t r i x of {,, is given by

The variance-

Va.r(47) = {X'(~-] -1 .~ I n ) X } -1 = ( E - t # D , , ) - ~

Now, we set. ASSUMPTION 1.

lim,,_~ n-lDn = A.

ASSUMPTION 2.

,/~(4,~ - ~) ~ N,,(0, (P,-~#a) -~), where L denotes convergence in distribution. Assumption 2 holds if Lindeberg's condition is satisfied. following contiguous sequence of alternatives A , : C r = co + O/v/~; 0 = ( e l , . . . ,

er)'.

We consider the

668

T.

SHIRAISHI

AND

Y. K O N N O

Then, we have the following Lemma from Saleh and Shiraishi (1992). LEMMA 2.1. Under Assumptions 1, 2 and "under A , , the asymptotic distributions of v~(~,~ - ~), v/-~C(~,, - ~,.), s and x / ~ ( ~ - ~r~) are given by

(2.2)

~/~(~n -- ~) ~ y r,o gq(_~to, (~--~-1}@~)-1 _ B 1 ) ,

(2.3)

, / ~ c ( ~ , , - ~,,) ~ z ~ N~(O, B~), s

and (2.4)

L Z,F_i Z,

V/~(~n _ ~.) ~(~-1 #/~)-1{ 6 , B 2 1 C } - 1 Z

~ Nq(#o , B1 ),

where B~ = ( E - ~ # A ) -~ C ' B 2 C ( E - I # A ) -~, #0 = ( E - ~ # A ) - I { C'B2 -1 C} -10, B2 = C ( N - I # A ) - ~ C ' and r = C ( E - l C / : A ) - I Q ( E - ~ C / : A ) - ~ C ' . Furthermore v/n(~, - ~), and v/n(~.n - ~,,) are asymptotically independent under A , . Assume that the asymptotic c.d.f, is obtained as +

G r ( ) = lira for an estimator ~+ of ~. Then we define the asymptotic distributional quadratic risk (ADQR) of ~..* by the expression /

(2.5)

^+

R(~+: Q) = lira lira E[min{n(~: - ~) Q(~,. - ~), b}] b~oc n ~

= / R ' x' QxdG+(x) = t r ( Q E + ) ' where E + = fR' xx'dG+(w) and Q is the weight matrix in the loss flmction (1.7). In order to obtain a sufficient condition for the proposed estimators to outperform the FGLSE, we investigate the ADQR for ~ . LEMMA 2.2. Under Assumptions 1, 2 and under A,, tl~e A D Q R R(~,.,,Q) for the estimator ~,,~ defined by (1.6) has the following expression, (2.6)

t r ( ( E - l # A ) -1 Q) - E[2rg( Y T -1 Y) + 4g'( Y ' r -1 Y) Y ' F -1 Y + g 2 ( Y ' F - 1 Y) y ' F -1 y ] .

PROOF.

(2.7)

Using (2.3) and (2.4), we get

v~(~,~ - ~ ) L y + [ { ( Z _ I # A ) _ I C , B 2 _ I C } _ ~ _ g ( z , r - 1 y ) ( ~ - ' # A ) -1 c , r -1] y ,

CONSTRUCTION

OF IMPROVED

ESTIMATORS

669

where Y and Z are independent and are respectively defined in (2.2) and (2.3). From (2.7), we get t h a t the A D Q R R ( ~ " Q) is expressed as

E{( Y + it0)' Q( Y + t~0)} + E[(Z - g ( Z ' r - 1 Z ) B 2 F - 1 Z - O)'B2-1rB2 -~ • ( Z - g ( z ' r - 1 Z ) S 2 r - 1 Z - 0)]. After a simple computation, it follows t h a t the A D Q R R(~,, : Q) is equal to t r ( E - t # Z ~ ) - 1 Q - 2 E [ t r ( Z - O ) ' B 2 - t ( g ( z ' r -~ z ) z )

(2.8)

_

trg2(z,r-~z)z,r-lz].

Under the regularity conditions stated in Stein (1981) or Bilodeau and Kariya (1989), we have

E[g(z'r-~ z)z(z

_ O)'B2 -1] = E [ ( O / O Z ) ( g ( Z T - 1 Z ) Z ) ]

(O/OZ)(g(ZT-1Z)Z) is an r x r matrix whose (i,j) element is ( O / O Z j ) ( g ( Z T - 1 Z ) Z ~ ) for Z = ( Z ~ , . . . , Z~)' and O/OZ = (O/OZ~,..., O/OZ~)'.

where

From chain-rule and straightforward calculation, it follows t h a t

E[(O/OZ) ( g ( Z T -1 Z)Z)] = E [ g ( Z T - 1 Z ) L + 2 g ' ( Z ' r -~ z)zzT-1]. Finally, putting this equation into (2.8), we get (2.6). [] 3.

Asymptotic dominance

Using L e m m a 2.2, we give a sufficient condition for the estimators (1.6) to outperform the FGLSE.

Suppose that r > 3. Under Assumptions 1, 2 and .under A,,, the estimator ~ asymptotically outperfo,mns ~ if (i) 0 < g(u) < 2(r - 2) for any (ii) is onincrea ing. THEOREM 3.1.

PROOF.

This is a direct consequence fi'om Lennna 2.2. []

Set 9(u) = I(u r - 2), and r u -1 in (1.6), where IA is an indicator function, r is increasing, limt~oo r = r - 2, (3.1)

r

> r

= r - 2 - 2f~(u)/

/0

t-~f~(t)dt,

and f,.(t) is a density of the chi-square distribution with r degrees of freedom. Then, we define a preliminary test estimator (PTE), a shrinkage estimator (SE), a

670

T. SHIRAISHI

AND

Y. K O N N O

positive-rule shrinkage e s t i m a t o r ( P S E ) , a n d a K u b o k a w a - t y p e shrinkage e s t i m a t o r *KS

(t 3. Under Assumptions 1, 2 and "under ~PS

^l 3.

Put Q,, = (~'#D,,)

3.2.

and suppose that -PS

Under Assumptions 1, 2 and under Ho, th,e A D Q R of~,, *S

^KS

-PT

those of ~,,, ~,, , ~,,

*PT

and k =

j~O ='~

j x

{1 - r

9 9

' s),R(LT

-.S

-g ^ lf,5 if(;, = ~,,

if ~,~ ^g = ~,,,

from which it. follows t h a t

- s ) 1 (< 1), ~,, is better (worse) t h a n ~,,. The A R R E ' s of the estimators depend on underlying distribution only through the noncentrality parameter ~2. For 62 = 0.0(0.5)20, the values of the risks for ~,/'s' a n d ~I~'~q' were estimated from a Monte Carlo simulation with 10,000 repetitions. In the first setting, we restricted our attention to the case t h a t q = 6 and r = 4. In Fig. 1, ^PS

^KS

we drew the graphical picture of the A R R E ' s of the estimators ~,~ and ~,, with respect to the estimator ~,. l~'om Fig. 1, we can see t h a t (i) the improvements of the PSE and the KSE over the F G L S E decrease in (52 and (ii) the PSE is better t h a n the KSE for all 52 satisfying 0 < ~2 < 2.6 while the latter is better t h a n the former for 2.8 < 6 '~. Further, we find that, for q = 2 + r, the A R R E ' s of the PSE and the KSE relative to the F G L S E increase in r.

672

T. S H I R A I S H I

AND

Y. K O N N O

ARRE

1.7 1.6 1.5 1.4. 1.3 ~'%,~

- I(S

-

1.2 1.1 1.0 0.9 0.8-

'''•'''•'''•'''•'''•'`'•'''•'''•'''•''`•''']'''•''']'''•'''•'''•'''•'''•`''•'''• 0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20

62 Fig. 1.

4.

Tile a s y m p t o t i c relative risk efficiency of ~PS and ~,KS with respect to ~,,.

Normal theory

In this section, we assume that the underlying distribution F ( x ) is a pvariate normal distribution with null mean and known positive-definite variancecox,ariance matrix E. We state the normal theory shrinkage estimators, which is a general theory of a classical result. Based on 4" defined by (2.1), estimators are constructed. The weighted combination of the generalized least squares estimator and restricted estimator corresponding to (1.6) is proposed, being of the form (4.1)

~,7 = ~,~ + {x~ - g ( C : ; ) ( r . - l # D . ) -

t C ' r * - ~ c } ( ~ , ., . . . - r

where

~,~ = ~,~ - ( E - I # D , , ) - I c ' { c ( E - t # D , , ) - I C ' } __/~* ~, C , F . _

c : , = ,~.(~,: . . . . .

-

-

1

....

c(~n

-

~..)

(C~,-* - -

C0)

~

~nd

r* = C(r.-~#D,,)-~Q(ZC*-I#D.)-~C'.

By setting 9(u) = I(u < ~,~,~), ( r - 2). u -1, 1 - {1 - ( r - 2). u - l } 9I(u > r and r

2),

^*PT

u -1 in (4.1), we define the preliminary test estimator (PTE) ~. . . . ^*S

^*PS

shrinkage estimator (SE) ~, , positive-rule shrinkage estimator (PSE) ~,~ Kubokawa-type shrinkage estimator (KSE) ~:J,:s, respectively.

, and

CONSTRUCTION

OF IMPROVED ESTIMATORS

673

The quadratic risk (QR) of ~n+ is given by

T~ ^+

Q)

+

=

_

,

-

}.

Corresponding to Corollary 3.1 and Theorem 3.3, we get Theorems 4.1 and 4.2 respectively. THEOREM 4.1. namely,

^*S

: Q),

THEOREIVl 4.2.

P~(tj,,

: Q),

Suppose that 'r >_ 3. ^*PS

Mahalanobis loss is taken, ~r,.

^*KS

and {n

g(~,,

: Q),

/~(~n

~*KS

and ~,~

-.

dominate ~,~,

: Q) _< ~.(~,, : Q).

For Q = E - I # D , , ,

namely, when

^*S

dominate ~.,~ , namely,

--*KS'

-*PS

~(~,~

^*PS

Suppose that r >_ 3. The ~n , ~,~

^*S

~*

: Q)