A Comparison of Some Tests for Determining the Number of Nonzero ...

Communications in Statistics—Simulation and Computation® , 35: 727–749, 2006 Copyright © Taylor & Francis Group, LLC ISSN: 0361-0918 print/1532-4141 online DOI: 10.1080/03610910600716290

Multivariate Analysis

A Comparison of Some Tests for Determining the Number of Nonzero Canonical Correlations 1 2 ´ ´ TADEUSZ CALINSKI , MIROSŁAW KRZYSKO , 2 ´ AND WALDEMAR WOŁYNSKI 1

Department of Mathematical and Statistical Methods, August Cieszkowski Agricultural University of Poznan, ´ Poznan, ´ Poland 2 Faculty of Mathematics and Computer Science, Adam Mickiewicz University of Poznan, ´ Poznan, ´ Poland When considering the relationships between two sets of variates, the number of nonzero population canonical correlations may be called the dimensionality. In the literature, several tests for dimensionality in the canonical correlation analysis are known. A comparison of seven sequential test procedures is presented, using results from some simulation study. The tests are compared with regard to the relative frequencies of underestimation, correct estimation, and overestimation of the true dimensionality. Some conclusions from the simulation results are drawn. Keywords Bartlett–Nanda–Pillai trace statistic; Canonical correlations; Lawley– Hotelling trace statistic; Simulation study; Testing dimensionality hypotheses; Wilks statistic. Mathematics Subject Classification Primary 62H20; Secondary 62H15.

1. Introduction For studying interrelations between two sets of variables, the canonical correlation analysis proposed by Hotelling (1936) is often used. It consists in determining a linear transformation of the original variables from both sets into two new sets of variables not correlated within the sets but most highly correlated between them. Pairs of the corresponding new variables are called the canonical variates and the coefficients of correlations within the pairs are called canonical correlations. Most frequently, one is interested in reducing the number of the resulting pairs, to consider only those which are really highly correlated. This means to reduce Received February 27, 2004; Accepted January 6, 2006 Address correspondence to Tadeusz Calinski, Department of Mathematical and ´ Statistical Methods, August Cieszkowski Agricultural University of Poznan, ´ Wojska Polskiego 28, 60-637 Poznan, ´ Poland; E-mail: [email protected]

727

728

Calin´ ski et al.

the dimensionality of the two new coordinate systems. In the literature, several methods for the estimation of dimensionality in canonical correlation analysis are known. They are either of the type of sequential test procedures or direct estimation methods. The former are based on the well-known statistics, such as the Lawley– Hotelling trace statistic, the Wilks statistic, and the Bartlett–Nanda–Pillai trace statistic. The latter are based on some information criteria (IC), the Akaike IC and the Bayesian IC, in particular. The main purpose of the present article is to compare seven different test procedures, using results from extensive simulation study. To get some idea on the performance of ICs, one of them is also included in the study. The tests are compared with regard to the relative frequencies of underestimation, correct estimation, and overestimation of the true dimensionality.

2. Canonical Correlation Analysis Let Y and X be two vectors of p and q p ≤ q random variables, respectively, having a joint p + q-variate normal distribution with means 1 and 2 , respectively, and a positive definite covariance matrix

= 11 21

12 22

(1)

Then the population canonical correlations between Y and X, 1 > 1 ≥ 2 ≥ · · · ≥ d > d+1 = · · · = p = 0

(2)

are the positive square roots of the eigenvalues of −1 −1 12 22 21 11

where d is the unknown rank of 12 0 ≤ d ≤ p. Furthermore, let S S = 11 S21

S12 S22

be the sample unbiased covariance matrix formed from a sample of N ≥ p + q + 1 observations, Y = y1 y2 yN and X = x1 x2 xN , on Y and X, respectively. The sample canonical correlations 1 > r1 > r2 > · · · > rp > 0

(3)

−1 are the positive square roots of the eigenvalues of S−1 11 S12 S22 S21 With probability 1, the sample canonical correlations r1 r2 rp are distinct and the rank of S12 is p (see Seber, 1984, Sec. 5.7.2). In the interpretation of the relationships between two sets of variables, the number d of nonzero population canonical correlations may be called the dimensionality, referring to the subspace to which the really correlated canonical variates belong. Of interest is the estimation of this dimensionality.

Tests for Determining the Number of Nonzero Canonical Correlations

729

3. Tests of Dimensionality The canonical correlation analysis makes sense only if 12 = 0, that is, d > 0. Therefore, one should first test the null hypothesis 12 = 0, equivalent to the hypothesis H0 1 = 2 = · · · = p = 0

(4)

If H0 is rejected, then a next question arises, namely, which of the canonical correlations i , i = 1 2 p, are 0. Because the coefficients are ordered as shown in (2), the relevant hypotheses, following H0 , are Hd d+1 = d+2 = · · · = p = 0 d = 1 2 p − 1

(5)

Thus, the question of interest is “what is the value of d?” To answer this, it is advisable to proceed as follows (see, e.g., Anderson, 1984, Sec. 12.4.1; Seber, 1984, Sec. 5.7.2). First test the overall hypothesis that d = 0, the hypothesis H0 in (4). If H0 is not rejected, then accept that d = 0. Otherwise, test Hd for d = 1. In case of not rejecting it, accept that d = 1. Otherwise, continue in this way until Hd is not rejected for the first time. Then conclude that the last p − d canonical correlations are all 0. Several testing procedures have been proposed in the literature for implementing this advice. For testing the hypothesis Hd d = 0 1 p − 1, the following three statistics, or suitable functions of them, analogous to the statistics proposed for MANOVA (see e.g., Fujikoshi, 1977, p. 64) can be considered: 1. Lawley–Hotelling trace statistic Td2 =

p

ri2

2 i=d+1 1 − ri

(6)

2. Wilks statistic (the likelihood ratio statistic) d =

p

1 − ri2

(7)

i=d+1

3. Bartlett–Nanda–Pillai trace statistic Vd =

p

ri2

(8)

i=d+1

Under Hd , Bartlett’s (1938, 1947) 2 -approximation can be applied to Td2 , d , and Vd when transformed to SB Td2 = N − p − q − 2Td2 1 SB d = − N − 1 − p + q + 1 ln d 2

(9) (10)

730

Calin´ ski et al.

and SB Vd = N − 1Vd

(11)

respectively. Their distributions (under Hd ) are approximately 2 with p − d q − d degrees of freedom. In the case of the statistic d , Lawley (1959) gave an improved transformation, d 1 −2 SL d = − N − 1 − d − p + q + 1 + ri ln d 2 i=1

(12)

which (under Hd ) is distributed approximately as 2 with p − dq − d degrees of freedom. Further, Fujikoshi (1977, p. 66) has given the modified test statistics for the other two statistics considered above: d SF Td2 = N − p − q − 2 + ri−2 Td2

(13)

i=1

and d −2 ri Vd SF Vd = N − 1 − 2d +

(14)

i=1

which are also approximately distributed as 2 with p − dq − d degrees of freedom. Procedures most commonly used in practice are those based on the d criterion, modified either according to Bartlett or to Lawley. Relevant computing procedures, using mainly the former modification, are available in many statistical program packages, e.g., BMD, STATISTICA, and STATGRAPHICS. Quite surprisingly, procedures based on other multivariate test criteria have not received much attention, despite the fact that their limiting distributions are known (see Fujikoshi, 1977, Sec. 4). Calinski and Krzy´sko (2005) have suggested a closed testing procedure, in the ´ sense of Marcus et al. (1976), based on the Lawley–Hotelling trace statistic Td2 . It may be noted that the hypotheses Hd for d = 0 1 p − 1 have the property that for any d > 0 the hypothesis Hd is implied by Hd−1 , and thus all of them are implied by H0 . Moreover, since for any d1 < d2 the equality Hd1 ∩ Hd2 = Hd1 holds, the set of hypotheses = H0 H1 Hp−1

(15)

forms a family closed under intersection. Therefore, any Hd ∈ should be tested by means of an -level test if and only if all hypotheses preceding Hd in have been tested and rejected. To assure that the probability of making no Type I error is at least 1 − , it is sufficient that the test employed for testing any Hd rejects a true Hd with probability not exceeding (see Marcus et al., 1976, p. 656).


731

When H0 is true, the null distribution of the Lawley–Hotelling trace statistic 2 T02 is indexed by the triple p mH mE and is denoted by Tpm , where mH = q, H mE mE = N − q − 1 and p ≤ mH mE . Although the exact distribution of Td2 under Hd is not available, the null distribution of the Lawley–Hotelling trace statistic related to the triple p − d mH − d mE can be shown to be an optimal upper bound for the distribution under Hd of the statistic Td2 , so that 2 Hd ≤ PTd2 > T p−dm H −dmE

This has been established in Calinski and Lejeune (1998, Sec. 2) by using some ´ results of Schott (1984). Thus, it is justified to propose to test the family of hypotheses (15) sequentially with the statistic Td2 , for d = 0 1 2 , until the inequality 2 Td2 ≤ T p−dm H −dmE

(16)

is reached. If Td2 , for some d, is the first for which (16) holds, it can be concluded that the dimensionality, i.e., the number of nonzero population canonical correlations, is d. The probability of making no Type I error, i.e., of not overestimating the true number d by stopping too late, is then at least 1 − . Unlike in Bartlett’s approximation (9) and in Fujikoshi’s approximation (13), in the method considered here, the distribution of Td2 is approximated by its optimal 2 upper bound, which is of the form of an exact Lawley–Hotelling Tp−dm H −dmE distribution. This allows the critical value for testing the hypothesis Hd , from the family (15), to be taken as given in (16). If Hd is true, the probability of exceeding the critical value by the statistic Td2 is then at most equal to the designated , assuring the control of the Type I familywise error rate at the level , as explained above.

4. The Simulation Study Consider the p + q × p + q covariance matrix of the combined random vector Y X of the form (1). The main problem that arises in designing a simulation study of the canonical correlation analysis, is the infinite number of matrices that may be selected. However, the problem can be alleviated considerably by confining the selection to ’s in the canonical form, i.e., such that =

Ip

Iq

where is a p × q matrix of the form  1 0 0 0 0  0 2 0 0 0  =  ...................................  0 0 p 0 0 

732

Calin´ ski et al.

This simplifies the selection problem, because results obtained from the canonical matrices are not at any case less general than results obtained from any other matrices. There are three reasons for this. • In the first place, canonical correlations are invariant under nonsingular linear transformation, and so are their test statistics (see, e.g., Kshirsagar, 1972, p. 278). • Secondly, for any matrix , one can always select two matrices, say P and Q, such that Y = PY and X = QX yield a covariance matrix in the canonical form. • Finally, and most importantly, the exact joint distribution of r12 rp2 depends only on the parameters 21 2p (see, e.g., Muirhead, 1982, p. 557). The simulations were conducted, at the significance level = 001, = 005, and = 01, applying the same method as that used by Mendoza et al. (1978). The results of simulations presented here are confined to samples of the size N = 25 50 75, generated from normal distributions with four cases of the numbers p and q: p = 4 q = 5, p = 4 q = 8, p = 6 q = 6, and p = 6 q = 8. From each population, specified by the assumed values of the canonical correlations (see Table 1), 10,000 samples were simulated for each N (10 times more than in the referred paper). To each sample seven testing procedures were applied: • three procedures based on the Lawley–Hotelling trace statistic Td2 : the proposed closed testing procedure denoted by Td2 , the Bartlett 2 -approximation procedure denoted by SB Td2 , and the Fujikoshi 2 -approximation procedure denoted by SF Td2 ; • two procedures based on the likelihood ratio statistic d : the Bartlett 2 -approximation procedure denoted by SB d and the Lawley 2 -approximation procedure denoted by SL d ; • two procedures based on the Bartlett–Nanda–Pillai trace statistic Vd : the Bartlett 2 -approximation procedure denoted by SB Vd and the Fujikoshi 2 -approximation procedure denoted by SF Vd . The seven tests were compared with regard to (a) their ability to correctly identify the number of nonzero canonical correlations in the population, (b) the frequency with which they underestimate the number of nonzero canonical correlations in the population, and (c) the frequency with which they overestimate the number of nonzero canonical correlations in the population. In addition,

Table 1 The values of canonical correlations in the matrices used in the simulation study p=4 0 1 2 3 4 5 6

1 1 1 1 1 1 1

= 2 = 3 = 4 = 0 = 08, 2 = 3 = 4 = 0 = 07, 2 = 3 = 4 = 0 = 08, 2 = 07, 3 = 4 = 0 = 07, 2 = 05, 3 = 4 = 0 = 08, 2 = 07, 3 = 05, 4 = 0 = 07, 2 = 05, 3 = 04, 4 = 0

p=6 1 1 1 1 1 1 1

= 2 = 3 = 4 = 5 = 6 = 0 = 08, 2 = 3 = 4 = 5 = 6 = 0 = 07, 2 = 3 = 4 = 5 = 6 = 0 = 08, 2 = 07, 3 = 4 = 5 = 6 = 0 = 07, 2 = 05, 3 = 4 = 5 = 6 = 0 = 08, 2 = 07, 3 = 05, 4 = 5 = 6 = 0 = 07, 2 = 05, 3 = 04, 4 = 5 = 6 = 0


733

for comparison reasons, the Akaike Information Criterion (AIC) has been included in the study, in the form adopted for canonical correlation analysis by Fujikoshi and Veitch (1979).

5. Results of a Simulation Study Results of the conducted simulation study are given in details in Tables A.1–A.12 (see the Appendix) and in a summarized form in Tables 2 and 3. The following conclusions can be drawn from the obtained results. In the case of generated samples of size N = 25, the mean percentages of correct choices of the actual dimension d are comparatively low for all the seven investigated testing procedures. Under the condition that the nonzero population canonical correlations are all above 0.5, the highest obtained percentage of correct choices is 47.6 at the significance level = 001 58.6 at = 005, and 63.3 at = 010 The best performance from this point of view is shown by the testing procedures SB Td2 and SF Td2 Very poor performance is shown by procedures based on the statistic Vd i.e., the procedures SB Vd and SF Vd If at least one of the nonzero population canonical correlations is equal to 0.5 or less, the obtained percentages of correct choices are small for all the procedures, the highest being for SF Td2 4.6 at = 0 01 11.4 at = 005, and 16.4 at = 010 Under this condition, all the compared procedures expose very high underestimation of the actual d In the case of samples of size N = 50 the mean percentage of correct choices of the actual d is quite high for all the compared testing procedures, provided that the nonzero population canonical correlations are all above 0.5. The best performance is shown by the three procedures based on the statistic Td2 i.e., the procedures Td2 SB Td2 , and SF Td2 with small differences between them. With regard to possible overestimation of d it should be noted that the procedure SF Td2 may overestimate the actual dimension more frequently than the nominal significance level allows for it. From this point of view, the best performance among the first three procedures is shown by the closed testing procedure Td2 Again, if at least one of the nonzero population canonical correlations is equal to 0.5 or less, then the frequency of correct choices is not satisfactory for any of the compared procedures. In this case, the best performance is shown by the procedure SF Td2 which gives the mean percentage of correct choices equal to 22.5 at = 001 37.5 at = 005, and 44.5 at = 010 In the sample size case of N = 75 the mean percentage of correct choices of the actual d is for all the compared testing procedures very high, provided that none of the nonzero population canonical correlations is equal or smaller than 0.5. Under this condition, satisfactory performance with about or above 90% of correct decisions is shown by each of the procedures, the differences between them being negligible. However, it is to be noted that the procedure SF Td2 leads to a slightly higher frequency of the overestimation of d than it is determined by the nominal significance level Similarly as before, the mean percentage of correct choices of d is for any procedure much lower, if at least one of the nonzero population canonical correlations is equal to 0.5 or smaller. Under the latter condition, the best performance is shown by the procedure SF Td2 for which the mean percentage of correct choices is 49.8 at = 001 65.6 at = 005, and 70.1 at = 010 When considering the means over all the generated samples, it can be noted that under the condition that all nonzero population canonical correlations are

2j

1

i

2j

1

i

2b

3c

4d

5e

6f

7g

1a

under correct over under correct over

14.1 9.1 8.3 85.2 89.7 89.9 0.7 1.1 1.8 77.8 74.5 69.6 14.6 17.9 22.5 0.1 0.2 0.5

19.6 79.7 0.7 78.7 13.6 0.1

4d

N = 25

3c

5e

6f

7g

1a

2b

3c

N = 50

39.2 55.0 5.8 86.8 12.0 1.3

4d

= 01

38.6 54.7 6.7 84.5 13.4 2.1

5e

7g

8h

66.6 68.7 5.5 28.4 27.9 58.2 5.0 3.4 36.3 86.8 90.3 52.6 11.0 8.2 37.6 2.2 1.6 9.8

6f

20.2 2.6 2.1 1.8 3.6 3.5 9.5 9.9 0.3 76.9 89.3 89.6 87.0 88.0 87.2 82.5 82.8 76.0 2.9 8.1 8.3 11.2 8.4 9.3 8.0 7.3 23.7 65.7 48.6 49.1 42.2 49.3 46.1 51.0 52.0 28.5 25.2 40.5 40.6 44.5 39.6 41.2 37.3 36.0 54.1 1.6 3.5 2.8 5.8 3.6 5.2 4.3 4.6 9.9

39.5 35.9 53.4 52.8 83.9 85.4 40.4 32.0 28.3 57.7 58.6 44.1 44.2 14.6 13.7 55.0 63.3 62.6 2.8 5.5 2.5 3.0 1.5 0.9 4.7 4.8 9.1 93.8 87.7 93.4 92.0 94.8 96.5 88.6 89.7 81.1 6.0 11.4 6.3 7.3 4.7 3.2 10.4 9.8 16.4 0.2 1.0 0.4 0.6 0.6 0.3 1.0 0.5 2.4

2b

= 005

19.2 50.3 51.2 5.1 3.7 3.3 7.2 7.0 19.6 80.0 49.4 48.5 91.0 91.8 90.5 88.9 88.5 77.2 0.8 0.4 0.3 3.8 4.5 6.2 4.0 4.5 3.2 77.1 83.2 83.8 60.2 58.9 52.3 61.1 58.2 64.7 15.2 9.2 8.5 31.0 32.4 37.5 30.1 32.1 26.3 0.3 0.1 0.1 1.3 1.2 2.7 1.4 2.2 1.5

under 76.3 53.5 50.5 77.1 76.8 98.0 98.3 53.9 correct 23.5 45.7 47.6 22.6 22.8 1.9 1.6 44.2 over 0.2 0.9 2.0 0.3 0.4 0.1 0.0 1.9 under 99.1 98.0 95.3 98.8 98.5 99.6 99.8 94.6 correct 0.8 2.0 4.6 1.2 1.5 0.4 0.2 5.1 over 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.3

1a

= 001

Testing procedures applied at the significance level

Table 2 Mean percentages of correct and incorrect dimensionality choices obtained from all simulations

734 Calin´ ski et al.

2j

1i

2j

1i

under correct over under correct over

30.6 68.9 0.6 78.2 19.2 0.2

21.1 77.8 1.1 75.6 21.7 0.2

19.8 78.4 1.8 71.4 25.6 0.5

33.0 66.4 0.6 79.1 18.2 0.2

32.7 66.5 0.7 77.6 19.6 0.3

52.4 47.2 0.3 83.2 14.1 0.1

52.9 46.8 0.3 83.7 13.7 0.2

19.8 76.9 3.3 63.5 32.6 1.4

14.5 81.5 4.0 62.6 33.7 1.2

20.3 76.0 3.6 63.9 32.2 1.4

Overall 13.1 81.0 5.9 56.8 38.1 2.6

20.1 75.8 4.1 61.6 33.8 2.1

35.0 62.1 2.9 66.6 29.3 1.6

35.8 61.7 2.5 67.6 28.2 1.7

14.4 78.4 7.3 54.2 39.9 3.4

11.4 81.2 7.4 54.9 39.8 2.8

i

Td2 , b SB Td2 , c SF Td2 , d SB d , e SL d , f SB Vd , g SF Vd , h AIC. Populations in which the values of the canonical correlations are all above 0.5. j Populations in which at least one of the nonzero canonical correlations has its true value equal to 0.5 or less.

a

N = 75

under 1.2 0.8 0.7 2.2 2.2 9.0 9.2 0.3 0.2 0.2 0.5 0.4 1.7 1.7 0.1 0.1 correct 98.0 98.1 97.7 96.9 96.9 90.4 90.2 95.4 95.0 93.9 95.2 94.8 94.4 94.6 90.9 90.8 over 0.8 1.1 1.6 0.8 1.0 0.6 0.5 4.3 4.8 6.0 4.4 4.8 3.9 3.7 9.0 9.1 under 57.6 54.4 49.4 59.8 57.4 66.9 67.3 35.8 35.1 30.3 37.2 34.6 40.5 40.6 25.5 25.9 correct 42.1 45.2 49.8 39.8 42.1 32.8 32.3 61.8 62.6 65.6 60.3 61.8 56.8 56.4 68.7 69.0 over 0.3 0.4 0.8 0.3 0.6 0.3 0.3 2.5 2.3 4.1 2.5 3.5 2.7 3.0 5.7 5.1 10.1 79.5 10.4 48.3 43.7 5.5

14.3 78.0 7.7 54.2 39.8 3.5

14.1 77.3 8.6 51.6 40.9 5.0

25.6 67.2 7.3 55.2 38.0 4.3

26.4 67.2 6.4 56.7 36.4 4.4

1.9 71.4 26.7 31.5 55.5 10.5

0.1 0.2 0.2 0.6 0.6 0.0 88.9 90.8 90.0 90.6 91.0 79.9 11.0 9.0 9.8 8.8 8.4 20.1 21.6 26.4 24.1 27.8 27.7 13.4 70.1 67.8 68.2 65.8 65.1 74.8 8.4 5.8 7.7 6.4 7.2 11.7

Tests for Determining the Number of Nonzero Canonical Correlations 735

736

Calin´ ski et al. Table 3 The ratios of the frequency results of the procedure SF Td2 to those of the procedure Td2 (the first), with regard to the correct choice of d, and also to the overestimation of d Populations in which the values of the canonical correlations are all above 0.5

Populations in which at least one of the nonzero canonical correlation has its true value equal to 0.5 or less

N

1a

2b

1a

2b

25

0.01 0.05 0.1 0.01 0.05 0.1 0.01 0.05 0.1 0.01 0.05 0.1

2.03 1.32 1.14 1.06 0.99 0.97 1.00 0.98 0.98 1.14 1.05 1.01

9.28 2.97 1.95 2.66 1.62 1.37 1.91 1.37 1.23 3.12 1.76 1.44

5.40 2.23 1.58 1.54 1.21 1.10 1.18 1.06 1.02 1.34 1.17 1.10

12.40 3.50 2.43 3.47 2.09 1.68 2.50 1.65 1.46 3.04 1.92 1.63

50

75

All

a b

Ratio of SF Td2 to Td2 with regard to the frequencies of the correct choice of d. Ratio of SF Td2 to Td2 with regard to the frequencies of the overestimation of d.

above 0.5, a comparatively successful performance is shown by the first three testing procedures, Td2 SB Td2 , and SF Td2 i.e., procedures based on the statistic Td2 with not much differences between them, particularly when testing at = 010 Under the other condition, when at least one of the nonzero population canonical correlations is equal to 0.5 or less, the frequencies of correct decisions are much smaller, with the best performance shown by the procedure SF Td2 . Results obtained for the added AIC are shown in the last column of Table 2. Although, if at least one of the nonzero population canonical correlations is equal to 0.5 or less, the performance of this criterion may be better than that of the investigated testing procedures, it has the disadvantage of always leading to much higher frequency of the overestimation of the actual d Thus, if it is desirable to keep this type of deviation under a reasonable control, that criterion cannot be recommended. To summarize the above observations, it may be concluded that for small samples the performance of any of the seven testing procedures is not satisfactory, particularly when small canonical correlations appear in the population. For medium-sized samples, the performance of the compared procedures is much better. The best performance in the sense of correct decisions on d can be expected from the first three procedures, those based on the Lawley–Hotelling statistic Td2 Among them, the most accurate in identifying the correct number of nonzero canonical


737

correlations when some of them are small, is the third procedure, SF Td2 based on the Fujikoshi approximation of the distribution of the statistic Td2 For large samples, all the compared testing procedures show satisfactory performance, with only small differences among them, rather in favor of the first three procedures, but with larger differences when small population canonical correlations are present. Then again, more successful is the procedure SF Td2 It has, however, the tendency to overestimate the actual d more frequently than it could be expected from the nominal significance level i.e., it does not always preserve the control of the Type I familywise error rate at the designated level From this point of view, the most rigorous is the proposed closed testing procedure based on the statistic Td2 i.e., the first procedure. It performs particularly well when applied at the significance level not smaller than = 005 and when the analyzed sample is not very small. In practice, it may be advisable to apply both of these testing procedures, Td2 and SF Td2 simultaneously (see Table 3). Anyway, the first three procedures, Td2 SB Td2 , and SF Td2 in most cases perform better than the other, though for large samples their dominance is not so evident. Finally, it should be noted that in view of the simulation study reported here, there is no reason for favouring in the commonly known statistical software packages, such as, e.g., BMD, STATISTICA, or STATGRAPHICS, the procedures based on the Wilks statistic d only.

References Anderson, T. W. (1984). An Introduction to Multivariate Statistical Analysis. 2nd ed. New York: Wiley. Bartlett, M. S. (1938). Further aspects of the theory of multiple regression. Proc. Camb. Phil. Soc. 34:33–40. Bartlett, M. S. (1947). Multivariate analysis (with discussion). J. Roy. Statist. Soc. Suppl. 9(B):176–197. Calinski, T., Lejeune, M. (1998). Dimensionality in MANOVA tested by a closed testing ´ procedure. J. Multivariate Anal. 65:181–194. Calinski, T., Krzy´sko, M. (2005). A closed testing procedure for canonical correlations. ´ Commun. Statist.-Theor. Meth. 34:1105–1116. Fujikoshi, Y. (1977). Asymptotic expansion for the distributions of some multivariate tests. In: Krishnaiah, P. R., ed. Multivariate Analysis. Vol. IV. Amsterdam: North-Holland, pp. 55–71. Fujikoshi, Y., Veitch, L. G. (1979). Estimation of dimensionality in canonical correlation analysis. Biometrika 66:345–351. Hotelling, H. (1936). Relations between two sets of variates. Biometrika 28:321–377. Kshirsagar, A. M. (1972). Multivariate Analysis. New York: Marcel Dekker. Lawley, D. N. (1959). Tests of significance in canonical analysis. Biometrika 46:59–66. Marcus, R., Peritz, E., Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63:655–660. Mendoza, J. L., Markos, V. H., Gonter, R. (1978). A new perspective on sequential testing procedures in canonical analysis: A Monte Carlo evaluation. Multivariate Behav. Res. 13:371–382. Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. New York: Wiley. Schott, J. R. (1984). Optimal bounds for the distributions of some test criteria for tests of dimensionality. Biometrika 71:561–567. Seber, G. A. F. (1984). Multivariate Observations. New York: Wiley.

7g

1a

2b

3c

4d

5e

6f

7g

8h

a

Td2 , b SB Td2 , c SF Td2 , d SB d , e SL d , f SB Vd , g SF Vd , h AIC.

under 9978 9966 9853 9971 9943 9983 9989 9689 9746 9252 9659 9463 9600 9675 9153 9377 8654 9115 8801 8896 9027 7320 correct 22 34 144 29 57 17 11 293 246 674 323 473 354 281 741 579 1129 770 968 893 749 2273 over 0 0 3 0 0 0 0 18 8 74 18 64 46 44 106 44 217 115 231 211 224 407

6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9906 9578 9578 9880 9880 9994 9994 9504 9017 9017 9464 9464 9812 9812 9010 8530 8530 8920 8920 9446 9446 6151 over 94 422 422 120 120 6 6 496 983 983 536 536 188 188 990 1470 1470 1080 1080 554 554 3849

1a

= 001

0

Estim. Cond. of d


Appendix Table A.1 Numbers of dimensionality choices among 10,000 simulations for the case of N = 25, p = 4, q = 5


7g

8h

For explanations see Table A.1.

9987 9324 9357 8312 9184 8969 9308 9622 8450 8821 7446 8250 7931 8192 8784 5082 13 627 629 1547 763 948 631 359 1392 1113 2187 1575 1763 1495 1044 4346 0 49 14 141 53 83 61 19 158 66 367 175 306 313 172 572

9986 9375 9025 8420 9179 9060 9595 9740 8667 8479 7675 8336 8158 8824 9147 4395 14 598 949 1455 781 879 365 236 1232 1457 2056 1524 1639 995 748 4523 0 27 26 125 40 61 40 24 101 64 269 140 203 181 105 1082

9940 6946 6028 4930 6740 6561 8537 8999 5317 5003 3885 5184 4990 6691 7396 1200 60 2934 3856 4657 3101 3223 1322 932 4313 4743 5355 4371 4432 2803 2310 6623 0 120 116 413 159 216 141 69 370 254 760 445 578 506 294 2177

9953 6196 4210 4210 6042 6042 9126 9126 4714 3351 3351 4496 4496 7682 7682 753 47 3680 5545 5305 3763 3729 798 825 4926 6204 5811 4989 4923 1958 2065 5974 0 124 245 485 195 229 76 49 360 445 838 515 581 360 253 3273

9888 3259 1709 1709 3513 3513 8461 8461 2050 1230 1230 2183 2183 6416 6416 131 109 6489 7839 7544 6143 6094 1395 1454 7320 8052 7621 7057 6975 3070 3246 6031 3 252 452 747 344 393 144 85 630 718 1149 760 842 514 338 3838


6f

6

5e

9927 9824 9400 9869 9823 9967 71 174 582 129 172 33 2 2 18 2 5 0

4d

under correct over

3c

5

2b

9906 9620 9293 9838 9814 9978 91 374 682 156 177 19 3 6 25 6 9 3

1a

under correct over

7g

4

6f

9107 7695 6746 8959 8878 9868 885 2282 3143 1023 1099 127 8 23 111 18 23 5

5e

under correct over

4d

3

3c

8427 5721 5721 8361 8361 9953 1559 4215 4129 1617 1614 44 14 64 150 22 25 3

2b

under correct over

1a

9999 9530 8818 8818 9380 9380 9903 9903 9041 8352 8352 8800 8800 9605 9605 4716 1 470 1182 1182 620 620 97 97 959 1648 1648 1200 1200 395 395 5284

7g

2

6f

5978 2843 2843 6336 6336 9888 3986 7003 6860 3613 3608 109 36 154 297 51 56 3

5e

under correct over

4d

1

3c

= 01

9907 9385 9385 9879 9879 9999 93 615 615 121 121 1

2b

= 005

correct over

1a

= 001

0

Estim. Cond. of d


Table A.2 Numbers of dimensionality choices among 10,000 simulations for the case of N = 25, p = 4, q = 8


7g

8h


9986 9481 9537 8619 9352 9174 9378 9653 8770 9144 7849 8648 8351 8404 9011 4440 14 488 453 1272 614 761 543 315 1143 823 1876 1235 1417 1311 825 4321 0 31 10 109 34 65 79 32 87 33 275 117 232 285 164 1239

9990 9493 9164 8564 9274 9182 9630 9768 8864 8684 7857 8510 8340 8822 9160 3901 10 490 825 1336 686 760 320 209 1065 1271 1911 1370 1484 984 717 4503 0 17 11 100 40 58 50 23 71 45 232 120 176 194 123 1596

9932 7293 6415 5192 7186 6996 8652 9100 5678 5350 4199 5561 5360 7000 7677 966 65 2587 3482 4446 2657 2785 1185 831 3993 4432 5118 4067 4153 2492 2038 6005 3 120 103 362 157 219 163 69 329 218 683 372 487 508 285 3029

9925 6331 4319 4319 6144 6144 9082 9082 4877 3492 3492 4698 4698 7695 7695 589 75 3505 5378 5111 3601 3565 774 835 4691 5992 5618 4737 4660 1865 2000 5362 0 164 303 570 255 291 144 83 432 516 890 565 642 440 305 4049

9863 3502 1856 1856 3863 3863 8526 8526 2258 1335 1335 2445 2445 6712 6712 103 133 6240 7710 7366 5792 5754 1296 1371 7133 7953 7486 6773 6694 2713 2899 5330 4 258 434 778 345 383 178 103 609 712 1179 782 861 575 389 4567


6f

6

5e

9953 9890 9512 9901 9872 9944 46 109 466 96 123 54 1 1 22 3 5 2

4d

under correct over

3c

5

2b

9941 9705 9400 9889 9867 9980 58 294 583 110 130 20 1 1 17 1 3 0

1a

under correct over

7g

4

6f

9186 7942 7038 9094 9026 9856 804 2038 2851 890 954 137 10 20 111 16 20 7

5e

under correct over

4d

3

3c

8472 5783 5783 8385 8385 9925 1514 4127 4000 1582 1575 73 14 90 217 33 40 2

2b

under correct over

1a

9994 9484 8747 8747 9377 9377 9893 9893 9018 8288 8288 8811 8811 9618 9618 4068 6 516 1253 1253 623 623 107 107 982 1712 1712 1189 1189 382 382 5932

7g

2

6f

6172 3010 3010 6665 6665 9863 3798 6826 6695 3280 3270 130 30 164 295 55 65 7

5e

under correct over

4d

1

3c

= 01

9888 9333 9333 9865 9865 9994 112 667 667 135 135 6

2b

= 005

correct over

1a

= 001

0

Estim. Cond. of d




6f

7g

8h


9996 9790 9808 9103 9596 9478 9602 9848 9412 9622 8590 9092 8880 8838 9439 4212 4 202 191 820 388 478 334 140 541 370 1217 824 968 931 466 4271 0 8 1 77 16 44 64 12 47 8 193 84 152 231 95 1517

9996 9731 9435 8904 9437 9381 9769 9885 9317 9130 8379 8871 8747 9225 9523 3211 4 258 554 1017 529 570 192 98 634 843 1459 1040 1117 615 411 4656 0 11 11 79 34 49 39 17 49 27 162 89 136 160 66 2133

9977 8561 7625 6361 8001 7864 9262 9589 7291 6832 5365 6658 6507 7969 8646 933 23 1382 2326 3320 1889 1981 617 377 2511 3065 4058 3030 3095 1619 1191 5205 0 57 49 319 110 155 121 34 198 103 577 312 398 412 163 3862

9976 7388 5034 5034 6725 6725 9510 9510 6105 4266 4266 5348 5348 8471 8471 410 24 2504 4724 4460 3051 3015 400 448 3598 5350 4939 4116 4047 1184 1324 4290 0 108 242 506 224 260 90 42 297 384 795 536 605 345 205 5300

9968 5038 2593 2593 4786 4786 9256 9256 3547 2005 2005 3298 3298 7846 7846 102 32 4783 7011 6621 4857 4822 626 693 5980 7390 6822 6002 5935 1712 1887 3985 0 179 396 786 357 392 118 51 473 605 1173 700 767 442 267 5913

9999 10000 9984 9986 9835 9946 9918 9921 9975 9913 9961 9653 9812 9721 9658 9861 7027 1 0 16 14 156 54 78 68 25 84 39 308 174 243 283 118 2413 0 0 0 0 9 0 4 11 0 3 0 39 14 36 59 21 560

5e

under 9998 9997 9967 9996 9995 correct 2 3 33 4 5 over 0 0 0 0 0

4d

6

3c

9981 18 1

2b


1a

5

7g

9986 14 0

6f


5e

4

4d

9951 47 2

3c


2b

3

1a

9976 24 0

7g


6f

2

5e

9968 30 2

4d


3c

= 01

1

2b

= 005

correct 9892 9158 9158 9818 9818 10000 10000 9513 8550 8550 9222 9222 9926 9926 9055 8085 8085 8605 8605 9660 9660 2531 over 108 842 842 182 182 0 0 487 1450 1450 778 778 74 74 945 1915 1915 1395 1395 340 340 7469

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9892 9795 9795 9897 9897 9951 9951 9502 9318 9318 9504 9504 9672 9672 9025 8836 8836 9002 9002 9223 9223 7533 over 108 205 205 103 103 49 49 498 682 682 496 496 328 328 975 1164 1164 998 998 777 777 2467

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h

708 292 0

623 372 5


754 246 0

761 239 0

738 262 0

838 162 0

857 143 0

477 499 24

470 509 21

393 562 45

490 486 24

457 506 37

537 437 26

562 413 25

339 601 60

355 596 49

281 635 84

347 593 60

319 601 80

356 574 70

374 555 71

202 722 76


6f

6

5e

under correct over

4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9906 9727 9727 9895 9895 9972 9972 9478 9177 9177 9460 9460 9698 9698 8964 8653 8653 8958 8958 9267 9267 7255 over 94 273 273 105 105 28 28 522 823 823 540 540 302 302 1036 1347 1347 1042 1042 733 733 2745

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9898 9747 9747 9898 9898 9967 9967 9503 9217 9217 9489 9489 9722 9722 8998 8673 8673 8947 8947 9266 9266 6770 over 102 253 253 102 102 33 33 497 783 783 511 511 278 278 1002 1327 1327 1053 1053 734 734 3230

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9914 9715 9715 9887 9887 9978 9978 9496 9124 9124 9477 9477 9742 9742 8976 8593 8593 8924 8924 9331 9331 6216 over 86 285 285 113 113 22 22 504 876 876 523 523 258 258 1024 1407 1407 1076 1076 669 669 3784

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9897 9828 9828 9889 9889 9938 9938 9510 9398 9398 9508 9508 9616 9616 9012 8864 8864 8998 8998 9139 9139 7887 over 103 172 172 111 111 62 62 490 602 602 492 492 384 384 988 1136 1136 1002 1002 861 861 2113

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9897 9807 9807 9893 9893 9954 9954 9527 9348 9348 9506 9506 9667 9667 9040 8864 8864 9028 9028 9197 9197 7824 over 103 193 193 107 107 46 46 473 652 652 494 494 333 333 960 1136 1136 972 972 803 803 2176

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9901 9815 9815 9909 9909 9957 9957 9545 9359 9359 9530 9530 9673 9673 9071 8873 8873 9062 9062 9241 9241 7431 over 99 185 185 91 91 43 43 455 641 641 470 470 327 327 929 1127 1127 938 938 759 759 2569

1a

= 001

0

Estim. Cond. of d




7g

1a

2b

3c

4d

5e

6f

7g

8h



6f

6

5e


4d

5

3c


2b

4

1a


7g

3

6f


5e

2

4d


3c

= 01

1

2b

= 005

correct 9884 9761 9761 9879 9879 9953 9953 9466 9254 9254 9450 9450 9623 9623 8936 8724 8724 8922 8922 9150 9150 7043 over 116 239 239 121 121 47 47 534 746 746 550 550 377 377 1064 1276 1276 1078 1078 850 850 2957

1a

= 001

0

Estim. Cond. of d




A Comparison of Some Tests for Determining the Number of Nonzero ...

A Comparison of Some Tests for Determining the Number of Nonzero ...

Suggest Documents

Comparison of the results of some serological tests for bovine ...

Comparison of the results of some serological tests for bovine

Power Comparison of Some Tests for the Error Component Model ...

A Comparison of Methods for Determining Optical

Comparison of Five Rules for Determining the

A theoretical framework for determining the minimum number of ...

a comparison of methods for determining the molecular ... - Google Sites

5.Determining the number of cards.indd

Number of Vector Bites Determining the Infection

Determining Optimum Number of Kanbans for Workstations in a

Comparison of Methods for Determining Precompression ... - SciELO

Comparison of methods for determining unsaturated hydraulic ...

Comparison of methods for determining unsaturated hydraulic ...

Comparison of methods for determining lime ...

comparison of 13 equations for determining ... - BioOne

Some families of graphs with no nonzero real domination roots

Comparison of Some Tests of Fit for the Inverse Gaussian ... - Core

Determining the number of components in mixture models for ...

NbClust: An R Package for Determining the Relevant Number of ...

NbClust: An R Package for Determining the Relevant Number of ...

NbClust package for determining the number of ...

Determining the Number of Accusations for Decision Making in Key ...

Twenty-step algorithm for determining the asymptotic number of trees ...

Determining the number of breaks in a piecewise linear regression ...