Monte Carlo Exact Conditional Tests for Quasi ... - CiteSeerX

Monte Carlo Exact Conditional Tests for Quasi-independence using Gibbs Sampling Peter W. F. Smith, John W. McDonald

Department of Social Statistics, University of Southampton, Southampton, SO17 1BJ, U.K.

and Jonathan J. Forster Department of Mathematical Sciences, Loughborough University of Technology, Loughborough, Leicestershire, LE11 3TU, U.K.

Summary A Gibbs sampling approach to estimating the exact conditional p-value for quasi-independence is described. As an example, a test for quasi-independence for the o-diagonal cells of a 8  8 table is presented. The results are compared with an alternative simulate-and-reject procedure. Keywords: Estimated p-values, Gibbs sampler, Monte Carlo exact conditional tests, Quasi-independence, Simulate and reject

1 Introduction For a two-way contingency table, the hypothesis of quasi-independence (QI) usually arises either because some cells involve structural zeros or because interest is focused on part of the table, e.g., the o-diagonal cells. Consider Table 1, analysed by Cazes (1990), which presents the distribution of marriages by lineage of each spouse amongst the Dogon of Boni, Mali. As exogamy (marriage outside the lineage) is prevalent, few of the observations lie on the diagonal and the hypothesis that the lineages of the spouses are independence is rejected. The hypothesis of QI for the o-diagonal cells, i.e., the lineages of the spouses are independent given that they dier, is considered. However, the sparseness of the table in some parts causes concern about the validity of using asymptotic tests, and an exact test should be used. In order to perform an exact test of QI the null distribution of an appropriate test statistic must be calculated or simulated. For both independence and QI, enumerating the required distribution is often computationally infeasible, and a Monte Carlo exact test is preferred. In Section 2 we review Monte Carlo exact conditional tests for independence and QI. In Section 3 we describe how the Gibbs sampler can be used to perform a Monte Carlo exact conditional test for QI. Section 4 contains an example and Section 5 a discussion.

Table 1: Distribution of marriages in the village of Nemgene by lineage of each spouse Husband's lineage

Wife's lineage Iariwa Ger.

Segiwa

Suraba

Pussuwo 

Iariwa

Tengo2

Tengo

Other

Total

Iariwa Ger.

0

2

3

3

3

4

1

3

19

Segiwa

2

0

2

5

1

1

6

3

20

Suraba

0

2

0

2

3

6

7

6

26

Pussuwo 

1

4

4

0

3

5

9

7

33

Iariwa

2

2

3

4

1

1

13

16

42

Tengo2

4

0

4

8

4

1

16

14

51

Tengo

4

6

8

10

8

14

12

16

78

Other

2

6

7

4

17

6

21

17

80

Total

15

22

31

36

40

38

85

82

349

2 Monte Carlo Exact Conditional Tests 2.1 Independence The usual approach to exact tests of independence for two-way tables is the conditional approach, which is reviewed by Agresti (1992) and Kreiner (1987), among others authors. Let X = fXij : ij 2 I = (1; : : :; r)  (1; : : :; c)g be a r  c table with either Poisson, multinomial or product-multinomial cell counts. A saturated log-linear model for ij = E(Xij ) is log ij =  + i + j + ij :

(1)

The hypothesis of independence corresponds to the interaction parameters ij equalling zero, with the others considered as nuisance parameters. The vector of sucient statistics for the parameter vector (; i; j ; ij ) is (x++ ; xi+; x+j ; xij ), where + denotes summation over the corresponding subscript. The distribution of the sucient statistics for the parameters of interest, conditional on the sucient statistics for the nuisance parameters, does not depend on the nuisance parameters. Therefore, an exact test for the hypothesis of independence can be constructed using the conditional distribution of the cell counts given the marginal totals. A Monte Carlo procedure for estimating the p-value of exact conditional tests of independence has been described in detail by Agresti, Wackerly and Boyett (1979), Kreiner (1987) and Whittaker (1990). Brie y, one generates a random sample of tables according to the conditional distribution of the cell counts given the marginal totals. For each generated table an appropriate test statistic is calculated and the exact conditional p-value is estimated by the proportion of generated tables which are at least as discrepant from the null as the observed table. When the tables are generated independently the accuracy of this unbiased estimate may easily be evaluated. In order to estimate the p-value to within 0.01 with 99% con dence, one needs to generate at least 16,590 tables (Agresti et al., 1979).

2.2 Quasi-independence For the two-way table X , consider a proper subset I  of the index set I for the complete table. We call the cells in I  the `cells of interest' and the cells not in I  ` xed'. The hypothesis of QI over I  corresponds to ij = 0 for ij 2 I . Now ij for ij 62 I  are considered additional nuisance parameters with sucient statistics xij for ij 62 I  and maximum likelihood estimates

bij = xij . Therefore, an exact test for the null hypothesis of QI can be constructed using the conditional distribution of the cell counts given the marginal totals and the observed counts in the xed cells. A common example, considered in this paper, is the hypothesis of QI for the o-diagonal cells of a r  r square table, where the sucient statistics for the nuisance parameters are xi+ ; x+j and xii , for i; j = 1; : : :; r. Unfortunately, in general, the probabilities of the required exact conditional distribution depend on a normalisation constant which is very dicult to evaluate in practice. Smith and McDonald (1993, 1994) propose a simulate-and-reject procedure for generating from this conditional distribution. This procedure involves simulating tables from a multivariate hypergeometric and rejecting tables when the simulated counts for the xed cells do not match the observed counts for the xed cells. While the simulate-and-reject procedure works well for moderate-sized tables, the rejection rate increases dramatically with table size, and for large tables the computational burden is enormous. An alternative, less computationally demanding, approach is now described.

3 The Gibbs Sampler The Gibbs sampler is a Markov chain Monte Carlo method which yields, in the limit, a realisation from a multivariate distribution of interest by sampling iteratively from univariate conditional distributions (Gelfand and Smith, 1990; Smith and Roberts, 1993). A sample from the required multivariate distribution is obtained by taking every n-th iteration from one or more chains after convergence. Kolassa and Tanner (1994) illustrate how the Gibbs sampler can be used by frequentists, who prefer to eliminate nuisance parameters via conditioning rather than integration (see also Tanner, 1993). As an illustrative example, they use the Gibbs sampler to generate samples from an approximation to the multivariate hypergeometric distribution, which is the required conditional distribution for exact tests of independence in two-way contingency tables. This is of no practical use because ecient algorithms for generating samples directly from this distribution exist, e.g., Pate eld (1981). We now review their approach and extend it to generate realisations from the required exact conditional distribution for QI, where no such algorithm exists.

3.1 Independence In order to construct a Gibbs sampler to generate realisations from the exact conditional distribution under independence for a r  c table, Kolassa and Tanner (1994) consider a log-linear model (1) with corner-point identi ability constraints, r = c = 0 and ij = 0 if i = r or j = c. The sucient statistics for the interaction parameters are the cell counts in all but the last row and the last column, and the sucient statistics for the nuisance parameters (, i , j ) are the table total, the rst r ? 1 row totals and the rst c ? 1 column totals (or equivalently, all the row and column totals). Kolassa and Tanner (1994) show that the univariate distribution, under independence, for any cell count Xij in the rst r ? 1 rows and rst c ? 1 columns, conditional on all other such cell counts and the observed row and column totals, is a hypergeometric distribution. This is de ned by the row and column totals of the 2  2 table xij xic r1 xrj xrc r2 c1 c2 t

and is given by P (Xij = x) =

r1 ! r2 ! c 1 ! c 2 ! ; x! (r1 ? x)! (c1 ? x)! (t ? r1 ? c1 + x)! t!

(2)

where maxf0; r1 + c1 ? tg  x  minfr1; c1g. The Gibbs sampler uses these univariate hypergeometric conditionals to generate from the required multivariate distribution.

3.2 Quasi-independence We require a parameterisation for the saturated model within which the hypothesis of QI can be expressed. If the table can be rearranged by permuting the rows and/or columns such that all the cells not in I  lie in the rst r ? 1 rows and c ? 1 columns, then the corner-point parameterisation can still be used. The counts for the cells not in I  are now sucient statistics for the nuisance parameters and should be conditioned on. This involves xing each of these counts at its observed value, instead of sampling from the appropriate conditional distribution at each iteration. For our example, where interest is focused on the o-diagonal cells of a r  r table, the corner-point parameterisation is inappropriate, since interaction parameters corresponding to xed cells must not be constrained to zero. For square tables, we consider the saturated model (1) with identi ability constraints, r = c = 0 and ij = 0 if i = r or j = r (but not i = j = r) and r?2 r?1 = 0. We refer to this as an `adjusted (r ? 2; r ? 1) corner-point' parameterisation. The null hypothesis of QI can now be expressed as H0 : ij = 0

i = 1; :::; r ? 1; j = 1; :::; r ? 1; i 6= j ; (i; j ) 6= (r ? 2; r ? 1)

and the sucient statistics for these parameters of interest are the cell counts in the rst r ? 1 rows and r ? 1 columns excluding cells on the main diagonal and cell (r ? 2; r ? 1). The required joint exact conditional distribution is of dimension (r ? 1)(r ? 2) ? 1. For r = 3, there is a single interaction parameter of interest, 21. The conditional distribution of its sucient statistic, X21, is de ned by the row and column totals of the o-diagonal cells of the following table: | x12 x13 r1 x21 | x23 r2 x31 x32 | r3 c1

c2

c3

t

and given by (up to a constant of proportionality)

/ x! (r ? x)! (c ? x)! (t ? r ? c 1? x)! (r ? c + x)! (c ? r + x)! ; (3) 2 1 3 3 3 1 3 2 where maxf0; r2 ? c3; c1 ? r3g  x  minfr2; c1; r3 + c3 ? tg. As this distribution involves P (X21 = x)

six factorial terms, we refer to it as a hexadial distribution. For the general case of r > 3, the (r ? 1)(r ? 2) ? 1 univariate conditional distributions for the sucient statistics of interest required for our Gibbs sampler fall into two classes. 1. For counts in cells (i; r ? 1); i = 1; :::; r ? 3 or (r ? 2; j ); j = 1; :::; r ? 3 the univariate conditional distributions are hypergeometric, de ned by the row and column totals of the tables xi r?1 xic r1 xr?2 r?1 xr?2 c r2 c1 c2 t

xr?2 j xr?2 r?1 r1 xr j xr r?1 r2 c1 c2 t

or

respectively, and given by (2). 2. For all other cell counts which are sucient statistics for parameters of interest, i.e., those in the rst r ? 2 columns and the rst r ? 1 rows, excluding those in the (r ? 2)-th row and on the diagonal, the univariate conditional distributions are hexadial, de ned by the row and column totals of the o-diagonal cells of the 3  3 table | xr?2 r?1 xr?2 c r1 xij | xic r2 xrj xr r?1 | r3 c1

c2

c3

t

and given by (3). For a particular sucient statistic, this follows by considering the cell counts that are not xed after conditioning on the remaining sucient statistics.

Generation from univariate hypergeometric distributions of the form of (2) is straightforward. However, the majority of the conditional distributions required for generating tables under QI are hexadial. There is no closed form expression for the normalising constant, and therefore the expression on the right hand side of (3) needs to be evaluated for all possible values of x each time a variate is required from such a distribution. Typically the support of these distributions is moderate, and the number of calculations involved is not prohibitive. Here, a Gibbs sampler is used to perform a Monte Carlo exact test for a common example of QI, where I  is the set of o-diagonal cells of a square table. In general, care is required to construct a suitable parameterisation for the saturated model, within which the QI hypothesis of interest can be expressed. For a general log-linear model, including the special case of QI for any I , Forster, McDonald and Smith (1994) show how to derive the required form of the conditional distributions in order to carry out Gibbs sampling.

3.3 Estimating the Exact P-value We estimate the exact conditional p-value by ranking the observed value of a test statistic among a random sample of values generated from the exact conditional distribution of the test statistic, using a Gibbs sampler. As successive realisations are dependent, care must be taken when assessing the accuracy of the estimated p-value. Raftery and Lewis (1992) describe a method for determining the number of iterations of a Gibbs sampler required to estimate a quantile to some speci ed accuracy by assessing the Markov properties of the process. We used a simple modi cation of their method to obtain the approximate con dence intervals for our estimated p-value.

4 Example: Intermarriage among the Dogon of Boni, Mali Cazes (1990) examines factors in uencing endogamy in a Dogon population of the Boni district of Mali. One model considered by Cazes for the data in Table 1 above is QI. This model has a deviance of 53.49 and a Pearson X 2 of 48.08, both on 41 degrees of freedom. Table 2 presents the asymptotic p-values along with those for the simulate-and-reject and Gibbs sampling approaches. The simulate-and-reject p-values are based on 20,000 independent realisations and are estimated to within 0:0070 for the deviance and 0:0073 for X 2, with 99% con dence. The Gibbs sampling p-values are based on 50,000 dependent realisations and are estimated to within 0:0047 and 0:0068 respectively, with approximate 99% con dence. The discrepancy between the deviance and X 2 statistics and their associated asymptotic p-values supports our concern about the validity of using asymptotic tests here. Furthermore, the asymptotic and Monte Carlo estimated p-values dier, whereas those estimated using the simulate-and-reject and Gibbs sampling approaches closely agree.

Table 2: P-values Deviance Pearson X 2 Asymptotic 0.0914 0.2079 Simulate and reject 0.1779 0.2027 Gibbs sampler 0.1749 0.2018

5 Discussion We have described a method, using the Gibbs sampler, for estimating exact p-values when testing for QI. Previous approaches include complete enumeration and simulate and reject. In general, it is only feasible to calculate exact p-values using complete enumeration for small tables, and to estimate the exact p-value using a simulate-and-reject approach for moderatesized tables. In our experience, the simulate-and-reject approach is infeasible for tables larger than 8  8. However, the Gibbs sampling approach described here is feasible for very large tables. Also, the Gibbs sampling approach is very much faster than the simulate-and-reject approach for all but some special cases, e.g., triangular tables (McDonald and Smith, 1994). Caution is required when using the Gibbs sampler because of the dependence of the realisations. However, in our experience, when the simulate-and-reject procedure is feasible the p-value closely agrees with that obtained using the Gibbs sampler. This is reassuring, since simulate-and-reject produces independent realisations directly from the exact conditional distribution. An alternative model for square tables, with less strong structural implications than QI, is quasi-symmetry. The quasi-symmetry model, instead of constraining all the o-diagonal interaction parameters to zero, constrains all symmetrically opposite interaction parameters to be equal, i.e., ij = ji for all ij . The approach outlined is this paper can be adapted for exact conditional tests of quasi-symmetry, as a similar conditioning argument yields hexadial distributions as the univariate conditionals required for a Gibbs sampler (Forster, McDonald and Smith, 1994). This is not too surprising since QI for the o-diagonal cells and quasisymmetry are equivalent for a 3  3 table.

Acknowledgement The work of the rst two authors was supported by the Economic and Social Research Council award H519255005 as part of the Analysis of Large and Complex Datasets Programme.

References Agresti, A. (1992) A survey of exact inference for contingency tables (with discussion). Statist. Sci., 7, 131{177. Agresti, A., Wackerly, D. and Boyett, J. M. (1979) Exact conditional tests for crossclassi cations: approximation of attained signi cance levels. Psychometrika, 44, 75{83. Cazes, M. H. (1990) Endogamy among the Dogon of Boni, Mali. J. Biosoc. Sci., 22, 85{99. Forster, J. J., McDonald, J. W. and Smith, P. W. F. (1994) Monte Carlo exact conditional tests for log-linear and logistic models. Working Paper, University of Southampton. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling based approaches to calculating marginal densities. J. Am. Statist. Soc., 85, 398{409. Kolassa, J. E. and Tanner, M. A. (1994) Approximate conditional inference in exponential families via the Gibbs sampler. J. Am. Statist. Soc., 89, 697-702. Kreiner, S. (1987) Analysis of multi-dimensional contingency tables by exact conditional tests: techniques and strategies. Scand. J. Statist., 14, 97{112. McDonald, J. W. and Smith, P. W. F. (1994) Exact conditional tests of quasi-independence for triangular contingency tables: estimating attained signi cance levels. Appl. Statist. (to appear). Pate eld, W. M. (1981) Algorithm AS159: An ecient method of generating random R  C tables with given row and column totals. Appl. Statist., 30, 91{97. Raftery, A. E. and Lewis, S. (1992) How many iterations in the Gibbs sampler? In Bayesian Statistics 4 (eds J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith) pp. 777{784. Oxford: Oxford University Press. Smith, A. F. M. and Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods (with discussion). J. R. Statist. Soc. B, 55, 3{23. Smith, P. W. F. and McDonald, J. W. (1993) Exact conditional tests for incomplete contingency tables: estimating attained signi cance levels. Working Paper, University of Southampton. Smith, P. W. F. and McDonald, J. W. (1994) Simulate and reject Monte Carlo exact conditional tests for quasi-independence. Proceedings of COMPSTAT 1994 (to appear). Tanner, M. A. (1993) Tools for Statistical Inference, 2nd edn. New York: Springer-Verlag. Whittaker, J. (1990) Graphical Models in Applied Multivariate Statistics. Chichester: Wiley.