Monte Carlo Estimation of Bayesian Credible and ... - Semantic Scholar

Monte Carlo Estimation of Bayesian Credible and HPD Intervals Ming-Hui Chen To appear in

and

Qi-Man Shaoy

Journal of Computational and Graphical Statistics, 7, 1998

This paper considers how to estimate Bayesian credible and highest probability density (HPD) intervals for parameters of interest and provides a simple Monte Carlo approach to approximate these Bayesian intervals when a sample of the relevant parameters can be generated from their respective marginal posterior distribution using a Markov chain Monte Carlo (MCMC) sampling algorithm. We also develop a Monte Carlo method to compute HPD intervals for the parameters of interest from the desired posterior distribution using a sample from an importance sampling distribution. We apply our methodology to a Bayesian hierarchical model that has a posterior density containing analytically intractable integrals that depend on the (hyper) parameters. We further show that our methods are useful not only for calculating the HPD intervals for the parameters of interest but also for computing the HPD intervals for functions of the parameters. Necessary theory is developed and illustrative examples including a simulation study are given.

Key Words: Bayesian computation; Markov chain Monte Carlo; Monte Carlo methods; Posterior distribution; Simulation.

1. INTRODUCTION One purpose of Bayesian posterior inference is to summarize posterior marginal densities. Graphical presentation of the entire posterior distribution is always desirable if this can be conveniently accomplished. However, summary statistics, which outline important features of the posterior distribution, are sometimes adequate. In particular, when one deals with a high dimensional posterior problem, graphical display of the entire posterior distribution is nearly infeasible, and therefore appropriate summary of the posterior distribution is most desirable.  Assistant Professor, Department of Mathematical Sciences, Worcester Polytechnic Institute, 100 Institute Road,

Worcester, MA 01609-2280. Email: [email protected]

yAssistant Professor, Department of Mathematics, University of Oregon, Eugene, OR 97403-1222. Email:

[email protected]

1

Nowadays, it almost becomes a routine practice that one summarizes the marginal posterior distributions by tabulating 100(1 ? )% posterior credible intervals for the parameters of interest in Bayesian inference. The primary reason for this is that credible intervals are easy to obtain. One can obtain such credible intervals analytically or using a Markov chain Monte Carlo (MCMC) method. However, when a marginal distribution is not symmetric, a 100(1 ? )% highest probability density (HPD) interval is more desirable. As discussed in Box and Tiao (1992), a HPD interval has two main properties: (a) the density for every point inside the interval is greater than that for every point outside the interval; (b) for a given probability content (say, 1 ? ) the interval is of the shortest length. Unlike Bayesian credible intervals, it is very computationally intensive to compute HPD intervals unless one deals with a simple Bayesian model such as a standard normal model in which one makes inference about the normal mean. Tanner (1996) provided a Monte Carlo algorithm to calculate the content and boundary of the HPD region. However, Tanner's algorithm requires evaluating the marginal posterior densities analytically or numerically. The implementation of his algorithm is also quite complicated and computationally intensive. To overcome such diculty, we propose a simple Monte Carlo method to estimate HPD intervals in this paper. Our approach requires only a MCMC sample generated from the marginal posterior distribution of the parameter of interest. Importance sampling is often used to estimate Bayesian posterior integrations of interest (e.g., see Geweke 1989). However, the literature on estimating Bayesian credible intervals, in particular HPD intervals, of parameters of interest via importance sampling is still sparse. Gonet and Wallach (1996) developed an ecient importance sampling method for estimating the quantiles of a test statistic, a function of a random sample of observations and showed that their quantile estimator has substantially lower mean squared error than the conventional estimator. Johns (1988) used importance sampling to obtain bootstrap con dence intervals. Their methods can certainly be applied to obtain Bayesian credible intervals. However, the problems considered in this paper are somewhat dierent from theirs in the sense that (i) we are mainly interested in calculating Bayesian HPD intervals; (ii) we use importance sampling because direct samples from the desired marginal posterior distributions are not available or very expensive to obtain. Further, when we deal with the problems arising from a Bayesian hierarchical model which has a posterior density containing analytically intractable integrals that depend on the (hyper) parameters under consideration, it is natural to use an importance sampling approach to evaluate posterior properties because directly sampling from the desired posterior distribution is nearly impossible (see Gelfand, Smith and Lee 1992 and Chen and Shao 1998 for detailed discussions). Chen and Shao (1998) developed an importance sampling approach to evaluate posterior properties using a technique of Chen and Shao (1997) for estimating normalizing constants. However, the approach developed there is useful only for estimating certain moment-types of posterior properties such as posterior means and posterior variances. For such a complicated and challenging Bayesian hierarchical model problem, we develop 2

a Monte Carlo method to estimate posterior credible and HPD intervals using a sample from an importance sampling distribution. The outline of this paper is organized as follows. In Section 2, we develop exact and simple Monte Carlo approaches for calculating Bayesian credible and HPD intervals when the closed forms of marginal posterior densities are known or a MCMC sample can be generated from a desired posterior distribution. In Section 3, we propose a Monte Carlo method to compute posterior credible and HPD intervals using a sample from an importance sampling distribution. In Section 4, we apply our Monte Carlo method to a Bayesian hierarchical model. A small scale simulation study is conducted for assessing empirical performance of the proposed Monte Carlo method in Section 5 and Section 6 uses the Meal, Ready-to-Eat (MRE) example of Chen, Nandram, and Ross (1996) to illustrate the proposed methodology. Finally, short remarks are given in Section 7.

2. EXACT OR SIMPLE MONTE CARLO APPROACHES Consider a Bayesian posterior density having the form (; 'jD) = c(1D) L(; '; D)(; '); (2.1) where D denotes data, the parameter  is one-dimensional, and ' may be a multi-dimensional vector of parameters other than  in the model. In (2.1), L(; '; D) is a likelihood function given data, (; ') is a prior, and c(D) is a normalizing constant. Our objective is to obtain Bayesian credible and HPD intervals for . Let  (jD) and (jD) be the marginal posterior density function and the marginal posterior cumulative distribution function (cdf) of , respectively. For ease of exposition, we rst assume that (jD) is unimodal. However, possible extensions to multimodal cases will be discussed in Section 7. In this section, we also assume that  can be generated from  (jD) using a direct random generation scheme (see, for example, Devroye 1986) or a Markov chain Monte Carlo sampling algorithm (see Gelfand and Smith 1990 for the Gibbs sampler and Tierney 1994 for the MetropolisHastings algorithm). We will obtain exact or approximate credible and HPD intervals for . To obtain exact credible and HPD intervals, we further assume that  (jD) and (jD) are analytically available, i.e., the closed forms of  (jD) and (jD) are known. Note that in the context of MCMC simulation,  (jD) is required to know only up to a normalizing constant. Given that the closed forms of  (jD) and (jD) are known, a Bayesian credible interval for  is readily available. For example, if we are interested in a 100(1 ? )% credible interval, we calculate (=2) and (1?=2) such that      (=2)jD = =2 and  (1?=2) jD = 1 ? =2:

3





Then, a 100(1 ? )% credible interval for  is (=2); (1?=2) . When  (jD) is not symmetric, a HPD interval is more plausible. A 100(1 ? )% HPD interval for  is simply given by

R ( ) = f : (jD)   g;

(2.2)

where  is the largest constant such that P ( 2 R ( ))  1 ? . See Berger (1985), Box and Tiao (1992), or Hyndman (1996) for the technical detail. Note that R ( ) can be viewed as a 100(1 ? )% HPD region for  when  (jD) is multimodal. It is often dicult to analytically calculate . To obtain an approximation of  , a Monte Carlo approach has been developed, see Box and Tiao (1992), Wei and Tanner (1990), Tanner (1996), or Hyndman (1996). Let  =  (jD), which is obtained by transforming  by its own density function. Then,  is the th quantile of  so that P (   ) = 1 ? . Assume that a MCMC sample fi ; i = 1; 2; : : :; ng is available from the marginal posterior distribution  (jD). (Note that if f(i ; 'i); i = 1; 2; : : :; ng is a MCMC sample from the joint posterior distribution  (; 'jD), then fi ; i = 1; 2; : : :; ng is a MCMC sample from  (jD) (see Gelfand and Smith 1990).) Now, let i =  (i jD) for i = 1; 2; : : :; n. We choose ^ = (j) , where (j) is the j th smallest of fig, j = [n], and [n] denotes the integer part of n. Then, ^ !  as n ! 1, and so R(^ ) ! R() as n ! 1. (See Hyndman 1996 for the detail.) For the cases where the closed forms of  (jD) and (jD) are known, the exact 100(1 ? )% HPD interval for  is also available. This result is given in the following theorem. Theorem 2.1 Assume that (jD) is continuous and unimodal. Then, a 100(1?)% HPD interval is (L ; U ) where L and U are the solution of the following optimization problem min (j (U jD) ?  (L jD)j + j(U jD) ? (L jD) ? (1 ? )j) :

L < P i

b  jD) = ( > :

if  < (1)  j =1 wj if (i)   < (i+1)  1 if   (n) :

0

(3.5)

b  jD) is in fact an empirical cdf of . When the  's are directly generated from Note that ( i b  jD) becomes a the desired marginal posterior distribution  (jD), then wi = 1=n and thus ( b  jD) given in (3.5) is a generalization of the usual empirical cdf. usual empirical cdf. Therefore, ( Under certain regularity conditions such as ergodicity, we can show that the central limit theorem b  jD). Using (3.5), () can be estimated by still holds for (

^() =

(

(1) (i)

if P= 0; Pi if ij?1 =1 wj <   j =1 wj :

(3.6)

To obtain a 100(1 ? )% HPD interval for , we let

? j+[(1?)n]




j Rj (n) = ^( n ); ^

7

n

;

(3.7)

for j = 1; 2; : : :; n ? [(1 ? )n]. Then, similar to Theorem 2.2, we have the following result. Theorem 3.1 Let Rj (n) be the interval that has the smallest width among all Rj (n)'s. If (jD)

is unimodal and (2.3) has unique solution, then we have

Rj  (n) ! R( ) as n ! 1; where R( ) is de ned in (2.2).

The proof is given in Appendix B. Next we present a central limit theorem (CLT) for ^() . In order to obtain the asymptotic normality, we assume that f(i; 'i); i = 1; 2; : : :ng is a random sample from g (; ').

Theorem 3.2 Let f(i; 'i); i = 1; 2; : : :ng be a random sample from g(; '). If Z

then we have

p2 (; 'jD)=g(; ')dd' < 1;

pn(^() ? ()) D N (0; 2);

where

R

(3.8)

R

(1 ? 2) 1f() g p2(; 'jD)=g (; ')dd' + 2 p2(; 'jD)=g (; ')dd' ; R ( p(; 'jD)dd')2  2(() jD) and p(; 'jD) is de ned in (3.1). If there exists  > 0 such that

2 =

inf

() ?t() +

and then

Z Z

(tjD) > 0

f(1 + jj)(1 + p(; 'jD)=g(; '))g2+ g(; ')dd' < 1; ^() ? () )2 =  2 :

nlim !1 nE (

(3.9) (3.10) (3.11)

The proof is given in Appendix C. Note that a similar CLT can also be obtained under certain regularity conditions such as ergodicity when f(i; 'i); i = 1; 2; : : :ng is a MCMC sample. However, the expression of the asymptotic variance  2 will be more complicated. Using (3.6), a 100(1 ? )% Bayesian credible interval for  is 



^(=2) ; ^(1?=2) ;

where the two limits of this credible interval, ^(=2) and ^(1?=2), are computed using (3.6). Similar to Corollary 2.1, we can obtain a HPD interval of  = h(; ') as follows. 8

(3.12)

Corollary 3.1 Let i = h(i; 'i) for i = 1; 2; : : :; n. Also let the (i) denote the ordered values of the i. Then, the th quantile of the marginal posterior distribution of  can be estimated by

^() =

(

(1) (i)

if P= 0; Pi if ij?1 =1 w(j ) <   j =1 w(j );

(3.13)

where w(j ) is the weight function associated with the j th ordered value (j ) . Using (3.13), we compute 

? j+[(1?)n]


j Rj (n) = ^( n ); ^

n

;

(3.14)

and a 100(1 ? )% HPD interval of  is Rj  (n) that has the smallest interval width among all Rj (n)'s.

We note that the same weight function wi can be used for computing the HPD interval for  as well as for any functions of  and '. Therefore, the proposed Monte Carlo methods are advantageous and computationally ecient when one wants to simultaneously compute the HPD intervals for many posterior quantities of interest.

4. APPLICATION TO A BAYESIAN HIERARCHICAL MODEL In this section, we apply the importance sampling approach developed in Section 3 to a Bayesian hierarchical model in which the resulting posterior density contains analytically intractable integrals that depend on hyperparameters. Suppose that a Bayesian hierarchical model has the following posterior density: 'j) 1 (; ')
()1 (); (4.1) (; '; jD) = c (1D) L(; '; D)
(;  c (  ) S w where  and ' are the parameters of interest,  is a vector of hyperparameters, and  (; 'j) and R R () are proper priors with the supports  and , that is,  (; 'j)dd' = 1 and  ()d = 1. In (4.1), S   is the constrained space that may or may not depend on the data according to the nature of the problem, Z c() = (; 'j)dd'; (4.2) S and Z cw (D) = L(; '; D) [(; 'j)=c()] ()dd'd;

where is the support of the  (; '; jD) de ned as

= S  = f(; '; ) : (; ') 2 S and  2 g : 9

Note that the posterior distributions given in (4.1) typically arise from constrained or ordered parameter problems (see, for example, Gelfand, Smith and Lee 1992 and Chen and Shao 1998). Also note that when S = , c() = 1 and thus, the constrained parameter problem disappears. Our goal is to obtain 100(1 ? )% credible and HPD intervals for  when S is a subset of . As discussed in Gelfand, Smith, and Lee (1992), directly sampling from  (; '; jD) is nearly impossible. Therefore, Chen and Shao (1998) proposed to sample from a natural importance sampling density g(; '; ) = c(1D) L(; '; D)(; 'j)1S(; ')()1(); where Z  c (D) = L(; '; D)(; 'j)()dd'd:

 Note that c (D) need not be calculated since it will be cancelled out in calculation.

To ease the presentation, we assume that f((i); '(i); (i)); i = 1; 2; : : :; ng is an ergodic MCMC sample from g where (1)  (2) 


 (n) . Similar to (3.5), the empirical cdf of  is given as follows: 8 > if  < (1) < 0 Pi  b ( jD) = > j =1 wj if (i)   < (i+1) ; (4.3) : 1  if   (n) where 1=c( ) (4.4) wi = Pn 1=c((i) ) :

In (4.4), the c((i))'s are unknown and

(j ) j =1 so is b j ( jD). Recall that

c() =

Z

S

(; 'j)dd':

Then, using a two-stage procedure proposed by Chen and Shao (1998), c((i)) can be estimated by m  (mix; 'mixj )1S (mix ; 'mix) X j j (i) j j ; (4.5) c^m((i)) = m1 mix mix mix(j ; 'j ) j =1 where mix(; ') is a \mixture" density with its support S and known up to a normalizing constant and f(jmix ; 'mix j ); j = 1; 2; : : :; mg is a MCMC sample from mix . Chen and Shao (1998) provided an automatic procedure to obtain a good mixture density mix (; '). Here, mix (; ') is said to be good in the sense that mix (; ') will \cover" every  (; 'j)1S (; ') for  2 , that is, mix (; ') has a shape similar to  (; 'j)1S(; '), a location close to that of  (; 'j)1S (; '), and a heavier tail than  (; 'j)1S (; '). Using (4.5), the empirical cdf of  at  is obtained by 8 > if  < (1) < 0 Pi  ~ m ( jD) = > j =1 w^j if (i)   < (i+1) ; (4.6) : 1  if   (n) 10

where Thus, j() can be estimated by

1=c^ ( ) w^i = Pn 1m=c^ ((i) ) : j =1 m (j )

(4.7)

(

(1) if P= 0 Pi (4.8) (i) if ij?1 =1 w^j <   j =1 w^j : Similarly, we can obtain a 100(1 ? )% credible interval given by (3.12) and we use (3.7) to obtain a 100(1 ? )% HPD interval for . Further, we can use (3.13) and (3.14) to obtain a 100(1 ? )% HPD interval for a function, h(; '; ), of , ', and . ^() =

5. SIMULATION STUDY In this section, we aim to study the performance of our Monte Carlo estimate of the HPD interval for  given in Theorem 2.2. We consider a Bayesian inference concerning a variance ratio of two independent normal populations. Suppose that Nj independent observations yjl ; l = 1; 2; : : :; Nj are drawn from normal population N (j ; j2) for j = 1; 2. We take a uniform prior for 1 , 2 , log 12, and log 22 , that is, (1; 2 ; 12; 22) / 1=(1222 ). Let s2j denote the sample variance of the yji 's and j = Nj ? 1 for j = 1; 2. We are interested in making inference on the variance ratio  = 22=12. In particular, we want to obtain a 100(1 ? )% HPD interval of  since the posterior distribution of  is skewed. From Box and Tiao (1992), the marginal posterior distribution of  is given by  2 1 =2 s 1 1

! 1

2 1 ? 2 (1 +2 ) s 2 2 1 + s12   ;  > 0: (5.1) (js1 ; s2) = 2 2 For illustrative purposes, we consider N1 = 20, s21 = 12, N2 = 12, and s22 = 50. Box and Tiao (1992) also used a similar example to illustrate how to derive a HPD interval for log(22=12) scaled in 22 =12. Since we are interested in a HPD interval of 22 =12, we cannot directly use their results because the HPD interval is not invariant under a nonlinear transformation. Therefore, we use Theorem 2.1 to obtain an exact 100(1 ? )% HPD interval, denoted by (L (); U ()), for  = 22 =12, by the Nelder-Mead algorithm. Because  is univariate, the Nelder-Mead algorithm is expected to work well. However, this optimization algorithm will become very inecient in high dimensional problems; see Nemhauser et al. (1989) for the detailed discussion. Assume that fi; i = 1; 2; : : :; ng is a random sample from (js21; s22) given by (5.1). Then, from Theorem 2.2, an estimated HPD interval of  is ((j  ) ; (j +[(1?)n])), where j  is de ned in (2.6). To study convergence of ((j  ) ; (j +[(1?)n]) ), we de ne the following mean relative error (ME): 1 1 ?1 s22 2 2  B(1 =2; 2=2)





MEn = E j(j  ) ? L ()j + j(j  +[(1?)n]) ? U ()j =(U () ? L ()); 11

(5.2)

where the expectation is taken with respect to the distribution of the i . For a given Monte Carlo sample size n, MEn quanti es the relative dierence between the estimated HPD interval ((j  ) ; (j +[(1? ) and the exact HPD interval (L (); U ()) for .  )n]) Since E j(j  ) ? L ()j + j(j  +[(1?)n]) ? U ()j) is analytically intractable, we use a usual simulation technique to estimate this expectation. We ran 500 simulations and we calculated MEn;k = (j(j );k ? L ()j + j(j +[(1?)n]);k ? U ()j)=(U () ? L ()) for k = 1; 2; : : :; 500. Then, MEn is approximated by (1=500) P500 k=1 MEn;k and the simulation standard error is the square root of the sample variance of the MEn;k 's. Table 1 gives the MEn 's with the simulation standard errors for various n and 1 ? . From Table 1, it can be observed that the disagreement between the estimated and exact HPD intervals is within 10% of the length of the exact one for all cases. So, the estimated HPD intervals are quite close to the exact ones even for n = 500. Further, based on the 75%, 90%, and 95% HPD intervals, 22 is signi cantly greater than 12 since all the three HPD intervals are located on the right hand side of 1.

Table 1: Mean Relative Errors of the Estimated HPD intervals of  with Simulation Standard Errors in Parentheses

=

22=12

MEn 1 ?  (L (); U ()) n = 500 n = 1; 000 n = 5; 000 0.75 (1.6435, 6.4546) .0891 .0614 .0341 (.0023) (.0016) (.0010) 0.90 (1.2407, 8.7862) .0636 .0447 .0364 (.0017) (.0012) (.0007) 0.95 (1.0263, 10.7439) .0650 .0482 .0238 (.0018) .0013) (.0006)

6. A REAL DATA EXAMPLE: MEAL, READY-TO-EAT To illustrate our proposed methods, we consider a constrained Bayesian hierarchical model for the Meal, Ready-to-Eat (MRE) data initially studied by Chen, Nandram and Ross (1996) and further elaborated by Chen and Shao (1998). As described in Chen, Nandram and Ross (1996), the MRE canned foods were purchased and then were inspected for completeness and stored at four dierent temperatures, c1 = 4 C , c2 = 21C , c3 = 30C and c4 = 38 C , then withdrawn and tested at t1 = 0, t2 = 6, t3 = 12, t4 = 18, t5 = 24, t6 = 30, t7 = 36, t8 = 48 and t9 = 60 months. At 4 C the food was not tested at 6, 18, and 24 months, at 21C the food was not tested at 6 months, and at 30C and 38C the food was not tested after 36 and 24 months, respectively. Upon purchase the foods were immediately tested at room temperature (21C ), and thus data were only available for time 0 at 12

room temperature. At each temperature-time combination each food item served to 36 untrained, randomly chosen subjects who judged its acceptability on a nine-point hedonic rating scale where 1 =dislike extremely, 9 =like extremely, and intermediate scores have graduated meanings. In this example we are interested in obtaining the HPD intervals for the mean rating scores for ham-chicken loaf, one of the canned foods. Notice that the panelists were used on only 23 temperature-time combinations. We have a total of 4 missing (by design) combinations, and, therefore, they are not included in the analysis. Also notice that the pastries data were available and hence, this data set is used to specify the prior distributions. Let Yijl be the score given by the lth panelist for ham-chicken loaf at temperature ci and withdrawn at time tj . The Yijl are independent and identically distributed with P (Yijl = kjpij ) = pijk where pij = (pij1; pij2; : : :; pij9) and P9k0=1 pijk0 = 1 for i = 1; : : :; 4, j = 1; : : :; 9 and k = P 1; : : :; 9. Thus, the mean score is !ij = 9k=1 kpijk . For temperature ci , as the quality of food deteriorates with time, we have the constraint

!ij  !i;j?1 ; j = 2; 3; : : :; 9

(6.1)

and for withdrawal time tj , as the quality of food deteriorates with temperature, we have the constraint !ij  !i?1;j ; i = 2; 3; 4: (6.2) In (6.1) and (6.2) there is one adjustment. That is, !21  !i2 for i = 2; 3; 4. For the missing temperature-time combinations there are obvious adjustments to the constraints (6.1) and (6.2). By introducing latent variables Zijl so that

Yijl = k if ak?1  Zijl  ak ; k = 1; 2; : : :; 9

(6.3)

and

Zijl jij i:i:d:  N (ij ; 2) for i = 1; 2; 3; 4 and j = 1; 2; : : :; 9, it can be shown that   9  X a ?  l ij !ij = 1 + 1? ;  l=1

where  is the standard normal cumulative probability function. In (6.3) we take a0 = ?1, a1 = 0, a8 = 1, a9 = 1, and the cutpoints 0  a2  : : :  a7  1 are speci ed by using the similar pastries data. The values are a2 = 0:0986, a3 = 0:191, a4 = 0:299, a5 = 0:377, a6 = 0:524 and a7 = 0:719 (see Chen, Nandram, and Ross 1996). Since !ij is an increasing function of ij , it follows that the constraints (6.1) and (6.2) on the mean scores are equivalent to the constraints on the means of the latent variables,

ij  i;j?1 ; j = 2; 3; : : :; 9 and ij  i?1;j ; j = 2; 3; 4: 13

(6.4)

For the missing temperature-time combinations, the constraints (6.4) require the same adjustment as for the constraints (6.1) and (6.2). Chen and Shao (1998) considered the following temperature-eect additive model of the form

ij = i + ij

(6.5)

subject to the constraints (6.4) where ij i:i:d:  N (0; 2) and 2 is the variance of ij for i = 1; 2; 3; 4 and j = 1; 2; : : :; 9. We also add the constraints

1   2   3   4

(6.6)

to ensure consistency with (6.4). We take a diuse prior for the i over the constrained parameter space de ned by (6.6) and we choose priors for  2 and  2 as  2 2 (6.7) 2   and 2   : In (6.7)  ,  ,  and are to be speci ed by using the pastries data, and the values are  = 16:88,  = 0:83,  = 4:60 and = 0:01 (see Chen, Nandram and Ross, 1996 for the detail). McCullaph (1980) considered general regression models for independent ordinal data and Johnson (1996) used the latent structure models to analyze the automated essay grading data, where the ordinal responses are dependent. The main dierence between our model and their models is the incorporation of the order restrictions, which may present further challenge in posterior computation. In (6.3), we introduced the latent variables Zijl into the model. The main reason for using the strategy involving latent variables is to simplify posterior computation. In particular, when partial information on cutpoints ak 's is available, which is the case for our MRE data, the implementation of the Gibbs sampler becomes simple and straightforward. See Albert and Chib (1993), Nandram and Chen (1996), and Chen, Nandram and Ross (1996) for the detailed discussions. Let S be the constrained parameter space associated with the order constraints (6.4) on the ij . Also let  = (ij ) and  = (1; : : :; 4;  2). Because of the constraints (6.4) and the temperatureeect additive model (6.5), the prior distribution for  given  depends on the normalizing constant

c() =

8 Z