0) for binary responses, and µ = (µ1 , . . . , µp ) is a p × 1 intercept vector, Λ =
(λ1 , . . . , λp )0 is a p × 1 vector of factor loadings, ηij is a latent morbidity score for the jth patient 9
having procedure i and where K(·; γ) is an unknown unimodal kernel that is symmetric about γ. The procedure-specific latent variable density functions are modeled as a flexible location-scale mixture of Gaussian densities. By using an unknown kernel, we favor fewer and more biologically −2 interpretable clusters. Letting σit = ci dt for continuous responses, we obtain an additive log-linear
model for the residual precision, with ci a procedure-specific multiple and dt a response type specific −2 multiple, while fixing σit = 1 for binary responses. This allows the residual variance to change
for the different procedures, while also allowing a shift specific to each measure of morbidity. The constraint on the residual variances for the continuous variables underlying the binary responses ∗ imply higher morbidity, we is a standard identifiability condition. Because higher values of yijt
constrain the factor loadings to be non-negative so that λt ≥ 0 for t = 1, . . . , p. For the scale mixture component, we let Q ∼ DP (α0 Q0 ) where Q0 = Gamma(c0 , d0 ) is the base measure. Within expression (7), fi denotes the density of the latent morbidity score specific to patients receiving procedure i. This density is treated as unknown using a flexible location-scale mixture of Gaussians. It is straightforward to show that fi is marginally modeled as a Dirichlet process mixture of mixture of normals, as in Lo (1984) and Escobar and West (1995). It is well known that such a model is highly flexible. We avoid using Pi directly as the distribution of the latent factor scores within procedure i, since that would assume that the factor scores follow a discrete distribution. It seems more biologically realistic to allow a continuum of patient morbidity, while allowing patients with similar but not identical morbidity to be clustered. This is accomplished by the proposed model in that patients allocated to the same mixture component will be clustered. As mentioned above, we are more interested in clustering and ranking of the medical procedures instead of the patients. Because K(·; γ) is monotonically stochastically increasing in γ, we maintained the stochastic ordering restriction in the Pi s. Note that two procedures i and i0 having Pi = Pi0 , which is allowed by the proposed prior, will also have fi = fi0 and hence have the same morbidity density. In addition, fi ≺ fi0 (the distribution of patient morbidity under procedure i is stochastically less than that under procedure i0 ) if and only if Pi ≺ Pi0 . Hence, the clustering and ranking properties of the prior for {Pi } proposed above extend directly to the continuous latent factor model in (7). To complete a Bayesian specification of the SO-LCM model in (7), we choose priors as follows. The intercept vector is assigned a normal prior, µt ∼ N(µ0 , σ02 ) for t = 1, . . . , p, and the factor
10
loadings are assigned robust truncated Cauchy priors by letting λt ∼ N+ (0, τ ) for t = 1, . . . , p with τ ∼ Inv-Gamma(1/2, 1/2). We use a common precision τ to include dependent shrinkage across the loadings. And the multiplicative terms in the variance model, {ci } and {dt }, are assigned gamma priors. Elicitation of the different hyperparameters in these priors is considered later. 3.
POSTERIOR COMPUTATION
Due to the structure of the model described in section 2.1, it becomes straightforward to adapt previously proposed algorithms for posterior computation in DPMs and logistic regression models. Because the structure of the base measure creates some difficulties in implementing P´olya urnbased algorithms, we will focus on the exact block Gibbs sampler (Yau et al. 2010) for posterior computation and update polychotomous weights through Holmes and Held (2006). We will focus R R on the simple model yij ∼ fi , fi (η) = K(η; γ)dPi (γ), K(η; γ) = N(η; γ, ϕ)dQ(ϕ), Pi ∼ SO-LCM and Q ∼ DP(α0 Q0 ). The sampling steps are as follows, 1. Sample πk (xi ) through the following steps. The polychotomous generalization of the logistic regression model is defined via ζi ∼ M(1; π1 (xi ), . . . , πK (xi )),
ψk exp(x0i β j ) , πk (xi ) = PK 0 l=1 ψl exp(xi β l )
(8)
where ζi is the procedure cluster indicator and M(1; ·) denotes the single sample multinomial distribution. Defining β [k] = (β 1 , . . . , β k−1 , β k+1 , . . . , β K ), we have L(β k |ζ, β [k] ) ∝
n Y K Y
[πk (xi )]1(ζi =j)
i=1 k=1 n Y ∝ [χij ]1(ζi =j) [1 − χij ]1(ζi 6=j)
(9)
i=1
where χij Cij
exp{x0i β j + log(ψj ) − Cij } exp(xˆi 0 βˆj − Cij ) = , = 1 + exp{x0i β j + log(ψj ) − Cij } 1 + exp(xˆi 0 βˆj − Cij ) £X ¤ £X ¤ = log exp{x0i β k + log(ψk )} = log exp(xˆi 0 βˆj ) k6=j
(10)
k6=j
where xˆi = (x0i , 1)0 and βˆj = (β 0j , log ψj )0 . The prior for log(ψk ) from (4) can be approxiD
mated as log(ψk ) ≈ N(m, v). We use this approximation to obtain an efficient Metropolis 11
ˆ |ζ, β ˆ ) has the form of a loindependence chain proposal. The conditional likelihood L(β j [j] gistic regression on class indicator 1(ζi = j), which allows us to use the algorithm of Holmes and Held (2006). Details are in appendix B. 2. Sample the procedure cluster indicators ζi , for i = 1, . . . , n, from a multinomial distribution with probabilities Pr(ζi = k| · · · ) ∝ πk (xi )
ni X L Y
vl N(yij ; θkl , ϕij )
j=1 l=1
and construct mk =
Pn
i=1 1(ζi
≡ k). For K, we first choose a reasonable upper bound and
then monitor the maximum index of the occupied components. If all the MCMC samples have maximum indices several units below the upper bound, then the upper bound is sufficiently high, while otherwise the upper bound can be increased, with the analysis re-run. 3. The joint prior distribution of the group indicator ξij and a latent variable qij can be written as X
f (ξij , qij |v) =
δj (·) =
∞ X
1(qij < vl )δl (·).
l=1
l:vl >qij
Implement the Exact Block Gibbs sampler steps: i. Sample qij ∼ Unif(0, vξij ) for j = 1, . . . , ni with vl = νl
Q
s 1 − min{qij }. iii. Sample ξij for i = 1, . . . , n and j = 1, . . . , ni from the multinomial conditional with Pr(ξij = l) ∝ 1(qij < vl )N(yij ; θζi l , ϕij ). 4. Sample γl∗ from p(γl∗ | · · · ) ∝
£
Y
¤ N(yij ; θζi l , ϕij ) P0 (γ ∗l )
{(i,j)|ζi ,ξij =l}
12
∗ , . . . , γ ∗ ) as defined in (3). P is the baseline where θkl = wk0 γ ∗l , wk = (10k , 00K−k )0 , γ ∗l = (γ1l 0 Kl
measure. If no observation is assigned to a specific cluster, then the parameters are drawn directly from the prior distribution P0 . Construct θkl through θkl = wk0 γ ∗l . 5. ϕij is updated through the exact blocked gibbs sampler similar to the above steps. 6. Use random walk Metropolis-Hastings method to update concentration parameter α1 . 7. Sample concentration parameter α2 with conjugate prior Gamma(a2 , b2 ) directly from L−1 X ¡ ¢ (α2 | · · · ) ∼ Gamma a2 + K(L − 1), b2 − K log(1 − vl ) l=1
We should note that the accuracy of the truncation depends on the values of α1 and α2 . Thus the hyperparameters (a1 , b1 ) and (a2 , b2 ) should be chosen to give little prior probability to values of α1 and α2 larger than those used to calculate the truncation level. Note that this algorithm can be generalized easily to accommodate model (7), so the details are omitted. 4.
SIMULATION STUDY
We separate this section into two parts. Predictors are not considered in the first simulation but will be considered in the second simulation. Model (7) is studied and both simulations mimic the structure of the medical procedure data. 4.1
Without predictors
Data yij are generated according to (7), with one continuous response (p1 = 1) and six binary responses (p2 = 6) for each of 100 patients (j = 1, . . . , 100) in each of 60 procedures (i = 1, . . . , 60). Parameters for the data generating model are µ = (0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4)0 , Λ = 10p×1 and Σ = diag(0.5, 1, 1, 1, 1, 1, 1). The latent morbidity ηij is generated from one of four mixtures of Gaussian components outlined in Table 1, with the first fifteen procedures being generated from mixture distribution T1 , the second fifteen procedures generated from T2 , the third fifteen procedures generated from T3 and the last fifteen procedures generated from T4 , where T1 ≺ T2 ≺ T3 ≺ T4 such that the generated latent morbidity distributions are stochastically ordered. As shown 13
in Figure 1 and Table 1, distributions share components with each other and the ordering of the distributions is subtle. [Figure 1 about here.] To obtain an initial clustering of the medical procedures using standard methods, we first averP i yij as aged the severity data for the different patients having each procedure to obtain y i = n1i nj=1 a p = 7 dimensional summary of severity for procedure i. We then applied model-based clustering (Fraley and Raftery 2002; Fraley, Raftery and Wehrens 2005) to the data {y 1 , . . . , y n } using the R functions available in the package described in Fraley et al. (2005). These approaches rely on fitting of finite mixture models with the EM algorithm, with the model fit for a variety of choices of the number of mixture components, which also corresponds to the number of clusters. The BIC is used to select the optimal number of clusters. Figure 2 plots the BIC for the simulated data vs the number of cluster for four different options on the cluster shapes. In this case, the best model according to BIC is EEI model (equal size and shape) with six clusters. Note that this approach does not utilize the patient-specific data and instead clusters based on the mean severity measure across patients, while the proposed approach should have advantages in clustering procedures based on the entire distribution across patients. [Figure 2 about here.] For the SO-LCM estimation, parameters α1 , α2 are fixed to be 1 and a normal inverse-gamma prior distribution, NIG(0, 0.1, 2, 3) is chosen for the baseline measure m0 and s20 described in (3), implying that E(m0 |s20 ) = 0,V (m0 |s20 ) = 10s20 , E(s20 ) = 1, and V (s20 ) = 3. Additionally, we assign priors w0 ∼ beta(1, 1), κ ∼ Gamma(1/2, 1/2), representing a robust and flexible default prior for the base measure P0 . Without predictors, the prior for the cluster-specific allocation probabilities turns out to be (π1 , . . . , πK )0 ∼ Dir(α1 /K, . . . , α1 /K). Posterior samples under this SO-LCM prior are obtained through the algorithm described in Section 3 with prespecified truncation bounds K = 20. This truncation tends to be accurate for α1 ≤ 1 and α2 ≤ 1, where such values of α1 and α2 favor a small number of mixture components. In this particular application, mixture components close to the upper bound are not occupied in any of the MCMC samples after the burn-in period. 14
20,000 iterations are found to be enough for parameters to converge. All results are based on 5,000 samples obtained after a burn-in period of 20,000 iterations. For each pair of distributions Pi and Pi0 , (i < i0 ), the probability Pr(Pi = Pi0 ) was calculated as the proportion of posterior samples for which Pi and Pi0 are assigned to the same cluster; and Pr(Pi ≺ Pi0 ) is calculated as the proportion of posterior samples for which Pi is assigned to a cluster with stochastically less morbidity than Pi0 . Results are shown in Figure 3, where Figure 3(a) is the ranking plot with the (i, j)th entry of the lower triangular matrix identifying the probability for Pi ≺ Pi0 and Figure 3(b) is the clustering plot with the (i, j)th entry identifying the probability for Pi = Pi0 . Figure 3 illustrates that there is not enough information in the data to differentiate the first thirty procedures, which is not surprising given the very subtle differences in T1 and T2 shown in Figure 2. However, the true rankings and clusterings in the medical procedures are otherwise accurately reflected in the results. The estimated density of T1 is shown in Figure 4(a). For comparison, this density is also estimated under a DPM prior with the same base measure and precision parameter α2 = 1 (in Figure 4(b)). The estimate obtained using the SO-LCM prior distribution appears to capture both the small and large modes more accurately than the DPM alternative. [Figure 3 about here.] [Figure 4 about here.] 4.2
With predictors
Potentially, the incorporation of predictors may improve the ability to detect subtle differences in the differences of patient morbidity between procedures. To assess this, we repeated the simulation study of Section 4.1 but modified the model to allow predictor-dependent mixture weights. Mimicking the real data, we assumed there was a single predictor corresponding to an initial physician severity score obtained from their clinical experience and not from examination of the current data. In particular, predictors for the first fifteen procedures are chosen uniformly from (-4, -3.5), the second fifteen procedures chosen uniformly from (-0.6, -0.5), the third fifteen procedures chosen uniformly from (0.2, 0.3) and the last fifteen procedures chosen uniformly from (3.5, 4). Data are
15
then generated from the assumed model exactly as described in Section 4.1 but assuming logistic regression model (4) for the weights with ψ = (0.5, 1, 1.5, 2)0 and β = (−1.5, −0.5, 0.5, 1.5)0 . In the analysis, priors are specified as described in Section 4.1 and model (4) and we additionally choose a N(0,10 I) prior for β to complete the specification. Posterior computation was performed using the MCMC algorithm shown in Section 3. Apparent convergence was rapid, and 20,000 iterations were judged to be sufficient. Prespecified truncation bounds K = 20 are still adequate. Results are based on 5,000 samples after a burn-in period of 20,000 iterations. Figure 5(a) depicts the ranking performance and Figure 5(b) depicts the clustering plot. Both ranking and clustering performances are improved compared to study 4.1 such that the first thirty procedures are ranked consistently with the true order and are clustered correctly. [Figure 5 about here.] 5.
MEDICAL PROCEDURE APPLICATION
The analysis of congenital heart surgery outcomes data is challenging because of the wide array of defects encountered and the large number of associated surgical procedures. Certain diagnoses occur relatively frequently, but variations on the typical anatomy are commonplace. To overcome this difficulty, researchers have proposed methods to allow procedures with similar mortality and morbidity risk to be grouped together for analysis. Two widely used methods are the Risk Adjustment for Congenital Heart Surgery (RAHCS-1) methodology (Jenkins 2004) and the Aristotle Basic Complexity Levels. RACHS-1 groups 143 types of congenital heart surgery procedures into 6 categories based on their estimated risk of in-hospital mortality. Similarly, the Aristotle method groups 143 types of procedures into 4 categories (levels) based on their potential for mortality, morbidity, and technical difficulty. For both RACHS-1 and Aristotle, procedure categories were determined by panels of subject matter experts without using a formal statistical framework. In this section, our goal is to show that the SO-LCM methodology provides a useful statistical framework for grouping procedures into categories of risk and for choosing the number of categories. More formally, we sought to identify clusters of congenital procedures with similar distributions of post-procedural morbidity. Data for this analysis were obtained from the Society of Thoracic Surgeons (STS) Congenital 16
Heart Surgery database. The study population consisted of N=79,635 patients who underwent one of 143 types of congenital cardiovascular procedures at an STS-participating center during the years 2002-2008. Post-operative morbidity was regarded as a patient-level unobserved latent variable. Indicators of morbidity included a single continuous variable, post-operative length of stay (PLOS), modeled as y1 = log(1 + PLOS); and 6 binary (yes/no) variables: y2 = renal failure, y3 = stroke, y3 = heart block, y4 = requirement for extracorporeal membrane oxygenation or ventricular assist device; y5 = phrenic nerve injury, and y6 = in-hospital mortality. Responses from different patients were assumed to be independent. Multiple responses from the same patient were conditionally independent given the latent morbidity variable. The joint model for all 7 endpoints is: yij1 |ηij
2 ∼ N[µ1 + λ1 ηij , σi1 ]
yij2 |ηij
∼ Bernoulli[Φ(µ2 + λ2 ηij )]
(Stroke)
yij3 |ηij
∼ Bernoulli[Φ(µ3 + λ3 ηij )]
(Renal failure)
yij4 |ηij
∼ Bernoulli[Φ(µ4 + λ4 ηij )]
(Heart block)
yij5 |ηij
∼ Bernoulli[Φ(µ5 + λ5 ηij )]
(ECMO/VAD)
yij6 |ηij
∼ Bernoulli[Φ(µ6 + λ6 ηij )]
(Phrenic nerve injury)
yij7 |ηij
∼ Bernoulli[Φ(µ7 + λ7 ηij )]
(In-hospital mortality)
(PLOS)
Let fi denote the density of ηij among patients undergoing the ith type of procedure, i.e. ηij ∼ fi . Our goal is to estimate the densities f1 , f2 , . . . , f143 nonparametrically under the assumption that they have an unknown stochastic ordering. This assumption facilitates ranking of the procedures and is less restrictive than alternative parametric models which assume a Gaussian distribution and procedure-specific location parameters. An important consideration for the analysis is that the procedure-specific sample sizes are small and highly variable (median = 50; range 10 to 2000). To account for these low sample sizes, we propose a method of analysis that permits borrowing of information across procedures and incorporates external prior information. A potentially useful auxiliary covariate is each procedure’s Aristotle Basic Complexity (ABC) score. The ABC score is a number ranging from 0.5 to 15 that represents the average subjective ranking by an international panel of congenital heart surgeons. Large ABC scores imply that the procedure is considered to be 17
a difficult operation with high potential for mortality and morbidity. The procedure-specific morbidity distributions are estimated nonparametrically under the SOLCM prior distribution, Z
Z
fi (η)
=
Pi
ind
K X
∗ {P1∗ , P2∗ , . . . , PK }
∼
rDPP(α2 P0 ),
∼
K(η; γ)dPi , πk (xi )δPk∗ ,
i=1
K(η; γ) =
N(η; γ, ϕ)dQ(ϕ)
ψk exp(x0 β k ) πk (x) = PK 0 l=1 ψl exp(x β l ) Q ∼ DP(αQ0 ). ind
Hyper-priors are ψk ∼ Gamma(α1 /K, 1), α1 ∼ Gamma(1, 6/ ln 143), and βk ∼ N(0, 10) and α2 ∼ Gamma(1, 1). The prior for α1 is chosen based on expert elicitation to favor 6 or fewer clusters in the procedures. The prespecified truncation bound K = 20 is found to be enough since mixture components close to the upper bound are not occupied after the algorithm converges. The baseline distribution P0 is constructed as in (3) with w0 ∼ beta(1, 1), κ ∼ Gamma(1/2, 1/2) and ind
ind
(m0 , s20 ) ∼ NIG(0, 0.1, 2, 3). Priors for the outcomes model are µt ∼ N(0, 10) and λt ∼ N+ (0, τ ), t = 1, 2, . . . , p, where τ ∼ InvGamma(0.5, 0.5). Estimates are calculated with 50,000 MCMC iterations after a burn-in period of 20,000 iterations. We took a long burn-in and collection interval to be conservative, but similar results are obtained using shorter chains. The estimated association between latent morbidity and risk of complications is depicted in Figure 6. To facilitate interpretation, morbidity is plotted on the scale of percentiles of the marginal distribution of η. The 100pth percentile is defined as F −1 (p) where F −1 is the inverse of the function P ni F (x) = Pr(η < x) = 143 i=1 n Fi (x) and n = n1 + n2 + · · · + n143 . Details are in Appendix C. For each endpoint, the increase of estimated risk ranges from 10% to 40% for patients in the 90th percentile of the morbidity distribution compared to patients in the 10th percentile. These results suggest that the selected outcomes are internally consistent and valid indicators of the concept of morbidity. [Figure 6 about here.] Figures 7 and 8 summarize posterior inferences for procedures (n = 66) having at least 200 P occurrences in the database. For the ith procedure, let Ri = 66 h=1 I(Fi ¹ Fh ) denote the number of procedures having a morbidity distribution that is stochastically no smaller than Fi . In each 18
figure, procedures are sorted in ascending order based on the posterior mean E[Ri ]. In Figure 7, the posterior means are plotted on the vertical axis along with 95% probability intervals. The relatively wide probability intervals indicate that there is uncertainty regarding the true rank ordering of the procedure-specific morbidity distributions. Nonetheless, several procedures have narrow intervals and are clearly distinguished as either low (e.g. atrial septal defect repair) or high (e.g. Norwood operation) latent morbidity. Ranking and clustering performances are depicted in Figure 8(a) and 8(b). The total 143 procedures can be grouped into four, five or six homogeneous clusters according to posterior clustering probabilities shown in Table 2. The data suggest high (99%) posterior probability of fewer than 8 clusters, with 32% probability assigned to the posterior mode of 5. We also propose a way to obtain an optimal point estimation of the ranked clustering as following. Laird and Louis (1989) represented the ranks by, Rk (θ) = rank(θk ) =
K X
I{θk ≥θj } ,
(11)
j=1
with the smallest θ having rank 1 and the largest having rank K. Denote that the true rank for θ ˜ i is the estimated rank (we ˜ , Ri is the rank variable for object i and R is p, the estimated rank is p drop the dependency on θ for notational convenience). To find the optimal ranked clustering, we ˜ ) as define the following loss function L(p, p L(p, p˜) =
X ¡
˜i < R ˜ j , Ri > Rj ] + 2 × 1[R ˜i > R ˜ j , Ri < Rj ] 2 × 1[R
(i,j)∈M
¢ ˜i = R ˜ j , Ri 6= Rj ] + 1[R ˜ i 6= R ˜ j , R i = Rj ] . +1[R
(12)
˜i = R ˜ j half of those pairs for which the ordering are in We penalize for the pairs with Ri = Rj or R the opposite direction. The posterior expected loss turns out to be X µ ˜i < R ˜ j )Pr{Ri > Rj |y} + 2 × 1(R ˜i > R ˜ j )Pr{Ri < Rj |y} 2 × 1(R E(L(p, p˜|y)) = (i,j)∈M
¶ ˜i = R ˜ j )Pr{Ri 6= Rj |y} + 1(R ˜ i 6= R ˜ j )Pr{Ri = Rj |y} , + 1(R where Pr{Ri > Rj |y}, Pr{Ri < Rj |y}, Pr{Ri = Rj |y} and Pr{Ri 6= Rj |y} are estimated through the MCMC outputs. We may obtain an optimal ranked clustering which achieves a smallest Bayes risk. 19
To compare the performance of our method with that of the Aristotle level(with 4 levels) based on the Bayes risk, we let K = 4 so that we will only obtain 4 clusters. The Bayes risks for the ranking clustering obtained from the Aristotle score is 7726.9. Our optimal Bayesian ranked clustering achieves smaller risk: 6533.1. We compare groupings based on Aristotle to our final point estimate in table 3. Several procedures that were predicted to be relatively low-risk by the Aristotle score were actually moderate-risk or high-risk according to our proposed methodology. Among 24 procedures that were Aristotle Category 1 (lowest risk), only 9 of these procedures were assigned to the lowest risk category according to our method. The correlation between these two ranked clustering is 0.45. The correlation between the ranked clustering of the SO-LCM and the log(1 + PLOS) is 0.86 and the correlation between the ranked clustering of the Aristotle level and the log(1 + PLOS) is 0.58. [Figure 7 about here.] [Figure 8 about here.] 6.
DISCUSSION
We have formulated a novel extension of the nested Dirichlet process(nDP) for a family of a priori subjects to a partial stochastic ordering that allows us to simultaneously rank and cluster procedures. The procedures are clustered by their entire distribution rather than by particular features of it. Similar to the nDP, the SO-LCM also allows us to cluster subjects within procedures. The SO-LCM is also straightforward to be imbedded for stochastically ordered mixture distributions within a large hierarchical model. Although inspired by the pioneering work of the nDP, this article makes several important contributions. First, the stochastically ordered priors that allow covariates to impact the allocation to clusters are developed to apply to nonparametrically estimate densities for multiple procedures subject to a stochastic ordering constraint. In addition, we can also test the hypothesis of equalities between procedures against stochastically ordered alternatives. After examining some of the theoretical properties of the model, we describe a computationally efficient implementation and demonstrate the flexibility of the model through both a simulation study and an application where
20
the SO-LCM is used within a hierarchical model. Heat maps are also offered to summarize the ranking and clustering structures generated by the model. It is straightforward to make several generalizations of the SO-LCM. One natural generalization is to include hyperparameters in the prior on the regression coefficients of the predictor dependent probabilities H and the baseline measure P0 . For H, we can choose a heavy-tailed Cauchy prior or a variable selection mixture prior with a mass at zero to shrink unimportant coefficients towards zero. We note that, conditional on P0 , the distinct atoms {Pk∗ }∞ k=1 are assumed to be independent. Therefore, including hyperparameters in P0 allows us to parametrically borrow information across the distinct distributions. Another natural generalization of the SO-LCM is to replace the beta(1, α2 ) stick-breaking densities with more general forms beta(ak , bk ) as considered in Ishwaran and James (2001), with the SO-LCM corresponding to the special case ak = 1, bk = α2 . Richer classes of priors that encompass the SO-LCM as a particular case will be obtained, though in some regression contexts it does not always outperform the DP model with beta(1, α2 ) in terms of the log-predictive marginal likelihood. We can also generalize the procedure to incorporate multivariate latent factors whose distributions are stochastically ordered. This generalization is inspired by the valuable suggestion from the editors. In having univariate stochastic ordering on the latent variable level, we actually induce multivariate stochastic ordering for the responses (albeit in a somewhat restrictive manner). To directly place the stochastic ordering constraint on multivariate distributions, we can adopt multivariate monotone functions. In particular, in place of the scalar θkl we could have a vector θ kl with P0 chosen (e.g. multivariate truncated normal) so that the different elements are appropriately ordered to satisfy the constraint. In the simple ordering case, we could just let (3) independently for each element of the θ kl vector instead of just for the θkl scalar. We could even have different orders for different variables and could have some variables with no ordering.
21
7.
APPENDIX
Clustering Probability Under (2), the probability that Pi = Pi0 so that groups i and i0 are allocated to the same cluster is, K K X X P(Pi = Pi0 ) = E( πk πk ) = E(πk2 ) k=1 K X
k=1
Var(πk ) + E(πk )2 =
=
k=1 α1 (α1 − αK1 ) K( K 2 α1 (α1 + 1)
=
+(
K X
k=1 α1 K 2
α1
) )=
α1 α1 K (α1 − K ) α12 (α1 + 1)
+(
α1 K 2
α1
)
α1 − αK1 1 + α1 (α1 + 1) K
And when K goes to infinity, α1 − αK1 1 lim + K→∞ α1 (α1 + 1) K
=
1 α1 + 1
MCMC supplement We would introduce a set of variables dij , i = 1, . . . , p, j = 1 . . . , K, and define yˆij = 1 if the ith observation belongs to class j, j ∈ {1, . . . , K} and yˆij = 0 otherwise. Notice that the equivalent representation of equation (8) is, ( 1, sij ≥ 0, yˆij = 0, else. sij
= xˆi 0 βˆj − Cij + ²ij ,
dij
= (2eij )2 ,
²ij ∼ N(0, dij )
eij ∼ KS,
ˆ ∼ π(β ˆ ) λj ∼ Gamma( α1 , 1) β j j K
(A.2)
Parameters of equation (A.2) are updated through following steps, ˆ , π(β ˆ ) = N(b0 , v0 ), the full conditional ˆ in the case of a normal prior on β 1. Sampling β j j j ˆ given s·j and d·j is still normal, distribution of β j ˆ |s·j , d·j β j ˆ 0 Wj (s·j + C·j )), Bj = Vj (v0−1 b0 + x
Vj
∼ MN(Bj , Vj ), ˆ 0 Wj x ˆ )−1 , = (v0−1 + x
−1 Wj = diag(d−1 1j , . . . , dnj )
ˆ = (ˆ ˆ p )0 , s·j = (s1j , . . . , snj )0 , C·j = (C1j , . . . , Cnj )0 and d·j = (d1j , . . . , dnj )0 . where x x1 , . . . , x ˆ , 2. Following advice of Holmes and Held (2006), we update {s·j , d·j } jointly given β j ˆ ,y ˆ ˆ )π(d·j |s·j , β ˆ ) π(s·j , d·j |β j ˆ ·j ) = π(s·j |β j , y ·j j 22
ˆ |s·j , d·j . The marginal densities for the sij ’s are independent followed by an update to β j truncated logistic distributions, ˆ + log(λj ) − Cij , 1)I(sij > 0), Logistic(ˆ x0i β j ˆ sij |β j , yˆij ∝ ˆ + log(λj ) − Cij , 1)I(sij ≤ 0), Logistic(ˆ x0i β j
yˆij = 1, else.
where Logistic(a, b) denotes the density function of the logistic distribution with mean a and scale parameter b. 3. Sampling d·j through rejection sampling. As advised by Holmes and Held (2006), we use r Generalized Inverse Gaussian distribution g(dlj ) = GIG(0.5, 1, r2 ) = InvGamma , where (1,r) ˆ )2 , as rejection sampling density. Following a draw from g(·) the sample is r = (sij − x ˆ0 β i
j
accepted with probability α(·), α(dlj ) = where M ≥ supdlj
L(dlj )π(dlj ) , g(dlj )
L(dlj )π(dlj ) M g(dlj ) −1/2
L(dlj ) denotes the likelihood, L(dlj ) ∝ dlj
exp(−0.5r2 /dlj ),
and π(dlj ) is the prior, π(dlj ) =
1 1/2 1 −1/2 dlj KS( dlj ) 4 2
where KS(·) denotes the Kolmogorov-Smirnov density. Section 5 supplement Define πi =
Pni i ni
the proportion of patients receiving procedure i, then F (x) = Pr(η ≤ x) =
p X
πi Fi (x)
(A.4)
i=1
where Fi (x) = Pr(η ≤ x|procedure = i) is the CDF for procedure i. Let F −1 (u) denote the quantile function associated with F . We can use the notation F (x|θ) and F −1 (u|θ) to emphasize that these functions depend on unknown parameters (namely, the matrix of mass points and the vector of associated probabilities). Take the first binary responses for example, yij1 = µ1 + λ1 ηij + ²1 , 23
where
²1 ∼ N(0, 1).
(A.5)
So, g(x) = Pr(yij1 > 0|ηij = x) = Φ(µ1 + λ1 x). Define the function h(u) ≡ h(u|µ1 , λ1 , θ) = Φ(µ1 + λ1 F −1 (u|θ)) p X = πi Φ(µ1 + λ1 Fi−1 (u|θ))
(A.6)
i=1
and let h(u) = {h(0.01), h(0.02), . . . , h(0.99)} . We would like to evaluate E[h(u)|data]. This can be done by evaluating (A.6) at each MCMC iteration and then averaging across iterations.
The hardest part is figuring out how to calculate F −1 (u|θ) efficiently. One approach would be to approximate F (X) as a piecewise linear function instead of a step function. R function approx can be implemented to evaluate F −1 (u) at specified values of u. The approx function takes as arguments a vector x, a vector y, and an optional vector xout. approx builds a piecewise linear function by interpolating between the y’s. It then evaluates the function at the points xout and returns the vector of associated y values. The trick used here is to reverse the roles of x and y. That is, let x = g(η) and y = η, and xout = c(0.01, 0.02, . . . , 0.99). η denotes a grid of values for η. To create the grid, we use values evenly spaced on the interval between the lowest and highest mass points that have any posterior probability. REFERENCES Dunson, D., and Peddada, S. (2008), “Bayesian nonparametric inference on stochastic ordering,” Biometrika, 95(4), 859–874. Escobar, M., and West, M. (1995), “Bayesian Density Estimation and Inference Using Mixtures,” Journal of the American Statistical Association, 90(430), 577–588. Fraley, C., and Raftery, A. E. (2002), “Model-based clustering, discriminant analysis, and density estimation,” Journal of the American Statistical Association, 97(458), 611–631. 24
Fraley, C., Raftery, A. E., and Wehrens, R. (2005), “mclust: Model-based Cluster Analysis,” R package version 2.1-11, . Holmes, C., and Held, L. (2006), “Bayesian auxiliary variable models for binary and multinomial regression,” Bayesian Analysis, 1(1), 145–168. Ishwaran, H., and James, L. (2001), “Gibbs sampling methods for stick-breaking priors,” Journal of the American Statistical Association, 96(453), 161–173. Ishwaran, H., and Zarepour, M. (2002), “Exact and approximate sum-representations for the Dirichlet process,” The Canadian Journal of Statistics, 30, 269–283. Jasra, A., Holmes, C. C., and Stephens, D. A. (2005), “Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modelling,” Statististical Science, 20(1), 50–67. Jenkins, K. (2004), “Risk adjustment for congenital heart surgery: the RACHS-1 method,” Seminars in thoracic and cardiovascular surgery. Pediatric cardiac surgery annual, 7, 180–184. Karabatsos, G., and Walker, S. (2007), “Bayesian nonparametric inference of stochastically ordered distributions, with Polya trees and Bernstein polynomials,” Statistics and Probability Letters, 77, 907–913. Laird, N. M., and Louis, T. A. (1989), “Empirical Bayes ranking methods,” Journal of Educational Statistics, 14, 29–46. Lo, A. (1984), “On a class of Bayesian nonparametric estimates: I. density estimates,” The Annals of Statistics, 12(1), 351–357. Rodriguex, A., Dunson, D., , and Gelfand, A. (2008), “The Nested Dirichlet Process,” Journal of the American Statistical Association, 103(483), 1131–1154. Stephens, M. (2000), “Dealing with label switching in mixture models,” Journal of the Royal Statistical Society, Ser. B, 62, 795–809. Teh, Y., Jordan, M., Beal, M., and Blei, D. (2006), “Hierarchical Dirichlet processes,” Journal of the American Statistical Association, 101(476). 25
Yau, C., Papaspiliopoulos, O., Roberts, G., and Holmes, C. (2010), “Nonparametric hidden Markov models with application to the analysis of copy-number-variation in mammalian genomes,” Journal of Royal Statistical Society: Series B (to appear), .
26
Table 1: True distribution used in simulation study 4.1 Comp1
Comp2
Dist
w
(µ, σ 2 )
w
(µ, σ 2 )
T1
0.75
(-3,0.5)
0.25
(0,1)
T2
0.75
(-2.5,0.5)
0.25
(0,1)
T3
0.2
(1,0.5)
0.5
T4
0.2
(2,1)
0.3
Comp3 w
(µ, σ 2 )
(1.5,1)
0.3
(2,1)
(2.5,0.5)
0.4
(3,1)
Comp4 w
(µ, σ 2 )
0.1
(3.5,0.5)
Table 2: Posterior Probability for clustering (Epidemiology Application) Number of clusters
Posterior Probability
1
0.01
2
0.02
3
0.12
4
0.21
5
0.32
6
0.25
7
0.06
8
0.01
27
Table 3: Clustering comparison between Aristotle Level and SO-LCM SO-LCM \ Aristotle
Level 1
Level 2
Level 3
Level 4
Level 1
14
12
10
1
Level 2
4
17
24
3
Level 3
4
7
12
16
Level 4
1
5
4
11
28
1 2 3 4 5 6 7 8
List of Figures True distributions used in simulation study 4.1 . . . . . . . . . . . . . . . . . . . . . 30 Frequentist model-based clustering results implemented via the EM algorithm using the Mclust function Posterior probability for ranking and clustering in study of section 4.1 with entry (i, j) in (a) being the l True (solid lines) and estimated (dashed lines) densities from SO-LCM and DPM for distribution T1 33 Posterior probability for ranking and clustering in study of section 4.2 . . . . . . . . 34 CDF plot for weighted latent morbidity score regarding different responses . . . . . . 35 Sorted Procedures Posterior Means with 95% Confidence Intervals . . . . . . . . . . 36 Ranking and Clustering for selected 66 procedures with observations larger than 200 37
29
0.45 T1
0.4
T2
T3 T4
0.35 0.3 0.25 0.2 0.15 0.1 0.05 0
−6
−4
−2
0
2
4
Figure 1: True distributions used in simulation study 4.1
30
6
−1300 −1400 −1500 −1600
BIC
−1700
EII EEV 2
4
6
EEI EEE
8
number of components
Figure 2: Frequentist model-based clustering results implemented via the EM algorithm using the Mclust function in R in simulation study 4.1, with the different symbols representing different model assumptions. ”EII”: spherical, equal volume; ”EEI”: spherical, equal volume and shape; ”EEV”: spherical, equal volume but varying orientation; ”EEE”: ellipsoidal, equal volume, shape, and orientation refered to R package Mclust.
31
0.8
0.9 10
10
0.8 0.7
20
0.7 0.6
20
0.6 30
0.5 30
0.5
0.4
0.4 40
0.2
0.2
50
0.3
40
0.3
50 0.1
0.1 60
60 10
20
30
40
50
60
10
(a)
20
30
40
50
60
0
(b)
Figure 3: Posterior probability for ranking and clustering in study of section 4.1 with entry (i, j) in (a) being the lower triangular matrix identifying the probability for Pi ≺ Pi0 and in (b) the probability for Pi = Pi0 .
32
0.4 0.3 0.2 0.1 0
−6
−4
−2
0
2
4
6
0
2
4
6
(a)
0.4 0.3 0.2 0.1 0
−6
−4
−2
(b)
Figure 4: True (solid lines) and estimated (dashed lines) densities from SO-LCM and DPM for distribution T1
33
0.8 10
10
0.8
20
0.7 0.6
20 0.6
0.5
30
30
0.4
0.4 40
0.3
40
0.2
50
0.2
50
60
0
60
0.1 10
20
30
40
50
60
10
(a)
20
30
40
50
60
(b)
Figure 5: Posterior probability for ranking and clustering in study of section 4.2
34
0
CDF plot for weighted latent morbidity score regarding stroke
0.18
h(u)
0.17
0.058
0.16
0.056
h(u)
0.060
0.19
0.062
CDF plot for weighted latent morbidity score regarding renalfail
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
u
CDF plot for weighted latent morbidity score regarding heartblock
CDF plot for weighted latent morbidity score regarding ecmovad
1.0
0.065
h(u)
0.055
0.0355
0.060
0.0365
h(u)
0.0375
0.070
0.0385
u
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
u
CDF plot for weighted latent morbidity score regarding phrenicnerve
CDF plot for weighted latent morbidity score regarding unplannedreop
0.22
h(u)
0.19
0.20
0.060
0.21
h(u)
0.065
0.23
0.24
0.070
u
0.0
0.2
0.4
0.6
0.8
1.0
0.0
u
0.2
0.4
0.6
0.8
u
0.60
0.62
h(u)
0.64
0.66
0.68
CDF plot for weighted latent morbidity score regarding PLOS
0.0
0.2
0.4
0.6
0.8
1.0
u
Figure 6: CDF plot for weighted latent morbidity score regarding different responses
35
1.0
3
2
1
0
−1
−2 V6
V3
V1
V80
V82
V40
V11
V39
V54
V14
V65
V79
V24
V88
procedure sorted by posterior mean
Figure 7: Sorted Procedures Posterior Means with 95% Confidence Intervals
36
0.9 10
0.7
20
0.6
10
0.8
0.5
20
0.6 30
0.4
40
0.4
30
0.5
0.3
40
0.3 50
0.1
60 10
20
30
40
50
0.2
50
0.2
0.1
60
60
10
(a)
20
30
40
50
60
0
(b)
Figure 8: Ranking and Clustering for selected 66 procedures with observations larger than 200
37
