Comparison between Bayesian approach and ...

Journal of Statistical Computation and Simulation

ISSN: 0094-9655 (Print) 1563-5163 (Online) Journal homepage: http://www.tandfonline.com/loi/gscs20

Comparison between Bayesian approach and frequentist methods for estimating relative risk in randomized controlled trials: a simulation study Leila Janani, Mohammad Ali Mansournia, Kazem Mohammad, Mahmood Mahmoodi, Kamran Mehrabani & Keramat Nourijelyani To cite this article: Leila Janani, Mohammad Ali Mansournia, Kazem Mohammad, Mahmood Mahmoodi, Kamran Mehrabani & Keramat Nourijelyani (2016): Comparison between Bayesian approach and frequentist methods for estimating relative risk in randomized controlled trials: a simulation study, Journal of Statistical Computation and Simulation, DOI: 10.1080/00949655.2016.1222610 To link to this article: http://dx.doi.org/10.1080/00949655.2016.1222610

Published online: 30 Aug 2016.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=gscs20 Download by: [Leila Janani]

Date: 01 September 2016, At: 04:35

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION, 2016 http://dx.doi.org/10.1080/00949655.2016.1222610

Comparison between Bayesian approach and frequentist methods for estimating relative risk in randomized controlled trials: a simulation study Leila Janania , Mohammad Ali Mansourniab , Kazem Mohammadb , Mahmood Mahmoodib , Kamran Mehrabanib and Keramat Nourijelyanib a Department of Biostatistics, School of Public Health, Iran University of Medical Sciences, Tehran, Iran; b Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran

ABSTRACT

ARTICLE HISTORY

Relative risks (RRs) are often considered as preferred measures of association in randomized controlled trials especially when the binary outcome of interest is common. To directly estimate RRs, log-binomial regression has been recommended. Although log-binomial regression is a special case of generalized linear models, it does not respect the natural parameter constraints, and maximum likelihood estimation is often subject to numerical instability that leads to convergence problems. Alternative methods for solving log-binomial regression convergence problems have been proposed. A Bayesian approach also was introduced, but the comparison between this method and frequentist methods has not been fully explored. We compared five frequentist and one Bayesian methods for estimating RRs under a variety of scenario. Based on our simulation study, there is not a method that can perform well based on different statistical properties, but COPY 1000 and modified log-Poisson regression can be considered in practice.

Received 21 April 2015 Accepted 7 August 2016 KEYWORDS

Binary outcome; relative risk; log-binomial regression; Bayesian approach; randomized controlled trials; simulation

1. Introduction Randomized controlled trials (RCTs) are considered as the gold standard for the assessment of the effect of an intervention.[1] In RCTs, binary outcomes are common and may relate to efficacy or safety parameters.[2] The most popular regression model for binary outcomes is logistic regression which expresses the estimate of the effect of treatment on the outcome as an odds ratio.[3] As a measure of effect odds ratio is difficult to interpret [4–6] and it is not collapsible;[7,8] this means that the size of the odds ratio will change if adjustment is made for a variable that is not a confounder. The argument in most of the literature has reached a consensus that the relative risk is preferred over the odds ratio for prospective studies.[5,9–11] Frequently, however, in prospective studies where relative risks (RRs) are the parameters of primary interest, odds ratios (ORs) are reported instead.[12] When the prevalence of the outcome is low, the ORs approximate the relative risk well [10] but, in spite of emphasis on the importance of the low prevalence assumption, general audience often interpret the odds ratio as a relative risk even in studies with common outcomes, see, e.g. Schwartz et al. [13]. To estimate RRs directly, log-binomial regression has been recommended.[14] However, as commonly known, convergence problems may arise with log-binomial regression.[9,11,14,15] Although CONTACT Keramat Nourijelyani [email protected] Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, PO Box 14155-6446, Tehran, Iran © 2016 Informa UK Limited, trading as Taylor & Francis Group

2

L. JANANI ET AL.

log-binomial regression is a member of the class of the generalized linear models (GLM) with a logarithmic link function and binomial response, the parameters in the log-binomial regression have to satisfy certain restrictions to ensure that the probability of outcome lies between 0 and 1. When we use the maximum likelihood method to fit a log-binomial regression, this restriction causes convergence problems when the maximum likelihood estimate (MLE) is close to the boundary of the parameter space.[16] Alternative methods for solving log-binomial regression convergence problems have been introduced.[3,9,17–21] A comparison of these methods in the context of RCTs has been reported by Yelland et al.[2] Bayesian methods provide a straightforward approach for the necessary inequality constraint with proper restriction on the support of prior and posterior distributions.[22,23] As a result of major progresses in computational methods, the use of Bayesian approaches is rapidly expanding in epidemiology and other applied fields.[24–28] Chu and Cole [29] introduced the Bayesian approach to estimate log-binomial regression parameters.[29] They evaluated the performance of their proposed method in comparison to a log-binomial regression and a modified log-Poisson model in a limited simulation study. They considered a cohort study setting and only used one continuous covariate in the model and also their comparisons were limited to only 12 scenarios. Their conclusion was that the Bayesian method may experience slight bias but generally has smaller mean-square error. The aim of this paper is to compare the relative performance of frequentist methods and Bayesian approach for estimating RRs in the context of RCTs with independent observations in a large simulation study.

2. Methods and materials 2.1. Setting and notation We consider a two-group parallel RCT design comparing a new treatment to a control. Assume N independent subjects randomly allocated to the ‘treatment’ or ‘control’ group. Let Yi denote a binary outcome for subject i (i = 1, . . . , N) and π i = P(Yi = 1|Xi ) be the probability of success, where X i = (X0 , X1i , X2i . . . , XKi ) is a (K+1) × 1 vector of covariates. We also assume that X 0 = 1, X 1i = 1 or 0 indicating treatment or control group, and X2i . . . , Xki be k−1 binary or continuous baseline covariates. The purpose of the analysis is to estimate the relative risk of success comparing treatment to control conditional on the pre-specified baseline covariates. Here, we consider six different methods for analysing such data: log-binomial regression, modified log-Poisson regression, log-normal regression, expanded logistic regression, COPY 1000 method and Bayesian log-binomial regression. These methods are described below. 2.2. Log-binomial regression The log-binomial regression model is a member of the class of the GLM with a log link, written as log(πi ) = X i β,

(1)

where β is a vector of model parameters and the responses Yi ’s are assumed to be independent and ˆ one can follow a Bernoulli distribution with parameter π i . Given the model parameter estimates β, easily show that the relative risk comparing treatment to control, adjusted for baseline covariates can be estimated by exp(βˆ1 ) where βˆ1 is the coefficient of the treatment indicator X 1i . This model has to satisfy certain restrictions to ensure that the predicted probabilities πˆ i = ˆ must lie between 0 and 1(0 ≤ πi ≤ 1) implying that X i βˆ must not exceed zero for any exp(X i β) X i , i = 1, . . . , N. The model may fail to converge as a result of these restrictions placed on the parameter space. Choosing different starting values for the parameter estimates may help to overcome convergence problems in some circumstances. However, if the MLE lies on the boundary of

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION

3

the parameter space then convergence will not occur.[19,30] The convergence problems with the log-binomial regression model have provided the development of alternative methods for estimating RRs.

2.3. Alternative methods 2.3.1. COPY method Deddens et al. [19] proposed an algorithm for obtaining solution close to the MLE. They suggested taking C-1 copies of the original data and 1 copy of the original dataset with the outcomes reversed (Y set to 1−Y) and fitting the log-binomial regression model to these modified data. They stated that for any finite C, the estimate is no longer on the boundary, and thus the estimate is a MLE for the new dataset. Lumley et al. [3] indicated that this is equivalent to forming a new data set which consists of one copy of the original data set with weight w = (c−1)/c and one copy of the original data set with the outcome values interchanged with weight 1−w = 1/c. Assuming independence between the new observations and the original data set and forming a weighted Bernoulli likelihood, Savu et al. [31] established that a unique maximum exists for the weighted likelihood because the weights are positive for both the original Yi and the additional (1−Yi ). Deddens et al. [19] suggested using C = 1000 in practice and called this the ‘COPY 1000 method’. To correct for the additional data in the modified dataset, the standard errors of the parameter estimates have to be multiplied by the square root of C. 2.3.2. Expanded logistic regression Schouten et al. [18] suggested duplicating each observation that has Y = 1, setting Y = 0 for the duplicate. The probability of success in the original dataset then equals the odds of success in the expanded dataset so that a logistic regression model fitting to the new dataset results in consistent estimates of the parameters in the log-binomial regression model. The standard errors will be incorrect, however and clustered robust standard errors are needed to account for the within-subject correlation from the duplicated observations. 2.3.3. Modified log-Poisson regression McNutt et al. [11] suggested using log-Poisson regression to estimate relative risk directly. Like logbinomial regression, this involves fitting a generalize linear model with a log link; however, the responses are assumed to follow a Poisson distribution. When used to estimate RRs from binary data, log-Poisson regression gives standard errors that are too large, because the variance of a Poisson random variable is always larger than that of a binary variable with the same mean. Zou [20] and Carter et al. [32] suggested using the model-robust sandwich estimator to correct the standard errors. 2.3.4. Log-normal regression Lumley et al. [3] proposed using a nonlinear least-squares estimation which corresponds to fitting model (1) and assuming a normal error distribution. As with the log-Poisson regression model, using a variance function other than the binomial results in biased standard errors and corrected standard errors can be obtained from a robust ‘sandwich’ variance estimator or from a jackknife or bootstrap. 2.3.5. Bayesian log-binomial regression Chu and Cole [29] proposed a novel Bayesian approach using a Markov-chain Monte-Carlo method, with a focus on implementing the linear inequality constraint for estimation of relative risk from a log-binomial model. Let β = (β0 , β1 , β2 . . . , βk ) and f (β) be the prior distributions for β in the

4

L. JANANI ET AL.

log-binomial model; the full posterior distribution of β given the data was defined f (β|Data) =

 i

×

{eyi (β0 +β1 x1i +β2 x2i +···+βk xki ) [1 − e(β0 +β1 x1i +β2 x2i +···+βk xki ) ]1−yi }



I(β0 + β1 x1i + β2 x2i + · · · + βk xki ≤ 0) × f (β),

(2)

i

which I(·) is the indicator function. They used a flat or constant prior for the model parameters, and fitted the model using WinBUGS.[33] 2.4. Simulations The performance of the mentioned six methods for estimating RRs was studied by simulation. A single binary, a single continuous or both a binary and continuous baseline covariates were considered. We conducted simulations for N = 200 or N = 500 with 1000 simulation replications performed for each scenario. Half of the subjects were assigned to the treatment group, while the other half were assigned to the control group. For the scenarios with single baseline covariate, the prevalence of the binary covariate was 0.5 or 0.75 and the continuous covariate had normal distribution with mean 0.5 and variance 0.05 or 0.25. For the two covariate scenarios, the prevalence of the binary covariate was 0.5 and the continuous covariate was normally distributed with mean 0.5 and variance 0.25. The baseline risk was considered 0.1 (β0 = −2.30) for all scenarios. For the scenarios with single baseline covariate, the following combinations of treatment and covariate effects were considered: a treatment relative risk of 1, 1.5, 2 or 3 (β1 = 0, 0.41, 0.69 or 1.10) with a covariate relative risk of 1, 1.5, 2 or 3 (β2 = 0, 0.41, 0.69 or 1.10). For scenarios with two covariates, the treatment and covariate RRs were all 1 or 2 (i.e. βk = 0 or 0.69 for k = 1, 2, 3), with at least one of the covariates having an effect for all scenarios. An outcome with probability πi = exp(X i β) was randomly assigned to each subject. If π i exceeded 1 for given values of the treatment indicator and baseline covariate(s), a new value was generated for the continuous covariate until π i < 1, effectively truncating the distribution of the continuous covariate. This has been used in previous simulation studies which compared different methods for estimating RRs.[2] We considered a total of 140 scenarios including a single binary (64 scenarios), a single continuous (64 scenarios) or both a binary and continuous baseline covariate (12 scenarios). For each simulated dataset, analyses were performed using each of the six methods described in Sections 2.2 and 2.3. We performed all analyses using R 3.0.1.[34] We used the glm function for performing log-binomial regression. Expanded logistic regression and modified log-Poisson regression were fitted using the geeglm function of R package geepack.[35–37] Log-normal regression was performed using the geese function of R package geepack and the COPY 1000 method was performed using the relRisk function of R package geepack. We used R package R2OpenBUGS [38] to fit Bayesian log-binomial regression. A flat prior was considered for each parameter. Specifically, the priors for the parameters were assumed to be N(0,103 ). One Markov chain, 20,000 iterations with length of burn-in equal to 10,000 was performed for each simulation. We used the default thin-in value which is equal to 1. The methods for estimating RRs were compared based on the following properties, convergence rate, median per cent relative bias which is defined as  median [100(R R − RR)/RR], the root-mean-square error (RMSE), the coverage probabilities of the 95% confidence intervals for β 1 and the median width of 95% confidence intervals for relative risk. We used 95% Wald confidence intervals for frequentist methods and 95% equal-tail posterior intervals for the Bayesian method. We also used median (range) for summarizing the simulation results.

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION

5

2.5. Computational resources In order to reduce the computational time involved, we used two Intel(R) Core (TM i7-4770 CPU@ 340 GHz processor with a 64-bit operating system, and 16.0GB RAM.

3. Simulation results 3.1. Convergence Convergence problems were observed only for log-binomial regression when adjusted analyses were performed. Log-binomial regression had convergence problems for 58.6% of the scenarios and median convergence rate was 88.0% for scenarios with convergence problems (Table 1). For scenarios with single binary covariate, we only observed convergence problems when the effect of treatment and covariate were considered strong (β1 and β2 = 1.10). Convergence problems occurred for all scenarios with single continuous covariate and two covariates in the model and very low convergence rate (0.30%) was observed when both treatment and continuous covariate effect were considered strong (β1 and β2 = 1.10). 3.2. Bias In all scenarios median per cent relative bias for log-binomial regression could be as large as (−16.84%). For scenarios without any convergence problems, log-binomial regression performed well with median per cent relative bias equal to 0.01% and range equal to (−2.82%, 1.34%). Underestimation and overestimation of the relative risk can occur for all methods. Bayesian log-binomial regression overestimated the relative risk with median per cent relative bias as large as 4.27%; however per cent relative bias did not exceed 3.26% for any other methods. Overall, Bayesian log-binomial regression had the poorest performance and there was not any substantial difference between the other methods (Table 2). 3.3. Root-mean-square error Overall, the COPY 1000 method had the lowest RMSE (median = 0.242) and log-normal regression had the highest one (median = 0.25). When log-binomial regression converged, log-binomial Table 1. Number (%) of simulation scenarios where the model had convergence problems and median (range) convergence rate for these scenarios. N (%)

Method Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression

Median (range) 88.0 (0.3a , 99.9) – – – – –

82 (56.8%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%)

a Very low convergence rate occurred when the treatment and continuous covariate effect were considered strong (RR

= 3).

Table 2. Median (range) for the median per cent relative bias of the estimated relative risk. Method

Total (140 scenarios)

Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression

−0.39 (−16.84, 3.18) −0.26 (−2.56, 2.57) 0.21 (−2.02, 3.26) −0.13 (−2.53, 2.35) −0.54 (−3.09, 2.01) 0.51 (−3.29, 4.27)

Log-binomial converged (58 scenarios) 0.01 (−2.82, −0.05 (−2.54, 0.36 (−1.94, 0.01 (−2.53, −0.22 (−3.09, 0.85 (−0.86,

1.34) 1.54) 2.09) 1.77) 1.15) 3.14)

Log-binomial not converged (82 scenarios) – −0.30 (−2.56, 0.08 (−2.02, −0.27 (−2.46, −0.67 (−2.28, 0.20 (−3.29,

2.57) 3.26) 2.35) 2.01) 4.27)

6

L. JANANI ET AL.

Table 3. Median (range) for the RMSE of the estimated slope. Method Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression

Total (140 scenarios) 0.24 (0.09, 0.25 (0.11, 0.25 (0.11, 0.25 (0.11, 0.24 (0.11, 0.25 (0.11,

1.44) 1.47) 0.90) 1.45) 0.48) 0.98)

Log-binomial converged (58 scenarios) 0.24 (0.12, 0.25 (0.12, 0.25 (0.13, 0.25 (0.13, 0.24 (0.12, 0.25 (0.12,

0.47) 0.47) 0.51) 0.47) 0.47) 0.51)

Log-binomial not converged (82 scenarios) – 0.25 (0.11, 0.25 (0.11, 0.25 (0.11, 0.24 (0.11, 0.24 (0.11,

1.47) 0.90) 1.45) 0.48) 0.98)

regression and the COPY 1000 method had the lowest RMSE (median = 0.24). In contrast, when the log-binomial regression failed to converge, Bayesian log-binomial regression and the COPY 1000 method had the lowest RMSE (median = 0.24) (Table 3). 3.4. Coverage Observed coverage rates for log-binomial regression ranged between 66.67% and 100.0%, for total scenarios, but for scenarios without convergence problems this method performed well and had coverage rates between 93.70% and 97.10%. The undercoverage rates were substantial for the COPY 1000 method and Bayesian method (90.60% and 92.10%, respectively) (Table 4). Log-normal regression and modified log-Poisson regression had good performance based on coverage rate only failing to maintain acceptable coverage rate for 3.6% and 5.7% of scenarios, respectively. 3.5. Precision The COPY 1000 method showed best performance both overall and by convergence status of log-binomial regression based on median confidence interval widths for relative risk, and median confidence interval widths did not exceed above 4.18 for this method. Precision was generally poor for Bayesian log-binomial regression compared to the other methods and the maximum median confidence interval width for this method was 4.62 (Table 5). Table 4. Median (range) for the coverage rate based on the 95% two-sided confidence interval for the relative risk. Method

Total (140 scenarios)

Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression

95.57 (66.67, 95.50 (93.40, 95.30 (93.00, 95.50 (93.40, 95.50 (90.60, 94.70 (92.10,

100.00) 97.10) 96.80) 97.20) 97.00) 96.70)

Log-binomial converged (58 scenarios) 95.30 (93.70, 95.30 (93.40, 95.25 (93.00, 95.30 (93.40, 95.35 (93.80, 94.55 (92.10,

97.10) 97.50) 96.80) 96.40) 97.00) 96.70)

Log-binomial not converged (82 scenarios) – 95.60 (94.10, 95.30 (94.10, 95.60 (94.20, 95.60 (90.60, 94.90 (93.20,

97.10) 96.40) 97.20) 96.60) 96.50)

Table 5. Median (range) for median confidence interval width for relative risk, both overall and by convergence status of the logbinomial regression model. Method Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression

Total (140 Scenarios) 1.60 (0.60, 1.61 (0.60, 1.64 (0.61, 1.61 (0.60, 1.59 (0.60, 1.63 (0.60,

4.21) 4.21) 4.40) 4.21) 4.18) 4.62)

Log-binomial converged (58 scenarios) 1.58 (0.60, 1.58 (0.60, 1.61 (0.61, 1.59 (0.60, 1.57 (0.60, 1.62 (0.60,

4.21) 4.21) 4.40) 4.21) 4.18) 4.62)

Log-binomial not converged (82 scenarios) – 1.64 (0.64, 1.65 (0.64, 1.68 (0.65, 1.59 (0.61, 1.64 (0.61,

4.19) 4.32) 4.19) 4.16) 4.60)

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION

7

3.6. Binary versus continuous covariate There were some differences in results based on whether a binary or continuous covariate was adjusted in the model. The convergence problem occurred only in 6 out of 64 scenarios with the binary covariate, but in all scenarios with the continuous covariate there were convergence problems. For all methods, RMSE and median confidence interval width were smaller for the binary covariate compared to the continuous covariate. For all methods, median per cent relative bias and coverage problems could be smaller or larger for the continuous covariate compared to the binary covariate. 3.7. Sample size = 200 versus sample size = 500 For all methods, RMSE, median per cent relative bias and median confidence interval width were smaller for sample size 500 compared to sample size 200. There were not substantial differences based on the convergence and coverage problems in two sample sizes and this was for all the methods.

4. Illustrative example Data provided for this analysis were supported by funding from the National, Heart, Lung, and Blood Institute: HL075263. Electronic Communications and Home Blood Pressure Monitoring (e-BP) and was previously published in JAMA.[39] Seven hundred and seventy-eight participants aged 25–75 years with uncontrolled essential hypertension and Internet access were enrolled. Participants were randomly assigned to usual care, home BP monitoring and secure patient Web site training only or home BP monitoring and secure patient Web site training plus pharmacist care management delivered through Web communications. They used blocked randomization to assign patients to three groups. Primary outcomes were defined percentage of patients with controlled BP (140/90 mmHg) and changes in systolic and diastolic BP at 12 months. Of 778 patients, 730 (94%) completed the 12-month follow-up visit. We considered only two groups of this study; home BP monitoring and secure patient Web site training plus pharmacist care (231 individuals who completed the 12 months follow-up) versus usual care (234 individuals who completed the 12 months follow-up) and the binary primary outcome whether the patient blood pressure was controlled at 12 months visit. Adjustment was made for Body Mass Index, sex, already having a home BP monitor before the trial, baseline systolic BP, and clinic like the original study. Previously introduced six methods and also logistic regression were used for making adjustment for covariates. We used three chains and 100,000 iterations for the Bayesian model for this example. Because we run multiple MCMC chains in the real data example, we used Gelman and Rubin diagnostic to assess convergence. The median potential scale reduction factor for parameter β 1 (treatment effect) and also a multivariate potential scale reduction factor were close to 1. 4.1. Example results The percentage of patients with controlled BP was 53.7% and 28.2% for the intervention and usual care group, respectively, producing an unadjusted relative risk (95% confidence interval (CI)) of 1.90 (1.50, 2.41). When adjustment was made for covariates, the log-binomial regression method failed to converge. Adjusted RRs varied depending on the method ranging between 1.82 and 1.99 (Table 6). Based on the unadjusted and adjusted results, the new intervention; home BP monitoring and secure patient Web site training plus pharmacist care have strong effect on controlling blood pressure at 12 months follow-up. Because the outcome was common in this study, there is a very large difference between odds ratio produced by logistic regression and adjusted RRs and odds ratio overestimated the effect of treatment on binary outcome.

8

L. JANANI ET AL.

Table 6. Adjusted relative risk comparing intervention group with usual care for the outcome of controlled BP for all patients completing 12 months follow-up. Type of model Log-binomial regression Modified log-Poisson regression Log-normal regression Expanded logistic regression COPY 1000 Bayesian log-binomial regression Logistic regressiona

Adjustedb relative risk

95% Confidence limitsc

– 1.94 1.85 1.99 1.87 1.82 3.19

– (1.54, 2.45) (1.47, 2.33) (1.58, 2.52) (1.48, 2.36) (1.45, 2.28) (2.14, 4.78)

a Odds ratio reported for logistic regression. b Models adjusted for Body Mass Index, sex, already having a home BP monitor before the trial, baseline systolic BP and clinic. c Based on 95% Wald confidence intervals for frequentist methods and 95% equal-tail posterior intervals for the Bayesian

log-

binomial model.

5. Discussion We have compared five frequentist methods and one Bayesian approach for estimating relative risk in an RCT setting with independent observations. Convergence rates can be improved by employing one of the alternatives to log-binomial regression for relative risk estimation. The most convergence problems for log-binomial regression were observed when adjustment was made on a continuous covariate and rate of convergence can be so low when strong effects were considered for both treatment and the continuous covariate. Our simulation results show that there is variability between the methods and that the best method to use depends on the statistical property considered. This study was the first large simulation study that has compared frequentist and Bayesian approaches for estimating relative risk in RCT settings and also the first one that used R [34] software for running all methods. Chu and Cole [29] introduced the Bayesian approach and used WinBUGS [33] to fit the Bayesian log-binomial model. They compared the proposed method to modified logPoisson regression and log-binomial regression and showed that the resulting Bayesian method may incur slight bias but generally has a smaller mean-square error and is therefore more accurate than the existing frequentist methods. They used 10,000 iterations for both burn-in and inference for simulation studies and implemented 2000 replicates with a moderate sample size of 400 for each replicate. They used a parameter space based on a rectangular region for covariates defined by the extreme values of each covariate and considered flat priors for parameters. We implemented MarkovChain Monte-Carlo methods without such restrictions and used R package R2OpenBUGS [38] to fit Bayesian log-binomial regression and also considered flat priors for the parameters. We also used 10,000 iterations for both burn-in and inference for simulation studies, but considered 1000 replicates with a sample size of 200 or 500 for each replicate. Based on our results, Bayesian log-binomial regression had larger median per cent relative bias compared to the frequentist methods and only showed good performance based on the RMSE when log-binomial regression failed to converge. A large simulation study for comparing 10 frequentist methods were implemented by Yelland et al. [2] they concluded that modified log-Poisson regression, log-normal regression and expanded logistic regression can be considered as good alternative methods for log-binomial regression but because of additional advantages of modified log-Poisson regression such as simple to describe and implementation, it can be a good choice in practice. They observed convergence problems for the COPY 1000 method and so concluded that this method is not a suitable alternative method in practice and they observed convergence problem for log-binomial regression in 33.3% of scenarios. We did not observe any convergence problem for the COPY 1000 method and based on RMSE and precision this method was found to be the best one. Based on the median per cent relative bias and the coverage rate, modified log-Poisson regression performed better than the COPY 1000 method in our study. There are other studies which also reported no convergence problem for the COPY method.[19,31] Savu et al. [31] showed that fitting algorithms of log-binomial regression should converge towards

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION

9

the global maximum point of the likelihood, if started at initial values selected in a neighbourhood of the COPY modified data’s MLE. However, they mentioned that convergence problems can still occur with the COPY method, for example, if the algorithm generates an intermediate value outside the restricted parameter space. We used the relRisk function of R package geepack but Yelland et al.’s used PROC GENMOD in SAS software to perform the COPY 1000 method. We suggest that using different strategies to select starting points and using different numerical estimation methods by different software can lead to different reports for convergence of the COPY method. There were convergence problems for log-binomial regression in 56.8% of scenarios in our study, this high rate can be explained by stronger effects that we considered for treatment and covariate in the model. In the illustrated example in which the binary outcome was not rare, we observed that there was a large difference between odds ratio and relative risk as a measure of association. So in the context of clinical trial studies in which common binary outcome is popular, relative risk is a preferred measure of association. There are several limitations to our research that are worth being mentioned here. First, we only used one Markov chain, 20,000 iterations with length of burn-in equals to 10,000 and implemented the analysis using R2OpenBUGS [38] which applies adaptive rejection Metropolis sampling within the Gibbs sampling algorithm.[40–42] Recently, Zhou et al. [16] used Slice sampling [43] and found that it can improve the accuracy of Monte-Carlo estimates. Maybe running multiple chains with higher iterations and using Slice sampling can improve the Bayesian log-binomial model estimates. Second, we assumed that all data were available for the final analysis and hence missing data were not considered. Missing data are a common problem in RCTs, although analysis may still be based on complete data if imputation methods are used to fill in the missing values. Finally, there are some new alternative methods for log-binomial regression which we did not consider in this research. Fitzmaurice et al. [12] introduced an almost efficient estimation of relative risk regression which uses near-optimal weights based on a Maclaurin series (Taylor series expanded around zero) approximation to the true Bernoulli or binomial weight function and found that the new method can improve the precision. Lipsits et al. [44] proposed using the jackknife as a bias reduction approach for the COPY method. Further studies are needed to establish usefulness of Bayesian methods as an alternative method for log-binomial regression. Also further research is needed to understand performance of the Bayesian approach in comparison to frequentist methods in settings of RCT’s when the data are correlated, and this work is currently in progress. We also came across a problem in geepack [35–37] which needs to be considered by authors of the package. There is a problem using the geeglm function to fit the model by the log link function and normal distribution, so we used the geese function instead to fit log-normal regression. Start argument of the geeglm function did not work in this circumstance, but we could use b argument in the geese function and assign a vector of zeros as starting values and fit the log-normal model.

6. Conclusion In conclusion, log-binomial regression can be the most useful when the scientific goal is to estimate the association between an intervention and a commonly occurring binary outcome while controlling for additional covariates. This model may fail to converge specially when adjustment was made on continuous covariate and also treatment and covariate have a strong effect. Alternative methods for estimating RRs are required in these situations. Based on our simulation study, there is no method that can perform well based on different statistical properties, but COPY 1000 and modified log-Poisson regression can be considered in practice.

Acknowledgements The authors would like to thank to Lisa Yelland for her kind support and valuable help during the research and Beverly B. Green and her colleagues for letting us to use their clinical trial dataset.

10

L. JANANI ET AL.

Disclosure statement The authors declare that there is no conflict of interest.

Funding This work was financially supported by Tehran University of Medical Sciences (TUMS), Tehran, Iran.

References [1] Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Brit Med J. 1996;312(7023):71–72. [2] Yelland LN, Salter AB, Ryan P. Relative risk estimation in randomized controlled trials: a comparison of methods for independent observations. Int J Biostat. 2011;7:1–31. [3] Lumley T, Kronmal R, Ma S. Relative risk regression in medical research: models, contrasts, estimators, and algorithms. UW Biostatistics working paper series, Working paper 293; 2006. [4] Lee J. Odds ratio or relative risk for cross-sectional data? Int J Epidemiol. 1994;23(1):201–203. [5] Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol. 1994;47(8):881–889. [6] Walter SD. Choice of effect measure for epidemiological data. J Clin Epidemiol. 2000;53(9):931–939. [7] Miettinen OS, Cook EF. Confounding: essence and detection. Am J Epidemiol. 1981;114(4):593–603. [8] Cummings P. The relative merits of risk ratios and odds ratios. Arch Pediat Adol Med. 2009;163(5):438–445. [9] Wacholder S. Binomial regression in GLIM: estimating risk ratios and risk differences. Am J Epidemiol. 1986;123(1):174–184. [10] Greenland S. Interpretation and choice of effect measures in epidemiologic analyses. Am J Epidemiol. 1987;125(5):761–768. [11] McNutt L-A, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol. 2003;157(10):940–943. [12] Fitzmaurice GM, Lipsitz SR, Arriaga A, Sinha D, Greenberg C, Gawande AA. Almost efficient estimation of relative risk regression. Biostatistics. 2014;15(4):745–756. [13] Schwartz LM, Woloshin S, Welch HG. Misunderstandings about the effects of race and sex on physicians’ referrals for cardiac catheterization. New Engl J Med. 1999;341(4):279–283. [14] Deddens J, Petersen MR, Endahl L. Prevalence proportion ratios: estimation and hypothesis testing. Int J Epidemiol. 1998;27(1):91–95. [15] Wallenstein S, Bodian C. Inferences on odds ratios, relative risks, and risk differences based on standard regression programs. Am J Epidemiol. 1987;126(2):346–355. [16] Zhou R, Sivaganesan S, Longla M. An objective Bayesian estimation of parameters in a log-binomial model. J Statist Plann Inference. 2014;146:113–121. [17] Flanders WD, Rhodes PH. Large sample confidence intervals for regression standardized risks, risk ratios, and risk differences. J Chronic Dis. 1987;40(7):697–704. [18] Schouten EG, Dekker JM, Kok FJ, et al. Risk ratio and rate ratio estimation in case-cohort designs: Hypertension and cardiovascular mortality. Stat Med. 1993;12(18):1733–1745. [19] Deddens JA, Petersen MR, Lei X. Estimation of prevalence ratios when PROC GENMOD does not converge. In Proceedings of the 28th Annual SAS Users Group International Conference. Cary, NC: SAS Institute Inc. Paper 270–28; 2003. [20] Zou G. A modified Poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004;159(7):702–706. [21] Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007;60(9):874–882. [22] Gelfand AE, Smith AF, Lee T-M. Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J Am Statist Assoc. 1992;87(418):523–532. [23] Smith AF, Roberts GO. Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. J R Statist Soc B (Methodol.). 1993;55(1):3–23. [24] Gelfand AE, Smith AF. Sampling-based approaches to calculating marginal densities. J Am Statist Assoc. 1990;85(410):398–409. [25] Ashby D. Bayesian statistics in medicine: a 25 year review. Stat Med. 2006;25(21):3589–3631. [26] Greenland S. Bayesian perspectives for epidemiological research: I. Foundations and basic methods. Int J Epidemiol. 2006;35(3):765–775. [27] Greenland S. Bayesian perspectives for epidemiological research. II. Regression analysis. Int J Epidemiol. 2007;36(1):195–202.

JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION

11

[28] Gelman A, Carlin JB, Stern HS, Rubin DB. Bayesian data analysis. Texts in Statistical Science. 2nd ed. Boca Raton, FL: Chapman & Hall/CRC; 2004. [29] Chu H, Cole SR. Estimation of risk ratios in cohort studies with common outcomes: a Bayesian approach. Epidemiology. 2010;21(6):855–862. [30] Barros AJ, Hirakata VN. Alternatives for logistic regression in cross-sectional studies: an empirical comparison of models that directly estimate the prevalence ratio. BMC Med Res Methodol. 2003;3(1). Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-3-21 [31] Savu A, Liu Q, Yasui Y. Estimation of relative risk and prevalence ratio. Stat Med. 2010;29(22):2269–2281. [32] Carter RE, Lipsitz SR, Tilley BC. Quasi-likelihood estimation for relative risk regression models. Biostatistics. 2005;6(1):39–44. [33] Spiegelhalter D, Thomas A, Best NG, Lunn D. WinBUGS Version 1.4 User Manual. Cambridge: Medical Research Council Biostatistics Unit; 2004. [34] R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2014. [35] Halekoh U, Højsgaard S, Yan J. The R package geepack for generalized estimating equations. J Statist Softw. 2006;15(2):1–11. [36] Yan J, Fine J. Estimating equations for association structures. Stat Med. 2004;23(6):859–874. [37] Yan J. Geepack: yet another package for generalized estimating equations. R News. 2002;2(3):12–14. [38] Sturtz S, Ligges U, Gelman AE. R2WinBUGS: a package for running WinBUGS from R. J Statist Softw. 2005;12(3):1–16. [39] Green BB, Cook AJ, Ralston JD, et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. J Am Med Assoc. 2008;299(24):2857–2867. [40] Gilks WR, Best N, Tan K. Adaptive rejection Metropolis sampling within Gibbs sampling. J Appl Stat. 1995;44(4):455–472. [41] Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. Equation of state calculations by fast computing machines. J Chem Phys. 1953;21(6):1087–1092. [42] Gelfand AE, Hills SE, Racine-Poon A, Smith AFM, et al. Illustration of Bayesian inference in normal data models using Gibbs sampling. J Am Statist Assoc. 1990;85(412):972–985. [43] Neal RM. Slice sampling. Ann Stat. 2003;31(3):705–767. [44] Lipsitz SR, Fitzmaurice GM, Arriaga A, Sinha D, Gawande AA. Using the jackknife for estimation in log link Bernoulli regression models. Stat Med. 2014;34(3):444–453.

Appendix 1. R codes for running the models library (geepack) library(R2OpenBUGS) data=read.table ("") #Log-binomial regression: glm(outcome ∼ treatment+covariate, data = data, family=binomial(link="log")) #We need to specify a unique subject identifier (id) for performing the following models. #Modified log-Poisson regression: geeglm(outcome ∼ treatment+covariate, data = data, family=poisson(link="log"), id=id) #Log-normal regression: geese(outcome ∼ treatment+covariate, data = data, family=gaussian(link="log"), id=id, b=c(0,0,0)) #COPY 1000 method: relRisk(outcome ∼ treatment+covariate, data = data, id=id, ncopy=1000) #Preparing data for expanded logistic regression method: data2