Bayesian methods for cluster randomized trials with continuous responses

Bayesian methods for cluster randomized trials extend the random-effects formulation by allowing both the use of external evidence on parameters and straightforward relaxation of the standard normality and constant variance assumptions. Care is required in specifying prior distributions on variance components, and a number of different options are explored with implied prior distributions for other parameters given in closed form. Markov chain Monte Carlo (MCMC) methods permit the fitting of very general models and the introduction of parameter uncertainty into power calculations. We illustrate these ideas using a published example in which general practices were randomized to intervention or control, and show that different choices of supposedly 'non-informative' prior distributions can have substantial influence on conclusions. We also illustrate the use of forward simulation methods in power calculations with uncertainty on multiple inputs. Bayesian methods have the potential to be very useful but guidance is required as to appropriate strategies for robust analysis. Our current experience leads us to recommend a standard 'non-informative' prior distribution for the within-cluster sampling variance, and an independent prior on the intraclass correlation coefficient (ICC). The latter may exploit background evidence or, as a reference analysis, be a uniform ICC or a 'uniform shrinkage' prior.     

Incomplete factorial designs for randomized clinical trials

Recently there has been increased interest in considering factorial designs for randomized clinical trials when one wishes to study two or more treatments. Such designs may offer impressive gains in efficiency compared with a series of trials studying one treatment at a time. This is especially true when the treatments do not interact with one another. If interactions are of special interest, factorial designs provide one sensible approach for studying them, but larger sample sizes would be required because tests for interactions have lower power than those for main effects. In trials designed to test putative agents for preventing cancer, interactions may be of less interest so that fractions of higher-order factorial designs might be appropriate. Sometimes it may not be reasonable, interesting, feasible, or ethical to study all treatment combinations required in a complete or fractional factorial design, yet one may want to preserve some of the factorial structure to increase efficiency and to aid understanding. For such situations, incomplete factorial designs are proposed. Although not all of the advantages of full factorial designs are preserved, such designs may provide reasonable compromises for certain situations.     

Novel methods for the analysis of stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.     

The design and analysis of paired cluster randomized trials: an application of meta-analysis techniques

The statistical issues in clinical trials where clusters, communities or groups rather than individuals are randomized are often not fully appreciated. In this paper we discuss the design and analysis of trials in which pairs of clusters are randomized in the context of one recent trial, the British Family Heart Study. Both sample size calculations and the analysis strategy need to take account of the between-cluster component of variance. The analysis can be considered as a random effects meta-analysis across cluster pairs, and can usefully be presented as such. Techniques developed in the context of meta-analysis can then be used in the analysis, for example using a profile likelihood method to derive a confidence interval for the overall treatment effect which takes into account the variability in the estimate of the between-cluster variance. The methods presented here are contrasted with previously published methods for cluster randomized trials. © 1997 by John Wiley & Sons, Ltd.     

Bayesian methods of analysis for cluster randomized trials with binary outcome data

We explore the potential of Bayesian hierarchical modelling for the analysis of cluster randomized trials with binary outcome data, and apply the methods to a trial randomized by general practice. An approximate relationship is derived between the intracluster correlation coefficient (ICC) and the between-cluster variance used in a hierarchical logistic regression model. By constructing an informative prior for the ICC on the basis of available information, we are thus able implicitly to specify an informative prior for the between-cluster variance. The approach also provides us with a credible interval for the ICC for binary outcome data. Several approaches to constructing informative priors from empirical ICC values are described. We investigate the sensitivity of results to the prior specified and find that the estimate of intervention effect changes very little in this data set, while its interval estimate is more sensitive. The Bayesian approach allows us to assume distributions other than normality for the random effects used to model the clustering. This enables us to gain insight into the robustness of our parameter estimates to the classical normality assumption. In a model with a more complex variance structure, Bayesian methods can provide credible intervals for a difference between two variance components, in order for example to investigate whether the effect of intervention varies across clusters. We compare our results with those obtained from classical estimation, discuss the relative merits of the Bayesian framework, and conclude that the flexibility of the Bayesian approach offers some substantial advantages, although selection of prior distributions is not straightforward.     

Bayesian methods of analysis for cluster randomized trials with count outcome data

Bayesian approaches to inference in cluster randomized trials have been investigated for normally distributed and binary outcome measures. However, relatively little attention has been paid to outcome measures which are counts of events. We discuss an extension of previously published Bayesian hierarchical models to count data, which usually can be assumed to be distributed according to a Poisson distribution. We develop two models, one based on the traditional rate ratio, and one based on the rate difference which may often be more intuitively interpreted for clinical trials, and is needed for economic evaluation of interventions. We examine the relationship between the intracluster correlation coefficient (ICC) and the between‐cluster variance for each of these two models. In practice, this allows one to use the previously published evidence on ICCs to derive an informative prior distribution which can then be used to increase the precision of the posterior distribution of the ICC. We demonstrate our models using a previously published trial assessing the effectiveness of an educational intervention and a prior distribution previously derived. We assess the robustness of the posterior distribution for effectiveness to departures from a normal distribution of the random effects. Copyright © 2009 John Wiley & Sons, Ltd.     

Three methods for analysing correlated ROC curves: a comparison in real data sets from multi-reader,multi-case studies with a factorial design

This paper compares three published methods for analysing multiple correlated ROC curves: a method using generalized estimating equations with marginal non-proportional ordinal regression models; a method using jackknifed pseudovalues of summary statistics; a method using a corrected F-test from analysis of variance of summary statistics. Use of these methods is illustrated through six real data examples from studies with the common factorial design, that is, multiple readers interpreting images obtained with each test modality on each study subject. The issue of the difference between typical summary statistics and summary statistics from typical ROC curves is explored. The examples also address similarities and differences among the analytical methods. In particular, while point estimates of differences between test modalities are similar, the standard errors of these differences do not agree for all three methods. A simulation study supports the standard errors provided by the generalized estimating equations with marginal non-proportional ordinal regression models.     

Intra‐cluster correlation selection for cluster randomized trials

In this paper, we give focus to cluster randomized trials, also known as group randomized trials, which randomize clusters, or groups, of subjects to different trial arms, such as intervention or control. Outcomes from subjects within the same cluster tend to exhibit an exchangeable correlation measured by the intra‐cluster correlation coefficient (ICC). Our primary interest is to test if the intervention has an impact on the marginal mean of an outcome. Using recently developed methods, we propose how to select a working ICC structure with the goal of choosing the structure that results in the smallest standard errors for regression parameter estimates and thus the greatest power for this test. Specifically, we utilize small‐sample corrections for the estimation of the covariance matrix of regression parameter estimates. This matrix is incorporated within correlation selection criteria proposed in the generalized estimating equations literature to choose one of multiple working ICC structures under consideration. We demonstrate the potential power and utility of this approach when used in cluster randomized trial settings via a simulation study and application example, and we discuss practical considerations for its use in practice. Copyright © 2016 John Wiley & Sons, Ltd.     

Sample size calculation for cluster randomized cross-over trials

The cluster randomized cross-over design has been proposed in particular because it prevents an imbalance that may bring into question the internal validity of parallel group cluster trials. We derived a sample size formula for continuous outcomes that takes into account both the intraclass correlation coefficient (representing the clustering effect) and the interperiod correlation (induced by the cross-over design).     

Allowing for imprecision of the intracluster correlation coefficient in the design of cluster randomized trials

The sample size required for a cluster randomized trial depends on the magnitude of the intracluster correlation coefficient (ICC). The usual sample size calculation makes no allowance for the fact that the ICC is not known precisely in advance. We develop methods which allow for the uncertainty in a previously observed ICC, using a variety of distributional assumptions. Distributions for the power are derived, reflecting this uncertainty. Further, the observed ICC in a future study will not equal its true value, and we consider the impact of this on power. We implement calculations within a Bayesian simulation approach, and provide one simplification that can be performed using simple simulation within spreadsheet software. In our examples, recognizing the uncertainty in a previous ICC estimate decreases expected power, especially when the power calculated naively from the ICC estimate is high. To protect against the possibility of low power, sample sizes may need to be very substantially increased. Recognizing the variability in the future observed ICC has little effect if prior uncertainty has already been taken into account. We show how our method can be extended to the case in which multiple prior ICC estimates are available. The methods presented in this paper can be used by applied researchers to protect against loss of power, or to choose a design which reduces the impact of uncertainty in the ICC.     

A comparison of confidence interval methods for the intraclass correlation coefficient in cluster randomized trials

This study compared different methods for assigning confidence intervals to the analysis of variance estimator of the intraclass correlation coefficient (rho). The context of the comparison was the use of rho to estimate the variance inflation factor when planning cluster randomized trials. The methods were compared using Monte Carlo simulations of unbalanced clustered data and data from a cluster randomized trial of an intervention to improve the management of asthma in a general practice setting. The coverage and precision of the intervals were compared for data with different numbers of clusters, mean numbers of subjects per cluster and underlying values of rho. The performance of the methods was also compared for data with Normal and non-Normally distributed cluster specific effects. Results of the simulations showed that methods based upon the variance ratio statistic provided greater coverage levels than those based upon large sample approximations to the standard error of rho. Searle's method provided close to nominal coverage for data with Normally distributed random effects. Adjusted versions of Searle's method to allow for lack of balance in the data generally did not improve upon it either in terms of coverage or precision. Analyses of the trial data, however, showed that limits provided by Thomas and Hultquist's method may differ from those of the other variance ratio statistic methods when the arithmetic mean differs markedly from the harmonic mean cluster size. The simulation results demonstrated that marked non-Normality in the cluster level random effects compromised the performance of all methods. Confidence intervals for the methods were generally wide relative to the underlying size of rho suggesting that there may be great uncertainty associated with sample size calculations for cluster trials where large clusters are randomized. Data from cluster based studies with sample sizes much larger than those typical of cluster randomized trials are required to estimate rho with a reasonable degree of precision.     

A comparison of methods to analyse continuous data from pseudo cluster randomized trials

A major methodological reason to use cluster randomization is to avoid the contamination that would arise in an individually randomized design. However, when patient recruitment cannot be completed before randomization of clusters, the non-blindedness of recruiters and patients may cause selection bias, while in the control clusters, it may slow recruitment due to patient or recruiter preferences for the intervention. As a compromise, pseudo cluster randomization has been proposed. Because no insight is available into the relative performance of methods to analyse data obtained from this design, we compared the type I and II error rates of mixed models, generalized estimating equations (GEE) and a paired t-test to those of the estimator originally proposed in this design. The bias in the point estimate and its standard error were also incorporated into this comparison. Furthermore, we evaluated the effect of the weighting scheme and the accuracy of the sample size formula that have been described previously. Power levels of the originally proposed estimator and the unweighted mixed models were in agreement with the sample size formula, but the power of paired t-test fell short. GEE produced too large type I errors, unless the number of clusters was large (>30-40 per arm). The use of the weighting scheme generally enhanced the power, but at the cost of increasing the type I error in mixed models and GEE. We recommend unweighted mixed models as the best compromise between feasibility and power to analyse data from a pseudo cluster randomized trial.     

Extending the CONSORT statement to cluster randomized trials: for discussion

The need for clear reporting of randomized controlled trials has been emphasized recently. The CONSORT Statement has made evidence-based suggestions for a checklist and a patient flow diagram. Adapting this for cluster randomized controlled trials presents particular challenges. Simple changes in the checklist and diagram for the completely randomized two level cluster randomized trials are suggested for discussion. An example taken from an unpublished trial demonstrates that these changes are less simple to implement, although extensions to electronic publications may be helpful. These suggestions should be formally evaluated. Further work is required to consider the cases of more levels and of stratified or pair-matched cluster randomized trials.     

Analysis of cluster randomized trials in primary care: a practical approach

BACKGROUND: Cluster randomized trials increasingly are being used in health services research and in primary care, yet the majority of these trials do not account appropriately for the clustering in their analysis. OBJECTIVES: We review the main implications of adopting a cluster randomized design in primary care and highlight the practical application of appropriate analytical techniques. METHODS: The application of different analytical techniques is demonstrated through the use of empirical data from a primary care-based case study. CONCLUSION: Inappropriate analysis of cluster trials can lead to the presentation of inaccurate results and hence potentially misleading conclusions. We have demonstrated that adjustment for clustering can be applied to real-life data and we encourage more routine adoption of appropriate analytical techniques.     

The randomized placebo-phase design for clinical trials

Randomized controlled trials are the criterion standard method for evaluating the effectiveness of medical treatments. There are situations, however, where the possibility of being in the control group in a randomized controlled trial is unacceptable to potential subjects or their physicians. This lack of acceptance is a reason for poor accrual. We developed and validated a new clinical trial design for survival data that may allay concerns about not receiving an investigational product and should be more acceptable. Called the randomized placebo-phase design, this new design asks whether, on average, those subjects who begin active treatment sooner respond sooner than those who begin it later. Using Monte Carlo computer simulations, we demonstrated that the design is valid and may offer advantages over traditional randomized controlled trials in some situations. The randomized placebo-phase design may be especially useful when highly potent therapies for rare diseases are tested or when accrual may be otherwise difficult.     

Bayesian methods for analysis of binary outcome data in cluster randomized trials on the absolute risk scale

A Bayesian hierarchical modelling approach to the analysis of cluster randomized trials has advantages in terms of allowing for full parameter uncertainty, flexible modelling of covariates and variance structure, and use of prior information. Previously, such modelling of binary outcome data required use of a log-odds ratio scale for the treatment effect estimate and an approximation linking the intracluster correlation (ICC) to the between-cluster variance on a log-odds scale. In this paper we develop this method to allow estimation on the absolute risk scale, which facilitates clinical interpretation of both the treatment effect and the between-cluster variance. We describe a range of models and apply them to data from a trial of different interventions to promote secondary prevention of coronary heart disease in primary care. We demonstrate how these models can be used to incorporate prior data about typical ICCs, to derive a posterior distribution for the number needed to treat, and to consider both cluster and individual level covariates. Using these methods, we can benefit from the advantages of Bayesian modelling of binary outcome data at the same time as providing results on a clinically interpretable scale.     

Issues in the meta-analysis of cluster randomized trials

Meta-analyses involving the synthesis of evidence from cluster randomization trials are being increasingly reported. These analyses raise challenging methodologic issues beyond those raised by meta-analyses which include only individually randomized trials. In this paper we review and comment on a selected number of these issues, including problems of study heterogeneity, difficulties in estimating design effects from individual trials and the choice of statistical methods.     

Analysis of counts for cluster randomized trials: Negative controls and test-negative designs

In cluster randomized trials (CRTs), the outcome of interest is often a count at the cluster level. This occurs, for example, in evaluating an intervention with the outcome being the number of infections of a disease such as HIV or dengue or the number of hospitalizations in the cluster. Standard practice analyzes these counts through cluster outcome rates using an appropriate denominator (eg, population size). However, such denominators are sometimes unknown, particularly when the counts depend on a passive community surveillance system. We consider direct comparison of the counts without knowledge of denominators, relying on randomization to balance denominators. We also focus on permutation tests to allow for small numbers of randomized clusters. However, such approaches are subject to bias when there is differential ascertainment of counts across arms, a situation that may occur in CRTs that cannot implement blinded interventions. We suggest the use of negative control counts as a method to remove, or reduce, this bias, discussing the key properties necessary for an effective negative control. A current example of such a design is the recent extension of test-negative designs to CRTs testing community-level interventions. Via simulation, we compare the performance of new and standard estimators based on CRTs with negative controls to approaches that only use the original counts. When there is no differential ascertainment by intervention arm, the count-only approaches perform comparably to those using debiasing negative controls. However, under even modest differential ascertainment, the count-only estimators are no longer reliable.     

Performance of analytical methods for overdispersed counts in cluster randomized trials: Sample size,degree of clustering and imbalance

Many different methods have been proposed for the analysis of cluster randomized trials (CRTs) over the last 30 years. However, the evaluation of methods on overdispersed count data has been based mostly on the comparison of results using empiric data; i.e. when the true model parameters are not known. In this study, we assess via simulation the performance of five methods for the analysis of counts in situations similar to real community‐intervention trials. We used the negative binomial distribution to simulate overdispersed counts of CRTs with two study arms, allowing the period of time under observation to vary among individuals. We assessed different sample sizes, degrees of clustering and degrees of cluster‐size imbalance. The compared methods are: (i) the two‐sample t‐test of cluster‐level rates, (ii) generalized estimating equations (GEE) with empirical covariance estimators, (iii) GEE with model‐based covariance estimators, (iv) generalized linear mixed models (GLMM) and (v) Bayesian hierarchical models (Bayes‐HM). Variation in sample size and clustering led to differences between the methods in terms of coverage, significance, power and random‐effects estimation. GLMM and Bayes‐HM performed better in general with Bayes‐HM producing less dispersed results for random‐effects estimates although upward biased when clustering was low. GEE showed higher power but anticonservative coverage and elevated type I error rates. Imbalance affected the overall performance of the cluster‐level t‐test and the GEE's coverage in small samples. Important effects arising from accounting for overdispersion are illustrated through the analysis of a community‐intervention trial on Solar Water Disinfection in rural Bolivia. Copyright © 2009 John Wiley & Sons, Ltd.