Sample size estimation for stratified individual and cluster randomized trials with binary outcomes

Individual randomized trials (IRTs) and cluster randomized trials (CRTs) with binary outcomes arise in a variety of settings and are often analyzed by logistic regression (fitted using generalized estimating equations for CRTs). The effect of stratification on the required sample size is less well understood for trials with binary outcomes than for continuous outcomes. We propose easy-to-use methods for sample size estimation for stratified IRTs and CRTs and demonstrate the use of these methods for a tuberculosis prevention CRT currently being planned. For both IRTs and CRTs, we also identify the ratio of the sample size for a stratified trial vs a comparably powered unstratified trial, allowing investigators to evaluate how stratification will affect the required sample size when planning a trial. For CRTs, these can be used when the investigator has estimates of the within-stratum intracluster correlation coefficients (ICCs) or by assuming a common within-stratum ICC. Using these methods, we describe scenarios where stratification may have a practically important impact on the required sample size. We find that in the two-stratum case, for both IRTs and for CRTs with very small cluster sizes, there are unlikely to be plausible scenarios in which an important sample size reduction is achieved when the overall probability of a subject experiencing the event of interest is low. When the probability of events is not small, or when cluster sizes are large, however, there are scenarios where practically important reductions in sample size result from stratification.     

A comparison of confidence interval methods for the intraclass correlation coefficient in cluster randomized trials

This study compared different methods for assigning confidence intervals to the analysis of variance estimator of the intraclass correlation coefficient (rho). The context of the comparison was the use of rho to estimate the variance inflation factor when planning cluster randomized trials. The methods were compared using Monte Carlo simulations of unbalanced clustered data and data from a cluster randomized trial of an intervention to improve the management of asthma in a general practice setting. The coverage and precision of the intervals were compared for data with different numbers of clusters, mean numbers of subjects per cluster and underlying values of rho. The performance of the methods was also compared for data with Normal and non-Normally distributed cluster specific effects. Results of the simulations showed that methods based upon the variance ratio statistic provided greater coverage levels than those based upon large sample approximations to the standard error of rho. Searle's method provided close to nominal coverage for data with Normally distributed random effects. Adjusted versions of Searle's method to allow for lack of balance in the data generally did not improve upon it either in terms of coverage or precision. Analyses of the trial data, however, showed that limits provided by Thomas and Hultquist's method may differ from those of the other variance ratio statistic methods when the arithmetic mean differs markedly from the harmonic mean cluster size. The simulation results demonstrated that marked non-Normality in the cluster level random effects compromised the performance of all methods. Confidence intervals for the methods were generally wide relative to the underlying size of rho suggesting that there may be great uncertainty associated with sample size calculations for cluster trials where large clusters are randomized. Data from cluster based studies with sample sizes much larger than those typical of cluster randomized trials are required to estimate rho with a reasonable degree of precision.     

Sample size requirements to detect an intervention by time interaction in longitudinal cluster randomized clinical trials

In designing a longitudinal cluster randomized clinical trial (cluster‐RCT), the interventions are randomly assigned to clusters such as clinics. Subjects within the same clinic will receive the identical intervention. Each will be assessed repeatedly over the course of the study. A mixed‐effects linear regression model can be applied in a cluster‐RCT with three‐level data to test the hypothesis that the intervention groups differ in the course of outcome over time. Using a test statistic based on maximum likelihood estimates, we derived closed‐form formulae for statistical power to detect the intervention by time interaction and the sample size requirements for each level. Importantly, the sample size does not depend on correlations among second‐level data units and the statistical power function depends on the number of second‐ and third‐level data units through their product. A simulation study confirmed that theoretical power estimates based on the derived formulae are nearly identical to empirical estimates. Copyright © 2009 John Wiley & Sons, Ltd.     

Allowing for imprecision of the intracluster correlation coefficient in the design of cluster randomized trials

The sample size required for a cluster randomized trial depends on the magnitude of the intracluster correlation coefficient (ICC). The usual sample size calculation makes no allowance for the fact that the ICC is not known precisely in advance. We develop methods which allow for the uncertainty in a previously observed ICC, using a variety of distributional assumptions. Distributions for the power are derived, reflecting this uncertainty. Further, the observed ICC in a future study will not equal its true value, and we consider the impact of this on power. We implement calculations within a Bayesian simulation approach, and provide one simplification that can be performed using simple simulation within spreadsheet software. In our examples, recognizing the uncertainty in a previous ICC estimate decreases expected power, especially when the power calculated naively from the ICC estimate is high. To protect against the possibility of low power, sample sizes may need to be very substantially increased. Recognizing the variability in the future observed ICC has little effect if prior uncertainty has already been taken into account. We show how our method can be extended to the case in which multiple prior ICC estimates are available. The methods presented in this paper can be used by applied researchers to protect against loss of power, or to choose a design which reduces the impact of uncertainty in the ICC.     

Sample size adjustments for varying cluster sizes in cluster randomized trials with binary outcomes analyzed with second‐order PQL mixed logistic regression

Adjustments of sample size formulas are given for varying cluster sizes in cluster randomized trials with a binary outcome when testing the treatment effect with mixed effects logistic regression using second‐order penalized quasi‐likelihood estimation (PQL). Starting from first‐order marginal quasi‐likelihood (MQL) estimation of the treatment effect, the asymptotic relative efficiency of unequal versus equal cluster sizes is derived. A Monte Carlo simulation study shows this asymptotic relative efficiency to be rather accurate for realistic sample sizes, when employing second‐order PQL. An approximate, simpler formula is presented to estimate the efficiency loss due to varying cluster sizes when planning a trial. In many cases sampling 14 per cent more clusters is sufficient to repair the efficiency loss due to varying cluster sizes. Since current closed‐form formulas for sample size calculation are based on first‐order MQL, planning a trial also requires a conversion factor to obtain the variance of the second‐order PQL estimator. In a second Monte Carlo study, this conversion factor turned out to be 1.25 at most. Copyright © 2010 John Wiley & Sons, Ltd.     

Cluster size variability and imbalance in cluster randomized controlled trials

Cluster randomized controlled trials are increasingly used to evaluate medical interventions. Research has found that cluster size variability leads to a reduction in the overall effective sample size. Although reporting standards of cluster trials have started to evolve, a far greater degree of transparency is needed to ensure that robust evidence is presented. The use of the numbers of patients recruited to summarize recruitment rate should be avoided in favour of an improved metric that illustrates cumulative power and accounts for cluster variability. Data from four trials is included to show the link between cluster size variability and imbalance. Furthermore, using simulations it is demonstrated that by randomising using a two block randomization strategy and weighting the second by cluster size recruitment, chance imbalance can be minimized.     

Intention-to-treat analysis in cluster randomized trials with noncompliance

In cluster randomized trials (CRTs), individuals belonging to the same cluster are very likely to resemble one another, not only in terms of outcomes but also in terms of treatment compliance behavior. Although the impact of resemblance in outcomes is well acknowledged, little attention has been given to the possible impact of resemblance in compliance behavior. This study defines compliance intraclass correlation as the level of resemblance in compliance behavior among individuals within clusters. On the basis of Monte Carlo simulations, it is demonstrated how compliance intraclass correlation affects power to detect intention-to-treat (ITT) effect in the CRT setting. As a way of improving power to detect ITT effect in CRTs accompanied by noncompliance, this study employs an estimation method, where ITT effect estimates are obtained based on compliance-type-specific treatment effect estimates. A multilevel mixture analysis using an ML-EM estimation method is used for this estimation.     

Measures of between‐cluster variability in cluster randomized trials with binary outcomes

Cluster randomized trials (CRTs) are increasingly used to evaluate the effectiveness of health‐care interventions. A key feature of CRTs is that the observations on individuals within clusters are correlated as a result of between‐cluster variability. Sample size formulae exist which account for such correlations, but they make different assumptions regarding the between‐cluster variability in the intervention arm of a trial, resulting in different sample size estimates. We explore the relationship for binary outcome data between two common measures of between‐cluster variability: k, the coefficient of variation and ρ, the intracluster correlation coefficient. We then assess how the assumptions of constant k or ρ across treatment arms correspond to different assumptions about intervention effects. We assess implications for sample size estimation and present a simple solution to the problems outlined. Copyright © 2009 John Wiley & Sons, Ltd.     

Sample size calculation for cluster randomized cross-over trials

The cluster randomized cross-over design has been proposed in particular because it prevents an imbalance that may bring into question the internal validity of parallel group cluster trials. We derived a sample size formula for continuous outcomes that takes into account both the intraclass correlation coefficient (representing the clustering effect) and the interperiod correlation (induced by the cross-over design).     

Sample size calculation in cost‐effectiveness cluster randomized trials: optimal and maximin approaches

In this paper, the optimal sample sizes at the cluster and person levels for each of two treatment arms are obtained for cluster randomized trials where the cost‐effectiveness of treatments on a continuous scale is studied. The optimal sample sizes maximize the efficiency or power for a given budget or minimize the budget for a given efficiency or power. Optimal sample sizes require information on the intra‐cluster correlations (ICCs) for effects and costs, the correlations between costs and effects at individual and cluster levels, the ratio of the variance of effects translated into costs to the variance of the costs (the variance ratio), sampling and measuring costs, and the budget. When planning, a study information on the model parameters usually is not available. To overcome this local optimality problem, the current paper also presents maximin sample sizes. The maximin sample sizes turn out to be rather robust against misspecifying the correlation between costs and effects at the cluster and individual levels but may lose much efficiency when misspecifying the variance ratio. The robustness of the maximin sample sizes against misspecifying the ICCs depends on the variance ratio. The maximin sample sizes are robust under misspecification of the ICC for costs for realistic values of the variance ratio greater than one but not robust under misspecification of the ICC for effects. Finally, we show how to calculate optimal or maximin sample sizes that yield sufficient power for a test on the cost‐effectiveness of an intervention. Copyright © 2014 John Wiley & Sons, Ltd.     

Relative efficiency of unequal versus equal cluster sizes in cluster randomized and multicentre trials

Cluster randomized and multicentre trials evaluate the effect of a treatment on persons nested within clusters, for instance, patients within clinics or pupils within schools. Optimal sample sizes at the cluster (centre) and person level have been derived under the restrictive assumption of equal sample sizes per cluster. This paper addresses the relative efficiency of unequal versus equal cluster sizes in case of cluster randomization and person randomization within clusters. Starting from maximum likelihood parameter estimation, the relative efficiency is investigated numerically for a range of cluster size distributions. An approximate formula is presented for computing the relative efficiency as a function of the mean and variance of cluster size and the intraclass correlation, which can be used for adjusting the sample size. The accuracy of this formula is checked against the numerical results and found to be quite good. It is concluded that the loss of efficiency due to variation of cluster sizes rarely exceeds 10 per cent and can be compensated by sampling 11 per cent more clusters.     

Sample size estimation in educational intervention trials with subgroup heterogeneity in only one arm

We present closed form sample size and power formulas motivated by the study of a psycho‐social intervention in which the experimental group has the intervention delivered in teaching subgroups whereas the control group receives usual care. This situation is different from the usual clustered randomized trial because subgroup heterogeneity only exists in one arm. We take this modification into consideration and present formulas for the situation in which we compare a continuous outcome at both a single point in time and longitudinally over time. In addition, we present the optimal combination of parameters such as the number of subgroups and number of time points for minimizing sample size and maximizing power subject to constraints such as the maximum number of measurements that can be taken (i.e., a proxy for cost). Copyright © 2012 John Wiley & Sons, Ltd.     

Sample size and robust marginal methods for cluster‐randomized trials with censored event times

In cluster‐randomized trials, intervention effects are often formulated by specifying marginal models, fitting them under a working independence assumption, and using robust variance estimates to address the association in the responses within clusters. We develop sample size criteria within this framework, with analyses based on semiparametric Cox regression models fitted with event times subject to right censoring. At the design stage, copula models are specified to enable derivation of the asymptotic variance of estimators from a marginal Cox regression model and to compute the number of clusters necessary to satisfy power requirements. Simulation studies demonstrate the validity of the sample size formula in finite samples for a range of cluster sizes, censoring rates, and degrees of within‐cluster association among event times. The power and relative efficiency implications of copula misspecification is studied, as well as the effect of within‐cluster dependence in the censoring times. Sample size criteria and other design issues are also addressed for the setting where the event status is only ascertained at periodic assessments and times are interval censored. Copyright © 2014 JohnWiley & Sons, Ltd.     

Modeling clustering and treatment effect heterogeneity in parallel and stepped‐wedge cluster randomized trials

Cluster randomized trials are frequently used in health service evaluation. It is common practice to use an analysis model with a random effect to allow for clustering at the analysis stage. In designs where clusters are exposed to both control and treatment conditions, it may be of interest to examine treatment effect heterogeneity across clusters. In designs where clusters are not exposed to both control and treatment conditions, it can also be of interest to allow heterogeneity in the degree of clustering between arms. These two types of heterogeneity are related. It has been proposed in both parallel cluster trials, stepped‐wedge, and other cross‐over designs that this heterogeneity can be allowed for by incorporating additional random effect(s) into the model. Here, we show that the choice of model parameterization needs careful consideration as some parameterizations for additional heterogeneity induce unnecessary or implausible assumptions. We suggest more appropriate parameterizations, discuss their relative advantages, and demonstrate the implications of these model choices using a real example of a parallel cluster trial and a simulated stepped‐wedge trial.     

Constructing intervals for the intracluster correlation coefficient using Bayesian modelling, and application in cluster randomized trials

Studies in health research are commonly carried out in clustered settings, where the individual response data are correlated within clusters. Estimation and modelling of the extent of between-cluster variation contributes to understanding of the current study and to design of future studies. It is common to express between-cluster variation as an intracluster correlation coefficient (ICC), since this measure is directly comparable across outcomes. ICCs are generally reported unaccompanied by confidence intervals. In this paper, we describe a Bayesian modelling approach to interval estimation of the ICC. The flexibility of this framework allows useful extensions which are not easily available in existing methods, for example assumptions other than Normality for continuous outcome data, adjustment for individual-level covariates and simultaneous interval estimation of several ICCs. There is also the opportunity to incorporate prior beliefs on likely values of the ICC. The methods are exemplified using data from a cluster randomized trial.     

Sample size calculations for micro‐randomized trials in mHealth

The use and development of mobile interventions are experiencing rapid growth. In “just‐in‐time” mobile interventions, treatments are provided via a mobile device, and they are intended to help an individual make healthy decisions ‘in the moment,’ and thus have a proximal, near future impact. Currently, the development of mobile interventions is proceeding at a much faster pace than that of associated data science methods. A first step toward developing data‐based methods is to provide an experimental design for testing the proximal effects of these just‐in‐time treatments. In this paper, we propose a ‘micro‐randomized’ trial design for this purpose. In a micro‐randomized trial, treatments are sequentially randomized throughout the conduct of the study, with the result that each participant may be randomized at the 100s or 1000s of occasions at which a treatment might be provided. Further, we develop a test statistic for assessing the proximal effect of a treatment as well as an associated sample size calculator. We conduct simulation evaluations of the sample size calculator in various settings. Rules of thumb that might be used in designing a micro‐randomized trial are discussed. This work is motivated by our collaboration on the HeartSteps mobile application designed to increase physical activity. Copyright © 2015 John Wiley & Sons, Ltd.     

Sample size calculation for stepped wedge and other longitudinal cluster randomised trials

The sample size required for a cluster randomised trial is inflated compared with an individually randomised trial because outcomes of participants from the same cluster are correlated. Sample size calculations for longitudinal cluster randomised trials (including stepped wedge trials) need to take account of at least two levels of clustering: the clusters themselves and times within clusters. We derive formulae for sample size for repeated cross‐section and closed cohort cluster randomised trials with normally distributed outcome measures, under a multilevel model allowing for variation between clusters and between times within clusters. Our formulae agree with those previously described for special cases such as crossover and analysis of covariance designs, although simulation suggests that the formulae could underestimate required sample size when the number of clusters is small. Whether using a formula or simulation, a sample size calculation requires estimates of nuisance parameters, which in our model include the intracluster correlation, cluster autocorrelation, and individual autocorrelation. A cluster autocorrelation less than 1 reflects a situation where individuals sampled from the same cluster at different times have less correlated outcomes than individuals sampled from the same cluster at the same time. Nuisance parameters could be estimated from time series obtained in similarly clustered settings with the same outcome measure, using analysis of variance to estimate variance components. Copyright © 2016 John Wiley & Sons, Ltd.     

Sample size calculation for cluster randomization trials with a time-to-event endpoint

Cluster randomization trials randomize groups (called clusters) of subjects (called subunits) between intervention arms, and observations are collected from each subject. In this case, subunits within each cluster share common frailties, so that the observations from subunits of each cluster tend to be correlated. Oftentimes, the outcome of a cluster randomization trial is a time-to-event endpoint with censoring. In this article, we propose a closed form sample size formula for weighted rank tests to compare the marginal survival distributions between intervention arms under cluster randomization with possibly variable cluster sizes. Extensive simulation studies are conducted to evaluate the performance of our sample size formula under various design settings. Real study examples are taken to demonstrate our method.     

A simple sample size formula for analysis of covariance in cluster randomized trials

For cluster randomized trials with a continuous outcome, the sample size is often calculated as if an analysis of the outcomes at the end of the treatment period (follow‐up scores) would be performed. However, often a baseline measurement of the outcome is available or feasible to obtain. An analysis of covariance (ANCOVA) using both the baseline and follow‐up score of the outcome will then have more power. We calculate the efficiency of an ANCOVA analysis using the baseline scores compared with an analysis on follow‐up scores only. The sample size for such an ANCOVA analysis is a factor r² smaller, where r is the correlation of the cluster means between baseline and follow‐up. This correlation can be expressed in clinically interpretable parameters: the correlation between baseline and follow‐up of subjects (subject autocorrelation) and that of clusters (cluster autocorrelation). Because of this, subject matter knowledge can be used to provide (range of) plausible values for these correlations, when estimates from previous studies are lacking. Depending on how large the subject and cluster autocorrelations are, analysis of covariance can substantially reduce the number of clusters needed. Copyright © 2012 John Wiley & Sons, Ltd.     

Varying cluster sizes in trials with clusters in one treatment arm: Sample size adjustments when testing treatment effects with linear mixed models

Trials in which treatments induce clustering of observations in one of two treatment arms, such as when comparing group therapy with pharmacological treatment or with a waiting‐list group, are examined with respect to the efficiency loss caused by varying cluster sizes. When observations are (approximately) normally distributed, treatment effects can be estimated and tested through linear mixed model analysis. For maximum likelihood estimation, the asymptotic relative efficiency of unequal versus equal cluster sizes is derived. In an extensive Monte Carlo simulation for small sample sizes, the asymptotic relative efficiency turns out to be accurate for the treatment effect, but less accurate for the random intercept variance. For the treatment effect, the efficiency loss due to varying cluster sizes rarely exceeds 10 per cent, which can be regained by recruiting 11 per cent more clusters for one arm and 11 per cent more persons for the other. For the intercept variance the loss can be 16 per cent, which requires recruiting 19 per cent more clusters for one arm, with no additional recruitment of subjects for the other arm. Copyright © 2009 John Wiley & Sons, Ltd.