Sample size estimation for stratified individual and cluster randomized trials with binary outcomes

Individual randomized trials (IRTs) and cluster randomized trials (CRTs) with binary outcomes arise in a variety of settings and are often analyzed by logistic regression (fitted using generalized estimating equations for CRTs). The effect of stratification on the required sample size is less well understood for trials with binary outcomes than for continuous outcomes. We propose easy-to-use methods for sample size estimation for stratified IRTs and CRTs and demonstrate the use of these methods for a tuberculosis prevention CRT currently being planned. For both IRTs and CRTs, we also identify the ratio of the sample size for a stratified trial vs a comparably powered unstratified trial, allowing investigators to evaluate how stratification will affect the required sample size when planning a trial. For CRTs, these can be used when the investigator has estimates of the within-stratum intracluster correlation coefficients (ICCs) or by assuming a common within-stratum ICC. Using these methods, we describe scenarios where stratification may have a practically important impact on the required sample size. We find that in the two-stratum case, for both IRTs and for CRTs with very small cluster sizes, there are unlikely to be plausible scenarios in which an important sample size reduction is achieved when the overall probability of a subject experiencing the event of interest is low. When the probability of events is not small, or when cluster sizes are large, however, there are scenarios where practically important reductions in sample size result from stratification.     

Measures of between‐cluster variability in cluster randomized trials with binary outcomes

Cluster randomized trials (CRTs) are increasingly used to evaluate the effectiveness of health‐care interventions. A key feature of CRTs is that the observations on individuals within clusters are correlated as a result of between‐cluster variability. Sample size formulae exist which account for such correlations, but they make different assumptions regarding the between‐cluster variability in the intervention arm of a trial, resulting in different sample size estimates. We explore the relationship for binary outcome data between two common measures of between‐cluster variability: k, the coefficient of variation and ρ, the intracluster correlation coefficient. We then assess how the assumptions of constant k or ρ across treatment arms correspond to different assumptions about intervention effects. We assess implications for sample size estimation and present a simple solution to the problems outlined. Copyright © 2009 John Wiley & Sons, Ltd.     

Sample size adjustments for varying cluster sizes in cluster randomized trials with binary outcomes analyzed with second‐order PQL mixed logistic regression

Adjustments of sample size formulas are given for varying cluster sizes in cluster randomized trials with a binary outcome when testing the treatment effect with mixed effects logistic regression using second‐order penalized quasi‐likelihood estimation (PQL). Starting from first‐order marginal quasi‐likelihood (MQL) estimation of the treatment effect, the asymptotic relative efficiency of unequal versus equal cluster sizes is derived. A Monte Carlo simulation study shows this asymptotic relative efficiency to be rather accurate for realistic sample sizes, when employing second‐order PQL. An approximate, simpler formula is presented to estimate the efficiency loss due to varying cluster sizes when planning a trial. In many cases sampling 14 per cent more clusters is sufficient to repair the efficiency loss due to varying cluster sizes. Since current closed‐form formulas for sample size calculation are based on first‐order MQL, planning a trial also requires a conversion factor to obtain the variance of the second‐order PQL estimator. In a second Monte Carlo study, this conversion factor turned out to be 1.25 at most. Copyright © 2010 John Wiley & Sons, Ltd.     

Inference and sample size calculation for clinical trials with incomplete observations of paired binary outcomes

We investigate the estimation of intervention effect and sample size determination for experiments where subjects are supposed to contribute paired binary outcomes with some incomplete observations. We propose a hybrid estimator to appropriately account for the mixed nature of observed data: paired outcomes from those who contribute complete pairs of observations and unpaired outcomes from those who contribute either pre‐intervention or post‐intervention outcomes. We theoretically prove that if incomplete data are evenly distributed between the pre‐intervention and post‐intervention periods, the proposed estimator will always be more efficient than the traditional estimator. A numerical research shows that when the distribution of incomplete data is unbalanced, the proposed estimator will be superior when there is moderate‐to‐strong positive within‐subject correlation. We further derive a closed‐form sample size formula to help researchers determine how many subjects need to be enrolled in such studies. Simulation results suggest that the calculated sample size maintains the empirical power and type I error under various design configurations. We demonstrate the proposed method using a real application example. Copyright © 2016 John Wiley & Sons, Ltd.     

Sample size formulae for intervention studies with the cluster as unit of randomization

This paper presents sample size formulae for both continuous and dichotomous endpoints obtained from intervention studies that use the cluster as the unit of randomization. The formulae provide the required number of clusters or the required number of individuals per cluster when the other number is given. The proposed formulae derive from Student's t-test with use of cluster summary measures and a variance that consists of within and between cluster components. Power contours are provided to help in the design of intervention studies that use cluster randomization. Sample size formulae for designs with and without stratification of clusters appear separately.     

Sample size calculation for cluster randomization trials with a time-to-event endpoint

Cluster randomization trials randomize groups (called clusters) of subjects (called subunits) between intervention arms, and observations are collected from each subject. In this case, subunits within each cluster share common frailties, so that the observations from subunits of each cluster tend to be correlated. Oftentimes, the outcome of a cluster randomization trial is a time-to-event endpoint with censoring. In this article, we propose a closed form sample size formula for weighted rank tests to compare the marginal survival distributions between intervention arms under cluster randomization with possibly variable cluster sizes. Extensive simulation studies are conducted to evaluate the performance of our sample size formula under various design settings. Real study examples are taken to demonstrate our method.     

Sample size requirements for stratified cluster randomization designs.

Sample size requirements are provided for designs of studies in which clusters are randomized within each of several strata, where cluster size itself may be a stratifying factor. The approach generalizes a formula derived by Woolson et al., which provides sample size requirements for the Cochran-Mantel-Haenszel statistic. Issues of data analysis are also discussed.     

Sample size calculation for stepped wedge and other longitudinal cluster randomised trials

The sample size required for a cluster randomised trial is inflated compared with an individually randomised trial because outcomes of participants from the same cluster are correlated. Sample size calculations for longitudinal cluster randomised trials (including stepped wedge trials) need to take account of at least two levels of clustering: the clusters themselves and times within clusters. We derive formulae for sample size for repeated cross‐section and closed cohort cluster randomised trials with normally distributed outcome measures, under a multilevel model allowing for variation between clusters and between times within clusters. Our formulae agree with those previously described for special cases such as crossover and analysis of covariance designs, although simulation suggests that the formulae could underestimate required sample size when the number of clusters is small. Whether using a formula or simulation, a sample size calculation requires estimates of nuisance parameters, which in our model include the intracluster correlation, cluster autocorrelation, and individual autocorrelation. A cluster autocorrelation less than 1 reflects a situation where individuals sampled from the same cluster at different times have less correlated outcomes than individuals sampled from the same cluster at the same time. Nuisance parameters could be estimated from time series obtained in similarly clustered settings with the same outcome measure, using analysis of variance to estimate variance components. Copyright © 2016 John Wiley & Sons, Ltd.     

Sample size/power calculation for stratified case–cohort design

The case–cohort (CC) study design usually has been used for risk factor assessment in epidemiologic studies or disease prevention trials for rare diseases. The sample size/power calculation for a stratified CC (SCC) design has not been addressed before. This article derives such result based on a stratified test statistic. Simulation studies show that the proposed test for the SCC design utilizing small sub‐cohort sampling fractions is valid and efficient for situations where the disease rate is low. Furthermore, optimization of sampling in the SCC design is discussed and compared with proportional and balanced sampling techniques. An epidemiological study is provided to illustrate the sample size calculation under the SCC design. Copyright © 2014 John Wiley & Sons, Ltd.     

A comparison of the statistical power of different methods for the analysis of cluster randomization trials with binary outcomes

Cluster randomization trials are randomized controlled trials (RCTs) in which intact clusters of subjects are randomized to either the intervention or to the control. Cluster randomization trials require different statistical methods of analysis than do conventional randomized controlled trials due to the potential presence of within-cluster homogeneity in responses. A variety of statistical methods have been proposed in the literature for the analysis of cluster randomization trials with binary outcomes. However, little is known about the relative statistical power of these methods to detect a statistically significant intervention effect. We conducted a series of Monte Carlo simulations to examine the statistical power of three methods that compare cluster-specific response rates between arms of the trial: the t-test, the Wilcoxon rank sum test, and the permutation test; and three methods that compare subject-level response rates: an adjusted chi-square test, a logistic-normal random effects model, and a generalized estimating equations (GEE) method. In our simulations we allowed the number of clusters, the number of subjects per cluster, the intraclass correlation coefficient and the magnitude of the intervention effect to vary. We demonstrated that the GEE approach tended to have the highest power for detecting a statistically significant intervention effect. However, in most of the 240 scenarios examined, the differences between the competing statistical methods were negligible. The largest mean difference in power between any two different statistical methods across the 240 scenarios was 0.02. The largest observed difference in power between two different statistical methods across the 240 scenarios and 15 pair-wise comparisons of methods was 0.14.     

A simple sample size formula for analysis of covariance in cluster randomized trials

For cluster randomized trials with a continuous outcome, the sample size is often calculated as if an analysis of the outcomes at the end of the treatment period (follow‐up scores) would be performed. However, often a baseline measurement of the outcome is available or feasible to obtain. An analysis of covariance (ANCOVA) using both the baseline and follow‐up score of the outcome will then have more power. We calculate the efficiency of an ANCOVA analysis using the baseline scores compared with an analysis on follow‐up scores only. The sample size for such an ANCOVA analysis is a factor r² smaller, where r is the correlation of the cluster means between baseline and follow‐up. This correlation can be expressed in clinically interpretable parameters: the correlation between baseline and follow‐up of subjects (subject autocorrelation) and that of clusters (cluster autocorrelation). Because of this, subject matter knowledge can be used to provide (range of) plausible values for these correlations, when estimates from previous studies are lacking. Depending on how large the subject and cluster autocorrelations are, analysis of covariance can substantially reduce the number of clusters needed. Copyright © 2012 John Wiley & Sons, Ltd.     

Design and analysis of trials with a partially nested design and a binary outcome measure

Where treatments are administered to groups of patients or delivered by therapists, outcomes for patients in the same group or treated by the same therapist may be more similar, leading to clustering. Trials of such treatments should take account of this effect. Where such a treatment is compared with an un‐clustered treatment, the trial has a partially nested design. This paper compares statistical methods for this design where the outcome is binary. Investigation of consistency reveals that a random coefficient model with a random effect for group or therapist is not consistent with other methods for a null treatment effect, and so this model is not recommended for this design. Small sample performance of a cluster‐adjusted test of proportions, a summary measures test and logistic generalised estimating equations and random intercept models are investigated through simulation. The expected treatment effect is biased for the logistic models. Empirical test size of two‐sided tests is raised only slightly, but there are substantial biases for one‐sided tests. Three formulae are proposed for calculating sample size and power based on (i) the difference of proportions, (ii) the log‐odds ratio or (iii) the arc‐sine transformation of proportions. Calculated power from these formulae is compared with empirical power from a simulations study. Logistic models appeared to perform better than those based on proportions with the likelihood ratio test performing best in the range of scenarios considered. For these analyses, the log‐odds ratio method of calculation of power gave an approximate lower limit for empirical power. © 2015 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.     

Mendelian randomization analysis of a time‐varying exposure for binary disease outcomes using functional data analysis methods

A Mendelian randomization (MR) analysis is performed to analyze the causal effect of an exposure variable on a disease outcome in observational studies, by using genetic variants that affect the disease outcome only through the exposure variable. This method has recently gained popularity among epidemiologists given the success of genetic association studies. Many exposure variables of interest in epidemiological studies are time varying, for example, body mass index (BMI). Although longitudinal data have been collected in many cohort studies, current MR studies only use one measurement of a time‐varying exposure variable, which cannot adequately capture the long‐term time‐varying information. We propose using the functional principal component analysis method to recover the underlying individual trajectory of the time‐varying exposure from the sparsely and irregularly observed longitudinal data, and then conduct MR analysis using the recovered curves. We further propose two MR analysis methods. The first assumes a cumulative effect of the time‐varying exposure variable on the disease risk, while the second assumes a time‐varying genetic effect and employs functional regression models. We focus on statistical testing for a causal effect. Our simulation studies mimicking the real data show that the proposed functional data analysis based methods incorporating longitudinal data have substantial power gains compared to standard MR analysis using only one measurement. We used the Framingham Heart Study data to demonstrate the promising performance of the new methods as well as inconsistent results produced by the standard MR analysis that relies on a single measurement of the exposure at some arbitrary time point.     

Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes

When designing a study to develop a new prediction model with binary or time-to-event outcomes, researchers should ensure their sample size is adequate in terms of the number of participants (n) and outcome events (E) relative to the number of predictor parameters (p) considered for inclusion. We propose that the minimum values of n and E (and subsequently the minimum number of events per predictor parameter, EPP) should be calculated to meet the following three criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of ≥ 0.9, (ii) small absolute difference of ≤ 0.05 in the model's apparent and adjusted Nagelkerke's R² , and (iii) precise estimation of the overall risk in the population. Criteria (i) and (ii) aim to reduce overfitting conditional on a chosen p, and require prespecification of the model's anticipated Cox-Snell R² , which we show can be obtained from previous studies. The values of n and E that meet all three criteria provides the minimum sample size required for model development. Upon application of our approach, a new diagnostic model for Chagas disease requires an EPP of at least 4.8 and a new prognostic model for recurrent venous thromboembolism requires an EPP of at least 23. This reinforces why rules of thumb (eg, 10 EPP) should be avoided. Researchers might additionally ensure the sample size gives precise estimates of key predictor effects; this is especially important when key categorical predictors have few events in some categories, as this may substantially increase the numbers required.     

Analysis of data arising from a stratified design with the cluster as unit of randomization

This paper discusses statistical techniques for the analysis of dichotomous data arising from a design in which the investigator randomly assigns each of two clusters of possibly varying size to interventions within strata. The problem addressed is that of assessing the statistical significance of the intervention effect over all strata. We propose a weighted paired t-test based on the empirical logistic transform for designs that randomize large aggregate clusters in each of several strata.     

Sample size calculation in cost‐effectiveness cluster randomized trials: optimal and maximin approaches

In this paper, the optimal sample sizes at the cluster and person levels for each of two treatment arms are obtained for cluster randomized trials where the cost‐effectiveness of treatments on a continuous scale is studied. The optimal sample sizes maximize the efficiency or power for a given budget or minimize the budget for a given efficiency or power. Optimal sample sizes require information on the intra‐cluster correlations (ICCs) for effects and costs, the correlations between costs and effects at individual and cluster levels, the ratio of the variance of effects translated into costs to the variance of the costs (the variance ratio), sampling and measuring costs, and the budget. When planning, a study information on the model parameters usually is not available. To overcome this local optimality problem, the current paper also presents maximin sample sizes. The maximin sample sizes turn out to be rather robust against misspecifying the correlation between costs and effects at the cluster and individual levels but may lose much efficiency when misspecifying the variance ratio. The robustness of the maximin sample sizes against misspecifying the ICCs depends on the variance ratio. The maximin sample sizes are robust under misspecification of the ICC for costs for realistic values of the variance ratio greater than one but not robust under misspecification of the ICC for effects. Finally, we show how to calculate optimal or maximin sample sizes that yield sufficient power for a test on the cost‐effectiveness of an intervention. Copyright © 2014 John Wiley & Sons, Ltd.     

Varying cluster sizes in trials with clusters in one treatment arm: Sample size adjustments when testing treatment effects with linear mixed models

Trials in which treatments induce clustering of observations in one of two treatment arms, such as when comparing group therapy with pharmacological treatment or with a waiting‐list group, are examined with respect to the efficiency loss caused by varying cluster sizes. When observations are (approximately) normally distributed, treatment effects can be estimated and tested through linear mixed model analysis. For maximum likelihood estimation, the asymptotic relative efficiency of unequal versus equal cluster sizes is derived. In an extensive Monte Carlo simulation for small sample sizes, the asymptotic relative efficiency turns out to be accurate for the treatment effect, but less accurate for the random intercept variance. For the treatment effect, the efficiency loss due to varying cluster sizes rarely exceeds 10 per cent, which can be regained by recruiting 11 per cent more clusters for one arm and 11 per cent more persons for the other. For the intercept variance the loss can be 16 per cent, which requires recruiting 19 per cent more clusters for one arm, with no additional recruitment of subjects for the other arm. Copyright © 2009 John Wiley & Sons, Ltd.     

Strategies for analyzing multilevel cluster-randomized studies with binary outcomes collected at varying intervals of time

Frequently, studies are conducted in a real clinic setting. When the outcome of interest is collected longitudinally over a specified period of time, this design can lead to unequally spaced intervals and varying numbers of assessments. In our study, these features were embedded in a randomized, factorial design in which interventions to improve blood pressure control were delivered to both patients and providers. We examine the effect of the intervention and compare methods of estimation of both fixed effects and variance components in the multilevel generalized linear mixed model. Methods of comparison include penalized quasi-likelihood (PQL), adaptive quadrature, and Bayesian Monte Carlo methods. We also investigate the implications of reducing the data and analysis to baseline and final measurements. In the full analysis, the PQL fixed-effects estimates were closest to zero and confidence intervals were generally narrower than those of the other methods. The adaptive quadrature and Bayesian fixed-effects estimates were similar, but the Bayesian credible intervals were consistently wider. Variance component estimation was markedly different across methods, particularly for the patient-level random effects. In the baseline and final measurement analysis, we found that estimates and corresponding confidence intervals for the adaptive quadrature and Bayesian methods were very similar. However, the time effect was diminished and other factors also failed to reach statistical significance, most likely due to decreased power. When analyzing data from this type of design, we recommend using either adaptive quadrature or Bayesian methods to fit a multilevel generalized linear mixed model including all available measurements. Published in 2008 by John Wiley & Sons, Ltd.     

Propensity score methods for estimating relative risks in cluster randomized trials with low‐incidence binary outcomes and selection bias

Despite randomization, selection bias may occur in cluster randomized trials. Classical multivariable regression usually allows for adjusting treatment effect estimates with unbalanced covariates. However, for binary outcomes with low incidence, such a method may fail because of separation problems. This simulation study focused on the performance of propensity score (PS)‐based methods to estimate relative risks from cluster randomized trials with binary outcomes with low incidence. The results suggested that among the different approaches used (multivariable regression, direct adjustment on PS, inverse weighting on PS, and stratification on PS), only direct adjustment on the PS fully corrected the bias and moreover had the best statistical properties. Copyright © 2014 John Wiley & Sons, Ltd.     

A note on sample size calculations for cluster randomised crossover trials with a fixed number of clusters

Girardeau, Ravaud and Donner in 2008 presented a formula for sample size calculations for cluster randomised crossover trials, when the intracluster correlation coefficient, interperiod correlation coefficient and mean cluster size are specified in advance. However, in many randomised trials, the number of clusters is constrained in some way, but the mean cluster size is not. We present a version of the Girardeau formula for sample size calculations for cluster randomised crossover trials when the number of clusters is fixed. Formulae are given for the minimum number of clusters, the maximum cluster size and the relationship between the correlation coefficients when there are constraints on both the number of clusters and the cluster size. Our version of the formula may aid the efficient planning and design of cluster randomised crossover trials.