Stepped wedge designs could reduce the required sample size in cluster randomized trials

ObjectiveThe stepped wedge design is increasingly being used in cluster randomized trials (CRTs). However, there is not much information available about the design and analysis strategies for these kinds of trials. Approaches to sample size and power calculations have been provided, but a simple sample size formula is lacking. Therefore, our aim is to provide a sample size formula for cluster randomized stepped wedge designs.Study Design and SettingWe derived a design effect (sample size correction factor) that can be used to estimate the required sample size for stepped wedge designs. Furthermore, we compared the required sample size for the stepped wedge design with a parallel group and analysis of covariance (ANCOVA) design.ResultsOur formula corrects for clustering as well as for the design. Apart from the cluster size and intracluster correlation, the design effect depends on choices of the number of steps, the number of baseline measurements, and the number of measurements between steps. The stepped wedge design requires a substantial smaller sample size than a parallel group and ANCOVA design.ConclusionFor CRTs, the stepped wedge design is far more efficient than the parallel group and ANCOVA design in terms of sample size.     

Strengths and weaknesses of a stepped wedge cluster randomized design: its application in a colorectal cancer follow-up study

ObjectivesTo determine the advantages and disadvantages of a stepped wedge design for a specific clinical application.Study Design and SettingThe clinical application was a pragmatic cluster randomized surgical trial intending to find an increased percentage of curable recurrences in patients in follow-up after colorectal cancer. Advantages and disadvantages of the stepped wedge design were evaluated, and for this application, new advantages and disadvantages were presented.ResultsA main advantage of the stepped wedge design was that the intervention rolls out to all participants, motivating patients and doctors, and a large number of patients who were included in this study. The stepped wedge design increased the complexity of the data analysis, and there were concerns regarding the informed consent procedure. The repeated measurements may bring burden to patients in terms of quality of life, satisfaction, and costs.ConclusionThe stepped wedge design is a strong alternative for pragmatic cluster randomized trials. The known advantages hold, whereas most of the disadvantages were not applicable to this application. The main advantage was that we were able to include a large number of patients. Main disadvantages were that the informed consent procedure can be problematic and that the analysis of the data can be complex.     

Improving the efficiency of individually randomized clinical trials by staggering the introduction of the intervention

In cluster randomized trials, the introduction of the intervention can be staggered in different clusters, leading to a stepped wedge design. This strategy can lead to gains in efficiency, which might also translate to the context of individually randomized trials, though this has been relatively unexplored. Here, we present one illustrative example. We consider trials in which participants start in a control condition such as routine care and can cross over at any stage to the active intervention. We make the assumption that the effect of the intervention is the same however long the delay before a participant crosses over to the intervention condition. We consider designs for a trial with three repeated assessments, including a baseline, and show that a three-sequence design with staggered introduction of the intervention in two of the sequences estimates the treatment effect after one period more efficiently than a parallel groups design.     

Modeling clustering and treatment effect heterogeneity in parallel and stepped‐wedge cluster randomized trials

Cluster randomized trials are frequently used in health service evaluation. It is common practice to use an analysis model with a random effect to allow for clustering at the analysis stage. In designs where clusters are exposed to both control and treatment conditions, it may be of interest to examine treatment effect heterogeneity across clusters. In designs where clusters are not exposed to both control and treatment conditions, it can also be of interest to allow heterogeneity in the degree of clustering between arms. These two types of heterogeneity are related. It has been proposed in both parallel cluster trials, stepped‐wedge, and other cross‐over designs that this heterogeneity can be allowed for by incorporating additional random effect(s) into the model. Here, we show that the choice of model parameterization needs careful consideration as some parameterizations for additional heterogeneity induce unnecessary or implausible assumptions. We suggest more appropriate parameterizations, discuss their relative advantages, and demonstrate the implications of these model choices using a real example of a parallel cluster trial and a simulated stepped‐wedge trial.     

Statistical efficiency and optimal design for stepped cluster studies under linear mixed effects models

In stepped cluster designs the intervention is introduced into some (or all) clusters at different times and persists until the end of the study. Instances include traditional parallel cluster designs and the more recent stepped‐wedge designs. We consider the precision offered by such designs under mixed‐effects models with fixed time and random subject and cluster effects (including interactions with time), and explore the optimal choice of uptake times. The results apply both to cross‐sectional studies where new subjects are observed at each time‐point, and longitudinal studies with repeat observations on the same subjects. The efficiency of the design is expressed in terms of a ‘cluster‐mean correlation’ which carries information about the dependency‐structure of the data, and two design coefficients which reflect the pattern of uptake‐times. In cross‐sectional studies the cluster‐mean correlation combines information about the cluster‐size and the intra‐cluster correlation coefficient. A formula is given for the ‘design effect’ in both cross‐sectional and longitudinal studies. An algorithm for optimising the choice of uptake times is described and specific results obtained for the best balanced stepped designs. In large studies we show that the best design is a hybrid mixture of parallel and stepped‐wedge components, with the proportion of stepped wedge clusters equal to the cluster‐mean correlation. The impact of prior uncertainty in the cluster‐mean correlation is considered by simulation. Some specific hybrid designs are proposed for consideration when the cluster‐mean correlation cannot be reliably estimated, using a minimax principle to ensure acceptable performance across the whole range of unknown values. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.     

Different methods to analyze stepped wedge trial designs revealed different aspects of intervention effects

ObjectivesWithin epidemiology, a stepped wedge trial design (i.e., a one-way crossover trial in which several arms start the intervention at different time points) is increasingly popular as an alternative to a classical cluster randomized controlled trial. Despite this increasing popularity, there is a huge variation in the methods used to analyze data from a stepped wedge trial design.Study Design and SettingFour linear mixed models were used to analyze data from a stepped wedge trial design on two example data sets. The four methods were chosen because they have been (frequently) used in practice. Method 1 compares all the intervention measurements with the control measurements. Method 2 treats the intervention variable as a time-independent categorical variable comparing the different arms with each other. In method 3, the intervention variable is a time-dependent categorical variable comparing groups with different number of intervention measurements, whereas in method 4, the changes in the outcome variable between subsequent measurements are analyzed.ResultsRegarding the results in the first example data set, methods 1 and 3 showed a strong positive intervention effect, which disappeared after adjusting for time. Method 2 showed an inverse intervention effect, whereas method 4 did not show a significant effect at all. In the second example data set, the results were the opposite. Both methods 2 and 4 showed significant intervention effects, whereas the other two methods did not. For method 4, the intervention effect attenuated after adjustment for time.ConclusionDifferent methods to analyze data from a stepped wedge trial design reveal different aspects of a possible intervention effect. The choice of a method partly depends on the type of the intervention and the possible time-dependent effect of the intervention. Furthermore, it is advised to combine the results of the different methods to obtain an interpretable overall result.     

The use of permutation tests for the analysis of parallel and stepped‐wedge cluster‐randomized trials

We investigate the use of permutation tests for the analysis of parallel and stepped‐wedge cluster‐randomized trials. Permutation tests for parallel designs with exponential family endpoints have been extensively studied. The optimal permutation tests developed for exponential family alternatives require information on intraclass correlation, a quantity not yet defined for time‐to‐event endpoints. Therefore, it is unclear how efficient permutation tests can be constructed for cluster‐randomized trials with such endpoints. We consider a class of test statistics formed by a weighted average of pair‐specific treatment effect estimates and offer practical guidance on the choice of weights to improve efficiency. We apply the permutation tests to a cluster‐randomized trial evaluating the effect of an intervention to reduce the incidence of hospital‐acquired infection. In some settings, outcomes from different clusters may be correlated, and we evaluate the validity and efficiency of permutation test in such settings. Lastly, we propose a permutation test for stepped‐wedge designs and compare its performance with mixed‐effect modeling and illustrate its superiority when sample sizes are small, the underlying distribution is skewed, or there is correlation across clusters. Copyright © 2017 John Wiley & Sons, Ltd.     

Design and analysis considerations for cohort stepped wedge cluster randomized trials with a decay correlation structure

A stepped wedge cluster randomized trial is a type of longitudinal cluster design that sequentially switches clusters to intervention over time until all clusters are treated. While the traditional posttest-only parallel design requires adjustment for a single intraclass correlation coefficient, the stepped wedge design allows multiple outcome measurements from the same cluster and so additional correlation parameters are necessary to characterize the within-cluster correlation structure. Although a number of studies have differentiated between the concepts of within-period and between-period correlations, only a few studies have allowed the between-period correlation to decay over time. In this article, we consider the proportional decay correlation structure for a cohort stepped wedge design, and provide a matrix-adjusted quasi-least squares approach to accurately estimate the correlation parameters along with the marginal intervention effect. We further develop the sample size and power procedures accounting for the correlation decay, and investigate the accuracy of the power procedure with continuous outcomes in a simulation study. We show that the empirical power agrees well with the prediction even with as few as nine clusters, when data are analyzed with matrix-adjusted quasi-least squares concurrently with a suitable bias-corrected sandwich variance. Two trial examples are provided to illustrate the new sample size procedure.     

Maintaining the validity of inference in small-sample stepped wedge cluster randomized trials with binary outcomes when using generalized estimating equations

Stepped wedge cluster trials are an increasingly popular alternative to traditional parallel cluster randomized trials. Such trials often utilize a small number of clusters and numerous time intervals, and these components must be considered when choosing an analysis method. A generalized linear mixed model containing a random intercept and fixed time and intervention covariates is the most common analysis approach. However, the sole use of a random intercept applies a constant intraclass correlation coefficient structure, which is an assumption that is likely to be violated given stepped wedge trials (SWTs) have multiple time intervals. Alternatively, generalized estimating equations (GEE) are robust to the misspecification of the working correlation structure, although it has been shown that small-sample adjustments to standard error estimates and the use of appropriate degrees of freedom are required to maintain the validity of inference when the number of clusters is small. In this article, we show, using an extensive simulation study based on a motivating example and a more general design, the use of GEE can maintain the validity of inference in small-sample SWTs with binary outcomes. Furthermore, we show which combinations of bias corrections to standard error estimates and degrees of freedom work best in terms of attaining nominal type I error rates.     

Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation

Objective

To describe the application of the stepped wedge cluster randomized controlled trial (CRCT) design.

Study Design and Setting

Systematic review. We searched Medline, Embase, PsycINFO, HMIC, CINAHL, Cochrane Library, Web of Knowledge, and Current Controlled Trials Register for articles published up to January 2010. Stepped wedge CRCTs from all fields of research were included. Two authors independently reviewed and extracted data from the studies.

Results

Twenty-five studies were included in the review. Motivations for using the design included ethical, logistical, financial, social, and political acceptability and methodological reasons. Most studies were evaluating an intervention during routine implementation. For most of the included studies, there was also a belief or empirical evidence suggesting that the intervention would do more good than harm. There was variation in data analysis methods and insufficient quality of reporting.

Conclusions

The stepped wedge CRCT design has been mainly used for evaluating interventions during routine implementation, particularly for interventions that have been shown to be effective in more controlled research settings, or where there is lack of evidence of effectiveness but there is a strong belief that they will do more good than harm. There is need for consistent data analysis and reporting.     

Robust analysis of stepped wedge trials using cluster‐level summaries within periods

In stepped‐wedge trials (SWTs), the intervention is rolled out in a random order over more than 1 time‐period. SWTs are often analysed using mixed‐effects models that require strong assumptions and may be inappropriate when the number of clusters is small. We propose a non‐parametric within‐period method to analyse SWTs. This method estimates the intervention effect by comparing intervention and control conditions in a given period using cluster‐level data corresponding to exposure. The within‐period intervention effects are combined with an inverse‐variance‐weighted average, and permutation tests are used. We present an example and, using simulated data, compared the method to (1) a parametric cluster‐level within‐period method, (2) the most commonly used mixed‐effects model, and (3) a more flexible mixed‐effects model. We simulated scenarios where period effects were common to all clusters, and when they varied according to a distribution informed by routinely collected health data. The non‐parametric within‐period method provided unbiased intervention effect estimates with correct confidence‐interval coverage for all scenarios. The parametric within‐period method produced confidence intervals with low coverage for most scenarios. The mixed‐effects models' confidence intervals had low coverage when period effects varied between clusters but had greater power than the non‐parametric within‐period method when period effects were common to all clusters. The non‐parametric within‐period method is a robust method for analysing SWT. The method could be used by trial statisticians who want to emphasise that the SWT is a randomised trial, in the common position of being uncertain about whether data will meet the assumptions necessary for mixed‐effect models.     

Bias and inference from misspecified mixed‐effect models in stepped wedge trial analysis

Many stepped wedge trials (SWTs) are analysed by using a mixed‐effect model with a random intercept and fixed effects for the intervention and time periods (referred to here as the standard model). However, it is not known whether this model is robust to misspecification. We simulated SWTs with three groups of clusters and two time periods; one group received the intervention during the first period and two groups in the second period. We simulated period and intervention effects that were either common‐to‐all or varied‐between clusters. Data were analysed with the standard model or with additional random effects for period effect or intervention effect. In a second simulation study, we explored the weight given to within‐cluster comparisons by simulating a larger intervention effect in the group of the trial that experienced both the control and intervention conditions and applying the three analysis models described previously. Across 500 simulations, we computed bias and confidence interval coverage of the estimated intervention effect. We found up to 50% bias in intervention effect estimates when period or intervention effects varied between clusters and were treated as fixed effects in the analysis. All misspecified models showed undercoverage of 95% confidence intervals, particularly the standard model. A large weight was given to within‐cluster comparisons in the standard model. In the SWTs simulated here, mixed‐effect models were highly sensitive to departures from the model assumptions, which can be explained by the high dependence on within‐cluster comparisons. Trialists should consider including a random effect for time period in their SWT analysis model. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.     

Sample size calculation for stepped wedge and other longitudinal cluster randomised trials

The sample size required for a cluster randomised trial is inflated compared with an individually randomised trial because outcomes of participants from the same cluster are correlated. Sample size calculations for longitudinal cluster randomised trials (including stepped wedge trials) need to take account of at least two levels of clustering: the clusters themselves and times within clusters. We derive formulae for sample size for repeated cross‐section and closed cohort cluster randomised trials with normally distributed outcome measures, under a multilevel model allowing for variation between clusters and between times within clusters. Our formulae agree with those previously described for special cases such as crossover and analysis of covariance designs, although simulation suggests that the formulae could underestimate required sample size when the number of clusters is small. Whether using a formula or simulation, a sample size calculation requires estimates of nuisance parameters, which in our model include the intracluster correlation, cluster autocorrelation, and individual autocorrelation. A cluster autocorrelation less than 1 reflects a situation where individuals sampled from the same cluster at different times have less correlated outcomes than individuals sampled from the same cluster at the same time. Nuisance parameters could be estimated from time series obtained in similarly clustered settings with the same outcome measure, using analysis of variance to estimate variance components. Copyright © 2016 John Wiley & Sons, Ltd.     

Heterogeneous treatment effects and bias in the analysis of the stepped wedge design

Health Services and Outcomes Research Methodology - The effect of an intervention in a stepped wedge design can vary across clusters or with time since exposure to treatment, but consequences of...     

Cluster randomized controlled trials in primary care: An introduction

Background: Cluster randomized trials occur when groups or clusters of individuals, rather than the individuals themselves, are randomized to intervention and control groups and outcomes are measured on individuals within those clusters. Within primary care, between 1997 and 2000, there has been a virtual doubling in the number of published cluster randomized trials. A recent systematic review, specifically within primary care, found study quality to be both generally lower than that reported elsewhere and not to have shown any recent quality improvement. Objective: To discuss the design, conduct and analysis of cluster randomized trials within primary care in terms of the appropriate expertise required, potential bias, ethical considerations and expense. Discussion: Compared with trials that involve the randomization of individual participants, cluster randomized trials are more complex to design and analyse and, for a given sample size, have decreased power and a broadening of confidence intervals. Cluster randomized trials are specifically prone to potential bias at two levels—the cluster and individual. Regarding the former, it is recommended that cluster allocation be undertaken by a party independent to the research team and careful consideration be given to ensure minimal cluster attrition. Bias at the individual level can be overcome by identifying trial participants before randomization and at this time obtaining consent for intervention, data collection or both. A unique ethical aspect to cluster randomized trials is that cluster leaders may consent to the trial on behalf of potential cluster members. Additional costs of cluster randomized trials include the increased number of patients required, the complexity in their design and conduct and, usually, the need to recruit clusters de novo.

Conclusion: Cluster randomized trials are a powerful and increasingly popular research tool. They are uniquely placed for the conduct of research within primary-care clusters where intracluster contamination can occur. Associated methodological issues are straightforward and surmountable and just need careful consideration and management.     

The analysis of terminal endpoint events in stepped wedge designs

The stepped wedge design is a unique clinical trial design that allows for a sequential introduction of an intervention. However, the statistical analysis is unclear when this design is applied in survival data. The time‐dependent introduction of the intervention in combination with terminal endpoints and interval censoring makes the analysis more complicated. In this paper, a time‐on‐study scale discrete survival model was constructed. Simulations were conducted primarily to study the performance of our model for different settings of the stepped wedge design. Secondary, we compared our approach to continuous Cox proportional hazard model. The results show that the discrete survival model estimates the intervention effects unbiasedly. If the length of the censoring interval is increased, the precision of the estimates is decreased. Without left truncation and late entry, the number of steps improves the precision of the estimates, whereas in combination of left truncation and late entry, the number of steps decreases the precision. Given the same number of participants and clusters, a parallel group design has higher precision than a stepped wedge design. Copyright © 2016 John Wiley & Sons, Ltd.     

Sample size calculations for stepped wedge and cluster randomised trials: a unified approach

ObjectivesTo clarify and illustrate sample size calculations for the cross-sectional stepped wedge cluster randomized trial (SW-CRT) and to present a simple approach for comparing the efficiencies of competing designs within a unified framework.Study Design and SettingWe summarize design effects for the SW-CRT, the parallel cluster randomized trial (CRT), and the parallel cluster randomized trial with before and after observations (CRT-BA), assuming cross-sectional samples are selected over time. We present new formulas that enable trialists to determine the required cluster size for a given number of clusters. We illustrate by example how to implement the presented design effects and give practical guidance on the design of stepped wedge studies.ResultsFor a fixed total cluster size, the choice of study design that provides the greatest power depends on the intracluster correlation coefficient (ICC) and the cluster size. When the ICC is small, the CRT tends to be more efficient; when the ICC is large, the SW-CRT tends to be more efficient and can serve as an alternative design when the CRT is an infeasible design.ConclusionOur unified approach allows trialists to easily compare the efficiencies of three competing designs to inform the decision about the most efficient design in a given scenario.     

The Devon Active Villages Evaluation (DAVE) trial: Study protocol of a stepped wedge cluster randomised trial of a community-level physical activity intervention in rural southwest England

ABSTRACT: BACKGROUND: Although physical inactivity has been linked with numerous chronic health conditions and overall mortality, the majority of English adults report doing insufficient physical activity. To increase population physical activity levels, researchers have called for more community-level interventions. To evaluate these complex public health interventions, innovative study designs are required. This study protocol describes Devon Active Villages, a community-level intervention providing physical activity opportunities to 128 rural villages in southwest England, and the methods used to evaluate its effectiveness in increasing physical activity levels. METHODS: A stepped wedge cluster randomised trial will be used to evaluate whether Devon Active Villages leads to increased physical activity levels in rural communities. Community engagement will help tailor activity programmes for each village; communities will then be supported for a further twelve months. The intervention will be delivered over four periods, each lasting twelve weeks. Data collection consists of a postal survey of a random sample of adults aged 18 years and over, at baseline and after each of the four intervention periods. The questionnaire includes questions on participant demographics, physical activity behaviour, local environment characteristics, awareness of local activity programmes, and psychosocial factors. Based on detecting an increase in the proportion of people who meet physical activity guidelines (from 25% to 30%), at least ten respondents are needed from each of the 128 villages at each stage (80% power at the 5% level of significance). Anticipating a 20% response rate, 6,400 questionnaires will be sent out at each stage (i.e., 50 surveys to each village). Using data from all five periods, a comparison of study outcomes between intervention and control arms will be performed, allowing for time period (as a fixed effect) and the random effect induced by correlation of outcomes (clustering) within villages. DISCUSSION: This paper describes the use of a stepped wedge cluster randomised trial to evaluate a complex, community-level physical activity intervention in an under-studied population of adults in rural communities in southwest England. The study addresses gaps in the current literature by providing new insights into physical activity levels in this population. Trial Registration Number: Current Controlled Trials ISRCTN37321160.     

Sample size and power calculations for open cohort longitudinal cluster randomized trials

When calculating sample size or power for stepped wedge or other types of longitudinal cluster randomized trials, it is critical that the planned sampling structure be accurately specified. One common assumption is that participants will provide measurements in each trial period, that is, a closed cohort, and another is that each participant provides only one measurement during the course of the trial. However some studies have an “open cohort” sampling structure, where participants may provide measurements in variable numbers of periods. To date, sample size calculations for longitudinal cluster randomized trials have not accommodated open cohorts. Feldman and McKinlay (1994) provided some guidance, stating that the participant-level autocorrelation could be varied to account for the degree of overlap in different periods of the study, but did not indicate precisely how to do so. We present sample size and power formulas that allow for open cohorts and discuss the impact of the degree of “openness” on sample size and power. We consider designs where the number of participants in each cluster will be maintained throughout the trial, but individual participants may provide differing numbers of measurements. Our results are a unification of closed cohort and repeated cross-sectional sample results of Hooper et al (2016), and indicate precisely how participant autocorrelation of Feldman and McKinlay should be varied to account for an open cohort sampling structure. We discuss different types of open cohort sampling schemes and how open cohort sampling structure impacts on power in the presence of decaying within-cluster correlations and autoregressive participant-level errors.     

Methods for modelling change in cluster randomization trials

Randomized trials are often designed to assess an intervention's ability to change patient knowledge, behaviour or health. The study outcome will then need to be measured at least twice for each subject--prior to random assignment and following implementation of the intervention. In this paper we consider methods for modelling change when data are obtained from cluster randomization trials where the unit of allocation is a family, school or community. Attention focuses on mixed effects linear regression extensions of (i) two-sample t-tests and (ii) analysis of covariance, in both cases accounting for dependencies among cluster members. Algebraic expressions for tests of the intervention effect are derived for the special case where there are a fixed number of subjects per cluster while simulation studies are used to compare the power of these procedures in the more realistic case where there is variability in cluster size. A key conclusion is that there can be considerable gains in power when allowing for different individual-level and cluster-level associations between the baseline and follow-up assessments. The discussion is illustrated using data from a school-based smoking prevention trial.