Comparing completely and stratified randomized designs in cluster randomized trials when the stratifying factor is cluster size: a simulation study

Stratified randomized designs are popular in cluster randomized trials (CRTs) because they increase the chance of the intervention groups being well balanced in terms of identified prognostic factors at baseline and may increase statistical power. The objective of this paper is to assess the gains in power obtained by stratifying randomization by cluster size, when cluster size is associated with an important cluster level factor which is otherwise unaccounted for in data analysis. A simulation study was carried out using a CRT where UK general practices were the randomized units as a template. The results show that when cluster size is strongly associated with a cluster level factor which is predictive of outcome, the stratified randomized design has superior power results to the completely randomized design and that the superiority is related to the number of clusters.     

The impact of adding cost information to a conversation aid to support shared decision making about low-risk prostate cancer treatment: Results of a stepped-wedge cluster randomised trial

Background

Decision aids help patients consider the benefits and drawbacks of care options but rarely include cost information. We assessed the impact of a conversation-based decision aid containing information about low-risk prostate cancer management options and their relative costs.

Methods

We conducted a stepped-wedge cluster randomised trial in outpatient urology practices within a US-based academic medical center. We randomised five clinicians to four intervention sequences and enroled patients newly diagnosed with low-risk prostate cancer. Primary patient-reported outcomes collected postvisit included the frequency of cost conversations and referrals to address costs. Other patient-reported outcomes included: decisional conflict postvisit and at 3 months, decision regret at 3 months, shared decision-making postvisit, financial toxicity postvisit and at 3 months. Clinicians reported their attitudes about shared decision-making pre- and poststudy, and the intervention's feasibility and acceptability. We used hierarchical regression analysis to assess patient outcomes. The clinician was included as a random effect; fixed effects included education, employment, telehealth versus in-person visit, visit date, and enrolment period.

Results

Between April 2020 and March 2022, we screened 513 patients, contacted 217 eligible patients, and enroled 117/217 (54%) (51 in usual care, 66 in the intervention group). In adjusted analyses, the intervention was not associated with cost conversations (β = .82, p = .27), referrals to cost-related resources (β = −0.36, p = .81), shared decision-making (β = −0.79, p = .32), decisional conflict postvisit (β = −0.34, p= .70), or at follow-up (β = −2.19, p = .16), decision regret at follow-up (β = −9.76, p = .11), or financial toxicity postvisit (β = −1.32, p = .63) or at follow-up (β = −2.41, p = .23). Most clinicians and patients had positive attitudes about the intervention and shared decision-making. In exploratory unadjusted analyses, patients in the intervention group experienced more transient indecision (p < .02) suggesting increased deliberation between visit and follow-up.

Discussion

Despite enthusiasm from clinicians, the intervention was not significantly associated with hypothesised outcomes, though we were unable to robustly test outcomes due to recruitment challenges. Recruitment at the start of the COVID-19 pandemic impacted eligibility, sample size/power, study procedures, and increased telehealth visits and financial worry, independent of the intervention. Future work should explore ways to support shared decision-making, cost conversations, and choice deliberation with a larger sample. Such work could involve additional members of the care team, and consider the detail, quality, and timing of addressing these issues.

Patient or Public Contribution

Patients and clinicians were engaged as stakeholder advisors meeting monthly throughout the duration of the project to advise on the study design, measures selected, data interpretation, and dissemination of study findings.     

Sample size calculation for cluster randomized cross-over trials

The cluster randomized cross-over design has been proposed in particular because it prevents an imbalance that may bring into question the internal validity of parallel group cluster trials. We derived a sample size formula for continuous outcomes that takes into account both the intraclass correlation coefficient (representing the clustering effect) and the interperiod correlation (induced by the cross-over design).     

Efficient design of cluster randomized trials with treatment‐dependent costs and treatment‐dependent unknown variances

Cluster randomized trials evaluate the effect of a treatment on persons nested within clusters, where treatment is randomly assigned to clusters. Current equations for the optimal sample size at the cluster and person level assume that the outcome variances and/or the study costs are known and homogeneous between treatment arms. This paper presents efficient yet robust designs for cluster randomized trials with treatment‐dependent costs and treatment‐dependent unknown variances, and compares these with 2 practical designs. First, the maximin design (MMD) is derived, which maximizes the minimum efficiency (minimizes the maximum sampling variance) of the treatment effect estimator over a range of treatment‐to‐control variance ratios. The MMD is then compared with the optimal design for homogeneous variances and costs (balanced design), and with that for homogeneous variances and treatment‐dependent costs (cost‐considered design). The results show that the balanced design is the MMD if the treatment‐to control cost ratio is the same at both design levels (cluster, person) and within the range for the treatment‐to‐control variance ratio. It still is highly efficient and better than the cost‐considered design if the cost ratio is within the range for the squared variance ratio. Outside that range, the cost‐considered design is better and highly efficient, but it is not the MMD. An example shows sample size calculation for the MMD, and the computer code (SPSS and R) is provided as supplementary material. The MMD is recommended for trial planning if the study costs are treatment‐dependent and homogeneity of variances cannot be assumed.     

Analysis of cluster randomized cross-over trial data: a comparison of methods

In a cluster randomized cross-over trial, all participating clusters receive both intervention and control treatments consecutively, in separate time periods. Patients recruited by each cluster within the same time period receive the same intervention, and randomization determines order of treatment within a cluster. Such a design has been used on a number of occasions. For analysis of the trial data, the approach of analysing cluster-level summary measures is appealing on the grounds of simplicity, while hierarchical modelling allows for the correlation of patients within periods within clusters and offers flexibility in the model assumptions. We consider several cluster-level approaches and hierarchical models and make comparison in terms of empirical precision, coverage, and practical considerations. The motivation for a cluster randomized trial to employ cross-over of trial arms is particularly strong when the number of clusters available is small, so we examine performance of the methods under small, medium and large (6, 18, 30) numbers of clusters. One hierarchical model and two cluster-level methods were found to perform consistently well across the designs considered. These three methods are efficient, provide appropriate standard errors and coverage, and continue to perform well when incorporating adjustment for an individual-level covariate. We conclude that choice between hierarchical models and cluster-level methods should be influenced by the extent of complexity in the planned analysis.     

Accounting for a decaying correlation structure in cluster randomized trials with continuous recruitment

A requirement for calculating sample sizes for cluster randomized trials (CRTs) conducted over multiple periods of time is the specification of a form for the correlation between outcomes of subjects within the same cluster, encoded via the within-cluster correlation structure. Previously proposed within-cluster correlation structures have made strong assumptions; for example, the usual assumption is that correlations between the outcomes of all pairs of subjects are identical (“uniform correlation”). More recently, structures that allow for a decay in correlation between pairs of outcomes measured in different periods have been suggested. However, these structures are overly simple in settings with continuous recruitment and measurement. We propose a more realistic “continuous-time correlation decay” structure whereby correlations between subjects' outcomes decay as the time between these subjects' measurement times increases. We investigate the use of this structure on trial planning in the context of a primary care diabetes trial, where there is evidence of decaying correlation between pairs of patients' outcomes over time. In particular, for a range of different trial designs, we derive the variance of the treatment effect estimator under continuous-time correlation decay and compare this to the variance obtained under uniform correlation. For stepped wedge and cluster randomized crossover designs, incorrectly assuming uniform correlation will underestimate the required sample size under most trial configurations likely to occur in practice. Planning of CRTs requires consideration of the most appropriate within-cluster correlation structure to obtain a suitable sample size.     

Information content of stepped-wedge designs when treatment effect heterogeneity and/or implementation periods are present

Stepped-wedge cluster randomized trials, which randomize clusters of subjects to treatment sequences in which clusters switch from control to intervention conditions, are being conducted with increasing frequency. Due to the real-world nature of this design, methodological and implementation challenges are ubiquitous. To account for such challenges, more complex statistical models to plan studies and analyze data are required. In this paper, we consider stepped-wedge trials that accommodate treatment effect heterogeneity across clusters, implementation periods during which no data are collected, or both treatment effect heterogeneity and implementation periods. Previous work has shown that the sequence-period cells of a stepped-wedge design contribute unequal amounts of information to the estimation of the treatment effect. In this paper, we extend that work by considering the amount of information available for the estimation of the treatment effect in each sequence-period cell, sequence, and period of stepped-wedge trials with more complex designs and outcome models. When either treatment effect heterogeneity and/or implementation periods are present, the pattern of information content of sequence-period cells tends to be clustered around the times of the switch from control to intervention condition, similarly to when these complexities are absent. However, the presence and degree of treatment effect heterogeneity and the number of implementation periods can influence the information content of periods and sequences markedly.     

Optimal multistage designs for randomised clinical trials with continuous outcomes

Multistage designs allow considerable reductions in the expected sample size of a trial. When stopping for futility or efficacy is allowed at each stage, the expected sample size under different possible true treatment effects (δ) is of interest. The δ-minimax design is the one for which the maximum expected sample size is minimised amongst all designs that meet the types I and II error constraints. Previous work has compared a two-stage δ-minimax design with other optimal two-stage designs. Applying the δ-minimax design to designs with more than two stages was not previously considered because of computational issues. In this paper, we identify the δ-minimax designs with more than two stages through use of a novel application of simulated annealing. We compare them with other optimal multistage designs and the triangular design. We show that, as for two-stage designs, the δ-minimax design has good expected sample size properties across a broad range of treatment effects but generally has a higher maximum sample size. To overcome this drawback, we use the concept of admissible designs to find trials which balance the maximum expected sample size and maximum sample size. We show that such designs have good expected sample size properties and a reasonable maximum sample size and, thus, are very appealing for use in clinical trials.     

Optimal designs in three-level cluster randomized trials with a binary outcome

Cluster randomized trials (CRTs) were originally proposed for use when randomization at the subject level is practically infeasible or may lead to a severe estimation bias of the treatment effect. However, recruiting an additional cluster costs more than enrolling an additional subject in an individually randomized trial. Under budget constraints, researchers have proposed the optimal sample sizes in two-level CRTs. CRTs may have a three-level structure, in which two levels of clustering should be considered. In this paper, we propose optimal designs in three-level CRTs with a binary outcome, assuming a nested exchangeable correlation structure in generalized estimating equation models. We provide the variance of estimators of three commonly used measures: risk difference, risk ratio, and odds ratio. For a given sampling budget, we discuss how many clusters and how many subjects per cluster are necessary to minimize the variance of each measure estimator. For known association parameters, the locally optimal design is proposed. When association parameters are unknown but within predetermined ranges, the MaxiMin design is proposed to maximize the minimum of relative efficiency over the possible ranges, that is, to minimize the risk of the worst scenario.     

Bayesian methods of analysis for cluster randomized trials with count outcome data

Bayesian approaches to inference in cluster randomized trials have been investigated for normally distributed and binary outcome measures. However, relatively little attention has been paid to outcome measures which are counts of events. We discuss an extension of previously published Bayesian hierarchical models to count data, which usually can be assumed to be distributed according to a Poisson distribution. We develop two models, one based on the traditional rate ratio, and one based on the rate difference which may often be more intuitively interpreted for clinical trials, and is needed for economic evaluation of interventions. We examine the relationship between the intracluster correlation coefficient (ICC) and the between‐cluster variance for each of these two models. In practice, this allows one to use the previously published evidence on ICCs to derive an informative prior distribution which can then be used to increase the precision of the posterior distribution of the ICC. We demonstrate our models using a previously published trial assessing the effectiveness of an educational intervention and a prior distribution previously derived. We assess the robustness of the posterior distribution for effectiveness to departures from a normal distribution of the random effects. Copyright © 2009 John Wiley & Sons, Ltd.     

Developments in cluster randomized trials and Statistics in Medicine

The design and analysis of cluster randomized trials has been a recurrent theme in Statistics in Medicine since the early volumes. In celebration of 25 years of Statistics in Medicine, this paper reviews recent developments, particularly those that featured in the journal. Issues in design such as sample size calculations, matched paired designs, cohort versus cross-sectional designs, and practical design problems are covered. Developments in analysis include modification of robust methods to cope with small numbers of clusters, generalized estimation equations, population averaged and cluster specific models. Finally, issues on presenting data, some other clustering issues and the general problem of evaluating complex interventions are briefly mentioned.     

Comparison of tests for categorical data from a stratified cluster randomized trial

Two features commonly exhibited by randomized trials of health promotion interventions are cluster randomization and stratification. Ignoring correlations between individuals within clusters can lead to an inflated type I error rate and hence a P-value which overstates the significance of the result. This paper compares several methods for analysing categorical data from a stratified cluster randomized trial. We propose an extension of a method from survey sampling that uses the design effect to reduce the effective sample size. We compare this with three methods from Zhang and Boos that extend the standard Cochran-Mantel-Haenszel (CMH) statistic by using appropriate covariance matrices, and with a bootstrap method. The comparison is based on empirical type I error rates from a simulation study, in which the number of clusters randomized is small, as in most public health intervention studies. The method that performs consistently well is one of the Zhang and Boos extensions of the standard CMH statistic.     

Relative efficiency of unequal versus equal cluster sizes in cluster randomized and multicentre trials

Cluster randomized and multicentre trials evaluate the effect of a treatment on persons nested within clusters, for instance, patients within clinics or pupils within schools. Optimal sample sizes at the cluster (centre) and person level have been derived under the restrictive assumption of equal sample sizes per cluster. This paper addresses the relative efficiency of unequal versus equal cluster sizes in case of cluster randomization and person randomization within clusters. Starting from maximum likelihood parameter estimation, the relative efficiency is investigated numerically for a range of cluster size distributions. An approximate formula is presented for computing the relative efficiency as a function of the mean and variance of cluster size and the intraclass correlation, which can be used for adjusting the sample size. The accuracy of this formula is checked against the numerical results and found to be quite good. It is concluded that the loss of efficiency due to variation of cluster sizes rarely exceeds 10 per cent and can be compensated by sampling 11 per cent more clusters.     

Admissible two-stage designs for phase II cancer clinical trials

In a typical two-stage design for a phase II cancer clinical trial for efficacy screening of cytotoxic agents, a fixed number of patients are initially enrolled and treated. The trial may be terminated for lack of efficacy if the observed number of tumour responses after the first stage is too small, thus avoiding treatment of patient with inefficacious regimen. Otherwise, an additional fixed number of patients are enrolled and treated to accumulate additional information on efficacy as well as safety. The minimax and the so-called 'optimal' designs by Simon have been widely used, and other designs have largely been ignored in the past for such two-stage cancer clinical trials. Recently Jung et al. proposed a graphical method to search for compromise designs with features more favourable than either the minimax or the optimal design. In this paper, we develop a family of two-stage designs that are admissible according to a Bayesian decision-theoretic criterion based on an ethically justifiable loss function. We show that the admissible designs include as special cases the Simon's minimax and the optimal designs as well as the compromise designs introduced by Jung et al. We also present a Java program to search for admissible designs that are compromises between the minimax and the optimal designs.     

Design of a Phase III cluster randomized trial to assess the efficacy and safety of a malaria transmission blocking vaccine

Vaccines interrupting Plasmodium falciparum malaria transmission targeting sexual, sporogonic, or mosquito-stage antigens (SSM-VIMT) are currently under development to reduce malaria transmission. An international group of malaria experts was established to evaluate the feasibility and optimal design of a Phase III cluster randomized trial (CRT) that could support regulatory review and approval of an SSM-VIMT. The consensus design is a CRT with a sentinel population randomly selected from defined inner and buffer zones in each cluster, a cluster size sufficient to assess true vaccine efficacy in the inner zone, and inclusion of ongoing assessment of vaccine impact stratified by distance of residence from the cluster edge. Trials should be conducted first in areas of moderate transmission, where SSM-VIMT impact should be greatest. Sample size estimates suggest that such a trial is feasible, and within the range of previously supported trials of malaria interventions, although substantial issues to implementation exist.     

Sample size calculation in cost‐effectiveness cluster randomized trials: optimal and maximin approaches

In this paper, the optimal sample sizes at the cluster and person levels for each of two treatment arms are obtained for cluster randomized trials where the cost‐effectiveness of treatments on a continuous scale is studied. The optimal sample sizes maximize the efficiency or power for a given budget or minimize the budget for a given efficiency or power. Optimal sample sizes require information on the intra‐cluster correlations (ICCs) for effects and costs, the correlations between costs and effects at individual and cluster levels, the ratio of the variance of effects translated into costs to the variance of the costs (the variance ratio), sampling and measuring costs, and the budget. When planning, a study information on the model parameters usually is not available. To overcome this local optimality problem, the current paper also presents maximin sample sizes. The maximin sample sizes turn out to be rather robust against misspecifying the correlation between costs and effects at the cluster and individual levels but may lose much efficiency when misspecifying the variance ratio. The robustness of the maximin sample sizes against misspecifying the ICCs depends on the variance ratio. The maximin sample sizes are robust under misspecification of the ICC for costs for realistic values of the variance ratio greater than one but not robust under misspecification of the ICC for effects. Finally, we show how to calculate optimal or maximin sample sizes that yield sufficient power for a test on the cost‐effectiveness of an intervention. Copyright © 2014 John Wiley & Sons, Ltd.     

Modeling clustering and treatment effect heterogeneity in parallel and stepped‐wedge cluster randomized trials

Cluster randomized trials are frequently used in health service evaluation. It is common practice to use an analysis model with a random effect to allow for clustering at the analysis stage. In designs where clusters are exposed to both control and treatment conditions, it may be of interest to examine treatment effect heterogeneity across clusters. In designs where clusters are not exposed to both control and treatment conditions, it can also be of interest to allow heterogeneity in the degree of clustering between arms. These two types of heterogeneity are related. It has been proposed in both parallel cluster trials, stepped‐wedge, and other cross‐over designs that this heterogeneity can be allowed for by incorporating additional random effect(s) into the model. Here, we show that the choice of model parameterization needs careful consideration as some parameterizations for additional heterogeneity induce unnecessary or implausible assumptions. We suggest more appropriate parameterizations, discuss their relative advantages, and demonstrate the implications of these model choices using a real example of a parallel cluster trial and a simulated stepped‐wedge trial.     

Ratio estimators of intervention effects on event rates in cluster randomized trials

We consider five asymptotically unbiased estimators of intervention effects on event rates in non-matched and matched-pair cluster randomized trials, including ratio of mean counts $\left(r_1\right)$, ratio of mean cluster-level event rates $\left(r_2\right)$, ratio of event rates $\left(r_3\right)$, double ratio of counts $\left(r_4\right)$, and double ratio of event rates $\left(r_5\right)$. In the absence of an indirect effect, they all estimate the direct effect of the intervention. Otherwise, $r_1$, $r_2,$ and $r_3$ estimate the total effect, which comprises the direct and indirect effects, whereas $r_4$ and $r_5$ estimate the direct effect only. We derive the conditions under which each estimator is more precise or powerful than its alternatives. To control bias in studies with a small number of clusters, we propose a set of approximately unbiased estimators. We evaluate their properties by simulation and apply the methods to a trial of seasonal malaria chemoprevention. The approximately unbiased estimators are practically unbiased and their confidence intervals usually have coverage probability close to the nominal level; the asymptotically unbiased estimators perform well when the number of clusters is approximately 32 or more per trial arm. Despite its simplicity, $r_1$ performs comparably with $r_2$ and $r_3$ in trials with a large but realistic number of clusters. When the variability of baseline event rate is large and there is no indirect effect, $r_4$ and $r_5$ tend to offer higher power than $r_1$, $r_2,$ and $r_3$. We discuss the implications of these findings to the planning and analysis of cluster randomized trials.