The design features and practicalities of conducting a pragmatic cluster randomized trial of obesity management in primary care

The aim of this paper is to describe the design features and practicalities of conducting a cluster randomized trial of obesity management in primary care. The aim of the trial is to assess the effectiveness of an obesity management educational intervention delivered to staff within primary care practices (unit of randomization) in terms of change in body weight of their patients (unit of analysis) at one year. The design features which merit particular attention in this cluster randomized trial include standardization of intervention, sample size considerations, recruitment of patients prior to randomization of practices, method of randomization to balance control and intervention practices with respect to practice and patient level characteristics, and blinding of outcome assessment. The practical problems (and our solutions) associated with implementing these design features, particularly those that result in a time delay between baseline data collection, randomization and intervention, are discussed.     

Accounting for expected attrition in the planning of community intervention trials

Trials in which intact communities are the units of randomization are increasingly being used to evaluate interventions which are more naturally administered at the community level, or when there is a substantial risk of treatment contamination. In this article we focus on the planning of community intervention trials in which k communities (for example, medical practices, worksites, or villages) are to be randomly allocated to each of an intervention and a control group, and fixed cohorts of m individuals enrolled in each community prior to randomization. Formulas to determine k or m may be obtained by adjusting standard sample size formulas to account for the intracluster correlation coefficient rho. In the presence of individual-level attrition however, observed cohort sizes are likely to vary. We show that conventional approaches of accounting for potential attrition, such as dividing standard sample size formulas by the anticipated follow-up rate pi or using the average anticipated cohort size m pi, may, respectively, overestimate or underestimate the required sample size when cluster follow-up rates are highly variable, and m or rho are large. We present new sample size estimation formulas for the comparison of two means or two proportions, which appropriately account for variation among cluster follow-up rates. These formulas are derived by specifying a model for the binary missingness indicators under the population-averaged approach, assuming an exchangeable intracluster correlation coefficient, denoted by tau. To aid in the planning of future trials, we recommend that estimates for tau be reported in published community intervention trials.     

Relative efficiency of unequal versus equal cluster sizes in cluster randomized and multicentre trials

Cluster randomized and multicentre trials evaluate the effect of a treatment on persons nested within clusters, for instance, patients within clinics or pupils within schools. Optimal sample sizes at the cluster (centre) and person level have been derived under the restrictive assumption of equal sample sizes per cluster. This paper addresses the relative efficiency of unequal versus equal cluster sizes in case of cluster randomization and person randomization within clusters. Starting from maximum likelihood parameter estimation, the relative efficiency is investigated numerically for a range of cluster size distributions. An approximate formula is presented for computing the relative efficiency as a function of the mean and variance of cluster size and the intraclass correlation, which can be used for adjusting the sample size. The accuracy of this formula is checked against the numerical results and found to be quite good. It is concluded that the loss of efficiency due to variation of cluster sizes rarely exceeds 10 per cent and can be compensated by sampling 11 per cent more clusters.     

Sample size calculations for intervention trials in primary care randomizing by primary care group: an empirical illustration from one proposed intervention trial

Because of the central role of the general practice in the delivery of British primary care, intervention trials in primary care often use the practice as the unit of randomization. The creation of primary care groups (PCGs) in April 1999 changed the organization of primary care and the commissioning of secondary care services. PCGs will directly affect the organization and delivery of primary, secondary and social care services. The PCG therefore becomes an appropriate target for organizational and educational interventions. Trials testing these interventions should involve randomization by PCG. This paper discusses the sample size required for a trial in primary care assessing the effect of a falls prevention programme among older people. In this trial PCGs will be randomized. The sample size calculations involve estimating intra-PCG correlation in primary outcome: fractured femur rate for those 65 years and over. No data on fractured femur rate were available at PCG level. PCGs are, however, similar in size and often coterminous with local authorities. Therefore, intra-PCG correlation in fractured femur rate was estimated from the intra-local authority correlation calculated from routine data. Three alternative trial designs are considered. In the first design, PCGs are selected for inclusion in the trial from the total population of England (eight regions). In the second design, PCGs are selected from two regions only. The third design is similar to the second except that PCGs are stratified by region and baseline value of fracture rate. Intracluster correlation is estimated for each of these designs using two methods: an approximation which assumes cluster sizes are equal and an alternative method which takes account of the fact that cluster sizes vary. Estimates of sample size required vary between 26 and 7 PCGs in each intervention group, depending on the trial design and the method used to calculate sample size. Not unexpectedly, stratification by baseline value of the outcome variable decreases the sample size required. In our analyses, geographic restriction of the population to be sampled reduces between-cluster variability in the primary outcome. This leads to an increase in precision. When allowance for variable cluster size is made, the increase in precision is not as great as would be expected with equal cluster sizes. This paper highlights the usefulness of routine data in work of this kind, and establishes one of the essential prerequisites for our proposed trial and other trials using primary outcomes with similar between-PCG variation: a feasible sample size.     

Sample size requirement to detect an intervention effect at the end of follow‐up in a longitudinal cluster randomized trial

It is often anticipated in a longitudinal cluster randomized clinical trial (cluster‐RCT) that the course of outcome over time will diverge between intervention arms. In these situations, testing the significance of a local intervention effect at the end of the trial may be more clinically relevant than evaluating overall mean differences between treatment groups. In this paper, we present a closed‐form power function for detecting this local intervention effect based on maximum likelihood estimates from a mixed‐effects linear regression model for three‐level continuous data. Sample size requirements for the number of units at each data level are derived from the power function. The power function and the corresponding sample size requirements are verified by a simulation study. Importantly, it is shown that sample size requirements computed with the proposed power function are smaller than that required when testing group mean difference using data only at the end of trial and ignoring the course of outcome over the entire study period. Copyright © 2009 John Wiley & Sons, Ltd.     

Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993.

OBJECTIVES. This methodological review aims to determine the extent to which design and analysis aspects of cluster randomization have been appropriately dealt with in reports of primary prevention trials. METHODS. All reports of primary prevention trials using cluster randomization that were published from 1990 to 1993 in the American Journal of Public Health and Preventive Medicine were identified. Each article was examined to determine whether cluster randomization was taken into account in the design and statistical analysis. RESULTS. Of the 21 articles, only 4 (19%) included sample size calculations or discussions of power that allowed for clustering, while 12 (57%) took clustering into account in the statistical analysis. CONCLUSIONS. Design and analysis issues associated with cluster randomization are not recognized widely enough. Reports of cluster randomized trials should include sample size calculations and statistical analyses that take clustering into account, estimates of design effects to help others planning trials, and a table showing the baseline distribution of important characteristics by intervention group, including the number of clusters and average cluster size for each group.     

Sample size formulae for intervention studies with the cluster as unit of randomization

This paper presents sample size formulae for both continuous and dichotomous endpoints obtained from intervention studies that use the cluster as the unit of randomization. The formulae provide the required number of clusters or the required number of individuals per cluster when the other number is given. The proposed formulae derive from Student's t-test with use of cluster summary measures and a variance that consists of within and between cluster components. Power contours are provided to help in the design of intervention studies that use cluster randomization. Sample size formulae for designs with and without stratification of clusters appear separately.     

A comparison of methods to analyse continuous data from pseudo cluster randomized trials

A major methodological reason to use cluster randomization is to avoid the contamination that would arise in an individually randomized design. However, when patient recruitment cannot be completed before randomization of clusters, the non-blindedness of recruiters and patients may cause selection bias, while in the control clusters, it may slow recruitment due to patient or recruiter preferences for the intervention. As a compromise, pseudo cluster randomization has been proposed. Because no insight is available into the relative performance of methods to analyse data obtained from this design, we compared the type I and II error rates of mixed models, generalized estimating equations (GEE) and a paired t-test to those of the estimator originally proposed in this design. The bias in the point estimate and its standard error were also incorporated into this comparison. Furthermore, we evaluated the effect of the weighting scheme and the accuracy of the sample size formula that have been described previously. Power levels of the originally proposed estimator and the unweighted mixed models were in agreement with the sample size formula, but the power of paired t-test fell short. GEE produced too large type I errors, unless the number of clusters was large (>30-40 per arm). The use of the weighting scheme generally enhanced the power, but at the cost of increasing the type I error in mixed models and GEE. We recommend unweighted mixed models as the best compromise between feasibility and power to analyse data from a pseudo cluster randomized trial.     

Randomizing patients by family practice: sample size estimation,intracluster correlation and data analysis

BACKGROUND: Cluster randomized controlled trials increasingly are used to evaluate health interventions where patients are nested within larger clusters such as practices, hospitals or communities. Patients within a cluster may be similar to each other relative to patients in other clusters on key variables; therefore, sample size calculations and analyses of results require special statistical methods. OBJECTIVE: The purpose of this study was to illustrate the calculations used for sample size estimation and data analysis and to provide estimates of the intraclass correlation coefficients (ICCs) for several variables using data from the Seniors Medication Assessment Research Trial (SMART), a community-based trial of pharmacists consulting to family physicians to optimize the drug therapy of older patients. METHODS: The study was a paired cluster randomized trial, where the family physician's practice was the cluster. The sample size calculation was based on a hypothesized reduction of 15% in mean daily units of medication in the intervention group compared with the control group, using an alpha of 0.05 (one-tailed) with 80% power, and an ICC from pilot data of 0.08. ICCs were estimated from the data for several variables. The analyses comparing the two groups used a random effects model for a meta-analysis over pairs. RESULTS: The design effect due to clustering was 2.12, resulting in an inflation in sample size from 340 patients required using individual randomization, to 720 patients using randomization of practices, with 15 patients from each of 48 practices. ICCs for medication use, health care utilization and general health were <0.1; however, the ICC for mean systolic blood pressure over the trial period was 0.199. CONCLUSIONS: Compared with individual randomization, cluster randomization may substantially increase the sample size required to maintain adequate statistical power. The differences in ICCs among potential outcome variables reinforce the need for valid estimates to ensure proper study design.     

Measures of between‐cluster variability in cluster randomized trials with binary outcomes

Cluster randomized trials (CRTs) are increasingly used to evaluate the effectiveness of health‐care interventions. A key feature of CRTs is that the observations on individuals within clusters are correlated as a result of between‐cluster variability. Sample size formulae exist which account for such correlations, but they make different assumptions regarding the between‐cluster variability in the intervention arm of a trial, resulting in different sample size estimates. We explore the relationship for binary outcome data between two common measures of between‐cluster variability: k, the coefficient of variation and ρ, the intracluster correlation coefficient. We then assess how the assumptions of constant k or ρ across treatment arms correspond to different assumptions about intervention effects. We assess implications for sample size estimation and present a simple solution to the problems outlined. Copyright © 2009 John Wiley & Sons, Ltd.     

Rank tests for clustered survival data when dependent subunits are randomized

In clustered survival data, subunits within each cluster share similar characteristics, so that observations made from them tend to be positively correlated. In clinical trials, the correlated subunits from the same cluster are often randomized to different treatment groups. In this case, the variance formulas of the standard rank tests such as the logrank, Gehan-Wilcoxon or Prentice-Wilcoxon, proposed for independent samples, need to be adjusted for intracluster correlations both within and between treatment groups for testing equality of marginal survival distributions. In this paper we derive a general form of simple variance formulas of the rank tests when subunits from the same cluster are randomized into different treatment groups. Extensive simulation studies are conducted to investigate small sample performance of the variance formulas. We compare our non-parametric rank tests based on the adjusted variances with one from a shared frailty model, which is an optimal semi-parametric testing procedure when the intracluster correlations within and between groups are the same.     

Sample size considerations for stratified cluster randomization design with binary outcomes and varying cluster size

Stratified cluster randomization trials (CRTs) have been frequently employed in clinical and healthcare research. Comparing with simple randomized CRTs, stratified CRTs reduce the imbalance of baseline prognostic factors among different intervention groups. Due to the popularity, there has been a growing interest in methodological development on sample size estimation and power analysis for stratified CRTs; however, existing work mostly assumes equal cluster size within each stratum and uses multilevel models. Clusters are often naturally formed with random sizes in CRTs. With varying cluster size, commonly used ad hoc approaches ignore the variability in cluster size, which may underestimate (overestimate) the required number of clusters for each group per stratum and lead to underpowered (overpowered) clinical trials. We propose closed-form sample size formulas for estimating the required total number of subjects and for estimating the number of clusters for each group per stratum, based on Cochran-Mantel-Haenszel statistic for stratified cluster randomization design with binary outcomes, accounting for both clustering and varying cluster size. We investigate the impact of various design parameters on the relative change in the required number of clusters for each group per stratum due to varying cluster size. Simulation studies are conducted to evaluate the finite-sample performance of the proposed sample size method. A real application example of a pragmatic stratified CRT of a triad of chronic kidney disease, diabetes, and hypertension is presented for illustration.     

Repairing the efficiency loss due to varying cluster sizes in two‐level two‐armed randomized trials with heterogeneous clustering

In two‐armed trials with clustered observations the arms may differ in terms of (i) the intraclass correlation, (ii) the outcome variance, (iii) the average cluster size, and (iv) the number of clusters. For a linear mixed model analysis of the treatment effect, this paper examines the expected efficiency loss due to varying cluster sizes based upon the asymptotic relative efficiency of varying versus constant cluster sizes. Simple, but nearly cost‐optimal, correction factors are derived for the numbers of clusters to repair this efficiency loss. In an extensive Monte Carlo simulation, the accuracy of the asymptotic relative efficiency and its Taylor approximation are examined for small sample sizes. Practical guidelines are derived to correct the numbers of clusters calculated under constant cluster sizes (within each treatment) when planning a study. Because of the variety of simulation conditions, these guidelines can be considered conservative but safe in many realistic situations. Copyright © 2016 John Wiley & Sons, Ltd.     

Pseudo cluster randomization dealt with selection bias and contamination in clinical trials

BACKGROUND AND OBJECTIVES: When contamination is present, randomization on a patient level leads to dilution of the treatment effect. The usual solution is to randomize on a cluster level, but at the cost of efficiency and more importantly, this may introduce selection bias. Furthermore, it may slow down recruitment in the clusters that are randomized to the "less interesting" treatment. We discuss an alternative randomization procedure to approach these problems. METHODS: Pseudo cluster randomization is a two-stage randomization procedure that balances between individual randomization and cluster randomization. For common scenarios, the design factors needed to calculate the appropriate sample size are tabulated. RESULTS: A pseudo cluster randomized design can reduce selection bias and contamination, while maintaining good efficiency and possibly improving enrollment. To make a well-informed choice of randomization procedure, we discuss the advantages of each method and provide a decision flow chart. CONCLUSION: When contamination is thought to be substantial in an individually randomized setting and a cluster randomized design would suffer from selection bias and/or slow recruitment, pseudo cluster randomization can be considered.     

Sample size re-estimation in cluster randomization trials

Cluster randomization trials in which families are the unit of allocation are commonly adopted for the evaluation of disease prevention interventions. Sample size estimation for cluster randomization trials depends on parameters that quantify the variability within and between clusters and the variability in cluster size. Accurate advance estimates of these nuisance parameters may be difficult to obtain and misspecification may lead to an underpowered study. Since families are typically recruited over time, we propose using a portion of the data to estimate the nuisance parameters and to re-estimate sample size based on the estimates. This extends the standard internal pilot study methods to the setting of cluster randomization trials. The effect of this design on the power, significance level and sample size is analysed via simulation and is shown to provide a flexible and practical approach to cluster randomization trials.     

Sample size calculations for clustered binary data.

In this paper we propose a sample size calculation method for testing on a binomial proportion when binary observations are dependent within clusters. In estimating the binomial proportion in clustered binary data, two weighting systems have been popular: equal weights to clusters and equal weights to units within clusters. When the number of units varies cluster by cluster, performance of these two weighting systems depends on the extent of correlation among units within each cluster. In addition to them, we will also use an optimal weighting method that minimizes the variance of the estimator. A sample size formula is derived for each of the estimators with different weighting schemes. We apply these methods to the sample size calculation for the sensitivity of a periodontal diagnostic test. Simulation studies are conducted to evaluate a finite sample performance of the three estimators. We also assess the influence of misspecified input parameter values on the calculated sample size. The optimal estimator requires equal or smaller sample sizes and is more robust to the misspecification of an input parameter than those assigning equal weights to units or clusters.     

A comparison of the statistical power of different methods for the analysis of cluster randomization trials with binary outcomes

Cluster randomization trials are randomized controlled trials (RCTs) in which intact clusters of subjects are randomized to either the intervention or to the control. Cluster randomization trials require different statistical methods of analysis than do conventional randomized controlled trials due to the potential presence of within-cluster homogeneity in responses. A variety of statistical methods have been proposed in the literature for the analysis of cluster randomization trials with binary outcomes. However, little is known about the relative statistical power of these methods to detect a statistically significant intervention effect. We conducted a series of Monte Carlo simulations to examine the statistical power of three methods that compare cluster-specific response rates between arms of the trial: the t-test, the Wilcoxon rank sum test, and the permutation test; and three methods that compare subject-level response rates: an adjusted chi-square test, a logistic-normal random effects model, and a generalized estimating equations (GEE) method. In our simulations we allowed the number of clusters, the number of subjects per cluster, the intraclass correlation coefficient and the magnitude of the intervention effect to vary. We demonstrated that the GEE approach tended to have the highest power for detecting a statistically significant intervention effect. However, in most of the 240 scenarios examined, the differences between the competing statistical methods were negligible. The largest mean difference in power between any two different statistical methods across the 240 scenarios was 0.02. The largest observed difference in power between two different statistical methods across the 240 scenarios and 15 pair-wise comparisons of methods was 0.14.     

Sample size considerations in observational health care quality studies

A common objective in health care quality studies involves measuring and comparing the quality of care delivered to cohorts of patients by different health care providers. The data used for inference involve observations on units grouped within clusters, such as patients treated within hospitals. Unlike cluster randomization trials where often clusters are randomized to interventions to learn about individuals, the target of inference in health quality studies is the cluster. Furthermore, randomization is often not performed and the resulting biases may invalidate standard tests. In this paper, we discuss approaches to sample size determination in the design of observational health quality studies when the outcome is binary. Methods for calculating sample size using marginal models are briefly reviewed, but the focus is on hierarchical binomial models. Sample size in unbalanced clusters and stratified designs are characterized. We draw upon the experiences that have arisen from a study funded by the Agency for Healthcare Research and Quality involving assessment of quality of care for patients with cardiovascular disease. If researchers are interested in comparing clusters, hierarchical models are preferred.     

The effect of cluster randomization on sample size in prevention research

BACKGROUND: This paper concerns the issue of cluster randomization in primary care practice intervention trials. We present information on the cluster effect of measuring the performance of various preventive maneuvers between groups of physicians based on a successful trial. We discuss the intracluster correlation coefficient of determining the required sample size and the implications for designing randomized controlled trials where groups of subjects (e.g., physicians in a group practice) are allocated at random. METHODS: We performed a cross-sectional study involving data from 46 participating practices with 106 physicians collected using self-administered questionnaires and a chart audit of 100 randomly selected charts per practice. The population was health service organizations (HSOs) located in Southern Ontario. We analyzed performance data for 13 preventive maneuvers determined by chart review and used analysis of variance to determine the intraclass correlation coefficient. An index of "up-to-datedness" was computed for each physician and practice as the number of a recommended preventive measure done divided by the number of eligible patients. An index called "inappropriateness" was computed in the same manner for the not-recommended measures. The intraclass correlation coefficients for 2 key study outcomes (up-to-datedness and inappropriateness) were also calculated and compared. RESULTS: The mean up-to-datedness score for the practices was 53.5% (95% confidence interval [CI], 51.0%-56.0%), and the mean inappropriateness score was 21.5% (95% CI, 18.1%-24.9%). The intraclass correlation for up-to-datedness was 0.0365 compared with inappropriateness at 0.1790. The intraclass correlation for preventive maneuvers ranged from 0.005 for blood pressure measurement to 0.66 for chest radiographs of smokers, and as a consequence required the sample size ranged from 20 to 42 physicians per group. CONCLUSIONS: Randomizing by practice clusters and analyzing at the level of the physician has important implications for sample size requirements. Larger intraclass correlations indicate interdependence among the physicians within a cluster; as a consequence, variability within clusters is reduced, and the required sample size increased. The key finding that many potential outcome measures perform differently in terms of the intracluster correlation reinforces the need for researchers to carefully consider the selection of outcome measures and adjust sample sizes accordingly when the unit of analysis and randomization are not the same.     

Sample size in cluster‐randomized trials with time to event as the primary endpoint

In cluster‐randomized trials, groups of individuals (clusters) are randomized to the treatments or interventions to be compared. In many of those trials, the primary objective is to compare the time for an event to occur between randomized groups, and the shared frailty model well fits clustered time‐to‐event data. Members of the same cluster tend to be more similar than members of different clusters, causing correlations. As correlations affect the power of a trial to detect intervention effects, the clustered design has to be considered in planning the sample size. In this publication, we derive a sample size formula for clustered time‐to‐event data with constant marginal baseline hazards and correlation within clusters induced by a shared frailty term. The sample size formula is easy to apply and can be interpreted as an extension of the widely used Schoenfeld's formula, accounting for the clustered design of the trial. Simulations confirm the validity of the formula and its use also for non‐constant marginal baseline hazards. Findings are illustrated on a cluster‐randomized trial investigating methods of disseminating quality improvement to addiction treatment centers in the USA. Copyright © 2012 John Wiley & Sons, Ltd.