ON DESIGN CONSIDERATIONS AND RANDOMIZATION-BASED INFERENCE FOR COMMUNITY INTERVENTION TRIALS

This paper discusses design considerations and the role of randomization-based inference in randomized community intervention trials. We stress that longitudinal follow-up of cohorts within communities often yields useful information on the effects of intervention on individuals, whereas cross-sectional surveys can usefully assess the impact of intervention on group indices of health. We also discuss briefly special design considerations, such as sampling cohorts from targeted subpopulations (for example, heavy smokers), matching the communities, calculating sample size, and other practical issues. We present randomization tests for matched and unmatched cohort designs. As is well known, these tests necessarily have proper size under the strong null hypothesis that treatment has no effect on any community response. It is less well known, however, that the size of randomization tests can exceed nominal levels under the ‘weak’ null hypothesis that intervention does not affect the average community response. Because this weak null hypothesis is of interest in community intervention trials, we study the size of randomization tests by simulation under conditions in which the weak null hypothesis holds but the strong null hypothesis does not. In unmatched studies, size may exceed nominal levels under the weak null hypothesis if there are more intervention than control communities and if the variance among community responses is larger among control communities than among intervention communities; size may also exceed nominal levels if there are more control than intervention communities and if the variance among community responses is larger among intervention communities. Otherwise, size is likely near nominal levels. To avoid such problems, we recommend use of the same numbers of control and intervention communities in unmatched designs. Pair-matched designs usually have size near nominal levels, even under the weak null hypothesis. We have identified some extreme cases, unlikely to arise in practice, in which even the size of pair-matched studies can exceed nominal levels. These simulations, however, tend to confirm the robustness of randomization tests for matched and unmatched community intervention trials, particularly if the latter designs have equal numbers of intervention and control communities. We also describe adaptations of randomization tests to allow for covariate adjustment, missing data, and application to cross-sectional surveys. We show that covariate adjustment can increase power, but such power gains diminish as the random component of variation among communities increases, which corresponds to increasing intraclass correlation of responses within communities. We briefly relate our results to model-based methods of inference for community intervention trials that include hierarchical models such as an analysis of variance model with random community effects and fixed intervention effects. Although we have tailored this paper to the design of community intervention trials, many of the ideas apply to other experiments in which one allocates groups or clusters of subjects at random to intervention or control treatments.     

Person-time analysis of paired community intervention trials when the number of communities is small

Community intervention trials involve randomization of communities to either an intervention or control arm. The objective of this paper is to evaluate person-time methods of analysis of paired community intervention trials when the number of community pairs is small. We consider several test procedures and evaluate their performance by simulation. Naive methods that ignore intracluster correlation, such as standard Mantel–Haenszel type statistics, can be misleading. The performance of the paired t-test depends on the distribution of the random community effects. Permutation tests perform well for the ranges of situations considered. However, there can be considerable loss of power with permutation methods compared to standard Mantel–Haenszel methods if in fact there is no intracluster correlation when the number of pairs is small. We consider methods to account for individual level covariates. Motivation for this work came from recent randomized community intervention trials in Africa to prevent transmission of the human immunodeficiency virus (HIV). © 1998 John Wiley & Sons, Ltd.     

The importance and role of intracluster correlations in planning cluster trials

There is increasing recognition of the critical role of intracluster correlations of health behavior outcomes in cluster intervention trials. This study examines the estimation, reporting, and use of intracluster correlations in planning cluster trials. We use an estimating equations approach to estimate the intracluster correlations corresponding to the multiple-time-point nested cross-sectional design. Sample size formulae incorporating 2 types of intracluster correlations are examined for the purpose of planning future trials. The traditional intracluster correlation is the correlation among individuals within the same community at a specific time point. A second type is the correlation among individuals within the same community at different time points. For a "time x condition" analysis of a pretest-posttest nested cross-sectional trial design, we show that statistical power considerations based upon a posttest-only design generally are not an adequate substitute for sample size calculations that incorporate both types of intracluster correlations. Estimation, reporting, and use of intracluster correlations are illustrated for several dichotomous measures related to underage drinking collected as part of a large nonrandomized trial to enforce underage drinking laws in the United States from 1998 to 2004.     

Cluster randomized controlled trials in primary care: An introduction

Background: Cluster randomized trials occur when groups or clusters of individuals, rather than the individuals themselves, are randomized to intervention and control groups and outcomes are measured on individuals within those clusters. Within primary care, between 1997 and 2000, there has been a virtual doubling in the number of published cluster randomized trials. A recent systematic review, specifically within primary care, found study quality to be both generally lower than that reported elsewhere and not to have shown any recent quality improvement. Objective: To discuss the design, conduct and analysis of cluster randomized trials within primary care in terms of the appropriate expertise required, potential bias, ethical considerations and expense. Discussion: Compared with trials that involve the randomization of individual participants, cluster randomized trials are more complex to design and analyse and, for a given sample size, have decreased power and a broadening of confidence intervals. Cluster randomized trials are specifically prone to potential bias at two levels—the cluster and individual. Regarding the former, it is recommended that cluster allocation be undertaken by a party independent to the research team and careful consideration be given to ensure minimal cluster attrition. Bias at the individual level can be overcome by identifying trial participants before randomization and at this time obtaining consent for intervention, data collection or both. A unique ethical aspect to cluster randomized trials is that cluster leaders may consent to the trial on behalf of potential cluster members. Additional costs of cluster randomized trials include the increased number of patients required, the complexity in their design and conduct and, usually, the need to recruit clusters de novo.

Conclusion: Cluster randomized trials are a powerful and increasingly popular research tool. They are uniquely placed for the conduct of research within primary-care clusters where intracluster contamination can occur. Associated methodological issues are straightforward and surmountable and just need careful consideration and management.     

Cutoff designs for community-based intervention studies

Public health interventions are often designed to target communities defined either geographically (e.g. cities, counties) or socially (e.g. schools or workplaces). The group randomized trial (GRT) is regarded as the gold standard for evaluating these interventions. However, community leaders may object to randomization as some groups may be denied a potentially beneficial intervention. Under a regression discontinuity design (RDD), individuals may be assigned to treatment based on the levels of a pretest measure, thereby allowing those most in need of the treatment to receive it. In this article, we consider analysis, power, and sample size issues in applying the RDD and related cutoff designs in community-based intervention studies. We examine the power of these designs as a function of intraclass correlation, number of groups, and number of members per group and compare results to the traditional GRT.     

The effects of alternative study designs on the power of community randomized trials: evidence from three studies of human immunodeficiency virus prevention in East Africa

BACKGROUND: Randomized intervention trials in which the community is the unit of randomization are increasingly being used to evaluate the impact of public health interventions. In the design of community randomized trials (CRT), the power of the study is likely to be affected by two issues: the matching or stratification of communities, and the number and size of the communities to be randomized. METHODS: Data from three East African community intervention trials, designed to evaluate the impact of interventions to reduce human immunodeficiency virus (HIV) incidence, are used to compare the efficiency of different trial designs. RESULTS: Compared with an unmatched design, stratification reduced the between-community variation in the Mwanza trial (from 0.51 to 0.24) and in the Masaka trial (from 0.38 to 0.28). The reduction was smaller in the Rakai trial where the selected communities were more homogeneous (from 0.15 to 0.11). For all trials, individual matching of communities produced estimates of between-community variation similar to those from the stratified designs. The linear association between HIV prevalence and incidence was strong in the Mwanza trial (correlation coefficient R = 0.83) and the Masaka trial (R = 0.83), but weak in the Rakai trial (R = 0.28). Unmatched study designs that use smaller communities tend to increase between-community variation, but reduce the design effect and improve study power. CONCLUSIONS: These empirical data suggest that selection of homogeneous communities, or stratification of communities prior to randomization, may improve the power of CRT.     

Interval estimation for rank correlation coefficients based on the probit transformation with extension to measurement error correction of correlated ranked data

The Spearman (rho(s)) and Kendall (tau) rank correlation coefficient are routinely used as measures of association between non-normally distributed random variables. However, confidence limits for rho(s) are only available under the assumption of bivariate normality and for tau under the assumption of asymptotic normality of tau. In this paper, we introduce another approach for obtaining confidence limits for rho(s) or tau based on the arcsin transformation of sample probit score correlations. This approach is shown to be applicable for an arbitrary bivariate distribution. The arcsin-based estimators for rho(s) and tau (denoted by rho(s,a), tau(a)) are shown to have asymptotic relative efficiency (ARE) of 9/pi2 compared with the usual estimators rho(s) and tau when rho(s) and tau are, respectively, 0. In some nutritional applications, the Spearman rank correlation between nutrient intake as assessed by a reference instrument versus nutrient intake as assessed by a surrogate instrument is used as a measure of validity of the surrogate instrument. However, if only a single replicate (or a few replicates) are available for the reference instrument, then the estimated Spearman rank correlation will be downwardly biased due to measurement error. In this paper, we use the probit transformation as a tool for specifying an ANOVA-type model for replicate ranked data resulting in a point and interval estimate of a measurement error corrected rank correlation. This extends previous work by Rosner and Willett for obtaining point and interval estimates of measurement error corrected Pearson correlations.     

Methods for modelling change in cluster randomization trials

Randomized trials are often designed to assess an intervention's ability to change patient knowledge, behaviour or health. The study outcome will then need to be measured at least twice for each subject--prior to random assignment and following implementation of the intervention. In this paper we consider methods for modelling change when data are obtained from cluster randomization trials where the unit of allocation is a family, school or community. Attention focuses on mixed effects linear regression extensions of (i) two-sample t-tests and (ii) analysis of covariance, in both cases accounting for dependencies among cluster members. Algebraic expressions for tests of the intervention effect are derived for the special case where there are a fixed number of subjects per cluster while simulation studies are used to compare the power of these procedures in the more realistic case where there is variability in cluster size. A key conclusion is that there can be considerable gains in power when allowing for different individual-level and cluster-level associations between the baseline and follow-up assessments. The discussion is illustrated using data from a school-based smoking prevention trial.     

The effect of cluster randomization on sample size in prevention research

BACKGROUND: This paper concerns the issue of cluster randomization in primary care practice intervention trials. We present information on the cluster effect of measuring the performance of various preventive maneuvers between groups of physicians based on a successful trial. We discuss the intracluster correlation coefficient of determining the required sample size and the implications for designing randomized controlled trials where groups of subjects (e.g., physicians in a group practice) are allocated at random. METHODS: We performed a cross-sectional study involving data from 46 participating practices with 106 physicians collected using self-administered questionnaires and a chart audit of 100 randomly selected charts per practice. The population was health service organizations (HSOs) located in Southern Ontario. We analyzed performance data for 13 preventive maneuvers determined by chart review and used analysis of variance to determine the intraclass correlation coefficient. An index of "up-to-datedness" was computed for each physician and practice as the number of a recommended preventive measure done divided by the number of eligible patients. An index called "inappropriateness" was computed in the same manner for the not-recommended measures. The intraclass correlation coefficients for 2 key study outcomes (up-to-datedness and inappropriateness) were also calculated and compared. RESULTS: The mean up-to-datedness score for the practices was 53.5% (95% confidence interval [CI], 51.0%-56.0%), and the mean inappropriateness score was 21.5% (95% CI, 18.1%-24.9%). The intraclass correlation for up-to-datedness was 0.0365 compared with inappropriateness at 0.1790. The intraclass correlation for preventive maneuvers ranged from 0.005 for blood pressure measurement to 0.66 for chest radiographs of smokers, and as a consequence required the sample size ranged from 20 to 42 physicians per group. CONCLUSIONS: Randomizing by practice clusters and analyzing at the level of the physician has important implications for sample size requirements. Larger intraclass correlations indicate interdependence among the physicians within a cluster; as a consequence, variability within clusters is reduced, and the required sample size increased. The key finding that many potential outcome measures perform differently in terms of the intracluster correlation reinforces the need for researchers to carefully consider the selection of outcome measures and adjust sample sizes accordingly when the unit of analysis and randomization are not the same.     

Accounting for cluster randomization: a review of primary prevention trials, 1990 through 1993.

OBJECTIVES. This methodological review aims to determine the extent to which design and analysis aspects of cluster randomization have been appropriately dealt with in reports of primary prevention trials. METHODS. All reports of primary prevention trials using cluster randomization that were published from 1990 to 1993 in the American Journal of Public Health and Preventive Medicine were identified. Each article was examined to determine whether cluster randomization was taken into account in the design and statistical analysis. RESULTS. Of the 21 articles, only 4 (19%) included sample size calculations or discussions of power that allowed for clustering, while 12 (57%) took clustering into account in the statistical analysis. CONCLUSIONS. Design and analysis issues associated with cluster randomization are not recognized widely enough. Reports of cluster randomized trials should include sample size calculations and statistical analyses that take clustering into account, estimates of design effects to help others planning trials, and a table showing the baseline distribution of important characteristics by intervention group, including the number of clusters and average cluster size for each group.     

Sample size considerations for stratified cluster randomization design with binary outcomes and varying cluster size

Stratified cluster randomization trials (CRTs) have been frequently employed in clinical and healthcare research. Comparing with simple randomized CRTs, stratified CRTs reduce the imbalance of baseline prognostic factors among different intervention groups. Due to the popularity, there has been a growing interest in methodological development on sample size estimation and power analysis for stratified CRTs; however, existing work mostly assumes equal cluster size within each stratum and uses multilevel models. Clusters are often naturally formed with random sizes in CRTs. With varying cluster size, commonly used ad hoc approaches ignore the variability in cluster size, which may underestimate (overestimate) the required number of clusters for each group per stratum and lead to underpowered (overpowered) clinical trials. We propose closed-form sample size formulas for estimating the required total number of subjects and for estimating the number of clusters for each group per stratum, based on Cochran-Mantel-Haenszel statistic for stratified cluster randomization design with binary outcomes, accounting for both clustering and varying cluster size. We investigate the impact of various design parameters on the relative change in the required number of clusters for each group per stratum due to varying cluster size. Simulation studies are conducted to evaluate the finite-sample performance of the proposed sample size method. A real application example of a pragmatic stratified CRT of a triad of chronic kidney disease, diabetes, and hypertension is presented for illustration.     

Some considerations for the planning of total-community prevention trials-when is sample size adequate?

Despite the large accumulated experience of statisticians with sample-size calculations for clinical trials, little information is available on extending this methodology tototal community trials, in which the units of randomization are total communities and surveillance methods are used to assess event rates. As in clinical trials, the asymptotic formula for total sample size is used. However, the assumptions underlying the usual method of computing the expected T-year even rate for the experimental group, p_e, are no longer valid in total community trials: all emigrants (dropouts) from large communities cannot practically be identified or followed. Immigrants, similar to dropouts in clinical trials in that they are exposed to the treatment for only a portion of the period but are followed to the end of the study, present an additional problem. This paper presents a method for the computation of p_e in total community trials, taking into account in- and outmigration as well as the determinants usually considered in clinical trials. Sample computations are presented, and general problems of design, execution and data analysis of total community prevention trials are briefly discussed.Presented at the 10th Annual Meeting of the Society for Epidemiologic Research, Seattle, WA, June 15, 1977. This work was supported in part by NIH Research Career Development Award #HL-00329 (Richard F. Gillum, M.D.)     

The merits of breaking the matches: a cautionary tale

Matched-pair cluster randomization trials are frequently adopted as the design of choice for evaluating an intervention offered at the community level. However, previous research has demonstrated that a strategy of breaking the matches and performing an unmatched analysis may be more efficient than performing a matched analysis on the resulting data, particularly when the total number of communities is small and the matching is judged as relatively ineffective.The research concerning this question has naturally focused on testing the effect of intervention. However, a secondary objective of many community intervention trials is to investigate the effect of individual-level risk factors on one or more outcome variables. Focusing on the case of a continuous outcome variable, we show that the practice of performing an unmatched analysis on data arising from a matched-pair design can lead to bias in the estimated regression coefficient, and a corresponding test of significance which is overly liberal. However, for large-scale community intervention trials, which typically recruit a relatively small number of large clusters, such an analysis will generally be both valid and efficient.We also consider other approaches to testing the effect of an individual-level risk factor in a matched-pair cluster randomization design, including a generalized linear model approach that preserves the matching, a two-stage cluster-level analysis, and an approach based on generalized estimating equations.     

A method to estimate the variance of an endpoint from an on-going blinded trial

Blinded estimation of variance allows for changing the sample size without compromising the integrity of the trial. Some of the methods that estimate the variance in a blinded manner either make untenable assumptions or are only applicable to two-treatment trials. We propose a new method for continuous endpoints that makes minimal assumptions. The method uses the enrollment order of subjects and the randomization block size to estimate the variance. It can be applied to normal or non-normal data, trials with two or more treatments, equal or unequal allocation schemes, fixed or random randomization block sizes, and single or multi-centre trials. The variance estimator is unbiased and performs best when the randomization block size is the smallest. Simulation results suggest that for many commonly used randomization block sizes the proposed estimator is expected to perform well. The proposed method is used to estimate the variance of the endpoint for two trials and is shown to perform well by comparison with its unblinded counterpart.     

Clinical trials of cancer prevention agents: true versus observed prevention effect

Important considerations in the design and conduct of cancer and other long-term prevention trials include intervention compliance and frequency of events during follow-up. Each of these can produce observed proportions of events for control and test groups that differ less than anticipated. Methods and approximate 'rules of thumb' are presented that relate power and sample size to the impact of compliance and latent intervention effect. Results are based upon specific modifications in well-known formulae that reflect long-term prevention trials. Data from an ongoing cancer prevention trial illustrate the methods. Compliance was estimated accurately using the low cost capsule count method.     

Rank tests for clustered survival data when dependent subunits are randomized

In clustered survival data, subunits within each cluster share similar characteristics, so that observations made from them tend to be positively correlated. In clinical trials, the correlated subunits from the same cluster are often randomized to different treatment groups. In this case, the variance formulas of the standard rank tests such as the logrank, Gehan-Wilcoxon or Prentice-Wilcoxon, proposed for independent samples, need to be adjusted for intracluster correlations both within and between treatment groups for testing equality of marginal survival distributions. In this paper we derive a general form of simple variance formulas of the rank tests when subunits from the same cluster are randomized into different treatment groups. Extensive simulation studies are conducted to investigate small sample performance of the variance formulas. We compare our non-parametric rank tests based on the adjusted variances with one from a shared frailty model, which is an optimal semi-parametric testing procedure when the intracluster correlations within and between groups are the same.     

Sample size calculation for cluster randomization trials with a time-to-event endpoint

Cluster randomization trials randomize groups (called clusters) of subjects (called subunits) between intervention arms, and observations are collected from each subject. In this case, subunits within each cluster share common frailties, so that the observations from subunits of each cluster tend to be correlated. Oftentimes, the outcome of a cluster randomization trial is a time-to-event endpoint with censoring. In this article, we propose a closed form sample size formula for weighted rank tests to compare the marginal survival distributions between intervention arms under cluster randomization with possibly variable cluster sizes. Extensive simulation studies are conducted to evaluate the performance of our sample size formula under various design settings. Real study examples are taken to demonstrate our method.     

Stochastic variation in network epidemic models: implications for the design of community level HIV prevention trials

Important sources of variation in the spread of HIV in communities arise from overlapping sexual networks and heterogeneity in biological and behavioral risk factors in populations. These sources of variation are not routinely accounted for in the design of HIV prevention trials. In this paper, we use agent‐based models to account for these sources of variation. We illustrate the approach with an agent‐based model for the spread of HIV infection among men who have sex with men in South Africa. We find that traditional sample size approaches that rely on binomial (or Poisson) models are inadequate and can lead to underpowered studies. We develop sample size and power formulas for community randomized trials that incorporate estimates of variation determined from agent‐based models. We conclude that agent‐based models offer a useful tool in the design of HIV prevention trials. Copyright © 2014 John Wiley & Sons, Ltd.     

Randomizing patients by family practice: sample size estimation,intracluster correlation and data analysis

BACKGROUND: Cluster randomized controlled trials increasingly are used to evaluate health interventions where patients are nested within larger clusters such as practices, hospitals or communities. Patients within a cluster may be similar to each other relative to patients in other clusters on key variables; therefore, sample size calculations and analyses of results require special statistical methods. OBJECTIVE: The purpose of this study was to illustrate the calculations used for sample size estimation and data analysis and to provide estimates of the intraclass correlation coefficients (ICCs) for several variables using data from the Seniors Medication Assessment Research Trial (SMART), a community-based trial of pharmacists consulting to family physicians to optimize the drug therapy of older patients. METHODS: The study was a paired cluster randomized trial, where the family physician's practice was the cluster. The sample size calculation was based on a hypothesized reduction of 15% in mean daily units of medication in the intervention group compared with the control group, using an alpha of 0.05 (one-tailed) with 80% power, and an ICC from pilot data of 0.08. ICCs were estimated from the data for several variables. The analyses comparing the two groups used a random effects model for a meta-analysis over pairs. RESULTS: The design effect due to clustering was 2.12, resulting in an inflation in sample size from 340 patients required using individual randomization, to 720 patients using randomization of practices, with 15 patients from each of 48 practices. ICCs for medication use, health care utilization and general health were <0.1; however, the ICC for mean systolic blood pressure over the trial period was 0.199. CONCLUSIONS: Compared with individual randomization, cluster randomization may substantially increase the sample size required to maintain adequate statistical power. The differences in ICCs among potential outcome variables reinforce the need for valid estimates to ensure proper study design.     

Methodological considerations on the design and analysis of an equivalence stratified cluster randomization trial

The World Health Organization and collaborating institutions in four developing countries have conducted a multi-centre randomized controlled trial, in which clinics were allocated at random to two antenatal care (ANC) models. These were the standard 'Western' ANC model and a 'new' ANC model consisting of tests, clinical procedures and follow-up actions scientifically demonstrated to be effective in improving maternal and newborn outcomes. The two models were compared using the equivalence approach. This paper discusses the implications of the equivalence approach in the sample size calculation, analysis and interpretation of results of this cluster randomized trial. It reviews the ethical aspects regarding informed consent, concluding that the Zelen design has a place in cluster randomization trials. It describes the estimation of the intracluster correlation coefficient (ICC) in a stratified cluster randomized trial using two methods and reports estimates of the ICC obtained for many maternal, newborn and perinatal outcomes. Finally, it discusses analytical problems that arose: issues encountered using a composite index, heterogeneity of the intervention effect across sites, the choice of the method of analysis and the importance of efficacy analyses. The choice of the clustered Woolf estimator and the generalized estimating equations (GEE) as the methods of analysis applied is discussed.