Intraclass correlation coefficients for cluster randomized trials in primary care: the cholesterol education and research trial (CEART)

Cluster randomization trials are increasingly being used in primary care research. The main feature of these trials is that patients are nested within large clusters such as physician practices or communities and the intervention is applied to the cluster. This study design necessitates calculation of intraclass correlation coefficients in order to determine the required sample size. The purpose of this study is to determine intraclass correlation coefficients for a number of outcome measures at the primary care practice level. The CEART study is a randomized trial testing the effectiveness of translating ATP III guidelines into clinical practice, with primary care physician practices as the unit of randomization and patients as the unit of data collection. The intraclass correlation coefficient (ICC) was<0.02 and the design effect ranged from 1.0 to 2.3, respectively, for weight, total cholesterol, LDL, non-HDL, glucose, creatinine, and % at non-HDL goal. For smoking status, body mass index, systolic blood pressure, HDL cholesterol triglycerides, total cholesterol/HDL ratio and % at LDL goal, the ICC was 0.02-0.047 and the design effect was 2.6-4.1. The largest ICCs (0.05-0.12) and design effects (4.4-9.4) were found for height and diastolic blood pressure. These findings suggest that cluster randomization may substantially increase the sample size necessary to maintain adequate statistical power for selected outcomes such as diastolic blood pressure studies compared with simple randomization for most outcomes evaluated in this study where the design effect is small to moderate. Overall, the ICCs presented will be useful in calculating sample sizes at the primary care level.     

IRINI: random group allocation of multiple prognostic factors

Statistically sound experimental design in pharmacology studies ensures that the known prognostic factors, if any, are equally represented across investigational groups to avoid bias and imbalance which could render the experiment invalid or lead to false conclusions. Complete randomization can be effective to reduce bias in the created groups especially in large sample size situations. However, in small studies which involve only few treatment subjects, as in preclinical trials, there is a high chance of imbalance. The effects of this imbalance may be removed through covariate analysis or prevented with stratified randomization, however small studies limit the number of covariates to be analyzed this way. The problem is accentuated when there are multiple baseline covariates with varying scales and magnitudes to be considered in the randomization, and creating a balanced solution becomes a combinatorial challenge. Our method, IRINI, uses an optimization technique to achieve treatment to subject group allocation across multiple prognostic factors concurrently. It ensures that the created groups are equal in size and statistically comparable in terms of mean and variance. This method is a novel application of genetic algorithms to solve the allocation problem and simultaneously ensure quality, speed of the results and randomness of the process. Results from preclinical trials demonstrate the effectiveness of the method.     

The advantages of community-randomized trials for evaluating lifestyle modification

Observational studies may provide suggestive evidence for the results of behavior change and lifestyle modification, but they do not replace randomized trials for comparing interventions. To obtain a valid comparison of competing intervention strategies, randomized trials of adequate size are the recommended approach. Randomization avoids bias, achieves balance (on average) of both known and unknown predictive factors between intervention and comparison groups, and provides the basis of statistical tests. The value of randomization is as relevant when investigating community interventions as it is for studies that are directed at individuals. Randomization by group is less efficient statistically than randomization by individual, but there are reasons why randomization by group (such as community) may be chosen, including feasibility of delivery of the intervention, political and administrative considerations, avoiding contamination between individuals allocated to competing interventions, and the very nature of the intervention. One example is the Community Intervention Trial for Smoking Cessation (COMMIT), which involved 11 matched pairs of communities and randomized within these pairs to active community-level intervention versus comparison. For analysis of results, community-level permutation tests (and corresponding test-based confidence intervals) can be designed based on the randomization distribution. The advantages of this approach are that it is robust, and the unit of randomization is the unit of analysis, yet it can incorporate individual-level covariates. Such covariates can play a role in imputation for missing values, adjustment for imbalances, and separate analyses in demographic subsets (with appropriate tests for interaction). A communityrandomized trial can investigate a multichannel community-based approach to lifestyle modification, thus providing generalizability coupled with a rigorous evaluation of the intervention.     

Properties of the urn randomization in clinical trials

In this article we review the important statistical properties of the urn randomization (design) for assigning patients to treatment groups in a clinical trial. The urn design is the most widely studied member of the family of adaptive biased-coin designs. Such designs are a compromise between designs that yield perfect balance in treatment assignments and complete randomization which eliminates experimental bias. The urn design forces a small-sized trial to be balanced but approaches complete randomization as the size of the trial (n) increases. Thus, the urn design is not as vulnerable to experimental bias as are other restricted randomization procedures. In a clinical trial it may be difficult to postulate that the study subjects constitute a random sample from a well-defined homogeneous population. In this case, a randomization model provides a preferred basis for statistical inference. We describe the large-sample permutational null distributions of linear rank statistics for testing the equality of treatment groups based on the urn design. In general, these permutation tests may be different from those based on the population model, which is equivalent to assuming complete randomization. Poststratified subgroup analyses can also be performed on the basis of the urn design permutational distribution. This provides a basis for analyzing the subset of patients with observed responses when some patients' responses can be assumed to be missing-at-random. For multiple mutually exclusive strata, these tests are correlated. For this case, a combined covariate-adjusted test of treatment effect is described. Finally, we show how to generalize the urn design to a prospectively stratified trial with a fairly large number of strata.     

Imbalance in treatment assignments in stratified blocked randomization

Blocking and stratification are used in preparing randomization assignments to ensure that there will be nearly equal numbers of patients in each treatment group and that the groups will be similar with respect to important covariates. Stratified blocked randomization will create near balance within strata, but imbalance for the total trial may still occur. The variance for the total trial imbalance D is derived and examples from clinical trials are given. Under reasonable assumptions, if the blocking factor is size B in each of K strata, then max D = KB/2 and var D = K(B + 1)/6. These results may be used in planning a trial to estimate the overall imbalance expected for various choices of B and K. A conditional variance is given that allows the probability of an observed imbalance at the completion of a trial to be evaluated. Overall imbalance is about as likely with stratified blocked randomization as with simple randomization unless the total sample size N is appreciably larger than K X B. So long as the blinding is maintained, the block sizes should be chosen to be as small as possible.     

Properties of simple randomization in clinical trials

This article presents the properties of complete randomization (e.g., coin toss) and of the random allocation rule (random permutation of n/2 of n elements). The latter is principally used in cases where the total sample size n is known exactly a priori. The likelihood of treatment imbalances is readily computed and is shown to be negligible for large trials (n greater than 200), regardless of whether a stratified randomization is used. It is shown that substantial treatment imbalances are extremely unlikely in large trials, and therefore there is likely to be no substantial effect on power. The large-sample permutational distribution of the family of linear rank tests is presented for complete randomization unconditionally and conditionally, and for the random allocation rule. Asymptotically the three are equivalent to the distribution of these tests under a sampling-based population model. Permutation tests are also presented for a stratified analysis within one or more subgroups of patients defined post hoc on the basis of a covariate. This provides a basis for analysis when some patients' responses are assumed to be missing-at-random. Using the Blackwell-Hodges model, it is shown that complete randomization eliminates the potential for selection bias, but that the random allocation rule yields a substantial potential for selection bias in an unmasked trial. Finally, the Efron model for accidental bias is used to assess the potential for bias in the estimate of treatment effect due to covariate imbalance. Asymptotically, this probability approaches zero for complete randomization and for the random allocation rule. However, for finite n, complete randomization minimizes the probability of accidental bias, whereas this probability is slightly higher with a random allocation rule. It is concluded that complete randomization has merit in large clinical trials.     

Randomizing research participants: Promoting balance and concealment in small samples

Randomization is central to rigorous scientific trials. An effective but underutilized approach is urn randomization. To test the ability of urn randomization versus simple randomization to produce balanced groups with small sample sizes, we conducted simulated randomizations: 10 times under sample size scenarios of 20, 40, 60 (group sizes of 10, 20, and 30, respectively), for 30 trials in total. For groups of 20–30, urn surpassed simple randomization in the equal distribution of confounding variables between groups, leading to effects of these variables that were both smaller on average and more consistently close to zero over multiple trials. The urn method is easy to implement and has the advantages of unpredictability of assignment and decreased susceptibility to investigator bias. © 2010 Wiley Periodicals, Inc. Res Nurs Health 33:243–253, 2010     

Statistical comparison of random allocation methods in cancer clinical trials

The selection of a trial design is an important issue in the planning of clinical trials. One of the most important considerations in trial design is the method of treatment allocation and appropriate analysis plan corresponding to the design. In this article, we conducted computer simulations using the actual data from 2158 rectal cancer patients enrolled in the surgery-alone group from seven randomized controlled trials in Japan to compare the performance of allocation methods, simple randomization, stratified randomization and minimization in relatively small-scale trials (total number of two groups are 50, 100, 150 or 200 patients). The degree of imbalance in prognostic factors between groups was evaluated by changing the allocation probability of minimization from 1.00 to 0.70 by 0.05. The simulation demonstrated that minimization provides the best performance to ensure balance in the number of patients between groups and prognostic factors. Moreover, to achieve the 1 percentile for the p-value of chi-square test around 0.50 with respect to balance in prognostic factors, the allocation probability of minimization was required to be set to 0.95 for 50, 0.80 for 100, 0.75 for 150 and 0.70 for 200 patients. When the sample size was larger, sufficient balance could be achieved even if reducing allocation probability. The simulation using actual data demonstrated that unadjusted tests for the allocation factors resulted in conservative type I errors when dynamic allocation, such as minimization, was used. In contrast, adjusted tests for allocation factors as covariates improved type I errors closer to the nominal significance level and they provided slightly higher power. In conclusion, both the statistical and clinical validity of minimization was demonstrated in our study.     

Reading a research article part II: parametric and nonparametric statistics

Researchers often try to use a randomization technique in an attempt to reduce bias and ensure that treatment and control groups are as similar as possible. This article has provided an overview of how researchers might use parametric and nonparametric statistics when analyzing data and looking for differences between groups. Researchers must consider the types of data and choose the tests that are appropriate for the variable types to draw appropriate conclusions. The next article in this series will address comparison of more than two groups and repeated measures and other design issues.     

Randomizing responders. Cardiac Arrhythmia Suppression Trial (CAST) investigators.

Three clinical trial designs for use in testing the effect of long-term drug therapy on an outcome are considered: empiric (randomization to one of several specified and fixed therapies); randomization followed by dose-adjustment of the drug; and dose adjustment followed by randomization of responders. It is shown that the latter, though some information may be lost and bias may be introduced, can be more efficient because of the reduction of noise. These results are illustrated using information gathered by the Cardiac Arrhythmia Pilot Study (CAPS) and Cardiac Arrhythmia Suppression Trial (CAST), the former being a pilot study using the second design, and the latter a large clinical trial adopting the third design. For CAST, the efficiency (represented by ratio of sample sizes) is 0.78 relative to the empiric design, and 0.6 relative to the design incorporating randomization followed by dose adjustment.     

The impact of screening and eliminating preexisting cases on sample size requirements for cancer prevention trials

A cancer prevention trial may seek to test the effectiveness of an intervention in disease-free individuals, but the study population may include subjects with preexisting (but undiagnosed) disease. If sample size calculations assume all events are incident cases, the study will have less power than anticipated because preexisting cases cannot be expected to benefit from the intervention. Sample size can be increased appropriately by using revised event rates that include both preexisting and incident cases. These rates incorporate screening parameters and are applicable to the situation where subjects are screened before randomization. A simple cost model is given that permits examination of the tradeoffs involved in prescreening subjects versus increasing total sample size.     

Small N designs for rehabilitation research

Rehabilitation research presents unique and challenging problems to investigators during both the design and analysis periods. Statistical issues regarding sample size requirements for an adequately powered study may be in direct conflict with realistic recruitment and subject retention goals. Issues of underpowered studies, sample size requirements, and recruitment goals plague rehabilitation research. Randomized clinical trials (RCTs) are typically narrow in scope and thus lack generalizability to everyday, yet specific, clinical problems; they are also costly and time-consuming and require large numbers of participants for randomization to have optimal, desired effects. Further, the RCT design may not be applicable to assistive technologies and environmental modifications-vital components of disability and rehabilitation research-nor is it appropriate in situations in which theoretical models of change are lacking or premature. Single-case designs are better suited for studies in which understanding and changing patient behavior and functional status are primary goals and the targeted sample sizes are less than 30 and frequently less than 10. Theoretical, methodological, and clinical reasons for using experimental and quasi-experimental single-case designs are presented. Recommendations for designing and conducting single-case studies that contribute to the evidence base are also discussed.     

Lack of concealment may lead to selection bias in cluster randomized trials of palliative care

Comprehensive palliative care programs are often implemented on a community level, and to evaluate such interventions, randomization by cluster (community) may be the only feasible method. In trials randomizing individual subjects, the importance of proper concealment has been stressed. In cluster randomized trials, however, concealment of individual patient allocation is often impossible. The following risk of selection bias has been given little attention. In the present study, comparing palliative care to conventional care, community health care districts were defined as clusters and randomized. The patients' treatment assignment was determined by the allocation of the cluster in which they resided, and hence predictable by their address. A biased selection based on practical considerations related to patients' diagnoses and hospital departments was suspected. To explore this, cancer diagnoses were grouped according to local tradition for sharing of treatment responsibility among hospital departments. A significant difference between trial arms in distribution of these groups was revealed and strongly supported our suspicion. The finding carries an important message to future researchers: when using cluster randomization, any evidence of selection bias should be carefully checked and reported.     

Advanced Statistics:Statistical Methods for Analyzing Cluster and Cluster-randomized Data

Sometimes interventions in randomized clinical trials are not allocated to individual patients, but rather to patients in groups. This is called cluster allocation, or cluster randomization, and is particularly common in health services research. Similarly, in some types of observational studies, patients (or observations) are found in naturally occurring groups, such as neighborhoods. In either situation, observations within a cluster tend to be more alike than observations selected entirely at random. This violates the assumption of independence that is at the heart of common methods of statistical estimation and hypothesis testing. Failure to account for the dependence between individual observations and the cluster to which they belong can have profound implications on the design and analysis of such studies. Their p-values will be too small, confidence intervals too narrow, and sample size estimates too small, sometimes to a dramatic degree. This problem is similar to that caused by the more familiar "unit of analysis error" seen when observations are repeated on the same subjects, but are treated as independent. The purpose of this paper is to provide an introduction to the problem of clustered data in clinical research. It provides guidance and examples of methods for analyzing clustered data and calculating sample sizes when planning studies. The article concludes with some general comments on statistical software for cluster data and principles for planning, analyzing, and presenting such studies.     

On measurements and their quality: Paper 2: Random measurement error and the power of statistical tests

This is the second in a short series of papers on measurement theory and practice with particular relevance to intervention research in nursing, midwifery, and healthcare. This paper begins with an illustration of how random measurement error decreases the power of statistical tests and a review of the roles of sample size and effect size in hypothesis testing. A simple formula is presented and discussed for calculating sample size during the planning stages of intervention studies. Finally, an approach for incorporating reliability estimates into a priori power analyses is introduced and illustrated with a practical example. The approach permits researchers to compare alternative study designs, in terms of their statistical power. An SPSS program is provided to facilitate this approach and to assist researchers in making optimal decisions when choosing among alternative study designs.     

Sample size determination under an exponential model in the presence of a confounder and type I censoring.

In controlled clinical trials, random assignment of treatments to individuals is usually used to eliminate the effects of confounding variables. When there is censorship in data, however, confounding effects may not be automatically removed solely by random assignment of treatments to individuals under the exponential model. Therefore, it is important to incorporate the confounding effect into the sample size calculation even after randomization of treatments to individuals. In this paper, the discussion is restricted only to the situation where there are two comparison groups and one single Bernoulli confounding variable. Based on an exponential covariate model, an explicit sample size formula considering the confounding effect has been derived for the design of trials with type I censoring, in which an end time is fixed in advance and all responses occurring after that time are censored. The resulting sample size formula can also be applied to nonrandomized clinical trials. Finally, to provide insight into the influence of different factors on sample size calculation, a discussion on the effects of treatments, the confounder, the length of follow-up times for studied individuals, and the joint distribution of the treatment and the confounder has been included.     

Randomized clinical trials in medical rehabilitation research.

The purpose of this article is to provide researchers and clinicians with a basic understanding of randomized clinical trials and to discuss their potential application to and limitations in the field of physical medicine and rehabilitation. A brief history of the development of randomized clinical trials, definitions of clinical trials, types of trials, and overview of methodological issues related to design are offered. Information is provided about the need to establish clear and concise study objectives and to explicitly define interventions and expected outcomes. Recommendations for developing clinical protocols and determining adequate sample size are presented, and various statistical considerations, including power, are discussed. Issues related to sampling strategies, and recruitment are reviewed. Importance of randomization and blinding is emphasized. Readers are also referred to other resources available on this topic. Finally, the authors describe shortfalls associated with the use of this design in rehabilitation research. These are further explored and discussed in terms of the actual benefits and limitations of randomized clinical trials in physical medicine and rehabilitation research. Recommendations are made regarding the use of this methodology to address relevant needs in clinical practice.     

An application of a modified constrained randomization process to a practice-based cluster randomized trial to improve colorectal cancer screening

BackgroundWhen designing cluster randomized trials, it is important for researchers to be familiar with strategies to achieve valid study designs given limited resources. Constrained randomization is a technique to help ensure balance on pre-specified baseline covariates.MethodsThe goal was to develop a randomization scheme that balanced 16 intervention and 16 control practices with respect to 7 factors that may influence improvement in study outcomes during a 4-year cluster randomized trial to improve colorectal cancer screening within a primary care practice-based research network. We used a novel approach that included simulating 30,000 randomization schemes, removing duplicates, identifying which schemes were sufficiently balanced, and randomly selecting one scheme for use in the trial. For a given factor, balance was considered achieved when the frequency of each factor's sub-classifications differed by no more than 1 between intervention and control groups. The population being studied includes approximately 32 primary care practices located in 19 states within the U.S. that care for approximately 56,000 patients at least 50 years old.ResultsOf 29,782 unique simulated randomization schemes, 116 were determined to be balanced according to pre-specified criteria for all 7 baseline covariates. The final randomization scheme was randomly selected from these 116 acceptable schemes.ConclusionsUsing this technique, we were successfully able to find a randomization scheme that allocated 32 primary care practices into intervention and control groups in a way that preserved balance across 7 baseline covariates. This process may be a useful tool for ensuring covariate balance within moderately large cluster randomized trials.     

Reporting of clinical details in randomized controlled trials of acupuncture for the treatment of migraine/headaches and nausea/vomiting

OBJECTIVES: The main purpose of this study was to investigate the degree of reporting of clinical details in a selected sample of 30 randomized control trials (RCT) on acupuncture efficacy for the treatment of migraine/headaches (n = 11) and for nausea/vomiting (n = 19). Additional goals included the assessment of the quality of randomization of the trials, and the assessment of the degree of reporting of information about the outcome measures used in each trial. METHODS: A checklist of 50 clinical details selected on the basis of a previous survey was used to evaluate the degree of reporting of information about patients, practitioners, diagnosis, and acupuncture treatments presented in each RCT. The likelihood of bias in the randomization process was assessed using a previously validated scale. Information about reliability/validity and clinical significance of the outcome measures used in each trial was assessed in terms of fully, partially, or not reported, with no attempt to evaluate their quality or appropriateness. RESULTS: In this sample of RCTs an average of 38.7% of important clinical details per trial were either fully or partially reported by researchers, the lowest being 26.4% for the migraine/headaches group. Studies with better quality of randomization were not more likely to report important clinical details. Only five studies (16.7%) provided information about reliability and validity of the outcome measures used, and only four studies (13.3%), all from the migraines/headache group, discussed the clinical significance of the outcome measures selected. CONCLUSIONS: In this sample of 30 RCTs of acupuncture efficacy for the treatment of migraines/headache and nausea/vomiting, researchers neglected to report adequately on important clinical details, and often did not discuss the reliability, validity, and clinical significance of the outcome measures used in the trials, thus rendering potential readers of the articles unable to critically appraise them from a clinical standpoint. In addition, the fact that the quality of randomization of the trials was totally unrelated to the degree of reporting of clinical details renders potential reviewers of these studies unable to establish valid conclusions about acupuncture efficacy based on the general quality of the methodological design. In the future, all areas of clinical acupuncture research need to be reviewed in a similar manner, and recommendations about proper reporting of important clinical details and proper discussion of the validity, reliability, and clinical significance of the outcome measures used in each trial should be made. Only then could this research be used to generate meaningful evidence-based recommendations for the contemporary practice of acupuncture.     

Emergency Medicine Animal Research: Does Use of Randomization and Blinding Affect the Results?

OBJECTIVES: It has been shown that human clinical trials that lack randomization (RND) or blinding (BLD) often overestimate the magnitude of treatment effects. However, no studies have evaluated the effect of RND and BLD on animal research. The authors' objectives were to determine the proportion of animal studies presented at a national academic emergency medicine meeting that utilize randomization, blinding, or both; and to determine whether failure to employ these techniques changes the likelihood of observing a difference between treatment groups. METHODS: Two trained researchers reviewed abstracts presented at the 1997-2001 Society for Academic Emergency Medicine (SAEM) annual meetings using a standard data collection sheet. Studies that used an animal or cell line, compared two or more study groups, and measured an effect caused by the intervention or drugs were included. Studies were classified as randomized (RND+) if any part of the experiment involved random assignment of subjects to treatment groups, blinded (BLD+) if any assessment of the outcome was made by an investigator blinded to treatment group, and outcome-positive (Outcome+) if any difference between the study groups met the author's definition of significant. Following the initial review, differences in classification were resolved by consensus. The association between outcome and study methodology (RND, BLD or both) was measured using odds ratios (ORs) with 95% confidence intervals (95% CIs). RESULTS: A total of 2,592 studies were published as abstracts. Three hundred eighty-nine were animal studies, and 290 of these studies had two or more study groups. RND- and BLD- studies were more likely to be Outcome+ than RND+ or BLD+ studies (OR = 3.4; 95% CI = 1.7 to 6.9 and OR = 3.2; 95% CI = 1.3 to 7.7, respectively). When studies that used both RND and BND were compared with studies that used neither, the OR for a positive study was 5.2 (95% CI = 2.0 to 13.5). CONCLUSIONS: These results suggest that animal studies that do not utilize RND and BLD are more likely to report a difference between study groups than studies that employ these methods.