Cluster randomized controlled trials in primary care: An introduction

Background: Cluster randomized trials occur when groups or clusters of individuals, rather than the individuals themselves, are randomized to intervention and control groups and outcomes are measured on individuals within those clusters. Within primary care, between 1997 and 2000, there has been a virtual doubling in the number of published cluster randomized trials. A recent systematic review, specifically within primary care, found study quality to be both generally lower than that reported elsewhere and not to have shown any recent quality improvement. Objective: To discuss the design, conduct and analysis of cluster randomized trials within primary care in terms of the appropriate expertise required, potential bias, ethical considerations and expense. Discussion: Compared with trials that involve the randomization of individual participants, cluster randomized trials are more complex to design and analyse and, for a given sample size, have decreased power and a broadening of confidence intervals. Cluster randomized trials are specifically prone to potential bias at two levels—the cluster and individual. Regarding the former, it is recommended that cluster allocation be undertaken by a party independent to the research team and careful consideration be given to ensure minimal cluster attrition. Bias at the individual level can be overcome by identifying trial participants before randomization and at this time obtaining consent for intervention, data collection or both. A unique ethical aspect to cluster randomized trials is that cluster leaders may consent to the trial on behalf of potential cluster members. Additional costs of cluster randomized trials include the increased number of patients required, the complexity in their design and conduct and, usually, the need to recruit clusters de novo.

Conclusion: Cluster randomized trials are a powerful and increasingly popular research tool. They are uniquely placed for the conduct of research within primary-care clusters where intracluster contamination can occur. Associated methodological issues are straightforward and surmountable and just need careful consideration and management.     

The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications

ObjectivesTo assess the proportion of clinical trials explicitly reporting the risk of unblinding, to evaluate the completeness of reporting on unblinding risk, and to describe the reported procedures involved in assessing unblinding.Study Design and SettingWe sampled at random 300 blinded randomized clinical trials indexed in PubMed in 2010. Two authors read the trial publications and extracted data independently.ResultsTwenty-four trial publications, or 8% (95% confidence interval [CI], 5, 12%), explicitly reported the risk of unblinding, of which 16 publications, or 5% (95% CI, 3, 8%), reported compromised blinding; and 8 publications, or 3% (95% CI, 1, 5%), intact blinding. The reporting on risk of unblinding in the 24 trial publications was generally incomplete. The median proportion of assessments per trial affected by unblinding was 3% (range 1–30%). The most common mechanism for unblinding was perceptible physical properties of the treatments, for example, a difference in the taste and odor of a typhoid vaccine compared with its placebo.ConclusionPublished articles on randomized clinical trials infrequently reported risk of unblinding. This may reflect a tendency for avoiding reporting actual or suspected unblinding or a genuine low risk of unblinding.     

Pragmatic trials can be designed as optimal medical care: principles and methods of care trials

ObjectivesThe way clinical research and care are currently separated encourages the practice of unverifiable medicine. Some pragmatic trials can be designed (1) to guide proper medical conduct in the presence of uncertainty and (2) to govern the distinction between unvalidated and validated care.MethodsCare trials are simple randomized trials integrated into a practice they regulate in the interest of present patients. The fundamental principle guiding the design of a care trial is the protection of the patient being offered medical care that has not yet been validated. Selection criteria are inclusive, to assist most current patients confronted with the problem. The trial entails no extra tests or risks beyond what is proven beneficial. Endpoints are pre-defined, simple, valuable and resistant to bias. Follow-up visits and tests are routine. Data is collected in simple case-report forms.ResultsCare trials protect present patients from both unverifiable medicine and research performed for extraneous interests. They provide prudent care when evidence is lacking. They should not be obstructed by the need for separate funding, or by bureaucracy.ConclusionCare trials can identify which medical alternative should be standard therapy. In the meantime, they provide optimal care in the presence of uncertainty.     

A methodological review of recent meta-analyses has found significant heterogeneity in age between randomized groups

BackgroundThere is evidence to suggest that component randomized controlled trials (RCTs) within systematic reviews may be biased. It is important that these reviews are identified to prevent erroneous conclusions influencing health care policies and decisions.PurposeTo assess the likelihood of bias in trials in 12 meta-analyses.DesignA review of 12 systematic reviews.Data SourcesTwelve recently published systematic reviews with 503 component randomized trials, published in the British Medical Journal, The Lancet, Journal of the American Medical Association, and The Annals of Internal Medicine before May 2012.Study Selection and Data ExtractionSystematic reviews were eligible for inclusion if they included only RCTs. We obtained the full text for the component RCTs of the 12 systematic reviews (in English only). We extracted summary data on age, number of participants in each treatment group, and the method of allocation concealment for each RCT.Data SynthesisFive of the 12 meta-analyses exhibited heterogeneity in age differences (I² > 0.30), when there should have been none. In two meta-analyses, the age of the intervention group was significantly greater than that of the control group. Inadequate allocation concealment was a statistically significant predictor of heterogeneity in one trial as observed by a metaregression.ConclusionsMost of the sample of recent meta-analyses showed that there were signs of imbalance and/or heterogeneity in ages between treatment groups, when there should have been none. Systematic reviewers might consider using the techniques described here to assess the validity of their findings.     

Is there a solution to publication bias? Researchers call for changes in dissemination of clinical research results

ObjectivesTo explore opinions of authors of published reports of clinical trials and Cochrane systematic reviews on the causes and methods of preventing publication bias.Study Design and SettingAn online questionnaire was developed and sent to researchers publishing in high-impact or national general medical journals, authors of Cochrane systematic reviews, and a general population of researchers. Open-ended questions about publication bias were qualitatively analyzed. We also held a focus group with experienced researchers and/or Cochrane reviewers.ResultsPublication bias was common: 48 (36%) respondents had own unpublished trials and 40 (30%) admitted selective outcome reporting; but researchers felt strongly that blame rested also with the system that promotes and augments publication bias practices. Qualitative analysis of both survey responses and focus group discussion identified possible ways of reducing publication bias through increased transparency, improvements to trial registries, search engines and databases, enhancement of the role of institutional review boards, positive encouragement of scientists, and policy changes.ConclusionAlthough well aware of the problem, clinical researchers knowingly contribute to problems of selective reporting and nonpublication of trials. They call for changes in current practices of journal-based communication of trial reporting and for systematic evaluation of measures to decrease publication bias.     

Statistical Decision Properties of Imprecise Trials Assessing Coronavirus Disease 2019 (COVID-19) Drugs

ObjectivesResearchers studying treatment of coronavirus disease 2019 (COVID-19) have reported findings of randomized trials comparing standard care with care augmented by experimental drugs. Many trials have small sample sizes, so estimates of treatment effects are imprecise. Hence, clinicians may find it difficult to decide when to treat patients with experimental drugs. A conventional practice when comparing standard care and an innovation is to choose the innovation only if the estimated treatment effect is positive and statistically significant. This practice defers to standard care as the status quo. We study treatment choice from the perspective of statistical decision theory, which considers treatment options symmetrically when assessing trial findings.MethodsWe use the concept of near-optimality to evaluate criteria for treatment choice. This concept jointly considers the probability and magnitude of decision errors. An appealing criterion from this perspective is the empirical success rule, which chooses the treatment with the highest observed average patient outcome in the trial.ResultsConsidering the design of some COVID-19 trials, we show that the empirical success rule yields treatment choices that are much closer to optimal than those generated by prevailing decision criteria based on hypothesis tests.ConclusionUsing trial findings to make near-optimal treatment choices rather than perform hypothesis tests should improve clinical decision making.     

Single-center trials tend to provide larger treatment effects than multicenter trials: a systematic review

ObjectivesTo assess whether the reported trial characteristics are associated with treatment effects on all-cause mortality within critical care medicine.Study Design and SettingWe identified all eligible randomized controlled trials (RCTs) from Cochrane Reviews on patients with sepsis, septic shock, and cardiogenic shock. Risk of bias was judged on 12 trial characteristics, including the differentiation between single-center and multicenter trials. Hierarchical random-effects models quantified the impact of the risk of bias items on the reported effect estimates of mortality.ResultsTwelve meta-analyses that involved 82 RCTs were selected and judged. Single-center trials estimated a significant larger treatment effect compared with multicenter trials (ratio of odds ratios, 0.64; 95% confidence interval: 0.47, 0.87). Treatment effect tended to be overestimated with selective reporting of preplanned end points. Biases in different trial characteristics are unlikely to operate independently and may have modified these associations.ConclusionThe results of this study highlight a substantial difference in treatment effect estimates between single-center and multicenter trials. Therefore, we recommend that results from single-center trials should be cautiously used for decision making.     

Eliminating bias in randomized controlled trials: importance of allocation concealment and masking

Randomization in randomized controlled trials involves more than generation of a random sequence by which to assign subjects. For randomization to be successfully implemented, the randomization sequence must be adequately protected (concealed) so that investigators, involved health care providers, and subjects are not aware of the upcoming assignment. The absence of adequate allocation concealment can lead to selection bias, one of the very problems that randomization was supposed to eliminate. Authors of reports of randomized trials should provide enough details on how allocation concealment was achieved so the reader can determine the likelihood of success. Fortunately, a plan of allocation concealment can always be incorporated into the design of a randomized trial. Certain methods minimize the risk of concealment failing more than others. Keeping knowledge of subjects' assignment after allocation from subjects, investigators/health care providers, or those assessing outcomes is referred to as masking (also known as blinding). The goal of masking is to prevent ascertainment bias. In contrast to allocation concealment, masking cannot always be incorporated into a randomized controlled trial. Both allocation concealment and masking add to the elimination of bias in randomized controlled trials.     

Studywise minimization: A treatment allocation method that improves balance among treatment groups and makes allocation unpredictable

ObjectivesIn randomized controlled trials with many potential prognostic factors, serious imbalance among treatment groups regarding these factors can occur. Minimization methods can improve balance but increase the possibility of selection bias. We described and evaluated the performance of a new method of treatment allocation, called studywise minimization, that can avoid imbalance by chance and reduce selection bias.Study Design and SettingThe studywise minimization algorithm consists of three steps: (1) calculate the imbalance for all possible allocations, (2) list all allocations with minimum imbalance, and (3) randomly select one of the allocations with minimum imbalance. We carried out a simulation study to compare the performance of studywise minimization with three other allocation methods: randomization, biased-coin minimization, and deterministic minimization. Performance was measured, calculating maximal and average imbalance as a percentage of the group size.ResultsIndependent of trial size and number of prognostic factors, the risk of serious imbalance was the highest in randomization and absent in studywise minimization. The largest differences among the allocation methods regarding the risk of imbalance were found in small trials.ConclusionStudywise minimization is particularly useful in small trials, where it eliminates the risk of serious imbalances without generating the occurrence of selection bias.     

Reactions to treatment debriefing among the participants of a placebo controlled trial

Background  

A significant proportion of trial participants respond to placebos for a variety of conditions. Despite the common conduct of these trials and the strong emphasis placed on informed consent, very little is known about informing participants about their individual treatment allocation at trial closure. This study aims to address this gap in the literature by exploring treatment beliefs and reactions to feedback about treatment allocation in the participants of a placebo-controlled randomized clinical trial (RCT).     

Trial sequential analyses of meta-analyses of complications in laparoscopic vs. small-incision cholecystectomy: more randomized patients are needed

ObjectiveConclusions based on meta-analyses of randomized trials carry a status of “truth.” Methodological components may identify trials with systematic errors (“bias”). Trial sequential analysis (TSA) evaluates random errors in meta-analysis. We analyzed meta-analyses on laparoscopic vs. small-incision cholecystectomy regarding different outcome measures for the occurrence of type I errors.Study Design and SettingUsing TSA, we calculated the required information size (IS) and the trial sequential monitoring boundaries regarding complications in our Cochrane review with meta-analyses of cholecystectomy. For each outcome, we calculated a low risk of bias heterogeneity-adjusted IS. As a sensitivity analysis, we calculated an a priori heterogeneity-adjusted IS.ResultsAccording to the trial sequential analyses based on a low risk of bias heterogeneity-adjusted IS definitive evidence may be reached by conducting one more randomized trial. Information may be required on 582 and 119 additional randomized patients to evaluate the effect on severe complications and serious adverse events (SAEs), respectively.ConclusionOur results provide incentives to conduct a new trial with a low risk of bias focusing on a new composite outcome measure of SAEs to obtain conclusive evidence on which operative method to recommend.     

So much at stake: Ethical tradeoffs in accelerating SARSCoV-2 vaccine development

BackgroundA sense of urgency exists to develop vaccines against SARS CoV-2, responsible for numerous global cases and deaths, as well as widespread social and economic disruption. Multiple approaches have been proposed to speed up vaccine development, including accelerated randomized controlled trials (RCT), controlled human challenge trials (CHI), and wide distribution through an emergency use authorization after collecting initial data. There is a need to examine how best to accelerate vaccine development in the setting of a pandemic, without compromising ethical and scientific norms.MethodsTrade-offs in scientific and social value between generating reliable evidence about safety and efficacy while promoting rapid vaccine availability are examined along five ethically relevant dimensions: (1) confidence in and generalizability of data, (2) feasibility, (3) speed and cost, (4) participant risks, and (5) social risks.ResultsAccelerated individually randomized RCTs permit expeditious evaluation of vaccine candidates using established methods, expertise, and infrastructure. RCTs are more likely than other approaches to be feasible, increase speed and reduce cost, and generate reliable data about safety and efficacy without significantly increasing risks to participants or undermining societal trust.ConclusionEthical analysis suggests that accelerated RCTs are the best approach to accelerating vaccine development in a pandemic, and more likely than other approaches to enhance social value without compromising ethics or science. RCTs can expeditiously collect rigorous data about vaccine safety and efficacy. Innovative and flexible designs and implementation strategies to respond to shifting incidence and test vaccine candidates in parallel or sequentially would add value, as will coordinated data sharing across vaccine trials. CHI studies may be an important complementary strategy when more is known. Widely disseminating a vaccine candidate without efficacy data will not serve the public health nor achieve the goal of identifying safe and effective SARS Co-V-2 vaccines.     

Published methodological quality of randomized controlled trials does not reflect the actual quality assessed in protocols

ObjectivesTo assess whether the reported methodological quality of randomized controlled trials (RCTs) reflects the actual methodological quality and to evaluate the association of effect size (ES) and sample size with methodological quality.Study Design and SettingSystematic review. This is a retrospective analysis of all consecutive phase III RCTs published by eight National Cancer Institute Cooperative Groups up to 2006. Data were extracted from protocols (actual quality) and publications (reported quality) for each study.ResultsFour hundred twenty-nine RCTs met the inclusion criteria. Overall reporting of methodological quality was poor and did not reflect the actual high methodological quality of RCTs. The results showed no association between sample size and actual methodological quality of a trial. Poor reporting of allocation concealment and blinding exaggerated the ES by 6% (ratio of hazard ratio [RHR]: 0.94; 95% confidence interval [CI]: 0.88, 0.99) and 24% (RHR: 1.24; 95% CI: 1.05, 1.43), respectively. However, actual quality assessment showed no association between ES and methodological quality.ConclusionThe largest study to date shows that poor quality of reporting does not reflect the actual high methodological quality. Assessment of the impact of quality on the ES based on reported quality can produce misleading results.     

Confronting publication bias: a cohort design for meta-analysis

In evaluating therapies, clinical investigators often need to rely on the published clinical trial literature which may be biased in favour of studies with positive or 'encouraging' results and this may lead to erroneous conclusions of therapeutic effectiveness. The problem of publication bias can be magnified when the evaluation is based on a pooled analysis of clinical trial results, since in this case even small differences between treatment groups may reach statistical significance. In this paper a model is developed for pooling the results of clinical trials which is free from publication bias. It is proposed that an international registry of all clinical trials be established with the objectives and endpoints of each trial clearly defined in the register. In this way for each therapeutic issue researchers can select a cohort of clinical trials independently from the trial results. The approach is illustrated using the International Cancer Research Data Bank (ICRDB) registry of cancer clinical trials to evaluate the effect of chemotherapy on survival in advanced ovarian cancer. In this example, the conclusions based on a pooled analysis of registered trials have important differences from a more traditional review of the published trials. Implications of the results and problems in implementing the model are discussed.     

Citation networks of related trials are often disconnected: implications for bidirectional citation searches

Background and ObjectivesReports of randomized controlled trials (RCTs) should set findings within the context of previous research. The resulting network of citations would also provide an alternative search method for clinicians, researchers, and systematic reviewers seeking to base decisions on all available evidence. We sought to determine the connectedness of citation networks of RCTs by examining direct (referenced trials) and indirect (through references of referenced trials, etc) citation of trials to one another.MethodsMeta-analyses were used to create citation networks of RCTs addressing the same clinical questions. The primary measure was the proportion of networks where following citation links between RCTs identifies the complete set of RCTs, forming a single connected citation group. Other measures included the number of disconnected groups (islands) within each network, the number of citations in the network relative to the maximum possible, and the maximum number of links in the path between two connected trials (a measure of indirectness of citations).ResultsWe included 259 meta-analyses with a total of 2,413 and a median of seven RCTs each. For 46% (118 of 259) of networks, the RCTs formed a single connected citation group—one island. For the other 54% of networks, where at least one RCT group was not cited by others, 39% had two citation islands and 4% (10 of 257) had 10 or more islands. On average, the citation networks had 38% of the possible citations to other trials (if each trial had cited all earlier trials). The number of citation islands and the maximum number of citation links increased with increasing numbers of trials in the network.ConclusionAvailable evidence to answer a clinical question may be identified by using network citations created with a small initial corpus of eligible trials. However, the number of islands means that citation networks cannot be relied on for evidence retrieval.