Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials

OBJECTIVE: To compare the feasibility of blinding and the perceived risk of unblinding in trials evaluating pharmacologic (PT) and nonpharmacologic treatments (NPT) of hip or knee osteoarthritis. STUDY DESIGN AND SETTING: Two independent reviewers assessed the feasibility of blinding patients, care providers, and outcome assessors, the perceived risk of unblinding, and whether blinding was reported in 110 reports of randomized controlled trials (RCTs) evaluating PT and NPT in patients with hip or knee osteoarthritis. RESULTS: Blinding was considered to be possible less often in NPT trials than in PT trials for patients (42 vs. 96%; P <.001), care providers (12 vs. 96%; P <.001), and outcome assessors (34 vs. 98%; P <.001). When blinding was judged feasible, the perceived risk of unblinding was more often considered moderate or important in NPT than PT trials for patients (35 vs. 14%, P=.02) and outcome assessors (44 vs. 10%, P=.0004). When blinding was judged feasible, it was reported less often in NPT reports than in PT reports for patients (46 vs. 98%, P <.001), care providers (43 vs. 83%, P=.03), and outcome assessors (72 vs. 98%, P=.0006). CONCLUSION: Blinding appears to be more difficult to achieve and unblinding may occur more often in NPT than PT trials.     

A review of blinding in randomized controlled trials found results inconsistent and questionable

BACKGROUND AND OBJECTIVE: To determine methods to assess the success of blinding in randomized controlled trials (RCTs). METHODS: We searched MEDLINE, the Cochrane Controlled Trials Register, and the Cochrane Method Register and performed a manual search to target studies that attempt to assess blinding and describe the methods used in those studies. RESULTS: A total of 90 reports were selected. Reports assessed the success of blinding participants (n = 58), care providers (n = 36), and outcome assessors (n = 15). Of the 58 reports assessing the success of blinding participants, 54 (93%) reported asking participants to guess their treatment assignment. There was no consistency in timing of assessment (e.g., once at the end of the trial, 57%, or several times during the trial, 26%) or modalities of answering (e.g., "do not know" answers, 43%, or participants forced to guess, 31%). A statistical analysis was performed in 57% of reports. The statistical analysis mainly compared the proportion of correct guesses to those produced by chance (32%) or checked for a relation between participants' guesses and treatment assignment (23%). CONCLUSIONS: Methods of assessing the success of blinding, analysis and reporting the results were inconsistent and questionable.     

An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods

BACKGROUND AND OBJECTIVE: Readers of randomized controlled trials (RCTs) commonly assume that what was not reported did not occur. We undertook an observational study to determine whether concealment of randomization or blinding was used in RCTs that failed to report these bias-reducing strategies. METHODS: We recorded the reporting of concealment of randomization and blinding in 105 RCTs. We subsequently contacted the authors and determined if they had used these methodological safeguards. RESULTS: We successfully obtained data from 98 authors. The authors in the full-text publications of these 98 RCTs failed to report the presence or absence of concealment of randomization in 55%, and the blinding status of participants in 26%, health care providers in 64%, data collectors in 84%, outcome assessors in 83%, and data analysts in 96%. In direct contact, authors frequently reported concealing randomization (96%; 95% confidence interval CI=87-100%), blinding participants (20%; 95% CI=7-41%), blinding health care providers (65%; 95% CI=52-77%), blinding data collectors (65%; 95% CI=53-75%), blinding outcome assessors (79%; 95% CI=69-87%), and blinding data analysts (50%; 95% CI=40-60%), despite not reporting the use of these methodological safeguards in their publications. CONCLUSIONS: Readers should not assume that bias-reducing procedures not reported in an RCT did not occur.     

The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications

ObjectivesTo assess the proportion of clinical trials explicitly reporting the risk of unblinding, to evaluate the completeness of reporting on unblinding risk, and to describe the reported procedures involved in assessing unblinding.Study Design and SettingWe sampled at random 300 blinded randomized clinical trials indexed in PubMed in 2010. Two authors read the trial publications and extracted data independently.ResultsTwenty-four trial publications, or 8% (95% confidence interval [CI], 5, 12%), explicitly reported the risk of unblinding, of which 16 publications, or 5% (95% CI, 3, 8%), reported compromised blinding; and 8 publications, or 3% (95% CI, 1, 5%), intact blinding. The reporting on risk of unblinding in the 24 trial publications was generally incomplete. The median proportion of assessments per trial affected by unblinding was 3% (range 1–30%). The most common mechanism for unblinding was perceptible physical properties of the treatments, for example, a difference in the taste and odor of a typhoid vaccine compared with its placebo.ConclusionPublished articles on randomized clinical trials infrequently reported risk of unblinding. This may reflect a tendency for avoiding reporting actual or suspected unblinding or a genuine low risk of unblinding.     

A theoretical analysis showed that blinding cannot eliminate potential for bias associated with beliefs about allocation in randomized clinical trials

ObjectivesTo explore the theoretical justification for blinding in randomized trials and make recommendations concerning the implementation and interpretation of blinded randomized trials.Study Design and SettingA theoretical analysis was conducted of the potential for bias in randomized trials with successful blinding (ie, trials in which beliefs about allocation to treatment or control groups are independent of actual allocation). The analysis identified conditions that must be satisfied to ensure that blinding eliminates the potential for bias associated with beliefs about allocation.ResultsEven when beliefs about allocation are independent of actual allocation, they can still cause bias. The potential for bias is eliminated when the belief is uniformly one of complete ambivalence about allocation.ConclusionEven when blinding succeeds in making beliefs about allocation independent of actual allocation, beliefs about allocation may still cause bias. It is difficult to determine the extent of bias in any particular trial. Bias could be eliminated by establishing a state of complete ambivalence about the allocation of every trial participant, but universal ambivalence may be difficult to achieve and may reduce the generalizability of the trial's findings.     

Participant blinding and gastrointestinal illness in a randomized, controlled trial of an in-home drinking water intervention.

We conducted a randomized, triple-blinded home drinking water intervention trial to determine if a large study could be undertaken while successfully blinding participants. Households were randomized 50:50 to use externally identical active or sham treatment devices. We measured the effectiveness of blinding of participants by using a published blinding index in which values >0.5 indicate successful blinding. The principal health outcome measured was "highly credible gastrointestinal illness" (HCGI). Participants (n=236) from 77 households were successfully blinded to their treatment assignment. At the end of the study, the blinding index was 0.64 (95% confidence interval 0.51-0.78). There were 103 episodes of HCGI during 10,790 person-days at risk in the sham group and 82 episodes during 11,380 person-days at risk in the active treatment group. The incidence rate ratio of disease (adjusted for the clustered sampling) was 1.32 (95% CI 0.75, 2.33) and the attributable risk was 0.24 (95% CI -0.33, 0.57). These data confirm that participants can be successfully blinded to treatment group assignment during a randomized trial of an in-home drinking water intervention.     

Inadequate planning and reporting of adjudication committees in clinical trials: Recommendation proposal

ObjectivesAdjudication committees (ACs) are recommended in randomized controlled trials (RCTs) to standardize the assessment of events. We aimed to assess the reporting and functioning of ACs (synonyms: clinical event committees, endpoint committees) in clinical trials.Study Design and SettingWe searched five high-impact-factor medical journals for reports of RCTs with clinical event endpoints published between January 1, 2004 and December 31, 2005.ResultsACs were reported in 33.4% of the 314 reports of RCTs. ACs were reported in 29.6% of trials with low risk of misclassification (i.e., “hard” main outcome), in 47.5% of trials with medium risk of misclassification (i.e., subjective main outcome and intervention delivered in a blinded fashion) and in 31% of trials with high risk of misclassification (i.e., subjective main outcome without intervention delivered in a blinded fashion). Selected cases to be adjudicated consisted largely of events identified by site investigators (93.3%). Data provided to the AC were reported for 47.4% of ACs.ConclusionReporting of ACs is not fitted to the risk of biased misclassification. Important aspects of the functioning of ACs are insufficiently reported or raise methodological issues. We propose some recommendations for planning and reporting ACs in clinical trials.     

Adjudication-related processes are underreported and lack standardization in clinical trials of venous thromboembolism: a systematic review

ObjectivesAlthough the use of an adjudication committee (AC) for outcomes is recommended in randomized controlled trials, there are limited data on the process of adjudication. We therefore aimed to assess whether the reporting of the adjudication process in venous thromboembolism (VTE) trials meets existing quality standards and which characteristics of trials influence the use of an AC.Study Design and SettingWe systematically searched MEDLINE and the Cochrane Library from January 1, 2003, to June 1, 2012, for randomized controlled trials on VTE. We abstracted information about characteristics and quality of trials and reporting of adjudication processes. We used stepwise backward logistic regression model to identify trial characteristics independently associated with the use of an AC.ResultsWe included 161 trials. Of these, 68.9% (111 of 161) reported the use of an AC. Overall, 99.1% (110 of 111) of trials with an AC used independent or blinded ACs, 14.4% (16 of 111) reported how the adjudication decision was reached within the AC, and 4.5% (5 of 111) reported on whether the reliability of adjudication was assessed. In multivariate analyses, multicenter trials [odds ratio (OR), 8.6; 95% confidence interval (CI): 2.7, 27.8], use of a data safety–monitoring board (OR, 3.7; 95% CI: 1.2, 11.6), and VTE as the primary outcome (OR, 5.7; 95% CI: 1.7, 19.4) were associated with the use of an AC. Trials without random allocation concealment (OR, 0.3; 95% CI: 0.1, 0.8) and open-label trials (OR, 0.3; 95% CI: 0.1, 1.0) were less likely to report an AC.ConclusionRecommended processes of adjudication are underreported and lack standardization in VTE-related clinical trials. The use of an AC varies substantially by trial characteristics.     

A systematic review and meta-analysis of the cardioprotective effects of remote ischaemic preconditioning in open cardiac surgery

Objective

To investigate the cardioprotective efficacy of remote ischaemic preconditioning (RIPC) in cardiac surgery.

Design

We have performed a systematic search of MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials to identify randomized controlled trials involving RIPC.

Setting

Randomized controlled trials of RIPC in open cardiac surgery patients.

Main outcome measures

Meta-analysis was performed with the primary outcome the standardized mean difference between intervention and control groups in 12 hour postoperative troponin concentration. Heterogeneity was examined by fixed effects meta-regression.

Results

Ten studies with a total of 693 participants were included in the meta-analysis. RIPC reduced troponin levels 12 hours after surgery compared with control. The fixed and random effects differences were 0.35 (95% CI 0.19 to 0.51) and 0.53 (95% CI 0.18-0.88) respectively. However, important heterogeneity was present. Fixed effects meta-regression partially accounted for heterogeneity based on whether studies had full blinding, comprising blinding of patients, surgeons, anaesthetists and investigators. Studies with incomplete or no blinding demonstrated a larger estimate of effect, 0.74 (95% CI 0.47 to 1.00) compared to those with full blinding, 0.13 (95% CI - 0.07 to 0.33).

Conclusions

Although our analysis suggests RIPC may result in cardiac protection during cardiac surgery, the effect was most marked in studies without full blinding, with a smaller and statistically non-significant effect in fully blinded studies. We propose that further double blind randomized controlled trials investigating the cardioprotective effects of RIPC in cardiac surgery are required to resolve the current clinical uncertainty.     

Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials

Meta-analysis is an important method of bridging the gap between undersized randomized control trials and the treatment of patients. However, as in any retrospective study, the opportunities for bias to distort the results are widespread. Attempts must be made to introduce the controls found in prospective studies by blinding the selection of papers and extraction of data and making blinded duplicate determinations. Informal and personalized methods of obtaining data are probably more liable to error and bias than employing only published data. Publication bias is a serious problem requiring further research. There also need to be more comparisons of meta-analysed small studies with large co-operative trials.     

Cluster randomized trials produced similar results to individually randomized trials in a meta-analysis of enhanced care for depression

OBJECTIVES: To examine whether cluster randomized trials (1) produce baseline imbalances between intervention and control conditions; (2) give results that are substantially different individually randomized trials; and (3) give different results when adjusted for unit of analysis error. STUDY DESIGN AND SETTING: We used 14 cluster randomized trials and 20 individualized trials of the same intervention (collaborative care for depression). We conducted a random effects meta-analysis to examine imbalance in baseline depression scores. We used meta-regression to test for differential effect size and heterogeneity between clustered and individualized studies. Unit of analysis error was corrected using a range of plausible published intraclass correlation coefficients (ICCs). RESULTS: There were no baseline imbalances in either cluster randomized (P=0.837) or individually randomized (P=0.737) studies. Cluster randomized studies gave almost identical estimates of effect size when compared to individually randomized studies (standardized mean difference, SMDcluster=0.25, 95% confidence interval [CI]: 0.17, 0.33; SMDindividual=0.24; 95% CI: 0.13, 0.36). Adjustment for clustering had minimal effect on clinical and statistical significance (pooled SMDICC 0.02=0.249 [95% CI: 0.174, 0.325] to SMDICC 0.05=0.258 [95% CI: 0.172, 0.345]). CONCLUSION: The additional effort and expense involved in cluster randomized trials needs to be justified when individualized studies might produce robust and believable results.     

Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid

ObjectiveTo test the reliability and validity of specific instructions to classify blinding, when unclearly reported in randomized trials, as “probably done” or “probably not done.”Study Design and SettingWe assessed blinding of patients, health care providers, data collectors, outcome adjudicators, and data analysts in 233 randomized trials in duplicate and independently using detailed instructions. The response options were “definitely yes,” “probably yes,” “probably no,” and “definitely no.” We contacted authors for data verification (46% response). For each of the five questions, we assessed reliability by calculating the agreement between the two reviewers and validity by calculating the agreement between reviewers’ consensus and verified data.ResultsThe percentage with unclear blinding status varied between 48.5% (patients) and 84.1% (data analysts). Reliability was moderate for blinding of outcome adjudicators (κ = 0.52) and data analysts (κ = 0.42) and substantial for blinding of patients (κ = 0.71), providers (κ = 0.68), and data collectors (κ = 0.65). The raw agreement between the consensus record and the author-verified record varied from 84.1% (blinding of data analysts) to 100% (blinding of health care providers).ConclusionWith the possible exception of blinding of data analysts, use of “probably yes” and “probably no” instead of “unclear” may enhance the assessment of blinding in trials.     

A systematic review of trends in the methodological quality of randomized controlled trials in various research fields

ObjectivesWe sought to evaluate the trends in the methodological quality of randomized controlled trials in various medical fields.Study Design and SettingRelevant studies were retrieved by the PubMed and the ISI Web of science databases.ResultsThirty-five out of 457 retrieved studies met the inclusion criteria. Twenty-one out of 35 selected studies reported significant improvement in at least one methodological quality factor. Overall quality scores were increased in 13 out of 26 studies providing relevant data. The most commonly separately examined key quality factors were allocation concealment and blinding in 13 out of 21 studies that reported relevant data. Allocation concealment was the quality characteristic most commonly reported as significantly improving during the reviewed period (in five out of eight studies reporting relevant comparative data).ConclusionCertain aspects of methodological quality have improved significantly over time, but others remain stagnant. Further efforts to improve study design, conduct, and reporting of randomized controlled trials are warranted.     

Eliminating bias in randomized controlled trials: importance of allocation concealment and masking

Randomization in randomized controlled trials involves more than generation of a random sequence by which to assign subjects. For randomization to be successfully implemented, the randomization sequence must be adequately protected (concealed) so that investigators, involved health care providers, and subjects are not aware of the upcoming assignment. The absence of adequate allocation concealment can lead to selection bias, one of the very problems that randomization was supposed to eliminate. Authors of reports of randomized trials should provide enough details on how allocation concealment was achieved so the reader can determine the likelihood of success. Fortunately, a plan of allocation concealment can always be incorporated into the design of a randomized trial. Certain methods minimize the risk of concealment failing more than others. Keeping knowledge of subjects' assignment after allocation from subjects, investigators/health care providers, or those assessing outcomes is referred to as masking (also known as blinding). The goal of masking is to prevent ascertainment bias. In contrast to allocation concealment, masking cannot always be incorporated into a randomized controlled trial. Both allocation concealment and masking add to the elimination of bias in randomized controlled trials.     

Reporting in randomized clinical trials improved after adoption of the CONSORT statement

OBJECTIVE: To examine the extent to which the Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines improved clinical trials reporting and subject attrition, which may undermine the credibility of published randomized clinical trials (RCTs). STUDY DESIGN AND SETTING: Published RCTs reported in two major medical journals before and after the CONSORT guidelines were systematically reviewed; one used the CONSORT statement (JAMA) and one did not (NEJM). RESULTS: The quality of RCT reporting improved for both journals, but JAMA showed more significant and consistent improvements in all aspects of RCT reporting. Subject attrition was better accounted for after the publication of CONSORT, although the attrition rates for various reasons actually increased. Attrition due to unknown reasons, as a percentage of total attrition, declined dramatically, from 68.7% pre-CONSORT to 13.0% post-CONSORT. CONCLUSIONS: Attrition of study subjects remains a serious problem in RCTs. Bias from selective attrition can undermine the presumptive scientific advantage of RCTs. The CONSORT guidelines improved RCT reporting when they were implemented but did not substantially improve reported attrition rates.