Reporting of methods was better in the Clinical Trials Registry-India than in Indian journal publications

ObjectiveWe sought to evaluate if editorial policies and the reporting quality of randomized controlled trials (RCTs) had improved since our 2004–05 survey of 151 RCTs in 65 Indian journals, and to compare reporting quality of protocols in the Clinical Trials Registry-India (CTRI).Study Design and SettingAn observational study of endorsement of Consolidated Standards for the Reporting of Trials (CONSORT) and International Committee of Medical Journal Editors (ICMJE) requirements in the instructions to authors in Indian journals, and compliance with selected requirements in all RCTs published during 2007–08 vs. our previous survey and between all RCT protocols in the CTRI on August 31, 2010 and published RCTs from both surveys.ResultsJournal policies endorsing the CONSORT statement (22/67, 33%) and ICMJE requirements (35/67, 52%) remained suboptimal, and only 4 of 13 CONSORT items were reported in more than 50% of the 145 RCTs assessed. Reporting of ethical issues had improved significantly, and that of methods addressing internal validity had not improved. Adequate methods were reported significantly more frequently in 768 protocols in the CTRI, than in the 296 published trials.ConclusionThe CTRI template facilitates the reporting of valid methods in registered trial protocols. The suboptimal compliance with CONSORT and ICMJE requirements in RCTs published in Indian journals reduces credibility in the reliability of their results.     

Influence of trial registration on reporting quality of randomized trials: Study from highest ranked journals

ObjectiveTo evaluate the reporting quality of key methodological items of randomized control trials (RCTs) in 55 of the highest ranked journals.Study Design and SettingA list of the highest top ranked journals was identified, and a search for detecting RCTs in those journals was made. Two hundred sixty four journals were screened and 55 of them were identified having at least one RCT. Three RCTs were randomly selected a priori from each journal; 148 RCTs were finally included. RCTs were assessed by two reviewers using the Consolidated Standards of Reporting Trials (CONSORT) statement.ResultsOnly 11 (8%) RCTs had all items adequately reported. In addition, 36% of RCTs reported that the study was registered in any trial registry. We found a significant difference in the quality of reporting for baseline characteristics, recruitment, participant's flow, and randomization implementation between those studies having reported the registration of their RCT in a trial registry and those that have not. Adherence to key methodological items of the CONSORT statement was as follows: sample size determination (60%), sequence generation (49%), allocation concealment (40%), and blinding (25%).ConclusionsReporting of varied CONSORT items remains suboptimal. Registration in a trial registry was associated with improved reporting. Further efforts to enhance RCT registration could contribute to this improvement.     

Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting

ObjectiveTo describe a novel CONsolidated Standards of Reporting Trials (CONSORT) adherence strategy implemented by the American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO) and to report its impact on the completeness of reporting of published trials.Study Design and SettingThe AJO-DO CONSORT adherence strategy, initiated in June 2011, involves active assessment of randomized clinical trial (RCT) reporting during the editorial process. The completeness of reporting CONSORT items was compared between trials submitted and published during the implementation period (July 2011 to September 2013) and trials published between August 2007 and July 2009.ResultsOf the 42 RCTs submitted (July 2011 to September 2013), 23 were considered for publication and assessed for completeness of reporting, seven of which were eventually published. For all published RCTs between 2007 and 2009 (n = 20), completeness of reporting by CONSORT item ranged from 0% to 100% (Median = 40%, interquartile range = 60%). All published trials in 2011–2013, reported 33 of 37 CONSORT (sub) items. Four CONSORT 2010 checklist items remained problematic even after implementation of the adherence strategy: changes to methods (3b), changes to outcomes (6b) after the trial commenced, interim analysis (7b), and trial stopping (14b), which are typically only reported when applicable.ConclusionTrials published following implementation of the AJO-DO CONSORT adherence strategy completely reported more CONSORT items than those published or submitted previously.     

A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals

ObjectiveStatisticians have criticized the use of significance testing to compare the distribution of baseline covariates between treatment groups in randomized controlled trials (RCTs). Furthermore, some have advocated for the use of regression adjustment to estimate the effect of treatment after adjusting for potential imbalances in prognostically important baseline covariates between treatment groups.Study Design and SettingWe examined 114 RCTs published in the New England Journal of Medicine, the Journal of the American Medical Association, The Lancet, and the British Medical Journal between January 1, 2007 and June 30, 2007.ResultsSignificance testing was used to compare baseline characteristics between treatment arms in 38% of the studies. The practice was very rare in British journals and more common in the U.S. journals. In 29% of the studies, the primary outcome was continuous, whereas in 65% of the studies, the primary outcome was either dichotomous or time-to-event in nature. Adjustment for baseline covariates was reported when estimating the treatment effect in 34% of the studies.ConclusionsOur findings suggest the need for greater editorial consistency across journals in the reporting of RCTs. Furthermore, there is a need for greater debate about the relative merits of unadjusted vs. adjusted estimates of treatment effect.     

Rationale for unequal randomization in clinical trials is rarely reported: a systematic review

ObjectivesTo assess the reporting of the unequal randomization ratio in reports of trials with this design and to identify the justification for the design.Study Design and SettingSystematic review of reports of trials with unequal randomization. We selected all original reports of two parallel-group randomized controlled trials with unequal randomization, which were published in 2009 and 2010 in core clinical journals in MEDLINE on the basis of the abstract (and full text, if necessary). Additional information was collected by an author survey.ResultsWe retrieved 106 reports (prevalence, 4.7%). The randomization ratio was not stated in 8.5% of reports and 51.9% of abstracts. Sample size calculation was reported in 70 reports, with unequal randomization not taken into account in 25.7% (n = 18). Justification for unequal randomization was not reported in 77.4% (n = 82) of reports. Combining information from reports and author surveys, we had justification for 41 trials. The main justification was safety issues for 20 trials. In 11 of those latter 20 reports, adverse events were not fully reported.ConclusionA better reporting of the randomization ratio, sample size calculation, and justification to unequal randomization could help readers appraise the quality and risk of bias of such trials.     

Reporting in randomized clinical trials improved after adoption of the CONSORT statement

OBJECTIVE: To examine the extent to which the Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines improved clinical trials reporting and subject attrition, which may undermine the credibility of published randomized clinical trials (RCTs). STUDY DESIGN AND SETTING: Published RCTs reported in two major medical journals before and after the CONSORT guidelines were systematically reviewed; one used the CONSORT statement (JAMA) and one did not (NEJM). RESULTS: The quality of RCT reporting improved for both journals, but JAMA showed more significant and consistent improvements in all aspects of RCT reporting. Subject attrition was better accounted for after the publication of CONSORT, although the attrition rates for various reasons actually increased. Attrition due to unknown reasons, as a percentage of total attrition, declined dramatically, from 68.7% pre-CONSORT to 13.0% post-CONSORT. CONCLUSIONS: Attrition of study subjects remains a serious problem in RCTs. Bias from selective attrition can undermine the presumptive scientific advantage of RCTs. The CONSORT guidelines improved RCT reporting when they were implemented but did not substantially improve reported attrition rates.     

Statistical inference in abstracts of major medical and epidemiology journals 1975–2014: a systematic review

Since its introduction in the twentieth century, null hypothesis significance testing (NHST), a hybrid of significance testing (ST) advocated by Fisher and null hypothesis testing (NHT) developed by Neyman and Pearson, has become widely adopted but has also been a source of debate. The principal alternative to such testing is estimation with point estimates and confidence intervals (CI). Our aim was to estimate time trends in NHST, ST, NHT and CI reporting in abstracts of major medical and epidemiological journals. We reviewed 89,533 abstracts in five major medical journals and seven major epidemiological journals, 1975–2014, and estimated time trends in the proportions of abstracts containing statistical inference. In those abstracts, we estimated time trends in the proportions relying on NHST and its major variants, ST and NHT, and in the proportions reporting CIs without explicit use of NHST (CI-only approach). The CI-only approach rose monotonically during the study period in the abstracts of all journals. In Epidemiology abstracts, as a result of the journal’s editorial policy, the CI-only approach has always been the most common approach. In the other 11 journals, the NHST approach started out more common, but by 2014, this disparity had narrowed, disappeared or reversed in 9 of them. The exceptions were JAMA, New England Journal of Medicine, and Lancet abstracts, where the predominance of the NHST approach prevailed over time. In 2014, the CI-only approach is as popular as the NHST approach in the abstracts of 4 of the epidemiology journals: the American Journal of Epidemiology (48%), the Annals of Epidemiology (55%), Epidemiology (79%) and the International Journal of Epidemiology (52%). The reporting of CIs without explicitly interpreting them as statistical tests is becoming more common in abstracts, particularly in epidemiology journals. Although NHST is becoming less popular in abstracts of most epidemiology journals studied and some widely read medical journals, it is still very common in the abstracts of other widely read medical journals, especially in the hybrid form of ST and NHT in which p values are reported numerically along with declarations of the presence or absence of statistical significance.     

A methodological review of recent meta-analyses has found significant heterogeneity in age between randomized groups

BackgroundThere is evidence to suggest that component randomized controlled trials (RCTs) within systematic reviews may be biased. It is important that these reviews are identified to prevent erroneous conclusions influencing health care policies and decisions.PurposeTo assess the likelihood of bias in trials in 12 meta-analyses.DesignA review of 12 systematic reviews.Data SourcesTwelve recently published systematic reviews with 503 component randomized trials, published in the British Medical Journal, The Lancet, Journal of the American Medical Association, and The Annals of Internal Medicine before May 2012.Study Selection and Data ExtractionSystematic reviews were eligible for inclusion if they included only RCTs. We obtained the full text for the component RCTs of the 12 systematic reviews (in English only). We extracted summary data on age, number of participants in each treatment group, and the method of allocation concealment for each RCT.Data SynthesisFive of the 12 meta-analyses exhibited heterogeneity in age differences (I² > 0.30), when there should have been none. In two meta-analyses, the age of the intervention group was significantly greater than that of the control group. Inadequate allocation concealment was a statistically significant predictor of heterogeneity in one trial as observed by a metaregression.ConclusionsMost of the sample of recent meta-analyses showed that there were signs of imbalance and/or heterogeneity in ages between treatment groups, when there should have been none. Systematic reviewers might consider using the techniques described here to assess the validity of their findings.     

Compliance with prospective trial registration guidance remained low in high-impact journals and has implications for primary end point reporting

ObjectivesTo examine compliance with International Committee of Medical Journal Editors' (ICMJE) policy on prospective trial registration along with predictors of compliance.Study Design and SettingCross-sectional analysis of all articles reporting trial results published in the six highest-impact general medicine journals in January–June 2014 that were registered in a public trial registry. The main outcome measure was compliance with ICMJE policy. The time frame for trial primary end point ascertainment was used to assess whether retrospective registration could have allowed changing of primary end points following an interim analysis.ResultsForty of 144 (28%) articles did not comply with the ICMJE policy. Trials of non–FDA-regulated interventions were less compliant than trials of FDA-regulated interventions (i.e., medicines, medical devices) (42% vs. 21%; P = 0.016). Twenty-nine of these 40 (72%; 20% overall) were registered before any interim analysis of primary end points could have been conducted; 11 (28%; 8% overall) were registered after primary end point ascertainment, such that investigators could have had the opportunity to conduct an interim analysis before trial registration.ConclusionTwenty-eight percent of trials published in high-impact journals were retrospectively registered including nearly 10% that were registered after primary end point ascertainment could have had taken place. Prospective registration should be prompted and enforced to ensure transparency and accountability in clinical research.     

中文遗传关联研究Meta分析文章报告质量评价

目的评价中文遗传关联研究Meta分析文章报告质量。方法按制定策略检索中文数据库收录的遗传关联研究Meta 分析文章,时间至2012年12月,2名研究者独立按制定纳入排除标准进行文章筛选,意见不一致时咨询第3位研究者,运用MOOSE规范对文章的报告质量进行评价。结果共纳入440篇文章,0.7%的文章陈述了研究假设,72.7%的文章报告了完整检索策略,15.2%的文章报告了处理只有摘要和未发表文献的方法,47.3%的文章报告了文献筛选方法,32.7%的文章评价了文献质量,8.6%的文章有流程图总结入选文献流程,32.0%的文章公布了研究资助来源。结论中文遗传关联研究Meta分析文章报告质量在报告研究假设、文献检索、只有摘要和未发表文献处理、文献筛选方法、文献质量评价、提供文献筛选流程图和公布资助来源等方面存在一定问题。     

A tool to improve quality of reporting published economic analyses

OBJECTIVES: To test the feasibility of obtaining a baseline level of quality of reporting for cost-utility analysis (CUA) studies using the British Medical Journal economic submissions checklist, test interrater reliability of this tool, and discuss its longer term implications. METHODS: CUA studies in peer-reviewed English language journals in 1996, assessed using the British Medical Journal checklist, a quality index, and interrater reliability correlations. RESULTS: Forty-three CUA studies were assessed, with 23 checklist items acceptable and 10 items inadequate. Lowest quality scores were reported in specialist medical journals. Proportional agreement between assessors was over 80%. CONCLUSIONS: The British Medical Journal checklist is a feasible tool to collect baseline information on the quality of reporting in journals other than the British Medical Journal. Editors of specialist medical journals are in greatest need of economic guidance. If handled carefully, they might consider adopting the British Medical Journal checklist.     

Systematic reviews published in higher impact clinical journals were of higher quality

ObjectivesTo compare the methodological quality of systematic reviews (SRs) published in high- and low–impact factor (IF) Core Clinical Journals. In addition, we aimed to record the implementation of aspects of reporting, including Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram, reasons for study exclusion, and use of recommendations for interventions such as Grading of Recommendations Assessment, Development and Evaluation (GRADE).Study Design and SettingWe searched PubMed for systematic reviews published in Core Clinical Journals between July 1 and December 31, 2012. We evaluated the methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR) tool.ResultsOver the 6-month period, 327 interventional systematic reviews were identified with a mean AMSTAR score of 63.3% (standard deviation, 17.1%), when converted to a percentage scale. We identified deficiencies in relation to a number of quality criteria including delineation of excluded studies and assessment of publication bias. We found that SRs published in higher impact journals were undertaken more rigorously with higher percentage AMSTAR scores (per IF unit: β = 0.68%; 95% confidence interval: 0.32, 1.04; P < 0.001), a discrepancy likely to be particularly relevant when differences in IF are large.ConclusionMethodological quality of SRs appears to be better in higher impact journals. The overall quality of SRs published in many Core Clinical Journals remains suboptimal.     

The quality of safety reporting in trials is still suboptimal: survey of major general medical journals

Objective

To evaluate whether the quality of reporting harms improved after the publication of the Extension of the Consolidated Standards of Reporting Trials (CONSORT) statement and predictors that influence the safety reporting in randomized controlled trials (RCTs)

Study Design and Setting

Systematic survey of published RCTs assessing drugs. In MEDLINE, we identified 228 RCTs published in Annals of Internal Medicine, British Medical Journal, Journal of American Medical Association, The Lancet, and The New England Journal of Medicine in 2003 and 2006.

Results

The reporting of harms have improved over time both in quality and extent of space. However, the mean score as an overall measure of adequacy in reporting harms was 0.58 in 2003 and increased to 0.67 in 2006, indicating a moderate safety reporting. Safety was more adequate in trials with statistically significant results for efficacy, private funding, primary harms outcome, and anti-infective, antineoplasmatic, or immunosuppressive agents.

Conclusion

The use of the Extension of the CONSORT statement may be associated with improving the quality of safety reporting in RCTs, but there are still deficiencies that need to be corrected to use quantitative objective evidence for harms in performing meta-analyses and making therapeutic decisions.     

Randomized controlled trials of herbal interventions underreport important details of the intervention

Objective

To examine the quality of reporting and predictors of reporting in randomized clinical trials (RCTs) of herbal medicine interventions.

Study Design and Setting

We searched Medical Literature Analysis and Retrieval System Online, Excerpta Medica Database, and Academy of Microscope Enhanced Dentistry up to December 2007 for any English language RCT of 11 commonly used herbal medicine interventions. Two individuals separately and independently assessed all trials using the Consolidated Standards of Reporting Trials (CONSORT) checklist for herbal medicines interventions. We randomly selected 100 of these trials, extracted a set of potential predictor variables identified through a literature search and consultation with experts, and performed a conceptually driven stepwise elimination regression analyses for predictor variables.

Results

The 406 trials reported on average 38% of the information suggested in the checklist. Regression analyses revealed better overall reporting in trials with a participant flow diagram (P = 0.008), those of Panax quinquefolius (P = 0.018), and those published in more recent years (P = 0.02).

Conclusion

Our results indicate that RCTs of herbal medicine interventions frequently do not report important characteristics of the intervention. Trialists should refer to the CONSORT for herbal medicines when reporting their trials.     

Systematic reviewers commonly contact study authors but do so with limited rigor

ObjectivesAuthor contact can enhance the quality of systematic reviews. We conducted a systematic review of the practice of author contact in recently published systematic reviews to characterize its prevalence, quality, and results.Study Design and SettingEligible studies were systematic reviews of efficacy published in 2005–2006 in the 25 journals with the highest impact factor publishing systematic reviews in clinical medicine and the Cochrane Library, identified by searching MEDLINE, EMBASE, and the Cochrane Library. Two researchers determined whether and why reviewers contacted authors. To assess the accuracy of the abstracted data, we surveyed reviewers by e-mail.ResultsForty-six (50%) of the 93 eligible systematic reviews published in top journals and 46 (85%) of the 54 eligible Cochrane reviews reported contacting authors of eligible studies. Requests were made most commonly for missing information: 40 (76%) clinical medicine reviews and 45 (98%) Cochrane reviews. One hundred and nine of 147 (74%) reviewers responded to the survey, and reported a higher rate of author contact than apparent from the published record.ConclusionAlthough common, author contact is not a universal feature of systematic reviews published in top journals and the Cochrane Library. The conduct and reporting of author contact purpose, procedures, and results require improvement.