Meta-analyses in systematic reviews of randomized controlled trials in perinatal medicine: comparison of fixed and random effects models.

There is a need for empirical work comparing the random effects model with the fixed effects model in the calculation of a pooled relative risk in the meta-analysis in systematic reviews of randomized controlled trials. Such comparisons are particularly important when trial results are heterogeneous. We considered 84 independent meta-analyses in which each trial included a set of different women/newborns. These meta-analyses were included in systematic reviews published in the Cochrane Library's pregnancy and childbirth module. Twenty-one of these 84 meta-analyses demonstrated statistical heterogeneity at p<0.10. The random effects model estimates showed wider confidence intervals, particularly in those meta-analyses showing heterogeneity in the trial results. The summary relative risk for the random effects model tended to show a larger protective treatment effect than the fixed effects model in the heterogeneous meta-analyses. In this set of meta-analyses, statistical evaluation of publication bias cannot be shown to account for heterogeneity. Our empirical conclusion is that there may be opposing effects if the random effects model is used in the meta-analysis of clinical trials showing heterogeneity in the results: stronger treatment effects reflected in the summary relative risk, but wider confidence intervals about this summary measure.     

Bias in clinical intervention research

Research on bias in clinical trials may help identify some of the reasons why investigators sometimes reach the wrong conclusions about intervention effects. Several quality components for the assessment of bias control have been suggested, but although they seem intrinsically valid, empirical evidence is needed to evaluate their effects on the extent and direction of bias. This narrative review summarizes the findings of methodological studies on the influence of bias in clinical trials. A number of methodological studies suggest that lack of adequate randomization in published trial reports may be associated with more positive estimates of intervention effects. The influence of double-blinding and follow-up is less clear. Several studies have found a significant association between funding sources and pro-industry conclusions. However, the methodological studies also show that bias is difficult to detect and appraise. The extent of bias in individual trials is unpredictable. A-priori exclusion of trials with certain characteristics is not recommended. Appraising bias control in individual trials is necessary to avoid making incorrect conclusions about intervention effects.     

Effects on overviews of early stopping rules for clinical trials

Use of inappropriate stopping rules for clinical trials results in an excess of false positive conclusions when no true survival differences exist. Overviews of such trials, however, consist mainly of trials which were not stopped early, plus a few of reduced sample size which were. Simulations confirm that the level of such an overview is minimally elevated. Additional follow-up for survival further corrects the level. In fact, for individual trials conducted according to inappropriate rules, late tests have nearly correct level. On the other hand, publication bias (differential reporting of positive results) can substantially increase the level of an overview if only published studies are included.     

Undisclosed Changes in Outcomes in Randomized Controlled Trials: An Observational Study

PURPOSE We wanted to investigate the frequency of undisclosed changes in the outcomes of randomized controlled trials (RCTs) between trial registration and publication.METHODS Using a retrospective, nonrandom, cross-sectional study design, we investigated RCTs published in consecutive issues of 5 major medical journals during a 6-month period and their associated trials registry entries. Articles were excluded if they did not have an available trial registry entry, did not have analyzable outcomes, or were secondary publications. The primary outcome was the proportion of publications in which the primary outcome of the trial was, without disclosure, changed between that recorded in the trial registry and that reported in the final publication. The secondary outcome was the proportion of publications in which the secondary outcome was changed without disclosure.RESULTS We reviewed 158 reports of RCTs and included 110 in the analysis. In 34 (31%), a primary outcome had been changed, and in 77 (70%), a secondary outcome had been changed.CONCLUSIONS There are substantial and important undisclosed changes made to the outcomes of published RCTs between trial registration and publication. This finding has important implications for the interpretation of trial results. Disclosure and discussion of changes would improve transparency in the performance and reporting of trials.     

Registration of clinical trials: a statement from the International Committee of Medical Journal Editors

Altruistic motives and trust are central to scientific investigations involving people. These prompt volunteers to participate in clinical trials. However, publication bias and other causes of the failure to report trial results may lead to an overly positive view of medical interventions in the published evidence available. Registration of randomised controlled trials right from the start is therefore warranted. The International Committee of Medical Journal Editors has issued a statement to the effect that the 11 journals represented in the Committee will not consider publication of the results of trials that have not been registered in a publicly accessible register such as www.clinicaltrials.gov. Patients who voluntarily participate in clinical trials need to know that their contribution to better human healthcare is available for decision making in clinical practice.     

Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis

Asymmetry in funnel plots may indicate publication bias in meta-analysis, but the shape of the plot in the absence of bias depends on the choice of axes. We evaluated standard error, precision (inverse of standard error), variance, inverse of variance, sample size and log sample size (vertical axis) and log odds ratio, log risk ratio and risk difference (horizontal axis). Standard error is likely to be the best choice for the vertical axis: the expected shape in the absence of bias corresponds to a symmetrical funnel, straight lines to indicate 95% confidence intervals can be included and the plot emphasises smaller studies which are more prone to bias. Precision or inverse of variance is useful when comparing meta-analyses of small trials with subsequent large trials. The use of sample size or log sample size is problematic because the expected shape of the plot in the absence of bias is unpredictable. We found similar evidence for asymmetry and between trial variation in a sample of 78 published meta-analyses whether odds ratios or risk ratios were used on the horizontal axis. Different conclusions were reached for risk differences and this was related to increased between-trial variation. We conclude that funnel plots of meta-analyses should generally use standard error as the measure of study size and ratio measures of treatment effect.     

Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research

Biases in systematic reviews and meta-analyses may be examined in 'meta-epidemiological' studies, in which the influence of trial characteristics such as measures of study quality on treatment effect estimates is explored. Published studies to date have analysed data from collections of meta-analyses with binary outcomes, using logistic regression models that assume that there is no between- or within-meta-analysis heterogeneity. Using data from a study of publication bias (39 meta-analyses, 394 published and 88 unpublished trials) and language bias (29 meta-analyses, 297 English language trials and 52 non-English language trials), we compare results from logistic regression models, with and without robust standard errors to allow for clustering on meta-analysis, with results using a 'meta-meta-analytic' approach that can allow for between- and within-meta-analysis heterogeneity. We also consider how to allow for the confounding effects of different trial characteristics. We show that both within- and between meta-analysis heterogeneity may be of importance in the analysis of meta-epidemiological studies, and that confounding exists between the effects of publication status and trial quality.     

A meta-analysis of acupuncture for chronic pain

Results of 14 randomized controlled trials of acupuncture for chronic pain were pooled in a meta-analysis and analysed in three subgroups according to site of pain; and in two subgroups each according to type to trial, type of treatment, type of control, 'blindness' of participating agents, trial size, and type of journal in which results were published. While few individual trials had statistically significant results, pooled results of many subgroups attained statistical significance in favour of acupuncture. Various potential sources of bias, including problems with blindness, precluded a conclusive finding although most results apparently favoured acupuncture.     

Women participation in clinical trials. A preliminary study in the files of the Spanish Agency for Medicinal Products and Medical Devices

BACKGROUND: The evaluation of the gender bias in clinical trials is not unanimous. The aim of this study has been the assessment of the current situation in Spain regarding the manipulation of data from clinical trials at the publication of the Law of Equality. METHODS: We have examined the study design, sample population, demographic data and baseline clinical characteristics of participants, results and conclusions of the trial in the summaries of the final reports of clinical trials received in the first four months of 2007 in the Agencia Espa?ola de Medicamentos y Productos Sanitarios. RESULTS: From twenty-five documents analysed, in their results and conclusions: Five subdivide data sets by age and gender, thirteen do not subdivide data by gender nor do they provide a conclusion, three do not subdivide results neither but do describe severe side effect by gender and four others provide no results or conclusions. CONCLUSIONS: Only twenty percent of the reviewed summaries provide data subdivided by gender. This could be due to the fact that it has not been made or that it was not given enough importance to include it in the summary, neither of which is satisfactory. A more detailed study is difficult due to the current quality of the summaries provided of the final reports of the clinical trials, which is the only document that is mandatory to send to the AEMPS once the trial has concluded. We expect that with the publication of the new Law of Equality this situation will improve, given the growing evidence of the relevance of the subdivision by gender.