Systematic overview and critical appraisal of meta‐analyses of interventions in intensive care medicine

Rationale

Meta‐analysed intervention effect estimates are perceived to represent the highest level of evidence. However, such effects and the randomized clinical trials which are included in them need critical appraisal before the effects can be trusted.

Objective

Critical appraisal of a predefined set of all meta‐analyses on interventions in intensive care medicine to assess their quality and assessed the risks of bias in those meta‐analyses having the best quality.

Methods

We conducted a systematic search to select all meta‐analyses of randomized clinical trials on interventions used in intensive care medicine. Selected meta‐analyses were critically appraised for basic scientific criteria, (1) presence of an available protocol, (2) report of a full search strategy, and (3) use of any bias risk assessment of included trials. All meta‐analyses which qualified these criteria were scrutinized by full “Risk of Bias in Systematic Reviews” ROBIS evaluation of 4 domains of risks of bias, and a “Preferred Reporting Items for Systematic Reviews and Meta‐Analyses” PRISMA evaluation.

Results

We identified 467 meta‐analyses. A total of 56 meta‐analyses complied with these basic scientific criteria. We scrutinized the risks of bias in the 56 meta‐analyses by full ROBIS evaluation and a PRISMA evaluation. Only 4 meta‐analyses scored low risk of bias in all the 4 ROBIS domains and 41 meta‐analyses reported all 27 items of the PRISMA checklist.

Conclusion

In contrast with what might be perceived as the highest level of evidence only 0.9% of all meta‐analyses were judged to have overall low risk of bias.