| Abstract: | In the era of “comparative effectiveness” research, each of the major stakeholders in healthcare—payors, patients, providers, and government—face a similar challenge. When making a decision about whether a new device, drug, or a diagnostic modality should be considered for use or coverage, what choices are best supported by the evidence? Medical evidence is defined by randomized controlled trials and by observational studies that vary greatly in their design, the accuracy of their analyses, and the relevance of their conclusions and recommendations. Hence, key decision makers increasingly rely on systematic reviews and meta-analyses to facilitate the interpretation and application of research evidence. Knowing how to evaluate meta-analyses and understanding the potential pitfalls of the method are crucial for those involved in designing drug benefits. The authors highlight the process, strengths, and weaknesses of meta-analysis and explain how to judge the value of the results.“Comparative effectiveness” research is the best tool available today for making decisions about which new medication, medical device, or diagnostic test is most supported by the evidence.1 The purpose of a systematic review is to synthesize the results of multiple primary studies using explicit and reproducible methods.2,3 Meta-analysis is a form of systematic review that goes one step further. Rather than qualitatively synthesizing the results of multiple studies, the purpose of a meta-analysis is to develop a quantitative summary of the evidence with the help of special statistical techniques. Whereas a systematic review may conclude that a certain drug is effective in preventing a morbid event, a meta-analysis will tell us that the drug is, say, 3.7 times more effective than a placebo in preventing that morbid outcome. Conceptually, the distinction between the 2 types of reviews is straightforward. Despite the appeal inherent in developing a quantitative synthesis, the decision to conduct, or to make decisions based on, a meta-analysis must be made carefully, with an understanding of the benefits and limitations of the method. This paper provides an overview of the meta-analysis process, highlights the strengths and weaknesses of the method, and offers guidance on how to interpret and judge the value of meta-analytic results. |
