Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.     

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

     

Why observational studies should be among the tools used in comparative effectiveness research

Doctors, patients, and other decision makers need access to the best available clinical evidence, which can come from systematic reviews, experimental trials, and observational research. Despite methodological challenges, high-quality observational studies have an important role in comparative effectiveness research because they can address issues that are otherwise difficult or impossible to study. In addition, many clinical and policy decisions do not require the very high levels of certainty provided by large, rigorous randomized trials. This paper provides insights and a framework to guide good decision making that involves the full range of high-quality comparative effectiveness research techniques, including observational research.     

Diet and survival after prostate cancer diagnosis

Prostate cancer is the most commonly diagnosed non-skin cancer in men in the United States. Among environmental factors, diet may play a particularly important role in its incidence, progression, and clinical outcome. This article reviews the findings of eight observational studies and 17 intervention or laboratory trials on the effect of plant-based diets and plant nutrients on both the progression and clinical outcome of prostate cancer as well as additional studies examining mechanisms that may explain dietary effects. While additional long-term therapeutic clinical trials are needed to further elucidate the role of diet, these early investigations suggest that a recommendation for individual patients to shift their diets toward plant foods may serve as an important component of the tertiary treatment of prostate cancer.     

Empirical estimation of life expectancy from large clinical trials: use of left-truncated, right-censored survival analysis methodology

In the current era of ever-increasing health care costs, economic analyses are an essential component in the comprehensive evaluation of new medical interventions. Cost-effectiveness analysis (CEA)--the most common form of economic analysis used in medicine--aids policy-makers in determining how to allocate finite health care dollars among possible alternative therapies. CEA relates the incremental benefits of a new technology to its incremental costs in a cost-effectiveness (CE) ratio. Although the generally agreed-upon standard of presentation for the CE ratio is the lifetime perspective (incremental lifetime cost to add one life year), this perspective presents an obvious challenge to the statistical analyst. Most large clinical trials collect limited follow-up data, and yet their findings form the basis of therapeutic recommendations that often extend far beyond the limits of the empirical data. Although clinical practice guidelines do not yet require explicit modeling to examine the long-term implications of their recommendations, health policy analyses routinely rely upon such extrapolations. This paper describes methods for using empirical patient-level data to extrapolate survival in large clinical trials and cohorts beyond a limited follow-up period in which most patients remain alive in order to estimate the entire survival distribution for a cohort of patients. We accomplish this task through a novel combination of models that estimate the hazard rate not only as a function of time but also as a function of patient age. Extrapolation of survival beyond a limited time frame is made possible by capitalizing on the extensive latitude of survival information available across the range of ages represented in the data. Variations in approach are presented, and issues arising in these analyses are discussed. The proposed methodology is developed, applied, and evaluated in both a large clinical trial cohort with 5-year follow-up on over 23,000 patients and a large observational database with long-term follow-up on over 4000 patients.     

疗效比较研究的方法学应用及其实施过程

疗效比较研究(CER)可有效比较诊疗措施的临床效果,为医疗决策提供证据.其设计除采用大规模临床随机对照试验、整群随机对照试验、类试验、数学模型外,以电子病历为基础的观察性研究备受重视.对研究课题的证据生成、综合、传播及应用的每一环节上,采用CER均需相应的统计学方法进行统计分析和质量控制.     

Analysis of longitudinal laboratory data in the presence of common selection mechanisms: a view toward greater emphasis on pre-marketing pharmaceutical safety

Pharmaceutical safety has received substantial attention in the recent past; however, longitudinal clinical laboratory data routinely collected during clinical trials to derive safety profiles are often used ineffectively. For example, these data are frequently summarized by comparing proportions (between treatment arms) of participants who cross pre-specified threshold values at some time during follow-up. This research is intended, in part, to encourage more effective utilization of these data by avoiding unnecessary dichotomization of continuous data, acknowledging and making use of the longitudinal follow-up, and combining data from multiple clinical trials. However, appropriate analyses require careful consideration of a number of challenges (e.g. selection, comparability of study populations, etc.). We discuss estimation strategies based on estimating equations and maximum likelihood for analyses in the presence of three response history-dependent selection mechanisms: dropout, follow-up frequency, and treatment discontinuation. In addition, because clinical trials' participants usually represent non-random samples from target populations, we describe two sensitivity analysis approaches. All discussions are motivated by an analysis that aims to characterize the dynamic relationship between concentrations of a liver enzyme (alanine aminotransferase) and three distinct doses (no drug, low dose, and high dose) of an nk-1 antagonist across four Phase II clinical trials.     

The Methodological Quality of Economic Evaluations of Guideline Implementation into Clinical Practice: A Systematic Review of Empiric Studies

OBJECTIVES: Despite the emphasis on efficiency of health-care services delivery, there is an imperfect evidence base to inform decisions about whether and how to develop and implement guidelines into clinical practice. In general, studies evaluating the economics of guideline implementation lack methodological rigor. We conducted a systematic review of empiric studies to assess advances in the economic evaluations of guideline implementation. METHODS: The Cochrane Effective Professional and Organisational Change Group specialized register and the MEDLINE database were searched for English publications between January 1998 and July 2004 that reported objective effect measures and implementation costs. We extracted data on study characteristics, quality of study design, and economic methodology. It was assessed whether the economic evaluations followed methodological guidance. RESULTS: We included 24 economic evaluations, involving 21 controlled trials and three interrupted time series designs. The studies involved varying settings, targeted professionals, targeted behaviors, clinical guidelines, and implementation strategies. Overall, it was difficult to determine the quality of study designs owing to poor reporting. In addition, most economic evaluations were methodologically flawed: studies did not follow guidelines for evaluation design, data collection, and data analysis. CONCLUSIONS: The increasing importance of the value for money of providing health care seems to be reflected by an increase in empiric economic evaluations of guideline implementation. Because of the heterogeneity and poor methodological quality of these studies, however, the resulting evidence is still of limited use in decision-making. There seems to be a need for more methodological guidance, especially in terms of data collection and data synthesis, to appropriately evaluate the economics of developing and implementing guidelines into clinical practice.     

Estimating the efficacy of interventions to prevent mother-to-child transmission of HIV in breast-feeding populations: development of a consensus methodology.

Postnatal transmission of HIV through breast milk complicates both the design of effective interventions to prevent mother-to-child transmission of HIV (PMTCT) and their evaluation. Estimated long-term efficacy in five African trials (four with peri-partum antiretrovirals and one with artificial feeding) varied from 25 to 50 per cent. This variation may be due, at least in part, to differences in analytical methodology. To facilitate direct comparison between trials, a methodological consensus approach to the analysis and presentation of the results of PMTCT trials was developed. The initial methodology used and results presented from African trials with available long-term efficacy data were reviewed during a workshop in Bordeaux, France, in September 2000. A consensus approach for evaluating efficacy applicable across PMTCT studies was developed. There are four typical situations defined by duration of follow-up (short versus long), and the available demographic (vital status) and biological data (single versus repeat HIV testing). Efficacy can be assessed from the risk of infection directly or from HIV-free survival by combining infection and death as a single endpoint. Studies should report results in a standardized format including infection, weaning, mortality and loss to follow-up. New statistical methods that account for the unknown date when a child would first test positive for HIV, for weaning as a competing risk for HIV infection, and for increased risk of death among HIV-infected children should be used in analysing data from PMTCT studies with repeat HIV testing. All estimates should be reported with confidence intervals. This standardized methodology that allows direct comparison between studies is now being applied to four randomized clinical trials.     

Novel approaches to incorporating pharmacoeconomic studies into phase III clinical trials for Alzheimer's disease

The societal and individual costs of Alzheimer's disease are significant, worldwide. As the world ages, these costs are increasing rapidly, while health systems face finite budgets. As a result, many regulators and payers will require or at least consider phase III cost-effectiveness data (in addition to safety and efficacy data) for drug approval and reimbursement, increasing the risks and costs of drug development. Incorporating pharmacoeconomic studies in phase III clinical trials for Alzheimer's disease presents a number of challenges. We propose several specific suggestions to improve the design of pharmacoeconomic studies in phase III clinical trials. We propose that acute episodes of care are key outcome measures for pharmacoeconomic studies. To improve the possibility of detecting a pharmacoeconomic impact in phase III, we suggest several strategies including; study designs for enrichment of pharmacoeconomic outcomes that include co-morbidity of patients; reducing variability of care that can affect pharmacoeconomic outcomes through standardized care management; employing administrative claims data to better capture meaningful pharmacoeconomic data; and extending clinical trials in open label follow-up periods in which pharmacoeconomic data are captured electronically by administrative claims. Specific aspects of power analysis for pharmacoeconomic studies are presented. The particular pharmacoeconomic challenges caused by the use of biomarkers in clinical trials, the increasing use of multinational studies, and the pharmacoeconomic challenges presented by biologicals in development for Alzheimer's disease are discussed. In summary, since we are entering an era in which pharmacoeconomic studies will be essential in drug development for supporting regulatory approval, payor reimbursement and integration of new therapies into clinical care, we must consider the design and incorporation of pharmacoeconomic studies in phase III clinical trials more seriously and more creatively.     

The efficacy and adverse effects of in vitro fertilization and embryo transfer.

This paper examines the current status of in vitro fertilization and embryo transfer (IVF-ET) as a treatment for various types of infertility. We reviewed studies on the efficacy and safety of IVF-ET and intracytoplasmic sperm injection (ICSI) plus IVF-ET, compared with conventional treatment or no treatment for various infertility diagnoses. Material retrieved included English language publications between 1992 and January 1997 that reported the results of prospective controlled clinical trials, cohort studies, and retrospective comparative studies with large series, and reviews presenting risks, complications, and longer-term health consequences associated with IVF-ET and ICSI. No adequate prospective comparative studies of sufficient power on the use of IVF-ET for specific infertility diagnoses have been reported to date. Most of the published reports concerning results with IVF-ET as a treatment of infertility have been based upon small, uncontrolled studies, with various methodological weaknesses. Reported results are not directly comparable. There are few follow-up data on outcomes after pregnancy is established or on long-term health consequences of the use of IVF-ET on mothers and their babies. IVF-ET has diffused widely without comprehensive assessment of its efficacy and safety. The available evidence supports its use only for severe bilateral tubal occlusion. For other diagnoses of infertility the evidence is limited and does not establish whether IVF-ET is effective. Long-term, well-designed, prospective clinical trials are required to determine when and for what indications IVF-ET is effective and what its health effects are on both mothers and their babies.     

El registro de los estudios observacionales: es el momento de cumplir el requerimiento de la Declaración de Helsinki

Publication bias is a serious deficiency in the current system of disseminating the results of human research studies. Clinical investigators know that, from an ethical standpoint, they should prospectively register clinical trials in a public registry before starting them. In addition, it is believed that this approach will help to reduce publication bias. However, most studies conducted in humans are observational rather than experimental. It is estimated that less than 2% out of 2 million concluded or ongoing observational studies have been registered. The 2013 revision of the Declaration of Helsinki requires registration of any type of research study involving humans or identifiable samples or data.It is proposed that funding agencies, such as the Fondo de Investigaciones Sanitarias, as well as private companies, require preregistration of observational studies before providing funding. It is also proposed that Research Ethics Committees which, following Spanish regulation, have been using the Declaration as the framework for assessing the ethics of clinical trials with medicines since 1990, should follow the same provisions for the assessment of health-related observational studies: therefore, they should require prospective registration of studies before granting their final approval. This would allow observational study investigators to be educated in complying with an ethical requirement recently introduced in the most important ethical code for research involving humans.     

New anti-epileptic drugs: overcoming the limits of randomised controlled trials

This commentary focuses on the designs of randomised controlled trials of new anti-epileptic drugs as treatment for focal epilepsy. Limits of these trials, with particular focus on placebo-controlled designs, are discussed and strategies to overcoming them proposed. To date there are only few head-to-head comparison trials between new anti-epileptic drugs. Ideally, direct head-to-head comparisons of new anti-epileptic drugs should be available in order to get the whole picture of each treatment, but usually randomised controlled trials have not such a direct-comparison design. Multiple-treatment meta-analysis may represent a promising way of overcoming this limit, providing information on ranking efficacy of new anti-epileptic drugs, thus allowing to answer several relevant questions regarding daily practice and decision-making. Although not free from concerns, also historical design trials might have several advantages in that all patients receive a promising anti-epileptic drug at dose(s) that are expected to be fully effective and eliminate the need for a parallel group on suboptimal treatment or placebo. All these strategies aimed to overcome the lack of head-to-head comparisons can't anyway be considered as a substitute for properly conducted direct-comparison randomised trials, which remain the most relevant source of data to inform clinical decisions.     

Risk indices associated with the insulin resistance syndrome, cardiovascular disease, and possible protection with yoga: a systematic review

OBJECTIVE: To conduct a systematic review of published literature regarding the effects of yoga, a promising mind-body therapy, on specific anthropometric and physiologic indices of cardiovascular disease (CVD) risk and on related clinical endpoints. METHODS: We performed a literature search using 4 computerized English and Indian scientific databases. The search was restricted to original studies (1970 to 2004) evaluating the effects of yoga on CVD or indices of CVD risk associated with the insulin resistance syndrome (IRS). Randomized controlled trials (RCTs), nonrandomized controlled trials, uncontrolled (pre and post) clinical trials, and cross-sectional (observational) studies were included if they met specific criteria. Data were extracted regarding study design, setting, population size and characteristics, intervention type and duration, comparison group or condition, outcome assessment, data analysis and presentation, follow-up, and key results, and the quality of each study was evaluated according to specific predetermined criteria. RESULTS: We identified 70 eligible studies, including 1 observational study, 26 uncontrolled clinical trials, 21 nonrandomized controlled clinical trials, and 22 RCTs. Together, the reported results of these studies indicate beneficial changes overall in several IRS-related indices of CVD risk, including glucose tolerance and insulin sensitivity, lipid profiles, anthropometric characteristics, blood pressure, oxidative stress, coagulation profiles, sympathetic activation, and cardiovagal function, as well as improvement in several clinical endpoints. CONCLUSIONS: Collectively, these studies suggest that yoga may reduce many IRS-related risk factors for CVD, may improve clinical outcomes, and may aid in the management of CVD and other IRS-related conditions. However, the methodologic and other limitations characterizing most of these studies preclude drawing firm conclusions. Additional high quality RCTs are needed to confirm and further elucidate the effects of standardized yoga programs on specific indices of CVD risk and related clinical endpoints.     

Challenges of long-term nutrition intervention studies on cognition: discordance between observational and intervention studies of vitamin B12 and cognition

Conducting long-term nutrition intervention studies on cognition can be challenging. The gaps in current methodology are addressed via a case study of the relationship between vitamin B(12) and cognition in people aged 60 and older. There is robust evidence from many observational studies, both cross-sectional and longitudinal, showing that a deficit of the vitamin is associated with poor or declining cognition in this age group, but supplementation of the vitamin in trials does not bring about improved cognition. The evidence from observational studies as well as clinical trials is reviewed here, and the potential difficulties in conducting long-term nutritional intervention studies in this area are highlighted.     

Methodological aspects in the assessment of treatment effects in observational health outcomes studies

Prospective observational studies, which provide information on the effectiveness of interventions in natural settings, may complement results from randomised clinical trials in the evaluation of health technologies. However, observational studies are subject to a number of potential methodological weaknesses, mainly selection and observer bias. This paper reviews and applies various methods to control for selection bias in the estimation of treatment effects and proposes novel ways to assess the presence of observer bias. We also address the issues of estimation and inference in a multilevel setting. We describe and compare the use of regression methods, propensity score matching, fixed-effects models incorporating investigator characteristics, and a multilevel, hierarchical model using Bayesian estimation techniques in the control of selection bias. We also propose to assess the existence of observer bias in observational studies by comparing patient- and investigator-reported outcomes. To illustrate these methods, we have used data from the SOHO (Schizophrenia Outpatient Health Outcomes) study, a large, prospective, observational study of health outcomes associated with the treatment of schizophrenia. The methods used to adjust for differences between treatment groups that could cause selection bias yielded comparable results, reinforcing the validity of the findings. Also, the assessment of observer bias did not show that it existed in the SOHO study. Observational studies, when properly conducted and when using adequate statistical methods, can provide valid information on the evaluation of health technologies.     

The Long-Term Care Data Cooperative: The Next Generation of Data Integration

Despite important advances in the linkage of residents’ Medicare claims and Minimum Data Set (MDS) information, the data infrastructure for long-term care remains inadequate for public health surveillance and clinical research. It is widely known that the evidence base supporting treatment decisions for older nursing home residents is scant as residents are systematically excluded from clinical trials. Electronic health records (EHRs) hold the promise to improve this population's representation in clinical research, especially with the more timely and detailed clinical information available in EHRs that are lacking in claims and MDS. The COVID-19 pandemic shined a spotlight on the data gap in nursing homes. To address this need, the National Institute on Aging funded the Long-Term Care (LTC) Data Cooperative, a collaboration among providers and stakeholders in academia, government, and the private sector. The LTC Data Cooperative assembles residents’ EHRs from major specialty vendors and facilitates linkage of these data with Medicare claims to create a comprehensive, longitudinal patient record. These data serve 4 key purposes: (1) health care operations and population health analytics; (2) public health surveillance; (3) observational, comparative effectiveness research; and (4) clinical research studies, including provider and patient recruitment into Phase 3 and Phase 4 randomized trials. Federally funded researchers wanting to conduct pragmatic trials can now enroll their partnering sites in this Cooperative to more easily access the clinical data needed to close the evidence gaps in LTC. Linkage to Medicare data facilitates tracking patients’ long-term outcomes after being discharged back to the community. As of August 2022, nearly 1000 nursing homes have joined, feedback reports to facilities are being piloted, algorithms for identifying infections are being tested, and proposals for use of the data have been reviewed and approved. This emerging EHR system is a substantial innovation in the richness and timeliness of the data infrastructure of the nursing home population.     

Long-Term Effects of Second-Generation Cholinesterase Inhibitors on Clinical Outcomes and Costs of Alzheimer’s Disease

Alzheimer’s disease, which is a common disorder among the elderly, not only has devastating health consequences, but also poses a substantial economic burden. Three second-generation cholinesterase inhibitors —donepezil, galantamine and rivastigmine — represent the best available treatment for patients with mild-to-moderate stages of the disease. While these drugs have been effective in short-term clinical trials, it is necessary to understand the outcomes over the longer term in order to assess the appropriateness of these treatments.Data on the effectiveness of these drugs from information beyond the short-term clinical trials (e.g. long-term clinical trials and non-trial data) are now emerging. In most cases, the results indicate that, at least for some patients, continued treatment with cholinesterase inhibitors is effective in slowing cognitive decline. Whether these results translate to clinically and economically relevant outcomes is less clear. The AD2000 trial in particular, a 1-year, placebo-controlled trial, seems to suggest that cognitive benefits, even if maintained over the mid-to-long term, may not adequately reflect overall deterioration in patients. Naturalistic studies, as well as analyses of administrative data, however, mostly suggest that these benefits are real and relevant.The cost effectiveness of treatment has been evaluated primarily through modeling. These studies have shown that the costs of treatment can be offset by savings in other areas as a result of slowed disease progression. When all medical costs are considered, relatively small delays in disease progression are required to offset treatment costs, but a large portion of these offsetting savings are a result of delayed institutionalization. To payors not responsible for institutional care costs, these predicted economic advantages are less relevant. While data are limited, some research indicates that reductions in other costs may be sufficiently large to fully offset the costs of treatment.Economic comparisons among active treatments have only been made in one study so far. Based on meta-analyses of trial data and a model estimating disease progression, that study concluded that galantamine provided the best health and economic projections. Head-to-head studies, however, are limited and provide conflicting results.While the treatment of patients with Alzheimer’s disease using cholinesterase inhibitors will likely continue to increase, there is no definitive answer regarding the appropriateness of long-term treatment. A reasonably complete answer will likely not be available until more long-term data from actual practice become available.     

Randomized clinical trials and observational studies in the assessment of drug safety

Randomized clinical trials are considered as the preferred design to assess the potential causal relationships between drugs or other medical interventions and intended effects. For this reason, randomized clinical trials are generally the basis of development programs in the life cycle of drugs and the cornerstone of evidence-based medicine. Instead, randomized clinical trials are not the design of choice for the detection and assessment of rare, delayed and/or unexpected effects related to drug safety. Moreover, the highly homogeneous populations resulting from restrictive eligibility criteria make randomized clinical trials inappropriate to describe comprehensively the safety profile of drugs. In that context, observational studies have a key added value when evaluating the benefit-risk balance of the drugs. However, observational studies are more prone to bias than randomized clinical trials and they have to be designed, conducted and reported judiciously. In this article, we discuss the strengths and limitations of randomized clinical trials and of observational studies, more particularly regarding their contribution to the knowledge of medicines’ safety profile. In addition, we present general recommendations for the sensible use of observational data.     

Empirical use of causal inference methods to evaluate survival differences in a real-world registry vs those found in randomized clinical trials

With heighted interest in causal inference based on real-world evidence, this empirical study sought to understand differences between the results of observational analyses and long-term randomized clinical trials. We hypothesized that patients deemed “eligible” for clinical trials would follow a different survival trajectory from those deemed “ineligible” and that this factor could partially explain results. In a large observational registry dataset, we estimated separate survival trajectories for hypothetically trial-eligible vs ineligible patients under both coronary artery bypass surgery (CABG) and percutaneous coronary intervention (PCI). We also explored whether results would depend on the causal inference method (inverse probability of treatment weighting vs optimal full propensity matching) or the approach to combine propensity scores from multiple imputations (the “across” vs “within” approaches). We found that, in this registry population of PCI/CABG multivessel patients, 32.5% would have been eligible for contemporaneous RCTs, suggesting that RCTs enroll selected populations. Additionally, we found treatment selection bias with different distributions of propensity scores between PCI and CABG patients. The different methodological approaches did not result in different conclusions. Overall, trial-eligible patients appeared to demonstrate at least marginally better survival than ineligible patients. Treatment comparisons by eligibility depended on disease severity. Among trial-eligible three-vessel diseased and trial-ineligible two-vessel diseased patients, CABG appeared to have at least a slight advantage with no treatment difference otherwise. In conclusion, our analyses suggest that RCTs enroll highly selected populations, and our findings are generally consistent with RCTs but less pronounced than major registry findings.