An evaluation of existing risk stratification models as a tool for comparison of surgical performances for coronary artery bypass grafting between institutions

Objective: Risk stratification systems are used in cardiac surgery to estimate mortality risk for individual patients and to compare surgical performance between institutions or surgeons. This study investigates the suitability of six existing risk stratification systems for these purposes. Methods: Data on 5471 patients who underwent isolated coronary artery bypass grafting at two UK cardiac centres between 1993 and 1999 were extracted from a prospective computerised clinical data base. Of these patients, 184 (3.3%) died in hospital. In-hospital mortality risk scores were calculated for each patient using the Parsonnet score, the EuroSCORE, the ACC/AHA score and three UK Bayes models (old, new complex and new simple). The accuracy for predicting mortality at an institutional level was assessed by comparing total observed and predicted mortality. The accuracy of the risk scores for predicting mortality for a patient was assessed by the Hosmer-Lemeshow test. The receiver operating characteristic (ROC) curve was used to evaluate how well a system ranks the patient with respect to their risk of mortality and can be useful for patient management. Results: Both EuroSCORE and the simple Bayes model were reasonably accurate at predicting overall mortality. However predictive accuracy at the patient level was poor for all systems, although EuroSCORE was accurate for low to medium risk patients. Discrimination was fair with the following ROC areas: Parsonnet 0.73, EuroSCORE 0.76, ACC/AHA system 0.76, old Bayes 0.77, complex Bayes 0.76, simple Bayes 0.76. Conclusions: This study suggests that two of the scores may be useful in comparing institutions. None of the risk scores provide accurate risk estimates for individual patients in the two hospitals studied although EuroSCORE may have some utility for certain patients. All six systems perform moderately at ranking the patients and so may be useful for patient management. More results are needed from other institutions to confirm that the EuroSCORE and the simple Bayes model are suitable for institutional risk-adjusted comparisons.