A comparison of severity of illness scoring systems for elderly patients with severe pneumonia

OBJECTIVE: To evaluate the predictive ability of three severity of illness scoring systems in elderly patients with severe pneumonia requiring mechanical ventilation compared to a younger age group. DESIGN: Prospective cohort study. SETTING: Two university-affiliated tertiary care hospitals. PATIENTS AND PARTICIPANTS: One hundred four patients 75 years of age and older and 253 patients younger than 75 years of age enrolled from medical intensive care units. MEASUREMENTS AND RESULTS: Probabilities of hospital death for patients were estimated by the Acute Physiology and Chronic Health Evaluation (APACHE) II, the Mortality Probability Model (MPM) II and the Simplified Acute Physiology Score (SAPS) II. Predicted risks of hospital death were compared with observed outcomes using three methods of assessing the overall goodness of fit. The actual mortality of the elderly group was 54.87 % (95 % confidence interval [CI]: 45.2-64.4 %) compared to 28.9 % (95 % CI, 23.3-34.4 %) in the younger age group. There was a significant difference in the predictive accuracy of the scoring systems as assessed by the c-index, which is equivalent to the area under the receiver operator characteristics (ROC) curve, between the two groups, but not within individual groups. Calibration was insufficient for APACHE II and SAPS II in the elderly cohort as in-hospital mortality was lower than the predicted mortality for both models. CONCLUSIONS: Although the three severity of illness scoring systems (APACHE II, MPM II and SAPS II) demonstrated average discrimination when applied to estimate hospital mortality in the elderly patients with severe pneumonia, MPM II had the closest fit to our database. Alternative modeling approaches might be needed to customize the model coefficients to the elderly population for more accurate probabilities or to develop specialized models targeted to the designed population.     

Mortality discrimination in acute myocardial infarction: comparison between APACHE III and SAPS II prognosis systems

Objective: To compare the Acute Physiology, Age and Chronic Health Evaluation (APACHE) III with the Simplified Acute Physiology Score (SAPS II) in discriminating in-hospital mortality for intensive care unit (ICU) patients with acute myocardial infarction (AMI). Design: Prospective, observational, multicenter study. Setting: 70 Spanish ICUs. Patients and participants: 1711 patients with AMI and representative of Spanish ICUs. Measurements and results: APACHE III score, APACHE III system probability of death (APACHE III probability), SAPS II score and in-hospital mortality were noted for each patient. Two hundred and twenty three (13.0 %) patients died in the hospital. The sensitivity (± SE), specificity (± SE), and accuracy (± SE) for the APACHE III score were, respectively, 75.8 ± 2.9, 75.9 ± 1.1, and 75.9 ± 1.0. The corresponding figures for APACHE III probability were 75.3 ± 2.9, 79.2 ± 1.1, and 78.7 ± 1.0, and for SAPS II 72.2 ± 3.0, 75.9 ± 1.1, and 75.4 ± 1.0. Conclusions: The results indicate good discrimination by the three tests. APACHE III probability shows a statistically significant improvement in accuracy and specificity when compared with the two scores. Received: 4 July 1996 Accepted: 27 November 1996     

Severity scores in respiratory intensive care: APACHE II predicted mortality better than SAPS II

BACKGROUND: In recent years several scoring systems have been developed to describe the severity of illness, to establish the individual prognosis, and to group adult ICU patients by predicted risk of mortality. In addition, these scores can be used to measure and/or compare the quality of care in different ICUs. We compared the mortality predictions of the Acute Physiology and Chronic Health Evaluation (APACHE II) score and a new Simplified Acute Physiology Score (SAPS II) in patients with respiratory disease who require intensive care. PATIENTS & METHODS: We prospectively studied all 306 admissions from January 1, 1992 through December 31, 1994. McNemar and Hosmer-Lemeshow tests, and receiver operating characteristic (ROC) curves were used to describe and analyze our data. RESULTS: The average APACHE II score was 17.5 (SD 6.0), corresponding to a mean predicted death rate of 24.9% (SD 17.2%) as compared to an observed overall RICU mortality rate of 21.6%. The average SAPS II score was 39.1 (SD 11.1) corresponding to a mean predicted death rate of 26.0% (SD 18.4%). The ratio between the actual and predicted hospital mortality was 86% for APACHE II and 83% for SAPS II. Survivors had a significantly lower predicted risk of death than nonsurvivors (p < 0.0001) with both indices, and a higher Glasgow coma scale score (p < 0.0001). The ROC-curve analysis suggested the superior predictive ability of APACHE II in our patients. Area under the APACHE II ROC curve was 80.88% (standard error [SE] 2.89%), significantly larger (p < 0.01) than that found for SAPS II (73.52%, SE 3.61%). CONCLUSIONS: The APACHE II score was a good predictor of hospital outcome and better than SAPS II in our population.     

Assessment of the performance of five intensive care scoring models within a large Scottish database

OBJECTIVE: To assess and compare the performance of five severity of illness scoring systems used commonly for intensive care unit (ICU) patients in the United Kingdom. The five models analyzed were versions II and III of the Acute Physiology and Chronic Health Evaluation (APACHE) system, a version of APACHE II using United Kingdom (UK)-derived coefficients (UK APACHE II), version II of the Simplified Acute Physiology Score (SAPS), and version II of the Mortality Probability Model, computed at admission (MPM0) and after 24 hrs in the ICU (MPM24). DESIGN: A 2-yr prospective cohort study of consecutive admissions to intensive care units. SETTING: A total of 22 general ICUs in Scotland PATIENTS: A total of 13,291 admissions to the study, which after prospectively agreed exclusions left a total of 10,393 patients for the analysis. OUTCOME MEASURES: Death or survival at hospital discharge. MEASUREMENTS AND MAIN RESULTS: All the models showed reasonable discrimination using the area under the receiver operating characteristic curve (APACHE III, 0.845; APACHE II, 0.805; UKAPACHE II, 0.809; SAPS II, 0.843; MPM0, 0.785; MPM24, 0.799). The levels of observed mortality were significantly different than that predicted by all models, using the Hosmer-Lemeshow goodness-of-fit C test (p < .001), with the results of the test being confirmed by calibration curves. When excluding patients discharged in the first 24 hrs to allow for comparisons using the same patient group, APACHE III, MPM24, and SAPS II (APACHE III, 0.795; MPM24, 0.791; SAPS II, 0.784) showed significantly better discrimination than APACHE II, UK APACHE II, and MPM0 (APACHE II, 0.763; UK APACHE II, 0.756; MPM0, 0.741). However, calibration changed little for all models with observed mortality still significantly different from that predicted by the scoring systems (p < .001). For equivalent data sets, APACHE II demonstrated superior calibration to all the models using the chi-squared value from the Hosmer-Lemeshow test for both populations (APACHE III, 366; APACHE II, 67; UKAPACHE II, 237; SAPS II, 142; MPM0, 452; MPM24, 101). CONCLUSIONS: SAPS II demonstrated the best overall performance, but the superior calibration of APACHE II makes it the most appropriate model for comparisons of mortality rates in different ICUs. The significance of the Hosmer-Lemeshow C test in all the models suggest that new logistic regression coefficients should be generated and the systems retested before they could be used with confidence in Scottish ICUs.     

Is a specific oncological scoring system better at predicting the prognosis of cancer patients admitted for an acute medical complication in an intensive care unit than general gravity scores?

Objective To evaluate the effectiveness of a specific oncologic scoring system—the ICU Cancer Mortality model (ICM)—in predicting hospital mortality in comparison to two general severity scores—the Acute Physiology and Chronic Health Evaluation (APACHE II) and the Simplified Acute Physiology Score (SAPS II).Patients and methods All 247 patients admitted for a medical acute complication over an 18-month period in an oncological medical intensive care unit were prospectively registered. Their data, including type of complication, vital status at discharge and cancer characteristics as well as other variables necessary to calculate the three scoring systems were retrospectively assessed.Results Observed in-hospital mortality was 34%. The predicted in-hospital mortality rate for APACHE II was 32%; SAPS II, 24%; and ICM, 28%. The goodness of fit was inadequate except for the ICM score. Comparison of the area under the ROC curves revealed a better fit for ICM (area 0.79). The maximum correct classification rate was 72% for APACHE II, 74% for SAPS II and 77% for ICM. APACHE II and SAPS II were better at predicting outcome for survivors to hospital discharge, although ICM was better for non-survivors. Two variables were independently predicting the risk of death during hospitalisation: ICM (OR=2.31) and SAPS II (OR=1.05).Conclusions Gravity scores were the single independent predictors for hospital mortality, and ICM was equivalent to APACHE II and SAPS II.     

Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit

OBJECTIVE: To evaluate Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II scoring systems in a single intensive care unit (ICU), independent from the ICUs of the developmental sample; and to compare the performance of APACHE II and SAPS II by means of statistical analyses in such a clinical setting. DESIGN: Prospective, cohort study. SETTING: A single ICU in a Greek university hospital. PATIENTS: In a time interval of 5 yrs, data for 681 patients admitted to our ICU were collected. The original exclusion criteria of both systems were employed. Patients <17 yrs of age were dropped from the study to keep compatibility with both systems. Eventually, a total of 661 patients were included in the analysis. INTERVENTIONS: Demographics, clinical parameters essential for the calculation of APACHE II and SAPS II scores, and risk of hospital death were recorded. Patient vital status was followed up to hospital discharge. MEASUREMENTS AND MAIN RESULTS: Both systems showed poor calibration and underestimated mortality but had good discriminative power, with SAPS II performing better than APACHE II. The evaluation of uniformity of fit in various subgroups for both systems confirmed the pattern of underprediction of mortality from both models and the better performance of APACHE II over our data sample. CONCLUSIONS: APACHE II and SAPS II failed to predict mortality in a population sample other than the one used for their development. APACHE II performed better than SAPS II. Validation in such a population is essential. Because there is a great variation in clinical and other patient characteristics among ICUs, it is doubtful that one system can be validated in all types of populations to be used for comparisons among different ICUs.     

Validation of severity scoring systems SAPS II and APACHE II in a single-center population

OBJECTIVE: To validate two severity scoring systems, the Simplified Acute Physiology Score (SAPS II) and Acute Physiology and Chronic Health Evaluation (APACHE II), in a single-center ICU population. DESIGN AND SETTING: Prospective data collection in a two four-bed multidisciplinary ICUs of a teaching hospital. PATIENTS AND METHODS: Data were collected in ICU over 4 years on 1,721 consecutively admitted patients (aged 18 years or older, no transferrals, ICU stay at least 24 h) regarding SAPS II, APACHE II, predicted hospital mortality, and survival upon hospital discharge. RESULTS: At the predicted risk of 0.5, sensitivity was 39.4 % for SAPS II and 31.6 % for APACHE II, specificity 95.6 % and 97.2 %, and correct classification rate 85.6 % and 85.5 %, respectively. The area under the ROC curve was higher than 0.8 for both models. The goodness-of-fit statistic showed no significant difference between observed and predicted hospital mortality (H = 7.62 for SAPS II, H = 3.87 for APACHE II; and C = 9.32 and C = 5.05, respectively). Observed hospital mortality of patients with risk of death higher than 60 % was overpredicted by SAPS II and underpredicted by APACHE II. The observed hospital mortality was significantly higher than that predicted by the models in medical patients and in those admitted from the ward. CONCLUSIONS: This study validates both SAPS II and APACHE II scores in an ICU population comprised mainly of surgical patients. The type of ICU admission and the location in the hospital before ICU admission influence the predictive ability of the models.     

Mortality predictions using current physiologic scoring systems in patients meeting criteria for early goal-directed therapy and the severe sepsis resuscitation bundle

Physiologic scoring systems are often used to prognosticate mortality in critically ill patients. This study examined the performance of Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, Mortality in Emergency Department Sepsis (MEDS), and Mortality Probability Models (MPM) II0 in predicting in-hospital mortality of patients in the emergency department meeting criteria for early goal-directed therapy and the severe sepsis resuscitation bundle. The discrimination and calibration characteristics of APACHE II, SAPS II, MEDS, and MPM II0 were evaluated. Data are presented as median and quartiles (25th, 75th). Two-hundred forty-six patients aged 68 (52, 81) years were analyzed from a prospectively maintained sepsis registry, with 76.0% of patients in septic shock, 45.5% blood culture positive, and 35.0% in-hospital mortality. Acute Physiology and Chronic Health Evaluation II, SAPS II, and MEDS scores were 29 (21, 37), 54 (40, 70), and 13 (11, 16), with predicted mortalities of 64% (40%, 85%), 58% (25%, 84%), and 16% (9%, 39%), respectively. Mortality Probability Models II0 showed a predicted mortality of 60% (27%, 80%). The area under the receiver operating characteristic curves was 0.73 for APACHE II, 0.71 for SAPS II, 0.60 for MEDS, and 0.72 for MPM II0. The standardized mortality ratios were 0.59, 0.63, 1.68, and 0.64, respectively. Thus, APACHE II, SAPS II, MEDS, and MPM II0 have variable abilities to discriminate early and estimate in-hospital mortality of patients presenting to the emergency department requiring the severe sepsis resuscitation bundle. Adoption of these prognostication tools in this setting may influence therapy and resource use for these patients.     

Training in data definitions improves quality of intensive care data

Background  

Our aim was to assess the contribution of training in data definitions and data extraction guidelines to improving quality of data for use in intensive care scoring systems such as the Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II in the Dutch National Intensive Care Evaluation (NICE) registry.     

Outcome prediction in intensive care: results of a prospective, multicentre, Portuguese study

Objective: To compare the performance of the New Simplified Acute Physiology Score (SAPS II) and Acute Physiology and Chronic Health Evaluation (APACHE) II in an independent database, using formal statistical assessment. Design: Analysis of the database of a multicentre, prospective study. Setting: 19 intensive care units (ICUs) in Portugal. Patients: Data for 1094 patients consecutively admitted to the ICUs were collected over a period of 4 months. Following the original SAPS II and APACHE II criteria, the analysis excluded patients younger than 18 years of age, readmissions, acute myocardial infarction, burns, patients in the post-operative period after coronary artery bypass surgery, and patients with a length of stay in the ICU of less than 24 h. The group analysed comprised 982 patients. Interventions: Collection of the first 24 h admission data necessary for the calculation of SAPS II, APACHE II, Therapeutic Intervention Scoring System (TISS), Simplified TISS, organ system failure and basic demographic statistics. Vital status at discharge from the hospital was registered. Measurements and results: In this cohort, discrimination was better for SAPS II than for APACHE II (SAPS II: area under the receiver operating characteristic curve 0.817, standard error 0.015; APACHE II: 0.787, 0.015; p < 0.001); however, both models presented a poor calibration, with significant differences between observed and predicted mortality (Hosmer-Lemeshow goodness-of-fit tests H and C, p < 0.001). In a stratified analysis, this study was unable to demonstrate any definite pattern of association between the poor performance of the models and specific subgroups of patients except for the most severely ill patients, where both models overestimated mortality. Conclusions: SAPS II performed better than APACHE II in this independent database, but the results do not allow its use, at least without being customised, to analyse quality of care or performance among ICUs in the target population. Received: 2 April 1996 Accepted: 24 October 1996     

The use of intensive care information systems alters outcome prediction

Objective: To study the effect of using an Intensive Care Information System (ICIS) on severity scores and prognostic indices: Acute Physiology and Chronic Health Evaluation II (APACHE II), Simplified Acute Physiology Score II (SAPS II), and Mortality Probability Models II (MPM II). Design: Prospective pilot study. Setting: A 20-bed medical-surgical intensive care unit (ICU) in a teaching hospital. Patients: 50 consecutive adult patients admitted to the ICU on a bed equipped with an ICIS. Interventions: None. Measurements and results: In each patient all the physiologic variables, as required by the severity scores, were both manually charted and recorded by ICIS. ICIS registration resulted in the extraction of more abnormal values for all physiologic variables (except temperature): p < 0.05. Higher severity scores and mortality prediction were achieved by using ICIS charting: predicted mortality increased by 15 % for APACHE II compared to manual charting, 25 % for SAPS II, and 24 % for MPM₀. ICIS charting resulted in higher severity scores and mortality prediction for 29 of the 50 patients using APACHE II with a mean increase in mortality prediction in this subgroup of 27 %. In the case of SAPS II, ICIS charting resulted in higher scores in 23 of the 50 patients and in the case of MPM₀ in 13 patients, the mean increase in mortality in these subgroups being 64 and 148 %, respectively. Conclusions: The use of ICIS charting to acquire the most abnormal physiologic values for severity scores and the derived prognostic indices results in a higher mortality prediction. Comparison of groups of patients and/or ICUs based on severity scores is impossible without standardization of data collection. The mortality prediction models have to be revalidated for the use of ICIS charting. While awaiting this, we suggest that every patient record in local regional, national, or international ICU databases should be marked as being recorded by manual or by ICIS charting. Received: 16 December 1997 Accepted: 11 June 1998     

Comparison of three severity scores for critically ill cancer patients

Objective To compare three scoring systems, the Acute Physiology and Chronic Health Evaluation (APACHE) II, the Simplified Acute Physiology Score (SAPS) II and a modified Mortality Probability Model II (ICU cancer mortality model, ICMM) for their prognostic value for mortality during hospital stay in a group of cancer patients admitted to a medical ICU.Design Prospective cohort study.Setting Medical ICU of a tertiary care hospital.Patients Two hundred forty-two consecutive cancer patients admitted to the ICU.Measurements and results Variables included in APACHE II, SAPS II and the ICMM scores as well as demographic data were assessed during the first 24 h of stay in the ICU. Hospital mortality was measured; it was 44%. Calibration for all three scoring systems was acceptable, SAPS II yielded a significantly superior discrimination between survivors and non-survivors. The areas under the receiver operating characteristic curves were 0.776 for APACHE II, 0.825 for SAPS II and 0.698 for the ICMM.Conclusion The SAPS II was superior to APACHE II and ICMM. The newly developed ICMM does not improve mortality prediction in critically ill cancer patients.     

Comparison of acute physiology and chronic health evaluations II and III and simplified acute physiology score II: a prospective cohort study evaluating these methods to predict outcome in a German interdisciplinary intensive care unit

OBJECTIVE: To evaluate the ability of three scoring systems to predict hospital mortality in adult patients of an interdisciplinary intensive care unit in Germany. DESIGN: A prospective cohort study. SETTING: A mixed medical and surgical intensive care unit at a teaching hospital in Germany. PATIENTS: From a total of 3,108 patients, 2,795 patients (89.9%) for Acute Physiology and Chronic Health Evaluation (APACHE) II and 2,661 patients (85.6%) for APACHE III and Simplified Acute Physiology Score (SAPS) II could be enrolled to the study because of defined exclusion criteria. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Probabilities of hospital death for patients were estimated by applying APACHE II and III and SAPS II and compared with observed outcomes. The overall goodness-of-fit of the three models was assessed. Hospital death rates were equivalent to those predicted by APACHE II but higher than those predicted by APACHE III and SAPS II. Calibration was good for APACHE II. For the other systems, it was insufficient, but better for SAPS II than for APACHE III. The overall correct classification rate, applying a decision criterion of 50%, was 84% for APACHE II and 85% for APACHE III and SAPS II. The areas under the receiver operating characteristic curve were 0.832 for APACHE II and 0.846 for APACHE III and SAPS II. Risk estimates for surgical and medical admissions differed between the three systems. For all systems, risk predictions for diagnostic categories did not fit uniformly across the spectrum of disease categories. CONCLUSIONS: Our data more closely resemble those of the APACHE II database, demonstrating a higher degree of overall goodness-of-fit of APACHE II than APACHE III and SAPS II. Although discrimination was slightly better for the two new systems, calibration was good with a close fit for APACHE II only. Hospital mortality was higher than predicted for both new models but was underestimated to a greater degree by APACHE III. Both score systems demonstrated a considerable variation across the spectrum of diagnostic categories, which also differed between the two models.     

Outcome of critically ill patients treated with intermittent high-volume haemofiltration: a prospective cohort analysis

Objective: To evaluate intervention and outcome in critically ill patients treated with high-volume haemofiltration (HV-HF). Design: Prospective cohort analysis. Setting: 18-bed closed format general intensive care unit (ICU) of a teaching hospital. Patients: 30-month cohort of ICU patients treated with HV-HF. Interventions: Intermittent high-volume venovenous haemofiltration. Endpoints: Observed and predicted mortality in prospectively stratified prognostic groups. Measurements and results: Clinical and filtration data, Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II and the Madrid Acute Renal Failure (ARF) score and predicted mortality. A total of 306 patients were haemofiltrated (140 medical, 166 surgical), 52 % were oliguric. Mean APACHE II score was 31 (SD 8) and mean SAPS II score 60 (SD 16). Mean ultrafiltrate rate was 63 ml/min (SD 20). A median total of 160 litres (90 % range 49 to 453) were filtrated per patient, material costs were 565 ECU (90 % range 199 to 1514). ICU mortality was 33 %, hospital mortality 40 % [95 % confidence interval (CI) 34 to 45], predicted mortality by the ARF score 67 % (CI 66 to 69). Non-cardiac surgery mortality was 47 % (CI 39 to 54), 73 % (CI 70 to 76) predicted by APACHE II and 67 % (CI 64 to 70) by SAPS II. Observed mortality was significantly lower than predicted in all prognostic groups. The standardised mortality ratio (SMR) was no higher than the SMR in the overall ICU population. Conclusions: Mortality in HV-HF patients was lower than that predicted by illness severity scores, as was the case in all patients in our ICU. Treatment with HV-HF appears to be safe and feasible. The efficacy of HV-HF should be tested in randomised, controlled trials of suitable power. Received: 4 December 1998 Final revision received: 20 April 1999 Accepted: 17 May 1999     

Performance assessment of the Simplified Acute Physiology Score II,the Acute Physiology and Chronic Health Evaluation II score,and the Sequential Organ Failure Assessment score in predicting the outcomes of adult patients with hepatic portal venous gas in the ED

ObjectiveThis study aims to evaluate the performance of Simplified Acute Physiology Score II (SAPS II), the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, and the Sequential Organ Failure Assessment (SOFA) score for predicting illness severity and the mortality of adult hepatic portal venous gas (HPVG) patients presenting to the emergency department (ED). This will assist emergency physicians in risk stratification.MethodsData for 48 adult HPVG patients who visited our ED between December 2009 and December 2013 were analyzed. The SAPS II, APACHE II score, and SOFA score were calculated based on the worst laboratory values in the ED. The probability of death was calculated for each patient based on these scores. The ability of the SAPS II, APACHE II score, and SOFA score to predict group mortality was assessed by using receiver operating characteristic curve analysis and calibration analysis.ResultsThe sensitivity, specificity, and accuracy were 92.6%,71.4%, and 83.3%, respectively, for the SAPS II method; 77.8%, 81%, and 79.2%, respectively, for the APACHE II scoring system, and 77.8%, 76.2%, and 79.2%, respectively, for the SOFA score. In the receiver operating characteristic curve analysis, the areas under the curve for the SAPS II, APACHE II scoring system, and SOFA score were 0.910, 0.878, and 0.809, respectively.ConclusionThis is one of the largest series performed in a population of adult HPVG patients in the ED. The results from the present study showed that SAPS II is easier and more quickly calculated than the APACHE II and more superior in predicting the mortality of ED adult HPVG patients than the SOFA. We recommend that the SAPS II be used for outcome prediction and risk stratification in adult HPVG patients in the ED.     

Simple translational equations to compare illness severity scores in intensive care trials

Purpose

Comparison of illness severity for intensive care unit populations assessed according to different scoring systems should increase our ability to compare and meta-analyze past and future trials but is currently not possible. Accordingly, we aimed to establish a methodology to translate illness severity scores obtained from one system into another.

Materials and methods

Using the Australian and New-Zealand intensive care adult patient database, we obtained simultaneous admission Acute Physiology and Chronic Health Evaluation (APACHE) II and APACHE III scores and Simplified Acute Physiology Score (SAPS) II in 634 428 patients admitted to 153 units between 2001 and 2010. We applied linear regression analyses to create models enabling translation of one score into another. Sensitivity analyses were performed after removal of diagnostic categories excluded from the original APACHE database, after matching for similar risk of death, after splitting data according to country of origin (Australia or New Zealand) and after splitting admissions occurring before or after 2006.

Results

The translational models were APACHE III = 3.08 × APACHE II + 5.75; APACHE III = 1.47 × SAPS II + 8.6; and APACHE II = 0.36 × SAPS II + 4.4. The area under the receiver operating curve for mortality prediction was 0.853 (95% confidence interval, 0.851-0.855) for the “APACHE II derived APACHE III” score and 0.854 (0.852-0.855) for the “SAPS II derived APACHE III” vs 0.854 (0.852-0.855) for the original APACHE III score. Similarly, it was 0.841 (0.839-0.843) for the “SAPS II derived APACHE II score” vs 0.842 (0.840-0.843) for the original APACHE II score. Correlation coefficients as well as intercepts remained very similar in all subgroups analyses.

Conclusions

Simple and robust translational formulas can be developed to allow clinicians to compare illness severity between studies involving critically ill patients. Further studies in other countries and health care systems are needed to confirm the generalizability of these results.     

Outcome prediction models on admission in a medical intensive care unit: do they predict individual outcome?

Prospectively acquired data from 941 patients staying greater than 24 h in a medical ICU were analyzed to determine the relevance of scoring on ICU admission by the following methods of outcome prediction: Acute Physiology and Chronic Health Evaluation (APACHE II), Simplified Acute Physiology Score (SAPS), and Mortality Prediction Model (MPM). Analysis was performed separately for all patients (group A) and for a subsample (group B), obtained by excluding coronary care patients. Calculation of risk and classification of patients were carried out as recommended in the literature for MPM, APACHE II, and SAPS. In group A, sensitivities (correct prediction of hospital mortality) were 44.7%, 51.1%, and 21.2% and specificities (correct prediction of survival) were 84.5%, 85.4%, and 96.8%, respectively; overall correct classification rates were 73.3%, 75.8%, and 75.6%. In group B, sensitivities were slightly higher, but total correct classification rates did not reach group A levels. Goodness-of-fit testing showed low levels of fit for all methods in both groups. Application of APACHE II to diagnostic subgroups, using disease-adapted risk calculations, revealed marked inconsistencies between the estimated risk and the observed mortality. We conclude that the estimation of risk on admission by the three methods investigated might be helpful for global comparisons of ICU populations, although the lack of disease specificity reduces their applicability for severity grading of a given illness. The inaccuracy of these methods makes them ineffective for predicting individual outcome; thus, they provide little advantage in clinical decision-making.     

Validation of the new Intensive Care Nursing Scoring System (ICNSS)

OBJECTIVES: To validate a new Intensive Care Nursing Scoring System (ICNSS). DESIGN: Retrospective data collection. SETTING: Adult 19-bed intensive care unit (ICU) in a tertiary care university hospital. PATIENTS: A total of 1,538 patient records of which 30 documents were included in the validation. MEASUREMENTS AND RESULTS: Data included admission scores of the Acute Physiology and Chronic Health Evaluation II (APACHE II) and the Simplified Acute Physiology Scores II (SAPS II), daily Therapeutic Intervention Scores (TISS) and ICNSS scores. Data were compared using Spearman's correlation, t-test and chi-square test. Receiver operating characteristics (ROC) curve analysis was used to assess the ability of ICNSS and TISS to predict mortality. Intra-class correlation, percentage agreement and kappa statistics were used to test the validity of given scores. Nursing workload assessment using ICNSS showed that medical and emergency-operated patients caused a greater nursing workload than electively operated patients (p<0.001). Six variables of the sub-scale that described vital function nursing accounted for 27.4% of the variation of SAPS II and for 37% of the variation of APACHE II. The ICNSS sub-scale of vital function nursing accounted for a ROC area of 0.91. In the validity of the given ICNSS scores, kappa was 0.81 and weighted kappa 0.82. CONCLUSIONS: Nursing workload varied between the different admission types. ICNSS explained a similar percentage of the variation of the admission scores of APACHE II and SAPSS II as TISS and discriminated between non-survivors and survivors. ICNSS is a suitable nursing workload instrument to be used with the TISS score.     

Operational performance of validated physiologic scoring systems for predicting in-hospital mortality among critically ill emergency department patients

OBJECTIVES: New Simplified Acute Physiology Score (SAPS) II, Morbidity Probability Model at admission (MPM0 II), and Logistic Organ Dysfunction System (LODS) have all demonstrated high accuracy for predicting mortality in intensive care unit populations. We tested the prognostic accuracy of these instruments for predicting mortality among a cohort of critically ill emergency department patients. DESIGN: Secondary analysis of a randomized controlled trial. SETTING: Urban, tertiary emergency department, census >100,000. PATIENTS: Nontrauma emergency department patients admitted to an intensive care unit, aged >17 yrs, with initial emergency department vital signs consistent with shock (systolic blood pressure <100 mm Hg or shock index >1.0), and with agreement of two independent observers for at least one sign and symptom of inadequate tissue perfusion. INTERVENTIONS: Emergency department variables needed for calculation of each scoring system were prospectively collected, and published formulas were used to calculate the probability of in-hospital death for each scoring system. The main outcome was actual in-hospital mortality. The area under the receiver operating characteristic curve was used to evaluate the predictive ability of each scoring system. MEASUREMENTS AND MAIN RESULTS: Ninety-one of 202 patients (45%) were included. The mean age was 56 +/- 16 yrs, 42% were female, the mean initial systolic blood pressure was 84 +/- 13 mm Hg, and the average length of stay in the emergency department was 4.2 +/- 2.0 hrs. The in-hospital mortality rate was 21%. The area under the receiver operating characteristic curve for calculated probability of in-hospital mortality for SAPS II was 0.72 (95% confidence interval, 0.57-0.87), for MPM0 II 0.69 (95% confidence interval, 0.54-0.84), and for LODS 0.60 (95% confidence interval, 0.45-0.76). CONCLUSIONS: Using variables available in the emergency department, three previously validated intensive care unit scoring systems demonstrated moderate accuracy for predicting in-hospital mortality.     

Impact of exclusion criteria on case mix, outcome, and length of stay for the severity of disease scoring methods in common use in critical care

OBJECTIVE: To identify the exclusion criteria for the major severity of disease scoring methods in critical care and to investigate the impact of the exclusion criteria on the case mix, outcomes and length of stay for admissions to intensive care units (ICUs) in England, Wales and Northern Ireland. DESIGN: Cohort study-analysis of prospectively collected data. SETTING: 127 adult, general (mixed medical/surgical) ICUs in England, Wales and Northern Ireland. PATIENTS: 120,503 admissions between 1995 and 2001. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Thirteen different exclusion criteria were identified from the original methodological/validation papers and data collection manuals for APACHE II, APACHE III, SAPS II and MPM II. Application of the original exclusion criteria for the four, major severity of disease scoring methods resulted in exclusion of between 11.5% and 14.6% of admissions. Hospital mortality for the overall cohort was 29.0% but ranged from 4.7% to 50.1% among those groups excluded. After application of the exclusion criteria for each scoring method, there was little difference in overall hospital mortality or median ICU and hospital length of stay for the included admissions when compared with the original cohort. At the level of individual ICUs, there were differences in hospital mortality before and after exclusions-minimum -3.1% to maximum 9.5% (APACHE II), minimum -2.8% to maximum 9.4% (APACHE III), minimum -3.1% to maximum 16.1% (SAPS II) and minimum -3.1% to maximum 16.5% (MPM II). The mean difference across individual ICUs was -0.5 % (95% CI -0.7% to -0.2%) for APACHE II, -0.2% (95% CI -0.2% to 0.1%) for APACHE III, 2.0% (95% CI 1.7% to 2.4%) for SAPS II and 2.1% (95% CI 1.7% to 2.5%) for MPM II. SAPS II and MPM II showed systematic variation. A survey of the literature found wide variation in the exclusion criteria reported in subsequent, published research using a single severity of disease scoring method (APACHE II). CONCLUSIONS: Exclusion criteria used in critical care research are often ill-defined and poorly reported. More attention to the choice of exclusion criteria and their effect on the reported results is essential. We hope this study will raise the need for both better reporting of exclusion criteria applied in studies and promote the need for a common set of explicit exclusion criteria for these methods.