Reliability and Validity of Scores on the Emergency Severity Index Version 3

OBJECTIVES: No widely used triage instrument accurately assesses patient acuity. The Emergency Severity Index (ESI) promises to facilitate reliable acuity assessment and possibly predict patient disposition. However, reliability and validity of ESI scores have not been established in emergency departments (EDs) outside the original research sites, and version 3 (v.3) of the ESI has not been evaluated. The study hypothesis was that scores on the ESI v.3 show good interrater reliability and predict hospital admission, admission site, and death. METHODS: The authors conducted an ED-based cross-sectional retrospective study of 403 systematically selected ED records of patients who presented to an academic medical center. Twenty-seven variables were abstracted, including triage level assigned, admission status, site, and death. Using a standard process, the researchers determined the true triage level. Weighted kappa and Pearson correlation were used to calculate interrater reliability between true triage level and triage score assigned by the registered nurse (RN). The relationships between the true ESI level and admission, admission site, and death were assessed. RESULTS: Interrater reliability between RN ESI level and the true ESI level was kappa = 0.89; Pearson r = 0.83 (p < 0.001). Hospital admission by ESI level was as follows: 1 (80%), 2 (73%), 3 (51%), 4 (6%), and 5 (5%). A higher percentage of ESI level-1 and level-2 patients (40%, 12%) were admitted to the intensive care unit than ESI levels 3-5 (2%, 0%, 0%). Admission to telemetry for ESI levels 1-5 was 20%, 19%, 7%, 1%, and 0%, respectively. Three of four patients who died were ESI level 1 or 2. CONCLUSIONS: Scores on the ESI assigned by nurses have excellent interrater reliability and predict hospital admission and location of admission.     

Reliability and Validity of the Emergency Severity Index for Pediatric Triage

Objectives: The Emergency Severity Index (ESI) triage algorithm is a five‐level triage acuity tool used by emergency department (ED) triage nurses to rate patients from Level 1 (most acute) to Level 5 (least acute). ESI has established reliability and validity in an all‐age population, but has not been well studied for pediatric triage. This study assessed the reliability and validity of the ESI for pediatric triage at five sites. Methods: Interrater reliability was measured with weighted kappa for 40 written pediatric case scenarios and 100 actual patient triages at each of five research sites (independently rated by both a triage nurse and a research nurse). Validity was evaluated with a sample of 200 patients per site. The ESI ratings were compared with outcomes, including hospital admission, resource consumption, and ED length of stay. Results: Interrater reliability was 0.77 (95% confidence interval [CI] = 0.76 to 0.78) for the scenarios (n = 155 nurses) and 0.57 (95% CI = 0.52 to 0.62) for actual patients (n = 498 patients). Inconsistencies in triage were noted for the most acute and least acute patients, as well as those less than 1 year of age and those with medical (rather than trauma) chief complaints. For the validity cohort (n = 1,173 patients), outcomes differed by ESI level, including hospital admission, which went from 83% for Level 1 patients to 0% for Level 5 (chi‐square, p < 0.0001). Nurses from dedicated pediatric EDs were 31% less likely to undertriage patients than nurses in general EDs (odds ratio [OR] = 0.31, 95% CI = 0.14 to 0.67). Conclusions: Reliability of the ESI for pediatric triage is moderate. The ESI provides a valid stratification of pediatric patients into five distinct groups. We found several areas in which nurses have difficulty triaging pediatric patients consistently. The study results are being used to develop pediatric‐specific ESI educational materials to strengthen reliability and validity for pediatric triage.     

Assessment of Interrater Reliability of the Emergency Severity Index After Implementation in Emergency Departments in Jamaica Using a Learning Collaborative Approach

IntroductionIn 2016, the Ministry of Health in Jamaica selected the Emergency Severity Index as the triage tool to be used nationally. This study evaluated the effectiveness of this approach by assessing the interrater reliability among new users trained with minimal resources by 2 experienced trainers, 1 local and 1 international.MethodsA retrospective case series review was conducted within an online learning collaborative framework. After completion of the training, the participants from each of the 19 clinical sites were asked to submit 2 triage cases per month for blinded review by the expert trainers. The triage categories assigned by each reviewer were compared with those assigned by the newly trained Emergency Severity Index providers. A weighted kappa value was calculated to assess the degree of agreement between the sites and the expert trainers.ResultsA total of 166 cases were received over the study period. Participation in the learning collaborative was consistently below 50%. The interrater reliability between the expert trainers (κ = 0.48) as well as between each scorer and each accident and emergency department site (κ_SF = 0.33, κ_PT = 0.26) was low, although there was improvement over the study period. Incomplete triage documentation limited raters' ability to assign triage categories and assess interrater reliability.DiscussionDespite a rigorous implementation process, the interrater reliability of the Emergency Severity Index skills of Jamaican emergency nurses and doctors when compared with that of the 2 experts was poor. Several areas were identified for strengthening. Considerations for the implementation of the Emergency Severity Index in countries outside of the US were also discussed.     

Establishing Research Priorities for the Emergency Severity Index Using a Modified Delphi Approach

IntroductionThe accuracy of an initial ED triage decision has been reported to drive the clinical trajectory for ED patients, and, therefore, this assessment is critical to patient safety. The Emergency Severity Index—a 5-point score assigned by a triage nurse and based on disease acuity, patient potential for decompensation, and anticipated resource use—is used both in the United States and internationally. In the US, the Emergency Severity Index is used by up to 94% of the academic medical center emergency departments. In 2020, the Emergency Nurses Association acquired the intellectual property rights to the Emergency Severity Index and is responsible for its maintenance and improvement.ObjectiveThe purpose of this study was to establish a research agenda for the improvement of individual and institutional understanding and use of the Emergency Severity Index.MethodsModified Delphi process was used with 3 rounds of data collection.ResultsRound 1 yielded 112 issues, which were collapsed into 18 potential research questions in 4 general categories: education and training (6 questions), workplace environment (3 questions), emergency care services (7 questions), and special populations (2 questions). These questions were used in round 2 to establish importance. Round 3 yielded a rank ordering of both categories and research questions.DiscussionThe research priorities as set through the use of this modified Delphi process align well with current gaps in the literature. Research in these areas should be encouraged to improve the understanding of educational, environmental, and process challenges to emergency nurses’ triage decisions and accuracy of Emergency Severity Index assignments.     

Different triage categorization using Emergency Severity Index (ESI) method in emergency department

Objective

Triage is basically a categorization process to prioritize various treatments for patients based on the types of disease, severity, prognosis and resource availability. However, the term triage is more appropriate to be used in the context of natural disaster or mass casualties. Within the context of emergency situation in emergency department, the term triage refers to a method used to assess the severity of patients’ condition, determine the level of priority, and mobilize the patients to the suitable care unit. ESI is a new concept of triage using five scales in classifying the patients in emergency department. The real implementation of this concept demands nurses have to immediately make assessment about patients’ condition right away, besides they must give their final decision, whether to move the patients to the ward or to let them leave the hospital.

Comparison between Emergency Severity Index and Heart Failure Triage Scale in heart failure patients: A randomized clinical trial

BACKGROUND: It is not clear whether Emergency Severity Index (ESI) is valid to triage heart failure (HF) patients and if HF patients benefit more from a customized triage scale or not. The aim of study is to compare the effect of Heart Failure Triage Scale (HFTS) and ESI on mistriage among patients with HF who present to the emergency department (ED).METHODS: A randomized clinical trial was conducted from April to June 2017. HF patients with dyspnea were randomly assigned to HFTS or ESI groups. Triage level, used resources and time to electrocardiogram (ECG) were compared between both groups among HF patients who were admitted to coronary care unit (CCU), cardiac unit (CU) and discharged patients from the ED. Content validity was examined using Kappa designating agreement on relevance (K*). Reliability of both scale was evaluated using inter-observer agreement (Kappa).RESULTS: Seventy-three and 74 HF patients were assigned to HFTS and ESI groups respectively. Time to ECG in HFTS group was significantly shorter than that of ESI group (2.05 vs. 16.82 minutes). Triage level between HFTS and ESI groups was significantly different among patients admitted to CCU (1.0 vs. 2.8), cardiac unit (2.26 vs. 3.06) and discharged patients from the ED (3.53 vs. 2.86). Used resources in HFTS group were significantly different among triage levels (H=25.89; df=3; P<0.001).CONCLUSION: HFTS is associated with less mistriage than ESI for triaging HF patients. It is recommended to make use of HFTS to triage HF patients in the ED.     

Refining Emergency Severity Index Triage Criteria

Objectives: The Emergency Severity Index (ESI) version 3 is a five‐level triage acuity scale with demonstrated reliability and validity. Patients are rated from ESI level 1 (highest acuity) to ESI level 5 (lowest acuity). Clinical experience has demonstrated two levels of ESI level 2 patients: those who require immediate intervention and those who are stable to wait for at least ten minutes. Studies have found that few patients are rated ESI level 1, and it has been suggested that revisions to the ESI might result in appropriate reclassification of some sickest level 2 patients as level 1. The purpose of this study was to identify level 2 patients who might be reclassified as level 1 patients. Methods: This was a multisite, prospective study. The authors identified ESI level 2 patients who required immediate, lifesaving intervention and calculated chi‐square statistics and odds ratios for variables that predicted which ESI level 2 patients actually received immediate intervention. Results: Immediate lifesaving interventions were provided for 117 (20.2%) of the 589 patients included in the study. Seventeen predictors of the need for immediate intervention were identified. The strongest predictor was the triage nurse's judgment of the need for immediate intervention, especially airway and medications. Conclusions: Specific clinical findings at triage for a subset of ESI level 2 patients were associated with immediate delivery of lifesaving interventions. Revisions to the ESI level 1 criteria may be beneficial.     

3L急危重病人指引模型在急诊室的应用和效果评价

目的探讨急危重病人在急诊室的应急处理流程。方法美国北卡罗大学附属医院急诊科使用的3L急危重病人指引模型(3LevelEmergencySeveritylndexModel)做为评估工具。结果应用3L指引有效评估了342例病人,符合率88.9%。需要在1L、2L、3L规范处理的病人例数分别为20例、130例和192例,3L各层面的敏感率和特征率平均分别为84.2%和75.7%。结论3L应用简便可靠,可作为急诊室护士对危重病人快速识别的护理指引。     

The Emergency Severity Index Triage Algorithm Version 2 Is Reliable and Valid

OBJECTIVES: Initial studies have shown improved reliability and validity of a new triage tool, the Emergency Severity Index (ESI), over conventional three-level scales at two university medical centers. After pilot implementation and validation, the ESI was revised to include pediatric and updated vital signs criteria. The goal of this study was to assess ESI version (v.) 2 reliability and validity at seven emergency departments (EDs) in three states. METHODS: In part 1, interrater reliability was assessed using weighted kappa analysis of written training cases and postimplementation by a random sampling of actual patient triages. In part 2, validity was analyzed using a prospective cohort with stratified random sampling at each site. The ESI was compared with outcomes including resource consumption, inpatient admission, ED length of stay, and 60-day all-cause mortality. RESULTS: Weighted kappa analysis of interrater reliability ranged from 0.70 to 0.80 for the written scenarios (n = 3289) and 0.69 to 0.87 for patient triages (n = 386). Outcomes for the validity cohort (n = 1042) included hospitalization rates by ESI triage level: level 1, 83%; 2, 67%; 3, 42%; 4, 8%; level 5, 4%. Sixty-day all-cause mortality by triage level was as follows: level 1, 25%; 2, 4%; 3, 2%; 4, 1%; and 5, 0%. CONCLUSIONS: ESI v. 2 triage produced reliable, valid stratification of patients across seven sites. ESI triage should be evaluated as an ED casemix identification system for uniform data collection in the United States and compared with other major ED triage methods.     

Prognostic performance of Emergency Severity Index (ESI) combined with qSOFA score

Objective

We conducted this study to investigate whether ESI combined with qSOFA score (ESI + qSOFA) predicts hospital outcome better than ESI alone in the emergency department (ED).

Improving ED Emergency Severity Index Acuity Assignment Using Machine Learning and Clinical Natural Language Processing

IntroductionTriage is critical to mitigating the effect of increased volume by determining patient acuity, need for resources, and establishing acuity-based patient prioritization. The purpose of this retrospective study was to determine whether historical EHR data can be used with clinical natural language processing and machine learning algorithms (KATE) to produce accurate ESI predictive models.MethodsThe KATE triage model was developed using 166,175 patient encounters from two participating hospitals. The model was tested against a random sample of encounters that were correctly assigned an acuity by study clinicians using the Emergency Severity Index (ESI) standard as a guide.ResultsAt the study sites, KATE predicted accurate ESI acuity assignments 75.7% of the time compared with nurses (59.8%) and the average of individual study clinicians (75.3%). KATE’s accuracy was 26.9% higher than the average nurse accuracy (P <.001). On the boundary between ESI 2 and ESI 3 acuity assignments, which relates to the risk of decompensation, KATE’s accuracy was 93.2% higher, with 80% accuracy compared with triage nurses 41.4% accuracy (P <.001).DiscussionKATE provides a triage acuity assignment more accurate than the triage nurses in this study sample. KATE operates independently of contextual factors, unaffected by the external pressures that can cause under triage and may mitigate biases that can negatively affect triage accuracy. Future research should focus on the impact of KATE providing feedback to triage nurses in real time, on mortality and morbidity, ED throughput, resource optimization, and nursing outcomes.     

Emergency Severity Index Triage Category Is Associated with Six-month Survival

OBJECTIVE: The Emergency Severity Index (ESI) is a new five-level triage instrument. The objective of this study was to determine whether there is an association between ESI triage status and short-term survival. METHODS: This was a survival analysis for a population-based, stratified random sample of patients over the age of 14 years who visited an urban, university-affiliated hospital emergency department (ED). Measures included ESI triage category (1 through 5), vital status obtained from the Social Security Administration, date of death (if applicable), and survival time in days. Data were analyzed with Kaplan-Meier survival analysis. RESULTS: Eighty-seven percent (202/232) of a random sample of patients appeared in the Social Security vital status registry. During the 252-day follow-up period, 19 patients (9%) died, 161 (80%) lived, and 22 (11%) had an unknown vital status. The ESI triage level was strongly associated with vital status at six months (Kaplan-Meier chi-square 25.9, p<0.0001). No patient in triage categories 4 and 5 died (lower limits of the 95% confidence interval for survival, 92% and 93%, respectively); whereas survival in triage category 1 was 68%, and in categories 2 and 3 it was 86% and 83%, respectively. Most of the deaths occurred within 60 days after the index ED visit. Sensitivity analyses biased against the instrument continued to demonstrate the association between triage status and survival. CONCLUSIONS: The ESI triage status is associated with six-month survival. Patients with the highest three triage groups experienced decreased survival during the follow-up period, whereas all patients in the two lowest triage strata survived at least six months.     

Accuracy of the Emergency Severity Index Triage Instrument for Identifying Elder Emergency Department Patients Receiving an Immediate Life‐saving Intervention

Objectives: The study objective was to determine the sensitivity and specificity of the Emergency Severity Index (ESI) triage instrument for the identification of elder patients receiving an immediate life‐saving intervention in the emergency department (ED). Methods: The authors reviewed medical records for consecutive patients 65 years or older who presented to a single academic ED serving a large community of elders during a 1‐month period. ESI triage scores were compared to actual ED course with attention to the occurrence of an immediate life‐saving intervention. The sensitivity and specificity of an ESI triage level of 1 for the identification of patients receiving an immediate intervention was calculated. For 50 cases, the triage nurse ESI designation was compared to the triage level determined by an expert triage nurse based on retrospective record review. Results: Of 782 consecutive patients 65 years or older who presented to the ED, 18 (2%) had an ESI level of 1, 176 (23%) had an ESI level of 2, 461 (60%) had an ESI level of 3, 100 (13%) had an ESI level of 4, and 18 (2%) had an ESI level of 5. Twenty‐six patients received an immediate life‐saving intervention. ESI triage scores for these 26 individuals were as follows: ESI 1, 11 patients; ESI 2, nine patients; and ESI 3, six patients. The sensitivity of ESI to identify patients receiving an immediate intervention was 42.3% (95% confidence interval [CI] = 23.3% to 61.3%); the specificity was 99.2% (95% CI = 98.0% to 99.7%). For 17 of 50 cases in which actual triage nurse and expert nurse ESI levels disagreed, undertriage by the triage nurses was more common than overtriage (13 vs. 4 patients). Conclusions: The ESI triage instrument identified fewer than half of elder patients receiving an immediate life‐saving intervention. Failure to follow established ESI guidelines in the triage of elder patients may contribute to apparent undertriage. ACADEMIC EMERGENCY MEDICINE 2010; 17:238–243 © 2010 by the Society for Academic Emergency Medicine     

Emergency Department Triage of Patients Infected with HIV

OBJECTIVE: The emergency department (ED) and HIV specialty clinics are primary sources of care for persons infected with HIV. HIV disease may be complicated by vague and complex symptomatology, and determining the degree of illness at triage is often difficult. The goals of this project were to characterize the ED presentation of HIV-related conditions, to develop a clinical decision rule to triage HIV-infected patients, and to validate the rule in clinical practice. METHODS: The study population consisted of ambulatory patients with self-reported HIV infection who presented for care to the ED of a 553-bed public hospital that serves a medically indigent, minority population. An Illness Severity Instrument was developed by an expert panel to serve as the criterion standard for defining medical urgency for HIV-infected patients presenting to the ED for care. Two phases of the study were conducted. Data from the first phase, a noninterventional cohort study, were used to develop a clinical decision rule for the ED triage of HIV-infected patients. The second phase was a prospective validation of the clinical decision rule. RESULTS: During phase I, data from 542 patient visits were collected. Data from 441 (81%) patient visits were used in a classification and regression tree (CART) analysis to produce a decision rule, the Clinical Triage Instrument. During phase II, the prospective validation of the Clinical Triage Instrument, 156 patient visits occurred. Of these, 88 (56%) patient visits were triaged using the Clinical Triage Instrument and could be scored using the Illness Severity Instrument. The Clinical Triage Instrument accurately triaged 45 [51%; 95% confidence interval (95% CI) = 40% to 62%] patient visits, undertriaged 11 (13%; 95% CI = 6% to 21%) patient visits, and overtriaged 32 (36%; 95% CI = 26% to 47%) patient visits. Sensitivities and specificities for determining emergent, urgent, and nonurgent medical conditions by the Clinical Triage Instrument were 56% (95% CI = 31% to 75%) and 84% (95% CI = 74% to 92%), 71% (95% CI = 55% to 84%) and 39% (95% CI = 25% to 55%), and 18% (95% CI = 6% to 37%) and 93% (95% CI = 84% to 98%), respectively. The positive and negative predictive values for determining an emergent medical condition using the Clinical Triage Instrument were 48% (95% CI = 26% to 70%) and 88% (95% CI = 78% to 95%), respectively. The positive and negative predictive values for determining a nonurgent medical condition using the Clinical Triage Instrument were 56% (95% CI = 21% to 86%) and 71% (95% CI = 60% to 81%), respectively. CONCLUSIONS: The Clinical Triage Instrument was not sufficiently accurate for clinical use. Until accurate and reliable triage methods are developed, all patients infected with HIV who present to the ED for care should receive timely evaluation and care.     

网络智能信息化预检分诊系统在急诊中的应用

目的:探讨网络智能信息化预检分诊系统在急诊中的应用方法及效果。方法:随机选取2014年1月1日~2016年12月31日急诊就诊患者500例为对照组,采用传统的急诊预检分诊方法;随机选取2017年1月1日~2019年12月31日急诊就诊患者500例为观察组,采用网络智能信息化预检分诊系统进行预检分诊。比较两组分诊准确率、分诊所需时间及患者就医满意度。结果:两组分诊准确率、分诊所需时间、患者就医满意度比较差异有统计学意义(P<0.05,P<0.01)。结论:网络智能信息化预检分诊系统可有效提高患者分诊准确率,缩短分诊时间,提高就医满意度。     

Derivation of a Triage Algorithm for Chest Radiography of Community-acquired Pneumonia Patients in the Emergency Department

Background:  Community-acquired pneumonia (CAP) accounts for 1.5 million emergency department (ED) patient visits in the United States each year.
Objectives:  To derive an algorithm for the ED triage setting that facilitates rapid and accurate ordering of chest radiography (CXR) for CAP.
Methods:  The authors conducted an ED-based retrospective matched case–control study using 100 radiographic confirmed CAP cases and 100 radiographic confirmed influenzalike illness (ILI) controls. Sensitivities and specificities of characteristics assessed in the triage setting were measured to discriminate CAP from ILI. The authors then used classification tree analysis to derive an algorithm that maximizes sensitivity and specificity for detecting patients with CAP in the ED triage setting.
Results:  Temperature greater than 100.4°F (likelihood ratio = 4.39, 95% confidence interval [CI] = 2.04 to 9.45), heart rate greater than 110 beats/minute (likelihood ratio = 3.59, 95% CI = 1.82 to 7.10), and pulse oximetry less than 96% (likelihood ratio = 2.36, 95% CI = 1.32 to 4.20) were the strongest predictors of CAP. However, no single characteristic was adequately sensitive and specific to accurately discriminate CAP from ILI. A three-step algorithm (using optimum cut points for elevated temperature, tachycardia, and hypoxemia on room air pulse oximetry) was derived that is 70.8% sensitive (95% CI = 60.7% to 79.7%) and 79.1% specific (95% CI = 69.3% to 86.9%).
Conclusions:  No single characteristic adequately discriminates CAP from ILI, but a derived clinical algorithm may detect most radiographic confirmed CAP patients in the triage setting. Prospective assessment of this algorithm will be needed to determine its effects on the care of ED patients with suspected pneumonia.     

Comparison of Triage Assessments among Pediatric Registered Nurses and Pediatric Emergency Physicians

OBJECTIVES: To compare triage level assignments, using simulated written case scenarios, in a pediatric emergency department (ED) among registered nurses (RNs) and pediatric emergency physicians (PEPs) and to compare the triage level assignments among RNs and PEPs with a consensus criterion standard. METHODS: This was a cross-sectional mailed questionnaire survey. The study was conducted at a pediatric tertiary care center with more than 65,000 annual patient visits. Participants were PEPs and RNs working in the ED. Dillman's Total Design Method, with three mailouts, was used for questionnaire construction and implementation. The survey included 55 case scenarios of patients presenting to the ED. Participants were instructed to assign triage level on each case, using the following four-level triage scale: 1 = resuscitation/emergent, 2 = urgent, 3 = less-urgent, and 4 = non-urgent. A priori, all cases were assigned a triage level by consensus agreement of three PEPs, using established triage guidelines from the RNs' teaching manual. Kappa statistics (95% CI) and the mean percentage of correct responses (+/-1 SD) were calculated. RESULTS: There was a 100% response rate (39 RNs, 24 PEPs). The kappa level of agreement (95% CI) was 0.453 (0.447 to 0.459) among the RNs and was 0.419 (0.409 to 0.429) among the PEPs. The mean percentage of correct responses (+/-1 SD) for the RNs was 64.2% (+/-8.0%) and for the PEPs was 53.5% (+/-8.1%, p < 0.01). There was no significant difference within groups by experience level (< 10 vs. > or =10 years) or by the type of work schedule (day vs. evening vs. overnight) or full-time vs. part-time status. CONCLUSIONS: The level of agreement and accuracy of triage assignment was only moderate for both RNs and PEPs. Triage, a crucial step in emergency care, requires improved measurement.