Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward

Objective Physiological track and trigger warning systems (TTs) are used to identify patients outside critical care areas at risk of deterioration and to alert a senior clinician, Critical Care Outreach Service, or equivalent. The aims of this work were: to describe published TTs and the extent to which each has been developed according to established procedures; to review the published evidence and available data on the reliability, validity and utility of existing systems; and to identify the best TT for timely recognition of critically ill patients. Design and setting Systematic review of studies identified from electronic, citation and hand searching, and expert informants. Cohort study of data from 31 acute hospitals in England and Wales. Measurements and results Thirty-six papers were identified describing 25 distinct TTs. Thirty-one papers described the use of a TT, and five were studies examining the development or testing of TTs. None of the studies met all methodological quality standards. For the cohort study, outcome was measured by a composite of death, admission to critical care, ‘do not attempt resuscitation’ or cardiopulmonary resuscitation. Fifteen datasets met pre-defined quality criteria. Sensitivities and positive predictive values were low, with median (quartiles) of 43.3 (25.4–69.2) and 36.7 (29.3–43.8), respectively. Conclusion A wide variety of TTs were in use, with little evidence of reliability, validity and utility. Sensitivity was poor, which might be due in part to the nature of the physiology monitored or to the choice of trigger threshold. Available data were insufficient to identify the best TT. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users     

Reliability of intensive care unit admitting and comorbid diagnoses, race, elements of Acute Physiology and Chronic Health Evaluation II score, and predicted probability of mortality in an electronic intensive care unit database

Background

Although reliability of severity of illness and predicted probability of hospital mortality have been assessed, interrater reliability of the abstraction of primary and other intensive care unit (ICU) admitting diagnoses and underlying comorbidities has not been studied.

Methods

Patient data from one ICU were originally abstracted and entered into an electronic database by an ICU nurse. A research assistant reabstracted patient demographics, ICU admitting diagnoses and underlying comorbidities, and elements of Acute Physiology and Chronic Health Evaluation II (APACHE II) score from 100 random patients of 474 admitted during 2005 using an identical electronic database. Chamberlain's percent positive agreement was used to compare diagnoses and comorbidities between the 2 data abstractors. A κ statistic was calculated for demographic variables, Glasgow Coma Score, APACHE II chronic health points, and HIV status. Intraclass correlation was calculated for acute physiology points and predicted probability of hospital mortality.

Results

Percent positive agreement for ICU primary and other admitting diagnoses ranged from 0% (primary brain injury) to 71% (sepsis), and for underlying comorbidities, from 40% (coronary artery bypass graft) to 100% (HIV). Agreement as measured by κ statistic was strong for race (0.81) and age points (0.95), moderate for chronic health points (0.50) and HIV (0.66), and poor for Glasgow Coma Score (0.36). Intraclass correlation showed a moderate-high agreement for acute physiology points (0.88) and predicted probability of hospital mortality (0.71).

Conclusion

Reliability for ICU diagnoses and elements of the APACHE II score is related to the objectivity of primary data in the medical charts.     

Interexaminer reliability of supine leg checks for discriminating leg-length inequality

Objective

The purpose of this study was to quantify interexaminer reliability of a standardized supine leg check procedure used to screen for leg-length inequality.

Methods

Two doctors of chiropractic used a standardized supine leg check procedure to examine 50 volunteers for leg-length inequality. The order of examination was randomized. The side and magnitude of leg-length inequality were determined to the nearest 1/8 in. Subjects and examiners were blinded. Interexaminer reliability was assessed with a Bland-Altman plot, tolerance table of absolute differences, a quadratic weighted κ statistic for quantitative scores, and a Gwet's first-order agreement coefficient for dichotomous ratings.

Results

The quadratic weighted κ statistic to quantify the reliability of the rating scale was 0.44 (95% confidence interval, 0.21-0.67), indicating moderate reliability. The 2 examiners agreed exactly 32% of the time, within 1/8 in 58% of the time, within 3/16 in 72% of the time, and within 3/8 in 92% of the time. The Bland-Altman plot revealed possible heterogeneity in reliability that requires additional study. The examiners agreed on the presence of a leg-length inequality of at least 1/8 in in 40 (80%) of 50 subjects (first-order agreement coefficient, 0.76), suggesting good agreement for this diagnostic category.

Conclusion

The examiners showed moderate reliability in assessing leg-length inequality at 1/8-in increments and good reliability in determining the presence of a leg-length inequality.     

Alarm limit settings for early warning systems to identify at-risk patients

Title.  Alarm limit settings for early warning systems to identify at-risk patients.
Aim.  This paper is a report of a study conducted to provide objective data to assist with setting alarm limits for early warning systems.
Background.  Early warning systems are used to provide timely detection of patient deterioration outside of critical care areas, but with little data from the general ward population to guide alarm limit settings. Monitoring systems used in critical care areas are known for excellent sensitivity in detecting signs of deterioration, but give high false positive alarm rates, which are managed with nurses caring for two or fewer patients. On general wards, nurses caring for four or more patients will be unable to manage a high number of false alarms. Physiological data from a general ward population would help to guide alarm limit settings.
Methods.  A dataset of continuous heart rate and respiratory rate data from a general ward population, previously collected from July 2003–January 2006, was analyzed for adult patients with no severe adverse events. Dataset modeling was constructed to analyze alarm frequency at varying heart rate and respiratory rate alarm limits.
Results.  A total of 317 patients satisfied the inclusion criteria, with 780·71 days of total monitoring. Sample alarm settings appeared to optimize false positive alarm rates for the following settings: heart rate high 130–135, low 40–45; respiratory rate high 30–35, low 7–8. Rates for each selected limit can be added to calculate the total alarm frequency, which can be used to judge the impact on nurse workflow.
Conclusion.  Alarm frequency data will assist with evidence-based configuration of alarm limits for early warning systems.     

Reliability and validity of the Chinese version of the behavioral pain scale in intubated and non-intubated critically ill patients: Two cross-sectional studies

BackgroundSelf-report pain assessment scales may be inappropriate when critically ill patients are incapable of adequate communication because of sedation or mechanical ventilation. The Behavioral Pain Scale (BPS, for intubated patients) and the BPS-non intubated (BPS-NI, for non-intubated patients) measure objective behavioral indicators of pain in non-communicating critically ill patients.ObjectivesTo develop a Chinese version of the BPS combining the original version of the BPS and BPS-NI suitable for pain assessment among critically ill patients and to determine its reliability and validity.DesignTwo cross-sectional studies.SettingsA 15-bed surgical intensive care unit (ICU) in a teaching hospital in Beijing, China.ParticipantsIn the first study, 129 patients (53 intubated and 76 non-intubated) were recruited; in the second study, 83 (43 intubated and 40 non-intubated) were recruited.MethodsThe Chinese version of the BPS (BPS-C) was developed via rigorous translation methods, including double back-translation and content validation involving 13 clinical experts. Internal consistency, discriminative validity, and criterion-related validity were established using the BPS-C and the Numeric Rating Scale (NRS). The BPS-C and NRS were used to assess pain in 53 intubated and 76 non-intubated post-abdominal surgery patients during low pain exposure and increased pain exposure in the first study. To establish interrater reliability, a researcher and a bedside nurse independently performed 172 paired assessments in 43 intubated patients and 160 paired assessments in 40 non-intubated patients with the BPS-C under the same conditions in the second study.ResultsThe BPS-C achieved conceptual and semantic equivalence with the original tool. Internal consistency was established through Cronbach's alpha (α = 0.724–0.743 in intubated patients, α = 0.701–0.762 in non-intubated patients). Interrater reliability was confirmed through the intraclass correlation coefficients (ICCs), which ranged from 0.962 to 1.000 in both intubated and non-intubated patients with high agreement percentages (95.3–100.0% in intubated and 95.0–100.0% in non-intubated patients). BPS-C scores during increased exposure to pain were significantly higher than those obtained during low exposure to pain, indicating discriminative validity. Criterion-related validity was confirmed by strong positive correlations between BPS-C and NRS scores (Pearson's correlations r = 0.815–0.937 for intubated patients, Pearson's correlations r = 0.755–0.899 for non-intubated patients).ConclusionsThe Chinese version of the BPS (BPS-C) is appropriate for pain assessment among intubated and non-intubated ICU patients.     

Reproducibility and observer variability of transrectal ultrasound measurements of prostatic volume

Transrectal ultrasound (TRUS) is an established investigation in benign and malignant prostatic disease though the level of reproducibility of TRUS volume measurements is not known. A group of observers in the UK and the USA who were participating in linked prospective studies of benign prostatic hyperplasia each carried out measurements during real-time scanning and on a series of static TRUS images. Results demonstrated good reproducibility for measurements of antero-posterior, transverse, and longitudinal dimensions of the prostate by an experienced UK urologist; only a minor degree of interobserver variation occurred in measurements made between experienced UK and USA observers. Using static images, USA observers obtained results that were in good agreement, whereas the UK observers, only two of whom used TRUS regularly, were in poor agreement with each other. This study demonstrates a high level of reproducibility for TRUS volume measurements performed by experienced observers.     

Reproducibility of walking at self-preferred and maximal speed in patients with postpoliomyelitis syndrome

OBJECTIVE: To assess the reproducibility of walking performance, heart rate, and perceived exertion at self-preferred speed and maximal walking speed in patients with the postpoliomyelitis syndrome (PPS). DESIGN: Repeated measurement at a 3-week interval. SETTING: University hospital. PARTICIPANTS: Convenience sample of 65 patients with PPS. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Walking performance: the distance walked in 2 minutes at a self-preferred speed and the time needed to walk 75 m at maximal speed, heart rate, and rating of perceived exertion (RPE) on an 11-point scale. RESULTS: Test-retest reliability of walking performance was excellent for both tests (intraclass correlation coefficient [ICC] range, .94-.97). No systematic differences existed between test and retest. The smallest detectable change for an individual was 15% for both tests. Test-retest reliability for heart rate was good (ICC=.86) but moderate for RPE (Spearman rho range, .67-.70). The smallest detectable change for RPE was between 4 and 6 scale points. The variability in walking performance was significantly correlated with the variability in heart rate at self-preferred speed (r =.36, P <.01) but not with the variability in RPE (r =.20, P =.11). CONCLUSIONS: Both walking tests showed good reproducibility and may be appropriate to monitor (individual) changes in walking capacity in patients with PPS. Because of its moderate reproducibility, RPE does not seem to be suitable to monitor physical exertion. The usefulness of an objective measure such as heart rate for this purpose needs further investigation.     

超声检测颈动脉内-中膜厚度的重复性评价

目的 评价"十一五"国家科技支撑计划<冠心病早期诊断和综合治疗技术体系的研究>之动脉粥样硬化性疾病临床前病变队列研究中,超声检测颈动脉内-中膜厚度(intima-media thickness,IMT)的重复性,论证其作为主要研究指标的可靠性.方法 抽取10例受检者,分别使用不同超声仪各检查次,评价仪器间IMT检测重复性;2个中心各抽取2名超声医生,分别独立对共15例受检者进行重复检查,评价检查者自身及检查者间重复性;抽取20份图像,同一阅读者重复阅读2次,参考阅读者阅读1次,评价阅读者自身及阅读者间重复性.所有图像均行中心阅读测量.结果 平均IMT的测量均差(meandifference,MD)在仪器间、检查者自身间、不同检查者间、测量者自身间、不同测量者间分别为(0.028±0.017)mm、(0.048±0.052)mm、(0.035±0.022)mm、(0.030±0.032)mm、(0.081±0.043)mm;最大IMT的上述测值依次为(0.047±0.029)mm、(0.090±0.09)mm、(0.055±0.044)mm、(0.062±0.053)mm、(0.085±0.050)mm.结论 平均IMT测量重复性优于最大IMT,且测量点越多,重复性越佳.与国外资料相比,本课题颈动脉超声测量IMT重复性良好.     

Reproducibility of loading measurements with skin-mounted accelerometers during walking

OBJECTIVE: To examine reproducibility of load measurements with skin-mounted accelerometers (SMAs) during walking. DESIGN: Reliability study. SETTING: A motion analysis laboratory. PARTICIPANTS: Ten healthy young men. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Two triaxial accelerometers were fixed to the subjects' skin above and below the knee joint. The subjects walked barefoot at their preferred speed and at a constant speed (1.3m/s, +/-5%) in a gait laboratory and along a corridor. The same protocol was repeated over 2 days. Initial peak acceleration (IPA), peak-to-peak (PP) acceleration, and maximal and average acceleration transient rates (ATRs) were calculated. The coefficient of variation (CV) and Pearson linear correlation coefficient were calculated to measure reproducibility of SMA load measurements. RESULTS: IPA and PP acceleration had good interday repeatability (CV <15%). The repeatability of average ATR and maximal ATR parameters was generally not acceptable. The loading variables obtained from ground reaction forces and SMA measurements during gait revealed high linear correlations, indicating that with SMA measurements it is possible to predict certain ground reaction force loading parameters. CONCLUSIONS: SMAs are practical for use in clinical environments to collect acceleration data that may be used to estimate joint loads.     

Reliability assessment and approaches to determining agreement between measurements: Classic methods paper

This classic methods paper (Bland and Altman, 2010) considers the assessment of agreement between measures, an often overlooked aspect of assessing measurements taken for use in research and practice and (re) introduces the ubiquitous ‘Bland Altman’ procedures for assessing agreement. The importance of these procedures is high and they address issues that are not always considered in research which uses measurement scales or describes the characteristics of scales developed for use in clinical practice. Many widely used approaches for reliability assessment can fail to consider the agreement between measures at all and can give an entirely misleading impression of an instrument's suitability for use in research or practice.     

超声剪切波弹性成像评价成人正常甲状腺的重复性

目的 评价剪切波弹性成像（SWE）技术检测正常甲状腺组织弹性参数的重复性。方法 选取健康成年志愿者28名,应用SWE技术,由同一检查者于不同时间及不同检查者分别行甲状腺弹性成像,应用Q-Box获取甲状腺双侧叶中部的弹性参数（Emean）,评价组内及组间的重复性。结果 ①检查者内同一天两测值Emean差异无统计学意义（P>0.05）,甲状腺左、右叶的组内相关系数（ICC）值分别为 0.81、0.94,Pearson相关系数分别为r=0.77、0.88（P均<0.001）,提示组内重复性很好;②检查者内不同天两测值Emean 差异无统计学意义（P>0.05）,组内ICC值分别为0.59、0.86,Pearson相关系数分别r=0.69、0.72（P均<0.001）,组内重复性好;③检查者间两测值Emean差异无统计学意义（P>0.05）,左、右叶ICC值分别为 0.64、0.58,Pearson相关系数分别为r_左=0.45（P_左=0.014）、r_右=0.57（P_右=0.002）,组间重复性好。Bland-Altman图示组内及组间SWE测量值具有良好一致性。结论 SWE可定量获取甲状腺的杨氏模量值,重复性好。     

Reproducibility of maximal quadriceps strength and its relationship to maximal voluntary activation in postpoliomyelitis syndrome

Horemans HL, Beelen A, Nollet F, Jones DA, Lankhorst GJ. Reproducibility of maximal quadriceps strength and its relationship to maximal voluntary activation in postpoliomyelitis syndrome. Arch Phys Med Rehabil 2004;85:1273-8.

Objectives

To determine what changes in maximal isometric strength can be detected in a symptomatic quadriceps muscle in patients with postpoliomyelitis syndrome (PPS) and to investigate the association between the variability in maximal strength and maximal voluntary activation (MVA).

Design

Repeated-measures over a 3-week interval.

Setting

University hospital.

Patients

Convenience sample of 65 patients with PPS.

Intervention

Dynamometer testing.

Main outcome measures

Maximal voluntary contraction (MVC) torque of the quadriceps was measured with a Kin-Com dynamometer and MVA was determined by twitch interpolation.

Results

The mean difference between the 2 consecutive measurements was −0.7±12.8Nm (95% confidence interval [CI], −3.9 to 2.5). The test-retest reliability was excellent for MVC torque (intraclass correlation coefficient [ICC]=.96; 95% CI, .93-.98) and moderate for MVA (ICC=.73; 95% CI, .56-.85). The smallest detectable change in MVC torque was 25% for an individual. The variability in MVA explained 18% of the variability in maximal strength.

Conclusions

Variability in maximal quadriceps strength, measured with a fixed dynamometer, was large and partly related to variability in MVA. This implies that even with optimally standardized strength testing, a follow-up of many years is required to objectify progression of quadriceps weakness in an individual patient with PPS. To demonstrate changes in strength in groups of patients in follow-up or intervention studies, feasible sample sizes are required.     

The performance of SAPS II in a cohort of patients admitted to 99 Italian ICUs: Results from GiViTI

Objective To assess the validity of SAPS II (new Simplified Acute Physiology Score) in a cohort of patients admitted to a large sample of Italian intensive care units (ICU).Design and setting The ability of the SAPS II scoring system to predict the probability of hospital mortality was assessed with calibration and discrimination measures obtained using published coefficients. A new logistic regression equation was then developed and further formal calibration and discrimination measures were estimated for the customized model.Patients From the 2202 consecutive patients recruited during a 1-month period in 99 ICUs, a total of 1393 patients were included in this validation study.Results When the parameters based on the standard model were applied, the expected probability of mortality did not fit those actually observed in the cohort (p<0.001), although it showed satisfactory discrimination (area under the receiver operating characteristic curve=0.80). Such lack of fit yields an overall under prediction of mortality (observed/expected ratio=1.14) that reflects a uniform pattern across a preselected set of subgroups. Customization allowed new mortality estimates to be calculated, with satisfactory calibration (p=0.82) and a more uniform pattern across subgroups.Conclusions SAPS II maintained its validity in an independent sample of patients recruited in a large network of Italian ICUs only after appropriate adaptation (first-level customization). Whether the determinants of this relatively poor performance are related to differences in unmeasured case-mix, methods of application, or quality of care delivered is a matter for discussion that cannot be solved with the data presently available. However, these findings suggest that caution is warranted before implementing the standard SAPS II scoring system parameters outside formal research projects.GiViTI: Gruppo italiano per la Valutazione degli interventi in Terapia Intensiva (A complete list of study participants appears in Appendix 2)     

Reproducibility and validity of the Dutch translation of the de Morton Mobility Index (DEMMI) used by physiotherapists in older patients with knee or hip osteoarthritis

Jans MP, Slootweg VC, Boot CR, de Morton NA, van der Sluis G, van Meeteren NL. Reproducibility and validity of the Dutch translation of the de Morton Mobility Index (DEMMI) used by physiotherapists in older patients with knee or hip osteoarthritis.

Objective

To examine the reproducibility, construct validity, and unidimensionality of the Dutch translation of the de Morton Mobility Index (DEMMI), a performance-based measure of mobility for older patients.

Design

Cross-sectional study.

Setting

Rehabilitation center (reproducibility study) and hospital (validity study).

Participants

Patients (N=28; age >65y) after orthopedic surgery (reproducibility study) and patients (N=219; age >65y) waiting for total hip or total knee arthroplasty (validity study).

Intervention

Not applicable.

Main Outcome Measures

Not applicable.

Results

The intraclass correlation coefficient for interrater reliability was high (.85; 95% confidence interval, 71–.93), and minimal detectable change with 90% confidence was 7 on the 100-point DEMMI scale. Rasch analysis identified that the Dutch translation of the DEMMI is a unidimensional measure of mobility in this population. DEMMI scores showed high correlations with scores on other performance-based measures of mobility (Timed Up and Go test, Spearman r=−.73; Chair Rise Time, r=−.69; walking test, r=.74). A lower correlation of .44 was identified with the self-report measure Western Ontario and McMaster Universities Osteoarthritis Index.

Conclusions

The Dutch translation of the DEMMI is a reproducible and valid performance-based measure for assessing mobility in older patients with knee or hip osteoarthritis.     

Interexaminer Reliability of Seated Motion Palpation for the Stiffest Spinal Site

Objectives

The purpose of this study was to assess the interexaminer reliability of palpation for stiffness in the cervical, thoracic, and lumbar spinal regions.

不同评分量表在糖尿病周围神经病变筛查中的比较研究

目的 比较密歇根神经病变筛查评分系统（MNSI）、密歇根糖尿病神经病变评分（MDNS）、多伦多临床评分系统（TCSS）在糖尿病周围神经病变（DPN）筛查中的价值,寻求简单、快捷、准确的筛查手段.方法 选取我院327例2型糖尿病患者,对其分别行MNSI、MDNS、TCSS和神经电生理检查,以神经电生理检查为金标准,分析各量表的灵敏度、特异度、准确度、阳性预测值、阴性预测值、约登指数、K值及ROC曲线下面积,评价其诊断价值.结果 以神经电生理检查为标准,各量表的敏感度、特异度、准确度分别为MDNS 90.50％ （162/179）、68.92％（102/148）、80.73％ （264/327）,MNSI 87.71％ （157/179）、78.38％ （116/148）、83.49％ （273/327）,TCSS 79.33％（142/179）、65.54％ （97/148）、73.09％ （239/327）.MNSI、MDNS与神经电生理检查之间呈高度一致（K值分别0.6 648、0.6 045）,MNSI、TCSS一致性较差,呈中度一致（K值为0.4 524）.MNSI、MDNS、TCSS诊断DPN的ROC曲线下面积分别为0.757、0.719、0.667,MNSI的真实性最佳.结论 MNSI可靠性强,且简单、快捷、可行性好,更适用于DPN的早期大规模筛查.     

The interrater reliability of a pain mechanisms-based classification for patients with nonspecific neck pain

ObjectiveTo examine the interrater reliability and agreement of a pain mechanisms-based classification for patients with nonspecific neck pain (NSNP).MethodsDesign – Observational, cross-sectional reliability study with a simultaneous examiner design. Setting: University hospital-based outpatient physical therapy clinic. Participants: A random sample of 48 patients, aged between 18 and 75 years old, with a primary complaint of neck pain was included. Interventions: Subjects underwent a standardized subjective and clinical examination, performed by 1 experienced physical therapist. Two assessors independently classified the participants’ NSNP on 3 main outcome measures. Main outcome measures: The Cohen kappa, percent agreement, and 95% confidence intervals (CIs) were calculated to determine the interrater reliability for (1) the predominant pain mechanism; (2) the predominant pain pattern; and (3) the predominant dysfunction pattern (DP).ResultsThere was almost perfect agreement between the 2 physical therapists’ judgements on the predominant pain mechanism, kappa = .84 (95% CI, .65–1.00), p < .001. There was substantial agreement between the raters’ judgements on the predominant pain pattern and predominant DP with respectively kappa = .61 (95% CI, .42–.80); and kappa = .62 (95% CI, .44–.79), p < .001.Conclusion(s)The proposed classification exhibits substantial to almost perfect interrater reliability. Further validity testing in larger neck pain populations is required before the information is used in clinical settings.Clinical trial registration numberNCT03147508 (https://clinicaltrials.gov/ct2/show/NCT03147508).