Collection of annotated data in a clinical validation study for alarm algorithms in intensive care—a methodologic framework

Introduction

Monitoring of physiologic parameters in critically ill patients is currently performed by threshold alarm systems with high sensitivity but low specificity. As a consequence, a multitude of alarms are generated, leading to an impaired clinical value of these alarms due to reduced alertness of the intensive care unit (ICU) staff. To evaluate a new alarm procedure, we currently generate a database of physiologic data and clinical alarm annotations.

Methods

Data collection is taking place at a 12-bed medical ICU. Patients with monitoring of at least heart rate, invasive arterial blood pressure, and oxygen saturation are included in the study. Numerical physiologic data at 1-second intervals, monitor alarms, and alarm settings are extracted from the surveillance network. Bedside video recordings are performed with network surveillance cameras.

Results

Based on the extracted data and the video recordings, alarms are clinically annotated by an experienced physician. The alarms are categorized according to their technical validity and clinical relevance by a taxonomy system that can be broadly applicable. Preliminary results showed that only 17% of the alarms were classified as relevant, and 44% were technically false.

Discussion

The presented system for collecting real-time bedside monitoring data in conjunction with video-assisted annotations of clinically relevant events is the first allowing the assessment of 24-hour periods and reduces the bias usually created by bedside observers in comparable studies. It constitutes the basis for the development and evaluation of “smart” alarm algorithms, which may help to reduce the number of alarms at the ICU, thereby improving patient safety.