Turning Off Frequently Overridden Drug Alerts: Limited Opportunities for Doing It Safely

Objectives

This study sought to identify opportunities to safely turn off frequently overridden drug–drug interaction alerts (DDIs) in computerized physician order entry (CPOE).

Design

Quantitative retrospective analysis of drug safety alerts overridden during 1 month and qualitative interviews with 24 respondents (18 physicians and 6 pharmacists) about turning off frequently overridden DDI alerts, based on the Dutch drug database, in a hospital setting. Screen shots and complete texts of frequently overridden DDIs were presented to physicians of internal medicine, cardiology, and surgery and to hospital pharmacists who were asked whether these could be turned off hospital-wide without impairing patient safety, and the reasons for their recommendations.

Results

Data on the frequency of alerts overridden in 1 month identified 3,089 overrides, of which 1,963 were DDIs. The category DDIs showed 86 different alerts, of which 24 frequently overridden alerts, accounting for 72% of all DDI overrides, were selected for further evaluation. The 24 respondents together made 576 assessments. Upon investigation, differences in the reasons for turning off alerts were found across medical specialties and among respondents within a specialty. Frequently mentioned reasons for turning off were “alert well known,” “alert not serious,” or “alert not needing (additional) action,” or that the effects of the combination were monitored or intended. For none of the alerts did all respondents agree that it could be safely turned off hospital-wide. The highest agreement was 13 of 24 respondents (54%). A positive correlation was found between the number of alerts overridden and the number of clinicians recommending to turn them off.

Conclusion

Although the Dutch drug database is already a selected reduction from all DDIs mentioned in literature, the majority of respondents wanted to turn off DDI alerts to reduce alert overload. Turning off DDI alerts hospital-wide appeared to be problematic because of differences among physicians regarding drug-related knowledge and of differences across the hospital in routine drug monitoring practices. Furthermore, several reasons for suppression of alerts could be questioned from a safety perspective. Further research should investigate when each of the following might help: changes in alert texts; new differential alert triggers based on clinician knowledge or specialty; and nonintrusive alert presentation so long as serum levels and patient parameters are measured and stay within limits.     

Comparison of a basic and an advanced pharmacotherapy-related clinical decision support system in a hospital care setting in the Netherlands

Objective

To compare the clinical relevance of medication alerts in a basic and in an advanced clinical decision support system (CDSS).

Design

A prospective observational study.

Materials and methods

We collected 4023 medication orders in a hospital for independent evaluation in two pharmacotherapy-related decision support systems. Only the more advanced system considered patient characteristics and laboratory test results in its algorithms. Two pharmacists assessed the clinical relevance of the medication alerts produced. The alert was considered relevant if the pharmacist would undertake action (eg, contact the physician or the nurse). The primary analysis concerned the positive predictive value (PPV) for clinically relevant medication alerts in both systems.

Results

The PPV was significantly higher in the advanced system (5.8% vs 17.0%; p<0.05). Significant differences were found in the alert categories: drug–(drug) interaction (9.9% vs 14.8%; p<0.05), drug–age interaction (2.9% vs 73.3%; p<0.05), and dosing guidance (5.6% vs 16.9%; p<0.05). Including laboratory values and other patient characteristics resulted in a significantly higher PPV for the advanced CDSS compared to the basic medication alerts (12.2% vs 23.3%; p<0.05).

Conclusion

The advanced CDSS produced a higher proportion of clinically relevant medication alerts, but the number of irrelevant alerts remained high. To improve the PPV of the advanced CDSS, the algorithms should be optimized by identifying additional risk modifiers and more data should be made electronically available to improve the performance of the algorithms. Our study illustrates and corroborates the need for cyclic testing of technical improvements in information technology in circumstances representative of daily clinical practice.     

Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID�Cwarfarin co-prescribing as a test case

Background

Studies that have looked at the effectiveness of computerized decision support systems to prevent drug–drug interactions have reported modest results because of low response by the providers to the automated alerts.

Objective

To evaluate, within an inpatient computerized physician order entry (CPOE) system, the incremental effectiveness of an alert that required a response from the provider, intended as a stronger intervention to prevent concurrent orders of warfarin and non-steroidal anti-inflammatory drugs (NSAIDs).

Design

Randomized clinical trial of 1963 clinicians assigned to either an intervention group receiving a customized electronic alert requiring affirmative response or a control group receiving a commercially available passive alert as part of the CPOE. The study duration was 2 August 2006 to 15 December 2007.

Measurements

Alert adherence was compared between study groups.

Results

The proportion of desired ordering responses (ie, not reordering the alert-triggering drug after firing) was lower in the intervention group (114/464 (25%) customized alerts issued) than in the control group (154/560 (28%) passive alerts firing). The adjusted OR of inappropriate ordering was 1.22 (95% CI 0.69 to 2.16).

Conclusion

A customized CPOE alert that required a provider response had no effect in reducing concomitant prescribing of NSAIDs and warfarin beyond that of the commercially available passive alert received by the control group. New CPOE alerts cannot be assumed to be effective in improving prescribing, and need evaluation.     

Developing High-specificity Anti-hypertensive Alerts by Therapeutic State Analysis of Electronic Prescribing Records

Objective

This paper presents a model for analysis of chronic disease prescribing action over time in terms of transitions in status of therapy as indicated in electronic prescribing records. The quality of alerts derived from these therapeutic state transitions is assessed in the context of antihypertensive prescribing.

Design

A set of alert criteria is developed based on analysis of state-transition in past antihypertensive prescribing of a rural Australian General Practice. Thirty active patients coded as hypertensive with alerts on six months of previously un-reviewed prescribing, and 30 hypertensive patients without alerts, are randomly sampled and independently reviewed by the practice’s two main general practice physicians (GPs), each GP reviewing 20 alert and 20 non-alert cases (providing 10 alert and 10 non-alert cases for agreement assessment).

Measurements

GPs provide blind assessment of quality of hypertension management and retrospective assessment of alert relevance.

Results

Alerts were found on 66 of 611 cases with coded hypertension with 37 alerts on the 30 sampled alert cases. GPs assessed alerting sensitivity as 74% (CI 52% - 89%) and specificity as 61% (CI 45% - 74%) for the sample, which is estimated as 26% sensitivity and 93% specificity for the antihypertensive population. Agreement between the GPs on assessment of alert relevance was fair (kappa = 0.37).

Conclusions

Data-driven development of alerts from electronic prescribing records using analysis of therapeutic state transition shows promise for derivation of high-specificity alerts to improve the quality of chronic disease management activities.     

Practitioners’ Views on Computerized Drug–Drug Interaction Alerts in the VA System

Objectives

To assess Veterans Affairs (VA) prescribers’ and pharmacists’ opinions about computer-generated drug–drug interaction (DDI) alerts and obtain suggestions for improving DDI alerts.

Design

A mail survey of 725 prescribers and 142 pharmacists from seven VA medical centers across the United States.

Measurements

A questionnaire asked respondents about their sources of drug and DDI information, satisfaction with the combined inpatient and outpatient computerized prescriber order entry (CPOE) system, attitude toward DDI alerts, and suggestions for improving DDI alerts.

Results

The overall response rate was 40% (prescribers: 36%; pharmacists: 59%). Both prescribers and pharmacists indicated that the CPOE system had a neutral to positive impact on their jobs. DDI alerts were not viewed as a waste of time and the majority (61%) of prescribers felt that DDI alerts had increased their potential to prescribe safely. However, only 30% of prescribers felt DDI alerts provided them with what they needed most of the time. Both prescribers and pharmacists agreed that DDI alerts should be accompanied by management alternatives (73% and 82%, respectively) and more detailed information (65% and 89%, respectively). When asked about suggestions for improving DDI alerts, prescribers most preferred including management options whereas pharmacists most preferred making it more difficult to override lethal interactions. Prescribers and pharmacists reported primarily relying on electronic references for general drug information (62% and 55%, respectively) and DDI information (51% and 79%, respectively).

Conclusion

Respondents reported neutral to positive views regarding the effect of CPOE on their jobs. Their opinions suggest DDI alerts are useful but still require additional work to increase their clinical utility.     

Communication Outcomes of Critical Imaging Results in a Computerized Notification System

Objective

Communication of abnormal test results in the outpatient setting is prone to error. Using information technology can improve communication and improve patient safety. We standardized processes and procedures in a computerized test result notification system and examined their effectiveness to reduce errors in communication of abnormal imaging results.

Design

We prospectively analyzed outcomes of computerized notification of abnormal test results (alerts) that providers did not explicitly acknowledge receiving in the electronic medical record of an ambulatory multispecialty clinic.

Measurements

In the study period, 190,799 outpatient visits occurred and 20,680 outpatient imaging tests were performed. We tracked 1,017 transmitted alerts electronically. Using a taxonomy of communication errors, we focused on alerts in which errors in acknowledgment and reception occurred. Unacknowledged alerts were identified through electronic tracking. Among these, we performed chart reviews to determine any evidence of documented response, such as ordering a follow-up test or consultation. If no response was documented, we contacted providers by telephone to determine their awareness of the test results and any follow-up action they had taken. These processes confirmed the presence or absence of alert reception.

Results

Providers failed to acknowledge receipt of over one-third (368 of 1,017) of transmitted alerts. In 45 of these cases (4% of abnormal results), the imaging study was completely lost to follow-up 4 weeks after the date of study. Overall, 0.2% of outpatient imaging was lost to follow-up. The rate of lost to follow-up imaging was 0.02% per outpatient visit.

Conclusion

Imaging results continue to be lost to follow-up in a computerized test result notification system that alerted physicians through the electronic medical record. Although comparison data from previous studies are limited, the rate of results lost to follow-up appears to be lower than that reported in systems that do not use information technology comparable to what we evaluated.     

Making electronic prescribing alerts more effective: scenario-based experimental study in junior doctors

Objective

Expert authorities recommend clinical decision support systems to reduce prescribing error rates, yet large numbers of insignificant on-screen alerts presented in modal dialog boxes persistently interrupt clinicians, limiting the effectiveness of these systems. This study compared the impact of modal and non-modal electronic (e-) prescribing alerts on prescribing error rates, to help inform the design of clinical decision support systems.

Design

A randomized study of 24 junior doctors each performing 30 simulated prescribing tasks in random order with a prototype e-prescribing system. Using a within-participant design, doctors were randomized to be shown one of three types of e-prescribing alert (modal, non-modal, no alert) during each prescribing task.

Measurements

The main outcome measure was prescribing error rate. Structured interviews were performed to elicit participants'' preferences for the prescribing alerts and their views on clinical decision support systems.

Results

Participants exposed to modal alerts were 11.6 times less likely to make a prescribing error than those not shown an alert (OR 11.56, 95% CI 6.00 to 22.26). Those shown a non-modal alert were 3.2 times less likely to make a prescribing error (OR 3.18, 95% CI 1.91 to 5.30) than those not shown an alert. The error rate with non-modal alerts was 3.6 times higher than with modal alerts (95% CI 1.88 to 7.04).

Conclusions

Both kinds of e-prescribing alerts significantly reduced prescribing error rates, but modal alerts were over three times more effective than non-modal alerts. This study provides new evidence about the relative effects of modal and non-modal alerts on prescribing outcomes.     

A Randomized Trial of Electronic Clinical Reminders to Improve Medication Laboratory Monitoring

Objective

Recommendations for routine laboratory monitoring to reduce the risk of adverse medication events are not consistently followed. We evaluated the impact of electronic reminders delivered to primary care physicians on rates of appropriate routine medication laboratory monitoring.

Design

We enrolled 303 primary care physicians caring for 1,922 patients across 20 ambulatory clinics that had at least one overdue routine laboratory test for a given medication between January and June 2004. Clinics were randomized so that physicians received either usual care or electronic reminders at the time of office visits focused on potassium, creatinine, liver function, thyroid function, and therapeutic drug levels.

Measurements

Primary outcomes were the receipt of recommended laboratory monitoring within 14 days following an outpatient clinic visit. The effect of the intervention was assessed for each reminder after adjusting for clustering within clinics, as well as patient and provider characteristics.

Results

Medication-laboratory monitoring non-compliance ranged from 1.6% (potassium monitoring with potassium-supplement use) to 6.3% (liver function monitoring with HMG CoA Reductase Inhibitor use). Rates of appropriate laboratory monitoring following an outpatient visit ranged from 14% (therapeutic drug levels) to 64% (potassium monitoring with potassium-sparing diuretic use). Reminders for appropriate laboratory monitoring had no impact on rates of receiving appropriate testing for creatinine, potassium, liver function, renal function, or therapeutic drug level monitoring.

Conclusion

We identified high rates of appropriate laboratory monitoring, and electronic reminders did not significantly improve these monitoring rates. Future studies should focus on settings with lower baseline adherence rates and alternate drug-laboratory combinations.     

Provider management strategies of abnormal test result alerts: a cognitive task analysis

Objective

Electronic medical records (EMRs) facilitate abnormal test result communication through “alert” notifications. The aim was to evaluate how primary care providers (PCPs) manage alerts related to critical diagnostic test results on their EMR screens, and compare alert-management strategies of providers with high versus low rates of timely follow-up of results.

Design

28 PCPs from a large, tertiary care Veterans Affairs Medical Center (VAMC) were purposively sampled according to their rates of timely follow-up of alerts, determined in a previous study. Using techniques from cognitive task analysis, participants were interviewed about how and when they manage alerts, focusing on four alert-management features to filter, sort and reduce unnecessary alerts on their EMR screens.

Results

Provider knowledge of alert-management features ranged between 4% and 75%. Almost half (46%) of providers did not use any of these features, and none used more than two. Providers with higher versus lower rates of timely follow-up used the four features similarly, except one (customizing alert notifications). Providers with low rates of timely follow-up tended to manually scan the alert list and process alerts heuristically using their clinical judgment. Additionally, 46% of providers used at least one workaround strategy to manage alerts.

Conclusion

Considerable heterogeneity exists in provider use of alert-management strategies; specific strategies may be associated with lower rates of timely follow-up. Standardization of alert-management strategies including improving provider knowledge of appropriate tools in the EMR to manage alerts could reduce the lack of timely follow-up of abnormal diagnostic test results.     

In-home monitoring of older adults with vision impairment: exploring patients’, caregivers’ and professionals’ views

Objective

To develop a conceptual framework for the design of an in-home monitoring system (IMS) based on the requirements of older adults with vision impairment (VI), informal caregivers and eye-care rehabilitation professionals.

Materials and Methods

Concept mapping, a mixed-methods statistical research tool, was used in the construction of the framework. Overall, 40 participants brainstormed or sorted and rated 83 statements concerning an IMS for older adults with VI. Multidimensional scaling and hierarchical cluster analysis were employed to construct the framework. A questionnaire yielded further insights into the views of a wider sample of older adults with VI (n=78) and caregivers (n=25) regarding IMS.

Results

Concept mapping revealed a nine-cluster model of IMS-related aspects including affordability, awareness of system capabilities, simplicity of installation, operation and maintenance, system integrity and reliability, fall detection and safe movement, user customization, user preferences regarding information delivery, and safety alerts for patients and caregivers. From the questionnaire, independence, safety and fall detection were the most commonly reported reasons for older adults and caregivers to accept an IMS. Concerns included cost, privacy, security of the information obtained through monitoring, system accuracy, and ease of use.

Discussion

Older adults with VI, caregivers and professionals are receptive to in-home monitoring, mainly for fall detection and safety monitoring, but have concerns that must be addressed when developing an IMS.

Conclusion

Our study provides a novel conceptual framework for the design of an IMS that will be maximally acceptable and beneficial to our ageing and vision-impaired population.     

Can Surveillance Systems Identify and Avert Adverse Drug Events? A Prospective Evaluation of a Commercial Application

Objective

Computerized monitors can effectively detect and potentially prevent adverse drug events (ADEs). Most monitors have been developed in large academic hospitals and are not readily usable in other settings. We assessed the ability of a commercial program to identify and prevent ADEs in a community hospital.

Design and Measurement

We prospectively evaluated the commercial application in a community-based hospital. We examined the frequency and types of alerts produced, how often they were associated with ADEs and potential ADEs, and the potential financial impact of monitoring for ADEs.

Results

Among 2,407 patients screened, the application generated 516 high priority alerts. We were able to review 266 alerts at the time they were generated and among these, 30 (11.3%) were considered substantially important to warrant contacting the physician caring for the patient. These 30 alerts were associated with 4 ADEs and 11 potential ADEs. In all 15 cases, the responsible physician was unaware of the event, leading to a change in clinical care in 14 cases. Overall, 23% of high priority alerts were associated with an ADE (95% confidence interval [CI] 12% to 34%) and another 15% were associated with a potential ADE (95% CI 6% to 24%). Active surveillance used approximately 1.5 hours of pharmacist time daily.

Conclusions

A commercially available, computer-based ADE detection tool was effective at identifying ADEs. When used as part of an active surveillance program, it can have an impact on preventing or ameliorating ADEs.     

Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial

Background

Accurate clinical problem lists are critical for patient care, clinical decision support, population reporting, quality improvement, and research. However, problem lists are often incomplete or out of date.

Objective

To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation.

Study Design and Methods

Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009–5/2010) and intervention (5/2010–11/2010) periods.

Results

17 043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions.

Conclusion

Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement.

Trial Registration

ClinicalTrials.gov: {"type":"clinical-trial","attrs":{"text":"NCT01105923","term_id":"NCT01105923"}}NCT01105923.     

Prevalence of HIV infection and the correlates among homeless in Tehran,Iran

Objective

To determine the prevalence of HIV infection among homeless men and women and the related risk behaviors in Tehran, Iran.

Methods

In 2007-2008, Tehran municipality stacked up 10 657 homeless men and women for assessment of HIV and began collaboration with Iranian Research Center for HIV/AIDS (IRCHA) departments to conduct HIV infection prevalence surveys in homeless populations. The results were analyzed for associations with demographic information, family support, status of drug abuse and relation with family and friends.

Results

Overall HIV prevalence was 1.7% (95% confidence interval 1.4-1.9). Factors independently associated with HIV infection included history of using drugs [AOR 8.15 (4.86-13.67)], older age [AOR 1.80 (1.08-2.99) for 40- 55 yr], occupation [AOR 1.64 (1.19-2.24) for unemployed], and no relation with family [AOR 1.82 (1.30-2.54)].

Conclusions

This study supports the idea that injection drug use is contributing to the increased spread of HIV among Iranian homeless. Harm reduction programs should be expanded, particularly among homeless injection drug users.     

Failure to utilize functions of an electronic prescribing system and the subsequent generation of ‘technically preventable’ computerized alerts

Objectives

To determine the frequency with which computerized alerts occur and the proportion triggered as a result of prescribers not utilizing e-prescribing system functions.

Methods

An audit of electronic inpatient medication charts at a teaching hospital in Sydney, Australia, was conducted to identify alerts fired, to categorize the system functions used by prescribers, and to assess if use of short-cut system functions could have prevented the alerts.

Results

Of the 2209 active orders reviewed, 600 (27.2%) triggered at least one alert. Therapeutic duplication alerts were the most frequent (n=572). One third of these (20.2% of all alerts) was ‘technically preventable’ and would not have fired if prescribers had used a short-cut system function to prescribe. Under-utilized system functions included the option to ‘MODIFY’ existing orders and use of the ‘AND’ function for concurrent orders. Pregnancy alerts, set for women aged between 12 and 55 years, were triggered for 43% of drugs ordered for this group.

Conclusion

Developers of decision support systems should test the extent to which technically preventable alerts may arise when prescribers fail to use system functions as designed. Designs which aim to improve the efficiency of the prescribing process but which do not align with the cognitive processes of users may fail to achieve this desired outcome and produce unexpected consequences such as triggering unnecessary alerts and user frustration. Ongoing user training to support effective use of e-prescribing system functions and modifications to the mechanisms underlying alert generation are needed to ensure that prescribers are presented with fewer but more meaningful alerts.     

Computerized Clinical Decision Support During Medication Ordering for Long-term Care Residents with Renal Insufficiency

Objective

To determine whether a computerized clinical decision support system providing patient-specific recommendations in real-time improves the quality of prescribing for long-term care residents with renal insufficiency.

Design

Randomized trial within the long-stay units of a large long-term care facility. Randomization was within blocks by unit type. Alerts related to medication prescribing for residents with renal insufficiency were displayed to prescribers in the intervention units and hidden but tracked in control units.

Measurement

The proportions of final drug orders that were appropriate were compared between intervention and control units within alert categories: (1) recommended medication doses; (2) recommended administration frequencies; (3) recommendations to avoid the drug; (4) warnings of missing information.

Results

The rates of alerts were nearly equal in the intervention and control units: 2.5 per 1,000 resident days in the intervention units and 2.4 in the control units. The proportions of dose alerts for which the final drug orders were appropriate were similar between the intervention and control units (relative risk 0.95, 95% confidence interval 0.83, 1.1) for the remaining alert categories significantly higher proportions of final drug orders were appropriate in the intervention units: relative risk 2.4 for maximum frequency (1.4, 4.4); 2.6 for drugs that should be avoided (1.4, 5.0); and 1.8 for alerts to acquire missing information (1.1, 3.4). Overall, final drug orders were appropriate significantly more often in the intervention units—relative risk 1.2 (1.0, 1.4).

Conclusions

Clinical decision support for physicians prescribing medications for long-term care residents with renal insufficiency can improve the quality of prescribing decisions.Trial Registration: http://clinicaltrials.gov Identifier: NCT00599209