Renal medication-related clinical decision support (CDS) alerts and overrides in the inpatient setting following implementation of a commercial electronic health record: implications for designing more effective alerts

ObjectiveTo assess the appropriateness of medication-related clinical decision support (CDS) alerts associated with renal insufficiency and the potential/actual harm from overriding the alerts.Materials and MethodsOverride rate frequency was recorded for all inpatients who had a renal CDS alert trigger between 05/2017 and 04/2018. Two random samples of 300 for each of 2 types of medication-related CDS alerts associated with renal insufficiency—“dose change” and “avoid medication”—were evaluated by 2 independent reviewers using predetermined criteria for appropriateness of alert trigger, appropriateness of override, and patient harm.ResultsWe identified 37 100 “dose change” and 5095 “avoid medication” alerts in the population evaluated, and 100% of each were overridden. Dose change triggers were classified as 12.5% appropriate and overrides of these alerts classified as 90.5% appropriate. Avoid medication triggers were classified as 29.6% appropriate and overrides 76.5% appropriate. We identified 5 adverse drug events, and, of these, 4 of the 5 were due to inappropriately overridden alerts.ConclusionAlerts were nearly always presented inappropriately and were all overridden during the 1-year period studied. Alert fatigue resulting from receiving too many poor-quality alerts may result in failure to recognize errors that could lead to patient harm. Although medication-related CDS alerts associated with renal insufficiency had previously been found to be the most clinically beneficial alerts in a legacy system, in this system they were ineffective. These findings underscore the need for improvements in alert design, implementation, and monitoring of alert performance to make alerts more patient-specific and clinically appropriate.     

Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development

Objective Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems.Methods We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.Results Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules.Conclusion Significant improvements in the EHR certification and implementation procedures are necessary.     

Computerized clinical decision support for medication prescribing and utilization in pediatrics

Background and objective

Accurate and informed prescribing is essential to ensure the safe and effective use of medications in pediatric patients. Computerized clinical decision support (CCDS) functionalities have been embedded into computerized physician order entry systems with the aim of ensuring accurate and informed medication prescribing. Owing to a lack of comprehensive analysis of the existing literature, this review was undertaken to analyze the effect of CCDS implementation on medication prescribing and use in pediatrics.

Materials and methods

A literature search was performed using keywords in PubMed to identify research studies with outcomes related to the implementation of medication-related CCDS functionalities.

Results and discussion

Various CCDS functionalities have been implemented in pediatric patients leading to different results. Medication dosing calculators have decreased calculation errors. Alert-based CCDS functionalities, such as duplicate therapy and medication allergy checking, may generate excessive alerts. Medication interaction CCDS has been minimally studied in pediatrics. Medication dosing support has decreased adverse drug events, but has also been associated with high override rates. Use of medication order sets have improved guideline adherence. Guideline-based treatment recommendations generated by CCDS functionalities have had variable influence on appropriate medication use, with few studies available demonstrating improved patient outcomes due to CCDS use.

Conclusion

Although certain medication-related CCDS functionalities have shown benefit in medication prescribing for pediatric patients, others have resulted in high override rates and inconsistent or unknown impact on patient care. Further studies analyzing the effect of individual CCDS functionalities on safe and effective prescribing and medication use are required.     

Effects of computer-aided clinical decision support systems in improving antibiotic prescribing by primary care providers: a systematic review

Objective To assess the effectiveness of computer-aided clinical decision support systems (CDSS) in improving antibiotic prescribing in primary care.Methods A literature search utilizing Medline (via PubMed) and Embase (via Embase) was conducted up to November 2013. Randomized controlled trials (RCTs) and cluster randomized trials (CRTs) that evaluated the effects of CDSS aiming at improving antibiotic prescribing practice in an ambulatory primary care setting were included for review. Two investigators independently extracted data about study design and quality, participant characteristics, interventions, and outcomes.Results Seven studies (4 CRTs, 3 RCTs) met our inclusion criteria. All studies were performed in the USA. Proportions of eligible patient visits that triggered CDSS use varied substantially between intervention arms of studies (range 2.8–62.8%). Five out of seven trials showed marginal to moderate statistically significant effects of CDSS in improving antibiotic prescribing behavior. CDSS that automatically provided decision support were more likely to improve prescribing practice in contrast to systems that had to be actively initiated by healthcare providers.Conclusions CDSS show promising effectiveness in improving antibiotic prescribing behavior in primary care. Magnitude of effects compared to no intervention, appeared to be similar to other moderately effective single interventions directed at primary care providers. Additional research is warranted to determine CDSS characteristics crucial to triggering high adoption by providers as a perquisite of clinically relevant improvement of antibiotic prescribing.     

OrderRex clinical user testing: a randomized trial of recommender system decision support on simulated cases

ObjectiveTo assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases.Materials and Methods43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses.ResultsClinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows.DiscussionUser testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments.ConclusionsClinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.     

Development of a tool within the electronic medical record to facilitate medication reconciliation after hospital discharge

Serious medication errors occur commonly in the period after hospital discharge. Medication reconciliation in the postdischarge ambulatory setting may be one way to reduce the frequency of these errors. The authors describe the design and implementation of a novel tool built into an ambulatory electronic medical record (EMR) to facilitate postdischarge medication reconciliation. The tool compares the preadmission medication list within the ambulatory EMR to the hospital discharge medication list, highlights all changes, and allows the EMR medication list to be easily updated. As might be expected for a novel tool intended for use in a minority of visits, use of the tool was low at first: 20% of applicable patient visits within 30 days of discharge. Clinician outreach, education, and a pop-up reminder succeeded in increasing use to 41% of applicable visits. Review of feedback identified several usability issues that will inform subsequent versions of the tool and provide generalizable lessons for how best to design medication reconciliation tools for this setting.     

Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial

Objective

To determine the effects of a personal health record (PHR)-linked medications module on medication accuracy and safety.

Design

From September 2005 to March 2007, we conducted an on-treatment sub-study within a cluster-randomized trial involving 11 primary care practices that used the same PHR. Intervention practices received access to a medications module prompting patients to review their documented medications and identify discrepancies, generating ‘eJournals’ that enabled rapid updating of medication lists during subsequent clinical visits.

Measurements

A sample of 267 patients who submitted medications eJournals was contacted by phone 3 weeks after an eligible visit and compared with a matched sample of 274 patients in control practices that received a different PHR-linked intervention. Two blinded physician adjudicators determined unexplained discrepancies between documented and patient-reported medication regimens. The primary outcome was proportion of medications per patient with unexplained discrepancies.

Results

Among 121 046 patients in eligible practices, 3979 participated in the main trial and 541 participated in the sub-study. The proportion of medications per patient with unexplained discrepancies was 42% in the intervention arm and 51% in the control arm (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01). The number of unexplained discrepancies per patient with potential for severe harm was 0.03 in the intervention arm and 0.08 in the control arm (adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04).

Conclusions

When used, concordance between documented and patient-reported medication regimens and reduction in potentially harmful medication discrepancies can be improved with a PHR medication review tool linked to the provider''s medical record.

Trial registration number

This study was registered at ClinicalTrials.gov ({"type":"clinical-trial","attrs":{"text":"NCT00251875","term_id":"NCT00251875"}}NCT00251875).     

Patient characteristics associated with venous thromboembolic events: a cohort study using pooled electronic health record data

Objective

To demonstrate the potential of de-identified clinical data from multiple healthcare systems using different electronic health records (EHR) to be efficiently used for very large retrospective cohort studies.

Materials and methods

Data of 959 030 patients, pooled from multiple different healthcare systems with distinct EHR, were obtained. Data were standardized and normalized using common ontologies, searchable through a HIPAA-compliant, patient de-identified web application (Explore; Explorys Inc). Patients were 26 years or older seen in multiple healthcare systems from 1999 to 2011 with data from EHR.

Results

Comparing obese, tall subjects with normal body mass index, short subjects, the venous thromboembolic events (VTE) OR was 1.83 (95% CI 1.76 to 1.91) for women and 1.21 (1.10 to 1.32) for men. Weight had more effect then height on VTE. Compared with Caucasian, Hispanic/Latino subjects had a much lower risk of VTE (female OR 0.47, 0.41 to 0.55; male OR 0.24, 0.20 to 0.28) and African-Americans a substantially higher risk (female OR 1.83, 1.76 to 1.91; male OR 1.58, 1.50 to 1.66). This 13-year retrospective study of almost one million patients was performed over approximately 125 h in 11 weeks, part time by the five authors.

Discussion

As research informatics tools develop and more clinical data become available in EHR, it is important to study and understand unique opportunities for clinical research informatics to transform the scale and resources needed to perform certain types of clinical research.

Conclusions

With the right clinical research informatics tools and EHR data, some types of very large cohort studies can be completed with minimal resources.     

Evaluation of an Android-based mHealth system for population surveillance in developing countries

Objective

In parts of the developing world traditionally modeled healthcare systems do not adequately meet the needs of the populace. This can be due to imbalances in both supply and demand—there may be a lack of sufficient healthcare and the population most at need may be unable or unwilling to take advantage of it. Home-based care has emerged as a possible mechanism to bring healthcare to the populace in a cost-effective, useful manner. This study describes the development, implementation, and evaluation of a mobile device-based system to support such services.

Materials and Methods

Mobile phones were utilized and a structured survey was implemented to be administered by community health workers using Open Data Kit. This system was used to support screening efforts for a population of two million persons in western Kenya.

Results

Users of the system felt it was easy to use and facilitated their work. The system was also more cost effective than pen and paper alternatives.

Discussion

This implementation is one of the largest applications of a system utilizing handheld devices for performing clinical care during home visits in a resource-constrained environment. Because the data were immediately available electronically, initial reports could be performed and important trends in data could thus be detected. This allowed adjustments to the programme to be made sooner than might have otherwise been possible.

Conclusion

A viable, cost-effective solution at scale has been developed and implemented for collecting electronic data during household visits in a resource-constrained setting.     

Are physicians' perceptions of healthcare quality and practice satisfaction affected by errors associated with electronic health record use?

Background

Electronic health record (EHR) adoption is a national priority in the USA, and well-designed EHRs have the potential to improve quality and safety. However, physicians are reluctant to implement EHRs due to financial constraints, usability concerns, and apprehension about unintended consequences, including the introduction of medical errors related to EHR use. The goal of this study was to characterize and describe physicians'' attitudes towards three consequences of EHR implementation: (1) the potential for EHRs to introduce new errors; (2) improvements in healthcare quality; and (3) changes in overall physician satisfaction.

Methods

Using data from a 2007 statewide survey of Massachusetts physicians, we conducted multivariate regression analysis to examine relationships between practice characteristics, perceptions of EHR-related errors, perceptions of healthcare quality, and overall physician satisfaction.

Results

30% of physicians agreed that EHRs create new opportunities for error, but only 2% believed their EHR has created more errors than it prevented. With respect to perceptions of quality, there was no significant association between perceptions of EHR-associated errors and perceptions of EHR-associated changes in healthcare quality. Finally, physicians who believed that EHRs created new opportunities for error were less likely be satisfied with their practice situation (adjusted OR 0.49, p=0.001).

Conclusions

Almost one third of physicians perceived that EHRs create new opportunities for error. This perception was associated with lower levels of physician satisfaction.