Risk factors associated with medication ordering errors

ObjectiveWe utilized a computerized order entry system–integrated function referred to as “void” to identify erroneous orders (ie, a “void” order). Using voided orders, we aimed to (1) identify the nature and characteristics of medication ordering errors, (2) investigate the risk factors associated with medication ordering errors, and (3) explore potential strategies to mitigate these risk factors.Materials and MethodsWe collected data on voided orders using clinician interviews and surveys within 24 hours of the voided order and using chart reviews. Interviews were informed by the human factors–based SEIPS (Systems Engineering Initiative for Patient Safety) model to characterize the work systems–based risk factors contributing to ordering errors; chart reviews were used to establish whether a voided order was a true medication ordering error and ascertain its impact on patient safety.ResultsDuring the 16-month study period (August 25, 2017, to December 31, 2018), 1074 medication orders were voided; 842 voided orders were true medication errors (positive predictive value = 78.3 ± 1.2%). A total of 22% (n = 190) of the medication ordering errors reached the patient, with at least a single administration, without causing patient harm. Interviews were conducted on 355 voided orders (33% response). Errors were not uniquely associated with a single risk factor, but the causal contributors of medication ordering errors were multifactorial, arising from a combination of technological-, cognitive-, environmental-, social-, and organizational-level factors.ConclusionsThe void function offers a practical, standardized method to create a rich database of medication ordering errors. We highlight implications for utilizing the void function for future research, practice and learning opportunities.     

Association between limiting the number of open records in a tele-critical care setting and retract–reorder errors

BackgroundWrong patient selection errors may be tracked by retract–reorder (RAR) events. The aim of this quality improvement study was to assess the impact of reducing the number of concurrently open electronic health records from 4 to 2 on RAR errors generated by a tele-critical care service.MethodsThe study encompassed 32 months before and 21 months after restriction. Chi-Square test of proportions and T statistical process control chart for rare events were used.ResultsThere were 156 318 orders with 57 RAR errors (36.5/100 000 orders) before restriction, and 122 587 orders with 34 errors (27.7/100 000 orders) after. Rates were not statistically different (P = .20), but analysis was underpowered. When plotted on a T control chart, random variation was detected between RAR errors.ConclusionWe found no significant difference in RAR errors in the tele-critical care setting after open record limitation. Other strategies should be studied to reduce wrong patient selection errors.     

Effects of technology-supported exercise programs on the knee pain,physical function,and quality of life of individuals with knee osteoarthritis and/or chronic knee pain: A systematic review and meta-analysis of randomized controlled trials

ObjectiveThe study sought to examine the effects of technology-supported exercise programs on the knee pain, physical function, and quality of life of individuals with knee osteoarthritis and/or chronic knee pain by a systematic review and meta-analysis of randomized controlled trials.Materials and MethodsWe searched MEDLINE, EMBASE, CINAHL Plus, and the Cochrane Library from database inception to August 2020. A meta-analysis and subgroup analyses, stratified by technology type and program feature, were conducted.ResultsTwelve randomized controlled trials were reviewed, all of which implemented the programs for 4 weeks to 6 months. Telephone, Web, mobile app, computer, and virtual reality were used to deliver the programs. The meta-analysis showed that these programs were associated with significant improvements in knee pain (standardized mean difference [SMD] = −0.29; 95% confidence interval [CI], −0.48 to −0.10; P = .003) and quality of life (SMD = 0.25; 95% CI, 0.04 to 0.46; P = .02) but not with significant improvement in physical function (SMD = 0.22; 95% CI, 0 to 0.43; P = .053). Subgroup analyses showed that some technology types and program features were suggestive of potential benefits.ConclusionsUsing technology to deliver the exercise programs appears to offer benefits. The technology types and program features that were associated with health values have been identified, based on which suggestions are discussed for the further research and development of such programs.     

Impact of a problem-oriented view on clinical data retrieval

ObjectiveThe electronic health record (EHR) data deluge makes data retrieval more difficult, escalating cognitive load and exacerbating clinician burnout. New auto-summarization techniques are needed. The study goal was to determine if problem-oriented view (POV) auto-summaries improve data retrieval workflows. We hypothesized that POV users would perform tasks faster, make fewer errors, be more satisfied with EHR use, and experience less cognitive load as compared with users of the standard view (SV).MethodsSimple data retrieval tasks were performed in an EHR simulation environment. A randomized block design was used. In the control group (SV), subjects retrieved lab results and medications by navigating to corresponding sections of the electronic record. In the intervention group (POV), subjects clicked on the name of the problem and immediately saw lab results and medications relevant to that problem.ResultsWith POV, mean completion time was faster (173 seconds for POV vs 205 seconds for SV; P < .0001), the error rate was lower (3.4% for POV vs 7.7% for SV; P = .0010), user satisfaction was greater (System Usability Scale score 58.5 for POV vs 41.3 for SV; P < .0001), and cognitive task load was less (NASA Task Load Index score 0.72 for POV vs 0.99 for SV; P < .0001).DiscussionThe study demonstrates that using a problem-based auto-summary has a positive impact on 4 aspects of EHR data retrieval, including cognitive load.ConclusionEHRs have brought on a data deluge, with increased cognitive load and physician burnout. To mitigate these increases, further development and implementation of auto-summarization functionality and the requisite knowledge base are needed.     

How effective are electronic medication systems in reducing medication error rates and associated harm among hospital inpatients? A systematic review and meta-analysis

ObjectiveTo conduct a systematic review and meta-analysis to assess: 1) changes in medication error rates and associated patient harm following electronic medication system (EMS) implementation; and 2) evidence of system-related medication errors facilitated by the use of an EMS.Materials and MethodsWe searched Medline, Scopus, Embase, and CINAHL for studies published between January 2005 and March 2019, comparing medication errors rates with or without assessments of related harm (actual or potential) before and after EMS implementation. EMS was defined as a computer-based system enabling the prescribing, supply, and/or administration of medicines. Study quality was assessed.ResultsThere was substantial heterogeneity in outcomes of the 18 included studies. Only 2 were strong quality. Meta-analysis of 5 studies reporting change in actual harm post-EMS showed no reduced risk (RR: 1.22, 95% CI: 0.18–8.38, P = .8) and meta-analysis of 3 studies reporting change in administration errors found a significant reduction in error rates (RR: 0.77, 95% CI: 0.72–0.83, P = .004). Of 10 studies of prescribing error rates, 9 reported a reduction but variable denominators precluded meta-analysis. Twelve studies provided specific examples of system-related medication errors; 5 quantified their occurrence.Discussion and ConclusionDespite the wide-scale adoption of EMS in hospitals around the world, the quality of evidence about their effectiveness in medication error and associated harm reduction is variable. Some confidence can be placed in the ability of systems to reduce prescribing error rates. However, much is still unknown about mechanisms which may be most effective in improving medication safety and design features which facilitate new error risks.     

The quality of social determinants data in the electronic health record: a systematic review

ObjectiveThe aim of this study was to collect and synthesize evidence regarding data quality problems encountered when working with variables related to social determinants of health (SDoH).Materials and MethodsWe conducted a systematic review of the literature on social determinants research and data quality and then iteratively identified themes in the literature using a content analysis process.ResultsThe most commonly represented quality issue associated with SDoH data is plausibility (n = 31, 41%). Factors related to race and ethnicity have the largest body of literature (n = 40, 53%). The first theme, noted in 62% (n = 47) of articles, is that bias or validity issues often result from data quality problems. The most frequently identified validity issue is misclassification bias (n = 23, 30%). The second theme is that many of the articles suggest methods for mitigating the issues resulting from poor social determinants data quality. We grouped these into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully.DiscussionThe type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error. Problems encountered with the quality of SDoH data are rarely distributed randomly. Data from Hispanic patients are more prone to issues with plausibility and misclassification than data from other racial/ethnic groups.ConclusionConsideration of data quality and evidence-based quality improvement methods may help prevent bias and improve the validity of research conducted with SDoH data.     

Demonstrating an approach for evaluating synthetic geospatial and temporal epidemiologic data utility: results from analyzing >1.8 million SARS-CoV-2 tests in the United States National COVID Cohort Collaborative (N3C)

ObjectiveThis study sought to evaluate whether synthetic data derived from a national coronavirus disease 2019 (COVID-19) dataset could be used for geospatial and temporal epidemic analyses.Materials and MethodsUsing an original dataset (n = 1 854 968 severe acute respiratory syndrome coronavirus 2 tests) and its synthetic derivative, we compared key indicators of COVID-19 community spread through analysis of aggregate and zip code-level epidemic curves, patient characteristics and outcomes, distribution of tests by zip code, and indicator counts stratified by month and zip code. Similarity between the data was statistically and qualitatively evaluated.ResultsIn general, synthetic data closely matched original data for epidemic curves, patient characteristics, and outcomes. Synthetic data suppressed labels of zip codes with few total tests (mean = 2.9 ± 2.4; max = 16 tests; 66% reduction of unique zip codes). Epidemic curves and monthly indicator counts were similar between synthetic and original data in a random sample of the most tested (top 1%; n = 171) and for all unsuppressed zip codes (n = 5819), respectively. In small sample sizes, synthetic data utility was notably decreased.DiscussionAnalyses on the population-level and of densely tested zip codes (which contained most of the data) were similar between original and synthetically derived datasets. Analyses of sparsely tested populations were less similar and had more data suppression.ConclusionIn general, synthetic data were successfully used to analyze geospatial and temporal trends. Analyses using small sample sizes or populations were limited, in part due to purposeful data label suppression—an attribute disclosure countermeasure. Users should consider data fitness for use in these cases.     

A theory-based meta-regression of factors influencing clinical decision support adoption and implementation

ObjectiveThe purpose of the study was to explore the theoretical underpinnings of effective clinical decision support (CDS) factors using the comparative effectiveness results.Materials and MethodsWe leveraged search results from a previous systematic literature review and updated the search to screen articles published from January 2017 to January 2020. We included randomized controlled trials and cluster randomized controlled trials that compared a CDS intervention with and without specific factors. We used random effects meta-regression procedures to analyze clinician behavior for the aggregate effects. The theoretical model was the Unified Theory of Acceptance and Use of Technology (UTAUT) model with motivational control.ResultsThirty-four studies were included. The meta-regression models identified the importance of effort expectancy (estimated coefficient = −0.162; P = .0003); facilitating conditions (estimated coefficient = 0.094; P = .013); and performance expectancy with motivational control (estimated coefficient = 1.029; P = .022). Each of these factors created a significant impact on clinician behavior. The meta-regression model with the multivariate analysis explained a large amount of the heterogeneity across studies (R2 =88.32%).DiscussionThree positive factors were identified: low effort to use, low controllability, and providing more infrastructure and implementation strategies to support the CDS. The multivariate analysis suggests that passive CDS could be effective if users believe the CDS is useful and/or social expectations to use the CDS intervention exist.ConclusionsOverall, a modified UTAUT model that includes motivational control is an appropriate model to understand psychological factors associated with CDS effectiveness and to guide CDS design, implementation, and optimization.     

Human-centered development of an electronic health record-embedded,interactive information visualization in the emergency department using fast healthcare interoperability resources

ObjectiveDevelop and evaluate an interactive information visualization embedded within the electronic health record (EHR) by following human-centered design (HCD) processes and leveraging modern health information exchange standards.Materials and MethodsWe applied an HCD process to develop a Fast Healthcare Interoperability Resources (FHIR) application that displays a patient’s asthma history to clinicians in a pediatric emergency department. We performed a preimplementation comparative system evaluation to measure time on task, number of screens, information retrieval accuracy, cognitive load, user satisfaction, and perceived utility and usefulness. Application usage and system functionality were assessed using application logs and a postimplementation survey of end users.ResultsUsability testing of the Asthma Timeline Application demonstrated a statistically significant reduction in time on task (P < .001), number of screens (P < .001), and cognitive load (P < .001) for clinicians when compared to base EHR functionality. Postimplementation evaluation demonstrated reliable functionality and high user satisfaction.DiscussionFollowing HCD processes to develop an application in the context of clinical operations/quality improvement is feasible. Our work also highlights the potential benefits and challenges associated with using internationally recognized data exchange standards as currently implemented.ConclusionCompared to standard EHR functionality, our visualization increased clinician efficiency when reviewing the charts of pediatric asthma patients. Application development efforts in an operational context should leverage existing health information exchange standards, such as FHIR, and evidence-based mixed methods approaches.     

Quantification of abdominal fat from computed tomography using deep learning and its association with electronic health records in an academic biobank

ObjectiveThe objective was to develop a fully automated algorithm for abdominal fat segmentation and to deploy this method at scale in an academic biobank.Materials and MethodsWe built a fully automated image curation and labeling technique using deep learning and distributive computing to identify subcutaneous and visceral abdominal fat compartments from 52,844 computed tomography scans in 13,502 patients in the Penn Medicine Biobank (PMBB). A classification network identified the inferior and superior borders of the abdomen, and a segmentation network differentiated visceral and subcutaneous fat. Following technical evaluation of our method, we conducted studies to validate known relationships with visceral and subcutaneous fat.ResultsWhen compared with 100 manually annotated cases, the classification network was on average within one 5-mm slice for both the superior (0.4 ± 1.1 slice) and inferior (0.4 ± 0.6 slice) borders. The segmentation network also demonstrated excellent performance with intraclass correlation coefficients of 1.00 (P < 2 × 10^-16) for subcutaneous and 1.00 (P < 2 × 10^-16) for visceral fat on 100 testing cases. We performed integrative analyses of abdominal fat with the phenome extracted from the electronic health record and found highly significant associations with diabetes mellitus, hypertension, and renal failure, among other phenotypes.ConclusionsThis work presents a fully automated and highly accurate method for the quantification of abdominal fat that can be applied to routine clinical imaging studies to fuel translational scientific discovery.     

Characterizing phenotypic abnormalities associated with high-risk individuals developing lung cancer using electronic health records from the All of Us researcher workbench

ObjectiveThe study sought to test the feasibility of conducting a phenome-wide association study to characterize phenotypic abnormalities associated with individuals at high risk for lung cancer using electronic health records.Materials and MethodsWe used the beta release of the All of Us Researcher Workbench with clinical and survey data from a population of 225 000 subjects. We identified 3 cohorts of individuals at high risk to develop lung cancer based on (1) the 2013 U.S. Preventive Services Task Force criteria, (2) the long-term quitters of cigarette smoking criteria, and (3) the younger age of onset criteria. We applied the logistic regression analysis to identify the significant associations between individuals’ phenotypes and their risk categories. We validated our findings against a lung cancer cohort from the same population and conducted an expert review to understand whether these associations are known or potentially novel.ResultsWe found a total of 214 statistically significant associations (P < .05 with a Bonferroni correction and odds ratio > 1.5) enriched in the high-risk individuals from 3 cohorts, and 15 enriched in the low-risk individuals. Forty significant associations enriched in the high-risk individuals and 13 enriched in the low-risk individuals were validated in the cancer cohort. Expert review identified 15 potentially new associations enriched in the high-risk individuals.ConclusionsIt is feasible to conduct a phenome-wide association study to characterize phenotypic abnormalities associated in high-risk individuals developing lung cancer using electronic health records. The All of Us Research Workbench is a promising resource for the research studies to evaluate and optimize lung cancer screening criteria.     

Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program

ObjectiveTo give providers a better understanding of how to use the electronic health record (EHR), improve efficiency, and reduce burnout.Materials and MethodsAll ambulatory providers were offered at least 1 one-on-one session with an “optimizer” focusing on filling gaps in EHR knowledge and lack of customization. Success was measured using pre- and post-surveys that consisted of validated tools and homegrown questions. Only participants who returned both surveys were included in our calculations.ResultsOut of 1155 eligible providers, 1010 participated in optimization sessions. Pre-survey return rate was 90% (1034/1155) and post-survey was 54% (541/1010). 451 participants completed both surveys. After completing their optimization sessions, respondents reported a 26% improvement in mean knowledge of EHR functionality (P < .01), a 19% increase in the mean efficiency in the EHR (P < .01), and a 17% decrease in mean after-hours EHR usage (P < .01). Of the 401 providers asked to rate their burnout, 32% reported feelings of burnout in the pre-survey compared to 23% in the post-survey (P < .01). Providers were also likely to recommend colleagues participate in the program, with a Net Promoter Score of 41.DiscussionIt is possible to improve provider efficiency and feelings of burnout with a personalized optimization program. We ascribe these improvements to the one-on-one nature of our program which provides both training as well as addressing the feeling of isolation many providers feel after implementation.ConclusionIt is possible to reduce burnout in ambulatory providers with personalized retraining designed to improve efficiency and knowledge of the EHR.     

Can decision support combat incompleteness and bias in routine primary care data?

ObjectiveRoutine primary care data may be used for the derivation of clinical prediction rules and risk scores. We sought to measure the impact of a decision support system (DSS) on data completeness and freedom from bias.Materials and MethodsWe used the clinical documentation of 34 UK general practitioners who took part in a previous study evaluating the DSS. They consulted with 12 standardized patients. In addition to suggesting diagnoses, the DSS facilitates data coding. We compared the documentation from consultations with the electronic health record (EHR) (baseline consultations) vs consultations with the EHR-integrated DSS (supported consultations). We measured the proportion of EHR data items related to the physician’s final diagnosis. We expected that in baseline consultations, physicians would document only or predominantly observations related to their diagnosis, while in supported consultations, they would also document other observations as a result of exploring more diagnoses and/or ease of coding.ResultsSupported documentation contained significantly more codes (incidence rate ratio [IRR] = 5.76 [4.31, 7.70] P < .001) and less free text (IRR = 0.32 [0.27, 0.40] P < .001) than baseline documentation. As expected, the proportion of diagnosis-related data was significantly lower (b = −0.08 [−0.11, −0.05] P < .001) in the supported consultations, and this was the case for both codes and free text.ConclusionsWe provide evidence that data entry in the EHR is incomplete and reflects physicians’ cognitive biases. This has serious implications for epidemiological research that uses routine data. A DSS that facilitates and motivates data entry during the consultation can improve routine documentation.     

Use of a Personal Digital Assistant for Managing Antibiotic Prescribing for Outpatient Respiratory Tract Infections in Rural Communities

ObjectiveTo assess the acceptability and usage of a standalone personal digital assistant (PDA)-based clinical decision-support system (CDSS) for the diagnosis and management of acute respiratory tract infections (RTIs) in the outpatient setting.DesignObservational study performed as part of a larger randomized trial in six rural communities in Utah and Idaho from January 2002 to March 2004. Ninety-nine primary care providers received a PDA-based CDSS for use at the point-of-care, and were asked to use the tool with at least 200 patients with suspected RTIs.MeasurementsClinical data were collected electronically from the devices at periodic intervals. Providers also completed an exit questionnaire at the end of the study period.ResultsProviders logged 14,393 cases using the CDSS, the majority of which (n=7624; 53%) were from family practitioners. Overall adherence with CDSS recommendations for the five most common diagnoses (pharyngitis, otitis media, sinusitis, bronchitis, and upper respiratory tract infection) was 82%. When antibiotics were prescribed (53% of cases), adherence with the CDSS-recommended antibiotic was high (76%). By logistic regression analysis, the odds of adherence with CDSS recommendations increased significantly with each ten cases completed (P=0.001). Questionnaire respondents believed the CDSS was easy to use, and most (44/65; 68%) did not believe it increased their encounter time with patients, regardless of prior experience with PDAs.ConclusionA standalone PDA-based CDSS for acute RTIs used at the point-of-care can encourage better outpatient antimicrobial prescribing practices and easily gather a rich set of clinical data.     

Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis

ObjectiveTo derive 7 proposed core electronic health record (EHR) use metrics across 2 healthcare systems with different EHR vendor product installations and examine factors associated with EHR time.Materials and MethodsA cross-sectional analysis of ambulatory physicians EHR use across the Yale-New Haven and MedStar Health systems was performed for August 2019 using 7 proposed core EHR use metrics normalized to 8 hours of patient scheduled time.ResultsFive out of 7 proposed metrics could be measured in a population of nonteaching, exclusively ambulatory physicians. Among 573 physicians (Yale-New Haven N = 290, MedStar N = 283) in the analysis, median EHR-Time₈ was 5.23 hours. Gender, additional clinical hours scheduled, and certain medical specialties were associated with EHR-Time₈ after adjusting for age and health system on multivariable analysis. For every 8 hours of scheduled patient time, the model predicted these differences in EHR time (P < .001, unless otherwise indicated): female physicians +0.58 hours; each additional clinical hour scheduled per month −0.01 hours; practicing cardiology −1.30 hours; medical subspecialties −0.89 hours (except gastroenterology, P = .002); neurology/psychiatry −2.60 hours; obstetrics/gynecology −1.88 hours; pediatrics −1.05 hours (P = .001); sports/physical medicine and rehabilitation −3.25 hours; and surgical specialties −3.65 hours.ConclusionsFor every 8 hours of scheduled patient time, ambulatory physicians spend more than 5 hours on the EHR. Physician gender, specialty, and number of clinical hours practicing are associated with differences in EHR time. While audit logs remain a powerful tool for understanding physician EHR use, additional transparency, granularity, and standardization of vendor-derived EHR use data definitions are still necessary to standardize EHR use measurement.     

Public vs physician views of liability for artificial intelligence in health care

The growing use of artificial intelligence (AI) in health care has raised questions about who should be held liable for medical errors that result from care delivered jointly by physicians and algorithms. In this survey study comparing views of physicians and the U.S. public, we find that the public is significantly more likely to believe that physicians should be held responsible when an error occurs during care delivered with medical AI, though the majority of both physicians and the public hold this view (66.0% vs 57.3%; P = .020). Physicians are more likely than the public to believe that vendors (43.8% vs 32.9%; P = .004) and healthcare organizations should be liable for AI-related medical errors (29.2% vs 22.6%; P = .05). Views of medical liability did not differ by clinical specialty. Among the general public, younger people are more likely to hold nearly all parties liable.