Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

ObjectiveThe aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies.Materials and MethodsWe searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures.ResultsOne hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles.DiscussionVendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported.ConclusionThe number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.     

Optimizing the electronic health record: An inpatient sprint addresses provider burnout and improves electronic health record satisfaction

ObjectiveWe sought reduce electronic health record (EHR) burden on inpatient clinicians with a 2-week EHR optimization sprint.Materials and MethodsA team led by physician informaticists worked with 19 advanced practice providers (APPs) in 1 specialty unit. Over 2 weeks, the team delivered 21 EHR changes, and provided 39 one-on-one training sessions to APPs, with an average of 2.8 hours per provider. We measured Net Promoter Score, thriving metrics, and time spent in the EHR based on user log data.ResultsOf the 19 APPs, 18 completed 2 or more sessions. The EHR Net Promoter Score increased from 6 to 60 postsprint (1.0; 95% confidence interval, 0.3-1.8; P = .01). The NPS for the Sprint itself was 93, a very high rating. The 3-axis emotional thriving, emotional recovery, and emotional exhaustion metrics did not show a significant change. By user log data, time spent in the EHR did not show a significant decrease; however, 40% of the APPs responded that they spent less time in the EHR.ConclusionsThis inpatient sprint improved satisfaction with the EHR.     

Can decision support combat incompleteness and bias in routine primary care data?

ObjectiveRoutine primary care data may be used for the derivation of clinical prediction rules and risk scores. We sought to measure the impact of a decision support system (DSS) on data completeness and freedom from bias.Materials and MethodsWe used the clinical documentation of 34 UK general practitioners who took part in a previous study evaluating the DSS. They consulted with 12 standardized patients. In addition to suggesting diagnoses, the DSS facilitates data coding. We compared the documentation from consultations with the electronic health record (EHR) (baseline consultations) vs consultations with the EHR-integrated DSS (supported consultations). We measured the proportion of EHR data items related to the physician’s final diagnosis. We expected that in baseline consultations, physicians would document only or predominantly observations related to their diagnosis, while in supported consultations, they would also document other observations as a result of exploring more diagnoses and/or ease of coding.ResultsSupported documentation contained significantly more codes (incidence rate ratio [IRR] = 5.76 [4.31, 7.70] P < .001) and less free text (IRR = 0.32 [0.27, 0.40] P < .001) than baseline documentation. As expected, the proportion of diagnosis-related data was significantly lower (b = −0.08 [−0.11, −0.05] P < .001) in the supported consultations, and this was the case for both codes and free text.ConclusionsWe provide evidence that data entry in the EHR is incomplete and reflects physicians’ cognitive biases. This has serious implications for epidemiological research that uses routine data. A DSS that facilitates and motivates data entry during the consultation can improve routine documentation.     

Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program

ObjectiveTo give providers a better understanding of how to use the electronic health record (EHR), improve efficiency, and reduce burnout.Materials and MethodsAll ambulatory providers were offered at least 1 one-on-one session with an “optimizer” focusing on filling gaps in EHR knowledge and lack of customization. Success was measured using pre- and post-surveys that consisted of validated tools and homegrown questions. Only participants who returned both surveys were included in our calculations.ResultsOut of 1155 eligible providers, 1010 participated in optimization sessions. Pre-survey return rate was 90% (1034/1155) and post-survey was 54% (541/1010). 451 participants completed both surveys. After completing their optimization sessions, respondents reported a 26% improvement in mean knowledge of EHR functionality (P < .01), a 19% increase in the mean efficiency in the EHR (P < .01), and a 17% decrease in mean after-hours EHR usage (P < .01). Of the 401 providers asked to rate their burnout, 32% reported feelings of burnout in the pre-survey compared to 23% in the post-survey (P < .01). Providers were also likely to recommend colleagues participate in the program, with a Net Promoter Score of 41.DiscussionIt is possible to improve provider efficiency and feelings of burnout with a personalized optimization program. We ascribe these improvements to the one-on-one nature of our program which provides both training as well as addressing the feeling of isolation many providers feel after implementation.ConclusionIt is possible to reduce burnout in ambulatory providers with personalized retraining designed to improve efficiency and knowledge of the EHR.     

Measuring time clinicians spend using EHRs in the inpatient setting: a national,mixed-methods study

ObjectiveTo understand hospitals’ use of EHR audit-log-based measures to address burden associated with inpatient EHR use.Materials and MethodsUsing mixed methods, we analyzed 2018 American Hospital Association Information Technology Supplement Survey data (n = 2864 hospitals; 64% response rate) to characterize measures used and provided by EHR vendors to track clinician time spent documenting. We interviewed staff from the top 3 EHR vendors that provided these measures. Multivariable analyses identified variation in use of the measures among hospitals with these 3 vendors.Results53% of hospitals reported using EHR data to track clinician time documenting, compared to 68% of the hospitals using the EHR from the top 3 vendors. Among hospitals with EHRs from these vendors, usage was significantly lower among rural hospitals and independent hospitals (P < .05). Two of these vendors provided measures of time spent doing specific tasks while the third measured an aggregate of auditable activities. Vendors varied in the underlying data used to create measures, measure specification, and data displays.DiscussionTools to track clinicians’ documentation time are becoming more available. The measures provided differ across vendors and disparities in use exist across hospitals. Increasing the specificity of standards underlying the data would support a common set of core measures making these measures more widely available.ConclusionAlthough half of US hospitals use measures of time spent in the EHR derived from EHR generated data, work remains to make such measures and analyses more broadly available to all hospitals and to increase its utility for national burden measurement.     

The impact of time spent on the electronic health record after work and of clerical work on burnout among clinical faculty

ObjectiveTo identify specific thresholds of daily electronic health record (EHR) time after work and daily clerical time burden associated with burnout in clinical faculty.Materials and MethodsWe administered an institution-wide survey to faculty in all departments at Mount Sinai Health System from November 2018 to February 2019. The Maslach Burnout Inventory and Mayo Well-Being Index assessed burnout. Demographics, possible confounding variables, and time spent on EHR work/clerical burden were assessed.ResultsOf 4156 eligible faculty members, 1781(42.9%) participated in the survey. After adjustment for background factors, EHR frustration (odds ratio [OR]=1.64–1.66), spending >90 minutes on EHR-outside the workday by self-report (OR = 1.41–1.90) and >1 hour of self-reported clerical work/day (OR = 1.39) were associated with burnout. Reporting that one’s practice unloads clerical burden (OR = 0.50–0.66) and higher resilience scores (OR = 0.77–0.84) were negatively associated with burnout.Spending >90 minutes/day on EHR-outside work (OR = 0.66–0.67) and >60 minutes/day on clerical work (OR = 0.54–0.58) was associated with decreased likelihood of satisfactory work–life integration (WLI) and professional satisfaction (PS). Greater meaning in work was associated with an increasedlikelihoodof achieving WLI (OR = 2.51) and PS (OR = 21.67).ConclusionResults suggest there are thresholds of excessive time on the EHR-outside the workday (>90 minutes) and overall clerical tasks (>60 minutes), above which clinical faculty may be at increased risk for burnout, as well as reduced WLI and PS, independent of demographic characteristics and clinical work hours. These thresholds of EHR and clerical burden may inform interventions aimed at mitigating this burden to reduce physician burnout.     

Impact of a problem-oriented view on clinical data retrieval

ObjectiveThe electronic health record (EHR) data deluge makes data retrieval more difficult, escalating cognitive load and exacerbating clinician burnout. New auto-summarization techniques are needed. The study goal was to determine if problem-oriented view (POV) auto-summaries improve data retrieval workflows. We hypothesized that POV users would perform tasks faster, make fewer errors, be more satisfied with EHR use, and experience less cognitive load as compared with users of the standard view (SV).MethodsSimple data retrieval tasks were performed in an EHR simulation environment. A randomized block design was used. In the control group (SV), subjects retrieved lab results and medications by navigating to corresponding sections of the electronic record. In the intervention group (POV), subjects clicked on the name of the problem and immediately saw lab results and medications relevant to that problem.ResultsWith POV, mean completion time was faster (173 seconds for POV vs 205 seconds for SV; P < .0001), the error rate was lower (3.4% for POV vs 7.7% for SV; P = .0010), user satisfaction was greater (System Usability Scale score 58.5 for POV vs 41.3 for SV; P < .0001), and cognitive task load was less (NASA Task Load Index score 0.72 for POV vs 0.99 for SV; P < .0001).DiscussionThe study demonstrates that using a problem-based auto-summary has a positive impact on 4 aspects of EHR data retrieval, including cognitive load.ConclusionEHRs have brought on a data deluge, with increased cognitive load and physician burnout. To mitigate these increases, further development and implementation of auto-summarization functionality and the requisite knowledge base are needed.     

Evaluation of electronic health record-integrated digital health tools to engage hospitalized patients in discharge preparation

ObjectiveTo evaluate the effect of electronic health record (EHR)-integrated digital health tools comprised of a checklist and video on transitions-of-care outcomes for patients preparing for discharge.Materials and MethodsEnglish-speaking, general medicine patients (>18 years) hospitalized at least 24 hours at an academic medical center in Boston, MA were enrolled before and after implementation. A structured checklist and video were administered on a mobile device via a patient portal or web-based survey at least 24 hours prior to anticipated discharge. Checklist responses were available for clinicians to review in real time via an EHR-integrated safety dashboard. The primary outcome was patient activation at discharge assessed by patient activation (PAM)-13. Secondary outcomes included postdischarge patient activation, hospital operational metrics, healthcare resource utilization assessed by 30-day follow-up calls and administrative data and change in patient activation from discharge to 30 days postdischarge.ResultsOf 673 patients approached, 484 (71.9%) enrolled. The proportion of activated patients (PAM level 3 or 4) at discharge was nonsignificantly higher for the 234 postimplementation compared with the 245 preimplementation participants (59.8% vs 56.7%, adjusted OR 1.23 [0.38, 3.96], P = .73). Postimplementation participants reported 3.75 (3.02) concerns via the checklist. Mean length of stay was significantly higher for postimplementation compared with preimplementation participants (10.13 vs 6.21, P < .01). While there was no effect on postdischarge outcomes, there was a nonsignificant decrease in change in patient activation within participants from pre- to postimplementation (adjusted difference-in-difference of −16.1% (9.6), P = .09).ConclusionsEHR-integrated digital health tools to prepare patients for discharge did not significantly increase patient activation and was associated with a longer length of stay. While issues uncovered by the checklist may have encouraged patients to inquire about their discharge preparedness, other factors associated with patient activation and length of stay may explain our observations. We offer insights for using PAM-13 in context of real-world health-IT implementations.Trial RegistrationNIH US National Library of Medicine, {"type":"clinical-trial","attrs":{"text":"NCT03116074","term_id":"NCT03116074"}}NCT03116074, clinicaltrials.gov     

Physicians’ electronic inbox work patterns and factors associated with high inbox work duration

ObjectivesElectronic health record systems are increasingly used to send messages to physicians, but research on physicians’ inbox use patterns is limited. This study’s aims were to (1) quantify the time primary care physicians (PCPs) spend managing inboxes; (2) describe daily patterns of inbox use; (3) investigate which types of messages consume the most time; and (4) identify factors associated with inbox work duration.Materials and MethodsWe analyzed 1 month of electronic inbox data for 1275 PCPs in a large medical group and linked these data with physicians’ demographic data.ResultsPCPs spent an average of 52 minutes on inbox management on workdays, including 19 minutes (37%) outside work hours. Temporal patterns of electronic inbox use differed from other EHR functions such as charting. Patient-initiated messages (28%) and results (29%) accounted for the most inbox work time. PCPs with higher inbox work duration were more likely to be female (P < .001), have more patient encounters (P < .001), have older patients (P < .001), spend proportionally more time on patient messages (P < .001), and spend more time per message (P < .001). Compared with PCPs with the lowest duration of time on inbox work, PCPs with the highest duration had more message views per workday (200 vs 109; P < .001) and spent more time on the inbox outside work hours (30 minutes vs 9.7 minutes; P < .001).ConclusionsElectronic inbox work by PCPs requires roughly an hour per workday, much of which occurs outside scheduled work hours. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload.     

Electronic Health Records in Specialty Care: A Time-Motion Study

BackgroundElectronic health records (EHRs) have great potential to improve safety, quality, and efficiency in medicine. However, adoption has been slow, and a key concern has been that clinicians will require more time to complete their work using EHRs. Most previous studies addressing this issue have been done in primary care.ObjectiveTo assess the impact of using an EHR on specialists’ time.DesignProspective, before-after trial of the impact of an EHR on attending physician time in four specialty clinics at an integrated delivery system: cardiology, dermatology, endocrine, and pain.MeasurementsWe used a time-motion method to measure physician time spent in one of 85 designated activities.ResultsAttending physicians were monitored before and after the switch from paper records to a web-based ambulatory EHR. Across all specialties, 15 physicians were observed treating 157 patients while still using paper-based records, and 15 physicians were observed treating 146 patients after adoption. Following EHR implementation, the average adjusted total time spent per patient across all specialties increased slightly but not significantly (Δ = 0.94 min., p = 0.83) from 28.8 (SE = 3.6) to 29.8 (SE = 3.6) min.ConclusionThese data suggest that implementation of an EHR had little effect on overall visit time in specialty clinics.     

Human-centered development of an electronic health record-embedded,interactive information visualization in the emergency department using fast healthcare interoperability resources

ObjectiveDevelop and evaluate an interactive information visualization embedded within the electronic health record (EHR) by following human-centered design (HCD) processes and leveraging modern health information exchange standards.Materials and MethodsWe applied an HCD process to develop a Fast Healthcare Interoperability Resources (FHIR) application that displays a patient’s asthma history to clinicians in a pediatric emergency department. We performed a preimplementation comparative system evaluation to measure time on task, number of screens, information retrieval accuracy, cognitive load, user satisfaction, and perceived utility and usefulness. Application usage and system functionality were assessed using application logs and a postimplementation survey of end users.ResultsUsability testing of the Asthma Timeline Application demonstrated a statistically significant reduction in time on task (P < .001), number of screens (P < .001), and cognitive load (P < .001) for clinicians when compared to base EHR functionality. Postimplementation evaluation demonstrated reliable functionality and high user satisfaction.DiscussionFollowing HCD processes to develop an application in the context of clinical operations/quality improvement is feasible. Our work also highlights the potential benefits and challenges associated with using internationally recognized data exchange standards as currently implemented.ConclusionCompared to standard EHR functionality, our visualization increased clinician efficiency when reviewing the charts of pediatric asthma patients. Application development efforts in an operational context should leverage existing health information exchange standards, such as FHIR, and evidence-based mixed methods approaches.     

Clinical data sharing improves quality measurement and patient safety

ObjectiveAccurate and robust quality measurement is critical to the future of value-based care. Having incomplete information when calculating quality measures can cause inaccuracies in reported patient outcomes. This research examines how quality calculations vary when using data from an individual electronic health record (EHR) and longitudinal data from a health information exchange (HIE) operating as a multisource registry for quality measurement. Materials and MethodsData were sampled from 53 healthcare organizations in 2018. Organizations represented both ambulatory care practices and health systems participating in the state of Kansas HIE. Fourteen ambulatory quality measures for 5300 patients were calculated using the data from an individual EHR source and contrasted to calculations when HIE data were added to locally recorded data.ResultsA total of 79% of patients received care at more than 1 facility during the 2018 calendar year. A total of 12 994 applicable quality measure calculations were compared using data from the originating organization vs longitudinal data from the HIE. A total of 15% of all quality measure calculations changed (P < .001) when including HIE data sources, affecting 19% of patients. Changes in quality measure calculations were observed across measures and organizations.DiscussionThese results demonstrate that quality measures calculated using single-site EHR data may be limited by incomplete information. Effective data sharing significantly changes quality calculations, which affect healthcare payments, patient safety, and care quality.ConclusionsFederal, state, and commercial programs that use quality measurement as part of reimbursement could promote more accurate and representative quality measurement through methods that increase clinical data sharing.     

Effects of technology-supported exercise programs on the knee pain,physical function,and quality of life of individuals with knee osteoarthritis and/or chronic knee pain: A systematic review and meta-analysis of randomized controlled trials

ObjectiveThe study sought to examine the effects of technology-supported exercise programs on the knee pain, physical function, and quality of life of individuals with knee osteoarthritis and/or chronic knee pain by a systematic review and meta-analysis of randomized controlled trials.Materials and MethodsWe searched MEDLINE, EMBASE, CINAHL Plus, and the Cochrane Library from database inception to August 2020. A meta-analysis and subgroup analyses, stratified by technology type and program feature, were conducted.ResultsTwelve randomized controlled trials were reviewed, all of which implemented the programs for 4 weeks to 6 months. Telephone, Web, mobile app, computer, and virtual reality were used to deliver the programs. The meta-analysis showed that these programs were associated with significant improvements in knee pain (standardized mean difference [SMD] = −0.29; 95% confidence interval [CI], −0.48 to −0.10; P = .003) and quality of life (SMD = 0.25; 95% CI, 0.04 to 0.46; P = .02) but not with significant improvement in physical function (SMD = 0.22; 95% CI, 0 to 0.43; P = .053). Subgroup analyses showed that some technology types and program features were suggestive of potential benefits.ConclusionsUsing technology to deliver the exercise programs appears to offer benefits. The technology types and program features that were associated with health values have been identified, based on which suggestions are discussed for the further research and development of such programs.     

Inaccuracies in electronic health records smoking data and a potential approach to address resulting underestimation in determining lung cancer screening eligibility

ObjectiveThe US Preventive Services Task Force (USPSTF) requires the estimation of lifetime pack-years to determine lung cancer screening eligibility. Leading electronic health record (EHR) vendors calculate pack-years using only the most recently recorded smoking data. The objective was to characterize EHR smoking data issues and to propose an approach to addressing these issues using longitudinal smoking data.Materials and MethodsIn this cross-sectional study, we evaluated 16 874 current or former smokers who met USPSTF age criteria for screening (50–80 years old), had no prior lung cancer diagnosis, and were seen in 2020 at an academic health system using the Epic^® EHR. We described and quantified issues in the smoking data. We then estimated how many additional potentially eligible patients could be identified using longitudinal data. The approach was verified through manual review of records from 100 subjects.ResultsOver 80% of evaluated records had inaccuracies, including missing packs-per-day or years-smoked (42.7%), outdated data (25.1%), missing years-quit (17.4%), and a recent change in packs-per-day resulting in inaccurate lifetime pack-years estimation (16.9%). Addressing these issues by using longitudinal data enabled the identification of 49.4% more patients potentially eligible for lung cancer screening (P < .001).DiscussionMissing, outdated, and inaccurate smoking data in the EHR are important barriers to effective lung cancer screening. Data collection and analysis strategies that reflect changes in smoking habits over time could improve the identification of patients eligible for screening.ConclusionThe use of longitudinal EHR smoking data could improve lung cancer screening.     

A theory-based meta-regression of factors influencing clinical decision support adoption and implementation

ObjectiveThe purpose of the study was to explore the theoretical underpinnings of effective clinical decision support (CDS) factors using the comparative effectiveness results.Materials and MethodsWe leveraged search results from a previous systematic literature review and updated the search to screen articles published from January 2017 to January 2020. We included randomized controlled trials and cluster randomized controlled trials that compared a CDS intervention with and without specific factors. We used random effects meta-regression procedures to analyze clinician behavior for the aggregate effects. The theoretical model was the Unified Theory of Acceptance and Use of Technology (UTAUT) model with motivational control.ResultsThirty-four studies were included. The meta-regression models identified the importance of effort expectancy (estimated coefficient = −0.162; P = .0003); facilitating conditions (estimated coefficient = 0.094; P = .013); and performance expectancy with motivational control (estimated coefficient = 1.029; P = .022). Each of these factors created a significant impact on clinician behavior. The meta-regression model with the multivariate analysis explained a large amount of the heterogeneity across studies (R2 =88.32%).DiscussionThree positive factors were identified: low effort to use, low controllability, and providing more infrastructure and implementation strategies to support the CDS. The multivariate analysis suggests that passive CDS could be effective if users believe the CDS is useful and/or social expectations to use the CDS intervention exist.ConclusionsOverall, a modified UTAUT model that includes motivational control is an appropriate model to understand psychological factors associated with CDS effectiveness and to guide CDS design, implementation, and optimization.     

Public vs physician views of liability for artificial intelligence in health care

The growing use of artificial intelligence (AI) in health care has raised questions about who should be held liable for medical errors that result from care delivered jointly by physicians and algorithms. In this survey study comparing views of physicians and the U.S. public, we find that the public is significantly more likely to believe that physicians should be held responsible when an error occurs during care delivered with medical AI, though the majority of both physicians and the public hold this view (66.0% vs 57.3%; P = .020). Physicians are more likely than the public to believe that vendors (43.8% vs 32.9%; P = .004) and healthcare organizations should be liable for AI-related medical errors (29.2% vs 22.6%; P = .05). Views of medical liability did not differ by clinical specialty. Among the general public, younger people are more likely to hold nearly all parties liable.     

Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review

Background ObjectiveElectronic health records (EHRs) are linked with documentation burden resulting in clinician burnout. While clear classifications and validated measures of burnout exist, documentation burden remains ill-defined and inconsistently measured. We aim to conduct a scoping review focused on identifying approaches to documentation burden measurement and their characteristics.Materials and MethodsBased on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Extension for Scoping Reviews (ScR) guidelines, we conducted a scoping review assessing MEDLINE, Embase, Web of Science, and CINAHL from inception to April 2020 for studies investigating documentation burden among physicians and nurses in ambulatory or inpatient settings. Two reviewers evaluated each potentially relevant study for inclusion/exclusion criteria.ResultsOf the 3482 articles retrieved, 35 studies met inclusion criteria. We identified 15 measurement characteristics, including 7 effort constructs: EHR usage and workload, clinical documentation/review, EHR work after hours and remotely, administrative tasks, cognitively cumbersome work, fragmentation of workflow, and patient interaction. We uncovered 4 time constructs: average time, proportion of time, timeliness of completion, activity rate, and 11 units of analysis. Only 45.0% of studies assessed the impact of EHRs on clinicians and/or patients and 40.0% mentioned clinician burnout.DiscussionStandard and validated measures of documentation burden are lacking. While time and effort were the core concepts measured, there appears to be no consensus on the best approach nor degree of rigor to study documentation burden.ConclusionFurther research is needed to reliably operationalize the concept of documentation burden, explore best practices for measurement, and standardize its use.