Assessing hospital electronic health record vendor performance across publicly reported quality measures

ObjectiveLittle is known regarding variation among electronic health record (EHR) vendors in quality performance. This issue is compounded by selection effects in which high-quality hospitals coalesce to a subset of market leading vendors. We measured hospital performance, stratified by EHR vendor, across 4 quality metrics.Materials and MethodsWe used data on 1272 hospitals in 2018 across 4 quality measures: Leapfrog Computerized Provider Order Entry/EHR Evaluation, Centers for Medicare and Medicaid Services Hospital Compare Star Ratings, Hospital-Acquired Condition (HAC) score, and Hospital Readmission Reduction Program (HRRP) ratio. We examined score distributions and used multivariable regression to evaluate the association between vendor and score, recovering partial R² to assess the proportion of quality variation explained by vendor.ResultsWe found significant variation across and within EHR vendors. The largest vendor, vendor A, had the highest mean score on the Leapfrog Computerized Provider Order Entry/EHR Evaluation and HRRP ratio, vendor G had the highest Hospital Compare score, and vendor F had the highest HAC score. In adjusted models, no vendor was significantly associated with higher performance on more than 2 measures. EHR vendor explained between 1.2% (HAC) and 7.6 (HRRP) of the variation in quality performance.DiscussionNo EHR vendor was associated with higher quality across all measures, and the 2 largest vendors were not associated with the highest scores. Only a small fraction of quality variation was explained by EHR vendor choice.ConclusionsTop performance on quality measures can be achieved with any EHR vendor; much of quality performance is driven by the hospital and how it uses the EHR.     

Sequencing of EHR adoption among US hospitals and the impact of meaningful use

Objective

To examine whether there is a common sequence of adoption of electronic health record (EHR) functions among US hospitals, identify differences by hospital type, and assess the impact of meaningful use.

Materials and methods

Using 2008 American Hospital Association (AHA) Information Technology (IT) Supplement data, we calculate adoption rates of individual EHR functions, along with Loevinger homogeneity (H) coefficients, to assess the sequence of EHR adoption across hospitals. We compare adoption rates and Loevinger H coefficients for hospitals of different types to assess variation in sequencing. We qualitatively assess whether stage 1 meaningful use functions are those adopted early in the sequence.

Results

There is a common sequence of EHR adoption across hospitals, with moderate-to-strong homogeneity. Patient demographic and ancillary results functions are consistently adopted first, while physician notes, clinical reminders, and guidelines are adopted last. Small hospitals exhibited greater homogeneity than larger hospitals. Rural hospitals and non-teaching hospitals exhibited greater homogeneity than urban and teaching hospitals. EHR functions emphasized in stage 1 meaningful use are spread throughout the scale.

Discussion

Stronger homogeneity among small, rural, and non-teaching hospitals may be driven by greater reliance on vendors and less variation in the types of care they deliver. Stage 1 meaningful use is likely changing how hospitals sequence EHR adoption—in particular, by moving clinical guidelines and medication computerized provider order entry ahead in sequence.

Conclusions

While there is a common sequence underlying adoption of EHR functions, the degree of adherence to the sequence varies by key hospital characteristics. Stage 1 meaningful use likely alters the sequence.     

Measuring time clinicians spend using EHRs in the inpatient setting: a national,mixed-methods study

ObjectiveTo understand hospitals’ use of EHR audit-log-based measures to address burden associated with inpatient EHR use.Materials and MethodsUsing mixed methods, we analyzed 2018 American Hospital Association Information Technology Supplement Survey data (n = 2864 hospitals; 64% response rate) to characterize measures used and provided by EHR vendors to track clinician time spent documenting. We interviewed staff from the top 3 EHR vendors that provided these measures. Multivariable analyses identified variation in use of the measures among hospitals with these 3 vendors.Results53% of hospitals reported using EHR data to track clinician time documenting, compared to 68% of the hospitals using the EHR from the top 3 vendors. Among hospitals with EHRs from these vendors, usage was significantly lower among rural hospitals and independent hospitals (P < .05). Two of these vendors provided measures of time spent doing specific tasks while the third measured an aggregate of auditable activities. Vendors varied in the underlying data used to create measures, measure specification, and data displays.DiscussionTools to track clinicians’ documentation time are becoming more available. The measures provided differ across vendors and disparities in use exist across hospitals. Increasing the specificity of standards underlying the data would support a common set of core measures making these measures more widely available.ConclusionAlthough half of US hospitals use measures of time spent in the EHR derived from EHR generated data, work remains to make such measures and analyses more broadly available to all hospitals and to increase its utility for national burden measurement.     

Impact of the 2015 Health Information Technology Certification Edition on Interoperability among Hospitals

ObjectiveMost nonfederal acute care hospitals use electronic health records (EHRs) certified by the Office of the National Coordinator for Health Information Technology. In 2015, the Office of the National Coordinator for Health Information Technology finalized the 2015 Health IT Certification Edition and adoption by hospitals began in 2016. We examine the impact of the 2015 Edition on rates of interoperable exchange among nonfederal acute hospitals.Materials and MethodsThe study applies a standard difference-in-differences design and a recently developed fixed effects estimator that relaxes the assumption of treatment effects being constant across groups and time. In the analysis, we identify separate effects of the 2015 Edition for hospitals that switched EHR developers and forecast hospitals’ interoperability over 2015 Edition adoption rates.ResultsThe adoption of the 2015 Edition increased hospitals’ rates of interoperable exchange and especially benefited hospitals that switched EHR developers in the post-implementation period. Forecasting results indicate that if all hospitals adopted the 2015 Edition, 53% to 61% of hospitals would engage in interoperable health information exchange compared with the current rate of 46%.DiscussionHospitals’ levels of interoperability have been rising over the last few years. Adoption of newer technology improved hospitals’ interoperability and accounts for up to 12% of the rise in interoperability.ConclusionsCertified technology is one mechanism to ensure providers use recent and safe technologies for interoperable exchange. Adoption of certified EHRs improves the nation’s interoperable exchange; however, it has a clear limited effect. Other mechanisms are necessary for achieving comprehensive interoperable exchange.     

HIT Implementation in Critical Access Hospitals: Extent of Implementation and Business Strategies Supporting IT Use

Small rural hospitals face considerable financial and personnel resource shortages which hinder their efforts to implement complex health information technology (HIT) systems. A survey on the use of HIT was completed by 85% of Iowa’s 82 Critical Access Hospitals (CAH). Analyses indicate that low IT staffing in CAHs is a barrier to implementing HIT solutions. CAHs with fewer staff tend to employ alternative business strategies. There is a clear relationship between having IT staff at a CAH and the types of technologies used. Many CAHs report having difficulty expanding upon HIT functionalities due to the challenges of finding IT staff with healthcare expertise. Most CAHs are in the transition point of planning for or beginning implementation of complex clinical information systems. Strategies for addressing these challenges will need to evolve as the HIT investments by rural hospitals race to keep pace with the goals for the nation.     

The association between perceived electronic health record usability and professional burnout among US nurses

ObjectivesTo measure nurse-perceived electronic health records (EHR) usability with a standardized metric of technology usability and evaluate its association with professional burnout.MethodsA cross-sectional survey of a random sample of US nurses was conducted in November 2017. EHR usability was measured with the System Usability Scale (SUS; range 0–100) and burnout with the Maslach Burnout Inventory.ResultsAmong the 86 858 nurses who were invited, 8638 (9.9%) completed the survey. The mean nurse-rated EHR SUS score was 57.6 (SD 16.3). A score of 57.6 is in the bottom 24% of scores across previous studies and categorized with a grade of “F.” On multivariable analysis adjusting for age, gender, race, ethnicity, relationship status, children, highest nursing-related degree, mean hours worked per week, years of nursing experience, advanced certification, and practice setting, nurse-rated EHR usability was associated with burnout with each 1 point more favorable SUS score and associated with a 2% lower odds of burnout (OR 0.98; 95% CI, 0.97–0.99; P < .001).ConclusionsNurses rated the usability of their current EHR in the low marginal range of acceptability using a standardized metric of technology usability. EHR usability and the odds of burnout were strongly associated with a dose-response relationship.     

Predicting the Adoption of Electronic Health Records by Physicians: When Will Health Care be Paperless?

ObjectivesThe purpose of this study was threefold. First, we gathered and synthesized the historic literature regarding electronic health record (EHR) adoption rates among physicians in small practices (ten or fewer members). Next, we constructed models to project estimated future EHR adoption trends and timelines. We then determined the likelihood of achieving universal EHR adoption in the near future and articulate how barriers can be overcome in the small and solo practice medical environment.DesignThis study used EHR adoption data from six previous surveys of small practices to estimate historic market penetration rates. Applying technology diffusion theory, three future adoption scenarios, optimistic, best estimate, and conservative, are empirically derived.MeasurementEHR adoption parameters, external and internal coefficients of influence, are estimated using Bass diffusion models.ResultsAll three EHR scenarios display the characteristic diffusion S curve that is indicative that the technology is likely to achieve significant market penetration, given enough time. Under current conditions, EHR adoption will reach its maximum market share in 2024 in the small practice setting.ConclusionThe promise of improved care quality and cost control has prompted a call for universal EHR adoption by 2014. The EHR products now available are unlikely to achieve full diffusion in a critical market segment within the time frame being targeted by policy makers.     

Area-level factors associated with electronic health record adoption and meaningful use in the Regional Extension Center Program

Objective

To identify area-level correlates of electronic health record (EHR) adoption and meaningful use (MU) among primary care providers (PCPs) enrolled in the Regional Extension Center (REC) Program.

Materials and methods

County-level data on 2013 EHR adoption and MU among REC-enrolled PCPs were obtained from the Office of the National Coordinator for Health Information Technology and linked with other county-level data sources including the Area Resource File, American Community Survey, and Federal Communications Commission''s broadband availability database. Hierarchical models with random intercepts for RECs were employed to assess associations between a broad set of area-level factors and county-level rates of EHR adoption and MU.

Results

Among the 2715 counties examined, the average county-level EHR adoption and MU rates for REC-enrolled PCPs were 87.5% and 54.2%, respectively. Community health center presence and Medicaid enrollment concentration were positively associated with EHR adoption, while metropolitan status and Medicare Advantage enrollment concentration were positively associated with MU. Health professional shortage area status and minority concentration were negatively associated with EHR adoption and MU.

Discussion

Increased financial incentives in areas with greater concentrations of Medicaid and Medicare enrollees may be encouraging EHR adoption and MU among REC-enrolled PCPs. Disparities in EHR adoption and MU in some low-resource and underserved areas remain a concern.

Conclusions

Federal efforts to spur EHR adoption and MU have demonstrated some early success; however, some geographic variations in EHR diffusion indicate that greater attention needs to be paid to ensuring equitable uptake and use of EHRs throughout the US.     

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

ObjectiveThe aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies.Materials and MethodsWe searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures.ResultsOne hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles.DiscussionVendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported.ConclusionThe number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.     

Correlates of Electronic Health Record Adoption in Office Practices: A Statewide Survey

Objective

Despite emerging evidence that electronic health records (EHRs) can improve the efficiency and quality of medical care, most physicians in office practice in the United States do not currently use an EHR. We sought to measure the correlates of EHR adoption.

Design

Mailed survey to a stratified random sample of all medical practices in Massachusetts in 2005, with one physician per practice randomly selected for survey.

Measurements

EHR adoption rates.

Results

The response rate was 71% (1345/1884). Overall, while 45% of physicians were using an EHR, EHRs were present in only 23% of practices. In multivariate analysis, practice size was strongly correlated with EHR adoption; 52% of practices with 7 or more physicians had an EHR, as compared with 14% of solo practices (adjusted odds ratio, 3.66; 95% confidence interval, 2.28–5.87). Hospital-based practices (adjusted odds ratio, 2.44; 95% confidence interval, 1.53–3.91) and practices that teach medical students or residents (adjusted odds ratio, 2.30; 95% confidence interval, 1.60–3.31) were more likely to have an EHR. The most frequently cited barriers to adoption were start-up financial costs (84%), ongoing financial costs (82%), and loss of productivity (81%).

Conclusions

While almost half of physicians in Massachusetts are using an EHR, fewer than one in four practices in Massachusetts have adopted EHRs. Adoption rates are lower in smaller practices, those not affiliated with hospitals, and those that do not teach medical students or residents. Interventions to expand EHR use must address both financial and non-financial barriers, especially among smaller practices.     

Outcomes of Mayo Clinic reBoot camps for postimplementation training in the electronic health record

ObjectiveAfter a new electronic health record (EHR) was implemented at Mayo Clinic, a training program called reBoot Camp was created to enhance ongoing education in response to needs identified by physician leaders.Materials and MethodsA reBoot camp focused on EHR topics pertinent to ambulatory care was offered from April 2018 through June 2020. There were 37 2-day sessions and 43 1-day sessions, with 673 unique participants. To evaluate outcomes of the reBoot camp, we used survey data to study baseline, immediate, and long-term perceptions of program satisfaction and self-assessed skills with the EHR. The study was conducted among practitioners at a large ambulatory practice network based in several states. Data were collected from April 2018 through January 2021. We analyzed automatically collected metadata and scores that evaluated the amount of personalization and proficiency of use.ResultsConfidence in skills increased by 13.5 points for general EHR use and was significant in 5 subdomains of use (13–18 point improvement). This degree of user confidence was maintained at the 6-month reassessment. The outcomes of configuration and proficiency scores also improved significantly.DiscussionOngoing education regarding EHR tools is necessary to support continued use of technology. This study was novel because of the amount and breadth of data collected, diversity of user participation, and validation that improvements were maintained over time.ConclusionsParticipating in a reBoot camp significantly improved user confidence in each domain of the EHR and demonstrated use of best-practice tools. Users maintained gains at the 6-month evaluation phase.     

Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program

ObjectiveTo give providers a better understanding of how to use the electronic health record (EHR), improve efficiency, and reduce burnout.Materials and MethodsAll ambulatory providers were offered at least 1 one-on-one session with an “optimizer” focusing on filling gaps in EHR knowledge and lack of customization. Success was measured using pre- and post-surveys that consisted of validated tools and homegrown questions. Only participants who returned both surveys were included in our calculations.ResultsOut of 1155 eligible providers, 1010 participated in optimization sessions. Pre-survey return rate was 90% (1034/1155) and post-survey was 54% (541/1010). 451 participants completed both surveys. After completing their optimization sessions, respondents reported a 26% improvement in mean knowledge of EHR functionality (P < .01), a 19% increase in the mean efficiency in the EHR (P < .01), and a 17% decrease in mean after-hours EHR usage (P < .01). Of the 401 providers asked to rate their burnout, 32% reported feelings of burnout in the pre-survey compared to 23% in the post-survey (P < .01). Providers were also likely to recommend colleagues participate in the program, with a Net Promoter Score of 41.DiscussionIt is possible to improve provider efficiency and feelings of burnout with a personalized optimization program. We ascribe these improvements to the one-on-one nature of our program which provides both training as well as addressing the feeling of isolation many providers feel after implementation.ConclusionIt is possible to reduce burnout in ambulatory providers with personalized retraining designed to improve efficiency and knowledge of the EHR.     

应用电子健康记录达到数字化医院的更高阶段

从电子病历和电子健康记录的概念出发,运用电子病历采用度模型,分析了美国HIMSS数据库4000家医院达到数字化的程度。得出的结论是大部分美国医院还处在电子病历的初级阶段,要充分应用电子病历,达到数字化医院的更高阶段还有很长的路要走。     

Practice and market factors associated with provider volume of health information exchange

ObjectiveTo assess the practice- and market-level factors associated with the amount of provider health information exchange (HIE) use.Materials and MethodsProvider and practice-level data was drawn from the Meaningful Use Stage 2 Public Use Files from the Centers for Medicare and Medicaid Services, the Physician Compare National Downloadable File, and the Compendium of US Health Systems, among other sources. We analyzed the relationship between provider HIE use and practice and market factors using multivariable linear regression and compared primary care providers (PCPs) to non-PCPs. Provider volume of HIE use is measured as the percentage of referrals sent with electronic summaries of care (eSCR) reported by eligible providers attesting to the Meaningful Use electronic health record (EHR) incentive program in 2016.ResultsProviders used HIE in 49% of referrals; PCPs used HIE in fewer referrals (43%) than non-PCPs (57%). Provider use of products from EHR vendors was negatively related to HIE use, while use of Athenahealth and Greenway Health products were positively related to HIE use. Providers treating, on average, older patients and greater proportions of patients with diabetes used HIE for more referrals. Health system membership, market concentration, and state HIE consent policy were unrelated to provider HIE use.DiscussionHIE use during referrals is low among office-based providers with the capability for exchange, especially PCPs. Practice-level factors were more commonly associated with greater levels of HIE use than market-level factors.ConclusionThis furthers the understanding that market forces, like competition, may be related to HIE adoption decisions but are less important for use once adoption has occurred.     

Electronic health records in small physician practices: availability,use, and perceived benefits

Objective

To examine variation in the adoption of electronic health record (EHR) functionalities and their use patterns, barriers to adoption, and perceived benefits by physician practice size.

Design

Mailed survey of a nationally representative random sample of practicing physicians identified from the Physician Masterfile of the American Medical Association.

Measurements

We measured, stratified by practice size: (1) availability of EHR functionalities, (2) functionality use, (3) barriers to the adoption and use of EHR, and (4) impact of the EHR on the practice and quality of patient care.

Results

With a response rate of 62%, we found that <2% of physicians in solo or two-physician (small) practices reported a fully functional EHR and 5% reported a basic EHR compared with 13% of physicians from 11+ group (largest group) practices with a fully functional system and 26% with a basic system. Between groups, a 21–46% difference in specific functionalities available was reported. Among adopters there were moderate to large differences in the use of the EHR systems. Financial barriers were more likely to be reported by smaller practices, along with concerns about future obsolescence. These differences were sizable (13–16%) and statistically significant (p<0.001). All adopters reported similar benefits.

Limitations

Although we have adjusted for response bias, influences may still exist.

Conclusion

Our study found that physicians in small practices have lower levels of EHR adoption and that these providers were less likely to use these systems. Ensuring that unique barriers are addressed will be critical to the widespread meaningful use of EHR systems among small practices.     

Optimizing the electronic health record: An inpatient sprint addresses provider burnout and improves electronic health record satisfaction

ObjectiveWe sought reduce electronic health record (EHR) burden on inpatient clinicians with a 2-week EHR optimization sprint.Materials and MethodsA team led by physician informaticists worked with 19 advanced practice providers (APPs) in 1 specialty unit. Over 2 weeks, the team delivered 21 EHR changes, and provided 39 one-on-one training sessions to APPs, with an average of 2.8 hours per provider. We measured Net Promoter Score, thriving metrics, and time spent in the EHR based on user log data.ResultsOf the 19 APPs, 18 completed 2 or more sessions. The EHR Net Promoter Score increased from 6 to 60 postsprint (1.0; 95% confidence interval, 0.3-1.8; P = .01). The NPS for the Sprint itself was 93, a very high rating. The 3-axis emotional thriving, emotional recovery, and emotional exhaustion metrics did not show a significant change. By user log data, time spent in the EHR did not show a significant decrease; however, 40% of the APPs responded that they spent less time in the EHR.ConclusionsThis inpatient sprint improved satisfaction with the EHR.     

Reporting of demographic data and representativeness in machine learning models using electronic health records

ObjectiveThe development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility.Materials and MethodsWe searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019.ResultsAcross the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population.DiscussionThe demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.     

A novel application of SMART on FHIR architecture for interoperable and scalable integration of patient-reported outcome data with electronic health records

ObjectiveDespite a proliferation of applications (apps) to conveniently collect patient-reported outcomes (PROs) from patients, PRO data are yet to be seamlessly integrated with electronic health records (EHRs) in a way that improves interoperability and scalability. We applied the newly created PRO standards from the Office of the National Coordinator for Health Information Technology to facilitate the collection and integration of standardized PRO data. A novel multitiered architecture was created to enable seamless integration of PRO data via Substitutable Medical Apps and Reusable Technologies on Fast Healthcare Interoperability Resources apps and scaled to different EHR platforms in multiple ambulatory settings.Materials and MethodsWe used a standards-based approach to deploy 2 apps that source and surface PRO data in real-time for provider use within the EHR and which rely on PRO assessments from an external center to streamline app and EHR integration.ResultsThe apps were developed to enable patients to answer validated assessments (eg, a Patient-Reported Outcomes Measurement Information System including using a Computer Adaptive Test format). Both apps were developed to populate the EHR in real time using the Health Level Seven FHIR standard allowing providers to view patients’ data during the clinical encounter. The process of implementing this architecture with 2 different apps across 18 ambulatory care sites and 3 different EHR platforms is described.ConclusionOur approach and solution proved feasible, secure, and time- and resource-efficient. We offer actionable guidance for this technology to be scaled and adapted to promote adoption in diverse ambulatory care settings and across different EHRs.     

The Relationship between Electronic Health Record Use and Quality of Care over Time

Objective

Electronic health records (EHRs) have the potential to advance the quality of care, but studies have shown mixed results. The authors sought to examine the extent of EHR usage and how the quality of care delivered in ambulatory care practices varied according to duration of EHR availability.

Methods

The study linked two data sources: a statewide survey of physicians' adoption and use of EHR and claims data reflecting quality of care as indicated by physicians' performance on widely used quality measures. Using four years of measurement, we combined 18 quality measures into 6 clinical condition categories. While the survey of physicians was cross-sectional, respondents indicated the year in which they adopted EHR. In an analysis accounting for duration of EHR use, we examined the relationship between EHR adoption and quality of care.

Results

The percent of physicians reporting adoption of EHR and availability of EHR core functions more than doubled between 2000 and 2005. Among EHR users in 2005, the average duration of EHR use was 4.8 years. For all 6 clinical conditions, there was no difference in performance between EHR users and non-users. In addition, for these 6 clinical conditions, there was no consistent pattern between length of time using an EHR and physicians performance on quality measures in both bivariate and multivariate analyses.

Conclusions

In this cross-sectional study, we found no association between duration of using an EHR and performance with respect to quality of care, although power was limited. Intensifying the use of key EHR features, such as clinical decision support, may be needed to realize quality improvement from EHRs. Future studies should examine the relationship between the extent to which physicians use key EHR functions and their performance on quality measures over time.