Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development

Objective Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems.Methods We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.Results Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules.Conclusion Significant improvements in the EHR certification and implementation procedures are necessary.     

Trends in user-initiated health information exchange in the inpatient,outpatient, and emergency settings

Prior research on health information exchange (HIE) typically measured provider usage through surveys or they summarized the availability of HIE services in a healthcare organization. Few studies utilized user log files. Using HIE access log files, we measured HIE use in real-world clinical settings over a 7-year period (2011-2017). Use of HIE increased in inpatient, outpatient, and emergency department (ED) settings. Further, while extant literature has generally viewed the ED as the most relevant setting for HIE, the greatest change in HIE use was observed in the inpatient setting, followed by the ED setting and then the outpatient setting. Our findings suggest that in addition to federal incentives, the implementation of features that address barriers to access (eg, Single Sign On), as well as value-added services (eg, interoperability with external data sources), may be related to the growth in user-initiated HIE.     

A research agenda to support the development and implementation of genomics-based clinical informatics tools and resources

ObjectiveThe Genomic Medicine Working Group of the National Advisory Council for Human Genome Research virtually hosted its 13th genomic medicine meeting titled “Developing a Clinical Genomic Informatics Research Agenda”. The meeting’s goal was to articulate a research strategy to develop Genomics-based Clinical Informatics Tools and Resources (GCIT) to improve the detection, treatment, and reporting of genetic disorders in clinical settings.Materials and MethodsExperts from government agencies, the private sector, and academia in genomic medicine and clinical informatics were invited to address the meeting''s goals. Invitees were also asked to complete a survey to assess important considerations needed to develop a genomic-based clinical informatics research strategy.ResultsOutcomes from the meeting included identifying short-term research needs, such as designing and implementing standards-based interfaces between laboratory information systems and electronic health records, as well as long-term projects, such as identifying and addressing barriers related to the establishment and implementation of genomic data exchange systems that, in turn, the research community could help address.DiscussionDiscussions centered on identifying gaps and barriers that impede the use of GCIT in genomic medicine. Emergent themes from the meeting included developing an implementation science framework, defining a value proposition for all stakeholders, fostering engagement with patients and partners to develop applications under patient control, promoting the use of relevant clinical workflows in research, and lowering related barriers to regulatory processes. Another key theme was recognizing pervasive biases in data and information systems, algorithms, access, value, and knowledge repositories and identifying ways to resolve them.     

CSER and eMERGE: current and potential state of the display of genetic information in the electronic health record

Objective Clinicians’ ability to use and interpret genetic information depends upon how those data are displayed in electronic health records (EHRs). There is a critical need to develop systems to effectively display genetic information in EHRs and augment clinical decision support (CDS).Materials and Methods The National Institutes of Health (NIH)-sponsored Clinical Sequencing Exploratory Research and Electronic Medical Records & Genomics EHR Working Groups conducted a multiphase, iterative process involving working group discussions and 2 surveys in order to determine how genetic and genomic information are currently displayed in EHRs, envision optimal uses for different types of genetic or genomic information, and prioritize areas for EHR improvement.Results There is substantial heterogeneity in how genetic information enters and is documented in EHR systems. Most institutions indicated that genetic information was displayed in multiple locations in their EHRs. Among surveyed institutions, genetic information enters the EHR through multiple laboratory sources and through clinician notes. For laboratory-based data, the source laboratory was the main determinant of the location of genetic information in the EHR. The highest priority recommendation was to address the need to implement CDS mechanisms and content for decision support for medically actionable genetic information.Conclusion Heterogeneity of genetic information flow and importance of source laboratory, rather than clinical content, as a determinant of information representation are major barriers to using genetic information optimally in patient care. Greater effort to develop interoperable systems to receive and consistently display genetic and/or genomic information and alert clinicians to genomic-dependent improvements to clinical care is recommended.     

The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis

Objectives

To evaluate the impact of electronic health record (EHR) implementation on nursing care processes and outcomes.

Design

Interrupted time series analysis, 2003–2009.

Setting

A large US not-for-profit integrated health care organization.

Participants

29 hospitals in Northern and Southern California.

Intervention

An integrated EHR including computerized physician order entry, nursing documentation, risk assessment tools, and documentation tools.

Main outcome measures

Percentage of patients with completed risk assessments for hospital acquired pressure ulcers (HAPUs) and falls (process measures) and rates of HAPU and falls (outcome measures).

Results

EHR implementation was significantly associated with an increase in documentation rates for HAPU risk (coefficient 2.21, 95% CI 0.67 to 3.75); the increase for fall risk was not statistically significant (0.36; −3.58 to 4.30). EHR implementation was associated with a 13% decrease in HAPU rates (coefficient −0.76, 95% CI −1.37 to −0.16) but no decrease in fall rates (−0.091; −0.29 to 0.11). Irrespective of EHR implementation, HAPU rates decreased significantly over time (−0.16; −0.20 to −0.13), while fall rates did not (0.0052; −0.01 to 0.02). Hospital region was a significant predictor of variation for both HAPU (0.72; 0.30 to 1.14) and fall rates (0.57; 0.41 to 0.72).

Conclusions

The introduction of an integrated EHR was associated with a reduction in the number of HAPUs but not in patient fall rates. Other factors, such as changes over time and hospital region, were also associated with variation in outcomes. The findings suggest that EHR impact on nursing care processes and outcomes is dependent on a number of factors that should be further explored.     

Information blocking remains prevalent at the start of 21st Century Cures Act: results from a survey of health information exchange organizations

ObjectiveRecent policy making aims to prevent health systems, lectronic health record (EHR) vendors, and others from blocking the electronic sharing of patient data necessary for clinical care. We sought to assess the prevalence of information blocking prior to enforcement of these rules.Materials and MethodsWe conducted a national survey of health information exchange organizations (HIEs) to measure the prevalence of information blocking behaviors observed by these third-party entities. Eighty-nine of 106 HIEs (84%) meeting the inclusion criteria responded.ResultsThe majority (55%) of HIEs reported that EHR vendors at least sometimes engage in information blocking, while 30% of HIEs reported the same for health systems. The most common type of information blocking behavior EHR vendors engaged in was setting unreasonably high prices, which 42% of HIEs reported routinely observing. The most common type of information blocking behavior health systems engaged in was refusing to share information, which 14% of HIEs reported routinely observing. Reported levels of vendor information blocking was correlated with regional competition among vendors and information blocking was concentrated in some geographic regions.DiscussionOur findings are consistent with early reports, revealing persistently high levels of information blocking and important variation by actor, type of behavior, and geography. These trends reflect the observations and experiences of HIEs and their potential biases. Nevertheless, these data serve as a baseline against which to measure the impact of new regulations and to inform policy makers about the most common types of information blocking behaviors.ConclusionEnforcement aimed at reducing information blocking should consider variation in prevalence and how to most effectively target efforts.     

Reporting of demographic data and representativeness in machine learning models using electronic health records

ObjectiveThe development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility.Materials and MethodsWe searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019.ResultsAcross the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population.DiscussionThe demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.     

医院卫生资源结构优化配置研究

目的:依据医院运行规律与特点,探讨如何合理定位医院服务规模,适时调整医院资源结构,从而实现医院卫生资源的最优化配置.方法:依据"理论模型-数学模型-决策支持系统"的技术路线,采用管理科学逻辑模型法(logic model),界定医院卫生资源结构优化配置的目标、投入、产出和结果及其相互关系,构建医院卫生资源结构优化配置理论模型;采用运筹学、统计学方法,构建相关数学模型;运用基于关系数据库的"三库"结构研制医院卫生资源结构优化配置决策支持系统.结果:(1)建立了用于指导医院卫生资源结构优化配置,明确界定各环节任务与参与者的理论模型;(2)建立了包括服务量预测系列模型和卫生资源结构优化模型在内的数学模型;(3)构建了以上述模型为核心的医院卫生资源结构优化配置决策支持系统.结论:所构建的医院卫生资源优化配置决策支持系统生成的配置方案建议合理,有助于辅助医院决策者及时灵活的优化配置卫生资源,以实现医院的可持续发展.     

Factors associated with difficult electronic health record implementation in office practice

Little is known about physicians' perception of the ease or difficulty of implementing electronic health records (EHR). This study identified factors related to the perceived difficulty of implementing EHR. 163 physicians completed surveys before and after the implementation of EHR in an externally funded pilot program in three Massachusetts communities. Ordinal hierarchical logistic regression was used to identify baseline factors that correlated with physicians' report of difficulty with EHR implementation. Compared with physicians with ownership stake in their practices, physician employees were less likely to describe EHR implementation as difficult (adjusted OR 0.5, 95% CI 0.3 to 1.0). Physicians who perceived their staff to be innovative were also less likely to view EHR implementation as difficult (adjusted OR 0.4, 95% CI 0.2 to 0.8). Physicians who own their practice may need more external support for EHR implementation than those who do not. Innovative clinical support staff may ease the EHR implementation process and contribute to its success.     

Optimizing gender-affirming medical care through anatomical inventories,clinical decision support,and population health management in electronic health record systems

Recent advances in electronic health records and health information technology are providing new opportunities to improve the quality of care for transgender and gender diverse people, a population that experiences significant health disparities. This article recommends changes to electronic health record systems that have the potential to optimize gender-affirming care. Specifically, we discuss the importance of creating an anatomical inventory form that captures organ diversity, and of developing clinical decision support tools and population health management systems that consider each patient’s gender identity, sex assigned at birth, and anatomy.     

Recommendations for the safe,effective use of adaptive CDS in the US healthcare system: an AMIA position paper

The development and implementation of clinical decision support (CDS) that trains itself and adapts its algorithms based on new data—here referred to as Adaptive CDS—present unique challenges and considerations. Although Adaptive CDS represents an expected progression from earlier work, the activities needed to appropriately manage and support the establishment and evolution of Adaptive CDS require new, coordinated initiatives and oversight that do not currently exist. In this AMIA position paper, the authors describe current and emerging challenges to the safe use of Adaptive CDS and lay out recommendations for the effective management and monitoring of Adaptive CDS.     

Patient characteristics associated with venous thromboembolic events: a cohort study using pooled electronic health record data

Objective

To demonstrate the potential of de-identified clinical data from multiple healthcare systems using different electronic health records (EHR) to be efficiently used for very large retrospective cohort studies.

Materials and methods

Data of 959 030 patients, pooled from multiple different healthcare systems with distinct EHR, were obtained. Data were standardized and normalized using common ontologies, searchable through a HIPAA-compliant, patient de-identified web application (Explore; Explorys Inc). Patients were 26 years or older seen in multiple healthcare systems from 1999 to 2011 with data from EHR.

Results

Comparing obese, tall subjects with normal body mass index, short subjects, the venous thromboembolic events (VTE) OR was 1.83 (95% CI 1.76 to 1.91) for women and 1.21 (1.10 to 1.32) for men. Weight had more effect then height on VTE. Compared with Caucasian, Hispanic/Latino subjects had a much lower risk of VTE (female OR 0.47, 0.41 to 0.55; male OR 0.24, 0.20 to 0.28) and African-Americans a substantially higher risk (female OR 1.83, 1.76 to 1.91; male OR 1.58, 1.50 to 1.66). This 13-year retrospective study of almost one million patients was performed over approximately 125 h in 11 weeks, part time by the five authors.

Discussion

As research informatics tools develop and more clinical data become available in EHR, it is important to study and understand unique opportunities for clinical research informatics to transform the scale and resources needed to perform certain types of clinical research.

Conclusions

With the right clinical research informatics tools and EHR data, some types of very large cohort studies can be completed with minimal resources.     

The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety

With the proliferation of relatively mature health information technology (IT) systems with large numbers of users, it becomes increasingly important to evaluate the effect of these systems on the quality and safety of healthcare. Previous research on the effectiveness of health IT has had mixed results, which may be in part attributable to the evaluation frameworks used. The authors propose a model for evaluation, the Triangle Model, developed for designing studies of quality and safety outcomes of health IT. This model identifies structure-level predictors, including characteristics of: (1) the technology itself; (2) the provider using the technology; (3) the organizational setting; and (4) the patient population. In addition, the model outlines process predictors, including (1) usage of the technology, (2) organizational support for and customization of the technology, and (3) organizational policies and procedures about quality and safety. The Triangle Model specifies the variables to be measured, but is flexible enough to accommodate both qualitative and quantitative approaches to capturing them. The authors illustrate this model, which integrates perspectives from both health services research and biomedical informatics, with examples from evaluations of electronic prescribing, but it is also applicable to a variety of types of health IT systems.     

Organizational complements to electronic health records in ambulatory physician performance: the role of support staff

In industries outside healthcare, highly skilled employees enable substantial gains in productivity after adoption of information technologies. The authors explore whether the presence of highly skilled, autonomous clinical support staff is associated with higher performance among physicians with electronic health records (EHRs). Using data from a survey of general internists, the authors assessed whether physicians with EHRs were more likely to be top performers on cost and quality if they worked with nurse practitioners or physician assistants. It was found that, among physicians with EHRs, those with highly skilled, autonomous staff were far more likely to be top performing than those without such staff (OR 7.0, 95% CI 1.7 to 34.8, p=0.02). This relationship did not hold among physicians without EHRs (OR 1.0). As we begin a national push towards greater EHR adoption, it is critical to understand why some physicians gain from EHR use and others do not.     

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

ObjectivesThis systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.Materials and MethodsWe searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009–2019.ResultsWe identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009–2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.DiscussionOverall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.ConclusionImprovement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.     

Encoding laboratory testing data: case studies of the national implementation of HHS requirements and related standards in five laboratories

ObjectiveAssess the effectiveness of providing Logical Observation Identifiers Names and Codes (LOINC®)-to-In Vitro Diagnostic (LIVD) coding specification, required by the United States Department of Health and Human Services for SARS-CoV-2 reporting, in medical center laboratories and utilize findings to inform future United States Food and Drug Administration policy on the use of real-world evidence in regulatory decisions.Materials and MethodsWe compared gaps and similarities between diagnostic test manufacturers’ recommended LOINC® codes and the LOINC® codes used in medical center laboratories for the same tests.ResultsFive medical centers and three test manufacturers extracted data from laboratory information systems (LIS) for prioritized tests of interest. The data submission ranged from 74 to 532 LOINC® codes per site. Three test manufacturers submitted 15 LIVD catalogs representing 26 distinct devices, 6956 tests, and 686 LOINC® codes. We identified mismatches in how medical centers use LOINC® to encode laboratory tests compared to how test manufacturers encode the same laboratory tests. Of 331 tests available in the LIVD files, 136 (41%) were represented by a mismatched LOINC® code by the medical centers (chi-square 45.0, 4 df, P < .0001).DiscussionThe five medical centers and three test manufacturers vary in how they organize, categorize, and store LIS catalog information. This variation impacts data quality and interoperability.ConclusionThe results of the study indicate that providing the LIVD mappings was not sufficient to support laboratory data interoperability. National implementation of LIVD and further efforts to promote laboratory interoperability will require a more comprehensive effort and continuing evaluation and quality control.     

Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial

Objective

To determine the effects of a personal health record (PHR)-linked medications module on medication accuracy and safety.

Design

From September 2005 to March 2007, we conducted an on-treatment sub-study within a cluster-randomized trial involving 11 primary care practices that used the same PHR. Intervention practices received access to a medications module prompting patients to review their documented medications and identify discrepancies, generating ‘eJournals’ that enabled rapid updating of medication lists during subsequent clinical visits.

Measurements

A sample of 267 patients who submitted medications eJournals was contacted by phone 3 weeks after an eligible visit and compared with a matched sample of 274 patients in control practices that received a different PHR-linked intervention. Two blinded physician adjudicators determined unexplained discrepancies between documented and patient-reported medication regimens. The primary outcome was proportion of medications per patient with unexplained discrepancies.

Results

Among 121 046 patients in eligible practices, 3979 participated in the main trial and 541 participated in the sub-study. The proportion of medications per patient with unexplained discrepancies was 42% in the intervention arm and 51% in the control arm (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01). The number of unexplained discrepancies per patient with potential for severe harm was 0.03 in the intervention arm and 0.08 in the control arm (adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04).

Conclusions

When used, concordance between documented and patient-reported medication regimens and reduction in potentially harmful medication discrepancies can be improved with a PHR medication review tool linked to the provider''s medical record.

Trial registration number

This study was registered at ClinicalTrials.gov ({"type":"clinical-trial","attrs":{"text":"NCT00251875","term_id":"NCT00251875"}}NCT00251875).