A Case for Manual Entry of Structured,Coded Laboratory Data from Multiple Sources into an Ambulatory Electronic Health Record

Laboratory results provide necessary information for the management of ambulatory patients. To realize the benefits of an electronic health record (EHR) and coded laboratory data (e.g., decision support and improved data access and display), results from laboratories that are external to the health care enterprise need to be integrated with internal results. We describe the development and clinical impact of integrating external results into the EHR at Intermountain Health Care (IHC). During 2004, over 14,000 external laboratory results for 128 liver transplant patients were added to the EHR. The results were used to generate computerized alerts that assisted clinicians with managing laboratory tests in the ambulatory setting. The external results were sent from 85 different facilities and can now be viewed in the EHR integrated with IHC results. We encountered regulatory, logistic, economic, and data quality issues that should be of interest to others developing similar applications.     

Research and applications: Combining structured and unstructured data to identify a cohort of ICU patients who received dialysis

Objective

To develop a generalizable method for identifying patient cohorts from electronic health record (EHR) data—in this case, patients having dialysis—that uses simple information retrieval (IR) tools.

Methods

We used the coded data and clinical notes from the 24 506 adult patients in the Multiparameter Intelligent Monitoring in Intensive Care database to identify patients who had dialysis. We used SQL queries to search the procedure, diagnosis, and coded nursing observations tables based on ICD-9 and local codes. We used a domain-specific search engine to find clinical notes containing terms related to dialysis. We manually validated the available records for a 10% random sample of patients who potentially had dialysis and a random sample of 200 patients who were not identified as having dialysis based on any of the sources.

Results

We identified 1844 patients that potentially had dialysis: 1481 from the three coded sources and 1624 from the clinical notes. Precision for identifying dialysis patients based on available data was estimated to be 78.4% (95% CI 71.9% to 84.2%) and recall was 100% (95% CI 86% to 100%).

Conclusions

Combining structured EHR data with information from clinical notes using simple queries increases the utility of both types of data for cohort identification. Patients identified by more than one source are more likely to meet the inclusion criteria; however, including patients found in any of the sources increases recall. This method is attractive because it is available to researchers with access to EHR data and off-the-shelf IR tools.     

Validity of electronic health record-derived quality measurement for performance monitoring

Background

Since 2007, New York City''s primary care information project has assisted over 3000 providers to adopt and use a prevention-oriented electronic health record (EHR). Participating practices were taught to re-adjust their workflows to use the EHR built-in population health monitoring tools, including automated quality measures, patient registries and a clinical decision support system. Practices received a comprehensive suite of technical assistance, which included quality improvement, EHR customization and configuration, privacy and security training, and revenue cycle optimization. These services were aimed at helping providers understand how to use their EHR to track and improve the quality of care delivered to patients.

Materials and Methods

Retrospective electronic chart reviews of 4081 patient records across 57 practices were analyzed to determine the validity of EHR-derived quality measures and documented preventive services.

Results

Results from this study show that workflow and documentation habits have a profound impact on EHR-derived quality measures. Compared with the manual review of electronic charts, EHR-derived measures can undercount practice performance, with a disproportionately negative impact on the number of patients captured as receiving a clinical preventive service or meeting a recommended treatment goal.

Conclusion

This study provides a cautionary note in using EHR-derived measurement for public reporting of provider performance or use for payment.     

The Relationship between Electronic Health Record Use and Quality of Care over Time

Objective

Electronic health records (EHRs) have the potential to advance the quality of care, but studies have shown mixed results. The authors sought to examine the extent of EHR usage and how the quality of care delivered in ambulatory care practices varied according to duration of EHR availability.

Methods

The study linked two data sources: a statewide survey of physicians' adoption and use of EHR and claims data reflecting quality of care as indicated by physicians' performance on widely used quality measures. Using four years of measurement, we combined 18 quality measures into 6 clinical condition categories. While the survey of physicians was cross-sectional, respondents indicated the year in which they adopted EHR. In an analysis accounting for duration of EHR use, we examined the relationship between EHR adoption and quality of care.

Results

The percent of physicians reporting adoption of EHR and availability of EHR core functions more than doubled between 2000 and 2005. Among EHR users in 2005, the average duration of EHR use was 4.8 years. For all 6 clinical conditions, there was no difference in performance between EHR users and non-users. In addition, for these 6 clinical conditions, there was no consistent pattern between length of time using an EHR and physicians performance on quality measures in both bivariate and multivariate analyses.

Conclusions

In this cross-sectional study, we found no association between duration of using an EHR and performance with respect to quality of care, although power was limited. Intensifying the use of key EHR features, such as clinical decision support, may be needed to realize quality improvement from EHRs. Future studies should examine the relationship between the extent to which physicians use key EHR functions and their performance on quality measures over time.     

Measures of electronic health record use in outpatient settings across vendors

Electronic health record (EHR) log data capture clinical workflows and are a rich source of information to understand variation in practice patterns. Variation in how EHRs are used to document and support care delivery is associated with clinical and operational outcomes, including measures of provider well-being and burnout. Standardized measures that describe EHR use would facilitate generalizability and cross-institution, cross-vendor research. Here, we describe the current state of outpatient EHR use measures offered by various EHR vendors, guided by our prior conceptual work that proposed seven core measures to describe EHR use. We evaluate these measures and other reporting options provided by vendors for maturity and similarity to previously proposed standardized measures. Working toward improved standardization of EHR use measures can enable and accelerate high-impact research on physician burnout and job satisfaction as well as organizational efficiency and patient health.     

Measuring time clinicians spend using EHRs in the inpatient setting: a national,mixed-methods study

ObjectiveTo understand hospitals’ use of EHR audit-log-based measures to address burden associated with inpatient EHR use.Materials and MethodsUsing mixed methods, we analyzed 2018 American Hospital Association Information Technology Supplement Survey data (n = 2864 hospitals; 64% response rate) to characterize measures used and provided by EHR vendors to track clinician time spent documenting. We interviewed staff from the top 3 EHR vendors that provided these measures. Multivariable analyses identified variation in use of the measures among hospitals with these 3 vendors.Results53% of hospitals reported using EHR data to track clinician time documenting, compared to 68% of the hospitals using the EHR from the top 3 vendors. Among hospitals with EHRs from these vendors, usage was significantly lower among rural hospitals and independent hospitals (P < .05). Two of these vendors provided measures of time spent doing specific tasks while the third measured an aggregate of auditable activities. Vendors varied in the underlying data used to create measures, measure specification, and data displays.DiscussionTools to track clinicians’ documentation time are becoming more available. The measures provided differ across vendors and disparities in use exist across hospitals. Increasing the specificity of standards underlying the data would support a common set of core measures making these measures more widely available.ConclusionAlthough half of US hospitals use measures of time spent in the EHR derived from EHR generated data, work remains to make such measures and analyses more broadly available to all hospitals and to increase its utility for national burden measurement.     

PROTEMPA: A Method for Specifying and Identifying Temporal Sequences in Retrospective Data for Patient Selection

ObjectiveTo specify and identify disease and patient care processes represented by temporal patterns in clinical events and observations, and retrieve patient populations containing those patterns from clinical data repositories, in support of clinical research, outcomes studies, and quality assurance.DesignA data processing method called PROTEMPA (Process-oriented Temporal Analysis) was developed for defining and detecting clinically relevant temporal and mathematical patterns in retrospective data. PROTEMPA provides for portability across data sources, “pluggable” data processing environments, and the creation of libraries of pattern definitions and data processing algorithms.MeasurementsA proof-of-concept implementation of PROTEMPA in Java was evaluated against standard SQL queries for its ability to identify patients from a large clinical data repository who show the features of HELLP syndrome, and categorize those patients by disease severity and progression based on time sequence characteristics in their clinical laboratory test results. Results were verified by manual case review.ResultsThe proof-of-concept implementation was more accurate than SQL in identifying patients with HELLP and correctly assigned severity and disease progression categories, which was not possible using SQL only.ConclusionsPROTEMPA supports the identification and categorization of patients with complex disease based on the characteristics of and relationships between time sequences in multiple data types. Identifying patient populations who share these types of patterns may be useful when patient features of interest do not have standard codes, are poorly-expressed in coding schemes, may be inaccurately or incompletely coded, or are not represented explicitly as data values.     

Facilitating Clinical Outcomes Assessment through the Automated Identification of Quality Measures for Prostate Cancer Surgery

Objectives

The College of American Pathologists (CAP) Category 1 quality measures, tumor stage, Gleason score, and surgical margin status, are used by physicians and cancer registrars to categorize patients into groups for clinical trials and treatment planning. This study was conducted to evaluate the effectiveness of an application designed to automatically extract these quality measures from the postoperative pathology reports of patients having undergone prostatectomies for treatment of prostate cancer.

Design

An application was developed with the Clinical Outcomes Assessment Toolkit that uses an information pipeline of regular expressions and support vector machines to extract CAP Category 1 quality measures. System performance was evaluated against a gold standard of 676 pathology reports from the University of California at Los Angeles Medical Center and Brigham and Women''s Hospital. To evaluate the feasibility of clinical implementation, all pathology reports were gathered using administrative codes with no manual preprocessing of the data performed.

Measurements

The sensitivity, specificity, and overall accuracy of system performance were measured for all three quality measures. Performance at both hospitals was compared, and a detailed failure analysis was conducted to identify errors caused by poor data quality versus system shortcomings.

Results

Accuracies for Gleason score were 99.7%, tumor stage 99.1%, and margin status 97.2%, for an overall accuracy of 98.67%. System performance on data from both hospitals was comparable. Poor clinical data quality led to a decrease in overall accuracy of only 0.3% but accounted for 25.9% of the total errors.

Conclusion

Despite differences in document format and pathologists'' reporting styles, strong system performance indicates the potential of using a combination of regular expressions and support vector machines to automatically extract CAP Category 1 quality measures from postoperative prostate cancer pathology reports.     

Constructing Large Scale Cohort for Clinical Study on Heart Failure with Electronic Health Record in Regional Healthcare Platform: Challenges and Strategies in Data Reuse

Regional healthcare platforms collect clinical data from hospitals in specific areas for the purpose of healthcare management. It is a common requirement to reuse the data for clinical research. However, we have to face challenges like the inconsistence of terminology in electronic health records (EHR) and the complexities in data quality and data formats in regional healthcare platform. In this paper, we propose methodology and process on constructing large scale cohorts which forms the basis of causality and comparative effectiveness relationship in epidemiology. We firstly constructed a Chinese terminology knowledge graph to deal with the diversity of vocabularies on regional platform. Secondly, we built special disease case repositories (i.e., heart failure repository) that utilize the graph to search the related patients and to normalize the data. Based on the requirements of the clinical research which aimed to explore the effectiveness of taking statin on 180-days readmission in patients with heart failure, we built a large-scale retrospective cohort with 29647 cases of heart failure patients from the heart failure repository. After the propensity score matching, the study group (n=6346) and the control group (n=6346) with parallel clinical characteristics were acquired. Logistic regression analysis showed that taking statins had a negative correlation with 180-days readmission in heart failure patients. This paper presents the workflow and application example of big data mining based on regional EHR data.     

基于数据共享的中医电子病历研究

中医信息化水平的迅速发展促使中医电子病历数据共享成为迫切需要。重点论述中医电子病历的特色,研究基于HL7 CDA标准构建中医电子病历共享文档的方法和流程,最后讨论电子病历相关安全问题及解决途径,为中医电子病历数据共享进程的推进提供参考。     

Clinical data sharing improves quality measurement and patient safety

ObjectiveAccurate and robust quality measurement is critical to the future of value-based care. Having incomplete information when calculating quality measures can cause inaccuracies in reported patient outcomes. This research examines how quality calculations vary when using data from an individual electronic health record (EHR) and longitudinal data from a health information exchange (HIE) operating as a multisource registry for quality measurement. Materials and MethodsData were sampled from 53 healthcare organizations in 2018. Organizations represented both ambulatory care practices and health systems participating in the state of Kansas HIE. Fourteen ambulatory quality measures for 5300 patients were calculated using the data from an individual EHR source and contrasted to calculations when HIE data were added to locally recorded data.ResultsA total of 79% of patients received care at more than 1 facility during the 2018 calendar year. A total of 12 994 applicable quality measure calculations were compared using data from the originating organization vs longitudinal data from the HIE. A total of 15% of all quality measure calculations changed (P < .001) when including HIE data sources, affecting 19% of patients. Changes in quality measure calculations were observed across measures and organizations.DiscussionThese results demonstrate that quality measures calculated using single-site EHR data may be limited by incomplete information. Effective data sharing significantly changes quality calculations, which affect healthcare payments, patient safety, and care quality.ConclusionsFederal, state, and commercial programs that use quality measurement as part of reimbursement could promote more accurate and representative quality measurement through methods that increase clinical data sharing.     

Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory

ObjectiveWe identified challenges and solutions to using electronic health record (EHR) systems for the design and conduct of pragmatic research.Materials and MethodsSince 2012, the Health Care Systems Research Collaboratory has served as the resource coordinating center for 21 pragmatic clinical trial demonstration projects. The EHR Core working group invited these demonstration projects to complete a written semistructured survey and used an inductive approach to review responses and identify EHR-related challenges and suggested EHR enhancements.ResultsWe received survey responses from 20 projects and identified 21 challenges that fell into 6 broad themes: (1) inadequate collection of patient-reported outcome data, (2) lack of structured data collection, (3) data standardization, (4) resources to support customization of EHRs, (5) difficulties aggregating data across sites, and (6) accessing EHR data.DiscussionBased on these findings, we formulated 6 prerequisites for PCTs that would enable the conduct of pragmatic research: (1) integrate the collection of patient-centered data into EHR systems, (2) facilitate structured research data collection by leveraging standard EHR functions, usable interfaces, and standard workflows, (3) support the creation of high-quality research data by using standards, (4) ensure adequate IT staff to support embedded research, (5) create aggregate, multidata type resources for multisite trials, and (6) create re-usable and automated queries.ConclusionWe are hopeful our collection of specific EHR challenges and research needs will drive health system leaders, policymakers, and EHR designers to support these suggestions to improve our national capacity for generating real-world evidence.     

Electronic health records improve clinical note quality

Background and objective The clinical note documents the clinician''s information collection, problem assessment, clinical management, and its used for administrative purposes. Electronic health records (EHRs) are being implemented in clinical practices throughout the USA yet it is not known whether they improve the quality of clinical notes. The goal in this study was to determine if EHRs improve the quality of outpatient clinical notes.Materials and methods A five and a half year longitudinal retrospective multicenter quantitative study comparing the quality of handwritten and electronic outpatient clinical visit notes for 100 patients with type 2 diabetes at three time points: 6 months prior to the introduction of the EHR (before-EHR), 6 months after the introduction of the EHR (after-EHR), and 5 years after the introduction of the EHR (5-year-EHR). QNOTE, a validated quantitative instrument, was used to assess the quality of outpatient clinical notes. Its scores can range from a low of 0 to a high of 100. Sixteen primary care physicians with active practices used QNOTE to determine the quality of the 300 patient notes.Results The before-EHR, after-EHR, and 5-year-EHR grand mean scores (SD) were 52.0 (18.4), 61.2 (16.3), and 80.4 (8.9), respectively, and the change in scores for before-EHR to after-EHR and before-EHR to 5-year-EHR were 18% (p<0.0001) and 55% (p<0.0001), respectively. All the element and grand mean quality scores significantly improved over the 5-year time interval.Conclusions The EHR significantly improved the overall quality of the outpatient clinical note and the quality of all its elements, including the core and non-core elements. To our knowledge, this is the first study to demonstrate that the EHR significantly improves the quality of clinical notes.     

Automated detection of substance use information from electronic health records for a pediatric population

ObjectiveSubstance use screening in adolescence is unstandardized and often documented in clinical notes, rather than in structured electronic health records (EHRs). The objective of this study was to integrate logic rules with state-of-the-art natural language processing (NLP) and machine learning technologies to detect substance use information from both structured and unstructured EHR data.Materials and MethodsPediatric patients (10-20 years of age) with any encounter between July 1, 2012, and October 31, 2017, were included (n = 3890 patients; 19 478 encounters). EHR data were extracted at each encounter, manually reviewed for substance use (alcohol, tobacco, marijuana, opiate, any use), and coded as lifetime use, current use, or family use. Logic rules mapped structured EHR indicators to screening results. A knowledge-based NLP system and a deep learning model detected substance use information from unstructured clinical narratives. System performance was evaluated using positive predictive value, sensitivity, negative predictive value, specificity, and area under the receiver-operating characteristic curve (AUC).ResultsThe dataset included 17 235 structured indicators and 27 141 clinical narratives. Manual review of clinical narratives captured 94.0% of positive screening results, while structured EHR data captured 22.0%. Logic rules detected screening results from structured data with 1.0 and 0.99 for sensitivity and specificity, respectively. The knowledge-based system detected substance use information from clinical narratives with 0.86, 0.79, and 0.88 for AUC, sensitivity, and specificity, respectively. The deep learning model further improved detection capacity, achieving 0.88, 0.81, and 0.85 for AUC, sensitivity, and specificity, respectively. Finally, integrating predictions from structured and unstructured data achieved high detection capacity across all cases (0.96, 0.85, and 0.87 for AUC, sensitivity, and specificity, respectively).ConclusionsIt is feasible to detect substance use screening and results among pediatric patients using logic rules, NLP, and machine learning technologies.     

Harmonized patient-reported data elements in the electronic health record: supporting meaningful use by primary care action on health behaviors and key psychosocial factors

Background

Electronic health records (EHR) have the potential to improve patient care through efficient access to complete patient health information. This potential may not be reached because many of the most important determinants of health outcome are rarely included. Successful health promotion and disease prevention requires patient-reported data reflecting health behaviors and psychosocial issues. Furthermore, there is a need to harmonize this information across different EHR systems.

Methods

To fill this gap a three-phased process was used to conceptualize, identify and recommend patient-reported data elements on health behaviors and psychosocial factors for the EHR. Expert panels (n=13) identified candidate measures (phase 1) that were reviewed and rated by a wide range of health professionals (n=93) using the grid-enabled measures wiki social media platform (phase 2). Recommendations were finalized through a town hall meeting with key stakeholders including patients, providers, researchers, policy makers, and representatives from healthcare settings (phase 3).

Results

Nine key elements from three areas emerged as the initial critical patient-reported elements to incorporate systematically into EHR—health behaviors (eg, exercise), psychosocial issues (eg, distress), and patient-centered factors (eg, demographics). Recommendations were also made regarding the frequency of collection ranging from a single assessment (eg, demographic characteristics), to annual assessment (eg, health behaviors), or more frequent (eg, patient goals).

Conclusions

There was strong stakeholder support for this initiative reflecting the perceived value of incorporating patient-reported elements into EHR. The next steps will include testing the feasibility of incorporating these elements into the EHR across diverse primary care settings.     

Use of a support vector machine for categorizing free-text notes: assessment of accuracy across two institutions

Background

Electronic health record (EHR) users must regularly review large amounts of data in order to make informed clinical decisions, and such review is time-consuming and often overwhelming. Technologies like automated summarization tools, EHR search engines and natural language processing have been shown to help clinicians manage this information.

Objective

To develop a support vector machine (SVM)-based system for identifying EHR progress notes pertaining to diabetes, and to validate it at two institutions.

Materials and methods

We retrieved 2000 EHR progress notes from patients with diabetes at the Brigham and Women''s Hospital (1000 for training and 1000 for testing) and another 1000 notes from the University of Texas Physicians (for validation). We manually annotated all notes and trained a SVM using a bag of words approach. We then used the SVM on the testing and validation sets and evaluated its performance with the area under the curve (AUC) and F statistics.

Results

The model accurately identified diabetes-related notes in both the Brigham and Women''s Hospital testing set (AUC=0.956, F=0.934) and the external University of Texas Faculty Physicians validation set (AUC=0.947, F=0.935).

Discussion

Overall, the model we developed was quite accurate. Furthermore, it generalized, without loss of accuracy, to another institution with a different EHR and a distinct patient and provider population.

Conclusions

It is possible to use a SVM-based classifier to identify EHR progress notes pertaining to diabetes, and the model generalizes well.     

临床试验计量重复测量资料的统计

目的 探讨药物临床试验计量重复测量资料统计分析方法的选择。方法 通过实例列出各种分析方法并进行比较。结果 临床药物研究资料常为重复测量资料，比较治疗组与对照组测量值是否有显著性差别，在统计方法选择策略上一种是对一个代表性值进行比较，采用传统的统计方法如两独立样本t检验、方差分析和协方差分析；另一种策略是采用混合效应模型对整个研究过程中所有时点的测量值进行分析。结论 两种统计策略均可以采用，但混合效应模型是较好的分析方法。药物临床试验研究资料常是重复测量资料，各时点值之间存在相关性。混合效应模型可以充分利用信息，既可以分析随机效应和相关性，又能处理缺损值。     

科研型电子健康档案系统发展与展望

随着越来越多的医疗机构开始应用电子健康档案系统（Electronic Health Records,EHR）来管理患者资料,基于在临床研究工作对患者资料的需求,各研究机构也开始以电子健康档案系统作为临床研究的数据来源。EHRCR（Electronic Health Records／Clinical Research）项目是在2006年12月由HL7技术委员会（Health Level Seven Technical Committee,HL7TC）和欧洲健康档案研究所（European Institute for Health Records,EuroRec）发起,旨在研究未来可以支持临床研究的电子健康档案系统应具有的功能,以及与此相关的系统、网络和业务流程。因此,对该项目的最新研究成果加以介绍,作为我国电子健康档案行业发展的参考。     

有序多分类重复测量资料的广义估计方程分析

目的 探讨广义估计方程在有序多分类重复测量资料中的应用，为临床试验中的重复测量资料的正确分析提供方法学上的参考。方法 采用SAS软件包的GENMOD语句拟合广义估计方程，进行实例分析，并和独立logistic回归分析结果进行对比。结果 获得了各参数及其标准误的估计值。可以对各因素进行直观的参数估计。广义估计方程各参数估计值标准误普遍大于独立logistic回归估计值的标准误，从而使得检验结果发生了变化。结论 广义估计方程引入工作相关矩阵以处理非独立数据之间的相关性，可以有效地控制层次相关性、重复测量因素及其它混杂因素，为有序多分类重复测量资料提供了一种有效的分析方法。     

基于大数据糖尿病患者膳食管理系统的设计

作为5项糖尿病治疗方法（饮食、运动、药物、自我监测与教育）中最基本的治疗方法，饮食治疗在糖尿病患者病程的各个阶段都极为重要。针对临床中糖尿病患者膳食管理方案编制繁琐、饮食治疗效果不佳的问题，结合糖尿病患者饮食管理工作步骤，设计糖尿病患者膳食管理智能决策流程；借助相似度计算模型，实现数据挖掘与智能决策，设计糖尿病患者饮食智能决策系统架构；借助计算机编程技术建立系统可视化界面；利用实例推理方法实现基于大数据的糖尿病患者膳食智能管理，减少糖尿病患者膳食管理工作量，提高糖尿病饮食治疗效率，为辅助临床快速、准确制定个性化的糖尿病患者饮食方案提供有效途径。