Leveraging dialog systems research to assist biomedical researchers’ interrogation of Big Clinical Data

The worldwide adoption of electronic health records (EHR) promises to accelerate clinical research, which lies at the heart of medical advances. However, the interrogation of such Big Data by clinical researchers can be laborious and error-prone, involving iterative and ineffective communication of data requests to data analysts. Research on this communication process is rare. There also exists no contemporary system that offers intelligent solutions to assist clinical researchers in their quest for clinical data. In this article, we first provide a detailed characterization of the challenges encountered in this communication space. Second, we identify promising synergies between fields studying human-to-human and human–machine communication that can shed light on biomedical data query mediation. We propose a mixed-initiative dialog-based approach to support autonomous clinical data access and recommend needed technology development and communication study for accelerating clinical research.     

The profession of public health informatics: Still emerging?

Purpose

Although public health informatics (PHI) was defined in 1995, both then and still now it is an “emerging” profession. An emergent profession lacks a base of “technical specialized knowledge.” Therefore, we analyzed MEDLINE bibliographic citation records of the PHI literature to determine if a base of technical, specialized PHI literature exists, which could lead to the conclusion that PHI has emerged from its embryonic state.

Method

A MEDLINE search for PHI literature published from 1980-2006 returned 16,942 records. Record screening by two subject matter experts netted 2493 PHI records that were analyzed by the intervals of previous PHI CBMs 96-4 and 2001-2 for 1980-1995 (I₁₉₈₀) and 1996-2000 (I₁₉₉₆), respectively, and a new, third interval of 2001-2006 (I₂₀₀₁).

Results

The distribution of records was 676 (I₁₉₈₀), 839 (I₁₉₉₆) and 978 (I₂₀₀₁). Annual publication rates were 42 (I₁₉₈₀), 168 (I₁₉₉₆), and 163 (I₂₀₀₁). Cumulative publications were accelerating. A subset of 19 (2.5%) journals accounted for 730 (29.3%) of the records. The journal subset average (±SD) annual publication rates of 0.7 ± 0.6 (I₁₉₈₀), 2.9 ± 1.9 (I₁₉₉₆), and 3.1 ± 2.7 (I₂₀₀₁) were different, F(3, 64) = 7.12, p < .05. Only I₁₉₈₀ was different (p < .05) from I₁₉₉₆ or I₂₀₀₁. Average (±SE) annual rate of increase for all journals (8.4 ± 0.8 publications per year) was different from the subset of 19 (2.7 ± 0.3), t(36) = 5.74, p < .05. MeSH first time-to-indexing narrowed from 7.3 (±4.3) years to the year (0.5 ± 0.8) the term was introduced, t(30) = 7.03, p < .05.

Conclusion

A core set of journals, the proliferation of PHI articles in varied and numerous journals, and rapid uptake of MeSH suggest PHI is acquiring professional authority and now should not be tagged as an “emerging” profession.     

Factors affecting physicians’ use of a dedicated overview interface in an electronic health record: The importance of standard information and standard documentation

BackgroundIt remains a continual challenge to present information in user interfaces in large IT systems to support overview in the best possible way. We here examine how an electronic health record (EHR) supports the creation of overview among hospital physicians with a particular focus on the use of an interface designed to provide clinicians with a patient information overview. The overview interface integrates information flexibly from diverse places in the EHR and presents this information in one screen display. Our study revealed widespread non-use of the overview interface. We explore the reasons for its use and non-use.MethodWe conducted exploratory ethnographic fieldwork among physicians in two hospitals and gathered statistical data on their use of the overview interface. From the quantitative data, we identified where the interface was used most and conducted 18 semi-structured, open-ended interviews framed by the theoretical framework and the findings of the initial ethnographic fieldwork. We interviewed both physicians and employees from the IT units in different hospitals. We then analysed notes from the ethnographic fieldwork and the interviews and ordered these into themes forming the basis for the presentation of findings.ResultsThe overview interface was most used in departments or situations where the problem at hand and the need for information could be standardized—in particular, in anesthesiological departments and outpatient clinics. However, departments with complex and long patient histories did not make much use of the overview interface. Design and layout were not mentioned as decisive factors affecting its use or non-use. Many physicians questioned the completeness of data in the overview interface—either because they were skeptical about the hospital’s or the department’s documentation practices, or because they could not recognize the structure of the interface. This uncertainty discouraged physicians from using the overview interface.ConclusionDedicating a specific function or interface to supporting overview works best where information needs can be standardized. The narrative and contextual nature of creating clinical overview is unlikely to be optimally supported by using the overview interface alone. The use of these kinds of interfaces requires trust in data completeness and other clinicians' and administrative staff's documentation practices, as well as an understanding of the underlying structure of the EHR and how information is filtered when data are aggregated for the interface.     

Can data from an electronic medical record identify which patients with pneumonia have Pneumocystis carinii Infection?

BACKGROUND: Pneumocystis carinii is the leading opportunistic pulmonary infection in HIV-infected patients. Invasive diagnostic procedures might be avoided if available electronic data can accurately identify patients with Pneumocystis pneumonia (PCP). METHODS: We extracted data from electronic hospital records, emergency department records, and a pathology database for 299 HIV-infected patients with pneumonia who underwent bronchoscopy. We identified independent indicators of confirmed PCP using logistic regression analysis on a random half of the patients and validated the predictive power of the resulting model on the other half. RESULTS: Bronchoscopy confirmed pneumocystis carinii in 111 patients (37%). Five of the seven significant independent predictors of PCP came from patients' electronic medical records: infiltrate on chest radiograph, male gender, lower red cell distribution width, lower serum creatinine, and a prior positive HIV test. The other two (duration of illness and presence of dyspnea) came from the emergency department record. A simple index found 43% of patients at low risk (18% with pneumocystis), 37% at moderate risk (36% with pneumocystis), and 20% at high risk (74% with pneumocystis). CONCLUSIONS: Data from electronic medical records can help quantify the risk of PCP among HIV-infected patients. However, the model failed to identify 18% of patients with PCP in the low risk group, and empiric therapy would erroneously treat 26% of patients classified as high risk. Bronchoscopy is needed to accurately diagnose PCP among HIV-infected patients with pneumonia. However, if bronchoscopy is not available, the model can help with initial decisions about antibiotic therapy.     

The Geisinger MyCode community health initiative: an electronic health record–linked biobank for precision medicine research

PurposeGeisinger Health System (GHS) provides an ideal platform for Precision Medicine. Key elements are the integrated health system, stable patient population, and electronic health record (EHR) infrastructure. In 2007, Geisinger launched MyCode, a system-wide biobanking program to link samples and EHR data for broad research use.MethodsPatient-centered input into MyCode was obtained using participant focus groups. Participation in MyCode is based on opt-in informed consent and allows recontact, which facilitates collection of data not in the EHR and, since 2013, the return of clinically actionable results to participants. MyCode leverages Geisinger’s technology and clinical infrastructure for participant tracking and sample collection.ResultsMyCode has a consent rate of >85%, with more than 90,000 participants currently and with ongoing enrollment of ~4,000 per month. MyCode samples have been used to generate molecular data, including high-density genotype and exome sequence data. Genotype and EHR-derived phenotype data replicate previously reported genetic associations.ConclusionThe MyCode project has created resources that enable a new model for translational research that is faster, more flexible, and more cost-effective than traditional clinical research approaches. The new model is scalable and will increase in value as these resources grow and are adopted across multiple research platforms.Genet Med 18 9, 906–913.     

The clinician in the Driver’s Seat: Part 1 – A drag/drop user-composable electronic health record platform

Creating electronic health records that support the uniquely complex and varied needs of healthcare presents formidable challenges. To address some of these challenges we created a new model for healthcare information systems, embodied in MedWISE,² a widget-based highly configurable electronic health record (EHR) platform. Founded on the idea that providing clinician users with greater control of the EHR may result in greater fit to user needs and preferences, MedWISE allows drag/drop user configurations and the sharing of user-created elements such as custom laboratory result panels and user-created interface tabs.After reviewing the current state of EHR configurability, we describe the philosophical, theoretical and practical rationales for our model, and the specific functionality of MedWISE. The alternative approach may have several advantages for human–computer interaction, efficiency, cognition, and fit of EHR tools to different contexts and tasks. We discuss potential issues raised by this approach.     

Electronic health record as a research tool: Frequency of exposure to targeted clinical problems and health care providers’ clinical proficiency

ObjectivesThe Electronic Health Record (EHR) could provide insight into possible decay in health care providers’ (HCP) clinical knowledge and cognitive performance. Analyses of the contributions of variables such as frequency of exposure to targeted clinical problems could inform the development and testing of appropriate individualized interventions to mitigate these threats to quality and safety of care.Materials/MethodsNine targeted clinical problems (TCP) were selected for further study, and de-identified, aggregated study data were obtained for one calendar year. Task analysis interviews of subspecialty physicians defined optimal management of each TCP and guided specification of quality of care metrics that could be extracted from the EHR. The Δ-t statistic, days since the provider’s prior encounter with a given TCP, quantified frequency of exposure.ResultsFrequency of patient encounters ranged from 1566 to 220,774 visits across conditions. Mean Δ-t ranged from 1.72 days to 30.79 days. Maximum Δ-t ranged from 285 to 497 days. The distribution of Δ-t for the TCPs generally fit a Gamma distribution (P < 0.001), indicating that Δ-t conforms to a Poisson process. A quality of care metric derived for each TCP declined progressively with increasing Δ-t for 8 of the 9 TCPs, affirming that knowledge decay was detectable from EHR data.Discussion/ConclusionsThis project demonstrates the utility of the EHR as a research tool in studies of health care delivery in association with frequency of exposure of HCPs to TCPs. Subsequent steps in our research include multivariate modeling of clinical knowledge decay and randomized trials of pertinent preventive interventions.     

The quality of evidence in health informatics: how did the quality of healthcare IT evaluation publications develop from 1982 to 2005?

OBJECTIVE: To obtain an overview of study designs and study methods used in research evaluating IT in health care, to present a list of quality criteria by which all kinds of reported evaluation studies on IT systems in health care can be assessed, and to assess the quality of reported evaluation studies on IT in health care and its development over time (1982-2005). METHODS: A generic 10-item list of quality indicators was developed based on existing literature on quality of medical and medical informatics publications. It is applicable to all kind of IT evaluation papers and not restricted to randomized controlled trials. One hundred and twenty explanatory papers evaluating the effects of an IT system in health care published between 1982 and 2005 were randomly selected from PubMed, the study designs and study methods were extracted, and the quality indicators were used to assess the quality of each paper by two independent raters. RESULTS: The inter-rater variability of scoring the 10 quality indicators as assessed by a pre-test with nine papers was good (K=0.87). There was a trend towards more multi-centre studies and authors coming more frequently from various departments. About 70% of the studies used a design other than a randomized controlled trial (RCT). Forty percent of the studies combined at least two different data acquisition methods. The quality of IT evaluation papers, as defined by the quality indicators, was only slightly improving in time (Spearman correlation coefficient [rs]=0.19). The quality of RCTs publications was significantly higher than the quality of non-RCT studies (p<0.001). CONCLUSION: The continuous and dominant number of non-RCT studies reflects the various approaches applicable to evaluate IT systems in health care. Despite the increasing discussion on evidence-based health informatics, the quality of published evaluation studies on IT interventions in health care is still insufficient in some aspects. Journal editors and referees should take care that reports of evaluation on IT systems contain all aspects needed for a sufficient understanding and reproducibility of a paper. Publication guidelines should be developed to support more complete and better publications of IT evaluation papers.     

Comparing virtual consults to traditional consults using an electronic health record: an observational case¿control study

ABSTRACT: BACKGROUND: Patients have typically received health care through face-to-face encounters. However, expansion of electronic communication and electronic health records (EHRs) provide alternative means for patient and physicians to interact. Electronic consultations may complement regular healthcare by providing "better, faster, cheaper" processes for diagnosing, treating, and monitoring health conditions. Virtual consultation between physicians may provide a method of streamlining care, potentially saving patients the time and expense of added visits. The purpose of this study was to compare physician usage and patient satisfaction with virtual consultations (VCs) with traditional consultations (TCs) facilitated within an EHR. METHODS: We conducted an observational case-control survey study within Kaiser Permanente, Colorado. A sample of patients who had VC requests (N=270) were matched with patients who had TC requests (N=270) by patient age, gender, reason for the consult, and specialty department. These patients (VC and TC), were invited to participate in a satisfaction survey. In addition, 205 primary care physicians who submitted a VC or TC were surveyed. RESULTS: During the study period, 58,146 VC or TC were requested (TC = 96.3%). Patients who completed a satisfaction survey (268 out of 540 patients, 49.6% response rate) indicated they were satisfied with their care, irrespective of the kind of consult (mean 10-point Likert score of 8.5). 88 of 205 primary care physicians surveyed (42.9%) returned at least one survey; VC and TC survey response rates and consulted departments were comparable (p = 0.13). More TCs than VCs requested transfer of patient care (p=0.03), assistance with diagnosis (p = 0.04) or initiating treatment (p =0.04). Within 3 weeks of the consultation request, 72.1% of respondents reported receiving information from VCs, compared with 33.9% of the TCs (p < 0.001). Utility of information provided by consultants and satisfaction with consultations did not differ between VCs and TCs. CONCLUSIONS: Referring physicians received information from consultants more quickly from VCs compared with TCs, but the value and application of information from both types of consultations were similar. VCs may decrease the need for face-to-face specialty encounters without a decrease in the patient's perception of care.     

Giving patients granular control of personal health information: Using an ethics ‘Points to Consider’ to inform informatics system designers

ObjectiveThere are benefits and risks of giving patients more granular control of their personal health information in electronic health record (EHR) systems. When designing EHR systems and policies, informaticists and system developers must balance these benefits and risks. Ethical considerations should be an explicit part of this balancing. Our objective was to develop a structured ethics framework to accomplish this.MethodsWe reviewed existing literature on the ethical and policy issues, developed an ethics framework called a “Points to Consider” (P2C) document, and convened a national expert panel to review and critique the P2C.ResultsWe developed the P2C to aid informaticists designing an advanced query tool for an electronic health record (EHR) system in Indianapolis. The P2C consists of six questions (“Points”) that frame important ethical issues, apply accepted principles of bioethics and Fair Information Practices, comment on how questions might be answered, and address implications for patient care.DiscussionThe P2C is intended to clarify what is at stake when designers try to accommodate potentially competing ethical commitments and logistical realities. The P2C was developed to guide informaticists who were designing a query tool in an existing EHR that would permit patient granular control. While consideration of ethical issues is coming to the forefront of medical informatics design and development practices, more reflection is needed to facilitate optimal collaboration between designers and ethicists. This report contributes to that discussion.     

We work with them? Healthcare workers interpretation of organizational relations mined from electronic health records

ObjectiveModels of healthcare organizations (HCOs) are often defined up front by a select few administrative officials and managers. However, given the size and complexity of modern healthcare systems, this practice does not scale easily. The goal of this work is to investigate the extent to which organizational relationships can be automatically learned from utilization patterns of electronic health record (EHR) systems.MethodWe designed an online survey to solicit the perspectives of employees of a large academic medical center. We surveyed employees from two administrative areas: (1) Coding & Charge Entry and (2) Medical Information Services and two clinical areas: (3) Anesthesiology and (4) Psychiatry. To test our hypotheses we selected two administrative units that have work-related responsibilities with electronic records; however, for the clinical areas we selected two disciplines with very different patient responsibilities and whose accesses and people who accessed were similar. We provided each group of employees with questions regarding the chance of interaction between areas in the medical center in the form of association rules (e.g., Given someone from Coding & Charge Entry accessed a patient's record, what is the chance that someone from Medical Information Services access the same record?). We compared the respondent predictions with the rules learned from actual EHR utilization using linear-mixed effects regression models.ResultsThe findings from our survey confirm that medical center employees can distinguish between association rules of high and non-high likelihood when their own area is involved. Moreover, they can make such distinctions between for any HCO area in this survey. It was further observed that, with respect to highly likely interactions, respondents from certain areas were significantly better than other respondents at making such distinctions and certain areas’ associations were more distinguishable than others.ConclusionsThese results illustrate that EHR utilization patterns may be consistent with the expectations of HCO employees. Our findings show that certain areas in the HCO are easier than others for employees to assess, which suggests that automated learning strategies may yield more accurate models of healthcare organizations than those based on the perspectives of a select few individuals.     

Digital video analysis of health professionals’ interactions with an electronic whiteboard: A longitudinal,naturalistic study of changes to user interactions

As hospital departments continue to introduce electronic whiteboards in real clinical settings a range of human factor issues have emerged and it has become clear that there is a need for improved methods for designing and testing these systems. In this study, we employed a longitudinal and naturalistic method in the usability evaluation of an electronic whiteboard system. The goal of the evaluation was to explore the extent to which usability issues experienced by users change as they gain more experience with the system. In addition, the paper explores the use of a new approach to collection and analysis of continuous digital video recordings of naturalistic “live” user interactions. The method developed and employed in the study included recording the users’ interactions with system during actual use using screen-capturing software and analyzing these recordings for usability issues. In this paper we describe and discuss both the method and the results of the evaluation. We found that the electronic whiteboard system contains system-related usability issues that did not change over time as the clinicians collectively gained more experience with the system. Furthermore, we also found user-related issues that seemed to change as the users gained more experience and we discuss the underlying reasons for these changes. We also found that the method used in the study has certain advantages over traditional usability evaluation methods, including the ability to collect analyze live user data over time. However, challenges and drawbacks to using the method (including the time taken for analysis and logistical issues in doing live recordings) should be considered before utilizing a similar approach. In conclusion we summarize our findings and call for an increased focus on longitudinal and naturalistic evaluations of health information systems and encourage others to apply and refine the method utilized in this study.     

Supporting information retrieval from electronic health records: A report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE)

ObjectiveThis paper describes the University of Michigan’s nine-year experience in developing and using a full-text search engine designed to facilitate information retrieval (IR) from narrative documents stored in electronic health records (EHRs). The system, called the Electronic Medical Record Search Engine (EMERSE), functions similar to Google but is equipped with special functionalities for handling challenges unique to retrieving information from medical text.Materials and methodsKey features that distinguish EMERSE from general-purpose search engines are discussed, with an emphasis on functions crucial to (1) improving medical IR performance and (2) assuring search quality and results consistency regardless of users’ medical background, stage of training, or level of technical expertise.ResultsSince its initial deployment, EMERSE has been enthusiastically embraced by clinicians, administrators, and clinical and translational researchers. To date, the system has been used in supporting more than 750 research projects yielding 80 peer-reviewed publications. In several evaluation studies, EMERSE demonstrated very high levels of sensitivity and specificity in addition to greatly improved chart review efficiency.DiscussionIncreased availability of electronic data in healthcare does not automatically warrant increased availability of information. The success of EMERSE at our institution illustrates that free-text EHR search engines can be a valuable tool to help practitioners and researchers retrieve information from EHRs more effectively and efficiently, enabling critical tasks such as patient case synthesis and research data abstraction.ConclusionEMERSE, available free of charge for academic use, represents a state-of-the-art medical IR tool with proven effectiveness and user acceptance.