Extracting important information from Chinese Operation Notes with natural language processing methods

Extracting information from unstructured clinical narratives is valuable for many clinical applications. Although natural Language Processing (NLP) methods have been profoundly studied in electronic medical records (EMR), few studies have explored NLP in extracting information from Chinese clinical narratives. In this study, we report the development and evaluation of extracting tumor-related information from operation notes of hepatic carcinomas which were written in Chinese. Using 86 operation notes manually annotated by physicians as the training set, we explored both rule-based and supervised machine-learning approaches. Evaluating on unseen 29 operation notes, our best approach yielded 69.6% in precision, 58.3% in recall and 63.5% F-score.     

Personal narratives as maps of the social ecosystem.

Research and theory on the development and function of autobiographical narratives are reviewed and synthesized in this discussion article. In our synthesis, these stories are viewed as summaries of long-term and recent historical experiences describing patterns of social contingencies in the narrator's relationships with a variety of people. The pattern, extending from past to present, is assumed to serve a map function for the narrator, guiding this observer's deliberate study of personal social interactions in the here and now. Clarity of the pattern is thought to depend on how well the narrative is structured along dimensions of coherence and richness, characteristics amenable to change through the constructive feedback of listeners. Ways in which clarity is enhanced through natural changes in narrative structure are highlighted as relevant processes for clinical work with adults, adolescents, and children.     

A simple algorithm for identifying negated findings and diseases in discharge summaries

Narrative reports in medical records contain a wealth of information that may augment structured data for managing patient information and predicting trends in diseases. Pertinent negatives are evident in text but are not usually indexed in structured databases. The objective of the study reported here was to test a simple algorithm for determining whether a finding or disease mentioned within narrative medical reports is present or absent. We developed a simple regular expression algorithm called NegEx that implements several phrases indicating negation, filters out sentences containing phrases that falsely appear to be negation phrases, and limits the scope of the negation phrases. We compared NegEx against a baseline algorithm that has a limited set of negation phrases and a simpler notion of scope. In a test of 1235 findings and diseases in 1000 sentences taken from discharge summaries indexed by physicians, NegEx had a specificity of 94.5% (versus 85.3% for the baseline), a positive predictive value of 84.5% (versus 68.4% for the baseline) while maintaining a reasonable sensitivity of 77.8% (versus 88.3% for the baseline). We conclude that with little implementation effort a simple regular expression algorithm for determining whether a finding or disease is absent can identify a large portion of the pertinent negatives from discharge summaries.     

Evaluation of electronic discharge summaries: a comparison of documentation in electronic and handwritten discharge summaries

BACKGROUND: Hospital discharge summaries have traditionally been paper-based (handwritten or dictated), and deficiencies have often been reported. On the increase is the utilisation of electronic summaries, which are considered of higher quality than paper-based summaries. However, comparisons between electronic and paper-based summaries regarding documentation deficiencies have rarely been made and there have been none in recent years. OBJECTIVES: (1) To study the hospital discharge summaries, which were either handwritten or electronic, of a population of inpatients, with regard to documentation of information required for ongoing care; and (2) to compare the electronic with the handwritten summaries concerning documentation of this information. METHODS: The discharge summaries of 245 inpatients were examined for documentation of the items: discharge date; additional diagnoses; summary of the patient's progress in hospital; investigations; discharge medications; and follow-up (instructions to the patient's general practitioner). One hundred and fifty-one (62%) discharge summaries were electronically created and 94 (38%) were handwritten. Odds ratios (ORs) with their confidence intervals (CI) were estimated to show strength of association between the electronic summary and documentation of individual study items. RESULTS: Across all items studied, the electronic summaries contained a higher number of errors and/or omissions than the handwritten ones (OR 1.74, 95% CI 1.26-2.39, p<0.05). Electronic summaries more commonly documented a summary of the patient's progress in hospital (OR 18.3, 95% CI 3.33-100, p<0.05) and less commonly recorded date of discharge and additional diagnoses (respective ORs 0.17 (95% CI 0.09-0.31, p<0.05) and 0.33 (95% CI 0.15-0.89, p<0.05). CONCLUSION: It is not necessarily the case that electronic discharge summaries are of higher quality than handwritten ones, but free text items such as summary of the patient's progress may less likely be omitted in electronic summaries. It is unknown what factors contributed to incompleteness in creating the electronic discharge summaries investigated in this study. Possible causes for deficiencies include: insufficient training; insufficient education of, and thus realisation by, doctors regarding the importance of accurate, complete discharge summaries; inadequate computer literacy; inadequate user interaction design, and insufficient integration into routine work processes. Research into these factors is recommended. This study suggests that not enough care is taken by doctors when creating discharge summaries, and that this is independent of the type of method used. The importance of the discharge summary as a chief means of transferring patient information from the hospital to the primary care provider needs to be strongly emphasised.     

Structured data entry for narrative data in a broad specialty: patient history and physical examination in pediatrics

Background  

Whereas an electronic medical record (EMR) system can partly address the limitations, of paper-based documentation, such as fragmentation of patient data, physical paper records missing and poor legibility, structured data entry (SDE, i.e. data entry based on selection of predefined medical concepts) is essential for uniformity of data, easier reporting, decision support, quality assessment, and patient-oriented clinical research. The aim of this project was to explore whether a previously developed generic (i.e. content independent) SDE application to support the structured documentation of narrative data (called OpenSDE) can be used to model data obtained at history taking and physical examination of a broad specialty.     

Assessment of commercial NLP engines for medication information extraction from dictated clinical notes

PURPOSE: We assessed the current state of commercial natural language processing (NLP) engines for their ability to extract medication information from textual clinical documents. METHODS: Two thousand de-identified discharge summaries and family practice notes were submitted to four commercial NLP engines with the request to extract all medication information. The four sets of returned results were combined to create a comparison standard which was validated against a manual, physician-derived gold standard created from a subset of 100 reports. Once validated, the individual vendor results for medication names, strengths, route, and frequency were compared against this automated standard with precision, recall, and F measures calculated. RESULTS: Compared with the manual, physician-derived gold standard, the automated standard was successful at accurately capturing medication names (F measure=93.2%), but performed less well with strength (85.3%) and route (80.3%), and relatively poorly with dosing frequency (48.3%). Moderate variability was seen in the strengths of the four vendors. The vendors performed better with the structured discharge summaries than with the clinic notes in an analysis comparing the two document types. CONCLUSION: Although automated extraction may serve as the foundation for a manual review process, it is not ready to automate medication lists without human intervention.     

Narratives of Somatizing and Non somatizing Patients in a Primary Care Setting

Somatizing patients, who comprise approximately 20 percent of the primary care population, often present physicians with recurrent but confusing combinations of symptoms without organic explanations. Illness narratives presented during initial medical encounters with primary care physicians were examined qualitatively to determine if the narrative structure, chronological development of symptoms and temporal frame differed between somatizing and non-somatizing patients. Following a structured interview to identify somatization tendency and co-morbidities of depression and post-traumatic stress disorder, 116 patients' encounters with primary care physicians were video-recorded and transcribed. Somatizers demonstrated a narrative structure that was similar to that of non-somatizing patients, but they used a thematic rather than a chronological development of symptoms and they did not convey a clear time frame. Somatizing patients with a co-morbid psychological condition focused on concrete physical sensations, were unable to provide contextual history or chronological organization, and did not develop a temporal frame. The narratives of somatizing and non-somatizing patients differed sufficiently to warrant further research for use as a clinical aid in the diagnosis of somatization.     

Assessing explicit error reporting in the narrative electronic medical record using keyword searching

BACKGROUND: Many types of medical errors occur in and outside of hospitals, some of which have very serious consequences and increase cost. Identifying errors is a critical step for managing and preventing them. In this study, we assessed the explicit reporting of medical errors in the electronic record. METHOD: We used five search terms "mistake," "error," "incorrect," "inadvertent," and "iatrogenic" to survey several sets of narrative reports including discharge summaries, sign-out notes, and outpatient notes from 1991 to 2000. We manually reviewed all the positive cases and identified them based on the reporting of physicians. RESULT: We identified 222 explicitly reported medical errors. The positive predictive value varied with different keywords. In general, the positive predictive value for each keyword was low, ranging from 3.4 to 24.4%. Therapeutic-related errors were the most common reported errors and these reported therapeutic-related errors were mainly medication errors. CONCLUSION: Keyword searches combined with manual review indicated some medical errors that were reported in medical records. It had a low sensitivity and a moderate positive predictive value, which varied by search term. Physicians were most likely to record errors in the Hospital Course and History of Present Illness sections of discharge summaries. The reported errors in medical records covered a broad range and were related to several types of care providers as well as non-health care professionals.     

NLP based congestive heart failure case finding: A prospective analysis on statewide electronic medical records

BackgroundIn order to proactively manage congestive heart failure (CHF) patients, an effective CHF case finding algorithm is required to process both structured and unstructured electronic medical records (EMR) to allow complementary and cost-efficient identification of CHF patients.Methods and resultsWe set to identify CHF cases from both EMR codified and natural language processing (NLP) found cases. Using narrative clinical notes from all Maine Health Information Exchange (HIE) patients, the NLP case finding algorithm was retrospectively (July 1, 2012–June 30, 2013) developed with a random subset of HIE associated facilities, and blind-tested with the remaining facilities. The NLP based method was integrated into a live HIE population exploration system and validated prospectively (July 1, 2013–June 30, 2014). Total of 18,295 codified CHF patients were included in Maine HIE. Among the 253,803 subjects without CHF codings, our case finding algorithm prospectively identified 2411 uncodified CHF cases. The positive predictive value (PPV) is 0.914, and 70.1% of these 2411 cases were found to be with CHF histories in the clinical notes.ConclusionsA CHF case finding algorithm was developed, tested and prospectively validated. The successful integration of the CHF case findings algorithm into the Maine HIE live system is expected to improve the Maine CHF care.     

Query-oriented evidence extraction to support evidence-based medicine practice

BackgroundEvidence-based medicine practice requires medical practitioners to rely on the best available evidence, in addition to their expertise, when making clinical decisions. The medical domain boasts a large amount of published medical research data, indexed in various medical databases such as MEDLINE. As the size of this data grows, practitioners increasingly face the problem of information overload, and past research has established the time-associated obstacles faced by evidence-based medicine practitioners. In this paper, we focus on the problem of automatic text summarisation to help practitioners quickly find query-focused information from relevant documents.MethodsWe utilise an annotated corpus that is specialised for the task of evidence-based summarisation of text. In contrast to past summarisation approaches, which mostly rely on surface level features to identify salient pieces of texts that form the summaries, our approach focuses on the use of corpus-based statistics, and domain-specific lexical knowledge for the identification of summary contents. We also apply a target-sentence-specific summarisation technique that reduces the problem of underfitting that persists in generic summarisation models.ResultsIn automatic evaluations run over a large number of annotated summaries, our extractive summarisation technique statistically outperforms various baseline and benchmark summarisation models with a percentile rank of 96.8%. A manual evaluation shows that our extractive summarisation approach is capable of selecting content with high recall and precision, and may thus be used to generate bottom-line answers to practitioners’ queries.ConclusionsOur research shows that the incorporation of specialised data and domain-specific knowledge can significantly improve text summarisation performance in the medical domain. Due to the vast amounts of medical text available, and the high growth of this form of data, we suspect that such summarisation techniques will address the time-related obstacles associated with evidence-based medicine.     

Patients' and physicians' understanding of health and biomedical concepts: relationship to the design of EMR systems

OBJECTIVE: The aim of this paper is to examine knowledge organization and reasoning strategies involved in physician-patient communication and to consider how these are affected by the use of computer tools, in particular, electronic medical record (EMR) systems. DESIGN: In the first part of the paper, we summarize results from a study in which patients were interviewed before their interactions with physicians and where physician-patient interactions were recorded and analyzed to evaluate patients' and physicians' understanding of the patient problem. We give a detailed presentation of one of such interaction, with characterizations of physician and patient models. In a second set of studies, the contents of both paper and EMRs were compared and in addition, physician-patient interactions (involving the use of EMR technology) were video recorded and analyzed to assess physicians' information gathering and knowledge organization for medical decision-making. RESULTS: Physicians explained the patient problems in terms of causal pathophysiological knowledge underlying the disease (disease model), whereas patients explained them in terms of narrative structures of illness (illness model). The data-driven nature of the traditional physician-patient interaction allows physicians to capture the temporal flow of events and to document key aspects of the patients' narratives. Use of electronic medical records was found to influence the way patient data were gathered, resulting in information loss and disruption of temporal sequence of events in assessing patient problem. CONCLUSIONS: The physician-patient interview allows physicians to capture crucial aspects of the patient's illness model, which are necessary for understanding the problem from the patients' perspective. Use of computer-based patient record technology may lead to a loss of this relevant information. As a consequence, designers of such systems should take into account information relevant to the patient comprehension of medical problems, which will influence their compliance.     

De-identification of clinical narratives through writing complexity measures

PurposeElectronic health records contain a substantial quantity of clinical narrative, which is increasingly reused for research purposes. To share data on a large scale and respect privacy, it is critical to remove patient identifiers. De-identification tools based on machine learning have been proposed; however, model training is usually based on either a random group of documents or a pre-existing document type designation (e.g., discharge summary). This work investigates if inherent features, such as the writing complexity, can identify document subsets to enhance de-identification performance.MethodsWe applied an unsupervised clustering method to group two corpora based on writing complexity measures: a collection of over 4500 documents of varying document types (e.g., discharge summaries, history and physical reports, and radiology reports) from Vanderbilt University Medical Center (VUMC) and the publicly available i2b2 corpus of 889 discharge summaries. We compare the performance (via recall, precision, and F-measure) of de-identification models trained on such clusters with models trained on documents grouped randomly or VUMC document type.ResultsFor the Vanderbilt dataset, it was observed that training and testing de-identification models on the same stylometric cluster (with the average F-measure of 0.917) tended to outperform models based on clusters of random documents (with an average F-measure of 0.881). It was further observed that increasing the size of a training subset sampled from a specific cluster could yield improved results (e.g., for subsets from a certain stylometric cluster, the F-measure raised from 0.743 to 0.841 when training size increased from 10 to 50 documents, and the F-measure reached 0.901 when the size of the training subset reached 200 documents). For the i2b2 dataset, training and testing on the same clusters based on complexity measures (average F-score 0.966) did not significantly surpass randomly selected clusters (average F-score 0.965).ConclusionsOur findings illustrate that, in environments consisting of a variety of clinical documentation, de-identification models trained on writing complexity measures are better than models trained on random groups and, in many instances, document types.     

The influence of information structuring and health literacy on recall and satisfaction in a simulated discharge communication

Objective

We investigated the effects of information structuring and its potential interaction with pre-existing medical knowledge on recall in a simulated discharge communication.

Child, parent, and parent-child emotion narratives: implications for developmental psychopathology

Studies using narratives with children and parents offer ways to study affective meaning-making processes that are central in many theories of developmental psychopathology. This paper reviews theory regarding affective meaning making, and argues that narratives are particularly suited to examine such processes. The review of narrative studies and methods is organized into three sections according to the focus on child, parent, and parent-child narratives. Within each focus three levels of analysis are considered: (a) narrative organization and coherence, (b) narrative content, and (c) the behavior/interactions of the narrator(s). The implications of this research for developmental psychopathology and clinical work are discussed with an emphasis on parent-child jointly constructed narratives as the meeting point of individual child and parent narratives.     

A qualitative study of the experiences of one group of African Americans in pursuit of a career in academic medicine

Recent reports demonstrate that medical school enrollment of minority students has continuously declined over the past several years and underrepresented minorities (URMs) continue to account for a disproportionately low percentage (less than 4%) of full-time academic faculty at medical schools in the United States. This article reports on a qualitative research project to examine the sociocultural experiences that influenced one group of minority physicians pursuing an academic medical career. Nine African American faculty, one resident, and one fellow from a Southern medical school of 574 full-time clinical and basic faculty completed 25 open-ended questions on a structured, qualitative interview plus background demographics. These nine faculty represented 82% (N = 11) of the total number of African American clinical and basic scientist faculty on campus at the end of the 1999 academic year. The narrative interviews describe key decision points, environmental and economic influences, and cultural experiences that affected faculty career choices and illustrate the real-life experiences of current minority faculty and scientists. These narratives contain significant messages for addressing policy on school campuses to improve the opportunities and likelihood of increasing the proportion of minority physicians and scientists.     

Temporal reasoning with medical data--a review with emphasis on medical natural language processing

Temporal information is crucial in electronic medical records and biomedical information systems. Processing temporal information in medical narrative data is a very challenging area. It lies at the intersection of temporal representation and reasoning (TRR) in artificial intelligence and medical natural language processing (MLP). Some fundamental concepts and important issues in relation to TRR have previously been discussed, mainly in the context of processing structured data in biomedical informatics; however, it is important that these concepts be re-examined in the context of processing narrative data using MLP. Theoretical and methodological TRR studies in biomedical informatics can be classified into three main categories: category 1 applies theories and models from temporal reasoning in AI; category 2 defines frameworks that meet needs from clinical applications; category 3 resolves issues such as temporal granularity and uncertainty. Currently, most MLP systems are not designed with a formal representation of time, and their ability to reason about temporal relations among medical events is limited. Previous work in processing time with clinical narrative data includes processing time in clinical reports, modeling textual temporal expressions in clinical databases, processing time in clinical guidelines, and building time standards for data exchange and integration. In addition to common problems in MLP, there are challenges specific to TRR in medical text, which occur at each level of linguistic structure and analysis. Despite advances in temporal reasoning in biomedical informatics, processing time in medical text deserves more attention. Besides the need for more research in temporal granularity, fuzzy time, temporal contradiction, intermittent events and uncertainty, broad areas for future research include enhancing functions of current MLP systems on processing temporal information, incorporating medical knowledge into temporal reasoning systems, resolving coreference, integrating narrative data with structured data and evaluating these systems.     

Quality of discharge summaries prepared by first year internal medicine residents

ABSTRACT: BACKGROUND: Patients are particularly susceptible to medical error during transitions from inpatient to outpatient care. We evaluated discharge summaries produced by incoming postgraduate year 1 (PGY-1) internal medicine residents for their completeness, accuracy, and relevance to family physicians. METHODS: Consecutive discharge summaries prepared by PGY-1 residents for patients discharged from internal medicine wards were retrospectively evaluated by two independent reviewers for presence and accuracy of essential domains described by the Joint Commission for Hospital Accreditation. Family physicians rated the relevance of a separate sample of discharge summaries on domains that family physicians deemed important in previous studies. RESULTS: Ninety discharge summaries were assessed for completeness and accuracy. Most items were completely reported with a given item missing in 5 % of summaries or fewer, with the exception of the reason for medication changes, which was missing in 15.9 % of summaries. Discharge medication lists, medication changes, and the reason for medication changes---when present---were inaccurate in 35.7 %, 29.5 %, and 37.7 % of summaries, respectively. Twenty-one family physicians reviewed 68 discharge summaries. Communication of follow-up plans for further investigations was the most frequently identified area for improvement with 27.7 % of summaries rated as insufficient. CONCLUSIONS: This study found that medication details were frequently omitted or inaccurate, and that family physicians identified lack of clarity about follow-up plans regarding further investigations and visits to other consultants as the areas requiring the most improvement. Our findings will aid in the development of educational interventions for residents.     

Development and evaluation of RapTAT: A machine learning system for concept mapping of phrases from medical narratives

Rapid, automated determination of the mapping of free text phrases to pre-defined concepts could assist in the annotation of clinical notes and increase the speed of natural language processing systems. The aim of this study was to design and evaluate a token-order-specific naïve Bayes-based machine learning system (RapTAT) to predict associations between phrases and concepts. Performance was assessed using a reference standard generated from 2860 VA discharge summaries containing 567,520 phrases that had been mapped to 12,056 distinct Systematized Nomenclature of Medicine – Clinical Terms (SNOMED CT) concepts by the MCVS natural language processing system. It was also assessed on the manually annotated, 2010 i2b2 challenge data. Performance was established with regard to precision, recall, and F-measure for each of the concepts within the VA documents using bootstrapping. Within that corpus, concepts identified by MCVS were broadly distributed throughout SNOMED CT, and the token-order-specific language model achieved better performance based on precision, recall, and F-measure (0.95 ± 0.15, 0.96 ± 0.16, and 0.95 ± 0.16, respectively; mean ± SD) than the bag-of-words based, naïve Bayes model (0.64 ± 0.45, 0.61 ± 0.46, and 0.60 ± 0.45, respectively) that has previously been used for concept mapping. Precision, recall, and F-measure on the i2b2 test set were 92.9%, 85.9%, and 89.2% respectively, using the token-order-specific model. RapTAT required just 7.2 ms to map all phrases within a single discharge summary, and mapping rate did not decrease as the number of processed documents increased. The high performance attained by the tool in terms of both accuracy and speed was encouraging, and the mapping rate should be sufficient to support near-real-time, interactive annotation of medical narratives. These results demonstrate the feasibility of rapidly and accurately mapping phrases to a wide range of medical concepts based on a token-order-specific naïve Bayes model and machine learning.