Automated Detection of Critical Results in Radiology Reports

The goal of this study was to develop and validate text-mining algorithms to automatically identify radiology reports containing critical results including tension or increasing/new large pneumothorax, acute pulmonary embolism, acute cholecystitis, acute appendicitis, ectopic pregnancy, scrotal torsion, unexplained free intraperitoneal air, new or increasing intracranial hemorrhage, and malpositioned tubes and lines. The algorithms were developed using rule-based approaches and designed to search for common words and phrases in radiology reports that indicate critical results. Certain text-mining features were utilized such as wildcards, stemming, negation detection, proximity matching, and expanded searches with applicable synonyms. To further improve accuracy, the algorithms utilized modality and exam-specific queries, searched under the “Impression” field of the radiology report, and excluded reports with a low level of diagnostic certainty. Algorithm accuracy was determined using precision, recall, and F-measure using human review as the reference standard. The overall accuracy (F-measure) of the algorithms ranged from 81% to 100%, with a mean precision and recall of 96% and 91%, respectively. These algorithms can be applied to radiology report databases for quality assurance and accreditation, integrated with existing dashboards for display and monitoring, and ported to other institutions for their own use.     

Clinical Concept-Based Radiology Reports Classification Pipeline for Lung Carcinoma

Rising incidence and mortality of cancer have led to an incremental amount of research in the field. To learn from preexisting data, it has become important to capture maximum information related to disease type, stage, treatment, and outcomes. Medical imaging reports are rich in this kind of information but are only present as free text. The extraction of information from such unstructured text reports is labor-intensive. The use of Natural Language Processing (NLP) tools to extract information from radiology reports can make it less time-consuming as well as more effective. In this study, we have developed and compared different models for the classification of lung carcinoma reports using clinical concepts. This study was approved by the institutional ethics committee as a retrospective study with a waiver of informed consent. A clinical concept-based classification pipeline for lung carcinoma radiology reports was developed using rule-based as well as machine learning models and compared. The machine learning models used were XGBoost and two more deep learning model architectures with bidirectional long short-term neural networks. A corpus consisting of 1700 radiology reports including computed tomography (CT) and positron emission tomography/computed tomography (PET/CT) reports were used for development and testing. Five hundred one radiology reports from MIMIC-III Clinical Database version 1.4 was used for external validation. The pipeline achieved an overall F1 score of 0.94 on the internal set and 0.74 on external validation with the rule-based algorithm using expert input giving the best performance. Among the machine learning models, the Bi-LSTM_dropout model performed better than the ML model using XGBoost and the Bi-LSTM_simple model on internal set, whereas on external validation, the Bi-LSTM_simple model performed relatively better than other 2. This pipeline can be used for clinical concept-based classification of radiology reports related to lung carcinoma from a huge corpus and also for automated annotation of these reports.

     

Automated Detection of Radiology Reports that Document Non-routine Communication of Critical or Significant Results

The purpose of this investigation is to develop an automated method to accurately detect radiology reports that indicate non-routine communication of critical or significant results. Such a classification system would be valuable for performance monitoring and accreditation. Using a database of 2.3 million free-text radiology reports, a rule-based query algorithm was developed after analyzing hundreds of radiology reports that indicated communication of critical or significant results to a healthcare provider. This algorithm consisted of words and phrases used by radiologists to indicate such communications combined with specific handcrafted rules. This algorithm was iteratively refined and retested on hundreds of reports until the precision and recall did not significantly change between iterations. The algorithm was then validated on the entire database of 2.3 million reports, excluding those reports used during the testing and refinement process. Human review was used as the reference standard. The accuracy of this algorithm was determined using precision, recall, and F measure. Confidence intervals were calculated using the adjusted Wald method. The developed algorithm for detecting critical result communication has a precision of 97.0% (95% CI, 93.5–98.8%), recall 98.2% (95% CI, 93.4–100%), and F measure of 97.6% (ß = 1). Our query algorithm is accurate for identifying radiology reports that contain non-routine communication of critical or significant results. This algorithm can be applied to a radiology reports database for quality control purposes and help satisfy accreditation requirements.Key words: Critical results reporting, data mining, Joint Commission on Accreditation of Healthcare Organizations (JCAHO), natural language processing, online analytical processing (OLAP), quality assurance, quality control, radiology reporting     

Automated Extraction of BI-RADS Final Assessment Categories from Radiology Reports with Natural Language Processing

The objective of this study is to evaluate a natural language processing (NLP) algorithm that determines American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) final assessment categories from radiology reports. This HIPAA-compliant study was granted institutional review board approval with waiver of informed consent. This cross-sectional study involved 1,165 breast imaging reports in the electronic medical record (EMR) from a tertiary care academic breast imaging center from 2009. Reports included screening mammography, diagnostic mammography, breast ultrasound, combined diagnostic mammography and breast ultrasound, and breast magnetic resonance imaging studies. Over 220 reports were included from each study type. The recall (sensitivity) and precision (positive predictive value) of a NLP algorithm to collect BI-RADS final assessment categories stated in the report final text was evaluated against a manual human review standard reference. For all breast imaging reports, the NLP algorithm demonstrated a recall of 100.0 % (95 % confidence interval (CI), 99.7, 100.0 %) and a precision of 96.6 % (95 % CI, 95.4, 97.5 %) for correct identification of BI-RADS final assessment categories. The NLP algorithm demonstrated high recall and precision for extraction of BI-RADS final assessment categories from the free text of breast imaging reports. NLP may provide an accurate, scalable data extraction mechanism from reports within EMRs to create databases to track breast imaging performance measures and facilitate optimal breast cancer population management strategies.     

Non-Clinical Errors Using Voice Recognition Dictation Software for Radiology Reports: A Retrospective Audit

The purpose of this study is to ascertain the error rates of using a voice recognition (VR) dictation system. We compared our results with several other articles and discussed the pros and cons of using such a system. The study was performed at the Southern Health Department of Diagnostic Imaging, Melbourne, Victoria using the GE RIS with Powerscribe 3.5 VR system. Fifty random finalized reports from 19 radiologists obtained between June 2008 and November 2008 were scrutinized for errors in six categories namely, wrong word substitution, deletion, punctuation, other, and nonsense phrase. Reports were also divided into two categories: computer radiography (CR = plain film) and non-CR (ultrasound, computed tomography, magnetic resonance imaging, nuclear medicine, and angiographic examinations). Errors were divided into two categories, significant but not likely to alter patient management and very significant with the meaning of the report affected, thus potentially affecting patient management (nonsense phrase). Three hundred seventy-nine finalized CR reports and 631 non-CR finalized reports were examined. Eleven percent of the reports in the CR group had errors. Two percent of these reports contained nonsense phrases. Thirty-six percent of the reports in the non-CR group had errors and out of these, 5% contained nonsense phrases. VR dictation system is like a double-edged sword. Whilst there are many benefits, there are also many pitfalls. We hope that raising the awareness of the error rates will help in our efforts to reduce error rates and strike a balance between quality and speed of reports generated.     

An Audio/Video Reporting Workflow to Supplement Standardized Radiology Reports

Radiology studies are inherently visual and the information contained within is best conveyed by visual methodology. Advanced reporting software allows the incorporation of annotated key images into text reports, but such features may be less effective compared with in-person consultations. The use of web technology and screen capture software to create retrievable on-demand audio/visual reports has not yet been investigated. This approach may preempt potential curbside consultations while providing referring clinicians with a more engaged imaging service. In this work, we develop and evaluate a video reporting tool that utilizes modern screen capture software and web technology. We hypothesize that referring clinicians would find that recorded on-demand video reports add value to clinical practice, education, and that such technology would be welcome in future practice. A total of 45 case videos were prepared by radiologists for 14 attending and 15 trainee physicians from emergency and internal medicine specialties. Positive survey feedback from referring clinicians about the video reporting system was statistically significant in all areas measured, including video quality, clinical helpfulness, and willingness to use such technology in the future. Trainees unanimously found educational value in video reporting. These results suggest the potential for video technology to re-establish the radiologist’s role as a pivotal member of patient care and integral clinical educator. Future work is needed to streamline these methods in order to minimize work redundancy with traditional text reporting. Additionally, integration with an existing PACS and dictation system will be essential to ensuring ease of use and widespread adoption.     

抗感染药品严重不良反应报告回顾性分析

目的 分析抗感染药品严重不良反应的规律以及特点。方法 随机选取2016年3月~2018年3月青海省内上报的200例抗感染药品严重不良反应病例进行回顾性分析,分析患者的年龄、性别、怀疑感染药品种类、不良反应累及器官/系统以及临床特征等。结果 50~59岁的人群严重不良反应发生率较高,高于其他年龄段人群,差异具有统计学意义（P＜0.05）;喹诺酮类出现严重不良反应的概率最高,发生率高于其他抗感染药品,差异具有统计学意义（P＜0.05）;全身严重不良反应发生率高于其他器官/系统,差异具有统计学意义（P＜0.05）;消化系统损害表现为肝功能异常,转氨酶水平升高等;泌尿系统损害表现为肾功能异常、肾炎等;皮肤损害表现为多行性红斑、皮疹等;全身反应,表现为真菌感染、曲霉菌感染、过敏反应等。结论 抗感染药品导致的严重不良反应会累及患者多个器官和系统,应当引起临床医护人员的高度重视,提高对药品不良反应的警惕性。     

Incorporation of a Formalized Emergency Radiology Curriculum to Facilitate Population of a MIRC-based Digital Teaching File

Teaching files are integral to radiological training. Digital Imaging and Communication in Medicine compatible digital radiological data and technological advances have made digital teaching files a desirable way to preserve and share representative and/or unusual cases for training purposes. The Medical Imaging Resource Community (MIRC) system developed by the Radiological Society of North America (RSNA) is a robust multi-platform digital teaching file implementation that is freely available. An emergency radiology training curriculum developed by the American Society of Emergency Radiology (ASER) was incorporated to determine if such an approach might facilitate the entry, maintenance, and cataloguing of interesting cases. The RSNA MIRC software was obtained from the main MIRC website and installed. A coding system was developed based on the outline form of the ASER curriculum. Weekly reports were generated tallying the number of cases in each category of the curriculum. Resident participation in the entry and maintenance of cases markedly increased after incorporation of the ASER curriculum. The coding schema facilitated progress assessment. Ultimately, 454 total cases were entered into the MIRC database, representing at least 42% of the subcategories within the ASER curriculum (161 out of 376). The incorporation of the ASER emergency radiology curriculum greatly facilitated the location, cataloguing, tracking, and maintenance of representative cases and served as an effective means by which to unify the efforts of the department to develop a comprehensive teaching resource within this subspecialty. This approach and format will be extended to other educational curricula in other radiological subspecialties.     

Content Analysis of Reporting Templates and Free-Text Radiology Reports

The Radiological Society of North America (RSNA) has developed a set of templates for structured reporting of radiology results. To measure how much of the content of conventional narrative (“free-text”) reports is covered by the concepts included in the RSNA reporting templates, we selected five reporting templates that represented a variety of imaging modalities and organ systems. From a sample of 8,275 consecutive, de-identified radiology reports from an academic medical center, we identified one corresponding imaging procedure code for each reporting template. The reports were annotated with RadLex and SNOMED CT terms using the BioPortal Annotator web service. The reporting templates we examined accounted for 17 to 49 % of the concepts that actually appeared in a sample of corresponding radiology reports. The findings suggest that the concepts that appear in the reporting templates occur frequently within free-text clinical reports; thus, the templates provide useful coverage of the “domain of discourse” in radiology reports. The techniques used in this study may be helpful to guide the development of reporting templates by identifying concepts that occur frequently in radiology reports, to evaluate the coverage of existing templates, and to establish global benchmarks for reporting templates.     

Integrating Natural Language Processing and Machine Learning Algorithms to Categorize Oncologic Response in Radiology Reports

Journal of Digital Imaging - A significant volume of medical data remains unstructured. Natural language processing (NLP) and machine learning (ML) techniques have shown to successfully extract...     

Automated Classification of Leukocytes in Blood Smear Images

Leukocyte formula counting is an important step of clinical blood analysis. A classification system is presented for problems of classification and count of leukocytes on blood smears images. The classification is based on a variant of the AdaBoost algorithm. The results of implementation of the algorithm are presented.     

Using Natural Language Processing of Free-Text Radiology Reports to Identify Type 1 Modic Endplate Changes

Electronic medical record (EMR) systems provide easy access to radiology reports and offer great potential to support quality improvement efforts and clinical research. Harnessing the full potential of the EMR requires scalable approaches such as natural language processing (NLP) to convert text into variables used for evaluation or analysis. Our goal was to determine the feasibility of using NLP to identify patients with Type 1 Modic endplate changes using clinical reports of magnetic resonance (MR) imaging examinations of the spine. Identifying patients with Type 1 Modic change who may be eligible for clinical trials is important as these findings may be important targets for intervention. Four annotators identified all reports that contained Type 1 Modic change, using N = 458 randomly selected lumbar spine MR reports. We then implemented a rule-based NLP algorithm in Java using regular expressions. The prevalence of Type 1 Modic change in the annotated dataset was 10%. Results were recall (sensitivity) 35/50 = 0.70 (95% confidence interval (C.I.) 0.52–0.82), specificity 404/408 = 0.99 (0.97–1.0), precision (positive predictive value) 35/39 = 0.90 (0.75–0.97), negative predictive value 404/419 = 0.96 (0.94–0.98), and F1-score 0.79 (0.43–1.0). Our evaluation shows the efficacy of rule-based NLP approach for identifying patients with Type 1 Modic change if the emphasis is on identifying only relevant cases with low concern regarding false negatives. As expected, our results show that specificity is higher than recall. This is due to the inherent difficulty of eliciting all possible keywords given the enormous variability of lumbar spine reporting, which decreases recall, while availability of good negation algorithms improves specificity.     

Creation and Implementation of Department-Wide Structured Reports: An Analysis of the Impact on Error Rate in Radiology Reports

The purpose of this study was to evaluate and compare textual error rates and subtypes in radiology reports before and after implementation of department-wide structured reports. Randomly selected radiology reports that were generated following the implementation of department-wide structured reports were evaluated for textual errors by two radiologists. For each report, the text was compared to the corresponding audio file. Errors in each report were tabulated and classified. Error rates were compared to results from a prior study performed prior to implementation of structured reports. Calculated error rates included the average number of errors per report, average number of nongrammatical errors per report, the percentage of reports with an error, and the percentage of reports with a nongrammatical error. Identical versions of voice-recognition software were used for both studies. A total of 644 radiology reports were randomly evaluated as part of this study. There was a statistically significant reduction in the percentage of reports with nongrammatical errors (33 to 26 %; p = 0.024). The likelihood of at least one missense omission error (omission errors that changed the meaning of a phrase or sentence) occurring in a report was significantly reduced from 3.5 to 1.2 % (p = 0.0175). A statistically significant reduction in the likelihood of at least one comission error (retained statements from a standardized report that contradict the dictated findings or impression) occurring in a report was also observed (3.9 to 0.8 %; p = 0.0007). Carefully constructed structured reports can help to reduce certain error types in radiology reports.     

Automatically Correlating Clinical Findings and Body Locations in Radiology Reports Using MedLEE

In this paper, we describe and evaluate a system that extracts clinical findings and body locations from radiology reports and correlates them. The system uses Medical Language Extraction and Encoding System (MedLEE) to map the reports’ free text to structured semantic representations of their content. A lightweight reasoning engine extracts the clinical findings and body locations from MedLEE’s semantic representation and correlates them. Our study is illustrative for research in which existing natural language processing software is embedded in a larger system. We manually created a standard reference based on a corpus of neuro and breast radiology reports. The standard reference was used to evaluate the precision and recall of the proposed system and its modules. Our results indicate that the precision of our system is considerably better than its recall (82.32–91.37% vs. 35.67–45.91%). We conducted an error analysis and discuss here the practical usability of the system given its recall and precision performance.     

Application of Deep Learning in Generating Structured Radiology Reports: A Transformer-Based Technique

Journal of Digital Imaging - Since radiology reports needed for clinical practice and research are written and stored in free-text narrations, extraction of relative information for further...     

Simplified Readability Metric Drives Improvement of Radiology Reports: an Experiment on Ultrasound Reports at a Pediatric Hospital

Highly complex medical documents, including ultrasound reports, are greatly mismatched with patient literacy levels. While improving radiology reports for readability is a longstanding concern, few articles objectively measure the effectiveness of physician training for readability improvement. We hypothesized that writing styles may be evaluated using an objective two-dimensional measure and writing training could improve the writing styles of radiologists. To test it, a simplified “grade vs. length” readability metric is developed based on results from factor analysis of ten readability metrics applied to more than 500,000 radiology reports. To test the short-term effectiveness of a writing workshop, we measured the writing style improvement before and after the training. Statistically significant writing style improvement occurred as a result of the training. Although the degree of improvement varied for different measures, it is evident that targeted training could provide potential benefits to improve readability due to our statistically significant results. The simplified grade vs. length metric enables future clinical decision support systems to quantitatively guide physicians to improve writing styles through writing workshops.     

Improved Fine-Tuning of In-Domain Transformer Model for Inferring COVID-19 Presence in Multi-Institutional Radiology Reports

Journal of Digital Imaging - Building a document-level classifier for COVID-19 on radiology reports could help assist providers in their daily clinical routine, as well as create large numbers of...