Automated Detection of Critical Results in Radiology Reports

The goal of this study was to develop and validate text-mining algorithms to automatically identify radiology reports containing critical results including tension or increasing/new large pneumothorax, acute pulmonary embolism, acute cholecystitis, acute appendicitis, ectopic pregnancy, scrotal torsion, unexplained free intraperitoneal air, new or increasing intracranial hemorrhage, and malpositioned tubes and lines. The algorithms were developed using rule-based approaches and designed to search for common words and phrases in radiology reports that indicate critical results. Certain text-mining features were utilized such as wildcards, stemming, negation detection, proximity matching, and expanded searches with applicable synonyms. To further improve accuracy, the algorithms utilized modality and exam-specific queries, searched under the “Impression” field of the radiology report, and excluded reports with a low level of diagnostic certainty. Algorithm accuracy was determined using precision, recall, and F-measure using human review as the reference standard. The overall accuracy (F-measure) of the algorithms ranged from 81% to 100%, with a mean precision and recall of 96% and 91%, respectively. These algorithms can be applied to radiology report databases for quality assurance and accreditation, integrated with existing dashboards for display and monitoring, and ported to other institutions for their own use.     

Automated Detection of Radiology Reports that Document Non-routine Communication of Critical or Significant Results

The purpose of this investigation is to develop an automated method to accurately detect radiology reports that indicate non-routine communication of critical or significant results. Such a classification system would be valuable for performance monitoring and accreditation. Using a database of 2.3 million free-text radiology reports, a rule-based query algorithm was developed after analyzing hundreds of radiology reports that indicated communication of critical or significant results to a healthcare provider. This algorithm consisted of words and phrases used by radiologists to indicate such communications combined with specific handcrafted rules. This algorithm was iteratively refined and retested on hundreds of reports until the precision and recall did not significantly change between iterations. The algorithm was then validated on the entire database of 2.3 million reports, excluding those reports used during the testing and refinement process. Human review was used as the reference standard. The accuracy of this algorithm was determined using precision, recall, and F measure. Confidence intervals were calculated using the adjusted Wald method. The developed algorithm for detecting critical result communication has a precision of 97.0% (95% CI, 93.5–98.8%), recall 98.2% (95% CI, 93.4–100%), and F measure of 97.6% (ß = 1). Our query algorithm is accurate for identifying radiology reports that contain non-routine communication of critical or significant results. This algorithm can be applied to a radiology reports database for quality control purposes and help satisfy accreditation requirements.Key words: Critical results reporting, data mining, Joint Commission on Accreditation of Healthcare Organizations (JCAHO), natural language processing, online analytical processing (OLAP), quality assurance, quality control, radiology reporting     

Automated Extraction of BI-RADS Final Assessment Categories from Radiology Reports with Natural Language Processing

The objective of this study is to evaluate a natural language processing (NLP) algorithm that determines American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) final assessment categories from radiology reports. This HIPAA-compliant study was granted institutional review board approval with waiver of informed consent. This cross-sectional study involved 1,165 breast imaging reports in the electronic medical record (EMR) from a tertiary care academic breast imaging center from 2009. Reports included screening mammography, diagnostic mammography, breast ultrasound, combined diagnostic mammography and breast ultrasound, and breast magnetic resonance imaging studies. Over 220 reports were included from each study type. The recall (sensitivity) and precision (positive predictive value) of a NLP algorithm to collect BI-RADS final assessment categories stated in the report final text was evaluated against a manual human review standard reference. For all breast imaging reports, the NLP algorithm demonstrated a recall of 100.0 % (95 % confidence interval (CI), 99.7, 100.0 %) and a precision of 96.6 % (95 % CI, 95.4, 97.5 %) for correct identification of BI-RADS final assessment categories. The NLP algorithm demonstrated high recall and precision for extraction of BI-RADS final assessment categories from the free text of breast imaging reports. NLP may provide an accurate, scalable data extraction mechanism from reports within EMRs to create databases to track breast imaging performance measures and facilitate optimal breast cancer population management strategies.     

Non-Clinical Errors Using Voice Recognition Dictation Software for Radiology Reports: A Retrospective Audit

The purpose of this study is to ascertain the error rates of using a voice recognition (VR) dictation system. We compared our results with several other articles and discussed the pros and cons of using such a system. The study was performed at the Southern Health Department of Diagnostic Imaging, Melbourne, Victoria using the GE RIS with Powerscribe 3.5 VR system. Fifty random finalized reports from 19 radiologists obtained between June 2008 and November 2008 were scrutinized for errors in six categories namely, wrong word substitution, deletion, punctuation, other, and nonsense phrase. Reports were also divided into two categories: computer radiography (CR = plain film) and non-CR (ultrasound, computed tomography, magnetic resonance imaging, nuclear medicine, and angiographic examinations). Errors were divided into two categories, significant but not likely to alter patient management and very significant with the meaning of the report affected, thus potentially affecting patient management (nonsense phrase). Three hundred seventy-nine finalized CR reports and 631 non-CR finalized reports were examined. Eleven percent of the reports in the CR group had errors. Two percent of these reports contained nonsense phrases. Thirty-six percent of the reports in the non-CR group had errors and out of these, 5% contained nonsense phrases. VR dictation system is like a double-edged sword. Whilst there are many benefits, there are also many pitfalls. We hope that raising the awareness of the error rates will help in our efforts to reduce error rates and strike a balance between quality and speed of reports generated.     

An Audio/Video Reporting Workflow to Supplement Standardized Radiology Reports

Radiology studies are inherently visual and the information contained within is best conveyed by visual methodology. Advanced reporting software allows the incorporation of annotated key images into text reports, but such features may be less effective compared with in-person consultations. The use of web technology and screen capture software to create retrievable on-demand audio/visual reports has not yet been investigated. This approach may preempt potential curbside consultations while providing referring clinicians with a more engaged imaging service. In this work, we develop and evaluate a video reporting tool that utilizes modern screen capture software and web technology. We hypothesize that referring clinicians would find that recorded on-demand video reports add value to clinical practice, education, and that such technology would be welcome in future practice. A total of 45 case videos were prepared by radiologists for 14 attending and 15 trainee physicians from emergency and internal medicine specialties. Positive survey feedback from referring clinicians about the video reporting system was statistically significant in all areas measured, including video quality, clinical helpfulness, and willingness to use such technology in the future. Trainees unanimously found educational value in video reporting. These results suggest the potential for video technology to re-establish the radiologist’s role as a pivotal member of patient care and integral clinical educator. Future work is needed to streamline these methods in order to minimize work redundancy with traditional text reporting. Additionally, integration with an existing PACS and dictation system will be essential to ensuring ease of use and widespread adoption.     

Incorporation of a Formalized Emergency Radiology Curriculum to Facilitate Population of a MIRC-based Digital Teaching File

Teaching files are integral to radiological training. Digital Imaging and Communication in Medicine compatible digital radiological data and technological advances have made digital teaching files a desirable way to preserve and share representative and/or unusual cases for training purposes. The Medical Imaging Resource Community (MIRC) system developed by the Radiological Society of North America (RSNA) is a robust multi-platform digital teaching file implementation that is freely available. An emergency radiology training curriculum developed by the American Society of Emergency Radiology (ASER) was incorporated to determine if such an approach might facilitate the entry, maintenance, and cataloguing of interesting cases. The RSNA MIRC software was obtained from the main MIRC website and installed. A coding system was developed based on the outline form of the ASER curriculum. Weekly reports were generated tallying the number of cases in each category of the curriculum. Resident participation in the entry and maintenance of cases markedly increased after incorporation of the ASER curriculum. The coding schema facilitated progress assessment. Ultimately, 454 total cases were entered into the MIRC database, representing at least 42% of the subcategories within the ASER curriculum (161 out of 376). The incorporation of the ASER emergency radiology curriculum greatly facilitated the location, cataloguing, tracking, and maintenance of representative cases and served as an effective means by which to unify the efforts of the department to develop a comprehensive teaching resource within this subspecialty. This approach and format will be extended to other educational curricula in other radiological subspecialties.     

Content Analysis of Reporting Templates and Free-Text Radiology Reports

The Radiological Society of North America (RSNA) has developed a set of templates for structured reporting of radiology results. To measure how much of the content of conventional narrative (“free-text”) reports is covered by the concepts included in the RSNA reporting templates, we selected five reporting templates that represented a variety of imaging modalities and organ systems. From a sample of 8,275 consecutive, de-identified radiology reports from an academic medical center, we identified one corresponding imaging procedure code for each reporting template. The reports were annotated with RadLex and SNOMED CT terms using the BioPortal Annotator web service. The reporting templates we examined accounted for 17 to 49 % of the concepts that actually appeared in a sample of corresponding radiology reports. The findings suggest that the concepts that appear in the reporting templates occur frequently within free-text clinical reports; thus, the templates provide useful coverage of the “domain of discourse” in radiology reports. The techniques used in this study may be helpful to guide the development of reporting templates by identifying concepts that occur frequently in radiology reports, to evaluate the coverage of existing templates, and to establish global benchmarks for reporting templates.     

Automated Classification of Leukocytes in Blood Smear Images

Leukocyte formula counting is an important step of clinical blood analysis. A classification system is presented for problems of classification and count of leukocytes on blood smears images. The classification is based on a variant of the AdaBoost algorithm. The results of implementation of the algorithm are presented.     

Using Natural Language Processing of Free-Text Radiology Reports to Identify Type 1 Modic Endplate Changes

Electronic medical record (EMR) systems provide easy access to radiology reports and offer great potential to support quality improvement efforts and clinical research. Harnessing the full potential of the EMR requires scalable approaches such as natural language processing (NLP) to convert text into variables used for evaluation or analysis. Our goal was to determine the feasibility of using NLP to identify patients with Type 1 Modic endplate changes using clinical reports of magnetic resonance (MR) imaging examinations of the spine. Identifying patients with Type 1 Modic change who may be eligible for clinical trials is important as these findings may be important targets for intervention. Four annotators identified all reports that contained Type 1 Modic change, using N = 458 randomly selected lumbar spine MR reports. We then implemented a rule-based NLP algorithm in Java using regular expressions. The prevalence of Type 1 Modic change in the annotated dataset was 10%. Results were recall (sensitivity) 35/50 = 0.70 (95% confidence interval (C.I.) 0.52–0.82), specificity 404/408 = 0.99 (0.97–1.0), precision (positive predictive value) 35/39 = 0.90 (0.75–0.97), negative predictive value 404/419 = 0.96 (0.94–0.98), and F1-score 0.79 (0.43–1.0). Our evaluation shows the efficacy of rule-based NLP approach for identifying patients with Type 1 Modic change if the emphasis is on identifying only relevant cases with low concern regarding false negatives. As expected, our results show that specificity is higher than recall. This is due to the inherent difficulty of eliciting all possible keywords given the enormous variability of lumbar spine reporting, which decreases recall, while availability of good negation algorithms improves specificity.     

Creation and Implementation of Department-Wide Structured Reports: An Analysis of the Impact on Error Rate in Radiology Reports

The purpose of this study was to evaluate and compare textual error rates and subtypes in radiology reports before and after implementation of department-wide structured reports. Randomly selected radiology reports that were generated following the implementation of department-wide structured reports were evaluated for textual errors by two radiologists. For each report, the text was compared to the corresponding audio file. Errors in each report were tabulated and classified. Error rates were compared to results from a prior study performed prior to implementation of structured reports. Calculated error rates included the average number of errors per report, average number of nongrammatical errors per report, the percentage of reports with an error, and the percentage of reports with a nongrammatical error. Identical versions of voice-recognition software were used for both studies. A total of 644 radiology reports were randomly evaluated as part of this study. There was a statistically significant reduction in the percentage of reports with nongrammatical errors (33 to 26 %; p = 0.024). The likelihood of at least one missense omission error (omission errors that changed the meaning of a phrase or sentence) occurring in a report was significantly reduced from 3.5 to 1.2 % (p = 0.0175). A statistically significant reduction in the likelihood of at least one comission error (retained statements from a standardized report that contradict the dictated findings or impression) occurring in a report was also observed (3.9 to 0.8 %; p = 0.0007). Carefully constructed structured reports can help to reduce certain error types in radiology reports.     

Automatically Correlating Clinical Findings and Body Locations in Radiology Reports Using MedLEE

In this paper, we describe and evaluate a system that extracts clinical findings and body locations from radiology reports and correlates them. The system uses Medical Language Extraction and Encoding System (MedLEE) to map the reports’ free text to structured semantic representations of their content. A lightweight reasoning engine extracts the clinical findings and body locations from MedLEE’s semantic representation and correlates them. Our study is illustrative for research in which existing natural language processing software is embedded in a larger system. We manually created a standard reference based on a corpus of neuro and breast radiology reports. The standard reference was used to evaluate the precision and recall of the proposed system and its modules. Our results indicate that the precision of our system is considerably better than its recall (82.32–91.37% vs. 35.67–45.91%). We conducted an error analysis and discuss here the practical usability of the system given its recall and precision performance.     

Simplified Readability Metric Drives Improvement of Radiology Reports: an Experiment on Ultrasound Reports at a Pediatric Hospital

Highly complex medical documents, including ultrasound reports, are greatly mismatched with patient literacy levels. While improving radiology reports for readability is a longstanding concern, few articles objectively measure the effectiveness of physician training for readability improvement. We hypothesized that writing styles may be evaluated using an objective two-dimensional measure and writing training could improve the writing styles of radiologists. To test it, a simplified “grade vs. length” readability metric is developed based on results from factor analysis of ten readability metrics applied to more than 500,000 radiology reports. To test the short-term effectiveness of a writing workshop, we measured the writing style improvement before and after the training. Statistically significant writing style improvement occurred as a result of the training. Although the degree of improvement varied for different measures, it is evident that targeted training could provide potential benefits to improve readability due to our statistically significant results. The simplified grade vs. length metric enables future clinical decision support systems to quantitatively guide physicians to improve writing styles through writing workshops.     

An Efficient Approach for Automated Mass Segmentation and Classification in Mammograms

Breast cancer is becoming a leading death of women all over the world; clinical experiments demonstrate that early detection and accurate diagnosis can increase the potential of treatment. In order to improve the breast cancer diagnosis precision, this paper presents a novel automated segmentation and classification method for mammograms. We conduct the experiment on both DDSM database and MIAS database, firstly extract the region of interests (ROIs) with chain codes and using the rough set (RS) method to enhance the ROIs, secondly segment the mass region from the location ROIs with an improved vector field convolution (VFC) snake and following extract features from the mass region and its surroundings, and then establish features database with 32 dimensions; finally, these features are used as input to several classification techniques. In our work, the random forest is used and compared with support vector machine (SVM), genetic algorithm support vector machine (GA-SVM), particle swarm optimization support vector machine (PSO-SVM), and decision tree. The effectiveness of our method is evaluated by a comprehensive and objective evaluation system; also, Matthew’s correlation coefficient (MCC) indicator is used. Among the state-of-the-art classifiers, our method achieves the best performance with best accuracy of 97.73 %, and the MCC value reaches 0.8668 and 0.8652 in unique DDSM database and both two databases, respectively. Experimental results prove that the proposed method outperforms the other methods; it could consider applying in CAD systems to assist the physicians for breast cancer diagnosis.     

用于MRI脑组织分割的自动模糊连接方法

本研究提出了一种自动化的模糊连接（fuzzy connectedness，FC）方法，用于3维核磁共振（MRI）图像脑组织分割。方法的主要创新在于提出了FC方法中各项参数的自动指定方法，包括：利用灰质、白质各自的体素尺度（scale）值大小差异，自动估计组织的灰度概率密度函数；根据估计得到的组织灰度概率密度函数，自动指定种子点。从而避免了人工干预，保证了分割过程的自动化和可重复性。所提方法在IBSR（the Internet Brain Segmentation Repository）数据库所提供的MRI图像上进行了测试，并和同类研究进行了对比，分割精度优于同类研究。作为一种完全自动化的方法，该方法能够被广泛应用到3维可视化、放疗手术计划和医学数据库构造中。     

Automated Classification and Identification of Slow Wave Propagation Patterns in Gastric Dysrhythmia

The advent of high-resolution (HR) electrical mapping of slow wave activity has significantly improved the understanding of gastric slow wave activity in normal and dysrhythmic states. One of the current limitations of this technique is it generates a vast amount of data, making manual analysis a tedious task for research and clinical development. In this study we present new automated methods to classify, identify, and locate patterns of interest in gastric slow wave propagation. The classification method uses a similarity metric to classify slow wave propagations, while the identification algorithm uses the divergence and mean curvature of the slow wave propagation to identify and regionalize patterns of interest. The methods were applied to synthetic and experimental datasets and were also compared to manual analysis. The methods classified and identified patterns of slow wave propagation in less than 1 s, compared to manual analysis which took up to 40 min. The automated methods achieved 96% accuracy in classifying AT maps, and 95% accuracy in identifying the propagation pattern with a mean spatial error of 1.5 mm in comparison to manual methods. These new methods will facilitate the efficient translation of gastrointestinal HR mapping techniques to clinical practice.     

Classification of HLA-Matching for Retrospective Analysis of Unrelated Donor Transplantation: Revised Definitions to Predict Survival

The best unrelated donors (URD) for hematopoietic cell transplantation (HCT) are alleles matched at HLA-A, -B, -C, and DRB1. Earlier studies mostly used incomplete or lower resolution HLA typing for analysis of transplant outcome. To understand the impact of incomplete HLA characterization, we analyzed 14,797 URD HCT (1995-2006) using multivariable regression modeling adjusting for factors affecting survival. Of 21 matching cohorts, we identified 3 groups with significantly different outcomes. Well-matched cases had either no identified HLA mismatch and informative data at 4 loci or allele matching at HLA-A, -B, and -DRB1 (n = 7477, 50% of the population). Partially matched pairs had a defined, single-locus mismatch and/or missing HLA data (n = 4962, 34%). Mismatched cases had ≥2 allele or antigen mismatches (n = 2358, 16%). Multivariate adjusted 5-year survival estimates were: well-matched: 54.1 (95% confidence interval), 52.9-55.4), partially matched: 43.7 (42.3-45.2), and mismatched: 33.4 (32.5-36.5), P < .001. A better matched donor yielded 10%-11% better 5-year survival. Importantly, intermediate resolution -A, -B, and -DRB1 alleles matched “6/6 antigen matched” HCT had survival outcomes within the partially matched cohort. We suggest that these proposed HLA subgroupings be used when complete HLA typing is not available. This improved categorization of HLA matching status allows adjustment for donor-recipient HLA compatibility, and can standardize interpretations of prior URD HCT experience.     

Expanding the Scope of an Automated Radiology Recommendation-Tracking Engine: Initial Experiences and Lessons Learned

An automated radiology recommendation-tracking engine for incidental focal masses in the liver, pancreas, kidneys, and adrenal glands was launched within our institution in July 2013. For 2 years, the majority of CT, MR, and US examination reports generated within our health system were mined by the engine. However, the need to expand the system beyond the initial four organs was soon identified. In July 2015, the second phase of the system was implemented and expanded to include additional anatomic structures in the abdomen and pelvis, as well as to provide non-radiology and non-imaging options for follow-up. The most frequent organs with incidental findings, outside of the original four, included the ovaries and the endometrium, which also correlated to the most frequently ordered imaging follow-up study of pelvic ultrasound and non-imaging follow-up study of endometrial biopsies, respectively. The second phase expansion has demonstrated new venues for augmenting and improving radiologist roles in optimal communication and management of incidental findings.