Research and applications: Investigating the link between radiologists’ gaze,diagnostic decision,and image content

Objective

To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms.

Methods

Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eye-tracker. Image analysis was performed in mammographic regions that attracted radiologists’ attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored.

Results

By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers’ gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers’ diagnostic errors while confirming 97.3% of their correct diagnoses. The readers’ individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior.

Conclusions

There is clearly an interaction between radiologists’ gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.     

A phase II study on safety and efficacy of high-dose N-acetylcysteine in patients with cystic fibrosis

Objective

We conducted a single-centre, randomised, double-blinded, placebo-controlled phase II clinical study to test safety and efficacy of a 12-week therapy with low-dose (700 mg/daily) or high-dose (2800 mg/daily) of NAC.

Methods

Twenty-one patients (ΔF508 homo/heterozygous, FEV₁ > 40% pred.) were included in the study. After a 3-weeks placebo run-in phase, 11 patients received low-dose NAC, and 10 patients received high-dose NAC. Outcomes included safety and clinical parameters, inflammatory (total leukocyte numbers, cell differentials, TNF-α, IL-8) measures in induced sputum, and concentrations of extracellular glutathione in induced sputum and blood.

Results

High-dose NAC was a well-tolerated and safe medication. High-dose NAC did not alter clinical or inflammatory parameters. However, extracellular glutathione in induced sputum tended to increase on high-dose NAC.

Conclusions

High-dose NAC is a well-tolerated and safe medication for a prolonged therapy of patients with CF with a potential to increase extracellular glutathione in CF airways.     

Using machine learning for concept extraction on clinical documents from multiple data sources

Objective

Concept extraction is a process to identify phrases referring to concepts of interests in unstructured text. It is a critical component in automated text processing. We investigate the performance of machine learning taggers for clinical concept extraction, particularly the portability of taggers across documents from multiple data sources.

Methods

We used BioTagger-GM to train machine learning taggers, which we originally developed for the detection of gene/protein names in the biology domain. Trained taggers were evaluated using the annotated clinical documents made available in the 2010 i2b2/VA Challenge workshop, consisting of documents from four data sources.

Results

As expected, performance of a tagger trained on one data source degraded when evaluated on another source, but the degradation of the performance varied depending on data sources. A tagger trained on multiple data sources was robust, and it achieved an F score as high as 0.890 on one data source. The results also suggest that performance of machine learning taggers is likely to improve if more annotated documents are available for training.

Conclusion

Our study shows how the performance of machine learning taggers is degraded when they are ported across clinical documents from different sources. The portability of taggers can be enhanced by training on datasets from multiple sources. The study also shows that BioTagger-GM can be easily extended to detect clinical concept mentions with good performance.     

Comprehensive temporal information detection from clinical text: medical events,time, and TLINK identification

Background

Temporal information detection systems have been developed by the Mayo Clinic for the 2012 i2b2 Natural Language Processing Challenge.

Objective

To construct automated systems for EVENT/TIMEX3 extraction and temporal link (TLINK) identification from clinical text.

Materials and methods

The i2b2 organizers provided 190 annotated discharge summaries as the training set and 120 discharge summaries as the test set. Our Event system used a conditional random field classifier with a variety of features including lexical information, natural language elements, and medical ontology. The TIMEX3 system employed a rule-based method using regular expression pattern match and systematic reasoning to determine normalized values. The TLINK system employed both rule-based reasoning and machine learning. All three systems were built in an Apache Unstructured Information Management Architecture framework.

Results

Our TIMEX3 system performed the best (F-measure of 0.900, value accuracy 0.731) among the challenge teams. The Event system produced an F-measure of 0.870, and the TLINK system an F-measure of 0.537.

Conclusions

Our TIMEX3 system demonstrated good capability of regular expression rules to extract and normalize time information. Event and TLINK machine learning systems required well-defined feature sets to perform well. We could also leverage expert knowledge as part of the machine learning features to further improve TLINK identification performance.     

Life cycle costing as a decision making tool for technology acquisition in radio-diagnosis

Background

Life cycle costing analysis is an emerging conceptual tool to validate capital investment in healthcare.

Methods

A preliminary study was done to analyze the long-term cost impact of acquiring a new 3 T MRI system when compared to technological upgradation of the existing 1.5 T MRI system with a view to evolve a decision matrix for correct investment planning and technology management. Operating costing method was utilized to estimate cost per unit MRI scan, costing inputs were considered for the existing 1.5 T and the proposed 3 T machine. Cost for each expected year in the life span of both 1.5 T and 3 T MRI scan options were then discounted to its Net Present Value. Net Present Value thus calculated for both the alternative options of 1.5 T and 3 T MRI machine was charted along with various intangible but critical Figures of Merit (FOM) to create a decision matrix for capital investment planning.

Result

Considering all fixed and variable costs contributing towards assumed operation, unit cost per MRI procedure was found to be Rs. 4244.58 for the 1.5 T upgrade and Rs. 6059.37 for the new 3 T MRI machine. Life Cycle Cost Analysis of the proposed 1.5 T upgrade and new 3 T machine showed a Net Present Value of Rs. 42,148,587.80 and Rs. 27,587,842.38 respectively.

Conclusion

The utility of life cycle costing as a strategic decision making tool towards evaluating alternative options for capital investment planning in health care environment is reiterated.     

Pathologically confirmed breast cancer in Malawi: a descriptive study: Clinical profile of breast cancer

Background

Breast cancer is the most common female cancer in Africa, yet no published studies have investigated breast cancer in Malawi. Understanding the clinical profile of breast cancer is important to develop early diagnosis efforts.

Aim

To describe clinical and pathological characteristics of breast specimens from a pathology laboratory at a national teaching hospital.

Methods

Secondary analysis of pathology reports from July 2011 to September 2013.

Results

Among 85 breast cancer cases, 55% were < 50 years. Median tumor size was 4 cm and 49% were grade 3. Median symptom duration was eight months.

Conclusions

Malawian women with breast cancer commonly have long symptom durations prior to diagnosis, young age, and poorly differentiated tumors. Improved clinical and pathological characterization, including hormone receptor status, are urgently needed to better understand this disease in Malawi.     

Machine learning-based coreference resolution of concepts in clinical documents

Objective

Coreference resolution of concepts, although a very active area in the natural language processing community, has not yet been widely applied to clinical documents. Accordingly, the 2011 i2b2 competition focusing on this area is a timely and useful challenge. The objective of this research was to collate coreferent chains of concepts from a corpus of clinical documents. These concepts are in the categories of person, problems, treatments, and tests.

Design

A machine learning approach based on graphical models was employed to cluster coreferent concepts. Features selected were divided into domain independent and domain specific sets. Training was done with the i2b2 provided training set of 489 documents with 6949 chains. Testing was done on 322 documents.

Results

The learning engine, using the un-weighted average of three different measurement schemes, resulted in an F measure of 0.8423 where no domain specific features were included and 0.8483 where the feature set included both domain independent and domain specific features.

Conclusion

Our machine learning approach is a promising solution for recognizing coreferent concepts, which in turn is useful for practical applications such as the assembly of problem and medication lists from clinical documents.     

High accuracy information extraction of medication information from clinical notes: 2009 i2b2 medication extraction challenge

Objective

Medication information comprises a most valuable source of data in clinical records. This paper describes use of a cascade of machine learners that automatically extract medication information from clinical records.

Design

Authors developed a novel supervised learning model that incorporates two machine learning algorithms and several rule-based engines.

Measurements

Evaluation of each step included precision, recall and F-measure metrics. The final outputs of the system were scored using the i2b2 workshop evaluation metrics, including strict and relaxed matching with a gold standard.

Results

Evaluation results showed greater than 90% accuracy on five out of seven entities in the name entity recognition task, and an F-measure greater than 95% on the relationship classification task. The strict micro averaged F-measure for the system output achieved best submitted performance of the competition, at 85.65%.

Limitations

Clinical staff will only use practical processing systems if they have confidence in their reliability. Authors estimate that an acceptable accuracy for a such a working system should be approximately 95%. This leaves a significant performance gap of 5 to 10% from the current processing capabilities.

Conclusion

A multistage method with mixed computational strategies using a combination of rule-based classifiers and statistical classifiers seems to provide a near-optimal strategy for automated extraction of medication information from clinical records.Many of the potential benefits of the electronic medical record (EMR) rely significantly on our ability to automatically process the free-text content in the EMR. To understand the limitations and difficulties of exploiting the EMR we have designed an information extraction engine to identify medication events within patient discharge summaries, as specified by the i2b2 medication extraction shared task.     

A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction

Objective

The goal of this work was to evaluate machine learning methods, binary classification and sequence labeling, for medication–attribute linkage detection in two clinical corpora.

Data and methods

We double annotated 3000 clinical trial announcements (CTA) and 1655 clinical notes (CN) for medication named entities and their attributes. A binary support vector machine (SVM) classification method with parsimonious feature sets, and a conditional random fields (CRF)-based multi-layered sequence labeling (MLSL) model were proposed to identify the linkages between the entities and their corresponding attributes. We evaluated the system''s performance against the human-generated gold standard.

Results

The experiments showed that the two machine learning approaches performed statistically significantly better than the baseline rule-based approach. The binary SVM classification achieved 0.94 F-measure with individual tokens as features. The SVM model trained on a parsimonious feature set achieved 0.81 F-measure for CN and 0.87 for CTA. The CRF MLSL method achieved 0.80 F-measure on both corpora.

Discussion and conclusions

We compared the novel MLSL method with a binary classification and a rule-based method. The MLSL method performed statistically significantly better than the rule-based method. However, the SVM-based binary classification method was statistically significantly better than the MLSL method for both the CTA and CN corpora. Using parsimonious feature sets both the SVM-based binary classification and CRF-based MLSL methods achieved high performance in detecting medication name and attribute linkages in CTA and CN.     

Research and applications: Supervised machine learning and active learning in classification of radiology reports

Objective

This paper presents an automated system for classifying the results of imaging examinations (CT, MRI, positron emission tomography) into reportable and non-reportable cancer cases. This system is part of an industrial-strength processing pipeline built to extract content from radiology reports for use in the Victorian Cancer Registry.

Materials and methods

In addition to traditional supervised learning methods such as conditional random fields and support vector machines, active learning (AL) approaches were investigated to optimize training production and further improve classification performance. The project involved two pilot sites in Victoria, Australia (Lake Imaging (Ballarat) and Peter MacCallum Cancer Centre (Melbourne)) and, in collaboration with the NSW Central Registry, one pilot site at Westmead Hospital (Sydney).

Results

The reportability classifier performance achieved 98.25% sensitivity and 96.14% specificity on the cancer registry''s held-out test set. Up to 92% of training data needed for supervised machine learning can be saved by AL.

Discussion

AL is a promising method for optimizing the supervised training production used in classification of radiology reports. When an AL strategy is applied during the data selection process, the cost of manual classification can be reduced significantly.

Conclusions

The most important practical application of the reportability classifier is that it can dramatically reduce human effort in identifying relevant reports from the large imaging pool for further investigation of cancer. The classifier is built on a large real-world dataset and can achieve high performance in filtering relevant reports to support cancer registries.     

MITRE system for clinical assertion status classification

Objective

To describe a system for determining the assertion status of medical problems mentioned in clinical reports, which was entered in the 2010 i2b2/VA community evaluation ‘Challenges in natural language processing for clinical data’ for the task of classifying assertions associated with problem concepts extracted from patient records.

Materials and methods

A combination of machine learning (conditional random field and maximum entropy) and rule-based (pattern matching) techniques was used to detect negation, speculation, and hypothetical and conditional information, as well as information associated with persons other than the patient.

Results

The best submission obtained an overall micro-averaged F-score of 0.9343.

Conclusions

Using semantic attributes of concepts and information about document structure as features for statistical classification of assertions is a good way to leverage rule-based and statistical techniques. In this task, the choice of features may be more important than the choice of classifier algorithm.     

Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010

Objective

As clinical text mining continues to mature, its potential as an enabling technology for innovations in patient care and clinical research is becoming a reality. A critical part of that process is rigid benchmark testing of natural language processing methods on realistic clinical narrative. In this paper, the authors describe the design and performance of three state-of-the-art text-mining applications from the National Research Council of Canada on evaluations within the 2010 i2b2 challenge.

Design

The three systems perform three key steps in clinical information extraction: (1) extraction of medical problems, tests, and treatments, from discharge summaries and progress notes; (2) classification of assertions made on the medical problems; (3) classification of relations between medical concepts. Machine learning systems performed these tasks using large-dimensional bags of features, as derived from both the text itself and from external sources: UMLS, cTAKES, and Medline.

Measurements

Performance was measured per subtask, using micro-averaged F-scores, as calculated by comparing system annotations with ground-truth annotations on a test set.

Results

The systems ranked high among all submitted systems in the competition, with the following F-scores: concept extraction 0.8523 (ranked first); assertion detection 0.9362 (ranked first); relationship detection 0.7313 (ranked second).

Conclusion

For all tasks, we found that the introduction of a wide range of features was crucial to success. Importantly, our choice of machine learning algorithms allowed us to be versatile in our feature design, and to introduce a large number of features without overfitting and without encountering computing-resource bottlenecks.     

Research and applications: Predicting complications of percutaneous coronary intervention using a novel support vector method

Objective

To explore the feasibility of a novel approach using an augmented one-class learning algorithm to model in-laboratory complications of percutaneous coronary intervention (PCI).

Materials and methods

Data from the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) multicenter registry for the years 2007 and 2008 (n=41 016) were used to train models to predict 13 different in-laboratory PCI complications using a novel one-plus-class support vector machine (OP-SVM) algorithm. The performance of these models in terms of discrimination and calibration was compared to the performance of models trained using the following classification algorithms on BMC2 data from 2009 (n=20 289): logistic regression (LR), one-class support vector machine classification (OC-SVM), and two-class support vector machine classification (TC-SVM). For the OP-SVM and TC-SVM approaches, variants of the algorithms with cost-sensitive weighting were also considered.

Results

The OP-SVM algorithm and its cost-sensitive variant achieved the highest area under the receiver operating characteristic curve for the majority of the PCI complications studied (eight cases). Similar improvements were observed for the Hosmer–Lemeshow χ² value (seven cases) and the mean cross-entropy error (eight cases).

Conclusions

The OP-SVM algorithm based on an augmented one-class learning problem improved discrimination and calibration across different PCI complications relative to LR and traditional support vector machine classification. Such an approach may have value in a broader range of clinical domains.     

A rule based solution to co-reference resolution in clinical text

Objective

To build an effective co-reference resolution system tailored to the biomedical domain.

Methods

Experimental materials used in this study were provided by the 2011 i2b2 Natural Language Processing Challenge. The 2011 i2b2 challenge involves co-reference resolution in medical documents. Concept mentions have been annotated in clinical texts, and the mentions that co-refer in each document are linked by co-reference chains. Normally, there are two ways of constructing a system to automatically discoverco-referent links. One is to manually build rules forco-reference resolution; the other is to use machine learning systems to learn automatically from training datasets and then perform the resolution task on testing datasets.

Results

The existing co-reference resolution systems are able to find some of the co-referent links; our rule based system performs well, finding the majority of the co-referent links. Our system achieved 89.6% overall performance on multiple medical datasets.

Conclusions

Manually crafted rules based on observation of training data is a valid way to accomplish high performance in this co-reference resolution task for the critical biomedical domain.     

Twenty-seven years follow-up of a patient with congenital retinocephalofacial vascular malformation syndrome and additional congenital malformations (Bonnet-Dechaume-Blanc syndrome or Wyburn-Mason syndrome)

Purpose

Follow-up of vascular changes in a patient with congenital retinocephalofacial vascular malformation syndrome.

Methods

MRI and cerebral angiography.

Results

In a 36-year-old man, magnetic resonance im aging of the skull and cerebral angiography revealed left intracranial arteriovenous malformations. Follow-up observation of 27 years revealed no essential change of retinal and cerebral arteriovenous malformations. Additional congenital deficits in this patient were described.

Conclusion

Patients with retinal arteriovenous malformations should be early examined with neuroradiological methods.     

Diffusion weighted magnetic resonance imaging for acute stroke: practical and popular

Aim

To evaluate the feasibility and impact of diffusion weighted magnetic resonance imaging (DW MRI) as the first line neuroimaging of stroke at a district general hospital.

Methods

Prospective audit of all in‐patients admitted with clinically suspected acute stroke and referred for imaging over a consecutive 17 week period. The data collected included scan type, time from cerebral event to imaging request, and time from formal radiological request to neuroimaging. Clinicians'' (general physicians, neurologists, and radiologists) perceptions were assessed by a questionnaire.

Results

148 patients had neuroimaging for clinically suspected stroke during this period. Eighty one per cent of patients (120 of 148) had DW MRI as first line. Ninety two per cent of these patients had DW MRI within 24 hours of the formal radiological request. Twenty eight patients did not undergo DW MRI because lack of MRI safety, clinical state, unavailability because of maintenance service or lack of trained staff. Clinicians found the introduction of the DW MRI based service a significant improvement on computed tomography, especially for equivocal cases.

Conclusion

DW based MRI service is both feasible and sustainable in the setting of a district general hospital and most clinicians feel that this is a significant improvement to stroke services.     

Research and applications: Patient-level temporal aggregation for text-based asthma status ascertainment

Objective

To specify the problem of patient-level temporal aggregation from clinical text and introduce several probabilistic methods for addressing that problem. The patient-level perspective differs from the prevailing natural language processing (NLP) practice of evaluating at the term, event, sentence, document, or visit level.

Methods

We utilized an existing pediatric asthma cohort with manual annotations. After generating a basic feature set via standard clinical NLP methods, we introduce six methods of aggregating time-distributed features from the document level to the patient level. These aggregation methods are used to classify patients according to their asthma status in two hypothetical settings: retrospective epidemiology and clinical decision support.

Results

In both settings, solid patient classification performance was obtained with machine learning algorithms on a number of evidence aggregation methods, with Sum aggregation obtaining the highest F₁ score of 85.71% on the retrospective epidemiological setting, and a probability density function-based method obtaining the highest F₁ score of 74.63% on the clinical decision support setting. Multiple techniques also estimated the diagnosis date (index date) of asthma with promising accuracy.

Discussion

The clinical decision support setting is a more difficult problem. We rule out some aggregation methods rather than determining the best overall aggregation method, since our preliminary data set represented a practical setting in which manually annotated data were limited.

Conclusion

Results contrasted the strengths of several aggregation algorithms in different settings. Multiple approaches exhibited good patient classification performance, and also predicted the timing of estimates with reasonable accuracy.     

Coreference analysis in clinical notes: a multi-pass sieve with alternate anaphora resolution modules

Objective

This paper describes the coreference resolution system submitted by Mayo Clinic for the 2011 i2b2/VA/Cincinnati shared task Track 1C. The goal of the task was to construct a system that links the markables corresponding to the same entity.

Materials and methods

The task organizers provided progress notes and discharge summaries that were annotated with the markables of treatment, problem, test, person, and pronoun. We used a multi-pass sieve algorithm that applies deterministic rules in the order of preciseness and simultaneously gathers information about the entities in the documents. Our system, MedCoref, also uses a state-of-the-art machine learning framework as an alternative to the final, rule-based pronoun resolution sieve.

Results

The best system that uses a multi-pass sieve has an overall score of 0.836 (average of B³, MUC, Blanc, and CEAF F score) for the training set and 0.843 for the test set.

Discussion

A supervised machine learning system that typically uses a single function to find coreferents cannot accommodate irregularities encountered in data especially given the insufficient number of examples. On the other hand, a completely deterministic system could lead to a decrease in recall (sensitivity) when the rules are not exhaustive. The sieve-based framework allows one to combine reliable machine learning components with rules designed by experts.

Conclusion

Using relatively simple rules, part-of-speech information, and semantic type properties, an effective coreference resolution system could be designed. The source code of the system described is available at https://sourceforge.net/projects/ohnlp/files/MedCoref.     

Nipple-sparing Mastectomy in Breast Cancer: From an Oncologic Safety Perspective

Objective:

To evaluate the oncologic safety of nipple-sparing mastectomy (NSM) for breast cancer patients based on current literature.

Data Sources:

A comprehensive literature search of Medline, Embase databases was conducted for studies published through March 2014.

Study Selection:

Our search criteria included English-language studies that focused on NSM at nipple-areola complex (NAC) involvement, patient selection, and recurrence. Prophylaxis NSM, case series or reports that based on very small population were excluded. In the end, 42 studies concerning NSM and oncological safety were included into the review.

Results:

NSM is a surgical procedure that allows the preservation of the skin and NAC in breast cancer patients or in patients with prophylactic mastectomy. However, the oncologic safety and patient selection criteria associated with NSM are still under debate. The incidence of NAC involvement of breast cancer in recent studies ranges from 9.5% to 24.6%, which can be decreased through careful patient selection. Tumour-nipple distance, tumour size, lymph node involvement and molecular characteristics can be evaluated preoperatively by clinical examinations, imaging studies and biopsies to predict the risk of NAC involvement. Currently, there is no available standard protocol for surgical approaches to NSM or pathological examination of NSM specimens. The local recurrence (ranges from 0% to 24%) of NSM is not significantly higher than that of traditional mastectomy in selected patients based on long-term follow-up. The role of radiotherapy in NSM is still controversial and is not universally accepted.

Conclusions:

NSM appears to be oncologically safe following careful patient selection and assessment of margins.