2010 i2b2/VA challenge on concepts,assertions, and relations in clinical text

The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records presented three tasks: a concept extraction task focused on the extraction of medical concepts from patient reports; an assertion classification task focused on assigning assertion types for medical problem concepts; and a relation classification task focused on assigning relation types that hold between medical problems, tests, and treatments. i2b2 and the VA provided an annotated reference standard corpus for the three tasks. Using this reference standard, 22 systems were developed for concept extraction, 21 for assertion classification, and 16 for relation classification.These systems showed that machine learning approaches could be augmented with rule-based systems to determine concepts, assertions, and relations. Depending on the task, the rule-based systems can either provide input for machine learning or post-process the output of machine learning. Ensembles of classifiers, information from unlabeled data, and external knowledge sources can help when the training data are inadequate.     

The application of naive Bayes model averaging to predict Alzheimer's disease from genome-wide data

Objective

Predicting patient outcomes from genome-wide measurements holds significant promise for improving clinical care. The large number of measurements (eg, single nucleotide polymorphisms (SNPs)), however, makes this task computationally challenging. This paper evaluates the performance of an algorithm that predicts patient outcomes from genome-wide data by efficiently model averaging over an exponential number of naive Bayes (NB) models.

Design

This model-averaged naive Bayes (MANB) method was applied to predict late onset Alzheimer''s disease in 1411 individuals who each had 312 318 SNP measurements available as genome-wide predictive features. Its performance was compared to that of a naive Bayes algorithm without feature selection (NB) and with feature selection (FSNB).

Measurement

Performance of each algorithm was measured in terms of area under the ROC curve (AUC), calibration, and run time.

Results

The training time of MANB (16.1 s) was fast like NB (15.6 s), while FSNB (1684.2 s) was considerably slower. Each of the three algorithms required less than 0.1 s to predict the outcome of a test case. MANB had an AUC of 0.72, which is significantly better than the AUC of 0.59 by NB (p<0.00001), but not significantly different from the AUC of 0.71 by FSNB. MANB was better calibrated than NB, and FSNB was even better in calibration. A limitation was that only one dataset and two comparison algorithms were included in this study.

Conclusion

MANB performed comparatively well in predicting a clinical outcome from a high-dimensional genome-wide dataset. These results provide support for including MANB in the methods used to predict outcomes from large, genome-wide datasets.     

Automation bias: a systematic review of frequency,effect mediators,and mitigators

Automation bias (AB)—the tendency to over-rely on automation—has been studied in various academic fields. Clinical decision support systems (CDSS) aim to benefit the clinical decision-making process. Although most research shows overall improved performance with use, there is often a failure to recognize the new errors that CDSS can introduce. With a focus on healthcare, a systematic review of the literature from a variety of research fields has been carried out, assessing the frequency and severity of AB, the effect mediators, and interventions potentially mitigating this effect. This is discussed alongside automation-induced complacency, or insufficient monitoring of automation output. A mix of subject specific and freetext terms around the themes of automation, human–automation interaction, and task performance and error were used to search article databases. Of 13 821 retrieved papers, 74 met the inclusion criteria. User factors such as cognitive style, decision support systems (DSS), and task specific experience mediated AB, as did attitudinal driving factors such as trust and confidence. Environmental mediators included workload, task complexity, and time constraint, which pressurized cognitive resources. Mitigators of AB included implementation factors such as training and emphasizing user accountability, and DSS design factors such as the position of advice on the screen, updated confidence levels attached to DSS output, and the provision of information versus recommendation. By uncovering the mechanisms by which AB operates, this review aims to help optimize the clinical decision-making process for CDSS developers and healthcare practitioners.     

Targeted screening for pediatric conditions with the CHICA system

Objective

The Child Health Improvement through Computer Automation (CHICA) system is a decision-support and electronic-medical-record system for pediatric health maintenance and disease management. The purpose of this study was to explore CHICA''s ability to screen patients for disorders that have validated screening criteria—specifically tuberculosis (TB) and iron-deficiency anemia.

Design

Children between 0 and 11 years were randomized by the CHICA system. In the intervention group, parents were asked about TB and iron-deficiency risk, and physicians received a tailored prompt. In the control group, no screens were performed, and the physician received a generic prompt about these disorders.

Results

1123 participants were randomized to the control group and 1116 participants to the intervention group. Significantly more people reported positive risk factors for iron-deficiency anemia in the intervention group (17.5% vs 3.1%, OR 6.6, 95% CI 4.5 to 9.5). In general, far fewer parents reported risk factors for TB than for iron-deficiency anemia. Again, there were significantly higher detection rates of positive risk factors in the intervention group (1.8% vs 0.8%, OR 2.3, 95% CI 1.0 to 5.0).

Limitations

It is possible that there may be more positive screens without improving outcomes. However, the guidelines are based on studies that have evaluated the questions the authors used as sensitive and specific, and there is no reason to believe that parents misunderstood them.

Conclusions

Many screening tests are risk-based, not universal, leaving physicians to determine who should have a further workup. This can be a time-consuming process. The authors demonstrated that the CHICA system performs well in assessing risk automatically for TB and iron-deficiency anemia.     

Self-reported fever and measured temperature in emergency department records used for syndromic surveillance

Many public health agencies monitor population health using syndromic surveillance, generally employing information from emergency department (ED) visit records. When combined with other information, objective evidence of fever may enhance the accuracy with which surveillance systems detect syndromes of interest, such as influenza-like illness. This study found that patient chief complaint of self-reported fever was more readily available in ED records than measured temperature and that the majority of patients with an elevated temperature recorded also self-reported fever. Due to its currently limited availability, we conclude that measured temperature is likely to add little value to self-reported fever in syndromic surveillance for febrile illness using ED records.