Evaluating computer-aided detection algorithms

Computer-aided detection (CAD) has been attracting extensive research interest during the last two decades. It is recognized that the full potential of CAD can only be realized by improving the performance and robustness of CAD algorithms and this requires good evaluation methodology that would permit CAD designers to optimize their algorithms. Free-response receiver operating characteristic (FROC) curves are widely used to assess CAD performance, however, evaluation rarely proceeds beyond determination of lesion localization fraction (sensitivity) at an arbitrarily selected value of nonlesion localizations (false marks) per image. This work describes a FROC curve fitting procedure that uses a recent model of visual search that serves as a framework for the free-response task. A maximum likelihood procedure for estimating the parameters of the model from free-response data and fitting CAD generated FROC curves was implemented. Procedures were implemented to estimate two figures of merit and associated statistics such as 95% confidence intervals and goodness of fit. One of the figures of merit does not require the arbitrary specification of an operating point at which to evaluate CAD performance. For comparison a related method termed initial detection and candidate analysis was also implemented that is applicable when all suspicious regions are reported. The two methods were tested on seven mammography CAD data sets and both yielded good to excellent fits. The search model approach has the advantage that it can potentially be applied to radiologist generated free-response data where not all suspicious regions are reported, only the ones that are deemed sufficiently suspicious to warrant clinical follow-up. This work represents the first practical application of the search model to an important evaluation problem in diagnostic radiology. Software based on this work is expected to benefit CAD developers working in diverse areas of medical imaging.     

Automated volume analysis of head and neck lesions on CT scans using 3D level set segmentation

The authors have developed a semiautomatic system for segmentation of a diverse set of lesions in head and neck CT scans. The system takes as input an approximate bounding box, and uses a multistage level set to perform the final segmentation. A data set consisting of 69 lesions marked on 33 scans from 23 patients was used to evaluate the performance of the system. The contours from automatic segmentation were compared to both 2D and 3D gold standard contours manually drawn by three experienced radiologists. Three performance metric measures were used for the comparison. In addition, a radiologist provided quality ratings on a 1 to 10 scale for all of the automatic segmentations. For this pilot study, the authors observed that the differences between the automatic and gold standard contours were larger than the interobserver differences. However, the system performed comparably to the radiologists, achieving an average area intersection ratio of 85.4% compared to an average of 91.2% between two radiologists. The average absolute area error was 21.1% compared to 10.8%, and the average 2D distance was 1.38 mm compared to 0.84 mm between the radiologists. In addition, the quality rating data showed that, despite the very lax assumptions made on the lesion characteristics in designing the system, the automatic contours approximated many of the lesions very well.     

Computer-aided detection system for clustered microcalcifications: comparison of performance on full-field digital mammograms and digitized screen-film mammograms

We have developed a computer-aided detection (CAD) system to detect clustered microcalcifications automatically on full-field digital mammograms (FFDMs) and a CAD system for screen-film mammograms (SFMs). The two systems used the same computer vision algorithms but their false positive (FP) classifiers were trained separately with sample images of each modality. In this study, we compared the performance of the CAD systems for detection of clustered microcalcifications on pairs of FFDM and SFM obtained from the same patient. For case-based performance evaluation, the FFDM CAD system achieved detection sensitivities of 70%, 80% and 90% at an average FP cluster rate of 0.07, 0.16 and 0.63 per image, compared with an average FP cluster rate of 0.15, 0.38 and 2.02 per image for the SFM CAD system. The difference was statistically significant with the alternative free-response receiver operating characteristic (AFROC) analysis. When evaluated on data sets negative for microcalcification clusters, the average FP cluster rates of the FFDM CAD system were 0.04, 0.11 and 0.33 per image at detection sensitivity level of 70%, 80% and 90% compared with an average FP cluster rate of 0.08, 0.14 and 0.50 per image for the SFM CAD system. When evaluated for malignant cases only, the difference of the performance of the two CAD systems was not statistically significant with AFROC analysis.     

Relapsing polychondritis: reversible airway obstruction is not always asthma

Relapsing polychondritis (RP) is a rare disease characterized by recurrent inflammation of cartilaginous and other proteoglycan-rich tissues. Respiratory tract involvement is a common cause of morbidity and mortality in RP. We describe a patient whose clinical features at onset of disease were typical of asthma. Later, the patient developed symptoms and signs characteristic of RP. Tracheobronchomalacia necessitated airway support by stenting. The possibility that airway obstruction in the initial stages of RP is due to airway inflammation and that early, aggressive immunosuppressive treatment of RP may delay or prevent irreversible cartilaginous destruction and airway collapse are discussed.     

Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees

ObjectiveAfter deploying a clinical prediction model, subsequently collected data can be used to fine-tune its predictions and adapt to temporal shifts. Because model updating carries risks of over-updating/fitting, we study online methods with performance guarantees. Materials and MethodsWe introduce 2 procedures for continual recalibration or revision of an underlying prediction model: Bayesian logistic regression (BLR) and a Markov variant that explicitly models distribution shifts (MarBLR). We perform empirical evaluation via simulations and a real-world study predicting Chronic Obstructive Pulmonary Disease (COPD) risk. We derive “Type I and II” regret bounds, which guarantee the procedures are noninferior to a static model and competitive with an oracle logistic reviser in terms of the average loss.ResultsBoth procedures consistently outperformed the static model and other online logistic revision methods. In simulations, the average estimated calibration index (aECI) of the original model was 0.828 (95%CI, 0.818–0.938). Online recalibration using BLR and MarBLR improved the aECI towards the ideal value of zero, attaining 0.265 (95%CI, 0.230–0.300) and 0.241 (95%CI, 0.216–0.266), respectively. When performing more extensive logistic model revisions, BLR and MarBLR increased the average area under the receiver-operating characteristic curve (aAUC) from 0.767 (95%CI, 0.765–0.769) to 0.800 (95%CI, 0.798–0.802) and 0.799 (95%CI, 0.797–0.801), respectively, in stationary settings and protected against substantial model decay. In the COPD study, BLR and MarBLR dynamically combined the original model with a continually refitted gradient boosted tree to achieve aAUCs of 0.924 (95%CI, 0.913–0.935) and 0.925 (95%CI, 0.914–0.935), compared to the static model’s aAUC of 0.904 (95%CI, 0.892–0.916).DiscussionDespite its simplicity, BLR is highly competitive with MarBLR. MarBLR outperforms BLR when its prior better reflects the data.ConclusionsBLR and MarBLR can improve the transportability of clinical prediction models and maintain their performance over time.     

Improved Survival of Young Adults with Cancer Following the Passage of the Affordable Care Act

BackgroundCompared with their ensured counterparts, uninsured adolescents and young adults (AYAs) with cancer are more likely to present with advanced disease and have poor prognoses. The Patient Protection and Affordable Care Act (ACA), enacted in 2010, provided health care coverage to millions of uninsured young adults by allowing them to remain on their parents’ insurance until age 26 years (the Dependent Care Expansion, DCE). The impact of the expansion of insurance coverage on survival outcomes for young adults with cancer has not been assessed.ParticipantsUtilizing the Surveillance, Epidemiology, and End Results database, we identified all patients aged 12-16 (younger-AYAs), 19-23 (middle-AYAs), and 26-30 (older-AYAs) who were diagnosed with cancer between 2006-2008 (pre-ACA) and 2011-2013 (post-ACA).MethodsIn this population-based cohort study, we used an accelerated failure time model to assess changes in survival rates before and after the enactment of the ACA DCE.ResultsMiddle-AYAs ages 19-23 (thus eligible to remain on their parents’ insurance) experienced significantly increased 2-year survival after the enactment of the ACA DCE (survival time ratio 1.25, 95% confidence interval: 0.75-2.43, P = .029) and that did not occur in younger-AYAs (ages 12-16). Patients with sarcoma and acute myeloid leukemia accounted for the majority of improvement in survival. Middle-AYAs of hispanic ethnicity and those with low socioeconomic status experienced trends of improved survival after the ACA DCE was enacted.ConclusionSurvival outcomes improved for young adults with cancer following the expansion of health insurance coverage. Efforts are needed to expand coverage for the millions of young adults who do not have health insurance.     

Surface expression of Gp 165/95, the complement receptor CR3, as a marker of disease activity in systemic Lupus erythematosus

Complement-derived peptides capable of activating neutrophils appear in plasma during flares of systemic lupus erythematosus (SLE). One possible consequence of such activation is an increased expression of the surface adhesion promoting heterodimer gp165/95 (the complement receptor CR3). The quantity of gp165/95 was measured by indirect immunofluorescence using a monoclonal antibody of the CD11b group. Mol, directed to the alpha chain. Eighty-three percent of 26 patients with SLE expressed gp165/95 on their neutrophil surface to a greater extent than normals. The highest levels of surface gp165/95 were found in patients with the most severe disease, who also had the highest levels of the circulating anaphylatoxin C3a (mean = 560 ng/ml versus 147 ng/ml in controls). There was a negative correlation between expression of gp165/95 and absolute neutrophil count. Five individuals followed serially demonstrated an increase in surface gp165/95 during disease flares which returned to normal with clinical improvement. These data support the hypothesis that the neutrophils of patients with active SLE recruit increased numbers of gp165/95 molecules to their surface in respose to complement activation; these activated neutrophils bearing increased numbers of adhesion promoting gp165/95 may contribute to endothelial injury in SLE.