Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms

PURPOSE: To evaluate the effectiveness of a computerized classification method as an aid to radiologists reviewing clinical mammograms for which the diagnoses were unknown to both the radiologists and the computer. MATERIALS AND METHODS: Six mammographers and six community radiologists participated in an observer study. These 12 radiologists interpreted, with and without the computer aid, 110 cases that were unknown to both the 12 radiologist observers and the trained computer classification scheme. The radiologists' performances in differentiating between benign and malignant masses without and with the computer aid were evaluated with receiver operating characteristic (ROC) analysis. Two-tailed P values were calculated for the Student t test to indicate the statistical significance of the differences in performances with and without the computer aid. RESULTS: When the computer aid was used, the average performance of the 12 radiologists improved, as indicated by an increase in the area under the ROC curve (A(z)) from 0.93 to 0.96 (P <.001), by an increase in partial area under the ROC curve ((0.90)A(')(z)) from 0.56 to 0.72 (P <.001), and by an increase in sensitivity from 94% to 98% (P =.022). No statistically significant difference in specificity was found between readings with and those without computer aid (Delta = -0.014; P =.46; 95% CI: -0.054, 0.026), where Delta is difference in specificity. When we analyzed results from the mammographers and community radiologists as separate groups, a larger improvement was demonstrated for the community radiologists. CONCLUSION: Computer-aided diagnosis can potentially help radiologists improve their diagnostic accuracy in the task of differentiating between benign and malignant masses seen on mammograms.