Comparison of standard reading and computer aided diagnosis (CAD) on a proficiency test of screening mammography

PURPOSE: To evaluate the role of computer aided diagnosis (CAD) to improve screening mammograms interpretation. MATERIALS AND METHODS: Six radiologists underwent a screening mammography proficiency test first by conventional, then by CAD assisted reading. Sensitivity and recall rate at conventional and CAD reading were compared. Independent conventional double reading was simulated (15 pair combinations) and compared to single CAD reading. RESULTS: CAD marked 31 of 32 cancers (case-based sensitivity=96.8%). On a film and lesion basis, CAD identified 31 of 32 (96.8%) malignant calcifications and 29 of 42 (69.0%) malignant opacities, the only cancer not identified by CAD being depicted as an isolated opacity. CAD marked 348 areas (153 microcalcifications and 195 opacities) in 88 of 108 non cancer cases, with a case-based specificity of 18.5% (20/108). Considering all six readings, cancer was identified in 164 or 174 of 192 readings (85.4 vs 90.6%, c2 2.03, df=1, p=0.15) and recalls of non-cancer cases were 108 or 159 of 648 readings (16.6 vs 24.5%, c2 11.7, df=1, p<0.001) at conventional or CAD reading, respectively. CAD reading (average of 6 readings, 192 cancer, 648 non-cancer readings) was slightly, non significantly less sensitive (sensitivity 90.6 vs 92.9%, c2 0.73, df=1, p=0.39) and slightly, but not significantly more specific (recall rate 24.5 vs 26.1%, c2 0.56, df=1, p=0.45) as compared to simulated independent double reading (average of 15 combinations, 480 cancer, 1620 non-cancer readings). CONCLUSION: CAD seems to allow for a limited absolute increase (+5.2%) in sensitivity and for a limited absolute increase (+7.9%) in recall rate, the latter difference only reaching statistical significance. CAD reading showed no significant difference in diagnostic accuracy as compared to conventional (simulated) double reading, although further studies are needed to confirm it as possible alternative to double reading in the current screening practice.     

The assessment of the impact of a double reading by expert readers in a mass mammographic study

PURPOSE: To evaluate the role of double reading of screening mammograms by expert radiologists. MATERIAL AND METHODS: We analyzed the results of independent readings of a proficiency test of screening mammography (140 cases, 32 cancers) performed by four expert radiologists. Double reading was simulated by matching the four original readings in 6 possible combinations. The impact of double reading over single reading was evaluated in terms of increased sensitivity and increased recall rate. RESULTS: Of 32 carcinomas 22, 6, or 4 were identified by 4, 3, or 2 readers, respectively. Of 108 cases negative for cancer a recall for further investigations was suggested by 4, 3, 2, 1 or no reader(s) in 3, 3, 9, 14, or 79 cases, respectively. Inter-reader diagnostic repeatability was good (k = 0.65). Single readers achieved an average sensitivity of 89% (range 87.5-90.6%) and an average recall rate of 12.2% (range 7.4-16.6%). Simulated double reading achieved an average increase in sensitivity of 8.8% (range 6.2-10.95%) and an average increase of recall rate of 6.2% (range 3.8-8.3%). CONCLUSIONS: Even though the relative increase of recall rate is relevant (+53.2%), the corresponding gain in sensitivity justifies the use of double reading, which was confirmed to be worthwhile also when expert radiologists are involved. This study confirms the opportunity of adopting double reading as a routine procedure in mammographic screening.     

Diagnostic accuracy of commercial system for computer-assisted detection (CADx) as an adjunct to interpretation of mammograms

PURPOSE: To evaluate the diagnostic accuracy of the commercial computer-aided detection CADx system for the reading of mammograms. MATERIALS AND METHODS: The study assessed the Second Look system developed and marketed by CADx Medical Systems, Montreal, Canada. The diagnostic sensitivity was evaluated by means of a retrospective study on 98 consecutive cancers detected at screening by double independent reading. The specificity and the positive predictive value (PPV) for cancer of the CADx system were prospectively evaluated on a second group of 560 consecutive mammograms of asymptomatic women not included in screening program. The radiologist who was present during the test assessed the abnormal mammographic findings by one or more of the following diagnostic procedures: physical examination, additional mammographic detail views with or without magnification, ultrasonography, ultrasound- or mammography-guided fine needle aspiration cytology, and core-biopsy. The exams first underwent conventional reading and then a second reading carried out with the aid of the CADx system. RESULTS: The overall diagnostic sensitivity of the CADx system on the 98 screening cancers was 81.6%; in particular it was 89.3% for calcifications, 83.9% for masses and only 37.5% for architectural distortion. The CADx markings for each mammography were 4.7 on average. Identification of invasive carcinoma was independent from tumour size. In the second group of 560 mammograms, the CADx system marked all cases identified as positive by conventional reading and confirmed by biopsy (7/7), but did not permit the detection of any additional cancer. The CADx markings per exam were 4.2 on average, the specificity was 13.7% and the PPV was 0.55% versus 13.7% recall rate of conventional reading. CADx reading led to a 1.96% (11/560) increase of the women necessitating further diagnostic investigation. CONCLUSIONS: The results of our study show that the diagnostic sensitivity of the CADx system is lower than that obtained by double independent reading at screening. Used in association with conventional reading of mammograms of asymptomatic women the CADx system did not increase diagnostic sensitivity.     

Optimizing parameters for computer-aided diagnosis of microcalcifications at mammography

RATIONALE AND OBJECTIVES: The purpose of this study was to optimize selection of the mammographic features most useful in discriminating benign from malignant clustered microcalcifications. MATERIALS AND METHODS: The computer-aided diagnosis (CAD) system automatically extracted from digitized mammograms 13 quantitative features characterizing microcalcification clusters. Archival cases (n = 134; patient age range, 31-77 years; mean age, 56.8 years) with known histopathologic results (79 malignant, 55 benign) were selected. Three radiologists at three facilities independently analyzed the microcalcifications by using the CAD system. Stepwise discriminant analysis selected the features best discriminating benign from malignant microcalcifications. A classification scheme was constructed on the basis of these optimized features, and its performance was evaluated by using receiver operating characteristic (ROC) analysis. RESULTS: Six of the 13 variables extracted by the CAD system were selected by stepwise determinant analysis for generating the classification scheme, which yielded an ROC curve with an area (Az) of 0.98, specificity of 83.64%, positive predictive value of 89.53%, and accuracy of 91.79% for 98% sensitivity. When patient age was an additional variable, the scheme's performance improved, but this was not statistically significant (Az = 0.98). The ROC curve of the classifier (without age as an additional variable) yielded a high Az of 0.96 for patients younger than 50 years and an even higher (P < .02) Az of 0.99 for those 50 years or older. CONCLUSION: Stepwise discriminant analysis optimized performance of a classification scheme for microcalcifications by selecting six optimized features. Scheme performance was significantly (P < .02) higher for women 50 years or older, but the addition of patient age as a variable did not produce a statistically significant increase in performance.     

Computer-aided detection versus independent double reading of masses on mammograms

PURPOSE: To evaluate the use of a computer-aided detection (CAD) system (designed for mammographic mass detection) to help improve mass interpretation and to compare CAD results with independent double-reading results. MATERIALS AND METHODS: Screening mammograms from 500 cases were collected; 125 of these cases were screening-detected cancers, and 125 were interval cancers. Previously obtained screening mammograms (ie, prior mammograms) were available in all cases. All mammograms were analyzed by a CAD system, which detected mass regions and assigned a level of (cancer) suspicion to each mass. Ten experienced screening radiologists read the prior mammograms. For independent interpretation with CAD, the suspicion rating assigned to each finding by the radiologist was weighted with the CAD output at the area of the finding. CAD markers on areas that were not reported by the radiologist were not used. Independent double reading was implemented by using a rule to combine the levels of suspicion assigned to findings by two radiologists. Results were evaluated by using localized-response receiver operating characteristic analysis. RESULTS: In a total of 141 cases, there was a visible abnormality at the location of the cancer on the prior mammogram, and 115 of these were classified as mass cases. For prior mammograms that depicted masses, the mean sensitivity of the radiologists, as averaged among the false-positive rates lower than 10%, was 39.4%; this increased by 7.0% with CAD and by 10.5% with double reading. Differences among single, double, and CAD readings were statistically significant (P <.001). CONCLUSION: Although independent double reading yields the best detection performance, the presence and probability of CAD mass markers can improve mammogram interpretation.     

A study on two different CAD systems for mammography as an aid to radiological diagnosis in the search of microcalcification clusters

OBJECTIVE: The aim of the present study was to evaluate the efficacy of two different computer aided detection (CAD) systems for mammography in improving radiological diagnosis in the search of microcalcification clusters. The CAD systems used are: the SecondLooktrade mark (CADx Medical Systems, Canada) commercial system and the CALMA (computer assisted library in MAmmography) research CAD system. Three radiologists were asked to read mammographic images with and without the support of the CAD systems. MATERIAL AND METHODS: Three radiologists with respectively 3, 5 and 7 years of practice in mammogram reading in an Italian public hospital analysed a dataset composed of 120 digitized mammograms of healthy subjects with no lesion (proven by a radiological follow up of at least 3 years) and 70 images of patients with malignant cluster of microcalcification (proven by histopathological examination) both with no CAD support as well as with the help of the SecondLooktrade mark system. After 3 months they were asked to observe the same digitized mammograms with the assistance of the CALMA system. The radiologists worked independently and were unaware of the final diagnosis. The values of the area A(z) under the ROC curve, diagnostic sensitivity, specificity, positive and negative predictive values, and diagnostic accuracy were evaluated with and without the support of the CAD systems. The reading time and qualitative evaluations of each radiologist were also reported. RESULTS: With the support of the two CAD systems an improvement in A(z) area was obtained ranging from 0.01 to 0.04. Sensitivity increased from +8.6 to +15.7% and specificity decreased from 0.8 to 4.2%. CONCLUSION: In our study, not conditioned by the dataset, the CAD systems as second reader produced an increase in overall sensitivity of up to 15.7%, with a little decrease in specificity of up to 4.2%. Based on these results both CAD systems might be used in the current practise to improve the sensitivity values of conventional reading (radiologist alone). The results of this study show that no significant differences exist in term of A(z), sensitivity and specificity between CALMA and CADx.     

Comparison of two commercial systems for computer-assisted detection (CAD) as an aid to interpreting screening mammograms

PURPOSE: To compare the diagnostic accuracy of two commercial CAD systems (CADx and R2) and their impact as an aid to conventional reading of screening mammograms. MATERIALS AND METHODS: The image set considered consisted of 120 mammograms, 89 confirmed negative and 31 with subsequent interval cancers (11 classified as false negatives (FN), 20 as "minimal signs" (MS)). The set was digitised and processed with CAD, and printouts obtained of the mammograms with indications of the areas warranting review. Six expert radiologists read the mammograms three times, once using conventional reading and twice using CAD reading with CADx and R2, respectively. The two CAD systems were compared in terms of diagnostic accuracy of the marks and the impact of CAD reading compared to conventional reading and to the use of independent second reading simulated by combining pairs of single conventional readings. RESULTS: R2 highlighted more calcifications (218 vs 132, +65%) and CADx highlighted more masses (208 vs 105, +98%). CADx and R2 marked 15 and 17 out of 31 cancers, respectively (sensitivity 48.3% vs 54.8%, chi squared=6.4, p=0.79), 10 and 6 out of 11 FN (90.9% vs 54.5%, chi squared=2.0, p=0.15), respectively, and 5 and 11 out of 20 MS (25.0% vs 55.0%, chi squared=2.6, p=0.10), respectively. As for specificity, the false positive markings for masses were on average (per case) 1.60 for CADx and 0.75 for R2, those for calcifications were 1.08 for CADx and 1.77 for R2 and the total false positive markings were 2.68 for CADx and 2.52 for R2. CADx and R2 marked 73 and 63 of 89 negative controls (specificity = 0.18 vs 0.29, chi squared=2.52, p=0.11), respectively. All the radiologists showed greater sensitivity with CAD reading compared to conventional reading. On average, sensitivity with conventional reading was 58.6% (109/186), as against 70.9% (132/186) for CADx or R2 (chi squared=5.71, p=0.016). Sensitivity for FN cases was 71.2% (47/66) with conventional reading, 84.8% (56/66) with CADx (chi squared=2.82, p=0.09) and 80.3% (53/66) for R2 (chi squared=1.03, p=0.30) (CADx vs R2, chi squared=0.21, p=0.64). Sensitivity for MS cases was 51.6% (62/120) for conventional reading, 63.3% (76/120) for CADx (chi squared=2.88, p=0.08) and 65.8% (79/120) for R2 (chi squared=4.40, p=0.03) (CADx vs R2, chi squared=0.07, p=0.78). The recall rates were 18.1% (97/534) for conventional reading, 29.7% (159/534) for CADx (chi squared=5.72, p=0.01) and 24.3% (130/534) for R2 (chi squared=10.11, p=10-5) (CADx vs R2, chi squared=3.71, p=0.05). Double reading was significantly more sensitive than conventional reading (chi squared=29.6, p=10-6), CADx (chi squared=5.33, p=0.02) and R2 (chi squared=5.33, p=0.02). The recall rate for double reading was significantly higher than for conventional reading (chi squared=21.5, p=10-6) whereas no significant difference was detected when compared to CADx (chi squared=0.16, p=0.68) or R2 (chi squared=3.4, p=0.06). CONCLUSIONS: Despite using different algorithms, the two CAD systems exhibit comparable levels of diagnostic accuracy and a similar positive impact on sensitivity when used as an aid to conventional reading. Single reading with either CAD system is as specific but not as sensitive to double independent reading: its use as an alternative to double reading cannot be recommended and should be investigated further by means of controlled prospective studies.     

Comparison of standard reading and computer aided detection (CAD) on a national proficiency test of screening mammography

OBJECTIVE: To evaluate the role of computer aided detection (CAD) in improving the interpretation of screening mammograms MATERIAL AND METHODS: Ten radiologists underwent a proficiency test of screening mammography first by conventional reading and then with the help of CAD. Radiologists were blinded to test results for the whole study duration. Results of conventional and CAD reading were compared in terms of sensitivity and recall rate. Double reading was simulated combining conventional readings of four expert radiologists and compared with CAD reading. RESULTS: Considering all ten readings, cancer was identified in 146 or 153 of 170 cases (85.8 vs. 90.0%; chi(2)=0.99, df=1, P=0.31) and recalls were 106 or 152 of 1330 cases (7.9 vs. 11.4%; chi(2)=8.69, df=1, P=0.003) at conventional or CAD reading, respectively. CAD reading was essentially the same (sensitivity 97.0 vs. 96.0%; chi(2)=7.1, df=1, P=0.93; recall rate 10.7 vs. 10.6%; chi(2)=1.5, df=1, P=0.96) as compared with simulated conventional double reading. CONCLUSION: CAD reading seems to improve the sensitivity of conventional reading while reducing specificity, both effects being of limited size. CAD reading had almost the same performance of simulated conventional double reading, suggesting a possible use of CAD which needs to be confirmed by further studies inclusive of cost-effective analysis.     

Computed assisted detection of interval breast cancers

OBJECTIVES: To examine interval cancer detection rate for a system of computer assisted detection (CAD) and its influence on radiologists' sensitivity/specificity in a screen-like retrospective review situation. MATERIALS AND METHODS: Three screening radiologists reviewed previous screen images of 59 interval cancers mixed with other screening mammograms (ratio 1:5) and non-mixed. Mixed interval cases were interpreted both without and with aid of CAD. RESULTS: CAD detected a number of 14 interval cancers while the three radiologists detected 17, 12 and 11 without and 16, 10 and 13 with CAD. Although CAD specificity was low (38%) no reduction in radiologists' specificity occurred using CAD (73%, 82% and 89% without and 78%, 90% and 92% with CAD). Non-mixed reading increased radiologists' detection rate to 21, 17 and 19 interval cancers respectively. CONCLUSION: Despite sufficiently high sensitivity for CAD alone no increase in radiologist sensitivity (or decrease in specificity) occurred with CAD. Improving CAD specificity, with unaffectedly high sensitivity, should make radiologists more inclined to revise interpretations according to CAD. The potential sensitivity increase, noted when using CAD as a double reader, could be realised in this way.     

Improving digital breast tomosynthesis reading time: A pilot multi-reader,multi-case study using concurrent Computer-Aided Detection (CAD)

PurposeEvaluate concurrent Computer-Aided Detection (CAD) with Digital Breast Tomosynthesis (DBT) to determine impact on radiologist performance and reading time.Materials and methodsThe CAD system detects and extracts suspicious masses, architectural distortions and asymmetries from DBT planes that are blended into corresponding synthetic images to form CAD-enhanced synthetic images. Review of CAD-enhanced images and navigation to corresponding planes to confirm or dismiss potential lesions allows radiologists to more quickly review DBT planes. A retrospective, crossover study with and without CAD was conducted with six radiologists who read an enriched sample of 80 DBT cases including 23 malignant lesions in 21 women. Area Under the Receiver Operating Characteristic (ROC) Curve (AUC) compared the readings with and without CAD to determine the effect of CAD on overall interpretation performance. Sensitivity, specificity, recall rate and reading time were also assessed. Multi-reader, multi-case (MRMC) methods accounting for correlation and requiring correct lesion localization were used to analyze all endpoints. AUCs were based on a 0–100% probability of malignancy (POM) score. Sensitivity and specificity were based on BI-RADS scores, where 3 or higher was positive.ResultsAverage AUC across readers without CAD was 0.854 (range: 0.785-0.891, 95% confidence interval (CI): 0.769,0.939) and 0.850 (range: 0.746-0.905, 95% CI: 0.751,0.949) with CAD (95% CI for difference: −0.046,0.039), demonstrating non-inferiority of AUC. Average reduction in reading time with CAD was 23.5% (95% CI: 7.0–37.0% improvement), from an average 48.2 (95% CI: 39.1,59.6) seconds without CAD to 39.1 (95% CI: 26.2,54.5) seconds with CAD. Per-patient sensitivity was the same with and without CAD (0.865; 95% CI for difference: −0.070,0.070), and there was a small 0.022 improvement (95% CI for difference: ‐0.046,0.089) in per-lesion sensitivity from 0.790 without CAD to 0.812 with CAD. A slight reduction in specificity with a −0.014 difference (95% CI for difference: ‐0.079,0.050) and a small 0.025 increase (95% CI for difference: −0.036,0.087) in recall rate in non-cancer cases were observed with CAD.ConclusionsConcurrent CAD resulted in faster reading time with non-inferiority of radiologist interpretation performance. Radiologist sensitivity, specificity and recall rate were similar with and without CAD.     

Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms

PURPOSE: To evaluate the effectiveness of a computerized classification method as an aid to radiologists reviewing clinical mammograms for which the diagnoses were unknown to both the radiologists and the computer. MATERIALS AND METHODS: Six mammographers and six community radiologists participated in an observer study. These 12 radiologists interpreted, with and without the computer aid, 110 cases that were unknown to both the 12 radiologist observers and the trained computer classification scheme. The radiologists' performances in differentiating between benign and malignant masses without and with the computer aid were evaluated with receiver operating characteristic (ROC) analysis. Two-tailed P values were calculated for the Student t test to indicate the statistical significance of the differences in performances with and without the computer aid. RESULTS: When the computer aid was used, the average performance of the 12 radiologists improved, as indicated by an increase in the area under the ROC curve (A(z)) from 0.93 to 0.96 (P <.001), by an increase in partial area under the ROC curve ((0.90)A(')(z)) from 0.56 to 0.72 (P <.001), and by an increase in sensitivity from 94% to 98% (P =.022). No statistically significant difference in specificity was found between readings with and those without computer aid (Delta = -0.014; P =.46; 95% CI: -0.054, 0.026), where Delta is difference in specificity. When we analyzed results from the mammographers and community radiologists as separate groups, a larger improvement was demonstrated for the community radiologists. CONCLUSION: Computer-aided diagnosis can potentially help radiologists improve their diagnostic accuracy in the task of differentiating between benign and malignant masses seen on mammograms.     

Digital mammography: Comparison of adaptive and nonadaptive CAD methods for mass detection

RATIONALE AND OBJECTIVES: The authors compared the performance of adaptive and nonadaptive computer-aided diagnostic (CAD) methods for breast mass detection with digital mammography. MATERIALS AND METHODS: Both adaptive and nonadaptive modular CAD methods employed recent advances in multiresolution and mutiorientation wavelet transforms for improved feature extraction. The nonadaptive method uses fixed parameters for the image preprocessing modules. The adaptive method, a new class of algorithms, adapts to image content by selecting parameters for the image preprocessing modules within a parameter range. Comparison of the two methods was performed for each individual CAD module with a region-of-interest (ROI) database containing all mass types and normal tissue. RESULTS: Receiver operating characteristic (ROC) analysis clearly demonstrated an improvement in performance for the three adaptive modules and a significant overall difference between the two methods. The average ROC area index (Az) values were 0.86 and 0.95 for the nonadaptive and adaptive methods, respectively. The corresponding P value is .0145. For a previously reported database of full mammographic images containing 50 abnormal cases with all mass types and 50 normal images, the adaptive CAD method had a sensitivity of 96% (1.71 false-positive results per image) compared with 89% (1.91 false-positive results per image) for the nonadaptive CAD method. CONCLUSION: The adaptive CAD method demonstrated better performance. A study is in progress to determine the generalizability of the adaptive CAD method by applying it to larger retrospective image databases with different film digitizers.     

Computer-aided detection of amorphous calcifications

OBJECTIVE: Computer-aided detection (CAD) systems have been used successfully to detect malignant calcifications on mammography, with sensitivities ranging from 86% to 99%. Amorphous calcifications are a subset of small indistinct calcifications of intermediate concern that have a 20% likelihood of being malignant and that are frequently overlooked on mammography. The purpose of our study was to determine the sensitivity of one commercially available CAD system for detecting amorphous calcifications. MATERIALS AND METHODS: A commercially available CAD system evaluated mammograms of 82 patients with 85 mammographically detected and histologically sampled groups of amorphous calcifications (21 malignant, 14 high risk, and 50 benign). The sensitivity of the system for detecting the calcifications on at least one image of the two-view mammographic examination (case sensitivity) and on each individual mammographic image (image sensitivity) was determined. Findings were correlated with results from large core needle biopsy or surgical excision in each case. RESULTS: The CAD system detected amorphous calcifications in 43 of 85 cases (case sensitivity, 51%) and in 59 of 146 mammographic images (image sensitivity, 40%). The case sensitivities by histologic outcome were 57% for malignant calcifications, 29% for high-risk calcifications, and 54% for benign calcifications. An average of 2.0 false-positive marks were displayed per case. CONCLUSION: The CAD sensitivity for malignant amorphous calcifications is markedly lower than previously reported for all malignant calcifications. Breast imaging radiologists who use CAD systems should continue to search diligently for these difficult-to-detect lesions.     

Effect of computer-aided detection on mammographic performance: experimental study on readers with different levels of experience

Purpose: To evaluate the effect of computer-aided detection (CAD) on the reader's performance.

Material and Methods: Four screening radiologists, two novice radiologists, and two residents with no prior experience in CAD read films of 200 women without and with CAD. The films, including 16 screen-detected cancers and 35 cancers “missed” on prior screening, were divided into two rollers: A (free time schedule) and B (prompted time schedule). Reading times were noted. Individual readings without and with CAD were compared, sensitivities and specificities were calculated.

Results: The sensitivity of CAD was 70.6% and specificity 15.8%. In 408 cancer readings, the screeners found 10 and other readers 7 new cancers with the aid of CAD. The screeners changed their opinion four times and others six times from true positive to false negative when CAD was negative. CAD output produced 12 versus 13 new false-positive findings respectively after 2352 readings. CAD did not significantly affect the reader's sensitivities/specificities regardless of the time limit (P = not significant). The use of CAD increased mean time for roller reading from 56 to 63 min (P = 0.053).

Conclusion: Screening radiologists benefited slightly more from CAD than other readers did, but no statistical significant difference was found in personal readings without and with CAD.     

Application of a computer-aided detection (CAD) system to digitalized mammograms for identifying microcalcifications]

PURPOSE: It is estimated that during mammographic screening programs radiologists fail to detect approximately 25% of breast cancers visible on retrospective review; this percentage rises to 50% if minimal signs are considered. Independent double reading is now strongly recommended as it allows to reduce the rate of false negative examinations by 5-15%. Recent technological progress has allowed to develop a number of computer-aided detection (CAD) systems. The aim of CAD is to help radiologists interpret lesions by serving as a second reader. In this study the authors developed and applied a CAD system to measure its ability to microcalcifications detect and compare its performance with that of a human observer. MATERIAL AND METHODS: The study was performed as part of the CALMA (computer-aided library for mammography) project of the Pisa section of the National Institute of Nuclear Physics. The aim of this project is to set up a large database of digital mammograms and to develop a CAD system. Our study series consisted of 802 mammograms - corresponding to 213 patients - digitalized between March and June 2000. We performed traditional mammography and then digitalized the mammograms using a CCD linear scanner (pixel size of 85 x 85 microm2, 12 bits). The images were evaluated by two radiologists with similar experience and then by the CAD system. This CAD system searches for microcalcifications by using ad hoc algorithms and an artificial neural network (Sanger type). RESULTS: The number of clusters in our database was 141 corresponding to 140 images; 692 images were non pathological. The CAD system identified a variable number of clusters depending on the threshold values. The threshold value is a number over which the probability of finding a lesion is highest. With thresholds of 0.13 and 0.14 the CAD system identified 140/141 clusters (99.3%); with a threshold of 0.15 it identified 139/141 clusters (98.6%); with a threshold of 0.16, 137/141 (97.2%); with a threshold of 0.18, 133/141 (94.3%); with thresholds of 0.18 and 0.20, 130/141 (92.2%). With threshold values of 0.13, 0.14, 0.15, 0.16 and 0.17 the system's sensitivity was greater than 82%, whereas with values of 0.18 and 0.20 it was greater than 80%. The number of false positive region of interest (ROI) / image was greater with low threshold values: in particular, thresholds of 0.13 and 0.14 yielded 16 false positives /image, thresholds of 0.15 and 0.16 yielded 9 and 7 false positives/image, and both 0.18 and 0.20 only 5/image. DISCUSSION: ROC curve shows how the use of high threshold values determined a very high specificity despite very low sensitivity rates. Conversely, low threshold values allowed to have a high sensitivity and a very low specificity. The best performance of our CAD system was obtained with threshold values at 0.15 and 0.16. In fact these thresholds resulted in a high sensitivity (greater than 82%) with an acceptable number of false positives/image, 9 and 7/image, respectively. It is not yet known how radiologists can deal with large numbers of false positives in screening programmes but in our opinion the most important feature of a good CAD system is a high sensitivity. CONCLUSIONS: In the near future the use of CAD systems will be widespread and easier to apply to everyday practice above all in centers where digital mammography is performed. Mammograms could be directly shown to radiologists after the CAD system has selected the ROI and analysed the images. Thanks to its high sensitivity and despite its low specificity CAD represents a concrete aid for radiologists.     

Ovarian cancer: comparison of observer performance for four methods of interpreting CT scans.

PURPOSE: To assess the effects of four interpretative methods on observers' mean sensitivity and specificity by using computed tomography (CT) of ovarian carcinoma as a model. MATERIALS AND METHODS: CT scans in 98 patients with ovarian carcinoma and 49 women who were disease free were retrospectively reviewed by four experienced blinded radiologists to compare single-observer reading, single-observer reading with an anatomic checklist, paired-observer reading (simultaneous double reading), and replicated reading (combination of two independent readings). Confidence level scoring was used to identify three possible disease forms in each patient: extranodal tumor, lymphadenopathy, and ascites. Patient conditions were then categorized as abnormal or normal. RESULTS: There were no significant improvements in sensitivity or specificity for classification of patient conditions as abnormal or normal when comparing single-observer interpretation with single-observer interpretation with a checklist or paired-observer interpretation. Although there was no significant improvement in the mean sensitivity (93% vs 94%) by using the replicated reading method, there was a statistically significant improvement in mean specificity (85% vs 79%) for the replicated readings compared with single-observer interpretations (P < .05). CONCLUSION: Diagnostic aids such as checklists and paired simultaneous readings did not lead to an improved mean observer performance for experienced readers. However, an increase in the mean specificity occurred with replicated readings.     

Computer-aided detection in computed tomography colonography: current status and problems with detection of early colorectal cancer

PURPOSE: The aim of this study was to evaluate the usefulness of computer-aided detection (CAD) in diagnosing early colorectal cancer using computed tomography colonography (CTC). MATERIALS AND METHODS: A total of 30 CTC data sets for 30 early colorectal cancers in 30 patients were retrospectively reviewed by three radiologists. After primary evaluation, a second reading was performed using CAD findings. The readers evaluated each colorectal segment for the presence or absence of colorectal cancer using five confidence rating levels. To compare the assessment results, the sensitivity and specificity with and without CAD were calculated on the basis of the confidence rating, and differences in these variables were analyzed by receiver operating characteristic (ROC) analysis. RESULTS: The average sensitivities for the detection without and with CAD for the three readers were 81.6% and 75.6%, respectively. Among the three readers, only one reader improved sensitivity with CAD compared to that without. CAD decreased specificity in all three readers. CAD detected 100% of protruding lesions but only 69.2% of flat lesions. On ROC analysis, the diagnostic performance of all three readers was decreased by use of CAD. CONCLUSION: Currently available CAD with CTC does not improve diagnostic performance for detecting early colorectal cancer. An improved CAD algorithm is required for detecting flat lesions and reducing the false-positive rate.     

Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection

PURPOSE: To compare the performance of radiologists and of a computer-aided detection (CAD) algorithm for pulmonary nodule detection on thin-section thoracic computed tomographic (CT) scans. MATERIALS AND METHODS: The study was approved by the institutional review board. The requirement of informed consent was waived. Twenty outpatients (age range, 15-91 years; mean, 64 years) were examined with chest CT (multi-detector row scanner, four detector rows, 1.25-mm section thickness, and 0.6-mm interval) for pulmonary nodules. Three radiologists independently analyzed CT scans, recorded the locus of each nodule candidate, and assigned each a confidence score. A CAD algorithm with parameters chosen by using cross validation was applied to the 20 scans. The reference standard was established by two experienced thoracic radiologists in consensus, with blind review of all nodule candidates and free search for additional nodules at a dedicated workstation for three-dimensional image analysis. True-positive (TP) and false-positive (FP) results and confidence levels were used to generate free-response receiver operating characteristic (ROC) plots. Double-reading performance was determined on the basis of TP detections by either reader. RESULTS: The 20 scans showed 195 noncalcified nodules with a diameter of 3 mm or more (reference reading). Area under the alternative free-response ROC curve was 0.54, 0.48, 0.55, and 0.36 for CAD and readers 1-3, respectively. Differences between reader 3 and CAD and between readers 2 and 3 were significant (P < .05); those between CAD and readers 1 and 2 were not significant. Mean sensitivity for individual readings was 50% (range, 41%-60%); double reading resulted in increase to 63% (range, 56%-67%). With CAD used at a threshold allowing only three FP detections per CT scan, mean sensitivity was increased to 76% (range, 73%-78%). CAD complemented individual readers by detecting additional nodules more effectively than did a second reader; CAD-reader weighted kappa values were significantly lower than reader-reader weighted kappa values (Wilcoxon rank sum test, P < .05). CONCLUSION: With CAD used at a level allowing only three FP detections per CT scan, sensitivity was substantially higher than with conventional double reading.     

Impact of the Kaiser score on clinical decision-making in BI-RADS 4 mammographic calcifications examined with breast MRI

To investigate whether the application of the Kaiser score for breast magnetic resonance imaging (MRI) might downgrade breast lesions that present as mammographic calcifications and avoid unnecessary breast biopsies This IRB-approved, retrospective, cross-sectional, single-center study included 167 consecutive patients with suspicious mammographic calcifications and histopathologically verified results. These patients underwent a pre-interventional breast MRI exam for further diagnostic assessment before vacuum-assisted stereotactic-guided biopsy (95 malignant and 72 benign lesions). Two breast radiologists with different levels of experience independently read all examinations using the Kaiser score, a machine learning–derived clinical decision-making tool that provides probabilities of malignancy by a formalized combination of diagnostic criteria. Diagnostic performance was assessed by receiver operating characteristics (ROC) analysis and inter-reader agreement by the calculation of Cohen’s kappa coefficients. Application of the Kaiser score revealed a large area under the ROC curve (0.859–0.889). Rule-out criteria, with high sensitivity, were applied to mass and non-mass lesions alike. The rate of potentially avoidable breast biopsies ranged between 58.3 and 65.3%, with the lowest rate observed with the least experienced reader. Applying the Kaiser score to breast MRI allows stratifying the risk of breast cancer in lesions that present as suspicious calcifications on mammography and may thus avoid unnecessary breast biopsies. • The Kaiser score is a helpful clinical decision tool for distinguishing malignant from benign breast lesions that present as calcifications on mammography. • Application of the Kaiser score may obviate 58.3–65.3% of unnecessary stereotactic biopsies of suspicious calcifications. • High Kaiser scores predict breast cancer with high specificity, aiding clinical decision-making with regard to re-biopsy in case of negative results.     

Accuracy of screening mammography using single versus independent double interpretation

OBJECTIVE: We conducted an analysis among 31 community radiologists to identify the average change in screening mammography interpretive accuracy afforded by independent double interpretation. MATERIALS AND METHODS: We assessed interpretive accuracy using a stratified random sample of test mammograms that included 30 women with cancer and 83 without. Radiologists were unaware of clinical information and of each other's assessments. We describe accuracy for individual radiologists and for double interpretation, including average sensitivity, specificity, diagnostic likelihood ratios positive and negative, and area under the receiver operating characteristic (ROC) curve. We also assessed weighted and nonweighted kappa statistics among all 465 pairs of radiologists and 31,465 pairs of unique pairs. The assessment for double interpretations used the "highest" (i.e., most abnormal) assessment of the two radiologists. We calculated the difference between each radiologist's individual accuracy and the average accuracy across that radiologist's 30 double interpretations. RESULTS: We found the following average accuracy statistics for individual radiologists: sensitivity, 79%; specificity, 81%; diagnostic likelihood ratio positive, 5.53; diagnostic likelihood ratio negative, 0.26; and area under the ROC curve, 0.85. The mean kappa statistic among radiologists for cancer cases increased with double interpretation from 0.59 to 0.70, and for noncancer cases from 0.30 to 0.34. Double interpretation resulted in an average increase in sensitivity of 7%, an average decrease in specificity of 11%, a decrease in diagnostic likelihood ratio positive of 2.35, a decrease in diagnostic likelihood ratio negative of 0.06, and an increase in area under the ROC curve of 0.02. CONCLUSION: Independent double interpretation does not increase accuracy as measured by the area under the ROC curve.