Comparison of standard reading and computer aided diagnosis (CAD) on a proficiency test of screening mammography

PURPOSE: To evaluate the role of computer aided diagnosis (CAD) to improve screening mammograms interpretation. MATERIALS AND METHODS: Six radiologists underwent a screening mammography proficiency test first by conventional, then by CAD assisted reading. Sensitivity and recall rate at conventional and CAD reading were compared. Independent conventional double reading was simulated (15 pair combinations) and compared to single CAD reading. RESULTS: CAD marked 31 of 32 cancers (case-based sensitivity=96.8%). On a film and lesion basis, CAD identified 31 of 32 (96.8%) malignant calcifications and 29 of 42 (69.0%) malignant opacities, the only cancer not identified by CAD being depicted as an isolated opacity. CAD marked 348 areas (153 microcalcifications and 195 opacities) in 88 of 108 non cancer cases, with a case-based specificity of 18.5% (20/108). Considering all six readings, cancer was identified in 164 or 174 of 192 readings (85.4 vs 90.6%, c2 2.03, df=1, p=0.15) and recalls of non-cancer cases were 108 or 159 of 648 readings (16.6 vs 24.5%, c2 11.7, df=1, p<0.001) at conventional or CAD reading, respectively. CAD reading (average of 6 readings, 192 cancer, 648 non-cancer readings) was slightly, non significantly less sensitive (sensitivity 90.6 vs 92.9%, c2 0.73, df=1, p=0.39) and slightly, but not significantly more specific (recall rate 24.5 vs 26.1%, c2 0.56, df=1, p=0.45) as compared to simulated independent double reading (average of 15 combinations, 480 cancer, 1620 non-cancer readings). CONCLUSION: CAD seems to allow for a limited absolute increase (+5.2%) in sensitivity and for a limited absolute increase (+7.9%) in recall rate, the latter difference only reaching statistical significance. CAD reading showed no significant difference in diagnostic accuracy as compared to conventional (simulated) double reading, although further studies are needed to confirm it as possible alternative to double reading in the current screening practice.     

Comparison of two commercial systems for computer-assisted detection (CAD) as an aid to interpreting screening mammograms

PURPOSE: To compare the diagnostic accuracy of two commercial CAD systems (CADx and R2) and their impact as an aid to conventional reading of screening mammograms. MATERIALS AND METHODS: The image set considered consisted of 120 mammograms, 89 confirmed negative and 31 with subsequent interval cancers (11 classified as false negatives (FN), 20 as "minimal signs" (MS)). The set was digitised and processed with CAD, and printouts obtained of the mammograms with indications of the areas warranting review. Six expert radiologists read the mammograms three times, once using conventional reading and twice using CAD reading with CADx and R2, respectively. The two CAD systems were compared in terms of diagnostic accuracy of the marks and the impact of CAD reading compared to conventional reading and to the use of independent second reading simulated by combining pairs of single conventional readings. RESULTS: R2 highlighted more calcifications (218 vs 132, +65%) and CADx highlighted more masses (208 vs 105, +98%). CADx and R2 marked 15 and 17 out of 31 cancers, respectively (sensitivity 48.3% vs 54.8%, chi squared=6.4, p=0.79), 10 and 6 out of 11 FN (90.9% vs 54.5%, chi squared=2.0, p=0.15), respectively, and 5 and 11 out of 20 MS (25.0% vs 55.0%, chi squared=2.6, p=0.10), respectively. As for specificity, the false positive markings for masses were on average (per case) 1.60 for CADx and 0.75 for R2, those for calcifications were 1.08 for CADx and 1.77 for R2 and the total false positive markings were 2.68 for CADx and 2.52 for R2. CADx and R2 marked 73 and 63 of 89 negative controls (specificity = 0.18 vs 0.29, chi squared=2.52, p=0.11), respectively. All the radiologists showed greater sensitivity with CAD reading compared to conventional reading. On average, sensitivity with conventional reading was 58.6% (109/186), as against 70.9% (132/186) for CADx or R2 (chi squared=5.71, p=0.016). Sensitivity for FN cases was 71.2% (47/66) with conventional reading, 84.8% (56/66) with CADx (chi squared=2.82, p=0.09) and 80.3% (53/66) for R2 (chi squared=1.03, p=0.30) (CADx vs R2, chi squared=0.21, p=0.64). Sensitivity for MS cases was 51.6% (62/120) for conventional reading, 63.3% (76/120) for CADx (chi squared=2.88, p=0.08) and 65.8% (79/120) for R2 (chi squared=4.40, p=0.03) (CADx vs R2, chi squared=0.07, p=0.78). The recall rates were 18.1% (97/534) for conventional reading, 29.7% (159/534) for CADx (chi squared=5.72, p=0.01) and 24.3% (130/534) for R2 (chi squared=10.11, p=10-5) (CADx vs R2, chi squared=3.71, p=0.05). Double reading was significantly more sensitive than conventional reading (chi squared=29.6, p=10-6), CADx (chi squared=5.33, p=0.02) and R2 (chi squared=5.33, p=0.02). The recall rate for double reading was significantly higher than for conventional reading (chi squared=21.5, p=10-6) whereas no significant difference was detected when compared to CADx (chi squared=0.16, p=0.68) or R2 (chi squared=3.4, p=0.06). CONCLUSIONS: Despite using different algorithms, the two CAD systems exhibit comparable levels of diagnostic accuracy and a similar positive impact on sensitivity when used as an aid to conventional reading. Single reading with either CAD system is as specific but not as sensitive to double independent reading: its use as an alternative to double reading cannot be recommended and should be investigated further by means of controlled prospective studies.     

The assessment of the impact of a double reading by expert readers in a mass mammographic study

PURPOSE: To evaluate the role of double reading of screening mammograms by expert radiologists. MATERIAL AND METHODS: We analyzed the results of independent readings of a proficiency test of screening mammography (140 cases, 32 cancers) performed by four expert radiologists. Double reading was simulated by matching the four original readings in 6 possible combinations. The impact of double reading over single reading was evaluated in terms of increased sensitivity and increased recall rate. RESULTS: Of 32 carcinomas 22, 6, or 4 were identified by 4, 3, or 2 readers, respectively. Of 108 cases negative for cancer a recall for further investigations was suggested by 4, 3, 2, 1 or no reader(s) in 3, 3, 9, 14, or 79 cases, respectively. Inter-reader diagnostic repeatability was good (k = 0.65). Single readers achieved an average sensitivity of 89% (range 87.5-90.6%) and an average recall rate of 12.2% (range 7.4-16.6%). Simulated double reading achieved an average increase in sensitivity of 8.8% (range 6.2-10.95%) and an average increase of recall rate of 6.2% (range 3.8-8.3%). CONCLUSIONS: Even though the relative increase of recall rate is relevant (+53.2%), the corresponding gain in sensitivity justifies the use of double reading, which was confirmed to be worthwhile also when expert radiologists are involved. This study confirms the opportunity of adopting double reading as a routine procedure in mammographic screening.     

Comparison of independent double readings and computer-aided diagnosis (CAD) for the diagnosis of breast calcifications

RATIONALE AND OBJECTIVES: The aim of the study is to compare independent double readings by radiologists and computer-aided diagnosis (CAD) in diagnostic interpretation of mammographic calcifications. MATERIALS AND METHODS: Ten radiologists independently interpreted 104 mammograms containing clustered microcalcifications. Forty-six of these were malignant and 58 were benign at biopsy. Radiologists read the images with and without a computer aid by using a counterbalanced study design. Sensitivity and specificity were calculated from observer biopsy recommendations, and receiver operating characteristic (ROC) curves were computed from their diagnostic confidence ratings. Unaided double-reading sensitivity and specificity values were derived post hoc by using three different objective rules and an additional rule of simulated-optimal double reading that assumed that consultations for resolving two radiologists' different independent diagnoses always produce the correct clinical recommendation. ROC curves of unaided double readings were obtained according to the literature. RESULTS: Single reading without computer aid yielded 74% sensitivity and 32% specificity, whereas CAD reading yielded 87% sensitivity and 42% specificity and appeared on a higher ROC curve (P < .0001). Three methods of formulating independent double readings generated sensitivities between 59% and 89%, specificities between 50% and 13%, and operating points that moved essentially along the average unaided single-reading ROC curve. ROC curves of unaided independent double readings showed small, statistically insignificant improvement over those of unaided single readings. Results of the simulated-optimal double reading were similar to CAD: 89% sensitivity and 50% specificity. CONCLUSION: Independent double readings of mammographic calcifications may not improve diagnostic performance. CAD reading improves diagnostic performance to an extent approaching the maximum possible performance.     

Single reading with computer-aided detection and double reading of screening mammograms in the United Kingdom National Breast Screening Program

PURPOSE: To retrospectively determine if the use of a computer-aided detection (CAD) system can improve the performance of single reading of screening mammograms to match that of double reading in the United Kingdom. MATERIALS AND METHODS: Local research ethics committee approval was obtained; informed consent was not required. This study included a sample of 10 267 mammograms obtained in women aged 50 years or older who underwent routine screening at one of two breast screening centers in 1996. Mammograms that were double read in 1996 were randomly allocated to be re-read by eight different radiologists using CAD. The cancer detection and recall rates from double reading and single reading with CAD were compared. Statistical significance and confidence intervals were calculated with the McNemar test to account for the matched nature of the data. RESULTS: Single reading with CAD led to a cancer detection rate that was significantly (P = .02) higher than that achieved with double reading: 6.5% more cancers were detected by means of single reading with CAD than by means of double reading. However, the recall rate was higher for single reading with CAD than for double reading (8.6% vs 6.5%, respectively; P < .001). This was equivalent to relative increases of 15% and 32% in the cancer detection and recall rates, respectively. CONCLUSION: Single reading with CAD leads to an improved cancer detection rate and an increased recall rate.     

Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection

PURPOSE: To compare the performance of radiologists and of a computer-aided detection (CAD) algorithm for pulmonary nodule detection on thin-section thoracic computed tomographic (CT) scans. MATERIALS AND METHODS: The study was approved by the institutional review board. The requirement of informed consent was waived. Twenty outpatients (age range, 15-91 years; mean, 64 years) were examined with chest CT (multi-detector row scanner, four detector rows, 1.25-mm section thickness, and 0.6-mm interval) for pulmonary nodules. Three radiologists independently analyzed CT scans, recorded the locus of each nodule candidate, and assigned each a confidence score. A CAD algorithm with parameters chosen by using cross validation was applied to the 20 scans. The reference standard was established by two experienced thoracic radiologists in consensus, with blind review of all nodule candidates and free search for additional nodules at a dedicated workstation for three-dimensional image analysis. True-positive (TP) and false-positive (FP) results and confidence levels were used to generate free-response receiver operating characteristic (ROC) plots. Double-reading performance was determined on the basis of TP detections by either reader. RESULTS: The 20 scans showed 195 noncalcified nodules with a diameter of 3 mm or more (reference reading). Area under the alternative free-response ROC curve was 0.54, 0.48, 0.55, and 0.36 for CAD and readers 1-3, respectively. Differences between reader 3 and CAD and between readers 2 and 3 were significant (P < .05); those between CAD and readers 1 and 2 were not significant. Mean sensitivity for individual readings was 50% (range, 41%-60%); double reading resulted in increase to 63% (range, 56%-67%). With CAD used at a threshold allowing only three FP detections per CT scan, mean sensitivity was increased to 76% (range, 73%-78%). CAD complemented individual readers by detecting additional nodules more effectively than did a second reader; CAD-reader weighted kappa values were significantly lower than reader-reader weighted kappa values (Wilcoxon rank sum test, P < .05). CONCLUSION: With CAD used at a level allowing only three FP detections per CT scan, sensitivity was substantially higher than with conventional double reading.     

Single reading with computer-aided detection performed by selected radiologists in a breast cancer screening program

Objectives

To assess the impact of shifting from a standard double reading plus arbitration protocol to a single reading by experienced radiologists assisted by computer-aided detection (CAD) in a breast cancer screening program.

Methods

This was a prospective study approved by the ethics committee. Data from 21,321 consecutive screening mammograms in incident rounds (2010–2012) were read following a single reading plus CAD protocol and compared with data from 47,462 consecutive screening mammograms in incident rounds (2004–2010) that were interpreted following a double reading plus arbitration protocol. For the single reading, radiologists were selected on the basis of the appraisement of their previous performance.

Results

Period 2010–2012 vs. period 2004–2010: Cancer detection rate (CDR): 6.1‰ (95% confidence interval: 5.1–7.2) vs. 5.25‰; Recall rate (RR): 7.02% (95% confidence interval: 6.7–7.4) vs. 7.24% (selected readers before arbitration) and vs. 3.94 (all readers after arbitration); Predictive positive value of recall: 8.69% vs. 13.32%. Average size of invasive cancers: 14.6 ± 9.5 mm vs. 14.3 ± 9.5 mm. Stage: 0 (22.3/26.1%); I (59.2/50.8%); II (19.2/17.1%); III (3.1/3.3%); IV (0/1.9%). Specialized breast radiologists performed better than general radiologists.

Conclusions

The cancer detection rate of the screening program improved using a single reading protocol by experienced radiologists assisted by CAD, at the cost of a moderate increase of the recall rate mainly related to the lack of arbitration.     

“CADEAT”: considerations on the use of CAD (computer-aided diagnosis) in mammography

Computer-aided diagnosis (CAD) has been extensively reported to increase sensitivity by about 10% when added to a single reading while increasing recall rate by 12%, and its current use can be safely recommended in clinical practice. CAD has been suggested as a possible alternative to conventional double reading in screening. Uncontrolled comparison is consistent and suggests that CAD is comparable to double reading in incremental cancer detection rate (CAD +10.6%, double reading +9.1%) and possibly better in recall rate (CAD +12.5%, double reading +28.8%). However, controlled studies comparing single reading + CAD to conventional double reading are not consistent and on average suggest a lower cancer detection rate (?5.1%) and a lower recall rate (?9.8%) for CAD. Scientific evidence is not sufficient for a safe recommendation of single reading + CAD as a current alternative to conventional double reading.     

Computer-aided detection versus independent double reading of masses on mammograms

PURPOSE: To evaluate the use of a computer-aided detection (CAD) system (designed for mammographic mass detection) to help improve mass interpretation and to compare CAD results with independent double-reading results. MATERIALS AND METHODS: Screening mammograms from 500 cases were collected; 125 of these cases were screening-detected cancers, and 125 were interval cancers. Previously obtained screening mammograms (ie, prior mammograms) were available in all cases. All mammograms were analyzed by a CAD system, which detected mass regions and assigned a level of (cancer) suspicion to each mass. Ten experienced screening radiologists read the prior mammograms. For independent interpretation with CAD, the suspicion rating assigned to each finding by the radiologist was weighted with the CAD output at the area of the finding. CAD markers on areas that were not reported by the radiologist were not used. Independent double reading was implemented by using a rule to combine the levels of suspicion assigned to findings by two radiologists. Results were evaluated by using localized-response receiver operating characteristic analysis. RESULTS: In a total of 141 cases, there was a visible abnormality at the location of the cancer on the prior mammogram, and 115 of these were classified as mass cases. For prior mammograms that depicted masses, the mean sensitivity of the radiologists, as averaged among the false-positive rates lower than 10%, was 39.4%; this increased by 7.0% with CAD and by 10.5% with double reading. Differences among single, double, and CAD readings were statistically significant (P <.001). CONCLUSION: Although independent double reading yields the best detection performance, the presence and probability of CAD mass markers can improve mammogram interpretation.     

Ovarian cancer: comparison of observer performance for four methods of interpreting CT scans.

PURPOSE: To assess the effects of four interpretative methods on observers' mean sensitivity and specificity by using computed tomography (CT) of ovarian carcinoma as a model. MATERIALS AND METHODS: CT scans in 98 patients with ovarian carcinoma and 49 women who were disease free were retrospectively reviewed by four experienced blinded radiologists to compare single-observer reading, single-observer reading with an anatomic checklist, paired-observer reading (simultaneous double reading), and replicated reading (combination of two independent readings). Confidence level scoring was used to identify three possible disease forms in each patient: extranodal tumor, lymphadenopathy, and ascites. Patient conditions were then categorized as abnormal or normal. RESULTS: There were no significant improvements in sensitivity or specificity for classification of patient conditions as abnormal or normal when comparing single-observer interpretation with single-observer interpretation with a checklist or paired-observer interpretation. Although there was no significant improvement in the mean sensitivity (93% vs 94%) by using the replicated reading method, there was a statistically significant improvement in mean specificity (85% vs 79%) for the replicated readings compared with single-observer interpretations (P < .05). CONCLUSION: Diagnostic aids such as checklists and paired simultaneous readings did not lead to an improved mean observer performance for experienced readers. However, an increase in the mean specificity occurred with replicated readings.     

Improving digital breast tomosynthesis reading time: A pilot multi-reader,multi-case study using concurrent Computer-Aided Detection (CAD)

PurposeEvaluate concurrent Computer-Aided Detection (CAD) with Digital Breast Tomosynthesis (DBT) to determine impact on radiologist performance and reading time.Materials and methodsThe CAD system detects and extracts suspicious masses, architectural distortions and asymmetries from DBT planes that are blended into corresponding synthetic images to form CAD-enhanced synthetic images. Review of CAD-enhanced images and navigation to corresponding planes to confirm or dismiss potential lesions allows radiologists to more quickly review DBT planes. A retrospective, crossover study with and without CAD was conducted with six radiologists who read an enriched sample of 80 DBT cases including 23 malignant lesions in 21 women. Area Under the Receiver Operating Characteristic (ROC) Curve (AUC) compared the readings with and without CAD to determine the effect of CAD on overall interpretation performance. Sensitivity, specificity, recall rate and reading time were also assessed. Multi-reader, multi-case (MRMC) methods accounting for correlation and requiring correct lesion localization were used to analyze all endpoints. AUCs were based on a 0–100% probability of malignancy (POM) score. Sensitivity and specificity were based on BI-RADS scores, where 3 or higher was positive.ResultsAverage AUC across readers without CAD was 0.854 (range: 0.785-0.891, 95% confidence interval (CI): 0.769,0.939) and 0.850 (range: 0.746-0.905, 95% CI: 0.751,0.949) with CAD (95% CI for difference: −0.046,0.039), demonstrating non-inferiority of AUC. Average reduction in reading time with CAD was 23.5% (95% CI: 7.0–37.0% improvement), from an average 48.2 (95% CI: 39.1,59.6) seconds without CAD to 39.1 (95% CI: 26.2,54.5) seconds with CAD. Per-patient sensitivity was the same with and without CAD (0.865; 95% CI for difference: −0.070,0.070), and there was a small 0.022 improvement (95% CI for difference: ‐0.046,0.089) in per-lesion sensitivity from 0.790 without CAD to 0.812 with CAD. A slight reduction in specificity with a −0.014 difference (95% CI for difference: ‐0.079,0.050) and a small 0.025 increase (95% CI for difference: −0.036,0.087) in recall rate in non-cancer cases were observed with CAD.ConclusionsConcurrent CAD resulted in faster reading time with non-inferiority of radiologist interpretation performance. Radiologist sensitivity, specificity and recall rate were similar with and without CAD.     

Computed tomography colonography: feasibility of computer-aided polyp detection in a "first reader" paradigm

OBJECTIVE:: To determine the feasibility of a computer-aided detection (CAD) algorithm as the "first reader" in computed tomography colonography (CTC). METHODS:: In phase 1 of a 2-part blind trial, we measured the performance of 3 radiologists reading 41 CTC studies without CAD. In phase 2, readers interpreted the same cases using a CAD list of 30 potential polyps. RESULTS:: Unassisted readers detected, on average, 63% of polyps > or =10 mm in diameter. Using CAD, the sensitivity was 74% (not statistically different). Per-patient analysis showed a trend toward increased sensitivity for polyps > or =10 mm in diameter, from 73% to 90% with CAD (not significant) without decreasing specificity. Computer-aided detection significantly decreased interobserver variability (P = 0.017). Average time to detection of the first polyp decreased significantly with CAD, whereas total reading case reading time was unchanged. CONCLUSION:: Computer-aided detection as a first reader in CTC was associated with similar per-polyp and per-patient detection sensitivity to unassisted reading. Computer-aided detection decreased interobserver variability and reduced the time required to detect the first polyp.     

Reading protocol for dynamic contrast-enhanced MR images of the breast: sensitivity and specificity analysis

PURPOSE: To prospectively determine sensitivity and specificity of breast magnetic resonance (MR) imaging in a screening and symptomatic population by using independent double reading, with histologic or cytologic results or a minimum 18-month follow-up as the standard. MATERIALS AND METHODS: Informed consent and ethical approval were obtained. Reader performance was analyzed in 44 radiologists at 18 centers from 1541 examinations, including 1441 screening examinations in 638 high-risk women aged 24-51 years (mean, 40.5 years) and 100 examinations in symptomatic women aged 23-81 years (mean, 49.2 years). A screening protocol of dynamic T1-weighted three-dimensional imaging and 0.2 mmol/kg gadolinium-based intravenous contrast agent was used. Logistic and Poisson regressions were used to analyze reader performance in relation to experience. Correlation between readers was determined with kappa statistics. Sensitivity and specificity were analyzed according to reader, field strength, machine type, and histologic results. RESULTS: The proportion of studies with lesions analyzed reduced significantly with reader experience (odds ratio, 0.84 per 6 months; P < .001), and number of regions per lesion analyzed also diminished (incidence rate ratio, 0.98 per 6 months; P = .047). The two readers for each study agreed 87% of the time, with a moderately good kappa statistic of 0.52 (95% confidence interval [CI]: 0.45, 0.58). By taking the reading with the highest score (most likely to be malignant) from each double-read study, sensitivity was 91% (95% CI: 83%, 96%) and specificity was 81% (95% CI: 79%, 83%). Single readings had 7% lower sensitivity (95% CI: 4%, 11%) and 7% higher specificity (95% CI: 6%, 7%). Sensitivity did not differ between MR imager manufacturers or between 1.0- and 1.5-T field strength, but there were significant differences in specificity for machine type (P = .001) and for field strength adjusted for manufacturer (P = .001). Specificity, but not sensitivity, was higher in women younger than 50 years (P = .02). CONCLUSION: Independent double reading by 44 radiologists blinded to mammography results showed sensitivity and specificity acceptable for screening; sensitivity was higher when two readings were used, at the cost of specificity. Interreader correlation was moderately good, and evidence of learning was seen. Equipment manufacturer, field strength, and age affected specificity but not sensitivity.     

Does the accuracy of single reading with CAD (computer-aided detection) compare with that of double reading?: A review of the literature

AIM: To examine current evidence to determine whether the accuracy of single reading with computed-aided detection (CAD) compares with that of double reading. METHODS: We performed a literature review to identify studies where both protocols had been investigated and compared. We identified eight studies that compared single reading with CAD against double reading, of which six reported on comparisons of both sensitivity and specificity. RESULTS: Of the six studies identified, three showed no differences in either sensitivity or specificity. One showed single reading with CAD had a higher sensitivity at the same specificity, another that single reading with CAD had a higher specificity at the same sensitivity. However, one study, in a real-life setting, showed that single reading with CAD had a higher sensitivity but a lower specificity. CONCLUSION: As the majority of the studies were not in a real-life setting, used test sets, lacked sufficient training in the use of CAD and simulated double reading (using a protocol of recall if one suggests), current evidence is therefore limited as to the accuracy, in terms of sensitivity and specificity, of single reading with CAD in comparison with the most common practice in the UK of double reading using a protocol of consensus or arbitration.     

Effect of computer-aided detection on mammographic performance: experimental study on readers with different levels of experience

Purpose: To evaluate the effect of computer-aided detection (CAD) on the reader's performance.

Material and Methods: Four screening radiologists, two novice radiologists, and two residents with no prior experience in CAD read films of 200 women without and with CAD. The films, including 16 screen-detected cancers and 35 cancers “missed” on prior screening, were divided into two rollers: A (free time schedule) and B (prompted time schedule). Reading times were noted. Individual readings without and with CAD were compared, sensitivities and specificities were calculated.

Results: The sensitivity of CAD was 70.6% and specificity 15.8%. In 408 cancer readings, the screeners found 10 and other readers 7 new cancers with the aid of CAD. The screeners changed their opinion four times and others six times from true positive to false negative when CAD was negative. CAD output produced 12 versus 13 new false-positive findings respectively after 2352 readings. CAD did not significantly affect the reader's sensitivities/specificities regardless of the time limit (P = not significant). The use of CAD increased mean time for roller reading from 56 to 63 min (P = 0.053).

Conclusion: Screening radiologists benefited slightly more from CAD than other readers did, but no statistical significant difference was found in personal readings without and with CAD.     

Computed assisted detection of interval breast cancers

OBJECTIVES: To examine interval cancer detection rate for a system of computer assisted detection (CAD) and its influence on radiologists' sensitivity/specificity in a screen-like retrospective review situation. MATERIALS AND METHODS: Three screening radiologists reviewed previous screen images of 59 interval cancers mixed with other screening mammograms (ratio 1:5) and non-mixed. Mixed interval cases were interpreted both without and with aid of CAD. RESULTS: CAD detected a number of 14 interval cancers while the three radiologists detected 17, 12 and 11 without and 16, 10 and 13 with CAD. Although CAD specificity was low (38%) no reduction in radiologists' specificity occurred using CAD (73%, 82% and 89% without and 78%, 90% and 92% with CAD). Non-mixed reading increased radiologists' detection rate to 21, 17 and 19 interval cancers respectively. CONCLUSION: Despite sufficiently high sensitivity for CAD alone no increase in radiologist sensitivity (or decrease in specificity) occurred with CAD. Improving CAD specificity, with unaffectedly high sensitivity, should make radiologists more inclined to revise interpretations according to CAD. The potential sensitivity increase, noted when using CAD as a double reader, could be realised in this way.     

Computer-aided detection in the United Kingdom National Breast Screening Programme: prospective study

PURPOSE: To evaluate prospectively the recall and cancer detection rates with and without computer-aided detection (CAD) in the United Kingdom National Health Service Breast Screening Programme. MATERIALS AND METHODS: The study had appropriate ethics committee approval. Informed consent was not required; however, patients were informed that their mammograms might be used in research efforts, and all patients agreed to participate. Mammograms obtained in 6111 women (mean age, 58.4 years) undergoing routine screening every 3 years were analyzed with a CAD system. Mammograms were independently double read. Twelve readers participated. Readers recorded an initial evaluation, viewed the CAD prompts, and recorded a final evaluation. Recall to assessment was decided after arbitration. Sensitivities were calculated for single reading, single reading with CAD, and double reading, as a proportion of the total number of cancers detected by using double reading with CAD. RESULTS: A total of 62 cancers were detected in 61 women. CAD prompted 51 (84%) of 61 radiographically detected cancers. Of 12 cancers missed on single reading, nine were correctly prompted; however, seven prompts were overruled by the reader. Sensitivity of single reading was 90.2% (95% confidence interval [CI]: 83.0%, 95.0%), single reading with CAD was 91.5% (95% CI: 85.0%, 96.0%), and double reading without CAD was 98.4% (95% CI: 91.0%, 100%). Cancer detection rate was 1%. Recall to assessment rate was 6.1%, with an increase of 5.8% because of CAD. Average time required, per reader, to read a case was 25 seconds without CAD and 45 seconds with CAD. CONCLUSION: CAD increases sensitivity of single reading by 1.3%, whereas double reading increases sensitivity by 8.2%.     

Potential contribution of computer-aided detection to the sensitivity of screening mammography

PURPOSE: To determine the false-negative rate in screening mammography, the capability of computer-aided detection (CAD) to identify these missed lesions, and whether or not CAD increases the radiologists' recall rate. MATERIALS AND METHODS: All available screening mammograms that led to the detection of biopsy-proved cancer (n = 1,083) and the most recent corresponding prior mammograms (n = 427) were collected from 13 facilities. Panels of radiologists evaluated the retrospectively visible prior mammograms by means of blinded review. All mammograms were analyzed by a CAD system that marks features associated with cancer. The recall rates of 14 radiologists were prospectively measured before and after installation of the CAD system. RESULTS: At retrospective review, 67% (286 of 427) of screening mammography-detected breast cancers were visible on the prior mammograms. At independent, blinded review by panels of radiologists, 27% (115 of 427) were interpreted as warranting recall on the basis of a statistical evaluation index; and the CAD system correctly marked 77% (89 of 115) of these cases. The original attending radiologists' sensitivity was 79% (427 of [427 + 115]). There was no statistically significant increase in the radiologists' recall rate when comparing the values before (8.3%) with those after (7.6%) installation of the CAD system. CONCLUSION: The original attending radiologists had a false-negative rate of 21% (115 of [427 + 115]). CAD prompting could have potentially helped reduce this false-negative rate by 77% (89 of 115) without an increase in the recall rate.     

The potential contribution of a computer-aided detection system for lung nodule detection in multidetector row computed tomography

RATIONALE AND OBJECTIVES: We sought to evaluate the potential benefits of a computer-aided detection (CAD) system for detecting lung nodules in multidetector row CT (MDCT) scans. METHODS: A CAD system was developed for detecting lung nodules on MDCT scans and was applied to the data obtained from 15 patients. Two chest radiologists in consensus established the reference standard. The nodules were categorized according to their size and their relationship to the surrounding structures (nodule type). The differences in the sensitivities between an experienced chest radiologist and a CAD system without user interaction were evaluated using a chi2 analysis. The differences in the sensitivities also were compared in terms of the nodule size and the nodule type. RESULTS: A total of 309 nodules were identified as the reference standard. The sensitivity of a CAD system (81%) was not significantly different from that of a radiologist (85%; P > 0.05). The sensitivities of the CAD system for detecting nodules < or = 5 mm in diameter as well as detecting isolated nodules were higher than those of a radiologist (83% vs. 75%, P > 0.05; 93% vs. 76%, P < 0.001). The sensitivities of a radiologist for detecting nodules >5 mm and the nodules attached to other structures were higher than those of a CAD system (98% vs. 79%, P < 0.001; 91% vs. 71%, P < 0.001). There were 28.8 false-positive results of CAD per CT study. CONCLUSION: The CAD system developed in this study performed the nodule detection task in different ways to that of a radiologist in terms of the nodule size and the nodule type, which suggests that the CAD system can play a complementary role to a radiologist in detecting nodules from large CT data sets.     

Effect of high sensitivity in a computerized scheme for detecting extremely subtle solitary pulmonary nodules in chest radiographs: observer performance study

RATIONALES AND OBJECTIVES: This study investigated the effect of a high sensitivity in computer-aided diagnosis (CAD) for detecting lung nodules in chest radiographs when extremely subtle cases were presented to radiologists. MATERIAL AND METHODS: The chest radiographs used in this study consisted of 36 normal images and 54 abnormals containing solitary lung nodules, of which 25 were extremely subtle and 29 were very subtle. Receiver operating characteristic analysis for detecting lung nodules was performed without and with CAD. The levels of CAD output were simulated with a hypothetical ideal performance of 100% sensitivity, but with three or four false positives per image. Six radiologists participated in an observer study in which cases were interpreted first without and then with the use of CAD. RESULTS: The average A(z) values for radiologists without and with CAD were 0.682 and 0.808, respectively. The performance of radiologists was improved significantly when high sensitivity was used (P = .0003). However, the radiologists were not able to recognize some extremely subtle nodules (5 of 54 nodules by all radiologists), even with the correct CAD output; these nodules were then considered as non-actionable. None of 306 computer-false positives was incorrectly regarded as a nodule by all radiologists, but 63 false positives were incorrectly identified by one or more radiologists. CONCLUSION: The accuracy of radiologists in the detection of some extremely subtle solitary pulmonary nodules can be improved significantly when the sensitivity of a CAD scheme can be made to be at an extremely high level. However, all of the six radiologists failed to identify some nodules (about 10%), even with the correct output of the CAD.