Temporal subtraction for detection of solitary pulmonary nodules on chest radiographs: evaluation of a commercially available computer-aided diagnosis system

PURPOSE: To evaluate the usefulness of a commercially available computer-aided diagnosis (CAD) system that incorporates temporal subtraction for the detection of solitary pulmonary nodules on chest radiographs by readers with different levels of experience. MATERIALS AND METHODS: Sixty pairs of chest radiographs in 30 patients with newly detected solitary pulmonary nodules and 30 normal cases, all confirmed with serial chest computed tomography (CT), were obtained from screen-film or digital radiographic systems and were digitized (spatial resolution, 0.171 mm/pixel). Temporal subtraction images were produced with an iterative image-warping technique. Five chest radiologists and five residents evaluated both image sets for solitary nodules: set A, current and prior radiographs with temporal subtraction images, and set B, current and prior radiographs only. Assessment was performed with receiver operating characteristic (ROC) analysis of the images on a monitor (pixel size, 1,280 x 1,024) equipped with the system. The reading time needed by each reader was recorded in each case. RESULTS: For the chest radiologists, no statistically significant difference was found between set A (area under the ROC curve [A(z)] = 0.934) and set B (A(z) = 0.964). For the residents, however, observer performance in set A (A(z) = 0.907) was superior to that in set B (A(z) = 0.855) (P <.05). For both groups, the mean reading time per case for set A (chest radiologists, 16.7 seconds; residents, 15.7 seconds) was significantly (P <.05) shorter than that for set B (chest radiologists, 20.4 seconds; residents, 26.2 seconds). CONCLUSION: For the detection of solitary pulmonary nodules, the CAD system with temporal subtraction can promote efficiency for established chest radiologists and improvement in accuracy for less experienced readers.     

Evaluation of usefulness of color digital summation radiography for solitary pulmonary nodules on chest radiographs

Purpose The aim of this study was to evaluate the usefulness of novel color digital summation radiography (CDSR) for detecting solitary pulmonary nodules on chest radiographs by observers with different levels of experience. Materials and methods A total of 30 healthy controls and 30 patients with newly detected solitary pulmonary nodules were evaluated. Six radiologists and five residents evaluated three image sets: set A, current and prior radiographs only; set B, set A with temporal subtraction images; and set C, set A with CDSR. The observers were asked to rate each image set using a continuous rating scale. In addition, the reading time required for each set was recorded. Results The radiologists showed no significant differences in the mean A_z value between set A, set B, and set C. However, the residents showed significant differences between set A and set B and between set A and set C. In addition, for set B and set C, the mean reading time per case of all readers was significantly shorter than that for set A. Conclusion The detection capability of observers with little experience is comparable to that of experienced observers when reading radiographs with temporal subtraction images or with CDSR. The usefulness of CDSR is comparable to that of temporal subtraction.     

Effect of temporal subtraction technique on the diagnosis of primary lung cancer with chest radiography

PURPOSE: The purpose of this study was to assess the diagnostic accuracy of the temporal subtraction technique in the detection of primary lung cancers by readers with different levels of experience. METHODS: Previous and current chest radiographs from 40 patients with histologically proven lung cancer and 40 controls were studied. Temporal subtraction images were produced using an automated digital subtraction technique. We evaluated the effect of temporal subtraction images in the diagnosis of lung cancer with chest radiographs via an observer performance study with the use of receiver operating characteristic analysis. Six experienced radiologists and six residents participated as observers. RESULTS: Observer performance for all observers was superior when temporal subtraction images were used (mean Az value increased from 0.764 to 0.836, p=0.0006). Although the average Az value for residents increased significantly, from 0.707 to 0.795 (p=0.0038), the average Az value for experienced radiologists increased only from 0.821 to 0.878 (n.s.). CONCLUSION: In conclusion, the temporal subtraction technique clearly improves diagnostic accuracy for the detection of primary lung cancer. The results indicated that the use of temporal subtraction images was more beneficial for the residents than for the experienced radiologists. This method would compensate to some extent for experience-dependent diagnostic accuracy in the detection of lung cancer.     

Improved detection of lung nodules by using a temporal subtraction technique

PURPOSE: To evaluate the effect of a temporal subtraction technique for digital chest radiography with regard to the accuracy of detection of lung nodules. MATERIALS AND METHODS: Twenty solitary lung nodules smaller than 30 mm in diameter, including 10 lung cancers and 10 benign nodules, were used. The nodules were grouped subjectively according to their subtlety. For non-nodular cases, 20 nodules without perceptible interval changes were selected. All chest radiographs were obtained by using a computed radiographic system, and temporal subtraction images were produced by using a program developed at the University of Chicago. The effect of the temporal subtraction image was evaluated by using an observer performance study, with use of receiver operating characteristic analysis. RESULTS: Observer performance with temporal subtraction images was substantially improved (A(z) = 0.980 and 0.958), as compared with that without temporal subtraction images (A(z) = 0.920 and 0.825) for the certified radiologists and radiology residents, respectively. The temporal subtraction technique clearly improved diagnostic accuracy for detecting lung nodules, especially subtle cases. CONCLUSION: The temporal subtraction technique is useful for improving detection accuracy for peripheral lung nodules on digital chest radiographs.     

Evaluation of the usefulness of color digital summation radiography in temporally sequential digital radiographs: a phantom study

Purpose The purpose of this study was to assess the usefulness of color digital summation radiography (CDSR) for detection of nodules on chest radiographs by observers with different levels of experience. Materials and methods A total of 30 radiographs of chest phantoms with abnormalities and 30 normal ones were arranged at random. Set A was conventional radiographs only. Set B consisted of both conventional radiographs and CDSR images, which were colored with magenta. Five chest radiologists and five residents evaluated both image sets on a TFT monitor. The observers were asked to rate each image set using a continuous rating scale. The reading time for each set was also recorded. Results In set A, the performance of chest radiologists was significantly superior to that of the residents (P < 0.05). However, in set B, there was no significant difference in the performance of the chest radiologists and the residents. In both observer groups, the mean reading time per case in set B was significantly shorter than that in set A (P < 0.01). Conclusion By using CDSR, the detection capability of observers with little experience improves and is comparable to that of experienced observers. Moreover, the reading time becomes much shorter using CDSR.     

Improved detection of focal pneumonia by chest radiography with bone suppression imaging

Objective

To evaluate radiologists’ ability to detect focal pneumonia by use of standard chest radiographs alone compared with standard plus bone-suppressed chest radiographs.

Methods

Standard chest radiographs in 36 patients with 46 focal airspace opacities due to pneumonia (10 patients had bilateral opacities) and 20 patients without focal opacities were included in an observer study. A bone suppression image processing system was applied to the 56 radiographs to create corresponding bone suppression images. In the observer study, eight observers, including six attending radiologists and two radiology residents, indicated their confidence level regarding the presence of a focal opacity compatible with pneumonia for each lung, first by use of standard images, then with the addition of bone suppression images. Receiver operating characteristic (ROC) analysis was used to evaluate the observers’ performance.

Results

The mean value of the area under the ROC curve (AUC) for eight observers was significantly improved from 0.844 with use of standard images alone to 0.880 with standard plus bone suppression images (P?<?0.001) based on 46 positive lungs and 66 negative lungs.

Conclusion

Use of bone suppression images improved radiologists’ performance for detection of focal pneumonia on chest radiographs.

Key Points

? Bone suppression image processing can be applied to conventional digital radiography systems. ? Bone suppression imaging (BSI) produces images that appear similar to dual-energy soft tissue images. ? BSI improves the conspicuity of focal lung disease by minimizing bone opacity. ? BSI can improve the accuracy of radiologists in detecting focal pneumonia.     

ROC analysis of detection of interval changes in interstitial lung diseases on digital chest radiographs using the temporal subtraction technique

PURPOSE: To demonstrate the clinical usefulness of a temporal subtraction technique for the detection of interval changes in various interstitial lung diseases on digital chest radiographs. MATERIALS AND METHODS: One hundred pairs of chest radiographs in 34 patients (63 with and 37 without interval changes) with various interstitial lung diseases were selected. All cases were confirmed by serial chest computed tomography (CT) and ascertained by radiologists. All chest radiographs were obtained with a computed radiography (CR) system, and temporal subtraction images were produced with an iterative image-warping technique. Four radiologists and two thoracic physicians provided confidence levels for interval changes in interstitial lung diseases with and without temporal subtraction. Their performances with and without temporal subtraction were evaluated by means of receiver operating characteristic (ROC) analysis using a sequential test. RESULTS: The area under the ROC curve (Az) values of six observers obtained with and without temporal subtraction were 0.90 and 0.78, respectively. Results showed that the detection of interval changes in interstitial lung diseases was significantly improved by the use of temporal subtraction images compared with CR images alone(P = 0.002). Furthermore, the high detection rate was achieved with temporal subtraction images regardless of the subtlety and location of interval changes. CONCLUSION: Temporal subtraction improved the diagnostic accuracy of radiologists in detecting interval changes in interstitial lung diseases on chest radiographs. It was also useful for cases of multiple interval changes.     

Detection of lung nodules on digital chest radiographs: potential usefulness of a new contralateral subtraction technique

PURPOSE: To evaluate the potential usefulness of a contralateral subtraction technique developed for radiologists' performance in the detection of subtle lung nodules on chest radiographs. MATERIALS AND METHODS: Fifty chest radiographs (25 normal and 25 abnormal with a subtle lung nodule) that were digitized with a 0.175-mm pixel size and 4,096 gray levels were used. Twelve radiologists (10 attending and two residents) participated in observer tests and read both original and contralateral subtraction images with a sequential testing method. Radiologists' performance was evaluated by means of receiver operating characteristic analysis with use of a continuous rating scale. The beneficial and detrimental effects of the contralateral subtraction technique on the radiologists' performance were also evaluated. RESULTS: The area under the receiver operating characteristic curve values obtained without and with contralateral subtraction images were 0.926 and 0.962, respectively. Results indicated that the contralateral subtraction images significantly (P <.05) improved diagnostic accuracy, particularly for radiologists with limited experience. CONCLUSION: The contralateral subtraction technique can assist radiologists in the correct identification of subtle lung nodules on chest radiographs.     

Digital storage phosphor chest radiography: an ROC study of the effect of 2K versus 4K matrix size on observer performance

PURPOSE: To compare observer performance in the detection of abnormalities on 1,760 x 2,140 matrix (2K) and 3,520 x 4,280 matrix (4K) digital storage phosphor chest radiographs. MATERIALS AND METHODS: One hundred sixty patients who underwent dedicated computed tomography (CT) of the thorax were prospectively recruited into the study. Posteroanterior and lateral computed radiographs of the chest were acquired in each patient and printed in 2K and 4K formats. Six radiologists independently analyzed the hard-copy images and scored the presence of parenchymal (opacities 2 cm, and subtle interstitial), mediastinal, and pleural abnormalities on a five-point confidence scale. With CT as the reference standard, observer performance tests were carried out by using receiver operating characteristic (ROC) analysis. RESULTS: Analysis of averaged observer performance showed 2K and 4K images were equally effective in detection of all three groups of abnormalities. In the detection of the three subtypes of parenchymal abnormalities, there were no significant differences in averaged performance between the 2K and 4K formats (area below ROC curve [A(z)] values: opacities 2 cm, 0.86 +/-.025 and 0.85 +/- 0.030; subtle interstitial abnormalities, 0.73 +/- 0.041 and 0.72 +/- 0.041). Averaged performance in detection of mediastinal and pleural abnormalities was equivalent (A(z) values: mediastinal, 0.70 +/- 0.046 and 0.73 +/- 0.033; pleural, 0.85 +/- 0.032 and 0.86 +/- 0.033). CONCLUSION: Observer performance in detection of parenchymal, mediastinal, and pleural abnormalities was not significantly different on 2K and 4K storage phosphor chest radiographs.     

ROC Analysis of Detection of Metastatic Pulmonary Nodules on Digital Chest Radiographs with Temporal Subtraction

RATIONALE AND OBJECTIVES: The authors' purpose was to evaluate the effect of temporal subtraction on digital chest radiographs in the detection of metastatic pulmonary nodules. MATERIALS AND METHODS: The study included 21 cases with metastatic pulmonary nodule and 21 cases without metastatic nodule. Eleven radiologists, including eight residents and three certified radiologists, provided their confidence levels for the presence or absence of pulmonary nodules without and with temporal subtraction. Their performances without and with temporal subtraction were evaluated by means of receiver operating characteristic analysis with both independent and sequential tests. RESULTS: For the independent test, the radiologists' Az (area under the receiver operating characteristic curve) values were 0.871 without and 0.954 with temporal subtraction, compared with 0.882 and 0.955, respectively, for the sequential test. Diagnosis accuracy was significantly improved with the use of temporal subtraction. There was no significant difference in Az values between the independent and sequential tests. CONCLUSION: Temporal subtraction is useful in the detection of metastatic pulmonary nodules, and this technique augments the value of digital chest radiography.     

Improved detection of lung cancer arising in diffuse lung diseases on chest radiographs using temporal subtraction

RATIONALE AND OBJECTIVES: The purpose of this study was to evaluate the usefulness of temporal subtraction for the detection of lung cancer arising in pneumoconiosis, idiopathic pulmonary fibrosis, and pulmonary emphysema. MATERIALS AND METHODS: Fifteen cases of lung cancer arising in diffuse lung diseases, including three cases of pneumoconiosis, six of idiopathic pulmonary fibrosis, and six of pulmonary emphysema, were evaluated. Pathologic proof was obtained by surgery or transbronchial lung biopsy. The average interval between previous and current radiographs was 356 days (range, 31-947 days). All chest radiographs were obtained with a computed radiography system, and temporal subtraction images were produced by subtracting of a previous image from a current one with a nonlinear image-warping technique. The effect of the temporal subtraction image was evaluated by observer performance study with receiver operating characteristic analysis. RESULTS: The average observer performance with temporal subtraction was significantly improved (Az = 0.935) compared with that without temporal subtraction (Az = 0.857, P < .0001). CONCLUSION: The temporal subtraction technique is useful for the detection of lung cancer arising in pneumoconiosis, idiopathic pulmonary fibrosis, and pulmonary emphysema.     

深度卷积网络骨抑制技术对孤立性肺结节诊断效能的研究

目的评估新型骨抑制技术—深度卷积网络骨抑制成像(deep bone suppression imaging,deepBSI)对孤立性肺结节(solitary pulmonary nodule,SPN)的诊断效能,并与数字化X线片(digital radiograph,DR)、双能量减影技术(dual energy substraction,DES)进行对比分析。方法收集我院2016年12月~2017年9月拍摄标准胸部正位片247例(114例诊断SPN,133例无结节)。3位低年资及3位高年资医师按DR图像→DR+deepBSI图像→DR+DES图像的阅片顺序,分别在三组图像上标出结节可能位置并评分,进行Z检验,应用ROC曲线分析三种检查方法对SPN的诊断效能。结果6位医师,DR、deepBSI、DES三种检查方法诊断SPN的ROC曲线下面积分别约0.715、0.804、0.800,deepBSI、DES诊断效能均优于DR(P<0.05)。当结节与肋骨重叠面积>50%时,deepBSI、DES二种方法相比于DR诊断SPN的诊断效能越好。结论deepBSI、DES诊断效能均优于DR,有助于肺结节的检出,与肋骨重叠面积大的结节优势更显著,deepBSI、DES诊断敏感性及特异性始终相似。     

Usefulness of computerized method for lung nodule detection on digital chest radiographs using similar subtraction images from different patients

Purpose

The purpose of this study is to evaluate the usefulness of a novel computerized method to select automatically the similar chest radiograph for image subtraction in the patients who have no previous chest radiographs and to assist the radiologists’ interpretation by presenting the “similar subtraction image” from different patients.

Materials and methods

Institutional review board approval was obtained, and the requirement for informed patient consent was waived. A large database of approximately 15,000 normal chest radiographs was used for searching similar images of different patients. One hundred images of candidates were selected according to two clinical parameters and similarity of the lung field in the target image. We used the correlation value of chest region in the 100 images for searching the most similar image. The similar subtraction images were obtained by subtracting the similar image selected from the target image. Thirty cases with lung nodules and 30 cases without lung nodules were used for an observer performance test. Four attending radiologists and four radiology residents participated in this observer performance test.

Results

The AUC for all radiologists increased significantly from 0.925 to 0.974 with the CAD (P = .004). When the computer output images were available, the average AUC for the residents was more improved (0.960 vs. 0.890) than for the attending radiologists (0.987 vs. 0.960).

Conclusion

The novel computerized method for lung nodule detection using similar subtraction images from different patients would be useful to detect lung nodules on digital chest radiographs, especially for less experienced readers.     

Detection of lung cancer on the chest radiograph: a study on observer performance

STUDY OBJECTIVES: to study the validity and observers consistency in the detection of lung cancer on the chest radiograph. MATERIALS AND METHODS: the chest radiographs of 100 clinical cases were interpreted by 14 observers. The radiographs were obtained from 30 patients with initially missed but histopathologically proven non-small cell lung cancer (NSCLC), 35 patients with other cardiopulmonary diseases and 35 patients with no abnormalities. The observers consisted of ten experienced radiologists, two-experienced chest physicians and two residents in radiology. All observers were unaware of the study design. The validity and observer consistency was determined for each observer. RESULTS: the mean sensitivity and specificity of the ten radiologists were 0.36 and 0.90. For the two chest physicians, the mean sensitivity and specificity were 0.29 and 0.96. For the two residents in radiology, mean sensitivity and specificity were 0.25 and 0.94. The mean interobserver kappa and mean intraobserver kappa for the radiologists were 0.38 and 0.54. For the two chest physicians, the mean interobserver kappa was 0.43, while the intraobserver kappa was 0.59. For the two residents in radiology, mean interobserver kappa was 0.35 and the intraobserver kappa was 0.42. There was no significant relation between the consistency parameters and validity parameters. The interobserver and intraobserver kappa values showed good correlation. CONCLUSION: the validity of the chest radiograph and observers consistency in the detection of nodular lung cancer varies widely. The level of experience is likely to influence the diagnostic performance.     

Computer-aided diagnosis in the detection of temporal changes in sequential chest radiographs

To aid radiologists in the diagnosis of screening chest radiographs, a temporal subtraction technique using digital image processing was developed. The accurate image registration of two sequential images enables us to detect even subtle changes in the "difference image" between them. In this report, a new method based on matching "lung markings" is introduced. Twenty-nine pairs of sequential posteroanterior chest radiographs with and without temporal changes were selected from cases examined with the computed radiography system. Image registration was employed, with the local matching of "lung markings" in previous and current radiographs. Observer performance tests were carried out by eight radiologists, with and without the "difference image." Observer performance tests with the temporal subtraction image showed that six of eight observers diagnosed them with higher sensitivity (mean, 43.9% vs. 55.3%) and a comparable false positive response. Mean area under the AFROC (alternative free-response receiver operating characteristics) curve also improved from 0.596 to 0.647, a statistically significant difference. The subtraction image using this registration technique improved diagnostic accuracy for subtle temporal changes.     

Effect of temporal subtraction images on radiologists' detection of lung cancer on CT: results of the observer performance study with use of film computed tomography images

RATIONALE AND OBJECTIVES: To evaluate the effect of temporal subtraction images on the radiologists' detection of early primary lung cancer in computed tomography (CT) scans. MATERIALS AND METHODS: Fourteen cases with primary lung cancer and 16 normal cases were used for this study from a database of low-dose CT images, which were obtained from a lung cancer screening program in Nagano, Japan. Images were obtained with a single-detector helical CT scanner using 10 mm collimation and 2:1 pitch. Each case had both previous and current CT scans. Temporal subtraction images were obtained by subtracting the warped previous images from the current images. Seven radiologists, including four attendings and three residents, provided their confidence levels for the presence or absence of lung cancers with use of film CT images without and with temporal subtraction images. Receiver operating characteristic analysis was used to compare their performance without and with temporal subtraction images. RESULTS: The mean Az values (area under the receiver operating characteristic curve) of seven observers without and with temporal subtraction images were 0.868 and 0.930, respectively. Diagnostic accuracy was significantly improved by using temporal subtraction images (P = .007). Temporal subtraction images were especially useful when a nodule was present near the pulmonary hilum, where radiologists tended to overlook it. CONCLUSION: The temporal subtraction technique can significantly improve the sensitivity and specificity for detection of lung cancer on CT scans.     

Detection of simulated chest lesions by using soft-copy reading: comparison of an amorphous silicon flat-panel-detector system and a storage-phosphor system

PURPOSE: To compare observer performance by using soft-copy images produced by an amorphous silicon flat-panel-detector system and a storage-phosphor system for the detection of simulated chest lesions. MATERIALS AND METHODS: To test the diagnostic performance of these two systems, four types of simulated lesions (nodules, micronodules, lines, and reticular opacities) were superimposed over an anthropomorphic chest phantom. Digital chest radiographs were acquired with amorphous silicon flat-panel-detector (3-K [K = 1,000] matrix, 12 bits) and storage-phosphor radiography (4-K matrix, 10 bits). Six board-certified radiologists evaluated soft-copy images on a high-resolution video monitor (2,048 x 2,560 x 8 bits). A total of 14,400 observations were analyzed in terms of receiver operating characteristics. RESULTS: Average performance in terms of nodule detection was significantly better (P <.05) with the flat-panel-detector system than with the storage-phosphor system. For micronodules, lines, and reticular opacities, no significant detection differences in averaged performance were found between the two detector systems. CONCLUSION: In the evaluation of soft-copy images, the amorphous silicon detector system appears to be superior to the storage-phosphor system for the detection of pulmonary nodules.     

Detection of lung cancer on radiographs: receiver operating characteristic analyses of radiologists', pulmonologists', and anesthesiologists' performance

PURPOSE: To compare and quantify, by means of receiver operating characteristic (ROC) and localization ROC analyses, the performance of radiologists, pulmonologists, and anesthesiologists (residents and staff) in the detection of missed lung cancer. MATERIALS AND METHODS: The study was approved by the institutional review board, and informed consent was not required or obtained for review of radiographs. A set of 60 posteroanterior chest radiographs was presented to 36 observers: 12 radiologists, 12 pulmonologists, and 12 anesthesiologists. Each of these three observer categories included six residents and six staff. Thirty of the radiographs each depicted one lung cancer that was overlooked at prospective image interpretation; the other 30 were normal radiographs matched for age and smoking history. Observers were asked to rate their degree of suspicion concerning the presence of lung cancer by using a visual analog scale and to point out the zone of suspicion on a schematic of the lung. These data were used to generate combined ROC-localization ROC curves and to assess performance. Intraobserver consistency was evaluated by using intraclass correlation coefficients and weighted kappa statistics. RESULTS: Areas under the ROC curves indicated better performance for radiologists and pulmonologists compared with anesthesiologists (P < .002) and for staff compared with residents (P < .022). Performance was lower for all categories of observers when localization ROC curves were used. Radiologists and staff pulmonologists showed a higher degree of confidence in the assessment of normality than did other categories of physicians. Intraobserver consistency was poor. CONCLUSION: Experienced readers showed better ability to distinguish normality from abnormality. Combined ROC and localization ROC analyses gave a more reliable quantification of observer performance than did ROC analysis alone.     

Computed radiography versus screen-film radiography: detection of pulmonary edema in a rabbit model that simulates neonatal pulmonary infiltrates.

PURPOSE: To determine if computed radiography is equivalent to screen-film radiography in depicting pulmonary edema and to determine if radiation exposure can be reduced with computed radiography while maintaining equivalent diagnostic accuracy for pulmonary edema. MATERIALS AND METHODS: Oleic acid was intravenously injected into three rabbits at each of four doses: 0, 0.02, 0.04, and 0.06 mL/kg. Two hours later, chest computed radiographs and screen-film radiographs were obtained at 60 kVp and 1.1 mAs. Additional computed radiographs were obtained after reducing milliampere seconds or by reducing milliampere seconds and increasing the kilovolt peak, which reduced bone marrow exposure by up to 20%. The presence of pulmonary opacities, "truth," was established by the wet-dry weight ratio and by chest computed tomography (CT). The radiographs were masked and randomized. Four observers rated the images for the presence of parenchymal opacities with a dichotomous score and judged the quality of the radiographs on a scale from 1 (worst) to 6 (best). Cochran Q tests and McNemar tests were used to analyze the differences in paired comparisons. Image quality was evaluated with logistic regression analysis. RESULTS: There was no significant difference between truth and observer ability to detect opacity for either modality or for any exposure (P > .05). There was no significant difference between computed radiography and screen-film radiography for image quality (P > .05). CONCLUSION: Computed radiography is equivalent to screen-film radiography in the detection of pulmonary edema. Radiation exposure reduction of 20% can be achieved without affecting pulmonary edema detection or image quality.     

Computer-aided diagnosis in chest radiography: results of large-scale observer tests at the 1996-2001 RSNA scientific assemblies.

Since 1996, computer-aided diagnosis (CAD) schemes have been presented as interactive demonstrations on computer workstations at each scientific assembly of the Radiological Society of North America. The schemes involved (a) detection of pulmonary nodules, (b) temporal subtraction, (c) detection of interstitial lung disease, (d) differential diagnosis of interstitial lung disease, and (e) distinction between benign and malignant pulmonary nodules on chest radiographs. Large-scale observer tests were carried out to examine how radiologists can benefit from CAD systems. Observer performance was evaluated by analysis of receiver operating characteristic (ROC) curves. The statistical significance of the difference between the areas under the ROC curves without and with CAD was analyzed with the Student t test. In all of the tests, the diagnostic accuracy of the radiologists in total improved significantly when CAD was used. This result provides additional evidence that CAD has the potential to improve the performance of radiologists in their decision-making process in interpreting chest radiographs. Copyright RSNA, 2003.