Improved detection of lung nodules on chest radiographs using a commercial computer-aided diagnosis system

OBJECTIVE: The aim of this study was to evaluate the usefulness of a new commercially available computer-aided diagnosis (CAD) system with an automated method of detecting nodules due to lung cancers on chest radiograph. MATERIALS AND METHODS: For patients with cancer, 45 cases with solitary lung nodules up to 25 mm in diameter (nodule size range, 8-25 mm in diameter; mean, 18 mm; median, 20 mm) were used. For healthy patients, 45 cases were selected on the basis of confirmation on chest CT. All chest radiographs were obtained with a computed radiography system. The CAD output images were produced with a newly developed CAD system, which consisted of an image server including CAD software called EpiSight/XR. Eight radiologists (four board-certified radiologists and four radiology residents) participated in observer performance studies and interpreted both the original radiographs and CAD output images using a sequential testing method. The observers' performance was evaluated with receiver operating characteristic analysis. RESULTS: The average area under the curve value increased significantly from 0.924 without to 0.986 with CAD output images. Individually, the use of CAD output images was more beneficial to radiology residents than to board-certified radiologists. CONCLUSION: This CAD system for digital chest radiographs can assist radiologists and has the potential to improve the detection of lung nodules due to lung cancer.     

Improved detection of pulmonary nodules on energy-subtracted chest radiographs with a commercial computer-aided diagnosis software: comparison with human observers

Objective  

To retrospectively analyze the performance of a commercial computer-aided diagnosis (CAD) software in the detection of pulmonary nodules in original and energy-subtracted (ES) chest radiographs.     

Tumour detection rate of a new commercially available computer-aided detection system

The aim of this study was to determine the tumour detection rate and false positive rate of a new mammographic computer-aided detection system (CAD) in order to assess its clinical usefulness. The craniocaudal and oblique images of 150 suspicious mammograms from 150 patients that were histologically proven to be malignant were analysed using the Second Look CAD (CADx Medical Systems, Quebec, Canada). Cases were selected randomly using the clinic's internal tumour case sampler. Correct marking of the malignant lesion in at least one view was scored as a true positive. Marks not at the location of the malignant lesion were scored as false positives. In addition, mammograms with histologically proven benign masses ( n=50) and microcalcifications ( n=50), as well as 100 non-suspicious mammograms, were scanned in order to determine the value of false-positive marks per image. The 150 mammograms included 94 lesions that were suspicious due to masses, 26 due to microcalcifications and 30 showed both signs of malignancy. The overall sensitivity was 90.0% (135 of 150). Sensitivity on subsets of the data was 88.7% (110 of 124) for suspicious masses (MA) and 98.2% (55 of 56) for microcalcifications. Eight of 14 false-negative cases were large lesions. The overall false-positive rate was observed as 0.28 and 0.97 marks per image of microcalcifications and masses, respectively. The lowest false-positive rates for microcalcifications and MA were observed in the cancer subgroup, whereas the highest false-positive rates were scored in the benign but mammographically suspicious subgroups, respectively. The new CAD system shows a high tumour detection rate, with approximately 1.3 false positive marks per image. These results suggest that this system might be clinically useful as a second reader of mammograms. The system performance was particularly useful for detecting microcalcifications.     

New methods for using computer-aided detection information for the detection of lung nodules on chest radiographs

Objective:

To investigate two new methods of using computer-aided detection (CAD) system information for the detection of lung nodules on chest radiographs. We evaluated an interactive CAD application and an independent combination of radiologists and CAD scores.

Methods:

300 posteroanterior and lateral digital chest radiographs were selected, including 111 with a solitary pulmonary nodule (average diameter, 16 mm). Both nodule and control cases were verified by CT. Six radiologists and six residents reviewed the chest radiographs without CAD and with CAD (ClearRead +Detect™ 5.2; Riverain Technologies, Miamisburg, OH) in two reading sessions. The CAD system was used in an interactive manner; CAD marks, accompanied by a score of suspicion, remained hidden unless the location was queried by the radiologist. Jackknife alternative free response receiver operating characteristics multireader multicase analysis was used to measure detection performance. Area under the curve (AUC) and partial AUC (pAUC) between a specificity of 80% and 100% served as the measure for detection performance. We also evaluated the results of a weighted combination of CAD scores and reader scores, at the location of reader findings.

Results:

AUC for the observers without CAD was 0.824. No significant improvement was seen with interactive use of CAD (AUC = 0.834; p = 0.15). Independent combination significantly improved detection performance (AUC = 0.834; p = 0.006). pAUCs without and with interactive CAD were similar (0.128), but improved with independent combination (0.137).

Conclusion:

Interactive CAD did not improve reader performance for the detection of lung nodules on chest radiographs. Independent combination of reader and CAD scores improved the detection performance of lung nodules.

Advances in knowledge:

(1) Interactive use of currently available CAD software did not improve the radiologists'' detection performance of lung nodules on chest radiographs. (2) Independently combining the interpretations of the radiologist and the CAD system improved detection of lung nodules on chest radiographs.Chest radiography can be considered the workhorse of the radiology department. It is being used for the detection and diagnosis of multiple diseases, including lung nodules, which may represent early lung cancer. Since a chest radiograph is a two-dimensional image, overprojection of multiple anatomical structures is inevitable. This so-called anatomical noise substantially impedes interpretation of chest radiographs. Multiple studies have shown that a substantial amount of lung cancers are missed, ranging from 19% to 26%,^1,2 and even up to 90%.^3–5 More recent studies have shown that the problem of missing lung nodules is still present with the most modern digital radiographic technology.^6,7 Abnormalities can be missed as a result of inadequate search, perception errors or interpretation errors. It has been stated that interpretation by the radiologist is the most important factor for missing lung cancer on chest radiographs.^8,9To reduce miss rates, computer-aided detection (CAD) systems have been developed. Thus far, all studies dealing with chest radiography apply CAD as a second reader to the radiologist, meaning that the CAD marks are made available only after the radiologist has made a primary review. It remains the reader''s discretion to accept or disregard the CAD marks. Results of these studies were contradictory: some found an increased accuracy for the detection of lung nodules,^10–12 whereas other studies reported an increase in sensitivity only at the expense of loss in specificity.^13–16 One problem ameliorating the potential of CAD is the radiologist''s limited ability to reliably discriminate between true-positive (TP) and false-positive (FP) CAD marks.We therefore decided to explore alternative methods of using CAD information. First, we used CAD interactively. In the interactive mode, CAD marks remained hidden unless the radiologist queried a position in the image by clicking with the mouse on that location. If a CAD mark was present in this location, it was shown to the radiologist together with a score of suspicion. Such an interactive CAD system had been shown to be beneficial in chest radiography in an observer study that only used non-radiologists.¹⁷ Second, we computed a mathematical combination of reader and CAD scores. With this method, observers did not need to view the CAD marks at all during their reading of the images, but a mathematical combination of the reader and the CAD scores was computed afterwards. Both methods have been reported to outperform the use of CAD as a second reader for lesion detection in mammograms.^18–20The purpose of this observer study was to test the impact of these two alternative methods of using CAD information on nodule detection on chest radiographs. To optimize baseline performance without CAD, digitally bone-suppressed images (BSIs) were added to the original chest radiographs. BSIs have been shown to improve accuracy for the detection of focal lesions on chest radiographs;^21–24 a further increase in detection performance beyond that of BSIs by adding CAD has also been documented.²⁵     

Neural network based detection of pulmonary nodules on chest radiographs

PURPOSE: We investigated the capabilities of an artificial neural network-based Computer-Aided Diagnosis (CAD) system in improving early detection of pulmonary nodules on chest radiographs. MATERIAL AND METHODS: We used a data-set of 145 digitized chest films. Two different radiologists read the radiographs to detect the sites of possible nodules. The system uses two neural networks trained on a training-set of 100 radiographs selected from the data-set. The first network is used to focus attention on the sites of potential nodules while the second calculates the likeliness of nodule presence in ROIs. The clinical test was performed on 45 more radiographs from the training-set, but different from those in the data-set, which were positive for both benign and malignant nodules. These latter plain films showed 65 nodular lesions which differed by shape and acquisition technique. RESULTS: Sensitivity was 89% in all radiographs while specificity, evaluated by ROI, and accuracy, were 98%. CONCLUSIONS: There are potential limitations in nodule detection on plain radiographs. Some of them are operator-dependent, such as nonsystematic investigation, lesion underestimation, and poor reading, and some are technique-dependent, such as X-ray beam/tube, low voltage, patient positioning, focus-film distance and development process. CADs may contribute to improving detection of pulmonary nodules because the false-negative rate is decreased and sensitivity consequently increased. The high sensitivity and specificity rates of neural networks encourage further trials on wider data-sets to help the radiologist in the early detection of pulmonary nodules.     

应用计算机辅助检测系统对观察者在数字化胸片肺结节检测中的训练

摘要目的评估短期反馈能否帮助观察者提高在数字化胸片中运用计算机辅助系统(CAD)检测肺结节的能力。方法140例胸部平片(56例CT证实存在孤立性肺结节,84例为阴性对照)分为4组,每组各35例;每组均有6名观察者按不同顺序阅片。在有和没有CAD(IQQA-Chest,EDDA Technology)辅助情况下分别对病变的存在、部位、诊断的可靠性进行计分。观察者在每组阅片后获得独立的反馈。     

Evaluation of usefulness of color digital summation radiography for solitary pulmonary nodules on chest radiographs

Purpose The aim of this study was to evaluate the usefulness of novel color digital summation radiography (CDSR) for detecting solitary pulmonary nodules on chest radiographs by observers with different levels of experience. Materials and methods A total of 30 healthy controls and 30 patients with newly detected solitary pulmonary nodules were evaluated. Six radiologists and five residents evaluated three image sets: set A, current and prior radiographs only; set B, set A with temporal subtraction images; and set C, set A with CDSR. The observers were asked to rate each image set using a continuous rating scale. In addition, the reading time required for each set was recorded. Results The radiologists showed no significant differences in the mean A_z value between set A, set B, and set C. However, the residents showed significant differences between set A and set B and between set A and set C. In addition, for set B and set C, the mean reading time per case of all readers was significantly shorter than that for set A. Conclusion The detection capability of observers with little experience is comparable to that of experienced observers when reading radiographs with temporal subtraction images or with CDSR. The usefulness of CDSR is comparable to that of temporal subtraction.     

计算机辅助检测系统在数字化胸片肺结节检出中的应用价值

目的评价计算机辅助检测（computer-aided detection，CAD）系统在数字化胸片肺结节检出中的应用价值及其对放射科医师的辅助作用。方法选取数字化胸片328例。由2名专家组医师应用IQQATM-Chest系统阅读具有结节样阴影的胸片，2人意见达成一致后标记结节的位置和大小并保存标记结果，将标记结果作为金标准来评估CAD系统检测肺结节的能力。由8名不同年资的放射科医师首先独立阅读具有结节样阴影的数字X线摄影（DR）胸片并保存诊断结果，然后再应用CAD系统阅读胸片，将最终结果存入CAD系统。应用受试者操作特征曲线（ROC）和配对t检验来分析放射科医师应用CAD系统前后在肺结节检测能力上的差异。结果在100例DR胸片中，金标准结节151个，CAD系统肺结节检测敏感性为78．1％（118．0／151），低年资放射科医师不用和应用CAD系统时，肺结节的检测敏感性分别为62．4％（94．2／151）和77．4％（116．8／151），ROC曲线下面积分别为0．769和0．836，二者之间的差异具有统计学意义（P〈0．01）；高年资放射科医师不用和应用CAD系统时，肺结节的检测敏感性分别为73．8％（111．5／151）和76．2％（115．0／151），ROC曲线下面积分别为0．820和0．827，二者之间的差异无统计学意义（P〉0．05）。结论CAD系统能够辅助放射科医师提高肺小结节的检测敏感性，对低年资医师的帮助更大。     

Temporal subtraction for the detection of hazy pulmonary opacities on chest radiography

OBJECTIVE: The purpose of this study was to evaluate the accuracy of temporal subtraction with a commercially available computer-assisted diagnosis system for the detection of multifocal hazy pulmonary opacities on chest radiographs, which are sometimes difficult to detect directly on chest radiographs. MATERIALS AND METHODS: Thirty healthy patients and 30 patients with new multifocal hazy pulmonary opacities that were confirmed by serial chest CT examinations were evaluated with and without temporal subtraction images. Chest radiographs were taken from either film-screen or digital radiography images and were digitized with a spatial resolution of 0.171 mm per pixel. Temporal subtraction images were produced by an iterative image-warping technique. We designed an observer performance study in which observers (six chest radiologists and four residents) indicated their confidence level for the presence or absence of hazy pulmonary opacities on two sets of images, current and previous radiographs only (set A), and current and previous radiographs with temporal subtraction images (set B). Receiver operating characteristic curves were generated. RESULTS: For chest radiologists, observer performance with set B (with temporal subtraction images; A(z) = 0.947) was superior to that with set A (without temporal subtraction images; A(z) = 0.916) (p < 0.05). For residents, no statistically significant difference was found between sets A and B. CONCLUSION: The temporal subtraction technique clearly improves diagnostic accuracy for the detection of multifocal hazy pulmonary opacities on chest radiographs, especially when the observers are experienced chest radiologists who have sufficient skill to evaluate the patient's condition as revealed on the images.     

Gray-scale inversion radiographic display for the detection of pulmonary nodules on chest radiographs

The purpose of this study was to investigate gray-scale inversion in nodule detection on chest radiography. Simulated nodules were superimposed randomly onto normal chest radiographs. Six radiologists interpreted 144 chest radiographs during three reading sessions: traditional presentation, inverted gray-scale, and a choice session allowing use of traditional and gray-scale inverted views. Sensitivity and specificity were used to assess accuracy based on presence or absence of a nodule. Gray-scale inversion and choice display sessions resulted in significantly higher nodule detection specificity and decreased sensitivity compared to traditional display. Gray-scale inversion may decrease false-positive nodule findings during chest X-ray interpretation.     

Automatic detection of pulmonary nodules at spiral CT: clinical application of a computer-aided diagnosis system

The aim of this study was to evaluate a computer-aided diagnosis (CAD) workstation with automatic detection of pulmonary nodules at low-dose spiral CT in a clinical setting for early detection of lung cancer. Eighty-eight consecutive spiral-CT examinations were reported by two radiologists in consensus. All examinations were reviewed using a CAD workstation with a self-developed algorithm for automatic detection of pulmonary nodules. The algorithm is designed to detect nodules with diameters of at least 5 mm. A total of 153 nodules were detected with at least one modality (radiologists in consensus, CAD, 85 nodules with diameter < 5 mm, 68 with diameter > or = 5 mm). The results of automatic nodule detection were compared to nodules detected with any modality as gold standard. Computer-aided diagnosis correctly identified 26 of 59 (38%) nodules with diameters > or = 5 mm detected by visual assessment by the radiologists; of these, CAD detected 44% (24 of 54) nodules without pleural contact. In addition, 12 nodules > or = 5 mm were detected which were not mentioned in the radiologist's report but represented real nodules. Sensitivity for detection of nodules > or = 5 mm was 85% (58 of 68) for radiologists and 38% (26 of 68) for CAD. There were 5.8+/-3.6 false-positive results of CAD per CT study. Computer-aided diagnosis improves detection of pulmonary nodules at spiral CT and is a valuable second opinion in a clinical setting for lung cancer screening despite of its still limited sensitivity.     

Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance

PURPOSE: To evaluate the effect of computer-aided diagnosis (CAD) on radiologists' detection of pulmonary nodules. MATERIALS AND METHODS: Fifty chest computed tomographic (CT) examination cases were used. The mean nodule size was 0.81 cm +/- 0.60 (SD) (range, 0.3-2.9 cm). Alternative free-response receiver operating characteristic (ROC) analysis with a continuous rating scale was used to compare the observers' performance in detecting nodules with and without use of CAD. Five board-certified radiologists and five radiology residents participated in an observer performance study. First they were asked to rate the probability of nodule presence without using CAD; then they were asked to rate the probability of nodule presence by using CAD. RESULTS: For all radiologists, the mean areas under the best-fit alternative free-response ROC curves (Az) without and with CAD were 0.64 +/- 0.08 and 0.67 +/- 0.09, respectively, indicating a significant difference (P <.01). For the five board-certified radiologists, the mean Az values without and with CAD were 0.63 +/- 0.08 and 0.66 +/- 0.09, respectively, indicating a significant difference (P <.01). For the five resident radiologists, the mean Az values without and with CAD were 0.66 +/- 0.04 and 0.68 +/- 0.04, respectively, indicating a significant difference (P =.02). At observer performance analyses, there were no significant differences in Az values obtained either without (P =.61) or with (P =.88) CAD between the board-certified radiologists and the residents. For all radiologists, in the detection of pulmonary nodules 1.0 cm in diameter or smaller, the mean Az values without and with CAD were 0.60 +/- 0.11 and 0.64 +/- 0.11, respectively, indicating a significant difference (P <.01). CONCLUSION: Use of the CAD system improved the board-certified radiologists' and residents' detection of pulmonary nodules at chest CT.     

Enhanced characterization of solid solitary pulmonary nodules with Bayesian analysis-based computer-aided diagnosis

The aim of this study was to prospectively assess the accuracy gain of Bayesian analysis-based computer-aided diagnosis (CAD) vs human judgment alone in characterizing solitary pulmonary nodules (SPNs) at computed tomography (CT). The study included 100 randomly selected SPNs with a definitive diagnosis. Nodule features at first and follow-up CT scans as well as clinical data were evaluated individually on a 1 to 5 points risk chart by 7 radiologists, firstly blinded then aware of Bayesian Inference Malignancy Calculator (BIMC) model predictions. Raters’ predictions were evaluated by means of receiver operating characteristic (ROC) curve analysis and decision analysis. Overall ROC area under the curve was 0.758 before and 0.803 after the disclosure of CAD predictions (P = 0.003). A net gain in diagnostic accuracy was found in 6 out of 7 readers. Mean risk class of benign nodules dropped from 2.48 to 2.29, while mean risk class of malignancies rose from 3.66 to 3.92. Awareness of CAD predictions also determined a significant drop on mean indeterminate SPNs (15 vs 23.86 SPNs) and raised the mean number of correct and confident diagnoses (mean 39.57 vs 25.71 SPNs). This study provides evidence supporting the integration of the Bayesian analysis-based BIMC model in SPN characterization.