Digital and conventional chest imaging: a modified ROC study of observer performance using simulated nodules

A modified receiver operating characteristic (ROC) study was performed in which five readers were asked to locate multiple nodules on images of an anthropomorphic phantom obtained with a prototype digital radiographic chest unit and with a conventional chest unit. Results indicate that when nodules were projected over the lungs, a significantly greater number (significant at the 5% level) were identified on conventional radiographs, whereas for nodules projected over the mediastinum, the digital images were notably superior (difference significant at the 2% level). An error analysis of the multiple nodule problem and pseudo-ROC curves are presented. The modified ROC study does not suffer from the positional ambiguity inherent in most ROC studies and is efficient in acquiring data.     

Effects of optimization and image processing in digital chest radiography: an ROC study with an anthropomorphic phantom.

A digital system for chest radiography based on a large image intensifier was compared to a conventional film-screen system. The digital system was optimized with regard to spatial and contrast resolution and dose. The images were digitally processed for contrast and edge enhancement. A simulated pneumothorax and two simulated nodules were positioned over the lungs and the mediastinum of an anthropomorphic phantom. Observer performance was evaluated with ROC analysis. Five observers assessed the processed digital images and the conventional full-size radiographs. The time spent viewing the full-size radiographs and the digital images was recorded. For the simulated pneumothorax, the results showed perfect performance for the full-size radiographs and detectability was high also for the processed digital images. No significant difference in the detectability of the simulated nodules was seen between the two imaging systems. The results for the digital images showed a significantly improved detectability for the nodules in the mediastinum as compared to a previous ROC study where no optimization and image processing was available. No significant difference in detectability was seen between the former and the present ROC study for small nodules in the lung. No difference was seen in the time spent assessing the conventional full-size radiographs and the digital images. The study indicates that processed digital images produced by a large image intensifier are equal in image quality to conventional full-size radiographs for low-contrast objects such as nodules.     

Digital chest radiography with a large image intensifier. Evaluation of diagnostic performance and patient exposure

A digital system for chest radiography based on a large image intensifier was compared with a conventional film-screen system. The diagnostic performance was evaluated with special reference to the digital monitor images with a modified version of receiver operating characteristic (ROC) analysis--free response ROC (FROC) analysis--on a chest equivalent phantom. Measurements of spatial resolution and energy imparted were also performed. The detectability of low-contrast objects as well as spatial resolution was better for the full-size film-screen radiographs than for both the digital monitor images and the 100 mm photofluorograms. The image-intensifier system has a potential for considerable dose savings in relation to the conventional technique provided that fluoroscopy is excluded in the positioning of the patients.     

Nodule detection with and without a chest image

The detectability of nodules displayed on uniform backgrounds was compared with their detectability on chest images by using receiver operating characteristic (ROC) curves. The images were displayed using a digital television system. Two conditions, chest image and uniform background, were compared at three different levels of added random noise. The viewing conditions were made as similar as possible by setting the luminance of the uniform background equal to the luminance of the lung in the chest image. Nodule detectability was significantly lower for the chest image than for the uniform background at each level of added random noise. The anatomic structure of the chest image interferes with the structural integrity of the nodule image resulting in lower detection performance.     

Comparison of chest radiography,chest digital tomosynthesis and low dose MDCT to detect small ground-glass opacity nodules: an anthropomorphic chest phantom study

Objectives

The purpose of this study was to evaluate the diagnostic performance of chest radiography (CXR), chest digital tomosynthesis (DT) and low dose multidetector computed tomography (LDCT) for the detection of small pulmonary ground-glass opacity (GGO) nodules, using an anthropomorphic chest phantom.

Methods

Artificial pulmonary nodules were placed in a phantom and a total of 40 samples of different nodule settings underwent CXR, DT and LDCT. The images were randomly read by three experienced chest radiologists. Free-response receiver-operating characteristics (FROC) were used.

Results

The figures of merit for the FROC curves averaged for the three observers were 0.41, 0.37 and 0.76 for CXR, DT and LDCT, respectively. FROC analyses revealed significantly better performance of LDCT over CXR or DT for the detection of GGO nodules (P?P?=?0.73).

Conclusion

The diagnostic performance of DT for the detection of pulmonary small GGO nodules was not significantly different from that of CXR, but LDCT performed significantly better than both CXR and DT. DT is not a suitable alternative to CT for small GGO nodule detection, and LDCT remains the method of choice for this purpose.

Key Points

? For GGO nodule detection, DT was not significantly different from CXR. ? DT is not a suitable alternative to CT for GGO nodule detection. ? LDCT is the method of choice for GGO nodule detection.     

Effects of improved contrast on lung-nodule detection. A clinical ROC study

We evaluated the effects of unsharp masking and highly efficient scatter rejection on film-screen chest radiographs of cancer patients. Unsharp masking significantly improved the detectability of lung nodules and visibility of anatomic structures in poorly penetrated areas of the chest. Highly efficient scatter rejection by an improved antiscatter grid provided only modest additional benefits. The study supports the conclusion that nodule detection in poorly penetrated areas on conventional chest radiographs is limited primarily by display contrast, whereas in the well-penetrated lung fields it is limited primarily by confusing background structures, rather than inadequate contrast. A method for analyzing clinical nodule detection data by transforming the FROC data to ROC coordinates also is demonstrated.     

Screening for tracheobronchial diseases with digital storage phosphor radiography]

Digital storage phosphor radiography (FCR: Fuji computed radiography) has a wide dynamic range and unique postprocessing capabilities. This study was designed to test whether chest imaging with FCR and its image processing would increase the accuracy of and confidence in the diagnosis of tracheobronchial abnormalities. In a phantom study, the performance of digital images having the appearance of a conventional chest radiograph was compared with that of a conventional system in detecting simulated tracheobronchial nodules. The digital images of lower kilovoltage (ROC area = 0.647 +/- 0.035) were equivalent to the conventional radiographs (ROC area = 0.620 +/- 0.028). On the other hand, nodule detectability was significantly improved in the digital images of higher kilovoltage (ROC area = 0.826 +/- 0.020). The author also compared the impact of five postprocessing algorithms (standard image, wide latitude image, enhanced image, reversed image, and subtraction image). ROC analysis indicated that the default standard image (ROC area = 0.826 +/- 0.020) was as good as an image with a linear rather than a sigmoid gradation curve (ROC area = 0.843 +/- 0.020), an image with strong enhancement of high frequencies (ROC area = 0.804 +/- 0.020), and an image with reversed gray scale polarity (ROC area = 0.775 +/- 0.015). Therefore these specific algorithms had no effect on the detection of tracheobronchial nodules. However, digital subtraction soft tissue images (ROC area = 0.961 +/- 0.030) were significantly better than the control images. Clinical study also indicated that subtraction images improve diagnostic accuracy in tracheobronchial diseases.     

Clinical evaluation of pulmonary nodules with dual-exposure dual-energy subtraction chest radiography

PURPOSE: The purpose of this study was to assess the effect of dual-exposure dual-energy (DE) subtraction chest radiography with flat-panel detector. MATERIALS AND METHODS: One hundred patients underwent dual-exposure DE subtraction chest radiography and chest CT for evaluation of pulmonary nodules. Fifty-two patients with pulmonary nodules and 48 patients with normal lungs were selected for receiver operating characteristic (ROC) curve analysis. Ten radiologists who were unaware of the CT results evaluated chest radiography alone and chest radiography with DE subtraction images in the detection of pulmonary nodules. For each radiologist, we calculated the areas under the ROC curve (Az) for chest radiography alone and chest radiography with DE subtraction images. RESULTS: The average detectability of dual-exposure DE subtraction chest radiography was statistically significantly higher than that of chest radiography without subtraction images (mean Az value increased from 0.784 to 0.815, p<0.001). CONCLUSION: Dual-exposure DE subtraction chest radiography improves diagnostic accuracy of pulmonary nodules.     

Improved pulmonary nodule detection with scanning equalization radiography

The potential for improved pulmonary nodule detection with scanning equalization radiography (SER) was evaluated by means of observer performance testing during the interpretation of posteroanterior conventional radiographs and SER images of an anthropomorphic chest phantom with simulated nodules. A test set of 200 conventional and 200 SER radiographs of phantoms containing either one nodule or none was interpreted by four radiologists attempting to detect a nodule and indicate a confidence value. Their ability to detect nodules positioned over the lung was slightly improved with SER compared with conventional radiography (sensitivity, .56 vs .70); for nodules over the mediastinum or diaphragmatic areas, it was much improved (sensitivity, .29 vs .64). The results were also analyzed with receiver-operating characteristic methods, which revealed a significant improvement in lesion detect-ability over the thicker body parts with SER images. The capability of equalized chest radiographs to provide improved lesion detectability suggests that SER may set a new standard for film-based chest radiography and have a large clinical application.     

Digital chest radiography with a large image intensifier. An ROC study with an anthropomorphic phantom

The diagnostic performance of two systems for chest radiography was studied. One system was based on a large image intensifier, the other was a conventional film-screen system. The images from the image intensifier were studied either on a digital TV screen or on 100 mm photofluorograms. Receiver operating characteristic curve analysis was performed on images of an anthropomorphic chest phantom. Low-contrast MMAP (methyl methacrylate polymer) nodules and simulated vessels were positioned over the parenchymal and the mediastinal region of the phantom. Five observers assessed the digital monitor images, photofluorograms, and conventional full-size radiograms. The results showed a significantly superior detectability for the full-size radiograms over the digital monitor images both in the parenchyma and in the mediastinum. No significant difference was found between photofluorograms and digital images.     

"Single-exposure" dual energy digital radiography in the detection of pulmonary nodules and calcifications

We studied the detectability of mineralized and non-mineralized simulated pulmonary nodules with dual energy digital radiography. "Soft tissue" and "bone" images (pixel size = 0.2 mm, 10 bits deep) were obtained with subtraction image processing after a single simultaneous exposure (100 kVp, 8 mAs, 17 mR skin exposure dose) of two storage phosphors with an interleaved 0.9 mm copper wafer. Three classes of paraffin-based nodules (0.5 to 3.0 cm) of varying mineral concentration (0, 120 and 190 mg/cm3 K2HPO4) were randomly positioned on the chest wall of two healthy volunteers to simulate calcified and non-calcified nodules. The average receiver operating characteristics (ROC) area of six readers (n = 2880 observations) showed that digital "bone" images (ROC area: 0.77 +/- 0.03) were significantly better (P less than 0.04) than conventional radiographs (OC Film, Lanex medium screens, 141 kVp, 19 mR skin exposure dose) (ROC area: 0.71 +/- 0.05) in detecting calcification in nodules. The unsubtracted digital images of lower kilovoltage were not superior to the 141 kVp conventional radiographs in a subgroup of two readers (ROC area: 0.73 +/- 0.02). Digital "soft tissue" images were equivalent to conventional chest radiographs in detecting soft tissue pulmonary nodules (ROC areas: 0.92 +/- 0.04 and 0.92 +/- 0.05, respectively.     

Quantifying the clinical relevance of a laboratory observer performance paradigm

Objective Laboratory observer performance measurements, receiver operating characteristic (ROC) and free-response ROC (FROC) differ from actual clinical interpretations in several respects, which could compromise their clinical relevance. The objective of this study was to develop a method for quantifying the clinical relevance of a laboratory paradigm and apply it to compare the ROC and FROC paradigms in a nodule detection task. Methods The original prospective interpretations of 80 digital chest radiographs were classified by the truth panel as correct (C=1) or incorrect (C=0), depending on correlation with additional imaging, and the average of C was interpreted as the clinical figure of merit. FROC data were acquired for 21 radiologists and ROC data were inferred using the highest ratings. The areas under the ROC and alternative FROC curves were used as laboratory figures of merit. Bootstrap analysis was conducted to estimate conventional agreement measures between laboratory and clinical figures of merit. Also computed was a pseudovalue-based image-level correctness measure of the laboratory interpretations, whose association with C as measured by the area (rAUC) under an appropriately defined relevance ROC curve, is as a measure of the clinical relevance of a laboratory paradigm. Results Low correlations (e.g. κ=0.244) and near chance level rAUC values (e.g. 0.598), attributable to differences between the clinical and laboratory paradigms, were observed. The absolute width of the confidence interval was 0.38 for the interparadigm differences of the conventional measures and 0.14 for the difference of the rAUCs. Conclusion The rAUC measure was consistent with the traditional measures but was more sensitive to the differences in clinical relevance. A new relevance ROC method for quantifying the clinical relevance of a laboratory paradigm is proposed.     

Digital slot-scan charge-coupled device radiography versus AMBER and Bucky screen-film radiography: comparison of image quality in a phantom study

PURPOSE: To evaluate the image quality and performance of a chest digital radiography system and to compare this with the image quality and performance of advanced multiple-beam equalization radiography (AMBER) and Bucky screen-film radiography systems. MATERIALS AND METHODS: The chest digital radiography system is a digital charge-coupled device (CCD) chest imaging unit that uses slot-scan technology. A contrast-detail test object was used in combination with a phantom that simulates the primary and scatter transmission for the lungs and mediastinum. Twelve phantom images were obtained with each modality (ie, CCD digital radiography and AMBER and Bucky screen-film radiography) and were judged by six observers. CCD digital radiography soft-copy reading was compared with AMBER hard-copy reading. To measure image quality, contrast-detail curves were constructed from the observer data. The Wilcoxon signed rank test was used for statistical analysis. RESULTS: For the lung configuration, contrast-detail curves showed lower threshold depth for hard-copy images obtained with CCD digital radiography than for those obtained with Bucky screen-film radiography. For hard-copy images, the difference between contrast-detail curves for CCD digital radiography and those for Bucky screen-film radiography was statistically significant (P < .006). No significant difference was found between CCD digital radiography and AMBER for hard-copy images obtained in either the lung or mediastinum configuration. For the lung configuration, a lower threshold depth was observed for CCD digital radiography soft-copy reading than for AMBER hard-copy reading, with significantly different contrast-detail curves for CCD digital radiography soft copy and AMBER hard copy (P < .006). No significant difference was found between either system for the mediastinum configuration. CONCLUSION: Contrast-detail curves indicate that image quality for the CCD chest system provides a digital alternative to AMBER and Bucky screen-film radiography.     

Advanced multiple beam equalization radiography (AMBER) in the detection of diffuse lung disease

To evaluate the effects of scanning equalization radiography (SER) on the detection of diffuse lung disease a clinical comparison between an Advanced Multiple Beam Equalization Radiography (AMBER) unit and conventional chest radiography was performed. Even though the overall detection of focal pulmonary lesions with the AMBER unit has been shown to be significantly higher than with conventional radiography because of the improved demonstration of the costophrenic and retrocardiac regions, the utility of AMBER in the demonstration of diffuse lung disease has not been established. Twenty-one patients with diffuse lung disease (fibrosing alveolitis or sarcoidosis) and six patients with no pulmonary disease had high kVp frontal and lateral chest radiographs on both an AMBER unit and a conventional chest stand. The pooled results of five observers using Receiver Operating Characteristic (ROC) analysis indicate that there is a slight improvement but no statistically significant difference in observer performance between AMBER (Area under the ROC curve AZ = 0.934) and conventional radiography (AZ = 0.868) in the task of detecting diffuse lung disease.     

Detectability of simulated pulmonary nodules on chest radiographs: Comparison between irradiation side sampling indirect flat-panel detector and computed radiography

Objective

To compare the detectability of simulated pulmonary nodules on chest radiographs between an irradiation side sampling indirect flat-panel detector (ISS-FPD) and computed radiography (CR).

Materials and methods

This study was an observer performance study. Simulated pulmonary nodules of 8 mm in diameter were superimposed on an anthropomorphic chest phantom. Chest radiographs were acquired under 2 exposure levels (4 and 3.2 mAs) with the ISS-FPD and the CR. Six thoracic radiologists evaluated all 40 images (10 patterns × 2 different exposure doses × 2 different systems) for the presence or absence of a lesion over each of 12 defined areas on a 3-megapixel monochrome liquid-crystal display. Receiver operating characteristic (ROC) curves were obtained for observation in predefined 480 areas. A jackknife method was used for statistical analysis. Differences with a P value of <0.05 were considered significant.

Results

The analysis of the observer detection of simulated pulmonary nodules showed larger areas under the ROC curve (AUC) by the ISS-FPD than by the CR. There was a statistically significant difference between the two systems at 3.2 mAs (P = 0.0330).

Conclusion

The ISS-FPD was superior to the CR for the detection of simulated pulmonary nodules at 3.2 mAs.     

Effect of radiation dose level on the detectability of pulmonary nodules in chest tomosynthesis

Objective

To investigate the detectability of pulmonary nodules in chest tomosynthesis at reduced radiation dose levels.

Methods

Eighty-six patients were included in the study and were examined with tomosynthesis and computed tomography (CT). Artificial noise was added to simulate that the tomosynthesis images were acquired at dose levels corresponding to 12, 32, and 70 % of the default setting effective dose (0.12 mSv). Three observers (with >20, >20 and three years of experience) read the tomosynthesis cases for presence of nodules in a free-response receiver operating characteristics (FROC) study. CT served as reference. Differences between dose levels were calculated using the jack-knife alternative FROC (JAFROC) figure of merit (FOM).

Results

The JAFROC FOM was 0.45, 0.54, 0.55, and 0.54 for the 12, 32, 70, and 100 % dose levels, respectively. The differences in FOM between the 12 % dose level and the 32, 70, and 100 % dose levels were 0.087 (p?=?0.006), 0.099 (p?=?0.003), and 0.093 (p?=?0.004), respectively. Between higher dose levels, no significant differences were found.

Conclusions

A substantial reduction from the default setting dose in chest tomosynthesis may be possible. In the present study, no statistically significant difference in detectability of pulmonary nodules was found when reducing the radiation dose to 32 %.

Key Points

? A substantial radiation dose reduction in chest tomosynthesis may be possible. ? Pulmonary nodule detectability remained unchanged at 32 % of the effective dose. ? Tomosynthesis might be performed at the dose of a lateral chest radiograph.     

A JAFROC study of nodule detection performance in CT images of a thorax acquired during PET/CT

PurposeTwo types of CT images (modalities) are acquired in PET/CT: for attenuation correction (AC) and diagnosis. The purpose of the study was to compare nodule detection and localization performance between these two modalities.MethodsCT images, using both modalities, of an anthropomorphic chest phantom containing zero or more simulated spherical nodules of 5, 8, 10 and 12 mm diameters and contrasts −800, −630 and 100 HU were acquired. An observer performance study using nine observers interpreting 45 normal (zero nodules) images and 47 abnormal images (1–3 nodules; average 1.26) was conducted using the free-response receiver operating characteristic (FROC) paradigm. Data were analysed using an R software package implemented jackknife alternative FROC (JAFROC) analysis. Both empirical areas under the equally weighted AFROC curve (wAFROC) and under the highest rating inferred ROC (HR-ROC) curve were used as figures of merit (FOM). To control the probability of Type I error test alpha was set at 0.05.ResultsNodule detection as measured by either FOM was significantly better on the diagnostic quality images (2nd modality), irrespective of the method of analysis, [reader averaged inter-modality wAFROC FOM difference = −0.07 (−0.11,−0.04); reader averaged inter-modality HR-ROC FOM difference = −0.05 (−0.09, −0.01)].ConclusionNodule detection was statistically worse on images acquired for AC; suggesting that images acquired for AC should not be used to evaluate pulmonary pathology.     

A ROC study of AMBER and conventional chest imaging in the detection of simulated interstitial lung disease

Purpose: The aim of this study was to compare the detection of simulated interstitial lung disease with Advanced Multiple Beam Equalization Radiography (AMBER) and conventional wide latitude screen-film radiography.Materials and methods: Interstitial disease of varying severity was simulated with overlays on an anthropomorphic chest phantom. A total of 60 images per modality was used in a Receiver Operating Characteristic (ROC) study.Results: AMBER performed significantly better than conventional radiography for all readers (P < 0.02). The difference was even more significant for the radiologist readers (P = 0.001). In each case the difference in ROC areas was between 5% and 9%.Conclusion: AMBER is superior to conventional wide latitude screen-film imaging in detecting the subtle patterns used to simulate interstitial lung disease.     

不同分辨率单色显示器对数字化X线成像肺结节检出效能的评价

目的 评价医用单色液晶显示器的不同分辨率对肺结节检出效能的影响.方法 从PACS系统中在线选取胸部数字化X线成像(DR)影像图93幅,其中正常图(CT和DR均无结节)23幅、疑诊图(仅DR 上有结节征象)32幅、确诊图(CT图显示结节)38幅.影像图上结节直径0.30～1.90 cm,均无明显钙化影;确诊病例中多发结节9例.请住院(低年资)医师或研究生、主治(中年资)医师、副主任或主任(高年资)医师各3名,分别在3种显示器[Barco Z-2261(2 M)、Barco Z3620(3 M)、Barco Z-5620(5 M)]上集中3次读片.对于结节的评价采用5等分法:肯定有、可能有、不确定、可能没有、肯定没有,每名医师针对每个显示器上的每张图像给出自己的信任等级.采用SPSS 11.5对结果进行ROC曲线的绘制及Z检验.结果 高年资医师使用2、3、5 M的显示器检测肺结节的ROC曲线下面积(AUC)分别为0.774、0.784、0.816,中年资医师分别为0.754、0.764、0.768.低年资医师分别为0.695、0.754、0.774.在相同分辨率显示器上不同年资的医师之间及同年资医师在不同分辨率的显示器上对肺结节的检出效能差异均无统计学意义(P＞0.05).结论在检测肺结节时,使用分辨率为2、3、5 M医用单色液晶显示器的效能相当;高年资的医师使用5 M的显示器会得到更多的信息;通过显示器的合理配置可以提高性价比.     

Detection of simulated pulmonary nodules by single-exposure dual-energy computed radiography of the chest: effect of a computer-aided diagnosis system (Part 2)

OBJECTIVE: To evaluate the performance of the computer-aided diagnosis (CAD) scheme on the detection of pulmonary nodules (PNs) in single-exposure dual-energy subtraction computed radiography (CR) images of the chest, and to evaluate the effect of this CAD scheme on radiologists' detectibilaties. METHODS AND MATERIAL: We compared the detectability by the CAD scheme with the detectability by 12 observers by using conventional CR (C-CR) and bone-subtracted CR (BS-CR) images of 25 chest phantoms with a low-contrast nylon nodule. RESULTS: Both in the CAD scheme and for the observers, the detectability of BS-CR images was superior to that of C-CR images (P<0.005). The detection performance of the CAD scheme was equal to that of the observers. The nodules detected by the CAD did not necessarily coincide with those by the observers. Thus, if observers can use the results of the CAD system as a 'second opinion', their detectabilities increase. CONCLUSION: The CAD system for detection of PNs in the single-exposure dual-energy subtraction method is promising for improving radiologists' detectabilities of PNs.