Evaluation of internal noise methods for Hotelling observer models

The inclusion of internal noise in model observers is a common method to allow for quantitative comparisons between human and model observer performance in visual detection tasks. In this article, we studied two different strategies for inserting internal noise into Hotelling model observers. In the first strategy, internal noise was added to the output of individual channels: (a) Independent nonuniform channel noise, (b) independent uniform channel noise. In the second strategy, internal noise was added to the decision variable arising from the combination of channel responses. The standard deviation of the zero mean internal noise was either constant or proportional to: (a) the decision variable's standard deviation due to the external noise, (b) the decision variable's variance caused by the external noise, (c) the decision variable magnitude on a trial to trial basis. We tested three model observers: square window Hotelling observer (HO), channelized Hotelling observer (CHO), and Laguerre-Gauss Hotelling observer (LGHO) using a four alternative forced choice (4AFC) signal known exactly but variable task with a simulated signal embedded in real x-ray coronary angiogram backgrounds. The results showed that the internal noise method that led to the best prediction of human performance differed across the studied model observers. The CHO model best predicted human observer performance with the channel internal noise. The HO and LGHO best predicted human observer performance with the decision variable internal noise. The present results might guide researchers with the choice of methods to include internal noise into Hotelling model observers when evaluating and optimizing medical image quality.     

Rapid calculation of detectability in Bayesian single photon emission computed tomography

We consider the calculation of lesion detectability using a mathematical model observer, the channelized Hotelling observer (CHO), in a signal-known-exactly/background-known-exactly detection task for single photon emission computed tomography (SPECT). We focus on SPECT images reconstructed with Bayesian maximum a posteriori methods. While model observers are designed to replace time-consuming studies using human observers, the calculation of CHO detectability is usually accomplished using a large number of sample images, which is still time consuming. We develop theoretical expressions for a measure of detectability, the signal-to-noise-ratio (SNR) of a CHO observer, that can be very rapidly evaluated. Key to our expressions are approximations to the reconstructed image covariance. In these approximations, we use methods developed in the PET literature, but modify them to reflect the different nature of attenuation and distance-dependent blur in SPECT. We validate our expressions with Monte Carlo methods. We show that reasonably accurate estimates of the SNR can be obtained at a computational expense equivalent to approximately two projection operations, and that evaluating SNR for subsequent lesion locations requires negligible additional computation.     

Hardware assessment using the multi-module, multi-resolution system (M3R): a signal-detection study

The multi-module, multi-resolution system (M3R) is used for hardware assessment in objective, task-based signal detection studies in projection data. A phantom capable of generating multiple realizations of a random textured background is introduced. Measured backgrounds from this phantom are used along with simulated lumpy and uniform backgrounds to investigate signal-to-noise ratio as a function of exposure time. Results are shown to agree with theoretical predictions, exhibiting a power-law like dependence previously seen for studies performed either in simulation or without an imaging system, and help validate the use of simulated lumpy backgrounds in observer studies. A second study looks at signal-detection performance, measured by AUC (area under the receiver operating characteristic curve), in lumpy backgrounds for 20 M3R aperture combinations as a function of lump size and signal size. Observer performance reveals an improvement in AUC for certain ranges of signal and lump combinations through the use of multiplexed, multiple-pinhole apertures, indicating a need for task-specific aperture optimization. The channelized Hotelling observer is used with Laguerre-Gauss channels for both observer studies. Methods for selection of number of channels and channel width are discussed.     

Image covariance and lesion detectability in direct fan-beam x-ray computed tomography

We consider noise in computed tomography images that are reconstructed using the classical direct fan-beam filtered backprojection algorithm, from both full- and short-scan data. A new, accurate method for computing image covariance is presented. The utility of the new covariance method is demonstrated by its application to the implementation of a channelized Hotelling observer for a lesion detection task. Results from the new covariance method and its application to the channelized Hotelling observer are compared with results from Monte Carlo simulations. In addition, the impact of a bowtie filter and x-ray tube current modulation on reconstruction noise and lesion detectability are explored for full-scan reconstruction.     

X-ray imaging optimization using virtual phantoms and computerized observer modelling

This study develops and demonstrates a realistic x-ray imaging simulator with computerized observers to maximize lesion detectability and minimize patient exposure. A software package, ViPRIS, incorporating two computational patient phantoms, has been developed for simulating x-ray radiographic images. A tomographic phantom, VIP-Man, constructed from Visible Human anatomical colour images is used to simulate the scattered portion using the ESGnrc Monte Carlo code. The primary portion of an x-ray image is simulated using the projection ray-tracing method through the Visible Human CT data set. To produce a realistic image, the software simulates quantum noise, blurring effects, lesions, detector absorption efficiency and other imaging artefacts. The primary and scattered portions of an x-ray chest image are combined to form a final image for computerized observer studies and image quality analysis. Absorbed doses in organs and tissues of the segmented VIP-Man phantom were also obtained from the Monte Carlo simulations. Approximately 25,000 simulated images and 2,500,000 data files were analysed using computerized observers. Hotelling and Laguerre-Gauss Hotelling observers are used to perform various lesion detection tasks. Several model observer tasks were used including SKE/BKE, MAFC and SKEV. The energy levels and fluence at the minimum dose required to detect a small lesion were determined with respect to lesion size, location and system parameters.     

Identification of simulated microcalcifications in white noise and mammographic backgrounds

This work investigates human performance in discriminating between differently shaped simulated microcalcifications embedded in white noise or mammographic backgrounds. Human performance was determined through two alternative forced-choice (2-AFC) experiments. The signals used were computer-generated simple shapes that were designed such that they had equal signal energy. This assured equal detectability. For experiments involving mammographic backgrounds, signals were blurred to account for the imaging system modulation transfer function (MTF). White noise backgrounds were computer generated; anatomic background patches were extracted from normal mammograms. We compared human performance levels as a function of signal energy in the expected difference template. In the discrimination task, the expected difference template is the difference between the two signals shown. In white noise backgrounds, human performance in the discrimination task was degraded compared to the detection task. In mammographic backgrounds, human performance in the discrimination task exceeded that of the detection task. This indicates that human observers do not follow the optimum decision strategy of correlating the expected signal template with the image. Human observer performance was qualitatively reproduced by non-prewhitening with eye filter (NPWE) model observer calculations, in which spatial uncertainty was explicitly included by shifting the locations of the expected difference templates. The results indicate that human strategy in the discrimination task may be to match individual signal templates with the image individually, rather than to perform template matching between the expected difference template and the image.     

Effect of dose reduction on the detection of mammographic lesions: a mathematical observer model analysis

The effect of reduction in dose levels normally used in mammographic screening procedures on the detection of breast lesions were analyzed. Four types of breast lesions were simulated and inserted into clinically-acquired digital mammograms. Dose reduction by 50% and 75% of the original clinically-relevant exposure levels were simulated by adding corresponding simulated noise into the original mammograms. The mammograms were converted into luminance values corresponding to those displayed on a clinical soft-copy display station and subsequently analyzed by Laguerre-Gauss and Gabor channelized Hotelling observer models for differences in detectability performance with reduction in radiation dose. Performance was measured under a signal known exactly but variable detection task paradigm in terms of receiver operating characteristics (ROC) curves and area under the ROC curves. The results suggested that luminance mapping of digital mammograms affects performance of model observers. Reduction in dose levels by 50% lowered the detectability of masses with borderline statistical significance. Dose reduction did not have a statistically significant effect on detection of microcalcifications. The model results indicate that there is room for optimization of dose level in mammographic screening procedures.     

Computer aided detection of masses in mammography using subregion Hotelling observers

We propose to investigate the use of the subregion Hotelling observer for the basis of a computer aided detection scheme for masses in mammography. A database of 1320 regions of interest (ROIs) was selected from the DDSM database collected by the University of South Florida using the Lumisys scanner cases. The breakdown of the cases was as follows: 656 normal ROIs, 307 benign ROIs, and 357 cancer ROIs. Each ROI was extracted at a size of 1024 x 1024 pixels and sub-sampled to 128 x 128 pixels. For the detection task, cancer and benign cases were considered positive and normal was considered negative. All positive cases had the lesion centered in the ROI. We chose to investigate the subregion Hotelling observer as a classifier to detect masses. The Hotelling observer incorporates information about the signal, the background, and the noise correlation for prediction of positive and negative and is the optimal detector when these are known. For our study, 225 subregion Hotelling observers were set up in a 15 x 15 grid across the center of the ROIs. Each separate observer was designed to "observe," or discriminate, an 8 x 8 pixel area of the image. A leave one out training and testing methodology was used to generate 225 "features," where each feature is the output of the individual observers. The 225 features derived from separate Hotelling observers were then narrowed down by using forward searching linear discriminants (LDs). The reduced set of features was then analyzed using an additional LD with receiver operating characteristic (ROC) analysis. The 225 Hotelling observer features were searched by the forward searching LD, which selected a subset of 37 features. This subset of 37 features was then analyzed using an additional LD, which gave a ROC area under the curve of 0.9412 +/- 0.006 and a partial area of 0.6728. Additionally, at 98% sensitivity the overall classifier had a specificity of 55.9% and a positive predictive value of 69.3%. Preliminary results suggest that using subregion Hotelling observers in combination with LDs can provide a strong backbone for a CAD scheme to help radiologists with detection. Such a system could be used in conjunction with CAD systems for false positive reduction.     

Effects of quantum noise and binocular summation on dose requirements in stereoradiography

In the case of a quantum-noise limited detector, signal detection theory suggests that stereoradiographic images can be acquired with one half of the per-image dose needed for a standard radiographic projection, as information from the two stereo images can be combined. Previously, film-screen stereoradiography has been performed using the same per-image dose as in projection radiography, i.e., doubling the total dose. In this paper, the assumption of a possible decrease in dose for stereoradiography was tested by a series of contrast-detail experiments, using phantom images acquired over a range of exposures. The number of visible details, the effective reduction of the dose, and the effective decrease in the threshold signal-to-noise ratio were determined using human observers under several display and viewing conditions. These results were averaged over five observers and compared with multiple readings by a single observer and with the results of an additional observer with limited stereoscopic acuity. Experimental results show that the total dose needed to produce a stereoradiographic image pair is approximately 1.1 times the dose needed for a single projection in standard radiography, indicating that under these conditions the human visual system demonstrates almost ideal binocular summation.     

基于CT模拟数据研究多层CHO的低对比检测性能

【摘 要】 目的:基于CT模拟数据,比较多层通道化Hotelling观察器（CHO）和单层CHO对低对比物体的检测性能。 方法:利用计算机模拟产生有、无信号两类三维图像,分别改变信号幅度及尺寸,研究单层、多层CHO对不同幅度、尺寸的信号检测性。通过比较观察者性能指标PC,定量评估多层CHO较单层CHO的低对比检测能力。 结果:当信号尺寸为2.5 mm,信号幅度分别为2.5、5.0、10.0 HU时,PC分别为0.79、0.96、1.00,层数为10时,多层CHO性能PC达饱和,对应单层CHO性能PC分别为0.67、0.78、0.93,性能分别提升了17.91%、23.08%、7.53%;当信号幅度固定为2.5 HU,信号尺寸分别为2.0、2.5、3.5、5.0、10.0 mm时,多层CHO在层数分别为8、10、8、6、3时,PC达最大,分别为0.69、0.79、0.95、0.99、1.00,对应单层CHO性能PC分别为0.63、0.67、0.74、0.86、0.96,性能分别提升9.5%、17.9%、28.4%、15.1%、4.2%。 结论:由于综合多层信息制定决策,多层CHO相比于单层CHO对低对比物体的检测能力更强。但利用多层CHO进行图像质量评估时,层数选择至关重要,应根据信号幅度、尺寸、背景噪音等因素选择合适的层数。     

Detection performance analysis for time-of-flight PET

In this paper, we investigate the performance of time-of-flight (TOF) positron emission tomography (PET) in improving lesion detectability. We present a theoretical approach to compare lesion detectability of TOF versus non-TOF systems and perform computer simulations to validate the theoretical prediction. A single-ring TOF PET tomograph is simulated using SimSET software, and images are reconstructed in 2D from list-mode data using a maximum a posteriori method. We use a channelized Hotelling observer to assess the detection performance. Both the receiver operating characteristic (ROC) and localization ROC curves are compared for the TOF and non-TOF PET systems. We first studied the SNR gains for TOF PET with different scatter and random fractions, system timing resolutions and object sizes. We found that the TOF information improves the lesion detectability and the improvement is greater with larger fractions of randoms, better timing resolution and bigger objects. The scatters by themselves have little impact on the SNR gain after correction. Since the true system timing resolution may not be known precisely in practice, we investigated the effect of mismatched timing kernels and showed that using a mismatched kernel during reconstruction always degrades the detection performance, no matter whether it is narrower or wider than the real value. Using the proposed theoretical framework, we also studied the effect of lumpy backgrounds on the detection performance. Our results indicated that with lumpy backgrounds, the TOF PET still outperforms the non-TOF PET, but the improvement is smaller compared with the uniform background case. More specifically, with the same correlation length, the SNR gain reduces with bigger number of lumpy patches and greater lumpy amplitudes. With the same variance, the SNR gain reaches the minimum when the width of the Gaussian lumps is close to the size of the tumor.     

Fully 4D motion-compensated reconstruction of cardiac SPECT images

In this paper, we investigate the benefits of a spatiotemporal approach for reconstruction of image sequences. In the proposed approach, we introduce a temporal prior in the form of motion compensation to account for the statistical correlations among the frames in a sequence, and reconstruct all the frames collectively as a single function of space and time. The reconstruction algorithm is derived based on the maximum a posteriori estimate, for which the one-step late expectation-maximization algorithm is used. We demonstrated the method in our experiments using simulated single photon emission computed tomography (SPECT) cardiac perfusion images. The four-dimensional (4D) gated mathematical cardiac-torso phantom was used for simulation of gated SPECT perfusion imaging with Tc-99m-sestamibi. In addition to bias-variance analysis and time activity curves, we also used a channelized Hotelling observer to evaluate the detectability of perfusion defects in the reconstructed images. Our experimental results demonstrated that the incorporation of temporal regularization into image reconstruction could significantly improve the accuracy of cardiac images without causing any significant cross-frame blurring that may arise from the cardiac motion. This could lead to not only improved detection of perfusion defects, but also improved reconstruction of the heart wall which is important for functional assessment of the myocardium.     

A clinical evaluation of contrast-detail analysis for ultrasound images

We report on the reproducibility of human observers' vanishing detection thresholds for visual targets in contrast-detail (C/D) analysis of ultrasound B-mode images. The images used in this study contain visual targets which are circular cross sections of constant-contrast conical structures in the C/D phantom. The vanishing threshold diameters for these targets vary as a function of the perceived size of the imaged target, target-to-background contrast, image noise content, and reproducibility of the decision levels of human observers for repeated observations. Our study indicates that the determination of absolute vanishing threshold diameter values for several targets of different contrast by human observers yields a high degree of error that is not predicted by existing theoretical assumptions based on a static threshold detector. We find that systematic error is introduced by the observers during the course of the experiment and that the levels of sensitivity of the observers differ widely at all times, and increase the amount of total observer error. These results suggest that, due to the large total observer error, C/D analysis may be impractical in a clinical environment, unless there is access to a team of observers specifically and extensively trained in this task. We suggest that a computer-based observer may be more reliable for the objective performance of contrast-detail analysis as a method for evaluating ultrasound image quality and comparison of imaging systems.     

Human observer detection experiments with mammograms and power-law noise

We determined contrast thresholds for lesion detection as a function of lesion size in both mammograms and filtered noise backgrounds with the same average power spectrum, P(f)=B/f3. Experiments were done using hybrid images with digital images of tumors added to digitized normal backgrounds, displayed on a monochrome monitor. Four tumors were extracted from digitized specimen radiographs. The lesion sizes were varied by digital rescaling to cover the range from 0.5 to 16 mm. Amplitudes were varied to determine the value required for 92% correct detection in two-alternative forced-choice (2AFC) and 90% for search experiments. Three observers participated, two physicists and a radiologist. The 2AFC mammographic results demonstrated a novel contrast-detail (CD) diagram with threshold amplitudes that increased steadily (with slope of 0.3) with increasing size for lesions larger than 1 mm. The slopes for prewhitening model observers were about 0.4. Human efficiency relative to these models was as high as 90%. The CD diagram slopes for the 2AFC experiments with filtered noise were 0.44 for humans and 0.5 for models. Human efficiency relative to the ideal observer was about 40%. The difference in efficiencies for the two types of backgrounds indicates that breast structure cannot be considered to be pure random noise for 2AFC experiments. Instead, 2AFC human detection with mammographic backgrounds is limited by a combination of noise and deterministic masking effects. The search experiments also gave thresholds that increased with lesion size. However, there was no difference in human results for mammographic and filtered noise backgrounds, suggesting that breast structure can be considered to be pure random noise for this task. Our conclusion is that, in spite of the fact that mammographic backgrounds have nonstationary statistics, models based on statistical decision theory can still be applied successfully to estimate human performance.     

Toward objective and quantitative evaluation of imaging systems using images of phantoms

The use of imaging phantoms is a common method of evaluating image quality in the clinical setting. These evaluations rely on a subjective decision by a human observer with respect to the faintest detectable signal(s) in the image. Because of the variable and subjective nature of the human-observer scores, the evaluations manifest a lack of precision and a potential for bias. The advent of digital imaging systems with their inherent digital data provides the opportunity to use techniques that do not rely on human-observer decisions and thresholds. Using the digital data, signal-detection theory (SDT) provides the basis for more objective and quantitative evaluations which are independent of a human-observer decision threshold. In a SDT framework, the evaluation of imaging phantoms represents a "signal-known-exactly/background-known-exactly" ("SKE/ BKE") detection task. In this study, we compute the performance of prewhitening and nonprewhitening model observers in terms of the observer signal-to-noise ratio (SNR) for these "SK E/BKE" tasks. We apply the evaluation methods to a number of imaging systems. For example, we use data from a laboratory implementation of digital radiography and from a full-field digital mammography system in a clinical setting. In addition, we make a comparison of our methods to human-observer scoring of a set of digital images of the CDMAM phantom available from the internet (EUREF-European Reference Organization). In the latter case, we show a significant increase in the precision of the quantitative methods versus the variability in the scores from human observers on the same set of images. As regards bias, the performance of a model observer estimated from a finite data set is known to be biased. In this study, we minimize the bias and estimate the variance of the observer SNR using statistical resampling techniques, namely, "bootstrapping" and "shuffling" of the data sets. Our methods provide objective and quantitative evaluation of imaging systems with increased precision and reduced bias.     

Incorporation of a Laguerre–Gauss Channelized Hotelling Observer for False-Positive Reduction in a Mammographic Mass CAD System

Previously, we developed a simple Laguerre-Gauss (LG) channelized Hotelling observer (CHO) for incorporation into our mass computer-aided detection (CAD) system. This LG-CHO was trained using initial detection suspicious region data and was empirically optimized for free parameters. For the study presented in this paper, we wish to create a more optimal mass detection observer based on a novel combination of LG channels. A large set of LG channels with differing free parameters was created. Each of these channels was applied to the suspicious regions, and an output test statistic was determined. A stepwise feature selection algorithm was used to determine which LG channels would combine best to detect masses. These channels were combined using a HO to create a single template for the mass CAD system. Results from free-response receiver operating characteristic curves demonstrated that the incorporation of the novel LG-CHO into the CAD system slightly improved performance in high-sensitivity regions.     

Comparison of region-of-interest analysis and human observers in the diagnosis of Parkinson's disease using [99mTc]TRODAT-1 and SPECT

This study determined the relative accuracy of diagnosis of Parkinson's disease (PD) using SPECT imaging data, comparing a semi-quantitative region-of-interest (ROI) approach and human observers. A set of patients with PD and normal healthy control subjects were studied using the dopamine transporter tracer [(99m)Tc]TRODAT-1 and SPECT. The sample comprised 81 patients (mean age +/- SD, 63.4 +/- 10.4 years; age range, 39.0-84.2 years) and 94 healthy controls (mean age +/- SD, 61.8 +/- 11.0 years; age range, 40.9-83.3 years). A standardized template containing six ROIs was transposed onto subregions of the brain, and the ratio of striatal to background ROI values was used as a semi-quantitative outcome measure. All images were used in a human observer study, with four experienced investigators. The data from the observer and ROI studies were analysed using a receiver operating characteristic (ROC) analysis, where the area under the ROC curve (AUC) indicated the diagnostic accuracy. ROI analysis and human observers gave similar diagnostic performance (mean observer AUC = 0.89, best ROI AUC = 0.90). This suggested that the human observers are visually acquiring similar information from the images that are contained in the semi-quantitative striatal uptake.     

Evaluation of a software package for automated quality assessment of contrast detail images--comparison with subjective visual assessment

Contrast detail analysis is commonly used to assess image quality (IQ) associated with diagnostic imaging systems. Applications include routine assessment of equipment performance and optimization studies. Most frequently, the evaluation of contrast detail images involves human observers visually detecting the threshold contrast detail combinations in the image. However, the subjective nature of human perception and the variations in the decision threshold pose limits to the minimum image quality variations detectable with reliability. Objective methods of assessment of image quality such as automated scoring have the potential to overcome the above limitations. A software package (CDRAD analyser) developed for automated scoring of images produced with the CDRAD test object was evaluated. Its performance to assess absolute and relative IQ was compared with that of an average observer. Results show that the software does not mimic the absolute performance of the average observer. The software proved more sensitive and was able to detect smaller low-contrast variations. The observer's performance was superior to the software's in the detection of smaller details. Both scoring methods showed frequent agreement in the detection of image quality variations resulting from changes in kVp and KERMA(detector), which indicates the potential to use the software CDRAD analyser for assessment of relative IQ.     

Human efficiency for symmetry detection on 3D bumpy surfaces

Based on the ideal observer analysis, we investigated sampling properties of image information used by human visual system, for symmetrical pattern discrimination on 3D bumpy surface. There were three models of ideal observer (IO) to perform the task: 2D-IO using 2D projection image (i.e., retinal image), 2.5D-IO using image transformed to canonical view, and 3D-IO using recovered pattern image of 2D plane. We measured discrimination thresholds on the task for each IO model and subjects, and calculated human statistical efficiency relative to each ideal observer. The results indicated for the detection of a diagonal symmetry in the bumpy surface that human performance was similar to 3D-IO. This implies that human observers use the structure of the bumpy surface to detect the diagonal symmetry.     

Using visual direction in three-dimensional motion perception

The eyes receive slightly different views of the world, and the differences between their images (binocular disparity) are used to see depth. Several authors have suggested how the brain could exploit this information for three-dimensional (3D) motion perception, but here we consider a simpler strategy. Visual direction is the angle between the direction of an object and the direction that an observer faces. Here we describe human behavioral experiments in which observers use visual direction, rather than binocular information, to estimate an object's 3D motion even though this causes them to make systematic errors. This suggests that recent models of binocular 3D motion perception may not reflect the strategies that human observers actually use.