Analysis of variations in contrast-detail experiments

Three sources of variability in a contrast-detail (CD) experiment have been quantitated: within-observer variance, between-observer variance, and sample variance. It is concluded that (1) it is more efficient to increase the numbers of replicated images and observers than to increase the number of readings; (2) sampling and between-observer variations are approximately equal; (3) one can expect approximately 10% standard errors in the measured value of threshold detail or threshold contrast in a CD experiment which employs four observers, four replicate image samples, and one reading per observer.     

Amorphous selenium flat panel detectors for digital mammography: validation of a NPWE model observer with CDMAM observer performance experiments

Model observers have been developed which incorporate a specific imaging task, system performance, and human observer characteristics and can potentially overcome some of the limitations in using detective quantum efficiency for optimization and comparison of detectors. In this paper, a modified nonprewhitening matched filter (NPWE) model observer was developed and validated to predict object detectability for an amorphous selenium (a-Se) direct flat-panel imager (FPI) where aliasing is severe. A preclinical a-Se digital mammography FPI with 85 microm pixel size was used in this investigation. Its physical imaging properties including modulation transfer function (MTF), noise power spectrum, and DQE were fully characterized. An observer performance study was conducted by imaging the CDMAM 3.4 contrast-detail phantom designed specifically for digital mammography and presenting these images to a panel of seven observers. X-ray attenuation and scatter due to the phantom were determined experimentally for use in development of the model observer. The observer study results were analyzed via threshold averaging and signal detection theory (SDT) based techniques to produce contrast-detail curves where threshold contrast is plotted as a function of disk diameter. Validity of the model was established using SDT analysis of the experimental data. The effect of aliasing on the detectability of small diameter disks was determined using the NPWE model observer. The signal spectrum was calculated using the presampling MTF of the detector with and without including the aliased terms. Our results indicate that the NPWE model based on Fourier domain parameters provides reasonable prediction of object detectability for the signal-known-exactly task in uniform image noise for a-Se direct FPI.     

Effects of quantum noise and binocular summation on dose requirements in stereoradiography

In the case of a quantum-noise limited detector, signal detection theory suggests that stereoradiographic images can be acquired with one half of the per-image dose needed for a standard radiographic projection, as information from the two stereo images can be combined. Previously, film-screen stereoradiography has been performed using the same per-image dose as in projection radiography, i.e., doubling the total dose. In this paper, the assumption of a possible decrease in dose for stereoradiography was tested by a series of contrast-detail experiments, using phantom images acquired over a range of exposures. The number of visible details, the effective reduction of the dose, and the effective decrease in the threshold signal-to-noise ratio were determined using human observers under several display and viewing conditions. These results were averaged over five observers and compared with multiple readings by a single observer and with the results of an additional observer with limited stereoscopic acuity. Experimental results show that the total dose needed to produce a stereoradiographic image pair is approximately 1.1 times the dose needed for a single projection in standard radiography, indicating that under these conditions the human visual system demonstrates almost ideal binocular summation.     

Human efficiency for symmetry detection on 3D bumpy surfaces

Based on the ideal observer analysis, we investigated sampling properties of image information used by human visual system, for symmetrical pattern discrimination on 3D bumpy surface. There were three models of ideal observer (IO) to perform the task: 2D-IO using 2D projection image (i.e., retinal image), 2.5D-IO using image transformed to canonical view, and 3D-IO using recovered pattern image of 2D plane. We measured discrimination thresholds on the task for each IO model and subjects, and calculated human statistical efficiency relative to each ideal observer. The results indicated for the detection of a diagonal symmetry in the bumpy surface that human performance was similar to 3D-IO. This implies that human observers use the structure of the bumpy surface to detect the diagonal symmetry.     

Receiver operating characteristic and location analysis of simulated near-infrared tomography images

Receiver operating characteristic (ROC) analysis was performed on simulated near-infrared tomography images, using both human observer and contrast-to-noise ratio (CNR) computational assessment, for application in breast cancer imaging. In the analysis, a nonparametric approach was applied for estimating the ROC curves. Human observer detection of objects had superior capability to localize the presence of heterogeneities when the objects were small with high contrast, with a minimum detectable threshold of CNR near 3.0 to 3.3 in the images. Human observers were able to detect heterogeneities in the images below a size limit of 4 mm, yet could not accurately find the location of these objects when they were below 10 mm diameter. For large objects, the lower limit of a detectable contrast limit was near 10% increase relative to the background. The results also indicate that iterations of the nonlinear reconstruction algorithm beyond 4 did not significantly improve the human detection ability, and degraded the overall localization ability for the objects in the image, predominantly by increasing the noise in the background. Interobserver variance performance in detecting objects in these images was low, suggesting that because of the low spatial resolution, detection tasks with NIR tomography is likely consistent between human observers.     

Validation of a digital mammographic unit model for an objective and highly automated clinical image quality assessment

In mammography, image quality assessment has to be directly related to breast cancer indicator (e.g. microcalcifications) detectability. Recently, we proposed an X-ray source/digital detector (XRS/DD) model leading to such an assessment. This model simulates very realistic contrast-detail phantom (CDMAM) images leading to gold disc (representing microcalcifications) detectability thresholds that are very close to those of real images taken under the simulated acquisition conditions. The detection step was performed with a mathematical observer. The aim of this contribution is to include human observers into the disc detection process in real and virtual images to validate the simulation framework based on the XRS/DD model. Mathematical criteria (contrast-detail curves, image quality factor, etc.) are used to assess and to compare, from the statistical point of view, the cancer indicator detectability in real and virtual images. The quantitative results given in this paper show that the images simulated by the XRS/DD model are useful for image quality assessment in the case of all studied exposure conditions using either human or automated scoring. Also, this paper confirms that with the XRS/DD model the image quality assessment can be automated and the whole time of the procedure can be drastically reduced. Compared to standard quality assessment methods, the number of images to be acquired is divided by a factor of eight.     

Evaluation of a software package for automated quality assessment of contrast detail images--comparison with subjective visual assessment

Contrast detail analysis is commonly used to assess image quality (IQ) associated with diagnostic imaging systems. Applications include routine assessment of equipment performance and optimization studies. Most frequently, the evaluation of contrast detail images involves human observers visually detecting the threshold contrast detail combinations in the image. However, the subjective nature of human perception and the variations in the decision threshold pose limits to the minimum image quality variations detectable with reliability. Objective methods of assessment of image quality such as automated scoring have the potential to overcome the above limitations. A software package (CDRAD analyser) developed for automated scoring of images produced with the CDRAD test object was evaluated. Its performance to assess absolute and relative IQ was compared with that of an average observer. Results show that the software does not mimic the absolute performance of the average observer. The software proved more sensitive and was able to detect smaller low-contrast variations. The observer's performance was superior to the software's in the detection of smaller details. Both scoring methods showed frequent agreement in the detection of image quality variations resulting from changes in kVp and KERMA(detector), which indicates the potential to use the software CDRAD analyser for assessment of relative IQ.     

Optimization of a contrast-detail-based method for electronic image display quality evaluation

The authors previously reported a general technique based on contrast-detail methods to provide an overall quantitative evaluation of electronic image display quality. The figure-of-merit reflecting overall display quality is called maximum threshold contrast or MTC. In this work we have optimized the MTC technique through improvements in both the test images and the figure-of-merit computation. The test images were altered to match the average luminance with that observed for clinical computed radiographic images. The figure-of-merit calculation was altered to allow for contrast-detail data with slopes not equal to -1. Preliminary experiments also were conducted to demonstrate the response of the MTC measurements to increased noise in the displayed image. MTC measurements were obtained from five observers using the improved test images displayed with maximum monitor luminance settings of 30-, 50-, and 70-ft-Lamberts. Similar measurements were obtained from two observers using test images altered by the addition of a low level of image noise. The noise-free MTC and MTC difference measurements exhibited standard deviations of 0.77 and 1.55, respectively. This indicates good measurement precision, comparable or superior to that observed using the earlier MTC technique. No statistically significant image quality differences versus maximum monitor luminance were seen. The noise-added MTC measurements were greater than the noise-free values by an average of 4.08 pixel values, and this difference was statistically significant. This response is qualitatively correct, and is judged to indicate good sensitivity of the MTC measurement to increased noise levels.     

Comparison of low contrast detectability between a digital amorphous silicon and a screen-film based imaging system for thoracic radiography.

Low contrast threshold detectability is investigated theoretically and experimentally for an amorphous silicon (a-Si) x-ray detector designed for digital radiography and for a standard thoracic screen-film combination. A theoretical signal-to-noise ratio is described with a human observer signal detection model. It is calculated using the detective quantum efficiency (DQE) and the modulation transfer function of the imaging system, as well as a spatial response function for the human visual system. Using a four-alternative forced choice observer perception study, the threshold contrasts of disk shaped objects ranging in size from 0.3 to 4.0 mm are determined. Significantly better contrast detectability is obtained from the digital detector, which is attributed to its higher DQE. On average, the disk shaped objects can be detected at 45% less contrast than required for screen-film. With no free parameters the experimental data agree well with those predicted by the observer model. Based upon the data, the model predicts that x-ray exposure for the a-Si detector only needs to be 30% of the exposure required to perform equally well on the contrast-detail detectability task using screen-film.     

How Reproducibly Can Human Ear Ossicles Be Measured? A Study of Inter‐Observer Error

Ear ossicles have thus far received little attention in biological anthropology. For the use of these bones as a source of biological information, it is important to know how reproducibly they can be measured. We determined inter‐observer errors for measurements recorded by two observers on mallei (N = 119) and incudes (N = 124) obtained from human skeletons recovered from an early medieval cemetery in southern Germany. Measurements were taken on‐screen on images of the bones obtained with a digital microscope. In the case of separately acquired images, mean inter‐observer error ranged between 0.50 and 9.59% (average: 2.63%) for malleus measurements and between 0.67 and 7.11% (average: 2.01%) for incus measurements. Coefficients of reliability ranged between 0.72 and 0.99 for the malleus measurements and between 0.61 and 0.98 for those of the incus. Except for one incus measurement, readings performed by the two observers on the same set of photographs produced lower inter‐observer errors and higher coefficients of reliability than the method involving separate acquisition of images by the observers. Across all linear measurements, absolute inter‐observer error was independent of the mean size of the measured variable for both bones. So far, studies on human ear ossicles have largely neglected the issue of measurement error and its potential implication for the interpretation of the data. Knowledge of measurement error is of special importance if results obtained by different researchers are combined into a single database. It is, therefore, suggested that the reproducibility of measurements should be addressed in all future studies of ear ossicles. Anat Rec, 2010. © 2010 Wiley‐Liss, Inc.     

Human observer detection experiments with mammograms and power-law noise

We determined contrast thresholds for lesion detection as a function of lesion size in both mammograms and filtered noise backgrounds with the same average power spectrum, P(f)=B/f3. Experiments were done using hybrid images with digital images of tumors added to digitized normal backgrounds, displayed on a monochrome monitor. Four tumors were extracted from digitized specimen radiographs. The lesion sizes were varied by digital rescaling to cover the range from 0.5 to 16 mm. Amplitudes were varied to determine the value required for 92% correct detection in two-alternative forced-choice (2AFC) and 90% for search experiments. Three observers participated, two physicists and a radiologist. The 2AFC mammographic results demonstrated a novel contrast-detail (CD) diagram with threshold amplitudes that increased steadily (with slope of 0.3) with increasing size for lesions larger than 1 mm. The slopes for prewhitening model observers were about 0.4. Human efficiency relative to these models was as high as 90%. The CD diagram slopes for the 2AFC experiments with filtered noise were 0.44 for humans and 0.5 for models. Human efficiency relative to the ideal observer was about 40%. The difference in efficiencies for the two types of backgrounds indicates that breast structure cannot be considered to be pure random noise for 2AFC experiments. Instead, 2AFC human detection with mammographic backgrounds is limited by a combination of noise and deterministic masking effects. The search experiments also gave thresholds that increased with lesion size. However, there was no difference in human results for mammographic and filtered noise backgrounds, suggesting that breast structure can be considered to be pure random noise for this task. Our conclusion is that, in spite of the fact that mammographic backgrounds have nonstationary statistics, models based on statistical decision theory can still be applied successfully to estimate human performance.     

Characterization of the effects of the FineView algorithm for full field digital mammography

The aim of this study was to characterize the effect of an image processing algorithm (FineView) on both quantitative image quality parameters and the threshold contrast detail response of the GE Senographe DS full-field digital mammography system. The system was characterized using signal transfer property, pre-sampling modulation transfer function (MTF), normalized noise power spectrum (NNPS) and detective quantum efficiency (DQE) of the system. An algorithmic modulation transfer function (MTF(a)) was calculated from images acquired at a reduced detector air kerma (DAK) and with the FineView algorithm enabled. Two sets of beam conditions were used: Mo/Mo/28 kV and Rh/Rh/29 kV, both with 2 mm added Al filtration at the x-ray tube. Images were acquired with and without FineView at four DAK levels from 14 to 378 μGy. The threshold contrast detail response was assessed using the CDMAM contrast-detail test object which was imaged under standard clinical conditions with and without FineView at three DAK levels from 24 to 243 μGy. The images were scored by both human observers and by automated scoring software. Results indicated an improvement of up to 125% at 5 mm?1 in MTF(a) when FineView was activated, particularly at high DAK levels. A corresponding increase of up to 425% at 5 mm?1 was also seen in the NNPS, again with the same DAK dependence. FineView did not influence DQE, an indication that the signal to noise ratio transfer of the system remained unchanged. FineView did not affect the threshold contrast detectability of the system, a result that is consistent with the DQE results.     

A Clinical Evaluation of the Image Quality Computer Program, CoCIQ

To provide an objective way of measuring image quality, a computer program was designed that automatically analyzes the test images of a contrast-detail (CD) phantom. The program gives a quantified measurement of image quality by calculating an Image Quality Figure (IQF). The aim of this work was to evaluate the program and adjust it to clinical situations in order to find the detectable level where the program gives a reliable figure of the contrast resolution. The program was applied on a large variety of images with lumbar spine and urographic parameters, from very low to very high image qualities. It was shown that the computer program produces IQFs with small variations and there were a strong linear statistical relation between the computerized evaluation and the evaluation performed by human observers (R² = 0.98). This method offers a fast and easy way of conducting image quality evaluations.     

Image quality assessment in digital mammography: part II. NPWE as a validated alternative for contrast detail analysis

Assessment of image quality for digital x-ray mammography systems used in European screening programs relies mainly on contrast-detail CDMAM phantom scoring and requires the acquisition and analysis of many images in order to reduce variability in threshold detectability. Part II of this study proposes an alternative method based on the detectability index (d') calculated for a non-prewhitened model observer with an eye filter (NPWE). The detectability index was calculated from the normalized noise power spectrum and image contrast, both measured from an image of a 5 cm poly(methyl methacrylate) phantom containing a 0.2 mm thick aluminium square, and the pre-sampling modulation transfer function. This was performed as a function of air kerma at the detector for 11 different digital mammography systems. These calculated d' values were compared against threshold gold thickness (T) results measured with the CDMAM test object and against derived theoretical relationships. A simple relationship was found between T and d', as a function of detector air kerma; a linear relationship was found between d' and contrast-to-noise ratio. The values of threshold thickness used to specify acceptable performance in the European Guidelines for 0.10 and 0.25 mm diameter discs were equivalent to threshold calculated detectability indices of 1.05 and 6.30, respectively. The NPWE method is a validated alternative to CDMAM scoring for use in the image quality specification, quality control and optimization of digital x-ray systems for screening mammography.     

Using visual direction in three-dimensional motion perception

The eyes receive slightly different views of the world, and the differences between their images (binocular disparity) are used to see depth. Several authors have suggested how the brain could exploit this information for three-dimensional (3D) motion perception, but here we consider a simpler strategy. Visual direction is the angle between the direction of an object and the direction that an observer faces. Here we describe human behavioral experiments in which observers use visual direction, rather than binocular information, to estimate an object's 3D motion even though this causes them to make systematic errors. This suggests that recent models of binocular 3D motion perception may not reflect the strategies that human observers actually use.     

A comparative contrast-detail study of five medical displays

The objective of this study was to compare the contrast-detail performance of five different commercial liquid crystal displays (LCDs) to other LCD and cathode-ray tube (CRT) displays for medical applications. A contrast-detail phantom, supplemented with 5 in. of acrylic, was imaged on a commercial digital radiographic system using techniques comparable to chest radiography. The phantom design enabled observer evaluation by a four-alternative forced choice paradigm. The acquired images were independently scored by five observers on five medical display devices: a 5 megapixel monochrome LCD, a 3 megapixel monochrome LCD, a 9 megapixel color LCD, a 5 megapixel monochrome CRT, and a mammographic-grade monochrome CRT. The data were analyzed using the method suggested by the manufacturer based on a nearest neighbor correction technique. They were further analyzed using a logistic regression response model with a natural threshold using an overall chi-square test for display type followed by pairwise comparisons for individual display performance. The differences between the display devices were small. The standard analysis of the results based on the manufacturer-recommended method did not yield any statistically discernible trend among displays. The logistic regression analysis, however, indicated that the 5 megapixel monochrome LCD was statistically significantly (p <0.0001) superior to the others, followed by the 3 megapixel monochrome LCD (p<0.0001). The three other displays exhibited lower but generally similar performance characteristics. The findings suggest that 5 and 3 megapixel monochrome LCDs provide comparable but subtly superior contrast detectability than other tested displays, with the former performing slightly better in the detection of subtle and fine details.     

Automated localization of implanted seeds in 3D TRUS images used for prostate brachytherapy

An algorithm has been developed in this paper to localize implanted radioactive seeds in 3D ultrasound images for a dynamic intraoperative brachytherapy procedure. Segmentation of the seeds is difficult, due to their small size in relatively low quality of transrectal ultrasound (TRUS) images. In this paper, intraoperative seed segmentation in 3D TRUS images is achieved by performing a subtraction of the image before the needle has been inserted, and the image after the seeds have been implanted. The seeds are searched in a "local" space determined by the needle position and orientation information, which are obtained from a needle segmentation algorithm. To test this approach, 3D TRUS images of the agar and chicken tissue phantoms were obtained. Within these phantoms, dummy seeds were implanted. The seed locations determined by the seed segmentation algorithm were compared with those obtained from a volumetric cone-beam flat-panel micro-CT scanner and human observers. Evaluation of the algorithm showed that the rms error in determining the seed locations using the seed segmentation algorithm was 0.98 mm in agar phantoms and 1.02 mm in chicken phantoms.     

Morphometry of dermal collagen orientation by Fourier analysis is superior to multi-observer assessment

In human dermis, collagen bundle architecture appears randomly organized, whereas in pathological conditions, such as scar tissue and connective tissue disorders, collagen bundle architecture is arranged in a more parallel fashion. Histological examination by one or two observers using polarized light is the most common method to determine collagen orientation. The hypothesis on which this study is based is that an objective image analysis technique, Fourier analysis, would improve the reliability (are the measurements reproducible?) and the accuracy (does the method measure what it is supposed to measure?) of collagen orientation assessment, compared with observer ratings. Fourier analysis was applied to 271 images of scar tissue and normal skin that were acquired by confocal laser‐scanning microscopy. Observers rated the same areas using polarized light as well as the confocal microscopy images. Computer images consisting of different types of ellipses were generated with a fixed orientation. Observers and Fourier analysis evaluated the images to evaluate accuracy. The inter‐observer reliability was acceptable when at least three observers rated polarized light images (r > 0.69), whereas two observers were sufficient for rating confocal microscopy images (r > 0.71). Fourier analysis correlated better with observer ratings of confocal microscopy images (r = 0.69) than with polarized light microscopy images (r = 0.42). Fourier analysis was more accurate than four observers for the evaluation of the ‘true’ orientation for almost all types of computer‐generated images. For the first time it is shown that Fourier image analysis is suitable for the morphometry of dermal collagen orientation and leads to a superior measurement of collagen orientation compared with subjective histological evaluation by several experts. If an evaluation is performed by conventional light microscopy, at least three observers are required to attain an acceptable inter‐observer reliability. Copyright © 2002 John Wiley & Sons, Ltd.     

Dose efficiency and low-contrast detectability of an amorphous silicon x-ray detector for digital radiography

The effect of dose reduction on low-contrast detectability is investigated theoretically and experimentally for a production grade amorphous silicon (a-Si) x-ray detector and compared with a standard thoracic screen-film combination. A non-prewhitening matched filter observer model modified to include a spatial response function and internal noise for the human visual system (HVS) is used to calculate a signal-to-noise ratio (SNR) related to object detectability. Other inputs to the SNR calculation are the detective quantum efficiency (DQE) and the modulation transfer function (MTF) of the imaging system. Besides threshold detectability, the model predicts the equivalent perception dose ratio (EPDR), which is the fraction of the screen film exposure for which the digital detector provides equal detectability. Images of a contrast-detail phantom are obtained with the digital detector at dose levels corresponding to 27%, 41%, 63% and 100% of the dose used for screen-film. The images are used in a four-alternative forced choice (4-AFC) observer perception study in order to measure threshold detectability. A statistically significant improvement in contrast detectability is measured with the digital detector at 100% and 63% of the screen-film dose. There is no statistical difference between screen-film and digital at 41% of the dose. On average, the experimental EPDR is 44%, which agrees well with the model prediction of 40%.     

Rapid calculation of detectability in Bayesian single photon emission computed tomography

We consider the calculation of lesion detectability using a mathematical model observer, the channelized Hotelling observer (CHO), in a signal-known-exactly/background-known-exactly detection task for single photon emission computed tomography (SPECT). We focus on SPECT images reconstructed with Bayesian maximum a posteriori methods. While model observers are designed to replace time-consuming studies using human observers, the calculation of CHO detectability is usually accomplished using a large number of sample images, which is still time consuming. We develop theoretical expressions for a measure of detectability, the signal-to-noise-ratio (SNR) of a CHO observer, that can be very rapidly evaluated. Key to our expressions are approximations to the reconstructed image covariance. In these approximations, we use methods developed in the PET literature, but modify them to reflect the different nature of attenuation and distance-dependent blur in SPECT. We validate our expressions with Monte Carlo methods. We show that reasonably accurate estimates of the SNR can be obtained at a computational expense equivalent to approximately two projection operations, and that evaluating SNR for subsequent lesion locations requires negligible additional computation.