Sampling efficiency and internal noise for motion detection,discrimination, and summation

By comparing real observers to an ideal observer, previous studies have found that the detection of static patterns is limited by internal noise and by imperfect sampling efficiency. We developed and applied ideal observer models for the detection, discrimination, and summation of oppositely drifting gratings in Gaussian white noise. The three tasks share a common source of internal noise. The sampling efficiencies were on the order of 1-2% except for much lower efficiency in direction discrimination for faster moving gratings. The efficiency of direction discrimination relative to detection systematically declines as the speed is increased from 1 to 6 Hz. These results suggest that observers use mismatched filters tuned to slow speeds regardless of the signal speed. Human visual motion sensing appears to use distorted representations of the incoming signals, and this distortion is a major limitation to visual performance.     

Classification images for detection and position discrimination in the fovea and parafovea

Classification images provide an important new method for learning about which parts of the stimulus are used to make perceptual decisions and provide a new tool for measuring the template an observer uses to accomplish a task. Here we introduce a new method using one-dimensional sums of sinusoids as both test stimuli (discrete frequency patterns [DFP]) and as noise. We use this method to study and compare the templates used to detect a target and to discriminate the target's position in central and parafoveal vision. Our results show that, unsurprisingly, the classification images for detection in both foveal and parafoveal vision resemble the DFP test stimulus, but are considerably broader in spatial frequency tuning than the ideal observer. In contrast, the classification images for foveal position discrimination are not ideal, and depend on the size of the position offset. Over a range of offsets from close to threshold to about 90 arc sec, our observers appear to use a peak strategy (responding to the location of the peak of the luminance profile of the target plus noise). Position acuity is much less acute in the parafovea, and this is reflected in the reduced root efficiency (i.e., square root of efficiency) and the coarse classification images for peripheral position discrimination. The peripheral position template is a low spatial frequency template.     

The response of the amblyopic visual system to noise

Visual perception is limited by both the strength of the neural signals, and by the noise in the visual nervous system. Here we use one-dimensional white noise as input, to study the response of amblyopic visual system. We measured the thresholds for detection and discrimination of noise contrast. Using an N-pass reverse correlation technique, we derived classification images and estimated response consistency. Our results provide the first report of the sensitivity of the amblyopic visual system to white noise. We show that amblyopes have markedly reduced sensitivity for detecting noise, particularly at high spatial frequencies, and much less loss for discriminating suprathreshold noise contrast. Compensating for the detection loss almost (but not quite) equates performance of the amblyopic and normal visual system. The classification images suggest that the amblyopic visual system contains adjustable channels for noise, similar to those found in normal vision, but "tuned" to slightly lower spatial frequencies than in normal observers. Our N-pass results show that the predominant factor limiting performance in our task in both normal and amblyopic vision is internal random multiplicative noise. For the detection of white noise the raised thresholds of the amblyopic visual system can be attributed primarily to extra additive noise. However, for the discrimination of suprathreshold white noise contrast, there is surprisingly little additional deficit, after accounting for the visibility of the noise.     

Spatial profiles of local and nonlocal effects upon contrast detection/discrimination from classification images

We used classification images (A. J. Ahumada, Jr., & J. Lovell, 1971) to estimate the perceptual filter in a task designed to assess both local and nonlocal effects upon contrast detection/discrimination. Three observers performed a yes/no detection or discrimination task of a uniform circular decrement (radius = 0.68 deg) near threshold presented for 100 to 400 ms. Stimuli were presented in ring image noise that either covered the signal and an annular surrounding area (out to 1.36 deg), or only the surrounding annular area (out to 1.36 deg). Both the signal and the annular surround appeared on a uniform background. With ring noise over both the signal and surround, the amplitudes of the classification images in the signal area decreased as radial distance increased from the signal/surround border, and no effect of the surround was found. With ring noise only in the surround, classification images indicated noncontiguous effects at both the signal/surround border (local) and the surround/background border (nonlocal). The spatial extents of the nonlocal effects (< 0.07 deg) were smaller than local effects (0.25 deg), whereas the peak amplitudes of the local and nonlocal effects were comparable. These results suggest that the nonlocal effects were smaller than the local effects, and that the smaller effects would be due to smaller effective areas, as opposed to smaller amplitudes over the same area. Little or no change was found in the classification images across stimulus duration, suggesting that both the local and nonlocal processes found in this study were completed within 100 ms.     

Stereo slant discrimination of planar 3D surfaces: Frontoparallel versus planar matching

Binocular stereo cues are important for discriminating 3D surface orientation, especially at near distances. We devised a single-interval task where observers discriminated the slant of a densely textured planar test surface relative to a textured planar surround reference surface. Although surfaces were rendered with correct perspective, the stimuli were designed so that the binocular cues dominated performance. Slant discrimination performance was measured as a function of the reference slant and the level of uncorrelated white noise added to the test-plane images in the left and right eyes. We compared human performance with an approximate ideal observer (planar matching [PM]) and two subideal observers. The PM observer uses the image in one eye and back projection to predict a test image in the other eye for all possible slants, tilts, and distances. The estimated slant, tilt, and distance are determined by the prediction that most closely matches the measured image in the other eye. The first subideal observer (local planar matching [LPM]) applies PM over local neighborhoods and then pools estimates across the test plane. The second suboptimal observer (local frontoparallel matching [LFM]) uses only location disparity. We find that the ideal observer (PM) and the first subideal observer (LPM) outperforms the second subideal observer (LFM), demonstrating the additional benefit of pattern disparities. We also find that all three model observers can account for human performance, if two free parameters are included: a fixed small level of internal estimation noise, and a fixed overall efficiency scalar on slant discriminability.     

Classification image analysis: estimation and statistical inference for two-alternative forced-choice experiments

We consider estimation and statistical hypothesis testing on classification images obtained from the two-alternative forced-choice experimental paradigm. We begin with a probabilistic model of task performance for simple forced-choice detection and discrimination tasks. Particular attention is paid to general linear filter models because these models lead to a direct interpretation of the classification image as an estimate of the filter weights. We then describe an estimation procedure for obtaining classification images from observer data. A number of statistical tests are presented for testing various hypotheses from classification images based on some more compact set of features derived from them. As an example of how the methods we describe can be used, we present a case study investigating detection of a Gaussian bump profile.     

Is visual attention required for robust picture memory?

Humans can remember many scenes for a long time after brief presentation. Do scene understanding and encoding processes require visual selective attention, or do they occur even when observers are engaged in other visual tasks? We showed observers scene or texture images while they performed a visual search task, an auditory detection task, or no concurrent task. Concurrent tasks interfered with memory for both image types. Visual search interfered more than effects of auditory detection even when the two tasks were equally difficult. The same pattern of results was obtained with concurrent tasks presented during the encoding or consolidation phases. We conclude that visual attention modulates picture memory performance. We did not find any aspect of picture memory to be independent of attentional demands.     

Classification images predict absolute efficiency

How well do classification images characterize human observers' strategies in perceptual tasks? We show mathematically that from the classification image of a noisy linear observer, it is possible to recover the observer's absolute efficiency. If we could similarly predict human observers' performance from their classification images, this would suggest that the linear model that underlies use of the classification image method is adequate over the small range of stimuli typically encountered in a classification image experiment, and that a classification image captures most important aspects of human observers' performance over this range. In a contrast discrimination task and in a shape discrimination task, we found that observers' absolute efficiencies were generally well predicted by their classification images, although consistently slightly (approximately 13%) higher than predicted. We consider whether a number of plausible nonlinearities can account for the slight under prediction, and of these we find that only a form of phase uncertainty can account for the discrepancy.     

The footprints of visual attention in the Posner cueing paradigm revealed by classification images

In the Posner cueing paradigm, observers' performance in detecting a target is typically better in trials in which the target is present at the cued location than in trials in which the target appears at the uncued location. This effect can be explained in terms of a Bayesian observer where visual attention simply weights the information differently at the cued (attended) and uncued (unattended) locations without a change in the quality of processing at each location. Alternatively, it could also be explained in terms of visual attention changing the shape of the perceptual filter at the cued location. In this study, we use the classification image technique to compare the human perceptual filters at the cued and uncued locations in a contrast discrimination task. We did not find statistically significant differences between the shapes of the inferred perceptual filters across the two locations, nor did the observed differences account for the measured cueing effects in human observers. Instead, we found a difference in the magnitude of the classification images, supporting the idea that visual attention changes the weighting of information at the cued and uncued location, but does not change the quality of processing at each individual location.     

Visual search in noise: revealing the influence of structural cues by gaze-contingent classification image analysis

Visual search experiments have usually involved the detection of a salient target in the presence of distracters against a blank background. In such high signal-to-noise scenarios, observers have been shown to use visual cues such as color, size, and shape of the target to program their saccades during visual search. The degree to which these features affect search performance is usually measured using reaction times and detection accuracy. We asked whether human observers are able to use target features to succeed in visual search tasks in stimuli with very low signal-to-noise ratios. Using the classification image analysis technique, we investigated whether observers used structural cues to direct their fixations as they searched for simple geometric targets embedded at very low signal-to-noise ratios in noise stimuli that had the spectral characteristics of natural images. By analyzing properties of the noise stimulus at observers' fixations, we were able to reveal idiosyncratic, target-dependent features used by observers in our visual search task. We demonstrate that even in very noisy displays, observers do not search randomly, but in many cases they deploy their fixations to regions in the stimulus that resemble some aspect of the target in their local image features.     

Visual search: the role of peripheral information measured using gaze-contingent displays

Two of the factors limiting progress in understanding the mechanisms of visual search are the difficulty of controlling and manipulating the retinal stimulus when the eyes are free to move and the lack of an ideal observer theory for fixation selection during search. Recently, we developed a method to precisely control retinal stimulation with gaze-contingent displays (J. S. Perry & W. S. Geisler, 2002), and we derived a theory of optimal eye movements in visual search (J. Najemnik & W. S. Geisler, 2005). Here, we report a parametric study of visual search for sine-wave targets added to spatial noise backgrounds that have spectral characteristics similar to natural images (the amplitude spectrum of the noise falls inversely with spatial frequency). Search time, search accuracy, and eye fixations were measured as a function of target spatial frequency, 1/f noise contrast, and the resolution falloff of the display from the point of fixation. The results are systematic and similar for the two observers. We find that many aspects of search performance and eye movement pattern are similar to those of an ideal searcher that has the same falloff in resolution with retinal eccentricity as the human visual system.     

What is the signal in noise?

Visual perception is limited by both the strength of the neural signals, and by the noise in the visual nervous system; however, little is known about what aspects of the input noise the human visual system is sensitive to, i.e., what is the signal in noise? In order to investigate this question we asked observers to discriminate differences in the strength of one-dimensional white noise. We measured their response consistency and classification images and compared the results with an ideal energy detector. Our results and modelling show that discrimination of noise is limited by the observer's template (i.e., the weighted combination of energy in each stimulus component) plus higher order nonlinearities (systematic noise), and by sources of random internal noise. We found that systematic noise is present only near detection threshold. Surprisingly, we found that the human template is "adaptive"-- its shape depends on the spatial frequency band of the noise-suggesting that sensitivity to spatial noise is not simply determined via passive filtering.     

Phase noise and the classification of natural images

We measured the effect of global phase manipulations on a rapid animal categorization task. The Fourier spectra of our images of natural scenes were manipulated by adding zero-mean random phase noise at all spatial frequencies. The phase noise was the independent variable, uniformly and symmetrically distributed between 0 degrees and +/-180 degrees . Subjects were remarkably resistant to phase noise. Even with +/-120 degrees phase noise subjects were still performing at 75% correct. The high resistance of the subjects' animal categorization rate to phase noise suggests that the visual system is highly robust to such random image changes. The proportion of correct answers closely followed the correlation between original and the phase noise-distorted images. Animal detection rate was higher when the same task was performed with contrast reduced versions of the same natural images, at contrasts where the contrast reduction mimicked that resulting from our phase randomization. Since the subjects' categorization rate was better in the contrast experiment, reduction of local contrast alone cannot explain the performance in the phase noise experiment. This result obtained with natural images differs from those obtained for simple sinusoidal stimuli were performance changes due to phase changes are attributed to local contrast changes only. Thus the global phase-change accompanying disruption of image structure such as edges and object boundaries at different spatial scales reduces object classification over and above the performance deficit resulting from reducing contrast. Additional color information improves the categorization performance by 2%.     

Stimulus-driven mechanisms underlying visual search asymmetry revealed by classification image analyses

Search asymmetry is a robust phenomenon with various stimuli and is important for understanding determinants of efficiency in visual search. However, its underlying mechanism remains unknown due to the lack of a method for estimating visual features used by human observers. This study used a classification image technique to solve this problem. Standard classification image analyses with an experiment of visual search asymmetry between Q and O revealed that observers used the same features in both search tasks, rejecting a hypothesis incorporating top-down feature selection. More quantitative data analysis and an additional experiment with a singleton search task also rejected target-dependent selective tuning of the common feature. Further model-based analyses revealed that a standard signal detection model with nonlinear signal transduction and multiplicative internal noise is sufficient to account for the classification image data. Contrary to intuitively appealing accounts based on attention and spatial uncertainty, these findings suggest that search asymmetry is a characteristic of elementary visual processing of multiple items by a nonlinear system. The classification image technique is a valuable tool for investigating search behavior beyond mere visualization of visual features.     

Some observations on contrast detection in noise

The standard psychophysical model of our early visual system consists of a linear filter stage, followed by a nonlinearity and an internal noise source. If a rectification mechanism is introduced at the output of the linear filter stage, as has been suggested on some occasions, this model actually predicts that human performance in a classical contrast detection task might benefit from the addition of weak levels of noise. Here, this prediction was tested and confirmed in two contrast detection tasks. In Experiment 1, observers had to discriminate a low-contrast Gabor pattern from a blank. In Experiment 2, observers had to discriminate two low-contrast Gabor patterns identical on all dimensions, except for orientation (-45 degrees vs. +45 degrees). In both experiments, weak-to-modest levels of 2-D, white noise were added to the stimuli. Detection thresholds vary nonmonotonically with noise power, i.e., some noise levels improve contrast detection performance. Both simple uncertainty reduction and an energy discrimination strategy can be excluded as possible explanations for this effect. We present a quantitative model consistent with the effects and discuss the implications.     

What limits performance in the amblyopic visual system: seeing signals in noise with an amblyopic brain

Amblyopia results in a loss of visual acuity, contrast sensitivity, and position acuity. However, the nature of the neural losses is not yet fully understood. Here we report the results of experiments using noise to try to better understand the losses in amblyopia. Specifically, in one experiment we compared the performance of normal, amblyopic, and ideal observers for detecting a localized signal (a discrete frequency pattern or DFP) in fixed contrast white noise. In a second experiment, we used visibility-scaled noise and varied both the visibility of the noise (from 2 to 20 times the noise detection threshold) and the spatial frequency of the signal. Our results show a loss of efficiency for detection of known signals in noise that increases with the spatial frequency of the signal in observers with amblyopia. To determine whether the loss of efficiency was a consequence of a mismatched template, we derived classification images. We found that although the amblyopic observers' template was shifted to lower spatial frequencies, the shift was insufficient to account for their threshold elevation. Reduced efficiency in the amblyopic visual system may reflect a high level of internal noise, a poorly matched position template, or both. To analyze the type of internal noise we used an "N-pass" technique, in which observers performed the identical experiment N times (where N = 3 or 4). The amount of disagreement between the repeated trials enables us to parse the internal noise into random noise and consistent noise beyond that due to the poorly matched template. Our results show that the amblyopes' reduced efficiency for detecting signals in noise is explained in part by reduced template efficiency but to a greater extent by increased random internal noise. This loss is more or less independent of external noise contrast over a log unit range of external noise.     

Perceptual classification of chromatic modulation

We measured the regions of the equiluminant plane that are exploited by observers during a Yes/No detection task. The signal was a 640-ms Gaussian modulation (sigma(t) = 160 ms) of a Gaussian spatial patch (sigma(s) = 2.4 deg) presented in chromatically bivariate uniform noise. One component of the noise was along the direction axial with the signal in color space, the other perpendicular. Four signal directions were tested: along cardinal LM and S axes and two intermediate directions to which the cardinal axes were equally sensitive. The distribution of noise chromaticities from each trial was correlated with the observers' responses and the presence and absence of the signal to build a classification image of the distribution of chromaticities on which the decision of the observer was based. The images show a narrowly selective peak in the signal direction flanked by regions with a broader selectivity. These results raise the possibility that detection judgments are mediated by both linear and nonlinear mechanisms with peak sensitivities between the cardinal directions.     

Classification image weights and internal noise level estimation

For the linear discrimination of two stimuli in white Gaussian noise in the presence of internal noise, a method is described for estimating linear classification weights from the sum of noise images segregated by stimulus and response. The recommended method for combining the two response images for the same stimulus is to difference the average images. Weights are derived for combining images over stimuli and observers. Methods for estimating the level of internal noise are described with emphasis on the case of repeated presentations of the same noise sample. Simple tests for particular hypotheses about the weights are shown based on observer agreement with a noiseless version of the hypothesis.     

An investigation of the ability of the human visual system to encode spatial phase relationships

Evidence is presented that the human visual system contains broad-band mechanisms capable of encoding the spatial phase relationship between a fundamental spatial frequency and higher frequencies up to its third harmonic. Compounds of a fundamental and its harmonics above the third become progressively more difficult to discriminate by means of phase information alone. Measurements were also made of the amount of spatial summation found in various detection and phase discrimination tasks using simple or compound gratings composed of a fundamental frequency (F) and its third harmonic (3F). A task requiring the discrimination of the phase relationship between a low contrast (F) and a high contrast (3F) shows less spatial summation than does a task requiring the detection of the (F) component by itself. A simple model is advanced to account for these results qualitatively. The model is based upon the hypothesis that the human visual system analyses the retinal image in patches of a range of sizes and that phase relationships may be discerned only between components that are detected by different elements of the same patch mechanism.