Testing a counter-intuitive prediction of optimal cue combination

Weighted averaging is said to be optimal when the weights assigned to the cues minimize the variance of the final estimate. Since the variance of this optimal percept only depends on the variances of the individual cues, irrespective of their values, judgments about a cue conflict stimulus should have the same variance as ones about a cue consistent stimulus. We tested this counter-intuitive prediction with a slant matching experiment using monocular and binocular slant cues. We found that the slant was indeed matched with about the same variance when the cues indicated slants that differed by 15° as when they indicated the same slant.     

Slant cue are combined early in visual processing: Evidence from visual search

When looking for a target with a different slant than all the other objects, the time needed is independent of the number of other objects. Surface slant can be inferred from the two-dimensional images on the retinas using various cues. The information from different cues is subsequently combined to get a single estimate of slant. Is information from the individual cues or from the combined percept responsible for us so easily finding the target? To find out we compared combinations of two slant cues. The cues that we chose are retinal shape and binocular disparity. We compared search times for conditions with the same differences between the target and the other objects in each individual cue, but for each object the two cues either indicated the same slant or opposite slants. Search times were independent of the number of other items if the target clearly differed in perceived slant from the other items. Subjects systematically found the target faster when the cues indicated the same slant. We conclude that slant cues are combined locally throughout the visual field before the search process begins.     

Shape similarity and distance disparity as apparent motion correspondence cues

Apparent motion is perceived when two spots of light are presented successively in different locations. When more than one element is present in each frame, there is a correspondence problem in matching the elements in one frame with those in the other. We report the effects of shape similarity and distance disparity on the correspondence process. Twenty subjects were tested using a 2-AFC design. We found that both shape and distance cues are used by the correspondence process: when distance is the only cue the motion which is usually perceived is that involving the shorter distance; when shape is the only cue the motion involving two elements of the same shape is preferred. We also studied the interaction between the two cues when both were present. Quantitative measures of the relative strengths of these effects and of their interaction are reported. A Signal Detection Theory model is used to analyze these apparent motion correspondence effects.     

Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant

Most research on depth cue integration has focused on stimulus regimes in which stimuli contain the small cue conflicts that one might expect to normally arise from sensory noise. In these regimes, linear models for cue integration provide a good approximation to system performance. This article focuses on situations in which large cue conflicts can naturally occur in stimuli. We describe a Bayesian model for nonlinear cue integration that makes rational inferences about scenes across the entire range of possible cue conflicts. The model derives from the simple intuition that multiple properties of scenes or causal factors give rise to the image information associated with most cues. To make perceptual inferences about one property of a scene, an ideal observer must necessarily take into account the possible contribution of these other factors to the information provided by a cue. In the context of classical depth cues, large cue conflicts most commonly arise when one or another cue is generated by an object or scene that violates the strongest form of constraint that makes the cue informative. For example, when binocularly viewing a slanted trapezoid, the slant interpretation of the figure derived by assuming that the figure is rectangular may conflict greatly with the slant suggested by stereoscopic disparities. An optimal Bayesian estimator incorporates the possibility that different constraints might apply to objects in the world and robustly integrates cues with large conflicts by effectively switching between different internal models of the prior constraints underlying one or both cues. We performed two experiments to test the predictions of the model when applied to estimating surface slant from binocular disparities and the compression cue (the aspect ratio of figures in an image). The apparent weight that subjects gave to the compression cue decreased smoothly as a function of the conflict between the cues but did not shrink to zero; that is, subjects did not fully veto the compression cue at large cue conflicts. A Bayesian model that assumes a mixed prior distribution of figure shapes in the world, with a large proportion being very regular and a smaller proportion having random shapes, provides a good quantitative fit for subjects' performance. The best fitting model parameters are consistent with the sensory noise to be expected in measurements of figure shape, further supporting the Bayesian model as an account of robust cue integration.     

Integration of binocular disparity and monocular cues at near threshold level

We examined the dependency of the integration of multiple depth cues upon the combined cues and upon the consistency of depth information from different cues. For each observer, depth thresholds were measured by the use of stimuli in which different depth cues (motion parallax, binocular disparity, and monocular configuration) specified the surface undulating sinusoidally with different spatial frequencies and different phases. Analysis of d(') showed that the performance was better than the prediction of probability summation only when parallax and disparity cues specified an undulation with the same spatial frequency and same phase. The probability summation model overestimated the performance for the other conditions of combination of disparity and parallax, and for all of the conditions of combination of disparity and monocular configuration. These results suggest that the improvement in depth perception caused by integration of multiple cues depends on the type of combined cues, and that the visual system possibly integrates the depth information from different cues at different stages of the visual processing.     

Optimal integration of texture and motion cues to depth

We report the results of a depth-matching experiment in which subjects were asked to adjust the height of an ellipse until it matched the depth of a simulated cylinder defined by texture and motion cues. In one-third of the trials the shape of the cylinder was primarily given by motion information, in another one-third of the trials it was given by texture information, and on the remaining trials it was given by both sources of information. Two optimal cue combination models are described where optimality is defined in terms of Bayesian statistics. The parameter values of the models are set based on subjects' responses on trials when either the motion cue or the texture cue was informative. These models provide predictions of subjects' responses on trials when both cues were informative. The results indicate that one of the optimal models provides a good fit to the subjects' data, and the second model provides an exceptional fit. Because the predictions of the optimal models closely match the experimental data, we conclude that observers' cue-combination strategies are indeed optimal, at least under the conditions studied here.     

Experience-dependent integration of texture and motion cues to depth

Previous investigators have shown that observers' visual cue combination strategies are remarkably flexible in the sense that these strategies adapt on the basis of the estimated reliabilities of the visual cues. However, these researchers have not addressed how observers' acquire these estimated reliabilities. This article studies observers' abilities to learn cue combination strategies. Subjects made depth judgments about simulated cylinders whose shapes were indicated by motion and texture cues. Because the two cues could indicate different shapes, it was possible to design tasks in which one cue provided useful information for making depth judgments, whereas the other cue was irrelevant. The results of experiment 1 suggest that observers' cue combination strategies are adaptable as a function of training; subjects adjusted their cue combination rules to use a cue more heavily when the cue was informative on a task versus when the cue was irrelevant. Experiment 2 demonstrated that experience-dependent adaptation of cue combination rules is context-sensitive. On trials with presentations of short cylinders, one cue was informative, whereas on trials with presentations of tall cylinders, the other cue was informative. The results suggest that observers can learn multiple cue combination rules, and can learn to apply each rule in the appropriate context. Experiment 3 demonstrated a possible limitation on the context-sensitivity of adaptation of cue combination rules. One cue was informative on trials with presentations of cylinders at a left oblique orientation, whereas the other cue was informative on trials with presentations of cylinders at a right oblique orientation. The results indicate that observers did not learn to use different cue combination rules in different contexts under these circumstances. These results are consistent with the hypothesis that observers' visual systems are biased to learn to perceive in the same way views of bilaterally symmetric objects that differ solely by a symmetry transformation. Taken in conjunction with the results of Experiment 2, this means that the visual learning mechanism underlying cue combination adaptation is biased such that some sets of statistics are more easily learned than others.     

Combination of texture and color cues in visual segmentation

The visual system can use various cues to segment the visual scene into figure and background. We studied how human observers combine two of these cues, texture and color, in visual segmentation. In our task, the observers identified the orientation of an edge that was defined by a texture difference, a color difference, or both (cue combination). In a fourth condition, both texture and color information were available, but the texture and color edges were not spatially aligned (cue conflict). Performance markedly improved when the edges were defined by two cues, compared to the single-cue conditions. Observers only benefited from the two cues, however, when they were spatially aligned. A simple signal-detection model that incorporates interactions between texture and color processing accounts for the performance in all conditions. In a second experiment, we studied whether the observers are able to ignore a task-irrelevant cue in the segmentation task or whether it interferes with performance. Observers identified the orientation of an edge defined by one cue and were instructed to ignore the other cue. Three types of trial were intermixed: neutral trials, in which the second cue was absent; congruent trials, in which the second cue signaled the same edge as the target cue; and conflict trials, in which the second cue signaled an edge orthogonal to the target cue. Performance improved when the second cue was congruent with the target cue. Performance was impaired when the second cue was in conflict with the target cue, indicating that observers could not discount the second cue. We conclude that texture and color are not processed independently in visual segmentation.     

The role of positional and orientational disparity cues in human fusional response

Torsional retinal disparities about the line of sight are compensated by the cyclofusional mechanism. Cyclofusional stimuli generally contain two types of disparity cues: positional (point by point) and orientational disparities. All previously reported cyclofusional studies have utilized stimuli containing both positional and orientational disparities, but in this investigation the two disparity types were isolated in order to study their relative effectiveness in evoking fusional response. For most subjects, stimuli containing only positional disparities were found to evoke greater fusional ranges than stimuli containing both positional and orientational cues. Orientational disparities, however, always elicited much smaller ranges than stimuli containing either positional only or positional and orientational disparities.     

The intrinsic constraint approach to cue combination: an empirical and theoretical evaluation

We elucidate two properties of the intrinsic constraint (IC) model of depth cue combination (F. Domini, C. Caudek, & H. Tassinari, 2006). First, we show that IC combines depth cues in a weighted sum that maximizes the signal-to-noise ratio of the combined estimate. Second, we show that IC predicts that any two depth-matched pairs of stimuli are separated by equal numbers of just noticeable differences (JNDs) in depth. That is, IC posits a strong link between perceived depth and depth discrimination, much like some Fechnerian theories of sensory scaling. We test this prediction, and we find that it does not hold. We also find that depth discrimination performance approximately follows Weber's law, whereas IC assumes that depth discrimination thresholds are independent of baseline stimulus depth.     

The influence of perspective and disparity cues on the perception of slant.

A rectangle rotated about its vertical axis provides many cues regarding itsslantrelative to the frontoparallel plane includinglinear perspectiveandbinocular disparity. In this experiment, fiveOs estimated the slant of stereoscopic targets containing one, both, or neither of these cues. Data analysis indicated for figures with vertical contours separated by 3–80° horizontally that perspective was the most effective cue to slant. The results are discussed in relation to Freeman's theory of slant perception. Further quantitative work concerning the functional relationship between contour separation and the effectiveness of thedisparity codeis suggested.     

Visual summation of luminance lines and illusory contours induced by pictorial, motion, and disparity cues

Illusory contours where no contrast exists in the image can be seen between pairs of spatially separate but aligned inducing real contours defined either by pictorial cues (luminance contrasts or offset gratings), kinetic contrast, or binocular disparity contrast. In previous studies it has been shown that the detection of a thin luminous line is facilitated when the line is superimposed on illusory contours and the inducing flanking elements are defined by luminance contrast. By using a spatial forced-choice technique I show that luminous lines summate with illusory contours induced by luminance contrast, offset gratings, motion contrast, and disparity contrast when the line is superimposed on the illusory contour. Control experiments show that the positional cues, offered by the inducing contours, are unable to account for these results. It is suggested that real luminous lines or edges and illusory contours activate common neural mechanisms in the brain irrespectively of the stimulus attributes that induce the illusory contour.     

"Phase capture" in the perception of interpolated shape: cue combination and the influence function

This study was concerned with what stimulus information observers use to judge the shape of simple objects. We used a string of four Gabor patches to define a contour. A fifth, center patch served as a test pattern. The observers' task was to judge the location of the test pattern relative to the contour. The contour was either a straight line, or an arc with positive or negative curvature (the radius of curvature was either 2 or 6 deg). We asked whether phase shifts in the inner or outer pairs of patches distributed along the contour influence the perceived shape. That is, we measured the phase shift influence function. We found that shifting the inner patches of the string by 0.25 cycle results in almost complete phase capture (attraction) at the smallest separation (2 lambda), and the capture effect falls off rapidly with separation. A 0.25 cycle shift of the outer pair of patches has a much smaller effect, in the opposite direction (repulsion). In our experiments, the contour is defined by two cues--the cue provided by the Gabor carrier (the 'feature' cue) and that defined by the Gaussian envelope (the 'envelope' cue). Our phase shift influence function can be thought of as a cue combination task. An ideal observer would weight the cues by the inverse variance of the two cues. The variance in each of these cues predicts the main features of our results quite accurately.     

Optimal combination of form and motion cues in human heading perception

We examined what role motion-streak-like form information plays in heading perception. We presented observers with an integrated form and motion display in which random-dot pairs in a 3D cloud were oriented toward one direction on the screen (the form FOE) to form a radial Glass pattern while moving in a different direction in depth (the motion FOE). Observers' heading judgments were strongly biased toward the form FOE direction (weight: 0.78), and this bias decreased with the reduction of the salience of the global form structure in the Glass pattern. At the local level, the orientation of dot pairs in the Glass pattern can affect their perceived motion direction, leading to a shift of the perceived motion FOE direction in optic flow. However, this shift accounted for about half of the total bias. Using the measurements of the shifted motion FOE and the perceived form FOE directions, we found that at the global level, an optimal combination of these two cues could accurately predict the heading bias observed for the integrated display. Our findings support the claim that motion streaks are effective cues for self-motion perception, and humans make optimal use of both form and motion cues for heading perception.     

Depth from shading and disparity in humans and monkeys

A stimulus display was devised that enabled us to examine how effectively monkeys and humans can process shading and disparity cues for depth perception. The display allowed us to present these cues separately, in concert and in conflict with each other. An oddities discrimination task was used. Humans as well as monkeys were able to utilize both shading and disparity cues but shading cues were more effectively processed by humans. Humans and monkeys performed better and faster when the two cues were presented conjointly rather than singly. Performance was significantly degraded when the two cues were presented in conflict with each other suggesting that these cues are processed interactively at higher levels in the visual system. The fact that monkeys can effectively utilize depth information derived from shading and disparity indicates that they are a good animal model for the study of the neural mechanisms that underlie the processing of these two depth cues.     

Heterophoria and fixation disparity: a review

Heterophoria does not provide a reliable clue for ordering prisms in an asthenopic patient. The same reservation applies to associated phoria, as determined by prism correction of fixation disparity. Subjective tests for fixation disparity, even those with a fusionable fixation target, do not correctly indicate the vergence position of the eyes under natural viewing conditions. Attempts to measure fixation disparity on the basis of stereo disparity, using the "Measuring and Correction Methods of H.-J. Haase", have failed.     

Heterophoria and fixation disparity: A review

Heterophoria does not provide a reliable clue for ordering prisms in an asthenopic patient. The same reservation applies to associated phoria, as determined by prism correction of fixation disparity. Subjective tests for fixation disparity, even those with a fusionable fixation target, do not correctly indicate the vergence position of the eyes under natural viewing conditions. Attempts to measure fixation disparity on the basis of stereo disparity, using the "Measuring and Correction Methods of H.-J. Haase", have failed.     

Depth cue integration: stereopsis and image blur

Depth-of-focus limitations introduce spatial blur in images of three-dimensional scenes. It is not clear how the visual system combines depth information derived from image blur with information from other depth cues. Stereoscopic disparity is the pre-eminent depth cue, so experiments were conducted to investigate interactions between image blur and stereoscopic disparity. Observers viewed two random dot stereograms (RDSs) in a 2AFC task, and were required to identify the RDS depicting the greatest depth. In control observations, all dots in both RDSs were sharply defined. In experimental observations, one RDS contained only sharply defined dots, but the other contained differential spatial blur to introduce an additional depth cue. Results showed that the addition of differential blur made only a marginal difference to apparent depth separation, and only when the blur difference was consistent with the sign of disparity. Cue combination between blur and disparity cues is thus weighted very heavily in favour of the latter. It is shown that blur and disparity cues co-vary according to geometric optics. Since the two cues are effective over different distances, the visual system is not normally called upon to integrate them, and is most likely to make use of blur cues over distances beyond the range of disparity mechanisms.     

Salience from feature contrast: variations with texture density

The salience of popout targets was measured in regular line arrays as a function of texture density. Test targets (singletons with orientation, motion, or luminance contrast) presented at different raster widths were compared with reference lines (lines brighter than surrounding lines) presented at fixed raster width. The luminance at which the reference target appeared as salient as the particular test target was taken as a measure of the relative salience of the test target. For orientation or motion contrast, targets at medium to small raster widths were far more salient than targets in sparse or very dense line arrangements. For targets defined by luminance contrast, salience variations with texture density were less pronounced. Some subjects also reported salience for lines in sparse arrangements even when these did not display feature contrast. When such non-specific saliency effects were subtracted from the actual measurements, salience curves for orientation or motion contrast revealed peaks of increased sensitivity at line spacings below 2-3 deg and flat curves at larger grid sizes. In an additional experiment, saliency effects from orientation contrast were measured using texture lines of different size. Salience variations were commonly observed. However, the curves were not found to scale with the different sizes of texture elements but were constantly related to the free space between neighbouring lines. This suggests that peaks in the salience profiles reflect the limited spatial extent of the underlying neural mechanisms.     

Fixation disparity: clinical implications and utilization.

Research suggests that fixation disparity data is extremely useful in the assessment of the binocular and accommodative systems. However, clinicians have been slow to integrate this method of analysis into their practices. Fixation disparity measurement, under different fusional and accommodative demands, aids the practitioner in prescribing ophthalmic prism and/or plus lenses. In addition, it helps determine when vision therapy is appropriate and provides an objective assessment of the effectiveness of a vision therapy program. This paper discusses significant aspects of horizontal and vertical fixation disparity measurement, with particular emphasis on its clinical usefulness in understanding and resolving asthenopic symptoms attributable to vergence and/or accommodative dysfunction. The goal of this paper is to provide the practitioner with information so that fixation disparity testing can be incorporated within the clinical regimen.