Effect of vertical disparities on depth representation in macaque monkeys: MT physiology and behavior

Horizontal binocular disparities provide information about the distance of objects relative to the point of ocular fixation and must be combined with an estimate of viewing distance to recover the egocentric distance of an object. Vergence angle and the gradient of vertical disparities across the visual field are thought to provide independent sources of viewing distance information based on human behavioral studies. Although the effect of vergence angle on horizontal disparity selectivity in early visual cortex has been examined (with mixed results), the effect of the vertical disparity field has not been explored. We manipulated the vertical disparities in a large random-dot stimulus to simulate different viewing distances, and we examined the effect of this manipulation on both the responses of neurons in the middle temporal (MT) area and on the psychophysical performance of the animal in a curvature discrimination task. We report here that alterations to the vertical disparity field have no effect on the horizontal disparity tuning of MT neurons. However, the same manipulation strongly and systematically biases the monkey's judgments of curvature, consistent with previous human studies. We conclude that monkeys, like humans, make use of the vertical disparity field to estimate viewing distance, but that the physiological mechanisms for this effect occur either downstream of MT or in a different pathway.     

Coding of horizontal disparity and velocity by MT neurons in the alert macaque

We performed the first large-scale (n = 501), quantitative study of horizontal disparity tuning in the middle temporal (MT) visual area of alert, fixating macaque monkeys. Using random-dot stereograms, we quantified the direction tuning, speed tuning, horizontal disparity tuning, and size tuning of each neuron. The vast majority (93%) of MT neurons were significantly tuned for horizontal disparity. Although disparity tuning was generally quite robust, the average disparity sensitivity of MT neurons was significantly weaker than their direction or speed sensitivity as quantified using both an index of response modulation and an index of signal-to-noise ratio. Disparity tuning was not correlated with direction or size tuning but tended to be broader and weaker for neurons that preferred faster speeds of motion. By comparison with recent studies, we find that disparity selectivity in MT is substantially stronger than that seen in either primary visual cortex (V1) or area V4. In addition, MT neurons are more broadly tuned for disparity than V1 neurons at comparable eccentricities. Disparity tuning curves are very well described by Gabor functions for >80% of MT neurons. The distribution of Gabor phases shows clear bimodality, indicating that MT neurons tend to have odd-symmetric disparity tuning (unlike neurons in V1). The preferred disparities were more strongly correlated with the phase parameter of the Gabor function than with the positional offset parameter. In fact, for neurons with preferred disparities close to zero, the positional offset tended to oppose the phase shift in specifying the disparity preference. We suggest that this result reflects a strategy used to finely distribute the disparity preferences of MT neurons, given the predominance of odd-symmetry and broad tuning.     

Cell responses to vertical and horizontal retinal disparities in the monkey visual cortex

Because of the horizontal separation of both ocular globes, the projection angles are slightly different. These differences are commonly termed retinal disparities. Vertical and horizontal retinal disparities occur constantly in normal life. We have investigated the responses of single cells in cortical areas V1 and V2 of behaving Macaca mulatta monkeys to retinal disparities by using dynamic random dot stereograms. Our findings show that cortical visual cells are sensitive to both vertical and horizontal disparities. To calculate the distance between two objects in a three-dimensional space from horizontal disparities, it is necessary to know the fixation distance. It has been suggested that the horizontal gradient of vertical disparity contains information to estimate the fixation distance and therefore to scale horizontal disparities. We suggest that these cells sensitive to horizontal and vertical disparities represent a neural mechanism that provides information to the visual system in order to achieve a correct eye alignment and depth perception.     

Horizontal-disparity tuning of neurons in the visual forebrain of the behaving barn owl

Stereovision plays a major role in depth perception of animals having frontally-oriented eyes, most notably primates, cats, and owls. Neuronal mechanisms of disparity sensitivity have only been investigated in anesthetized owls so far. In the current study, responses of 160 visual Wulst neurons to static random-dot stereograms (RDS) were recorded via radiotelemetry in awake, fixating barn owls. The majority of neurons (76%) discharged significantly as a function of horizontal disparity in RDS. The distribution of preferred disparities mirrored the behaviorally relevant range of horizontal disparities that owls can exploit for depth vision. Most tuning profiles displayed periodic modulation and could well be fitted with a Gabor function as expected if disparity detectors were implemented according to the disparity energy model. Corresponding to this observation, a continuum of tuning profiles was observed rather than discrete categories. To assess a possible clustering of neurons with similar disparity-tuning properties, single units, and multi-unit activity recorded at individual recording sites were compared. Only a minority of neurons were clustered according to their disparity-tuning properties, suggesting that neurons in the visual Wulst are not organized into columns by preferred disparity. To assess whether variable vergence eye movements influenced tuning data, we correlated tuning peak positions on a trial-by-trial basis for units that were recorded simultaneously. The general lack of significant correlation between single-trial peak positions of simultaneously recorded units indicated that vergence, if at all, had only a minor influence on the data. Our study emphasizes the significance of visual Wulst neurons in analyzing stereoscopic depth information and introduces the barn owl as a second model system to study stereopsis in awake, behaving animals.     

Updating target distance across eye movements in depth

We tested between two coding mechanisms that the brain may use to retain distance information about a target for a reaching movement across vergence eye movements. If the brain was to encode a retinal disparity representation (retinal model), i.e., target depth relative to the plane of fixation, each vergence eye movement would require an active update of this representation to preserve depth constancy. Alternatively, if the brain was to store an egocentric distance representation of the target by integrating retinal disparity and vergence signals at the moment of target presentation, this representation should remain stable across subsequent vergence shifts (nonretinal model). We tested between these schemes by measuring errors of human reaching movements (n = 14 subjects) to remembered targets, briefly presented before a vergence eye movement. For comparison, we also tested their directional accuracy across version eye movements. With intervening vergence shifts, the memory-guided reaches showed an error pattern that was based on the new eye position and on the depth of the remembered target relative to that position. This suggests that target depth is recomputed after the gaze shift, supporting the retinal model. Our results also confirm earlier literature showing retinal updating of target direction. Furthermore, regression analyses revealed updating gains close to one for both target depth and direction, suggesting that the errors arise after the updating stage during the subsequent reference frame transformations that are involved in reaching.     

Disparity sensitivity of frontal eye field neurons

Information about depth is necessary to generate saccades to visual stimuli located in three-dimensional space. To determine whether monkey frontal eye field (FEF) neurons play a role in the visuo-motor processes underlying this behavior, we studied their visual responses to stimuli at different disparities. Disparity sensitivity was tested from 3 degrees of crossed disparity (near) to 3 degrees degrees of uncrossed disparity (far). The responses of about two thirds of FEF visual and visuo-movement neurons were sensitive to disparity and showed a broad tuning in depth for near or far disparities. Early phasic and late tonic visual responses often displayed different disparity sensitivity. These findings provide evidence of depth-related signals in FEF and suggest a role for FEF in the control of disconjugate as well as conjugate eye movements.     

Human cortical activity correlates with stereoscopic depth perception

Stereoscopic depth perception is based on binocular disparities. Although neurons in primary visual cortex (V1) are selective for binocular disparity, their responses do not explicitly code perceived depth. The stereoscopic pathway must therefore include additional processing beyond V1. We used functional magnetic resonance imaging (fMRI) to examine stereo processing in V1 and other areas of visual cortex. We created stereoscopic stimuli that portrayed two planes of dots in depth, placed symmetrically about the plane of fixation, or else asymmetrically with both planes either nearer or farther than fixation. The interplane disparity was varied parametrically to determine the stereoacuity threshold (the smallest detectable disparity) and the upper depth limit (largest detectable disparity). fMRI was then used to quantify cortical activity across the entire range of detectable interplane disparities. Measured cortical activity covaried with psychophysical measures of stereoscopic depth perception. Activity increased as the interplane disparity increased above the stereoacuity threshold and dropped as interplane disparity approached the upper depth limit. From the fMRI data and an assumption that V1 encodes absolute retinal disparity, we predicted that the mean response of V1 neurons should be a bimodal function of disparity. A post hoc analysis of electrophysiological recordings of single neurons in macaques revealed that, although the average firing rate was a bimodal function of disparity (as predicted), the precise shape of the function cannot fully explain the fMRI data. Although there was widespread activity within the extrastriate cortex (consistent with electrophysiological recordings of single neurons), area V3A showed remarkable sensitivity to stereoscopic stimuli, suggesting that neurons in V3A may play a special role in the stereo pathway.     

Multimodal representation of optic flow in area PEc of macaque monkey

The visual perception of self-motion is mainly provided by optic flow. Eyes usually scan the environment during locomotion, and the gaze is not always directed to the focus of expansion (FOE) of the flow field. Such eye movements change the retinal FOE position with respect to the fovea. Here, we assess if optic flow selective neurons in parietal area PEc are modulated by eye position. We recorded single neuron activity during radial optic flow stimulation in two monkeys, varying eye and retinal FOE positions. We found that the majority of PEc neurons are modulated by the FOE retinotopic position with different tuning for expansion and contraction. Although many neurons did not show any gaze field without visual stimulation, the eye position modulated optic flow responses in about half of the cells. These novel results suggest that PEc neurons integrate both visual and eye position signals, and allow us to hypothesize their role in guiding locomotion as a part of a cortical network involved in FOE representation during self-motion. Visual and eye position interaction in this area could be seen as a contribution to the building of the invariant space representation necessary to motor planning.     

Stimulus dependence of disparity coding in primate visual area V4

Disparity tuning in visual cortex has been shown using a variety of stimulus types that contain stereoscopic depth cues. It is not known whether different stimuli yield similar disparity tuning curves. We studied whether cells in visual area V4 of the macaque show similar disparity tuning profiles when the same set of disparity values were tested using bars or dynamic random dot stereograms, which are among the most commonly used stimuli for this purpose. In a majority of V4 cells (61%), the shape of the disparity tuning profile differed significantly for the two stimulus types. The two sets of stimuli yielded statistically indistinguishable disparity tuning profiles for only a small minority (6%) of V4 cells. These results indicate that disparity tuning in V4 is stimulus-dependent. Given the fact that bar stimuli contain two-dimensional (2-D) shape cues, and the random dot stereograms do not, our results also indicate that V4 cells represent 2-D shape and binocular disparity in an interdependent fashion, revealing an unexpected complexity in the analysis of depth and three-dimensional shape.     

Disparity selectivity of neurons in monkey inferior temporal cortex

The inferior temporal cortex (IT) of the monkey, a final stage in the ventral visual pathway, has been known to process information on two-dimensional (2-D) shape, color, and texture. On the other hand, the dorsal visual pathway leading to the posterior parietal cortex has been known to process information on location in space. Likewise, neurons selective for binocular disparity, which convey information on depth, have been found mainly in areas along the dorsal visual pathway. Here, we report that many neurons in the IT are also selective for binocular disparity. We recorded extracellular activity from IT neurons and found that more than half of the neurons changed their response depending on the disparity added. The change was not attributed to monocular responses or eye movements. Most neurons selective for disparity were "near" or "far" cells; they preferred either crossed or uncrossed disparity, and only a small population was tuned to zero disparity. Disparity-selective neurons were also selective for shape. Most preferred the same type of disparity irrespective of the shape presented. Disparity preference was also invariant for the fronto-parallel translation of the stimuli in most of the neurons. Finally, nearby neurons exhibited similar disparity selectivity, suggesting the existence of a functional module for processing of binocular disparity in the IT. From the above and our recent findings, we suggest that the IT integrates shape and binocular disparity information, and plays an important role in the reconstruction of three-dimensional (3-D) surfaces.     

Ocular dominance predicts neither strength nor class of disparity selectivity with random-dot stimuli in primate V1

We address two unresolved issues concerning the coding of binocular disparity in primary visual cortex. Experimental studies and theoretical models have suggested a relationship between a cell's ocular dominance, assessed with monocular stimuli, and its tuning to binocular disparity. First, the disparity energy model of disparity selectivity suggests that there should be a correlation between ocular dominance and the strength of disparity tuning. Second, several studies have reported a relationship between ocular dominance and the shape of the disparity tuning curve, with cells dominated by one eye more likely to have disparity tuning of the tuned-inhibitory type. We investigated both of these relationships in single neurons recorded from the primary visual cortex of awake fixating macaques, using dynamic random-dot patterns as a stimulus. To classify disparity tuning curves quantitatively, we develop a new measure of symmetry, which can be applied to any function. We find no evidence for any correlation between ocular dominance and the nature of disparity tuning. This places constraints on the circuitry underlying disparity tuning.     

Space coding by premotor cortex

Summary Many neurons in inferior area 6, a cortical premotor area, respond to visual stimuli presented in the space around the animal. We were interested to learn whether the receptive fields of these neurons are coded in retinotopic or in body-centered coordinates. To this purpose we recorded single neurons from inferior area 6 (F4 sector) in a monkey trained to fixate a light and detect its dimming. During fixation visual stimuli were moved towards the monkey both within and outside the neurons's receptive field. The fixation point was then moved and the neuron retested with the monkey's gaze deviated to the new location. The results showed that most inferior area 6 visual neurons code the stimulus position in spatial and not in retinal coordinates. It is proposed that these visual neurons are involved in generating the stable body-centered frame of reference necesary for programming visually guided movements.     

Processing of shape defined by disparity in monkey inferior temporal cortex

Neurons in the monkey inferior temporal cortex (IT) have been shown to respond to shapes defined by luminance, texture, or motion. In the present study, we determined whether IT neurons respond to shapes defined solely by binocular disparity, and if so, whether signals of disparity and other visual cues to define shape converge on single IT neurons. We recorded extracellular activity from IT neurons while monkeys performed a fixation task. Among the neurons that responded to at least one of eight random-dot stereograms (RDSs) containing different disparity-defined shapes, 21% varied their responses to different RDSs. Responses of most of the neurons were positively correlated between two sets of RDSs, which consisted of different dot patterns but defined the same set of eight shapes, whereas responses to RDSs and their monocular images were not correlated. This indicates that the response modulation for the eight RDSs reflects selectivity for shapes (or their component contours) defined by disparity, although responses were also affected by dot patterns per se. Among the neurons that showed selectivity for shapes defined by luminance or disparity, 44% were activated by both cues. Responses of these neurons to luminance-defined shapes and those to disparity-defined shapes were often positively correlated to each other. Furthermore the stimulus rank, which was determined by the magnitude of responses to shapes, generally matched between these cues. The same held true between disparity and texture cues. The results suggest that the signals of disparity, luminance, and texture cues to define the shapes converge on a population of single IT neurons to produce the selectivity for shapes.     

The contribution of vision and proprioception to judgements of finger proximity

We sought to determine whether an increase in judged egocentric distance created by increasing vergence-specified distance would be negated when participants pointed at their own finger. It was found that ocular position dominates limb proprioception in the judgement of finger distance in the sagittal plane when vision is available. In contrast, an increase in perceived egocentric distance was largely attenuated by the presence of limb proprioception in reduced visual cue conditions. We conclude that the relative contribution of vergence to perceived distance depends upon the strength of the vergence effort signal when there are other cues present. Furthermore, if the distance percept includes a major contribution from retinal cues, then the visual component will dominate the limb proprioception component. If the visual component is largely determined by vergence information, limb proprioception will make a significant contribution and actually dominate when the vergence effort signal is weak. The results extend previous studies that have found a similar relationship between ocular position and limb proprioception in the perception of a finger′s location in the coronal plane.     

Representation of 3-D surface orientation by velocity and disparity gradient cues in area MT

Neural coding of the three-dimensional (3-D) orientation of planar surface patches may be an important intermediate step in constructing representations of complex 3-D surface structure. Spatial gradients of binocular disparity, image velocity, and texture provide potent cues to the 3-D orientation (tilt and slant) of planar surfaces. Previous studies have described neurons in both dorsal and ventral stream areas that are selective for surface tilt based on one or more of these gradient cues. However, relatively little is known about whether single neurons provide consistent information about surface orientation from multiple gradient cues. Moreover, it is unclear how neural responses to combinations of surface orientation cues are related to responses to the individual cues. We measured responses of middle temporal (MT) neurons to random dot stimuli that simulated planar surfaces at a variety of tilts and slants. Four cue conditions were tested: disparity, velocity, and texture gradients alone, as well as all three gradient cues combined. Many neurons showed robust tuning for surface tilt based on disparity and velocity gradients, with relatively little selectivity for texture gradients. Some neurons showed consistent tilt preferences for disparity and velocity cues, whereas others showed large discrepancies. Responses to the combined stimulus were generally well described as a weighted linear sum of responses to the individual cues, even when disparity and velocity preferences were discrepant. These findings suggest that area MT contains a rudimentary representation of 3-D surface orientation based on multiple cues, with single neurons implementing a simple cue integration rule.     

Spatial frequency integration for binocular correspondence in macaque area V4

Neurons in the primary visual cortex (V1) detect binocular disparity by computing the local disparity energy of stereo images. The representation of binocular disparity in V1 contradicts the global correspondence when the image is binocularly anticorrelated. To solve the stereo correspondence problem, this rudimentary representation of stereoscopic depth needs to be further processed in the extrastriate cortex. Integrating signals over multiple spatial frequency channels is one possible mechanism supported by theoretical and psychophysical studies. We examined selectivities of single V4 neurons for both binocular disparity and spatial frequency in two awake, fixating monkeys. Disparity tuning was examined with a binocularly correlated random-dot stereogram (RDS) as well as its anticorrelated counterpart, whereas spatial frequency tuning was examined with a sine wave grating or a narrowband noise. Neurons with broader spatial frequency tuning exhibited more attenuated disparity tuning for the anticorrelated RDS. Additional rectification at the output of the energy model does not likely account for this attenuation because the degree of attenuation does not differ among the various types of disparity-tuned neurons. The results suggest that disparity energy signals are integrated across spatial frequency channels for generating a representation of stereoscopic depth in V4.     

Architecture of binocular disparity processing in monkey inferior temporal cortex

Neurons in the inferior temporal (IT) cortex respond not only to the shape, color or texture of objects, but to the horizontal positional disparity of visual features in the right and left retinal images. IT neurons with similar shape selectivity cluster in columns. In this study, we examined how IT neurons are spatially arranged in the IT according to their selectivity for binocular disparity. With a single electrode, we simultaneously recorded extracellular action potentials from a single neuron and those from background multiple neurons at the same sites or recorded multineuronal responses at successive sites along electrode penetrations, while monkeys performed a fixation task. For neurons at each recording site, effective shapes were first determined from a set of 20 shapes presented at the zero-disparity plane. The most effective shape was then presented with varying amounts of disparity. Single neuron responses and background multiunit responses recorded at the same sites showed a similar ability of disparity discrimination and tended to share the preferred disparity, suggesting that neurons with similar disparity selectivity are clustered in the IT. We estimated from sequential recordings along electrode penetrations that the size of the neuronal clusters with similar disparity selectivity was smaller than the size of clusters with similar shape selectivity.     

Quantitative characterization of disparity tuning in ventral pathway area V4

We performed a quantitative characterization of binocular disparity-tuning functions in the ventral (object-processing) pathway of the macaque visual cortex. We measured responses of 452 area V4 neurons to stimuli with disparities ranging from -1.0 to +1.0 degrees. Asymmetric Gaussian functions fit the raw data best (median R = 0.90), capturing both the modal components (local peaks in the -1.0 to +1.0 degrees range) and the monotonic components (linear or sigmoidal dependency on disparity) of the tuning patterns. Values derived from the asymmetric Gaussian fits were used to characterize neurons on a modal x monotonic tuning domain. Points along the modal tuning axis correspond to classic tuned excitatory and inhibitory patterns; points along the monotonic axis correspond to classic near and far patterns. The distribution on this domain was continuous, with the majority of neurons exhibiting a mixed modal/monotonic tuning pattern. The distribution in the modal dimension was shifted toward excitatory patterns, consistent with previous results in other areas. The distribution in the monotonic dimension was shifted toward tuning for crossed disparities (corresponding to stimuli nearer than the fixation plane). This could reflect a perceptual emphasis on objects or object parts closer to the observer. We also found that disparity-tuning strength was positively correlated with orientation-tuning strength and color-tuning strength, and negatively correlated with receptive field eccentricity.     

Attention-related neurons in the supplementary eye field of the macaque monkey

This study investigated whether the neuronal activity of a cortical area involved in the control of eye fixation is affected by the covert orienting of attention. We recorded single-unit activity from the supplementary eye field (SEF) of two macaque monkeys performing fixation and peripheral-attention tasks. Ninety-nine out of four hundred and fifteen cells were related to eye movements. The other neurons showed relationship with postural adjustments, and arm and ear movements. Fifty-five neurons were active during fixation (fixation cells) and 44 discharged in relation to saccades. The experiments reported here primarily concern the fixation cells. The activity of 64% (35/55) of fixation cells started with the onset of visual stimulus, before the visual input reached the fovea, and continued during active fixation. The activity of 27% (15/55) of fixation cells started with the onset of fixation. The activity of 9% (5/55) of fixation cells modified their timing trial by trial. Sixty-four percent of the fixation cells (35/55) were position-dependent, showing a selective spatial field of activity, 36% (20/55) were position-independent and characterized by a full spatial field. None of the 55 cells showed a visual receptive field. We tested both types of fixation cells by means of a peripheral attention task. When attention was oriented peripherally toward a target located in the selective spatial field, the cells discharged as if the gaze was held toward it. When attention was oriented peripherally toward a target, lying outside the selective spatial field, the cells were inactive as if gaze was held in that position. These results suggest that the supplementary eye field neurons may code for oriented attention in space and might be involved in the preparation of motor action. Preliminary results were presented at the 1994 ENA meeting in abstract form     

Apparent motion cues distort object localisation in egocentric space

The visual localisation of objects in space is thought to rely on retinal information defining the environmental context and non-retinal cues from proprioception and motor commands. Here, the influence of dynamic contextual cues on the perception of egocentric space in a reaching task was investigated. Compared to performances with realistic motion or static cues, target localisation was less accurate when apparent motion was used to provide contextual information about space between the hand and the target. This effect could not be explained by the 'presence' of motion, or a bias in depth perception. Since the distortion was connected with the reaching area it was concluded that cognitive factors can unconsciously influence the perception of egocentric space, in particular distance estimation. We propose a mechanism for this whereby signals from areas MT/MST (middle temporal/medial superior temporal) create a perceptual bias through cortico-cortical connections with posterior parietal cortex. Electronic Publication