Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study

Perception of surface orientation is an essential step for the reconstruction of the three-dimensional (3D) structure of an object. Human lesion and functional neuroimaging studies have demonstrated the importance of the parietal lobe in this task. In primate single-unit studies, neurons in the caudal part of the intraparietal sulcus (CIP) were found to be active during the extraction of surface orientation through monocular (two-dimensional) cues such as texture gradients and linear perspective as well as binocular (3D) cues such as disparity gradient and orientation disparity. We used event-related fMRI to study the functional neuroanatomy of surface orientation discrimination using stimuli with monocular depth cues in six volunteers. Both posterior (CIP) and anterior (AIP) areas within the intraparietal sulcus showed a stronger activation during surface orientation as compared with a control (color discrimination) task using identical stimuli. Furthermore, the signal changes in CIP showed a greater performance effect than those in AIP, suggesting that CIP is tightly linked to the discrimination task.     

Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP.

We investigated the effects of linear perspective and binocular disparity, as monocular and binocular depth cues, respectively, on the response of surface-orientation-selective (SOS) neurons in the caudal part of the lateral bank of the intraparietal sulcus (area CIP). During the single-unit recording, monkeys were required to perform the delayed-matching-to-sample (successive same/different discrimination) of discriminating surface orientation in stereoscopic computer graphics. Of 211 visually responsive neurons, 66 were intensively tested using the solid-figure stereogram (SFS) of a square plate with both disparity and perspective cues (D+P condition), and 62 of these were identified as SOS neurons for responding selectively to the orientation of stimuli. All these neurons were further tested using a solid figure with perspective cues alone (P-only condition), and 58% (36/62) of these showed selective response to the orientation of the stimuli. Of the 62 SOS neurons, 35 neurons were also tested using SFS with disparity cues alone (D-only condition) in addition to the D+P and P-only conditions. We classified these 35 neurons into four groups by comparing the response selectivity under the P-only and D-only conditions. More than one-half of these (19/35) were sensitive to both perspective and disparity cues (DP neurons), and nearly one-third (11/35) of these were sensitive to disparity cues alone (D neurons), but a few (2/35) were sensitive to perspective cues alone (P neurons). The remaining (3/35) neurons exhibited orientation selectivity only when both cues were present. In DP neurons, the preferred orientation under the D+P condition was correlated to those under the D-only and P-only conditions, and the response magnitude under the D+P condition was greater than those under the D-only and P-only conditions, suggesting the integration of both cues for the perception of surface orientation. However, in these neurons, the orientation tuning sharpness under the D+P and D-only conditions was higher than that under the P-only condition, suggesting the dominance of disparity cues. After the single-unit recording experiments, muscimol was microinjected into the recording site to temporarily inactivate its function. In all three effective cases out of six microinjection experiments, discrimination of a three-dimensional (3D) surface orientation was impaired when disparity cues alone were present. In only one effective case, when a relatively large amount of muscimol was microinjected, discrimination of a 3D surface orientation was impaired even when both disparity and perspective cues were present. These results suggest that linear perspective is an important cue for representations of a 3D surface of SOS neurons in area CIP, although it is less effective than binocular disparity, and that both of these depth cues may be integrated in area CIP for the perception of surface orientation in depth.     

Neural representation of three-dimensional features of manipulation objects with stereopsis

In the first part of this article, we review our neurophysiological studies of the hand-manipulation-related neurons in the anterior part of the lateral bank of the intraparietal sulcus (area AIP) . We describe the properties of visually responsive neurons in area AIP. Object-type visual-dominant neurons responded to the sight of objects and showed selectivity not only for simple geometrical shapes, but also for complex objects such as a knob-in-groove and a plate-in-groove. Some of the object-type visual-dominant neurons showed selectivity for the orientation of the longitudinal axis or the plane (surface) of a plate or a ring. In the second part of this article, we review our study of binocular visual neurons in the caudal part of the lateral bank of the intraparietal sulcus (c-IPS area), in particular, of axis-orientation-selective (AOS) neurons and surface-orientation-selective (SOS) neurons. AOS neurons preferred long and thin stimuli, were sensitive to binocular disparity, and tuned to the axis orientation in three-dimensional (3D) space. SOS neurons preferred broad and flat stimuli and were tuned to the surface orientation in depth. Some SOS neurons responded to a square in a random dot stereogram (RDS) with orientation tuning, suggesting that they encode surface orientation from a disparity gradient. Others responded to solid figure stereograms with orientation disparity and/or width disparity. It was concluded that the c-IPS area is a higher center for stereopsis, which integrates various binocular disparity signals received from the V3 complex and other prestriate areas to represent the neural code for 3D features. It may send projections to the AIP area and contribute to visual adjustment of the shape of the handgrip and/or hand orientation for manipulation and grasping. Neurons of the AIP area may also receive monocular cues of depth from the ventral visual pathway to discriminate the 3D shape of the object of manipulation. Received: 21 September 1998 / Accepted: 23 March 1999     

Representation of 3-D surface orientation by velocity and disparity gradient cues in area MT

Neural coding of the three-dimensional (3-D) orientation of planar surface patches may be an important intermediate step in constructing representations of complex 3-D surface structure. Spatial gradients of binocular disparity, image velocity, and texture provide potent cues to the 3-D orientation (tilt and slant) of planar surfaces. Previous studies have described neurons in both dorsal and ventral stream areas that are selective for surface tilt based on one or more of these gradient cues. However, relatively little is known about whether single neurons provide consistent information about surface orientation from multiple gradient cues. Moreover, it is unclear how neural responses to combinations of surface orientation cues are related to responses to the individual cues. We measured responses of middle temporal (MT) neurons to random dot stimuli that simulated planar surfaces at a variety of tilts and slants. Four cue conditions were tested: disparity, velocity, and texture gradients alone, as well as all three gradient cues combined. Many neurons showed robust tuning for surface tilt based on disparity and velocity gradients, with relatively little selectivity for texture gradients. Some neurons showed consistent tilt preferences for disparity and velocity cues, whereas others showed large discrepancies. Responses to the combined stimulus were generally well described as a weighted linear sum of responses to the individual cues, even when disparity and velocity preferences were discrepant. These findings suggest that area MT contains a rudimentary representation of 3-D surface orientation based on multiple cues, with single neurons implementing a simple cue integration rule.     

Parietal neurons represent surface orientation from the gradient of binocular disparity

In order to elucidate the neural mechanisms involved in the perception of the three-dimensional (3D) orientation of a surface, we trained monkeys to discriminate the 3D orientation of a surface from binocular disparity cues using a Go/No-go type delayed-matching-to-sample (DMTS) task and examined the properties of the surface-orientation-selective (SOS) neurons. We recorded 57 SOS neurons from the caudal part of the lateral bank of the intraparietal sulcus (area CIP) of three hemispheres of two Japanese monkeys (Macaca fuscata). We tested 29 of 57 SOS neurons using the square plate of a solid figure stereogram (SFS) and random-dot stereogram (RDS) without perspective cues; almost all of the tested neurons (28/29) showed surface orientation selectivity for the SFS and/or the RDS without perspective cues. Eight of these 28 neurons (28.6%) showed selectivity for both the RDS and SFS, 7 (25.0%) were dominantly selective for the RDS, and 13 (46.4%) were dominantly selective for the SFS. These results suggest that neurons that show surface orientation tuning for the RDS without perspective cues compute surface orientation from the gradient of the binocular disparity given by the random-dot across the surface. On the other hand, neurons that show surface orientation tuning for the SFS without perspective cues may represent surface orientation primarily from the gradient of the binocular disparity along the contours. In conclusion, the SOS neurons in the area CIP are likely to operate higher order processing of disparity signals for surface perception by integrating the input signals from many disparity-sensitive neurons with different disparity tuning.     

Processing of shape defined by disparity in monkey inferior temporal cortex

Neurons in the monkey inferior temporal cortex (IT) have been shown to respond to shapes defined by luminance, texture, or motion. In the present study, we determined whether IT neurons respond to shapes defined solely by binocular disparity, and if so, whether signals of disparity and other visual cues to define shape converge on single IT neurons. We recorded extracellular activity from IT neurons while monkeys performed a fixation task. Among the neurons that responded to at least one of eight random-dot stereograms (RDSs) containing different disparity-defined shapes, 21% varied their responses to different RDSs. Responses of most of the neurons were positively correlated between two sets of RDSs, which consisted of different dot patterns but defined the same set of eight shapes, whereas responses to RDSs and their monocular images were not correlated. This indicates that the response modulation for the eight RDSs reflects selectivity for shapes (or their component contours) defined by disparity, although responses were also affected by dot patterns per se. Among the neurons that showed selectivity for shapes defined by luminance or disparity, 44% were activated by both cues. Responses of these neurons to luminance-defined shapes and those to disparity-defined shapes were often positively correlated to each other. Furthermore the stimulus rank, which was determined by the magnitude of responses to shapes, generally matched between these cues. The same held true between disparity and texture cues. The results suggest that the signals of disparity, luminance, and texture cues to define the shapes converge on a population of single IT neurons to produce the selectivity for shapes.     

Disparity tuning and binocularity of single neurons in cat visual cortex

Summary Activity from single neurons in the visual cortex of anaesthetized and paralyzed cats were recorded in response to monocular and binocular stimuli of different retinal disparities.Three different types of disparity sensitive neurons were found, characterized by (1) a response maximum near zero disparity, (2) a response minimum near zero disparity, and (3) an asymmetric disparity sensitivity.Usually, neurons of the first type receive about the same monocular input from the two eyes; the other disparity sensitive neurons respond better to monocular stimulation of one eye as compared to the other.The results closely resemble those obtained recently in the rhesus monkey.     

Visuomotor sensitivity to visual information about surface orientation

We measured human visuomotor sensitivity to visual information about three-dimensional surface orientation by analyzing movements made to place an object on a slanted surface. We applied linear discriminant analysis to the kinematics of subjects' movements to surfaces with differing slants (angle away form the fronto-parallel) to derive visuomotor d's for discriminating surfaces differing in slant by 5 degrees. Subjects' visuomotor sensitivity to information about surface orientation was very high, with discrimination "thresholds" ranging from 2 to 3 degrees. In a first experiment, we found that subjects performed only slightly better using binocular cues alone than monocular texture cues and that they showed only weak evidence for combining the cues when both were available, suggesting that monocular cues can be just as effective in guiding motor behavior in depth as binocular cues. In a second experiment, we measured subjects' perceptual discrimination and visuomotor thresholds in equivalent stimulus conditions to decompose visuomotor sensitivity into perceptual and motor components. Subjects' visuomotor thresholds were found to be slightly greater than their perceptual thresholds for a range of memory delays, from 1 to 3 s. The data were consistent with a model in which perceptual noise increases with increasing delay between stimulus presentation and movement initiation, but motor noise remains constant. This result suggests that visuomotor and perceptual systems rely on the same visual estimates of surface slant for memory delays ranging from 1 to 3 s.     

Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area

The macaque anterior intraparietal area (AIP) is crucial for visually guided grasping. AIP neurons respond during the visual presentation of real-world objects and encode the depth profile of disparity-defined curved surfaces. We investigated the neural representation of curved surfaces in AIP using a stimulus-reduction approach. The stimuli consisted of three-dimensional (3-D) shapes curved along the horizontal axis, the vertical axis, or both the horizontal and the vertical axes of the shape. The depth profile was defined solely by binocular disparity that varied along either the boundary or the surface of the shape or along both the boundary and the surface of the shape. The majority of AIP neurons were selective for curved boundaries along the horizontal or the vertical axis, and neural selectivity emerged at short latencies. Stimuli in which disparity varied only along the surface of the shape (with zero disparity on the boundaries) evoked selectivity in a smaller proportion of AIP neurons and at considerably longer latencies. AIP neurons were not selective for 3-D surfaces composed of anticorrelated disparities. Thus the neural selectivity for object depth profile in AIP is present when only the boundary is curved in depth, but not for disparity in anticorrelated stereograms.     

Spatial frequency integration for binocular correspondence in macaque area V4

Neurons in the primary visual cortex (V1) detect binocular disparity by computing the local disparity energy of stereo images. The representation of binocular disparity in V1 contradicts the global correspondence when the image is binocularly anticorrelated. To solve the stereo correspondence problem, this rudimentary representation of stereoscopic depth needs to be further processed in the extrastriate cortex. Integrating signals over multiple spatial frequency channels is one possible mechanism supported by theoretical and psychophysical studies. We examined selectivities of single V4 neurons for both binocular disparity and spatial frequency in two awake, fixating monkeys. Disparity tuning was examined with a binocularly correlated random-dot stereogram (RDS) as well as its anticorrelated counterpart, whereas spatial frequency tuning was examined with a sine wave grating or a narrowband noise. Neurons with broader spatial frequency tuning exhibited more attenuated disparity tuning for the anticorrelated RDS. Additional rectification at the output of the energy model does not likely account for this attenuation because the degree of attenuation does not differ among the various types of disparity-tuned neurons. The results suggest that disparity energy signals are integrated across spatial frequency channels for generating a representation of stereoscopic depth in V4.     

Judging surface slant for placing objects: a role for motion parallax

People have a variety of sources of information (cues) about surface slant at their disposal. We used a simple placing task to evaluate the relative importance of three such cues (motion parallax, binocular disparity and texture) within the space in which people normally manipulate objects. To do so, we projected a stimulus onto a rotatable screen. This allowed us to manipulate texture cues independently of binocular disparity and motion parallax. We asked people to stand in front of the screen and place a cylinder on the screen. We analysed the cylinder’s orientation just before contact. Participants mainly relied on binocular cues (weight between 50 and 90%), in accordance with binocular cues being known to be reliable when the stimulus surface is nearby and almost frontal. Texture cues contributed between 2 and 18% to the estimated slant. Motion parallax was given a weight between 1 and 9%, despite the fact that it only provided information when the head began to move, which was just before the arm did. Thus motion parallax is used to judge surface slant, even when one is under the impression of standing still.     

Sensitivity of human visual cortical area V6 to stereoscopic depth gradients associated with self-motion

The principal visual cue to self-motion (egomotion) is optic flow, which is specified in terms of local 2D velocities in the retinal image without reference to depth cues. However, in general, points near the center of expansion of natural flow fields are distant, whereas those in the periphery are closer, creating gradients of horizontal binocular disparity. To assess whether the brain combines disparity gradients with optic flow when encoding egomotion, stereoscopic gradients were applied to expanding dot patterns presented to observers during functional MRI scanning. The gradients were radially symmetrical, disparity changing as a function of eccentricity. The depth cues were either consistent with egomotion (peripheral dots perceived as near and central dots perceived as far) or inconsistent (the reverse gradient, central dots near, peripheral dots far). The BOLD activity generated by these stimuli was compared in a range of predefined visual regions in 13 participants with good stereoacuity. Visual area V6, in the parieto-occipital sulcus, showed a unique pattern of results, responding well to all optic flow patterns but much more strongly when they were paired with consistent rather than inconsistent or zero-disparity gradients. Of the other areas examined, a region of the precuneus and parietoinsular vestibular cortex also differentiate between consistent and inconsistent gradients, but with weak or suppressive responses. V3A, V7, MT, and ventral intraparietal area responded more strongly in the presence of a depth gradient but were indifferent to its depth-flow congruence. The results suggest that depth and flow cues are integrated in V6 to improve estimation of egomotion.     

The integration of motion and disparity cues to depth in dorsal visual cortex

Humans exploit a range of visual depth cues to estimate three-dimensional structure. For example, the slant of a nearby tabletop can be judged by combining information from binocular disparity, texture and perspective. Behavioral tests show humans combine cues near-optimally, a feat that could depend on discriminating the outputs from cue-specific mechanisms or on fusing signals into a common representation. Although fusion is computationally attractive, it poses a substantial challenge, requiring the integration of quantitatively different signals. We used functional magnetic resonance imaging (fMRI) to provide evidence that dorsal visual area V3B/KO meets this challenge. Specifically, we found that fMRI responses are more discriminable when two cues (binocular disparity and relative motion) concurrently signal depth, and that information provided by one cue is diagnostic of depth indicated by the other. This suggests a cortical node important when perceiving depth, and highlights computations based on fusion in the dorsal stream.     

Disparity-selective neurons in area V4 of macaque monkeys

Area V4 is an intermediate stage of the ventral visual pathway providing major input to the final stages in the inferior temporal cortex (IT). This pathway is involved in the processing of shape, color, and texture. IT neurons are also sensitive to horizontal binocular disparity, suggesting that binocular disparity is processed along the ventral visual pathway. In the present study, we examined the processing of binocular disparity information by V4 neurons. We recorded responses of V4 neurons to binocularly disparate stimuli. A population of V4 neurons modified their responses according to changes of stimulus disparity; neither monocular responses nor eye movements could account for this modulation. Disparity-tuning curves were similar for different locations within a neuron's receptive field. Neighboring neurons recorded using a single electrode displayed similar disparity-tuning properties. These findings indicate that a population of V4 neurons is selective for binocular disparity, invariant for the position of the stimulus within the receptive field. The finding that V4 neurons with similar disparity selectivity are clustered suggests the existence of functional modules for disparity processing in V4.     

Neurons with radial receptive fields in monkey area V4A: evidence of a subdivision of prelunate gyrus based on neuronal response properties

In recordings from two awake, behaving macaque monkeys we found that neurons in the crown of the prelunate gyrus differed in their responsiveness to simple visual stimuli. Neurons in the posterior part of the gyrus (area V4) responded strongly to stationary or moving bars, while neurons in the anterior part (area V4A) responded only weakly to such stimuli. Most receptive fields in area V4A were elongated with long axes oriented radially towards the fovea. These neurons were sensitive to radial movements, especially to sudden shifts of real 3D objects. The border between areas V4 and V4A coincided with the representation of the horizontal meridian. Area V4A extended into the posterior bank of the superior temporal sulcus, where its border corresponded to the representation of the vertical meridian. The sequence of the representations of the horizontal and vertical meridians over the prelunate gyrus suggests the existence of another area between V4A and V4t.     

Architecture of binocular disparity processing in monkey inferior temporal cortex

Neurons in the inferior temporal (IT) cortex respond not only to the shape, color or texture of objects, but to the horizontal positional disparity of visual features in the right and left retinal images. IT neurons with similar shape selectivity cluster in columns. In this study, we examined how IT neurons are spatially arranged in the IT according to their selectivity for binocular disparity. With a single electrode, we simultaneously recorded extracellular action potentials from a single neuron and those from background multiple neurons at the same sites or recorded multineuronal responses at successive sites along electrode penetrations, while monkeys performed a fixation task. For neurons at each recording site, effective shapes were first determined from a set of 20 shapes presented at the zero-disparity plane. The most effective shape was then presented with varying amounts of disparity. Single neuron responses and background multiunit responses recorded at the same sites showed a similar ability of disparity discrimination and tended to share the preferred disparity, suggesting that neurons with similar disparity selectivity are clustered in the IT. We estimated from sequential recordings along electrode penetrations that the size of the neuronal clusters with similar disparity selectivity was smaller than the size of clusters with similar shape selectivity.     

Cortical effects of brief daily periods of unrestricted vision during early monocular form deprivation

Experiencing daily brief periods of unrestricted vision during early monocular form deprivation prevents or reduces the degree of resulting amblyopia. To gain insight into the neural basis for these "protective" effects, we analyzed the monocular and binocular response properties of individual neurons in the primary visual cortex (V1) of macaque monkeys that received intermittent unrestricted vision. Microelectrode-recording experiments revealed significant decreases in the proportion of units that were dominated by the treated eyes, and the magnitude of this ocular dominance imbalance was correlated with the degree of amblyopia. The sensitivity of V1 neurons to interocular spatial phase disparity was significantly reduced in all treated monkeys compared with normal adults. With unrestricted vision, however, there was a small but significant increase in overall disparity sensitivity. Binocular suppression was prevalent in monkeys with constant form deprivation but significantly reduced by the daily periods of unrestricted vision. If neurons exhibited consistent responses to stimulation of the treated eye, monocular response properties obtained by stimulation of the two eyes were similar. These results suggest that the observed protective effects of brief periods of unrestricted vision are closely associated with the ability of V1 neurons to maintain their functional connections from the deprived eye and that interocular suppression in V1 may play an important role in regulating synaptic plasticity of these monkeys.     

Disparity selectivity of neurons in monkey inferior temporal cortex

The inferior temporal cortex (IT) of the monkey, a final stage in the ventral visual pathway, has been known to process information on two-dimensional (2-D) shape, color, and texture. On the other hand, the dorsal visual pathway leading to the posterior parietal cortex has been known to process information on location in space. Likewise, neurons selective for binocular disparity, which convey information on depth, have been found mainly in areas along the dorsal visual pathway. Here, we report that many neurons in the IT are also selective for binocular disparity. We recorded extracellular activity from IT neurons and found that more than half of the neurons changed their response depending on the disparity added. The change was not attributed to monocular responses or eye movements. Most neurons selective for disparity were "near" or "far" cells; they preferred either crossed or uncrossed disparity, and only a small population was tuned to zero disparity. Disparity-selective neurons were also selective for shape. Most preferred the same type of disparity irrespective of the shape presented. Disparity preference was also invariant for the fronto-parallel translation of the stimuli in most of the neurons. Finally, nearby neurons exhibited similar disparity selectivity, suggesting the existence of a functional module for processing of binocular disparity in the IT. From the above and our recent findings, we suggest that the IT integrates shape and binocular disparity information, and plays an important role in the reconstruction of three-dimensional (3-D) surfaces.     

Response of MSTd neurons to simulated 3D orientation of rotating planes.

We studied whether the dorsal division of the medial superior temporal area (MSTd) in the macaque has activity related to structure-from-motion (SFM) processing. As the simplest form of three-dimensional (3D) structure, we chose a planar stimulus and examined the relation between the neural responses and the simulated 3D orientation of the plane defined by motion cues. We recorded from 114 MSTd neurons while monkeys were performing a visual fixation task. These neurons responded to a basic set of optic flow patterns such as translation, expansion/contraction, and rotation. Responses of these neurons to rotating plane stimuli were examined to see whether the MSTd neurons exhibited selectivity to the tilt and slant that characterize the 3D orientation of the plane. We found that most MSTd neurons tested (97 of 114) responded to the plane stimuli, and many neurons (65 of 97) exhibited selectivity to tilt and/or slant. Of 97 neurons, 18% (17/97) were selective only to tilt, 24% (23/97) only to slant, and 26% (25/97) to both. Control experiments rejected the possibility that the selectivity could be explained solely by the sensitivity to local stimulus components such as local translation, local speed, and local speed gradients. These results suggest that MSTd neurons are sensitive to stimulus features specific to the perceived 3D orientation of the rotating plane stimuli and suggest that area MSTd is involved in SFM processing.     

Postnatal development of disparity sensitivity in visual area 2 (v2) of macaque monkeys

Macaque monkeys do not reliably discriminate binocular depth cues until about 8 wk of age. The neural factors that limit the development of fine depth perception in primates are not known. In adults, binocular depth perception critically depends on detection of relative binocular disparities and the earliest site in the primate visual brain where a substantial proportion of neurons are capable of discriminating relative disparity is visual area 2 (V2). We examined the disparity sensitivity of V2 neurons during the first 8 wk of life in infant monkeys and compared the responses of V2 neurons to those of V1 neurons. We found that the magnitude of response modulation in V2 and V1 neurons as a function of interocular spatial phase disparity was adult-like as early as 2 wk of age. However, the optimal spatial frequency and binocular response rate of these disparity sensitive neurons were more than an octave lower in 2- and 4-wk-old infants than in adults. Consequently, despite the lower variability of neuronal firing in V2 and V1 neurons of infant monkeys, the ability of these neurons to discriminate fine disparity differences was significantly reduced compared with adults. This reduction in disparity sensitivity of V2 and V1 neurons is likely to limit binocular depth perception during the first several weeks of a monkey's life.