Driving strategy alters neuronal responses to self-movement: cortical mechanisms of distracted driving

We presented naturalistic combinations of virtual self-movement stimuli while recording neuronal activity in monkey cerebral cortex. Monkeys used a joystick to drive to a straight ahead heading direction guided by either object motion or optic flow. The selected cue dominates neuronal responses, often mimicking responses evoked when that stimulus is presented alone. In some neurons, driving strategy creates selective response additivities. In others, it creates vulnerabilities to the disruptive effects of independently moving objects. Such cue interactions may be related to the disruptive effects of independently moving objects in Alzheimer's disease patients with navigational deficits.     

Cortical neuronal responses to optic flow are shaped by visual strategies for steering

We hypothesized that neuronal responses to virtual self-movement would be enhanced during steering tasks. We recorded the activity of medial superior temporal (MSTd) neurons in monkeys trained to steer a straight-ahead course, using optic flow. We found smaller optic flow responses during active steering than during the passive viewing of the same stimuli. Behavioral analysis showed that the monkeys had learned to steer using local motion cues. Retraining the monkeys to use the global pattern of optic flow reversed the effects of the active-steering task: active steering then evoked larger responses than passive viewing. We then compared the responses of neurons during active steering by local motion and by global patterns: Local motion trials promoted the use of local dot movement near the center of the stimulus by occluding the peripheral visual field midway through the trial. Global pattern trials promoted the use of radial pattern movement by occluding the central visual field midway through the trial. In this study, identical full-field optic-flow stimuli evoked larger responses in global-pattern trials than in local motion trials. We conclude that the selection of specific visual cues reflects strategies for active steering and alters MSTd neuronal responses to optic flow.     

A neural model of motion processing and visual navigation by cortical area MST

Cells in the dorsal medial superior temporal cortex (MSTd) process optic flow generated by self-motion during visually guided navigation. A neural model shows how interactions between well-known neural mechanisms (log polar cortical magnification, Gaussian motion-sensitive receptive fields, spatial pooling of motion-sensitive signals and subtractive extraretinal eye movement signals) lead to emergent properties that quantitatively simulate neurophysiological data about MSTd cell properties and psychophysical data about human navigation. Model cells match MSTd neuron responses to optic flow stimuli placed in different parts of the visual field, including position invariance, tuning curves, preferred spiral directions, direction reversals, average response curves and preferred locations for stimulus motion centers. The model shows how the preferred motion direction of the most active MSTd cells can explain human judgments of self-motion direction (heading), without using complex heading templates. The model explains when extraretinal eye movement signals are needed for accurate heading perception, and when retinal input is sufficient, and how heading judgments depend on scene layouts and rotation rates.     

Visual binding through reentrant connectivity and dynamic synchronization in a brain-based device

Effective visual object recognition requires mechanisms to bind object features (e.g. color, shape and motion) while distinguishing distinct objects. Synchronously active neuronal circuits among reentrantly connected cortical areas may provide a basis for visual binding. To assess the potential of this mechanism, we have constructed a mobile brain-based device, Darwin VIII, which is guided by simulated analogues of cortical and sub-cortical areas required for visual processing, decision-making, reward and motor responses. These simulated areas are reentrantly connected and each area contains neuronal units representing both the mean activity level and the relative timing of the activity of groups of neurons. Darwin VIII learns to discriminate among multiple objects with shared visual features and associates 'target' objects with innately preferred auditory cues. We observed the co-activation of globally distributed neuronal circuits that corresponded to distinct objects in Darwin VIII's visual field. These circuits, which are constrained by a reentrant neuroanatomy and modulated by behavior and synaptic plasticity, are necessary for successful discrimination. By situating Darwin VIII in a rich real-world environment involving continual changes in the size and location of visual stimuli due to self-generated movement, and by recording its behavioral and neuronal responses in detail, we were able to show that reentrant connectivity and dynamic synchronization provide an effective mechanism for binding the features of visual objects.     

Functional sub-regions for optic flow processing in the posteromedial lateral suprasylvian cortex of the cat.

During locomotion, an observer sees a large and complex pattern of visual motion called optic flow. This phenomenon is characterized by elements in the environment accelerating and expanding as they move peripherally. In cats, previous studies have indicated that the posteromedial part of the lateral suprasylvian (PMLS) cortex may be involved in the processing of optic flow fields. We further addressed this issue by studying the importance of specific parameters of the optic flow patterns and investigating whether cell responses to these stimuli depend on receptive field (RF) location in the visual field. Results can be summarized as follows: approximately two-thirds of PMLS cells responded to optic flow fields and a subset of these (84/153) showed a clear direction selectivity for motion along the frontal axis. Of these units, the majority responded preferentially to expansion rather than contraction of the pattern. Cells' responses depend on RF location in the visual field. For centrally located RFs, tested both when the origin of motion was within the RF or at the area centralis, responses were generally comparable whether or not size or speed gradients were removed from the optic flow pattern. A different tendency was observed for peripherally located RFs. In general, these cells exhibited a preferred direction almost exclusively when the origin of motion was placed at the area centralis, and neuronal discharges and direction selectivity for many of them were reduced when the optic flow cues were removed from the pattern. The results of this study suggest that there may be functional differences in response properties between PMLS cells located in the central and peripheral parts of the visual field that may reflect a specialization of the PMLS cortex in optic flow processing.     

Vestibular inputs to human motion-sensitive visual cortex

Two crucial sources of information available to an organism when moving through an environment are visual and vestibular stimuli. Macaque cortical area MSTd processes visual motion, including cues to self-motion arising from optic flow and also receives information about self-motion from the vestibular system. In humans, whether human MST (hMST) receives vestibular afferents is unknown. We have combined 2 techniques, galvanic vestibular stimulation and functional MRI (fMRI), to show that hMST is strongly activated by vestibular stimulation in darkness, whereas adjacent area MT is unaffected. The activity cannot be explained in terms of somatosensory stimulation at the electrode site. Vestibular input appears to be confined to the anterior portion of hMST, suggesting that hMST as conventionally defined may contain 2 subregions. Vestibular activity was also seen in another area previously implicated in processing visual cues to self-motion, namely the cingulate sulcus visual area (CSv), but not in visual area V6. The results suggest that cross-modal convergence of cues to self-motion occurs in both hMST and CSv.     

Representation of the perceived 3-D object shape in the human lateral occipital complex

We used human functional magnetic resonance imaging (fMRI) to test whether the human lateral occipital complex (LOC), an area known to be involved in the analysis of visual shape, represents the perceived 3-D shape of objects or simply their 2-D contours. We employed an fMRI adaptation paradigm, in which repeated presentation of a stimulus results in decreased responses compared to responses to different stimuli. We found adaptation in the LOC for images of objects with the same perceived 3-D shape structure but different 2-D contours that resulted from small rotations of the objects in the frontal plane or in depth. However, no adaptation was observed in the LOC for images of objects that had the same 2-D contours but differed in their perceived 3-D shape; namely, 2-D silhouettes versus 3-D shaded images of objects, or convex versus concave objects. Differences in the fMRI adaptation responses across subregions in the LOC suggest that different neural populations in the LOC may mediate different mechanisms for the processing of object features.     

Speed selectivity for optic flow in area 7a of the behaving macaque

Area 7a, in the inferior parietal lobe, has been implicated in optic flow processing to obtain spatial information about the environment. Optic flow, angle-of-gaze and center-of-motion dependencies are already documented, but the selectivity of area 7a to speed is unknown. Such information is crucial as area 7a provides the final step in visual motion analysis that begins at the lateral geniculate nucleus and passes through MT, MST and LIP/VIP. Macaque area 7a neurons were tested with optic flows with speeds of 0.5-128 degrees /s. Of 161 neurons tested in four hemispheres of two adult male macaques, 53% (86/161) were speed selective at either the time of stimulus onset, at the end of the trial, or at both times. Speed selec- tivities resembling the basic filter types (band-pass, band-reject, high-pass, low-pass, broadband) were found. Area 7a neurons exhibited two novel properties not previously reported elsewhere. Speed selectivity was found to be dynamic in that many cells gained, lost or changed speed tuning over the course of a trial. In addition, speed dependence and optic flow selectivity interacted. For example, a cell could preferentially respond to one type of naviga- tional optic flow at a slow speed and a different navigational optic flow at a fast speed. The presence of speed selectivity combined with other properties of area 7a neurons indicates that these neurons may have a role in the concurrent representation of heading as well as multiple object speeds and directions.     

Natural vision reveals regional specialization to local motion and to contrast-invariant, global flow in the human brain

Visual changes in feature movies, like in real-live, can be partitioned into global flow due to self/camera motion, local/differential flow due to object motion, and residuals, for example, due to illumination changes. We correlated these measures with brain responses of human volunteers viewing movies in an fMRI scanner. Early visual areas responded only to residual changes, thus lacking responses to equally large motion-induced changes, consistent with predictive coding. Motion activated V5+ (MT+), V3A, medial posterior parietal cortex (mPPC) and, weakly, lateral occipital cortex (LOC). V5+ responded to local/differential motion and depended on visual contrast, whereas mPPC responded to global flow spanning the whole visual field and was contrast independent. mPPC thus codes for flow compatible with unbiased heading estimation in natural scenes and for the comparison of visual flow with nonretinal, multimodal motion cues in it or downstream. mPPC was functionally connected to anterior portions of V5+, whereas laterally neighboring putative homologue of lateral intraparietal area (LIP) connected with frontal eye fields. Our results demonstrate a progression of selectivity from local and contrast-dependent motion processing in V5+ toward global and contrast-independent motion processing in mPPC. The function, connectivity, and anatomical neighborhood of mPPC imply several parallels to monkey ventral intraparietal area (VIP).     

A dynamic object-processing network: metric shape discrimination of dynamic objects by activation of occipitotemporal, parietal, and frontal cortices

Shape perception is important for object recognition. However, behavioral studies have shown that rigid motion also contributes directly to the recognition process, in addition to providing visual cues to shape. Using psychophysics and functional brain imaging, we investigated the neural mechanisms involved in shape and motion processing for dynamic object recognition. Observers discriminated between pairs of rotating novel objects in which the 3-dimensional shape difference between the pair was systematically varied in metric steps. In addition, the objects rotated in either the same or the different direction to determine the effect of task-irrelevant motion on behavior and neural activity. We found that observers' shape discrimination performance increased systematically with shape differences, as did the hemodynamic responses of occipitotemporal, parietal, and frontal regions. Furthermore, responses in occipital regions were only correlated with observers' perceived shape differences. We also found different effects of object motion on shape discrimination across observers, which were reflected in responses of the superior temporal sulcus. These results suggest a network of regions that are involved in the discrimination of metric shape differences for dynamic object recognition.     

Behavioral influences on cortical neuronal responses to optic flow

Optic flow selectively activates neurons in medial superior temporal (MST) cortex. We find that many MST neurons yield larger and more selective responses when the optic flow guides a subsequent eye movement. Smaller, less selective responses are seen when optic flow is preceded by a flashed precue that guides eye movements. Selectivity can decrease by a third (32%) after a flashed precue is presented at a peripheral location as a small spot specifying the target location of the eye movement. Smaller decreases in selectivity (18%) occur when the precue is presented centrally with its shape specifying the target location. Shape precues presented centrally, but not linked to specific target locations, do not appear to alter optic flow selectivity. The effects of spatial precueing can be reversed so that the precue leads to larger and more selective optic flow responses: A flashed precue presented as a distracter before behaviorally relevant optic flow is associated with larger optic flow responses and a 45% increase in selectivity. Together, these findings show that spatial precues can decrease or increase the size and selectivity of optic flow responses depending on the associated behavioral contingencies.     

Processing shape,motion and three-dimensional shape-from-motion in the human cortex

Shape and motion are complementary visual features and each appears to be processed in unique cortical areas. However, object motion is a powerful cue for the perception of three-dimensional (3-D) shape, implying that the two types of information--motion and form--are well integrated. We conducted a series of fMRI experiments aimed at identifying the brain regions involved in inferring 3-D shape from motion cues. For each subject, we identified regions in occipital-temporal cortex that were activated when perceiving: (i) motion in unstructured random-dot patterns, (ii) 2-D and 3-D line drawing shapes, and (iii) 3-D shapes defined by motion cues (shape-from-motion, SFM). We found closely adjacent areas in the lateral occipital region activated by random motion and line-drawing shapes. In addition, we found that the SFM stimuli produced a greater MRI signal in only one of the areas identified with the random motion and line-drawing stimuli: the superior lateral occipital (SLO) region. High-resolution analysis showed that SFM objects and line drawings were processed in separate but adjacent sub-regions in SLO, suggesting that SLO codes object shape but retains topographic segregation based on shape cues. Expanding the analysis to the entire cortex identified a parietal area that had overlapping activation for both SFM and line drawings and increased MRI signal for 3-D versus 2-D shapes, suggesting this area is important for processing shape information.     

A contralateral preference in the lateral occipital area: sensory and attentional mechanisms

Here we examined the level of the lateral occipital (LO) area within the processing stream of the ventral visual cortex. An important determinant of an area's level of processing is whether it codes visual elements on both sides of the visual field, as do higher visual areas, or prefers those in the contralateral visual field, as do early visual areas. The former would suggest that LO, on one side, combines bilateral visual elements into a whole, while the latter suggests that it codes only the parts of forms. We showed that LO has a relative preference for visual objects in the contralateral visual field. LO responses were influenced by attention. However, relative changes in LO activity caused by changes in object location were preserved even when attention was shifted away from the objects to moving random dot patterns on the opposite side. Our data offer a new view on LO as an intermediate, but not a high-level, visual area in which neurons are driven by visual input and spatial attention in a multiplicative fashion.     

View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex

A view-invariant representation of objects in the brain would have many computational advantages. Here we describe a population of single neurons in the temporal visual cortex (IT) that have view-invariant representations of familiar objects. Ten real plastic objects were placed in the monkeys' home cages for a period of time before neurophysiological experiments in which neuronal responses were measured to four views of each object. The macaques performed a visual fixation task, and had never been trained in object discrimination. The majority of the visual neurons recorded were responsive to some views of some objects and/or to the control stimuli, as would be expected from previous studies. However, a small subset of these neurons were responsive to all views of one or more of the objects, providing evidence that these neurons were coding for objects, rather than simply for individual views or visual features within the image. This result was confirmed by information theoretic analyses, which showed that the neurons provided information about which object was being seen, independently of the view. The coding scheme was shown to be sparse distributed, with relatively independent information being provided by the different neurons. Hypotheses about how these view-invariant cells are formed are described.      

Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration?

Interactions between multisensory integration and attention were studied using a combined audiovisual streaming design and a rapid serial visual presentation paradigm. Event-related potentials (ERPs) following audiovisual objects (AV) were compared with the sum of the ERPs following auditory (A) and visual objects (V). Integration processes were expressed as the difference between these AV and (A + V) responses and were studied while attention was directed to one or both modalities or directed elsewhere. Results show that multisensory integration effects depend on the multisensory objects being fully attended--that is, when both the visual and auditory senses were attended. In this condition, a superadditive audiovisual integration effect was observed on the P50 component. When unattended, this effect was reversed; the P50 components of multisensory ERPs were smaller than the unisensory sum. Additionally, we found an enhanced late frontal negativity when subjects attended the visual component of a multisensory object. This effect, bearing a strong resemblance to the auditory processing negativity, appeared to reflect late attention-related processing that had spread to encompass the auditory component of the multisensory object. In conclusion, our results shed new light on how the brain processes multisensory auditory and visual information, including how attention modulates multisensory integration processes.     

Neuronal responses to optic flow in the monkey parietal area PEc

Area PEc, a high order association area, is located in the dorsocaudal portion of the superior parietal cortex. PEc neurons encode visual motion signals, especially the direction of stimulus motion. The present study tested if PEc neurons also process visual correlates of self-motion. The extracellular activity of single neurons in response to optic flow stimuli was recorded in two monkeys (Macaca fascicularis) trained in a fixation task. The stimuli were produced by random dots simulating planar motion, radial expansion and radial contraction. A substantial number of PEc neurons were specifically activated by radial optic flow and were selective for the position of the focus of expansion with respect to the fovea. Eccentric positions of the focus of expansion were preferred. Almost all neurons showed opponent excitatory-inhibitory activity to expanding-contracting visual fields. Planar motion elicited very weak responses. Optic flow responsiveness is not entirely explained by classical bar sensitivity in PEc neurons, suggesting that optic flow and classical bar responses could serve different mechanisms in the integration of visuo-motor signals to prepare body movements.     

Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device

The ability of organisms to categorize diverse and often novel stimuli depends on ongoing interactions with their environment. In a modality such as vision, categorization requires the generation of both selective and invariant responses of cortical neurons to complex visual stimuli. How does behavior contribute to shaping the responses of these neurons? Analysis of this question is made difficult by the complex multilevel interactions between many neural and behavioral variables. To mitigate this difficulty, we studied the development and ongoing plasticity of pattern-selective neuronal responses by means of synthetic neural modeling. For this purpose, we constructed Darwin V, which consists of a simulated neuronal model embedded in a real-world device that is capable of motion and autonomous behavior. The neuronal model consists of four major components: a visual system (containing cortical and subcortical networks); a taste system based on conductance; sets of motor neurons capable of triggering behavior; and a diffuse ascending (value) system. The modeled visual cortex consists of two areas: a topographic map responsive to elementary features connected to a higher-order map composed of initially non-selective neuronal units. During behavior over time in its environment, Darwin V encounters numerous objects consisting of black metal cubes displaying different patterns of white blobs and stripes. Initially, the lack of specific higher-order visual responses does not allow visual pattern discrimination, and appetitive and aversive behaviors are triggered by the 'taste' (surface conductivity of objects) alone. In the course of sensory experience, however, changes occur in visual and sensorimotor connection strengths, with two major consequences. First, units within the higher visual area acquire responses that are both pattern selective and translation invariant. Second, as a result of the operation of the value system, these responses become linked to appropriate behaviors. Analysis of Darwin V after such changes indicates that the continuity of self-generated movements is essential for the development of pattern-selective and translation-invariant responses. The concomitant development of a preference for foveal over parafoveal objects was found to be due to increased behavioral interactions with object cubes gripped by the centrally mounted effector (snout) of Darwin V. Finally, even after development of higher- order visual responses, visual responses to more frequently encountered objects continued to be enhanced, while other responses were diminished. Overall, the detailed study of Darwin V over multiple levels of organization provides a heuristically revealing example of the crucial role played by behavioral and environmental interactions in the development of complex responses by specialized neurons.      

Single cell integration of animate form, motion and location in the superior temporal cortex of the macaque monkey

This study investigated the cellular mechanisms in the anterior part of the superior temporal sulcus (STSa) that underlie the integration of different features of the same visually perceived animate object. Three visual features were systematically manipulated: form, motion and location. In 58% of a population of cells selectively responsive to the sight of a walking agent, the location of the agent significantly influenced the cell's response. The influence of position was often evident in intricate two- and three-way interactions with the factors form and/or motion. For only one of the 31 cells tested, the response could be explained by just a single factor. For all other cells at least two factors, and for half of the cells (52%) all three factors, played a significant role in controlling responses. Our findings support a reformulation of the Ungerleider and Mishkin model, which envisages a subdivision of the visual processing into a ventral 'what' and a dorsal 'where' stream. We demonstrated that at least part of the temporal cortex ('what' stream) makes ample use of visual spatial information. Our findings open up the prospect of a much more elaborate integration of visual properties of animate objects at the single cell level. Such integration may support the comprehension of animals and their actions.     

Area MST and heading perception in macaque monkeys

The macaque medial superior temporal area (MST) is proposed to be specialized for analyzing complex 'optic flow' information. Such space-varying motion patterns provide a rich source of information about self motion, scene structure and object shape. We report the performance of rhesus macaques on a two-alternative 'heading' task, in which they reported whether horizontally varying, simulated trajectories were to left or right of center. Monkeys were sensitive to small heading angles; thresholds averaged 1.5-3 degrees. Heading estimates were stable in the face of changing stimulus location and smooth pursuit eye movements. In addition, we tested the role of area MST in heading judgements by electrically activating columns of neurons in this area while the monkeys performed the heading task. Activation of MST frequently affected performance, usually causing choice biases. These induced biases were often large and usually concordant with the preference of the neurons being activated. In addition, the induced biases were often larger in the presence of smooth pursuit eye movements. These results favor the hypothesis that MST is involved in recovering self-motion direction from optic flow cues and in the process by which heading perception is compensated for ongoing eye movements.     

Localization of the human cortical visual area MT based on computer aided histological analysis

We describe human area MT histologically based on the observer independent analysis of cortical myeloarchiteture, multiple complementary staining techniques and 3-D reconstruction. The topography of an architectonic field that presented constant structural characteristics across specimens was studied in relation to the sulcal geography of the occipito-temporal region. Objective and semi-automated analysis of local microstructure revealed a distinct cortical architecture and matched topographically the localization of MT derived from functional imaging. MT was localized by the histotopographic method in relation to definite macroscopic landmarks. This study demonstrates a new set of distinguishing architectonic features of human MT that permit localization on structural grounds and suggests that the characteristic laminar structure of this area may be related to its unique pattern of connections and to its role in visual perception.