Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites

The role of attention in multisensory integration (MI) is presently uncertain, with some studies supporting an automatic, pre-attentive process and others suggesting possible modulation through selective attention. The goal of this functional magnetic resonance imaging study was to investigate the role of spatial attention on the processing of congruent audiovisual speech stimuli (here indexing MI). Subjects were presented with two simultaneous visual streams (speaking lips in the left and right visual hemifields) plus a single central audio stream (spoken words). In the selective attention conditions, the auditory stream was congruent with one of the two visual streams. Subjects attended to either the congruent or the incongruent visual stream, allowing the comparison of brain activity for attended vs. unattended MI while the amount of multisensory information in the environment and the overall attentional requirements were held constant. Meridian mapping and a lateralized 'speaking-lips' localizer were used to identify early visual areas and to localize regions responding to contralateral visual stimulations. Results showed that attention to the congruent audiovisual stimulus resulted in increased activation in the superior temporal sulcus, striate and extrastriate retinotopic visual cortex, and superior colliculus. These findings demonstrate that audiovisual integration and spatial attention jointly interact to influence activity in an extensive network of brain areas, including associative regions, early sensory-specific visual cortex and subcortical structures that together contribute to the perception of a fused audiovisual percept.     

Effects of attention and arousal on early responses in striate cortex

Humans employ attention to facilitate perception of relevant stimuli. Visual attention can bias the selection of a location in the visual field, a whole visual object or any visual feature of an object. Attention draws on both current behavioral goals and/or the saliency of physical attributes of a stimulus, and it influences activity of different brain regions at different latencies. Attentional effect in the striate and extrastriate cortices has been the subject of intense research interest in many recent studies. The consensus emerging from them places the first attentional effects in extrastriate areas, which in turn modulate activity of V1 at later latencies. In this view attention influences activity in striate cortex some 150 ms after stimulus onset. Here we use magnetoencephalography to compare brain responses to foveally presented identical stimuli under the conditions of passive viewing, when the stimuli are irrelevant to the subject and under an active GO/NOGO task, when the stimuli are cues instructing the subject to make or inhibit movement of his/her left or right index finger. The earliest striate activity was identified 40-45 ms after stimulus onset, and it was identical in passive and active conditions. Later striate response starting at about 70 ms and reaching a peak at about 100 ms showed a strong attentional modulation. Even before the striate cortex, activity of the right inferior parietal lobule was modulated by attention, suggesting this region as a candidate for mediating attentional signals to the striate cortex.     

Selective attention to face identity and color studied with f MRI

Cortical areas associated with selective attention to the color and identity of faces were located using functional magnetic resonance imaging (fMRI). Six subjects performed tasks which required selective attention to face identity or color similarity using the same color-washed face stimuli. Performance of the color attention task but not the face attention task was associated with a region of activity in the collateral sulcus and nearby regions of the lingual and fusiform gyri. Performance of both tasks was associated with a region of activity in ventral occipitotemporal cortex that was lateral to the color responsive area and had a greater spatial extent. These fMRI results converge with results obtained from PET and ERP studies to demonstrate similar anatomical locations of functional areas for face and color processing across studies. Hum. Brain Mapping5:293–297, 1997. Published 1997 Wiley-Liss, Inc. This article was prepared by a group consisting of both United States government employees and non-United States government employees, and as such is subject to 17 U.S.C. Sec. 105.     

Motion direction tuning in human visual cortex

A number of electrophysiological studies have been conducted in recent years in order to clarify the dynamics of visual motion processing in the human brain. Using a variety of event-related potential (ERP) measures, several parameters such as onset, offset, contrast and velocity have been investigated, while a critical aspect of visual motion, that of direction, has received less attention. Here we used multichannel electroencephalography and distributed source localization to study brain activity for different directions of visual motion using random dot stimuli. Our data reveal differential extrastriate activation at 164–226 ms after motion onset that coded for motion direction with different ERP maps and underlying electrical generators for each tested direction. This activation was paralleled initially (164–186 ms) by a distinct extrastriate activation encoding whether the motion stimulus consisted of directed motion stimuli (as above) or contained undirected incoherent motion (control stimulus). Application of a linear inverse solution localized the brain activity for each tested motion direction to distinct brain regions within the same larger network of extrastriate brain regions. These regions included bilateral temporo-occipital and bilateral parieto-occipital cortex. The present data in healthy subjects are compatible with extrastriate activity that is tuned to different directions of visual motion. This extends previous clinical data and suggests the presence of distributed macroscopic motion direction tuning in primate extrastriate cortex that may complement the classical microscopic motion tuning at the columnar level.     

Corticostriatal connections of extrastriate visual areas in rhesus monkeys

The striatal connections of extrastriate visual areas were examined by the autoradiographic technique in rhesus monkeys. The medial as well as the dorsolateral extrastriate regions project preferentially to dorsal and lateral portions of the head and of the body of the caudate nucleus, as well as to the caudodorsal sector of the putamen. The rostral portion of the annectant gyrus has connections to the caudal sector of the body and to the genu, whereas projections from the caudal portion of the lower bank of the superior temporal sulcus are directed to dorsal and central sectors of the head and the body, to the genu and the tail, as well as to the caudal putamen. The ventrolateral extrastriate region is related mainly to the ventral sector of the body, to the genu and the tail, and to the caudal putamen. In contrast, the striatal projections of the ventromedial extrastriate cortex resemble those of the medial and dorsolateral regions. The caudal inferotemporal cortex is related strongly to the tail of the caudate nucleus and to the ventral putamen. The differential corticostriatal connectivity of the various extrastriate regions may contribute to the specific functional roles of these cortices. Thus, the connections from the dorsomedial, dorsolateral, and ventromedial areas to dorsal portions of the caudate nucleus and of the putamen may serve a visuospatial function. In contrast, the connections from the ventrolateral extrastriate and inferotemporal regions to the tail of the caudate nucleus and to the ventral putamen may have a role in visual object-related processes. © 1995 Wiley-Liss, Inc.     

Selective tuning of the left and right auditory cortices during spatially directed attention

Effects of spatially directed auditory attention on human brain activity, as indicated by changes in regional cerebral blood flow (rCBF), were measured with positron emission tomography (PET). Subjects attended to left-ear tones, right-ear tones, or foveal visual stimuli presented at rapid rates in three concurrent stimulus sequences. It was found that attending selectively to the right-ear input activated the auditory cortex predominantly in the left hemisphere and vice versa. This selective tuning of the left and right auditory cortices according to the direction of attention was presumably controlled by executive attention mechanisms of the frontal cortex, where enhanced activation during auditory attention was also observed.     

Attentional modulation of early-stage visual processing in schizophrenia

This study shows that paying attention to the color of a visual stimulus is manifested by an early endogenous scalp-positive event-related brain potential (ERP) component, referred to as "selection positivity", that emerges within the first 100 ms after stimulus onset in healthy observers. In contrast, recently ill and chronically ill schizophrenia patients as well as patients at high risk for schizophrenia all failed to show this early ERP component while attending to color. These results suggest that a relatively early stage of visual-selective processing in posterior extrastriate cortex is disrupted in schizophrenia.     

Processing of color, form and disparity information in visual areas VP and V2 of ventral extrastriate cortex in the macaque monkey

The responses of single cells to light bars of different orientation, direction of motion, speed, binocular disparity, and wavelength were systematically analyzed in areas V2 and VP of ventral extrastriate visual cortex in the macaque monkey. Selectivity for each of these parameters was assessed quantitatively using computer-controlled procedures. In both VP and V2 (both representing the superior contralateral quadrant), more than half of the cells studied were selective for stimulus color and more than half for stimulus orientation. In contrast, only a small minority of the VP and V2 cells were selective for the direction of stimulus motion. Comparison with reports of single-unit properties in dorsal extrastriate cortex suggests there are no major differences in the incidence of orientation, direction, and color selectivity between ventral and dorsal subdivisions of V2. Between V3 and VP, though, there are marked differences: Color-selective cells are much less common in V3 than VP, whereas direction-selective cells are more common in V3. This dorsoventral difference in the distribution of neuronal response properties suggests a significant asymmetry in the way visual information is processed in upper and lower parts of the visual field. The properties of cells in VP suggest that it plays an important role in both form and color vision, similar to that attributed to area V4.     

Common effects of emotional valence, arousal and attention on neural activation during visual processing of pictures.

Emotion and attention heighten sensitivity to visual cues. How neural activation patterns associated with emotion change as a function of the availability of attentional resources is unknown. We used positron emission tomography (PET) and 15O-water to measure brain activity in male volunteers while they viewed emotional picture sets that could be classified according to valence or arousal. Subjects simultaneously performed a distraction task that manipulated the availability of attentional resources. Twelve scan conditions were generated in a 3 x 2 x 2 factorial design involving three levels of valence (pleasant, unpleasant and neutral), two levels of arousal and two levels of attention (low and high distraction). Extrastriate visual cortical and anterior temporal areas were independently activated by emotional valence, arousal and attention. Common areas of activation derived from a conjunction analysis of these separate activations revealed extensive areas of activation in extrastriate visual cortex with a focus in right BA18 (12, -88, -2) (Z=5.73, P < 0.001 corrected) and right anterior temporal cortex BA38 (42, 14, -30) (Z=4.03, P < 0.05 corrected). These findings support an hypothesis that emotion and attention modulate both early and late stages of visual processing.     

Task-specific recruitment of dorsal and ventral visual areas during tactile perception

Many studies have found that visual cortical areas are active during tactile perception. Here we used positron emission tomographic (PET) scanning in normally sighted humans to show that extrastriate cortical regions are recruited in a task-specific manner during perceptual processing of tactile stimuli varying in two dimensions. Mental rotation of tactile Forms activated a focus around the anterior part of the left intraparietal sulcus. Since prior studies have reported activity nearby during mental rotation of visual stimuli, this focus appears to be associated with the dorsal visual (visuospatial) pathway. Discrimination between tactile Forms activated the right lateral occipital complex, an object-selective region in the ventral visual (visual Form) pathway. Thus, tactile tasks appear to recruit cortical regions that are active during corresponding visual tasks. Activation of these areas in both visual and tactile tasks could reflect visual imagery during tactile perception, activity in multisensory representations, or both.     

Color Stroop and negative priming in schizophrenia: An fMRI study

Disturbances in selective attention represent a core characteristic of schizophrenia, whose neural underpinnings have yet to be fully elucidated. Consequently, we recorded brain activation using functional magnetic resonance imaging (fMRI) while 15 patients with schizophrenia and 15 age-matched controls performed a well-established measure of selective attention—the color Stroop negative priming task. We focused on two aspects of performance: overriding pre-potent responses (Stroop effect) and inhibition of prior negatively primed trials (negative priming effect). Behaviorally, controls demonstrated both significant Stroop and negative priming effects, while schizophrenic subjects only showed the Stroop effect. For the Stroop effect, fMRI indicated significantly greater activation in frontal regions–medial frontal gyrus/anterior cingulate gyrus and middle frontal gyrus for controls–but greater activation in medial parietal regions (posterior cingulate gyrus/precuneus) for patients. Negative priming elicited significant activation in right dorsolateral prefrontal cortex for both groups, but also in left dorsolateral prefrontal cortex for patients. These different patterns of fMRI activation may reflect faulty interaction in schizophrenia within networks of brain regions that are vital to selective attention.     

PET study of the human foveal fixation system.

Positron emission tomography (PET) was used to investigate the functional anatomy of the foveal fixation system in 10 subjects scanned under three different conditions: at rest (REST), during the fixation of a central point (FIX), and while fixating the same foveal target during the presentation of peripheral visual distractors (DIS). Compared with the REST condition, both FIX and DIS tasks activated a common set of cortical areas. First, in addition to the involvement of the occipital visual cortex, both the frontal eye field (FEF) and the intraparietal sulcus (IPS) were bilaterally activated. Right frontal activation was also found in the dorsolateral prefrontal cortex, the inferior part of the precentral gyrus, and the inferior frontal gyrus. These results suggest that both FEF and IPS may constitute the main cortical regions subserving bilaterally the foveal fixation system in humans. The remaining right frontal activations may be considered as part of the anterior attentional network, supporting a role for the right frontal lobe in the allocation of the attentional mechanisms. Compared with the FIX condition, the DIS task also revealed the perceptual and cognitive processes related to the presence of peripheral visual distractors during foveal fixation. In addition to a bilateral activation of the V5/MT motion-sensitive area, a right FEF-IPS network was activated which may correspond to the engagement of the visuospatial attention. Finally, normalized regional cerebral blood flow (NrCBF) decreases were also observed during both DIS and FIX condition performance. Such NrCBF decreases were centered in the superior and middle temporal gyri, the prefrontal cortex, and the precuneus and the posterior retrosplenial part of the cingulate gyrus.     

Prefrontal representation of stimulus attributes during delay tasks. II. The role of behavioral significance

Rhesus monkeys were trained to perform two visual discrimination tasks with delayed response. In both tasks, the response depended on the color of the cue, a lighted circle in the center of a panel. Red and green guided one task, yellow and blue the other. In the course of performance, a fifth color (violet), non-relevant and inconsequential, was presented at random in the same location as the cues. All 5 stimuli were of equal brightness. Many cells in the dorsolateral prefrontal cortex (sulcus principalis and superior convexity) treated the relevant cues differently than the irrelevant stimulus. In general, cellular reactions to that stimulus were of lesser magnitude than the reactions to the cues. Cell reaction differences as a function of stimulus significance outnumbered and overshadowed differences as a function of cue-color or any other task variable. The results indicate that, during visual delay tasks, units in the dorsolateral prefrontal cortex differentiate stimuli by their behavioral significance, as well as by other stimulus attributes, including color. Because the motivational evaluation of sensory stimuli is an integral part of the cognitive processes in delay tasks (together with short-term memory and motor set), these results support the notion that the prefrontal cortex integrates motivational inputs into the structure of behavioral action.     

Mental imagery of faces and places activates corresponding stiimulus-specific brain regions

What happens in the brain when you conjure up a mental image in your mind's eye? We tested whether the particular regions of extrastriate cortex activated during mental imagery depend on the content of the image. Using functional magnetic resonance imaging (fMRRI), we demonstrated selective activation within a region of cortex specialized for face perception during mental imagery of faces, and selective activation within a place-selective cortical region during imagery of places. In a further study, we compared the activation for imagery and perception in these regions, and found greater response magnitudes for perception than for imagery of the same items. Finally, we found that it is possible to determine the content of single cognitive events from an inspection of the fMRI data from individual imagery trials. These findings strengthen evidence that imagery and perception share common processing mechanisms, and demonstrate that the specific brain regions activated during mental imagery depend on the content of the visual image.     

Visual form discrimination from texture cues: A PET study

With the purpose of localising the cerebral cortical areas participating in the discrimination of visual form generated exclusively by texture cues, we measured changes in regional cerebral blood flow (rCBF) with positron emission tomography (PET) and ¹⁵O-butanol as the tracer. The subjects performed two odd-one-out discrimination tasks: a form-from-texture discrimination task (in which a visual form was defined by differences in texture) and its reference task, the discrimination of texture. During task performance, activated fields were present bilaterally in the primary visual cortex and its immediate extrastriate cortex, the right lateral occipital gyrus, bilaterally in the fusiform and superior temporal gyri and posterior parts of the superior parietal lobules, along the medial bank of the right intraparietal sulcus, and in the right supramarginal gyrus. Other fields were found in the cingulate and prefrontal cortex. The findings demonstrate that the discrimination of visual form as defined by texture engages cortical fields that are widely distributed in the human brain. In the visual cortex, the activated fields are present in both the occipito-temporal and occipito-parietal visual areas. These results suggest that the perception and discrimination of forms in the visual system requires the joint activation of neuronal populations in the visual cortex. Hum. Brain Mapping 6:115–127, 1998. © 1998 Wiley-Liss, Inc.     

Dissociating the neural mechanisms of visual attention in change detection using functional MRI

We investigated using functional magnetic resonance imaging (fMRI) the neural processes associated with performance of a change-detection task. In this task, two versions of the same picture are presented in alternation, separated by a brief mask interval. Even when the two pictures greatly differ (e.g., as when a building is in different locations), subjects report that identification of the change is difficult and often take 30 or more seconds to identify the change. This phenomenon of "change blindness" provides a powerful and novel paradigm for segregating components of visual attention using fMRI that can otherwise be confounded in short-duration tasks. By using a response-contingent event-related analysis technique, we successfully dissociated brain regions associated with different processing components of a visual change-detection task. Activation in the calcarine cortex was associated with task onset, but did not vary with the duration of visual search. In contrast, the pattern of activation in dorsal and ventral visual areas was temporally associated with the duration of visual search. As such, our results support a distinction between brain regions whose activation is modulated by attentional demands of the visual task (extrastriate cortex) and those that are not affected by it (primary visual cortex). A second network of areas including central sulcus, insular, and inferior frontal cortical areas, along with the thalamus and basal ganglia, showed phasic activation tied to the execution of responses. Finally, parietal and frontal regions showed systematic deactivations during task performance, consistent with previous reports that these regions may be associated with nontask semantic processing. We conclude that detection of change, when transient visual cues are not present, requires activation of extrastriate visual regions and frontal regions responsible for eye movements. These results suggest that studies of change blindness can inform understanding of more general attentional processing.     

Feature uncertainty activates anterior cingulate cortex

In visual discrimination tasks, the relevant feature to discriminate is defined before stimulus presentation. In feature uncertainty tasks, a cue about the relevant feature is provided after stimulus offset. We used (15)O-butanol positron emission tomography (PET) in order to investigate brain activation during a feature uncertainty task. There was greater activity during the feature uncertainty task, compared with stimulus detection and discrimination of orientation and spatial frequency, in the lateral and medial prefrontal cortex, the cuneus, superior temporal and inferior parietal cortex, cortical motor areas, and the cerebellum. The most robust and consistent activation was observed in the dorsal anterior cingulate cortex (Brodmann area 32; x = 0 y = 16, z = 40). The insula, located near the claustrum (x = -38, y = 8, z = 4), was activated during the discrimination tasks compared with the feature uncertainty condition. These results suggest that the dorsal anterior cingulate cortex is important in feature uncertainty conditions, which include divided attention, expectancy under uncertainty, and cognitive monitoring.     

Covariations in ERP and PET measures of spatial selective attention in human extrastriate visual cortex

In a previous study using positron emission tomography (PET), we demonstrated that focused attention to a location in the visual field produced increased regional cerebral blood flow in the fusiform gyrus contralateral to the attended hemifield (Heinze et al. [1994]: Nature 372:543). We related these effects to modulations in the amplitude of the P1 component (80–130 msec latency) of the visual event-related brain potentials (ERPs) recorded from the same subjects, under the identical stimulus and task conditions. Here, we replicate and extend these findings by showing that attention effects in the fusiform gyrus and the P1 component were similarly modulated by the perceptual load of the task. When subjects performed a perceptually demanding symbol-matching task within the focus of spatial attention, the fusiform activity and P1 component of the ERP were of greater magnitude than when the subjects performed a less perceptually demanding task that required only luminance detection at the attended location. In the latter condition, both the PET and ERP attention effects were reduced. In addition, in the present data significant activations were also obtained in the middle occipital gyrus contralateral to the attended hemifield, thereby demonstrating that multiple regions of extrastriate visual cortex are modulated by spatial attention. The findings of covariations between the P1 attention effect and activity in the posterior fusiform gyrus reinforce our hypothesis that common neural sources exist for these complementary, but very different measures of human brain activity. Hum. Brain Mapping 5:273–279, 1997. © 1997 Wiley-Liss, Inc.     

Brain mechanisms of involuntary visuospatial attention: an event-related potential study

The brain mechanisms mediating visuospatial attention were investigated by recording event-related potentials (ERPs) during a line-orientation discrimination task. Nonpredictive peripheral cues were used to direct participant's attention involuntarily to a spatial location. The earliest attentional modulation was observed in the P1 component (peak latency about 130 ms), with the valid trials eliciting larger P1 than invalid trials. Moreover, the attentional modulations on both the amplitude and latency of the P1 and N1 components had a different pattern as compared to previous studies with voluntary attention tasks. In contrast, the earliest visual ERP component, C1 (peak latency about 80 ms), was not modulated by attention. Low-resolution brain electromagnetic tomography (LORETA) showed that the earliest attentional modulation occurred in extrastriate cortex (middle occipital gyrus, BA 19) but not in the primary visual cortex. Later attention-related reactivations in the primary visual cortex were found at about 110 ms after stimulus onset. The results suggest that involuntary as well as voluntary attention modulates visual processing at the level of extrastriate cortex; however, at least some different processes are involved by involuntary attention compared to voluntary attention. In addition, the possible feedback from higher visual cortex to the primary visual cortex is faster and occurs earlier in involuntary relative to voluntary attention task.     

Enhanced effectiveness in visuo‐haptic object‐selective brain regions with increasing stimulus salience

The occipital and parietal lobes contain regions that are recruited for both visual and haptic object processing. The purpose of the present study was to characterize the underlying neural mechanisms for bimodal integration of vision and haptics in these visuo‐haptic object‐selective brain regions to find out whether these brain regions are sites of neuronal or areal convergence. Our sensory conditions consisted of visual‐only (V), haptic‐only (H), and visuo‐haptic (VH), which allowed us to evaluate integration using the superadditivity metric. We also presented each stimulus condition at two different levels of signal‐to‐noise ratio or salience. The salience manipulation allowed us to assess integration using the rule of inverse effectiveness. We were able to localize previously described visuo‐haptic object‐selective regions in the lateral occipital cortex (lateral occipital tactile‐visual area) and the intraparietal sulcus, and also localized a new region in the left anterior fusiform gyrus. There was no evidence of superadditivity with the VH stimulus at either level of salience in any of the regions. There was, however, a strong effect of salience on multisensory enhancement: the response to the VH stimulus was more enhanced at higher salience across all regions. In other words, the regions showed enhanced integration of the VH stimulus with increasing effectiveness of the unisensory stimuli. We called the effect “enhanced effectiveness.” The presence of enhanced effectiveness in visuo‐haptic object‐selective brain regions demonstrates neuronal convergence of visual and haptic sensory inputs for the purpose of processing object shape. Hum Brain Mapp, 2010. © 2009 Wiley‐Liss, Inc.