Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials

Results from event-related potential (ERP) studies are reviewed that investigated crossmodal links in spatial attention between vision, audition and touch to find out which stages in the processing of sensory stimuli are affected by such crossmodal links. ERPs were recorded in response to visual, auditory, and tactile stimuli under conditions where attention was directed to a specific location within one (primary) modality, while stimuli in another (secondary) modality were to be ignored regardless of their position. Systematic ERP effects of spatial attention were observed not only in the primary modality, but also for secondary modality stimuli, thus revealing crossmodal links in spatial attention. These links affected relatively early sensory-specific ERP components between 100 and 200 ms post-stimulus. Beyond 200 ms, ERPs to secondary modality stimuli were little affected by the current focus of attention within another modality. This pattern of results suggests that crossmodal links in spatial attention may affect sensory-perceptual processes within modality-specific cortical regions, but have little impact on later post-perceptual processing stages.     

Spatial attention and crossmodal interactions between vision and touch

In the present paper, we review several functional imaging studies investigating crossmodal interactions between vision and touch relating to spatial attention. We asked how the spatial unity of a multimodal event in the external world might be represented in the brain, where signals from different modalities are initially processed in distinct brain regions. The results highlight several links between visual and tactile spatial representations. First, we found that activity in the anterior part of the intraparietal sulcus was influenced by stimulus position independently of the modality of the stimulation. This is consistent with crossmodal interactions via sensory convergence from early modality-specific spatial maps to higher-order multimodal regions. Second, we found that stimulation in, or attention to, one modality could affect activity in areas dedicated to a different modality, in a spatially-specific manner. These spatial crossmodal effects in unimodal regions demonstrate congruous activity in anatomically distant brain areas that represent similar external locations, implicating a distributed network of spatial representations in crossmodal integration. Finally, the results suggest that the temporo-parietal junction may be involved in aspects of controlling spatial attention, for both vision and touch. A multimodal attentional system may influence activity in distinct brain areas representing common regions of space for different modalities, thus suggesting a link between spatial attention and crossmodal integration.     

Effects of hand posture on preparatory control processes and sensory modulations in tactile-spatial attention.

OBJECTIVE: Event-related brain potentials (ERPs) were measured to investigate spatial coordinate systems involved in the control of preparatory tactile-spatial orienting, and in subsequent attentional modulations of somatosensory processing. METHODS: On each trial, a visual precue directed attention to the left or right hand, where infrequent tactile targets had to be detected. Hands were positioned either close together or wide apart. ERPs were recorded in the cue-target interval and in response to attended and unattended tactile non-targets. RESULTS: A frontal anterior directing attention negativity (ADAN) and a posterior late directing attention positivity (LDAP) were elicited in the cue-target interval contralateral to the direction of an attentional shift. The ADAN was unaffected by hand posture, but the LDAP was attenuated when hands were close together. N140 amplitudes were enhanced in response to tactile stimuli presented to the attended hand, and this effect was more pronounced when hands were wide apart. CONCLUSIONS: ADAN and LDAP are linked to separable anterior and posterior attentional control systems, which use coordinate systems based on somatotopic and external space, respectively. Effects of spatial attention on somatosensory stimulus processing are affected by variations in body posture. SIGNIFICANCE: Our results demonstrate that representations of body locations in external space play a central role in the control of tactile attention.     

Cross-modal transfer between vision and touch of go, no-go discrimination learning in the monkey

Four monkeys were tested for specific cross-modal transfer of go, no-go discrimination training: an effect was found on initial error scores, but not on training scores.     

Orienting and maintenance of spatial attention in audition and vision: an event-related brain potential study

We examined the effects of orienting and maintenance of attention on performance and event-related brain potentials (ERPs) in audition and vision. Our subjects selectively attended to sounds or pictures in one location (Maintenance of attention) or alternated the focus of their auditory or visual attention between left and right locations (Orienting of attention) in order to detect and press a response button to infrequent targets among the attended stimuli. Reaction times were longer in the Auditory Orienting condition and hit rates were lower and false alarm rates higher in the Visual Orienting condition than in the corresponding Maintenance conditions. Comparison of ERPs to the attended and unattended stimuli in the Auditory and Visual Orienting and Maintenance conditions revealed attention-related modulations of ERPs. In each modality, ERPs to attended stimuli were negatively displaced in relation to unattended stimuli at 100-250 ms from stimulus onset. These negative differences (Nds) showed modality-specific distributions and they were larger over the hemisphere contralateral to the attended sounds and pictures than over the ipsilateral hemisphere. Moreover, the Nd was larger in the Auditory Orienting condition than in the Auditory Maintenance condition, while no such difference was observed in the visual modality. In addition to the Nd, attended visual stimuli elicited a late positive response (LPR) in both Orienting and Maintenance conditions. In contrast to our recent functional magnetic resonance imaging (fMRI) study employing the same experimental paradigm and indicating orienting-related activity in the frontal and parietal cortices, no ERP responses specifically related to orienting were found in either modality.     

Vision and touch in ageing: crossmodal selective attention and visuotactile spatial interactions

We investigated whether ageing affects crossmodal selective attention (the ability to focus on a relevant sensory modality and ignore an irrelevant modality) and the spatial constraints on such selective processing. Three groups of 24 participants were tested: Young (19-25 years), Young-Old (65-72 years) and Old-Old (76-92 years). The participants had to judge the elevation of vibrotactile targets (upper/index finger and lower/thumb), presented randomly to either hand while ignoring concurrent visual distractors. In a second task, the role of the target and distractor modalities was reversed. Crossmodal selective attention was assessed by comparing performance in the presence versus absence of distractors. Spatial constraints on selective attention were also investigated by comparing the effect of distractors presented on the same versus opposite side as the target. When attending to touch, the addition of visual distractors had a significantly larger effect on error rates in both of the older groups as compared to the Young group. This indicates that ageing has a detrimental effect on crossmodal selective attention. In all three age groups, performance was impaired when the target and distractor were presented at incongruent as compared to congruent elevations in both tasks. This congruency effect was modulated by the relative spatial location of the target and distractor in certain conditions for the Young and the Young-Old group. That is, participants in the two younger age groups found it harder to attend selectively to targets in one modality, when distractor stimuli came from the same side rather than from the opposite side. However, no significant spatial modulation was found in the Old-Old group. This suggests that ageing may also compromise spatial aspects of crossmodal selective attention.     

The spatial distribution of attentional selectivity in touch: evidence from somatosensory ERP components.

OBJECTIVE: Somatosensory event-related brain potentials (ERPs) were measured to investigate the spatial distribution of selective attention in touch, and whether the focus of tactile attention can be split between non-contiguous areas of the body surface. METHODS: On each trial, vibratory tactile stimuli were delivered to one of 4 possible locations of the right hand. Participants had to attend to either one or two locations in order to detect infrequently presented target stimuli there. ERPs were recorded to tactile non-targets at attended and unattended locations. RESULTS: Attention directed to one finger versus another was reflected by amplitude modulations of the sensory-specific P100 component and a subsequent attentional negativity (Nd). These effects were smaller for within-finger as compared to between-finger selection. When attention was directed simultaneously to non-adjacent fingers, ERPs in response to stimuli delivered to spatially and anatomically intervening fingers showed no attentional modulations whatsoever. CONCLUSIONS: Allocating tactile-spatial attention to one finger versus another affects early modality-specific somatosensory processing stages, and these effects of within-hand attentional selectivity decrease gradually with increasing distance from the current attentional focus. Unlike vision, the focus of tactile attention can be split, and directed simultaneously to non-adjacent areas, thus excluding spatially and anatomically intermediate regions from attentional processing.     

Neural correlates of endogenous attention,exogenous attention and inhibition of return in touch

Selective attention helps process the myriad of information constantly touching our body. Both endogenous and exogenous mechanisms are relied upon to effectively process this information; however, it is unclear how they relate in the sense of touch. In three tasks we contrasted endogenous and exogenous event‐related potential (ERP) and behavioural effects. Unilateral tactile cues were followed by a tactile target at the same or opposite hand. Clear behavioural effects showed facilitation of expected targets both when the cue predicted targets at the same (endogenous predictive task) and opposite hand (endogenous counter‐predictive task), and these effects also correlated with ERP effects of endogenous attention. In an exogenous task, where the cue was non‐informative, inhibition of return (IOR) was observed. The electrophysiological results demonstrated early effects of exogenous attention followed by later endogenous attention modulations. These effects were independent in both the endogenous predictive and exogenous tasks. However, voluntarily directing attention away from a cued body part influenced the early exogenous marker (N80). This suggests that the two mechanisms are interdependent, at least when the task requires more demanding shifts of attention. The early marker of exogenous tactile attention, the N80, was not directly related to IOR, which may suggest that exogenous attention and IOR are not necessarily two sides of the same coin. This study adds valuable new insight into how we process and select information presented to our body, showing both independent and interdependent effects of endogenous and exogenous attention in touch.     

Preattentive interference between touch and audition: a case study on multisensory alloesthesia

Alloesthesia is a rare clinical condition that corresponds to a spatial disorder of stimulus localization, in which patients experience a given stimulus on the side opposite to the side of stimulation. Whereas it has been mostly described for unisensory stimulations, evidence of multisensory alloesthesia is only anecdotal. Here, we investigated a case of multisensory auditory-tactile alloesthesia. Our data suggest that auditory-tactile integration and multisensory alloesthesia not only depend on attentional mechanisms, but also on somatotopic preattentive mechanisms.     

Crossmodal links in endogenous and exogenous spatial attention: evidence from event-related brain potential studies

The adaptive control of behaviour in response to relevant external objects and events often requires the selection of information delivered by different sensory systems, but from the same region in external space. This can be facilitated by crossmodal links in the attentional processing of information across sensory modalities. Results from recent event-related potential (ERP) studies are reviewed that investigated mechanisms underlying such crossmodal links in spatial attention between vision, audition and touch. Crossmodal attention effects were observed for early modality-specific visual, auditory, and somatosensory ERP components, indicating that crossmodal links in spatial attention affect sensory-perceptual processes within modality-specific cortical regions. ERP modulations prior to target events but sensitive to the direction of an attentional shift were remarkably similar during anticipatory covert shifts of visual, auditory, or tactile attention. These results suggest that such attentional shifts are mediated by supramodal frontoparietal control mechanisms. Finally, ERP evidence is reviewed suggesting that effects of crossmodal links in endogenous (voluntary) as well as exogenous (involuntary) spatial attention are mediated by a representations of external space which are updated across postural changes.     

Cross-modal interactions in time and space: auditory influence on visual attention in hemispatial neglect

Recent studies indicate that auditory tone presentation and auditory alerting can temporarily ameliorate visuospatial attention deficits in patients with unilateral neglect [Frassinetti, F., Pavani, F., & Ladavas, E. Acoustical vision of neglected stimuli: Interaction among spatially converging audiovisual inputs in neglect patients. Journal of Cognitive Neuroscience, 14, 62-69, 2002; Robertson, I. H., Mattingley, J. B., Rorden, C., & Driver, J. Phasic alerting of neglect patients overcomes their spatial deficit in visual awareness. Nature, 395, 169-172, 1998]. The current study investigated proposed mechanisms of cross-modal interaction to determine conditions in which auditory stimulation affects spatial and nonspatially lateralized attention deficits in a patient with hemispatial neglect. In Experiment 1, a target was presented among related distracters (conjunction search) while a tone was presented either bilaterally or in a congruent or incongruent spatial location with respect to the visual target. Whereas the results suggest a benefit of both general alerting and cross-modal spatial integration on visual search efficiency, the most significant improvement occurred when the target and tone were both presented in contralesional space. In Experiment 2, the effect of auditory alerting on selective attention was examined in a rapid serial visual search procedure with visual targets embedded in a stream of distracters presented at central fixation. When two targets were presented without an alerting tone, the patient missed the second target for up to 1000 msec after the first target appeared (a finding known as the "attentional blink" [AB] and, on average, about 400-500 msec in normals). An alerting tone presented at a fixed temporal location significantly reduced the AB in a tone-duration-dependent manner. Experiment 3 examined the effect of cross-modal space on selective attention in an AB paradigm in which T2 occurred randomly to the left or right of T1 with a spatially congruent or incongruent tone. Discrimination of T2 in contralesional space significantly improved when the tone was presented in the same location, and was impaired when the tone was presented on the ipsilesional side. The findings are discussed as they relate to cross-modal interactions and their influence on spatial and nonspatially lateralized attention deficits in neglect.     

An ERP investigation on visuotactile interactions in peripersonal and extrapersonal space: evidence for the spatial rule

The spatial rule of multisensory integration holds that cross-modal stimuli presented from the same spatial location result in enhanced multisensory integration. The present study investigated whether processing within the somatosensory cortex reflects the strength of cross-modal visuotactile interactions depending on the spatial relationship between visual and tactile stimuli. Visual stimuli were task-irrelevant and were presented simultaneously with touch in peripersonal and extrapersonal space, in the same or opposite hemispace with respect to the tactile stimuli. Participants directed their attention to one of their hands to detect infrequent tactile target stimuli at that hand while ignoring tactile targets at the unattended hand, all tactile nontarget stimuli, and any visual stimuli. Enhancement of ERPs recorded over and close to the somatosensory cortex was present as early as 100 msec after onset of stimuli (i.e., overlapping with the P100 component) when visual stimuli were presented next to the site of tactile stimulation (i.e., perihand space) compared to when these were presented at different locations in peripersonal or extrapersonal space. Therefore, this study provides electrophysiological support for the spatial rule of visual-tactile interaction in human participants. Importantly, these early cross-modal spatial effects occurred regardless of the locus of attention. In addition, and in line with previous research, we found attentional modulations of somatosensory processing only to be present in the time range of the N140 component and for longer latencies with an enhanced negativity for tactile stimuli at attended compared to unattended locations. Taken together, the pattern of the results from this study suggests that visuotactile spatial effects on somatosensory processing occur prior and independent of tactile-spatial attention.     

Interactions between spatial attention and global/local feature selection: an ERP study

The present study examined the interaction between spatial attention and global/local feature processing of visual hierarchical stimuli. Event-related brain potentials (ERPs) were recorded from subjects who detected global or local targets at attended locations while ignoring those at unattended locations. Spatial attention produced enhanced occipital P1 and N1 waves in both global and local conditions. Selection of local features enhanced posterior P1, N1 and N2 waves relative to selection of global features. However, the modulations of the P1 and N2 by global/local feature selection were stronger when spatial attention was directed to the left than the right visual fields. The results suggest neurophysiological bases for interactions between spatial attention and hierarchical analysis at multiple stages of visual processing.     

Relationship between ventral stream for object vision and dorsal stream for spatial vision: an fMRI + ERP study

Recent imaging studies indicated the existence of two visual pathways in humans: a ventral stream for object and form vision and a dorsal stream for spatial and motion vision. The present study was motivated by a stimulating question: Supposing shape and motion are processed separately in the two pathways, how do the respective cortical areas respond to the stimuli of "forms defined by motion"? fMRI and ERP recordings were combined in order to measure the spatiotemporal activation pattern in the two pathways responding to forms defined by motion, which were produced solely by coherent movement of random dots against a background of dynamic or static random dots. The fMRI data indicated that the stimuli of forms defined by motion indeed activated both dorsal MT/V5 and ventral GTi/GF. Furthermore, the RV curves resulting from fMRI-seeded dipole modeling indicated that each pair of dipoles located at MT/V5 or GTi/GF reached the same best-fit point; a single pair of free dipoles located near the fMRI foci of MT/V5 and GTi/GF could be identified at the corresponding best-fit point; and the source waveforms resulting from fixed dipole modeling also showed simultaneous activation of MT/V5 and GTi/GF dipoles in the time interval around the best-fit point. The present results, therefore, suggest that MT/V5 and GTi/GF appear to be activated in parallel and simultaneously responding to forms defined by motion. Such findings raise interesting issues about the hierarchical organization and the functional specialization in the two pathways.     

Spatial attention and conscious perception: interactions and dissociations between and within endogenous and exogenous processes

A current controversy exists about the relationship between spatial attention and conscious perception. While some authors propose that these phenomena are intimately related (Bartolomeo, 2008, Chun and Marois, 2002, O’Regan and Noë, 2001, Posner, 1994), others report dissociations between them (Kentridge et al., 1999, Koch and Tsuchiya, 2007, Wyart and Tallon-Baudry, 2008). However, spatial attention is not a unitary mechanism, and it is possible that not all forms of attention dissociate from conscious perception. In the present study we used a paradigm in which endogenous and exogenous forms of attention are orthogonally manipulated in order to investigate their relation with conscious perception within the same design. By analyzing two different cue-related components, our results demonstrated that while endogenous attention was electrophysiologically dissociated from conscious perception, exogenous attention was not, consistent with the hypothesis that exogenous attention is an important antecedent of our conscious experience. Our results support previous claims of dissociations between some forms of spatial attention and conscious perception, but also highlight the importance of exogenous orienting on the selection of information for conscious access.     

Additive effects of emotional, endogenous, and exogenous attention: behavioral and electrophysiological evidence

Selective attention is not a unitary construct, but is composed of several processes. Attention selection may be guided by low-level stimulus properties, by the emotional value of the stimulus, or more voluntarily by the goals and plans of the observer. Whether these three systems operate independently during attention selection or not remains a debated question. We report results from two studies investigating the extent to which these different attention mechanisms may interact with one another. Using a standard dot probe paradigm wherein effects of exogenous, emotional, and endogenous attention were orthogonally manipulated, we found attentional facilitation effects for each component, indicated by faster decision times for validly, as opposed to invalidly cued targets. Moreover, results confirmed that these three attentional effects added up in a linear fashion. Complementing ERP results allowed us to disentangle the respective contributions of the two reflexive, bottom-up attention processes (exogenous vs. emotional) by showing non-overlapping temporal loci for attentional effects related either to low-level physical properties or the emotional content of the stimulus. These findings suggest that multiple separate attention mechanisms can operate simultaneously to yield a rapid and efficient visual processing of various classes of potentially relevant stimuli.     

Manual response preparation and saccade programming are linked to attention shifts: ERP evidence for covert attentional orienting and spatially specific modulations of visual processing

The premotor theory of attention claims that attentional shifts are triggered during response programming, regardless of which response modality is involved. To investigate this claim, event-related brain potentials (ERPs) were recorded while participants covertly prepared a left or right response, as indicated by a precue presented at the beginning of each trial. Cues signalled a left or right eye movement in the saccade task, and a left or right manual response in the manual task. The cued response had to be executed or withheld following the presentation of a Go/Nogo stimulus. Although there were systematic differences between ERPs triggered during covert manual and saccade preparation, lateralised ERP components sensitive to the direction of a cued response were very similar for both tasks, and also similar to the components previously found during cued shifts of endogenous spatial attention. This is consistent with the claim that the control of attention and of covert response preparation are closely linked. N1 components triggered by task-irrelevant visual probes presented during the covert response preparation interval were enhanced when these probes were presented close to cued response hand in the manual task, and at the saccade target location in the saccade task. This demonstrates that both manual and saccade preparation result in spatially specific modulations of visual processing, in line with the predictions of the premotor theory.     

Fearful,surprised, happy,and angry facial expressions modulate gaze-oriented attention: Behavioral and ERP evidence

The impact of emotions on gaze-oriented attention was investigated in non-anxious participants. A neutral face cue with straight gaze was presented, which then averted its gaze to the side while remaining neutral or expressing an emotion (fear/surprise in Exp.1 and anger/happiness in Exp.2). Localization of a subsequent target was faster at the gazed-at location (congruent condition) than at the non-gazed-at location (incongruent condition). This Gaze-Orienting Effect (GOE) was enhanced for fear, surprise, and anger, compared to neutral expressions which did not differ from happy expressions. In addition, Event Related Potentials (ERPs) to the target showed a congruency effect on P1 for fear and surprise and a left lateralized congruency effect on P1 for happy faces, suggesting that target visual processing was also influenced by attention to gaze and emotions. Finally, at cue presentation, early postero-lateral (Early Directing Attention Negativity (EDAN)) and later antero-lateral (Anterior Directing Attention Negativity (ADAN)) attention-related ERP components were observed, reflecting, respectively, the shift of attention and its holding at gazed-at locations. These two components were not modulated by emotions. Together, these findings show that the processing of social signals such as gaze and facial expression interact rather late and in a complex manner to modulate spatial attention.     

Spatial coordinate systems for tactile spatial attention depend on developmental vision: evidence from event-related potentials in sighted and congenitally blind adult humans

Changes in limb posture (such as crossing the hands) can impair people's performance in tasks such as those involving temporal order judgements, when one tactile stimulus is presented to either hand. This crossed hands deficit has been attributed to a conflict between externally and anatomically anchored reference systems when people localize tactile stimuli. Interestingly, however, the performance of congenitally blind adults does not seem to be affected by crossing the hands, suggesting a default use of an anatomically rather than an externally anchored reference system for tactile localization. In the present study, 12 congenitally blind and 12 sighted adults were instructed to attend to either the left or the right hand on a trial-by-trial basis in order to detect rare deviants (consisting of a double touch) at that hand, while ignoring both deviants at the other hand and frequent standard stimuli (consisting of a single touch) presented to either hand. Only the sighted participants performed less accurately when they crossed their hands. Concurrent electroencephalogram recordings revealed an early contralateral attention positivity, followed by an attention negativity in the sighted group when they adopted the uncrossed hands posture. For the crossed hand posture, only the attention negativity was observed with reduced amplitude in the sighted group. By contrast, the congenitally blind group displayed an event-related potential attention negativity that did not vary when the posture of their hands was changed. These results demonstrate that the default use of an external frame of reference for tactile localization seems to depend on developmental vision.     

Shifts of attention in light and in darkness: an ERP study of supramodal attentional control and crossmodal links in spatial attention

Crossmodal links in spatial attention, uncovered by recent behavioural and electrophysiological studies, have been interpreted as evidence for supramodal processes controlling shifts of attention. However, previous experiments have usually been conducted in illuminated environments. Continuously available visuo-spatial information might result in shifts of attention being primarily guided by visible information, even when another modality is task-relevant. The present ERP study evaluated this. A symbolic auditory cue directed attention to the left or right hand. Participants had to detect infrequent tactile targets delivered to the cued hand, while ignoring any visual stimuli. Stimuli were presented either in a lit environment or in darkness. Although continuous ambient visuo-spatial information was eliminated in the latter condition, processing of task-irrelevant visual events was still modulated by spatial attention for the tactile task. Moreover, ERP correlates of attentional shifts in the cue-target interval were similar for both illumination conditions. This was further confirmed in a follow-up experiment where the darkness condition was repeated without any peripheral visual stimulation ever occurring. These findings demonstrate that the ERP correlates of crossmodal attention (both preparatory effects in the cue-target interval, and also modulations of stimulus-evoked components) do not depend on selection being guided by ambient visible information in a lit environment. They suggest instead that spatial shifts of attention are controlled supramodally.