Age-related multisensory integration elicited by peripherally presented audiovisual stimuli

Although age-related multisensory integration has been investigated previously, the effects of aging on multisensory integration elicited by peripherally presented audiovisual (AV) stimuli remain unclear. In this study, visual, auditory, and AV stimuli were randomly presented to the left or the right of the central fixation point; during this time, participants (young and old adults) were asked to respond promptly to target stimuli. Using a race model to analyze the response times, our results showed that the AV behavioral facilitation of young participants was significant (P<0.05), with response times ranging from 240 to 450 ms and peaking at 360 ms (14%). However, the AV behavioral facilitation of elderly participants was delayed and showed an extensive range, with response times ranging from 260 to 540 ms and with a lower peak (12.6%) and delay time (390 ms). We found that the time window of AV behavioral facilitation in elderly participants was longer but more delayed than that in the young participants when the AV stimuli were presented peripherally. This finding also further confirmed that peripheral resolution decreased with age.     

The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Object recognition benefits maximally from multimodal sensory input when stimulus presentation is noisy, or degraded. Whether this advantage can be attributed specifically to the extent of overlap in object‐related information, or rather, to object‐unspecific enhancement due to the mere presence of additional sensory stimulation, remains unclear. Further, the cortical processing differences driving increased multisensory integration (MSI) for degraded compared with clear information remain poorly understood. Here, two consecutive studies first compared behavioral benefits of audio‐visual overlap of object‐related information, relative to conditions where one channel carried information and the other carried noise. A hierarchical drift diffusion model indicated performance enhancement when auditory and visual object‐related information was simultaneously present for degraded stimuli. A subsequent fMRI study revealed visual dominance on a behavioral and neural level for clear stimuli, while degraded stimulus processing was mainly characterized by activation of a frontoparietal multisensory network, including IPS. Connectivity analyses indicated that integration of degraded object‐related information relied on IPS input, whereas clear stimuli were integrated through direct information exchange between visual and auditory sensory cortices. These results indicate that the inverse effectiveness observed for identification of degraded relative to clear objects in behavior and brain activation might be facilitated by selective recruitment of an executive cortical network which uses IPS as a relay mediating crossmodal sensory information exchange.     

Hemodynamic response function abnormalities in schizophrenia during a multisensory detection task

Functional magnetic resonance imaging (fMRI) of the blood oxygen level dependent (BOLD) response has commonly been used to investigate the neuropathology underlying cognitive and sensory deficits in patients with schizophrenia (SP) by examining the positive phase of the BOLD response, assuming a fixed shape for the hemodynamic response function (HRF). However, the individual phases (positive and post‐stimulus undershoot (PSU)) of the HRF may be differentially affected by a variety of underlying pathologies. The current experiment used a multisensory detection task with a rapid event‐related fMRI paradigm to investigate both the positive and PSU phases of the HRF in SP and healthy controls (HC). Behavioral results indicated no significant group differences during task performance. Analyses that examined the shape of the HRF indicated two distinct group differences. First, SP exhibited a reduced and/or prolonged PSU following normal task‐related positive BOLD activation in secondary auditory and visual sensory areas relative to HC. Second, SP did not show task‐induced deactivation in the anterior node of the default‐mode network (aDMN) relative to HC. In contrast, when performing traditional analyses that focus on the positive phase, there were no group differences. Interestingly, the magnitude of the PSU in secondary auditory and visual areas was positively associated with the magnitude of task‐induced deactivation within the aDMN, suggesting a possible common neural mechanism underlying both of these abnormalities (failure in neural inhibition). Results are consistent with recent views that separate neural processes underlie the two phases of the HRF and that they are differentially affected in SP. Hum Brain Mapp 37:745–755, 2016. © 2015 Wiley Periodicals, Inc.     

Reduced multisensory integration in patients with schizophrenia on a target detection task

A growing body of literature demonstrates impaired multisensory integration (MSI) in patients with schizophrenia compared to non-psychiatric individuals. One of the most basic measures of MSI is intersensory facilitation of reaction times (RTs), in which bimodal targets, with cues from two sensory modalities, are detected faster than unimodal targets. This RT speeding is generally attributed to super-additive processing of multisensory targets. In order to test whether patients with schizophrenia are impaired on this basic measure of MSI, we assessed the degree of intersensory facilitation for a sample of 20 patients compared to 20 non-psychiatric individuals using a very simple target detection task. RTs were recorded for participants to detect targets that were either unimodal (auditory alone, A; visual alone, V) or bimodal (auditory + visual, AV). RT distributions to detect bimodal targets were compared with predicted RT distributions based on the summed probability distribution of each participant's RTs to visual alone and auditory alone targets. Patients with schizophrenia showed less RT facilitation when detecting bimodal targets relative to non-psychiatric individuals, even when groups were matched for unimodal RTs. Within the schizophrenia group, RT benefit was correlated with negative symptoms, such that patients with greater negative symptoms showed the least RT facilitation (r² = 0.20, p < 0.05). Additionally, schizophrenia patients who experienced both auditory and visual hallucinations showed less multisensory benefit compared to patients who experienced only auditory hallucinations, indicating that the presence of hallucinations in two modalities may more strongly impair MSI compared to hallucinations in only one modality.     

Dissociation of perception and action in audiovisual multisensory integration

The ‘temporal rule’ of multisensory integration (MI) proposes that unisensory stimuli, and the neuronal responses they evoke, must fall within a window of integration. Ecological validity demands that MI should occur only for physically simultaneous events (which may give rise to non‐simultaneous neural activations), and spurious neural response simultaneities unrelated to environmental multisensory occurrences must somehow be rejected. Two experiments investigated the requirements of simultaneity for facilitative MI. Experiment 1 employed an reaction time (RT)/race model paradigm to measure audiovisual (AV) MI as a function of AV stimulus‐onset asynchrony (SOA) under fully dark adapted conditions for visual stimuli that were either rod‐ or cone‐isolating. Auditory stimulus intensity was constant. Despite a 155‐ms delay in mean RT to the scotopic vs. photopic stimulus, facilitative AV MI in both conditions occurred exclusively at an AV SOA of 0 ms. Thus, facilitative MI demands both physical and physiological simultaneity. Experiment 2 investigated the accuracy of simultaneity and temporal order judgements under the same stimulus conditions. Judgements of AV stimulus simultaneity or temporal order were significantly influenced by stimulus intensity, indicating different simultaneity requirements for these tasks. The possibility was considered that there are mechanisms by which the nervous system may take account of variations in response latency arising from changes in stimulus intensity in order to selectively integrate only those physiological simultaneities that arise from physical simultaneities. It was proposed that separate subsystems for AV MI exist that pertain to action and perception.     

Neural correlates of multisensory integration of ecologically valid audiovisual events

A question that has emerged over recent years is whether audiovisual (AV) speech perception is a special case of multi-sensory perception. Electrophysiological (ERP) studies have found that auditory neural activity (N1 component of the ERP) induced by speech is suppressed and speeded up when a speech sound is accompanied by concordant lip movements. In Experiment 1, we show that this AV interaction is not speech-specific. Ecologically valid nonspeech AV events (actions performed by an actor such as handclapping) were associated with a similar speeding-up and suppression of auditory N1 amplitude as AV speech (syllables). Experiment 2 demonstrated that these AV interactions were not influenced by whether A and V were congruent or incongruent. In Experiment 3 we show that the AV interaction on N1 was absent when there was no anticipatory visual motion, indicating that the AV interaction only occurred when visual anticipatory motion preceded the sound. These results demonstrate that the visually induced speeding-up and suppression of auditory N1 amplitude reflect multisensory integrative mechanisms of AV events that crucially depend on whether vision predicts when the sound occurs.     

Primary and multisensory cortical activity is correlated with audiovisual percepts

Incongruent auditory and visual stimuli can elicit audiovisual illusions such as the McGurk effect where visual /ka/ and auditory /pa/ fuse into another percept such as/ta/. In the present study, human brain activity was measured with adaptation functional magnetic resonance imaging to investigate which brain areas support such audiovisual illusions. Subjects viewed trains of four movies beginning with three congruent /pa/ stimuli to induce adaptation. The fourth stimulus could be (i) another congruent /pa/, (ii) a congruent /ka/, (iii) an incongruent stimulus that evokes the McGurk effect in susceptible individuals (lips /ka/ voice /pa/), or (iv) the converse combination that does not cause the McGurk effect (lips /pa/ voice/ ka/). This paradigm was predicted to show increased release from adaptation (i.e. stronger brain activation) when the fourth movie and the related percept was increasingly different from the three previous movies. A stimulus change in either the auditory or the visual stimulus from /pa/ to /ka/ (iii, iv) produced within‐modality and cross‐modal responses in primary auditory and visual areas. A greater release from adaptation was observed for incongruent non‐McGurk (iv) compared to incongruent McGurk (iii) trials. A network including the primary auditory and visual cortices, nonprimary auditory cortex, and several multisensory areas (superior temporal sulcus, intraparietal sulcus, insula, and pre‐central cortex) showed a correlation between perceiving the McGurk effect and the fMRI signal, suggesting that these areas support the audiovisual illusion. Hum Brain Mapp, 2010. © 2009 Wiley‐Liss, Inc.     

Age-related changes in performance on a visual-closure task

A sample of 645 volunteer subjects, 50-79 years of age, took part for two consecutive years in a neuropsychological study of perception and memory. The sample was divided into three age groups (50-59, 60-69, and 70-79) in order to examine any age-related changes in performance. On a visual-closure task, similar to that of Gollin (1960), all age groups showed significant improvements in performance (savings) with repeated exposure to the drawings, both after a 15-min, and again after a 1-year, time interval. There were significant differences, however, between the different age groups on all aspects of the task, including number of errors made during initial perceptual performance, percentage of savings over both short- and long-time intervals, and number of items recalled. These findings are contrasted with those from studies of implicit and explicit aspects of verbal memory in normal young and old people, which have found no age-related differences in implicit memory abilities. Implications of these findings, as they relate to the neuropsychological assessment of memory in older people, are discussed briefly.     

Age-related changes in performance on a visual-closure task

Abstract

A sample of 645 volunteer subjects, 50-79 years of age, took part for two consecutive years in a neuropsychological study of perception and memory. The sample was divided into three age groups (50-59, 60-69, and 70-79) in order to examine any age-related changes in performance. On a visual-closure task, similar to that of Gollin (1960), all age groups showed significant improvements in performance (savings) with repeated exposure to the drawings, both after a 15-min, and again after a 1-year, time interval. There were significant differences, however, between the different age groups on all aspects of the task, including number of errors made during initial perceptual performance, percentage of savings over both short- and long-time intervals, and number of items recalled. These findings are contrasted with those from studies of implicit and explicit aspects of verbal memory in normal young and old people, which have found no age-related differences in implicit memory abilities. Implications of these findings, as they relate to the neuropsychological assessment of memory in older people, are discussed briefly.     

Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception

We studied the interactions in neural processing of auditory and visual speech by recording event-related brain potentials (ERPs). Unisensory (auditory - A and visual - V) and audiovisual (AV) vowels were presented to 11 subjects. AV vowels were phonetically either congruent (e.g., acoustic /a/ and visual /a/) or incongruent (e.g., acoustic /a/ and visual /y/). ERPs to AV stimuli and the sum of the ERPs to A and V stimuli (A+V) were compared. Similar ERPs to AV and A+V were hypothesized to indicate independent processing of A and V stimuli. Differences on the other hand would suggest AV interactions. Three deflections, the first peaking at about 85 ms after the A stimulus onset, were significantly larger in the ERPs to A+V than in the ERPs to both congruent and incongruent AV stimuli. We suggest that these differences reflect AV interactions in the processing of general, non-phonetic, features shared by the acoustic and visual stimulus (spatial location, coincidence in time). The first difference in the ERPs to incongruent and congruent AV vowels peaked at 155 ms from the A stimuli onset. This and two later differences are suggested to reflect interactions at phonetic level. The early general AV interactions probably reflect modified activity in the sensory-specific cortices, whereas the later phonetic AV interactions are likely generated in the heteromodal cortices. Thus, our results suggest that sensory-specific and heteromodal brain regions participate in AV speech integration at separate latencies and are sensitive to different features of A and V speech stimuli.     

Enhanced multisensory integration and motor reactivation after active motor learning of audiovisual associations

Everyday experience affords us many opportunities to learn about objects through multiple senses using physical interaction. Previous work has shown that active motor learning of unisensory items enhances memory and leads to the involvement of motor systems during subsequent perception. However, the impact of active motor learning on subsequent perception and recognition of associations among multiple senses has not been investigated. Twenty participants were included in an fMRI study that explored the impact of active motor learning on subsequent processing of unisensory and multisensory stimuli. Participants were exposed to visuo-motor associations between novel objects and novel sounds either through self-generated actions on the objects or by observing an experimenter produce the actions. Immediately after exposure, accuracy, RT, and BOLD fMRI measures were collected with unisensory and multisensory stimuli in associative perception and recognition tasks. Response times during audiovisual associative and unisensory recognition were enhanced by active learning, as was accuracy during audiovisual associative recognition. The difference in motor cortex activation between old and new associations was greater for the active than the passive group. Furthermore, functional connectivity between visual and motor cortices was stronger after active learning than passive learning. Active learning also led to greater activation of the fusiform gyrus during subsequent unisensory visual perception. Finally, brain regions implicated in audiovisual integration (e.g., STS) showed greater multisensory gain after active learning than after passive learning. Overall, the results show that active motor learning modulates the processing of multisensory associations.     

Assessing age-related multisensory enhancement with the time-window-of-integration model

Although from multisensory research a great deal is known about how the different senses interact, there is little knowledge as to the impact of aging on these multisensory processes. In this study, we measured saccadic reaction time (SRT) of aged and young individuals to the onset of a visual target stimulus with and without an accessory auditory stimulus occurring (focused attention task). The response time pattern for both groups was similar: mean SRT to bimodal stimuli was generally shorter than to unimodal stimuli, and mean bimodal SRT was shorter when the auditory accessory was presented ipsilaterally rather than contralaterally to the target. The elderly participants were considerably slower than the younger participants under all conditions but showed a greater multisensory enhancement, that is, they seem to benefit more from bimodal stimulus presentation. In an attempt to weigh the contributions of peripheral sensory processes relative to more central cognitive processes possibly responsible for the difference in the younger and older adults, the time-window-of-integration (TWIN) model for crossmodal interaction in saccadic eye movements developed by the authors was fitted to the data from both groups. The model parameters suggest that (i) there is a slowing of the peripheral sensory processing in the elderly, (ii) as a result of this slowing, the probability of integration is smaller in the elderly even with a wider time-window-of-integration, and (iii) multisensory integration, if it occurs, manifests itself in larger neural enhancement in the elderly; however, because of (ii), on average the integration effect is not large enough to compensate for the peripheral slowing in the elderly.     

Intermanual transfer in a simple motor task

The present study examines the effects of a four-week training session in a pegboard task on uni- and bimanual performance. Of particular interest was whether practice transfer from the trained to the untrained hand takes place. Twenty-five consistently right-handed subjects were trained to perform the task with the dominant hand, or the subdominant hand, or with both hands. After this training, the learning effects for the trained and untrained hands were analysed. To summarise, we obtained the following findings: (1) After training, movement times were considerably reduced for all hands and for all training conditions (practice effects); (2) practice effects were found for the hand trained and also for the untrained hand; (3) there was not a great difference in the size of the practice effects for the right hand after left hand training or for the left hand after right hand training; (4) task difficulty had no clear influence on the practice effect; (5) and finally, we discovered that bimanual movements not only profit from bimanual training but also from unimanual training and conversely unimanual movements benefit from bimanual training. These findings are discussed in the context of different motor control models and in the light of recent brain imaging findings.     

Observing executive activity in a simple assembly task

Performance of a group of adult trainees in a workshop for mentally retarded persons and that of an equivalent-MA group of third-grade children was observed in a toy construction task. Performances were video taped and subsequently coded for executive activity and other classes of behavior. A range of executive activities was identified, and these proved to be a powerful influence in discriminating between the groups. Other analyses of event sequences examined where executive activity occurred and which types of events typically followed such activity.     

Age-related differences in auditory event-related potentials during a cued attention task.

OBJECTIVE: To determine if aging is associated with differences in attentional regulation using behavioral and event-related potential (ERP) measures. METHODS: Younger (n=13;M=20 years) and older (n=12;M=76 years) subjects performed an auditory cued attention task. Verbal cues correctly (valid) or incorrectly (invalid) predicted the ear receiving a target tone 1.5 s later, or were uninformative (neutral). Targets were either 'high' (2000 Hz) or 'low' (1000 Hz) pitch monaural tones. Subjects pressed one of 4 buttons to indicate target ear and pitch. ERPs following cues and targets (P50, N100, P200, slow waves), and negative slow potentials (CNV) between cues and targets were assessed. RESULTS: Cue information had significant effects on reaction time for both groups (validvalid) but not older subjects. Target slow waves were also affected by cue information (invalid>valid), and the difference was larger and lasted longer in older subjects. Slow waves following cues were significantly larger in older subjects, but the subsequent CNV amplitudes were comparable among groups. CONCLUSIONS: When performing a cued attention task, age differences are present in transient ERPs following cues and targets. SIGNIFICANCE: Age differences in ERPs associated with attentional regulation support the hypothesis that attentional changes contribute to cognitive aging.     

Age-related differences in corticomotor excitability and inhibitory processes during a visuomotor RT task

This study tested the postulation that change in the ability to modulate corticospinal excitability and inhibitory processes underlie age-related differences in response preparation and generation during tasks requiring either rapid execution of a motor action or actively withholding that same action. Younger (n = 13, mean age = 26.0 years) and older adults (n = 13, mean age = 65.5 years) performed an RT task in which a warning signal (WS) was followed by an imperative signal (IS) to which participants were required to respond with a rapid flexion of the right thumb (go condition) or withhold their response (no-go condition). We explored the neural correlates of response preparation, generation, and inhibition using single- and paired-pulse TMS, which was administered at various times between WS and IS (response preparation phase) and between IS and onset of response-related muscle activity in the right thumb (response generation phase). Both groups exhibited increases in motor-evoked potential amplitudes (relative to WS onset) during response generation; however, this increase began earlier and was more pronounced for the younger adults in the go condition. Moreover, younger adults showed a general decrease in short-interval intracortical inhibition during response preparation in both the go and no-go conditions, which was not observed in older adults. Importantly, correlation analysis suggested that for older adults the task-related increases of corticospinal excitability and intracortical inhibition were associated with faster RT. We propose that the declined ability to functionally modulate corticospinal activity with advancing age may underlie response slowing in older adults.     

Coregistration of EEG and fMRI in a simple motor task

The purpose of this study was to evaluate the adequacy of coregistration of movement-related cortical potentials (MRCPs) and functional magnetic resonance imaging (fMRI) data in the primary sensorimotor cortex. Data were acquired in four normal subjects during right and left simple index finger movements. In fMRI (single-slice, 1.5 Tesla, T2^*-weighted FLASH sequence), contralateral primary motor (M1) and primary sensory cortex (S1) were activated in all subjects. Spatiotemporal dipole modelling of electric MRCP generators (BESA) revealed two main sources in the central region contralateral to the moving finger. Both sources were tangentially oriented. Their configuration was consistent with source locations in the anterior (M1) and posterior (S1) banks of the central sulcus. Accordingly, the M1 source generated the pre-movement, the S1 source largely the immediate post-movement MRCP component. Taken together, MRCP modelling and fMRI data indicated a phasic sequential activation pattern of mostly sulcal portions of contralateral M1 and S1. After coregistration of anatomical MRI, fMRI, and dipole modelling results, the average 3D-distance between fMRI activation areas and MRCP source locations was 18.6 mm (SD 7.6), with the largest deviation in the anterior-posterior direction (12.1 ± 9.5 mm). Coregistration inaccuracies of similar magnitude (∼ 17 mm) have been reported previously with MEG and PET or fMRI. We conclude, therefore, that the combination of EEG and fMRI is a promising technique for validation of electrophysiological source models and for evaluation of human functional brain anatomy with both adequate spatial and temporal resolution. © 1996 Wiley-Liss, Inc.     

An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection

Physiological and behavioral studies in animal models have revealed that information from the different senses can be used synergistically to enhance the detection of objects and events. Although a great deal of evidence exists which demonstrates the synergistic use of multisensory cues in human behavior and perception, there is conflicting evidence about whether such interactions can be used to aid in stimulus detection. To examine this issue, we had volunteers indicate the presence or absence of a brief, low-intensity sound that was either presented alone or paired with a simultaneous light in a one-interval signal detection task. In the first experiment, the task-irrelevant light was found to enhance the detectability of the sound, but also to increase the subject's willingness to report the presence of a sound (i.e., response bias). In the second experiment, designed to eliminate this response bias, we found an improvement in stimulus detectability in the absence of any change in bias. These findings demonstrate the presence of a significant multisensory-mediated gain in stimulus detection in human subjects.     

No evidence for impaired multisensory integration of low-level audiovisual stimuli in adolescents and young adults with autism spectrum disorders

Abrupt click sounds can improve the visual processing of flashes in several ways. Here, we examined this in high functioning adolescents with Autism Spectrum Disorders (ASD) using three tasks: (1) a task where clicks improve sensitivity for visual temporal order (temporal ventriloquism); (2) a task where a click improves visual search (pip-and-pop), and (3) a task where a click speeds up the visual orienting to a peripheral target (clock reading). Adolescents with ASD were, compared to adolescents with typical development (TD), impaired in judgments of visual temporal order, but they were unimpaired in visual search and orienting. Importantly, in all tasks visual performance of the ASD group improved by the presence of clicks by at least equal amounts as in the TD group. This suggests that adolescents and young adults with ASD show no generalized deficit in the multisensory integration of low-level audiovisual stimuli and/or the phasic alerting by abrupt sounds.