Cross-modal selective attention to visual and auditory stimuli modulates endogenous ERP components

In two experiments event-related potentials (ERPs) to visual and auditory stimuli were measured in 12 healthy subjects. A cross-modal and delayed response paradigm was used that allows ERPs to be obtained separately to attended and unattended stimuli under conditions in which unattended stimuli are less likely to be covertly or randomly attended. The results showed: (1) N1 enhancement with attention for standard stimuli in auditory and visual modalities and for deviant stimuli in the visual modality; (2) The onset time and scalp distribution of both the N1 for attend condition and Nd1 were similar regardless of standard or deviant stimuli in the auditory and visual modality; the onset time of Nd1 elicited by auditory and visual deviant stimuli was earlier than that of the unattended N1, and their scalp distributions were different; and (3) The Nd1 components elicited by auditory and visual deviant stimuli were distributed over the respective primary sensory areas, but Nd1 components evoked by auditory and visual standard stimuli were distributed over the frontal scalp. These results suggest that the attended N1 enhancement is primarily caused by a component with endogenous origins and that the early attention effect occurs before the exogenous components. The results support the view that the cross-modal attention to deviant stimuli modulates modality-specific processing in the brain, whereas attention to standard stimuli affects modality-nonspecific or supramodal brain systems.     

Visual mismatch negativity among patients with schizophrenia

Event related potentials (ERPs) provide an insight into sensory and cognitive processes in health and disease. Studies of an ERP negative amplitude deflection elicited by a change in a series of auditory stimuli is known as mismatch negativity (MMN). The generation of MMN is impaired in schizophrenia. Its deficit is associated with lower everyday functioning and may be also interpreted as the marker of progression in schizophrenia. MMN elicited by visual stimuli (vMMN) was described by several research teams, but it has not been investigated in schizophrenia as yet. Using a motion-direction paradigm, we elicited visual MMN in 24 patients with schizophrenia and schizoaffective disorder. The vMMN was computed as differences in areas under curve of visual ERPs to standard and deviant motion-direction stimuli recorded from midline derivations at the interval of 100-200 ms. They were compared between groups of patients with schizophrenia and healthy controls. The significantly smaller vMMN indicated an impaired generation of mismatch negativity in patients with schizophrenia. In secondary analyses there was an association of vMMN impairment among patients with higher dose of medication, lower level of functioning and the presence of deficit syndrome. This impairment appears analogous to the impairment of MMN in the auditory domain and is probably related to early visual information processing. Its relationship to cognitive functioning of patients with schizophrenia deserves further attention.     

Mismatch negativity of the color modality during a selective attention task to auditory stimuli in children with mental retardation

Event-related potentials (ERPs) in response to a stimulus change in the visual color modality were recorded in normal subjects and children with mental retardation (MR) under selective attention conditions with auditory stimuli. The paradigm included the presentation of a standard (blue color screen, B) or deviant (red, R, or greenish blue color screen, GB) visual stimulus, and a target or non-target tone burst stimulus. In Experiment 1, negativity of the subtracted waveform in response to visual stimuli with a latency of 250-280 ms was clearly observed in the ERPs of normal adults. These potentials prominently appeared at posterior sites in one condition, for which the deviant was GB, but were frontal site-dominant for the other condition. A P300 response to visual deviance was not observed in the GB-B paradigm and the subtracted negativity for this paradigm seemed to be more evident than that for the R-B paradigm. The subtracted negativities could be detected in the range of 180-400 ms after the stimulus onset in control children for the GB-B paradigm. The grand average waves of subtracted ERPs in normal children showed a similar distribution to that in normal adults. Similar subtracted potentials could be recorded with the same paradigm in children with MR, however, the negativities were different in waveform and spatial distribution than in controls. Therefore, the subtracted negativity of the present visual modality represented the analogue of the auditory mismatch negativity (MMN), and so-called 'visual MMN' was detectable in children and even in MR patients when the selective attention was directed to other stimuli.     

Intracranial identification of an electric frontal-cortex response to auditory stimulus change: a case study

The aim of the present study was to clarify whether ERPs recorded directly from the human frontal cortex contributed to the auditory N1 and mismatch negativity (MMN) elicited by changes in non-phonetic and phonetic sounds. We examined the role of prefrontal cortex in the processing of stimulus repetition and change in a 6-year-old child undergoing presurgical evaluation for epilepsy. EEG was recorded from three bilateral sub-dural electrode strips located over lateral prefrontal areas during unattended auditory stimulation. EEG epochs were averaged to obtain event-related potentials (ERPs) to repeating (standard) tones and to infrequent (deviant) shorter duration tones and complex sounds (telephone buzz). In another condition, ERPs were recorded to standard and deviant syllables, /ba/ and /da/, respectively. ERPs to vibration stimuli delivered to the fingertips were not observed at any of the sub-dural electrodes, confirming modality specificity of the auditory responses. Focal auditory ERPs consisting of P100 and N150 deflections were recorded to both tones and phonemes over the right lateral prefrontal cortex. These responses were insensitive to the serial position of the repeating sound in the stimulus train. Deviant tones evoked an MMN peaking at around 128 ms. Deviant complex sounds evoked ERPs with a similar onset latency and morphology but with an approximately two-fold increase in peak-to-peak amplitude. We conclude that right lateral prefrontal cortex (Brodmann's area 45) is involved in early stages of processing repeating sounds and sound changes.     

An electrophysiological and behavioral investigation of involuntary attention towards auditory frequency,duration and intensity changes

We measured behavior and event-related brain potentials (ERPs) in 12 subjects performing on an audio-visual distraction paradigm to investigate the cerebral mechanisms of involuntary attention towards stimulus changes in the acoustic environment. Subjects classified odd/even numbers presented on a computer screen 300 ms after the occurrence of a task-irrelevant auditory stimulus, by pressing the corresponding response button. Auditory stimuli were standard tones (600 Hz, 200 ms, 85 dB; P=0.8) or deviant tones (P=0.2), these differing from the standard either in frequency (700 Hz), duration (50 ms) or intensity (79 dB), in separate blocks. In comparison to performance to visual stimuli following the standard tones, reaction time increased by 24 ms (F(1,11)=10.91, P<0.01) and hit rate decreased by 4.6% (F(1,11)=35.47, P<0.001) to visual stimuli following the deviant tones, indicating behavioral distraction. ERPs revealed the mismatch negativity (MMN) elicited to deviant tones, which was larger for the duration deviant than for the frequency and intensity deviants (F(2,22)=19.43, P<0.001, epsilon =0.83), and which had different scalp distribution for all three deviant conditions (F(16,176)=2.40, P<0.05, epsilon =0.12). As the shorter duration and softer intensity deviant tones were unlikely to engage fresh neurons responding to their specific physical features, the present results indicate that a genuine change detection mechanism is involved in triggering attention switching towards sound changes, and suggest a largely distributed neural network of the auditory cortex underlying such involuntary attention switching.     

Functional characterization of mismatch negativity to a visual stimulus.

OBJECTIVE: To record mismatch negativity (MMN) to a visual stimulus fulfilling similar criteria to those of auditory MMN (A-MMN). METHODS: Twelve normal adults were instructed to simultaneously listen to a story and to pay attention to a visual target. Three windmill patterns that differed in the number of vanes (standard, deviant, or target) were used as visual stimuli, and were randomly presented. To ensure endogeneity, standard and deviant stimuli were alternated. To vary differences between frequent (standard) and infrequent (deviant) stimuli, deviants were changed by modulating the number of vanes. To examine effects of physical features of the target stimulus on changes in detection, two target conditions were used. The deviant-related component (DRC) was obtained by subtracting event-related potentials (ERPs) to the deviant stimulus from those to the standard stimulus. RESULTS: Seven subjects completed all phases of the experiment. Behavioral performances indicated that subjects' attention was directed by auditory context and identification of the target stimulus. Visual DRC appeared 150-300 ms after stimulus onset, and consisted of an early (DRN1) and a late (DRN2) component. Magnitude of deviancy from standard stimulus significantly influenced latency of DRN2 but not its magnitude, while changes in target stimulus affected latencies of both DRN1 and DRN2. CONCLUSIONS: Our DRCs satisfied criteria for A-MMN. In contrast to A-MMN, only latency of the DRC was associated with visual sensory discrimination and attentional reorienting. SIGNIFICANCE: It is possible to record valid MMN to a visual stimulus, which allows the study of preattentive visual information processing.     

Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study.

This study investigated cross-modal interactions in spatial attention by means of recording event-related brain potentials (ERPs). Noise bursts and light flashes were presented in random order to both left and right field locations separated by 60 degrees in free-field. One group of subjects was instructed to attend selectively to the noise bursts (attend-auditory group), and a second group attended only to the flashes (attend-visual group). On different runs attention was directed to either the right or left field stimuli of the designated modality. In the attend-auditory group, noise bursts at the attended location elicited a broad, biphasic negativity (Nd) beginning at 70 ms. The cross-modal spatial attention effect on the auditory ERPs in the attend-visual group was very similar in morphology, but the Nd was reduced in amplitude relative to the intra-modal effect. In the attend-visual group, flashes at the attended location elicited enhanced early (100-200 ms) and late (200-350 ms) ERP components relative to unattended-location flashes. The cross-modal effect in the attend-auditory group included small but significant enhancements of early components of the visual ERPs. It was concluded that spatial attention has a multi-modal organization such that the processing of stimuli at attended locations is facilitated at an early, sensory level, even for stimuli of an unattended modality.     

Mismatch negativity evoked by the McGurk-MacDonald effect: a phonetic representation within short-term memory.

OBJECTIVES: The McGurk-MacDonald illusory percept is obtained by dubbing an incongruent articulatory movement on an auditory phoneme. This type of audiovisual speech perception contributes to the assessment of theories of speech perception. The mismatch negativity (MMN) reflects the detection of a deviant stimulus within the auditory short-term memory and besides an acoustic component, possesses, under certain conditions, a phonetic one. The present study assessed the existence of an MMN evoked by McGurk-MacDonald percepts elicited by audiovisual stimuli with constant auditory components. METHODS: Cortical evoked potentials were recorded using the oddball paradigm on 8 adults in 3 experimental conditions: auditory alone, visual alone and audiovisual stimulation. Obtaining illusory percepts was confirmed in an additional psychophysical condition. RESULTS: The auditory deviant syllables and the audiovisual incongruent syllables elicited a significant MMN at Fz. In the visual condition, no negativity was observed either at Fz, or at O(z). CONCLUSIONS: An MMN can be evoked by visual articulatory deviants, provided they are presented in a suitable auditory context leading to a phonetically significant interaction. The recording of an MMN elicited by illusory McGurk percepts suggests that audiovisual integration mechanisms in speech take place rather early during the perceptual processes.     

Abnormal mismatch negativity in women with sexual assault-related posttraumatic stress disorder.

BACKGROUND: Disturbances in sensory processing have been hypothesized in individuals with posttraumatic stress disorder (PTSD). The authors investigated this possibility by using mismatch negativity (MMN), an event-related potential (ERP) that reflects the operation of a preconscious cortical detector of stimulus change. METHODS: Thirteen medication-free women with sexual assault-related PTSD were compared with 16 age-matched, healthy comparison women without PTSD. ERPs were elicited by regularly presented "standard" auditory stimuli and by infrequently occurring "deviant" auditory stimuli, which differed slightly in frequency. The MMN was identified in the subtraction waveforms as the difference between ERPs elicited by the deviant and standard stimuli. Group comparisons of P50, N1, P2, and N2 to the standard and to the deviant stimuli, and of the MMN in the subtraction waveform were performed. RESULTS: The amplitude of the MMN was significantly greater in the PTSD compared to the non-PTSD women. MMN was significantly correlated with the total Mississippi PTSD Symptom Scale score in the PTSD group. No significant group differences were noted in P50, N1, or P2 responding. Significant group differences in N2 were due to the increased MMN in PTSD subjects. CONCLUSIONS: The data provide evidence for abnormalities in preconscious auditory sensory memory in PTSD, whereas earlier studies have reported abnormalities in conscious processing. These data suggest an increased sensitivity to stimulus changes in PTSD and implicate the auditory cortex in the pathophysiology of the disorder.     

The mismatch negativity to frequency deviant stimuli during natural sleep

Eight subjects spent a single night in the sleep laboratory. Event-related potentials (ERPs) were recorded during the presentation of two auditory ‘oddball’ stimulus conditions in which tonal frequency was manipulated. In the first condition, 1000 Hz ‘standard’ and 2000 Hz ‘deviant’ tones were presented. In the second condition, the deviant tone was reduced to 1050 Hz. In both conditions, deviant probability was 0.2. Stimuli were presented every 600 ms during wakefulness and stages 2, 4, and REM of sleep. A distinctive N1 wave was visible in both stimulus conditions when the subject was awake. The deviant stimuli elicited a ‘mismatch negativity’ (MMN) that inverted in polarity at the mastoid. In REM sleep, an N1 and a MMN were also elicited in both conditions. In the large deviance condition, the MMN had a slightly attenuated amplitude and was shorter in duration while in the small deviant condition, its peak latency was unusually early. Neither the N1 nor the MMN could be recorded in non-REM sleep.     

Fast vigilance decrement in closed head injury patients as reflected by the mismatch negativity (MMN)

Event-related potentials (ERPs) were measured from 24 chronic closed head injury (CHI) patients and 18 age- and education-matched controls. The oddball paradigm was applied while subjects were watching a silent movie. The standard (p=0.8) sound of 75 ms duration had a basic frequency of 500 Hz with harmonic partials of 1000 Hz and 1500 Hz, whereas these frequencies for the pitch deviant were each 10% higher. The frequencies of the duration deviant matched with those of the standard but was 25 ms in duration. The MMN (mismatch negativity), generated by the brain's automatic auditory change-detector mechanism, was elicited by both deviants. No significant differences in the MMN latency or amplitude for either pitch or duration deviants were found between the groups. However, the MMN amplitude for the pitch deviant decreased in the patient group during the experiment considerably faster than in controls, suggesting a faster vigilance decrement in the patients.     

Independent component analysis on the mismatch negativity in an uninterrupted sound paradigm

We compared the efficiency of the independent component analysis (ICA) decomposition procedure against the difference wave (DW) and optimal digital filtering (ODF) procedures in the analysis of the mismatch negativity (MMN). The comparison was made in a group of 54 children aged 8-16 years. The MMN was elicited in a passive oddball protocol presenting uninterrupted auditory stimulation consisting of two frequent alternating tones (600 and 800 Hz) of 100 ms duration each. Infrequently, one of the 600 Hz tones was shortened to 50 or 30 ms. The event related potentials (ERPs) were decomposed into the MMN-like and non-MMN-like independent components (ICs) through the FastICA algorithm. The ICA decomposition procedure extracted a cleaner MMN compared to the ODF or DW procedures. It extracted the MMN, whose characteristics concurred with the substantial number of publications demonstrating a significantly larger peak amplitude and shorter latency of the MMN in response to the more deviant stimulus (30 ms) compared to the less deviant stimulus (50 ms). The MMN to these two deviant stimuli did not differ in the peak amplitude or latency when it was extracted through the other two procedures. The ICA decomposition and ODF procedures, similarly, significantly improved the single trial signal-to-noise ratio (SNR) of the MMN compared to the DW procedure. Due to this improvement, the proposed ICA decomposition procedure might allow shortening of the recording session and could be used to study the MMN in paradigms similar to this with small modifications.     

Visual mismatch negativity for changes in orientation – a sensory memory‐dependent response

It remains unclear whether the mismatch negativity of event‐related potentials (ERPs) in vision resembles its auditory counterpart in terms of memory relatedness. We recorded ERPs to visual bars in adult humans engaged in an auditory task. In one condition, a bar (‘standard’) repeated at 400‐ or 1100‐ms non‐stimulated intervals was rarely (P = 0.1) replaced by another bar of a different orientation (‘deviant’). In the other condition (400‐ms intervals), the occurrences of the standards were replaced by 10 (P = 0.1 each) bars of different orientations, including that of the deviant (‘control‐deviant’). Deviants shifted ERPs towards negative polarity relative to standards in occipital electrodes and towards positive polarity in frontal electrodes at 185–205 ms post‐stimulus but only when 400‐ms non‐stimulated intervals were applied. Furthermore, the shift existed even relative to ERPs to control‐deviants. The findings suggest that, as in audition, vision supports the detection of voluntarily unattended changes per se within the constraints of sensory memory. The findings also pave the way for the future exploration of both intact and impaired memory‐based visual processing and memory capacity.     

Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity

We used event-related potentials (ERPs) to evaluate the role of attention in the integration of visual and auditory features of multisensory objects. This was done by contrasting the ERPs to multisensory stimuli (AV) to the sum of the ERPs to the corresponding auditory-only (A) and visual-only (V) stimuli [i.e., AV vs. (A + V)]. V, A, and VA stimuli were presented in random order to the left and right hemispaces. Subjects attended to a designated side to detect infrequent target stimuli in either modality there. The focus of this report is on the ERPs to the standard (i.e., nontarget) stimuli. We used rapid variable stimulus onset asynchronies (350-650 msec) to mitigate anticipatory activity and included "no-stim" trials to estimate and remove ERP overlap from residual anticipatory processes and from adjacent stimuli in the sequence. Spatial attention effects on the processing of the unisensory stimuli consisted of a modulation of visual P1 and N1 components (at 90-130 msec and 160-200 msec, respectively) and of the auditory N1 and processing negativity (100-200 msec). Attended versus unattended multisensory ERPs elicited a combination of these effects. Multisensory integration effects consisted of an initial frontal positivity around 100 msec that was larger for attended stimuli. This was followed by three phases of centro-medially distributed effects of integration and/or attention beginning at around 160 msec, and peaking at 190 (scalp positivity), 250 (negativity), and 300-500 msec (positivity) after stimulus onset. These integration effects were larger in amplitude for attended than for unattended stimuli, providing neural evidence that attention can modulate multisensory-integration processes at multiple stages.     

Effects of spatial separation and stimulus probability on the event-related potentials elicited by occasional changes in sound location

The ability to extract information about the spatial location of sounds plays an important role in auditory scene analysis. The present study examined the effects of spatial separation and stimulus probability on auditory event-related potentials (ERPs) to changes in sound location. In Experiment 1, we found that difference waves between ERPs elicited by standard and deviant stimuli showed a biphasic negative-positive response peaking around 126 and 226 ms after deviant onset. The amplitude of both responses increased with decreasing deviant stimulus probability, and increasing stimulus deviance. When the same stimuli were presented with equal probability for all locations (Experiment 2), there were no significant differences in the ERP amplitude and latency. These results suggest that the data reported in Experiment 1 are the result of contextual changes, rather than changes in simple acoustic features. Brain electrical source analyses are consistent with generators located in auditory cortices posterior to Heschel's gyrus. Although occasional changes in sound location elicit earlier peaks than the mismatch negativity (MMN) response reported for other types of deviation, their topographical distribution and behavior are consistent with MMN. The early latency of MMN for changes in sound location is interpreted in the context of an early-warning system to alert the organism to new sound sources in the environment.     

Involuntary attention in children as a function of sound source location: evidence from event-related potentials.

OBJECTIVES: The present study addressed the question of whether location of the auditory stimulation source affects an involuntary attention triggering to the deviant sounds in a passive oddball paradigm in 8-10-year-old children. METHODS: Using free-field stimulation two late event-related potentials components were examined: the mismatch negativity (MMN), indexing preconscious sound change detection and the P3a, indexing involuntary attention switch. Data were registered to frequency changes in sounds of different complexities in two experimental conditions. In the 'in-front' condition, the sound sequences were presented through the loudspeakers situated in front of a participant on both sides of the video display. In the 'on-sides' condition, the sources of auditory and visual stimuli were separated by moving the loudspeakers to the sides of the participant. RESULTS: The MMN amplitude or the MMN and P3a latencies varied in neither stimulus class significantly as a function of sound location. However, significantly larger P3 amplitude was found in the 'in-front', as compared to the 'on-sides' condition. CONCLUSIONS: The present results indicate enhanced involuntary attention switching in children when unattended auditory events occur within the space attended actively for visual modality. Such study design favouring cross-modal integration can be advantageous when studying involuntary auditory attention and its impairment in children.     

Electrophysiological correlates of automatic visual change detection in school-age children

Automatic stimulus-change detection is usually investigated in the auditory modality by studying Mismatch Negativity (MMN). Although the change-detection process occurs in all sensory modalities, little is known about visual deviance detection, particularly regarding the development of this brain function throughout childhood. The aim of the present study was to examine the maturation of the electrophysiological response to unattended deviant visual stimuli in 11-year-old children. Twelve children and 12 adults were presented with a passive visual oddball paradigm using dynamic stimuli involving changes in form and motion. Visual Mismatch responses were identified over occipito-parietal sites in both groups but they displayed several differences. In adults the response clearly culminated at around 210 ms whereas in children three successive negative deflections were evidenced between 150 and 330 ms. Moreover, the main mismatch response in children was characterized by a positive component peaking over occipito-parieto-temporal regions around 450 ms after deviant stimulus onset. The findings showed that the organization of the vMMN response is not mature in 11-year-old children and that a longer time is still necessary to process simple visual deviancy at this late stage of child development.     

Effects of sleep onset on the mismatch negativity (MMN) to frequency deviants using a rapid rate of presentation

This study examined the effects of sleep onset-the transition from a waking, conscious state to one of sleep and unconsciousness-on the mismatch negativity (MMN) following frequency deviants when a rapid rate of stimulus presentation is employed. The MMN is thought to reflect a brief-lasting sensory memory. Rapid rates of stimulus presentation should guard the sensory memory from fading. A 1,000 Hz standard stimulus was presented every 150 ms. At random, on 6.6% of the trials, the standard was changed to either a large 2,000 or a small 1,100 Hz deviant. During alert wakefulness (when subject ignored the stimuli and read a book), the large deviant elicited a larger deviant related negativity (DRN) than did the small deviant. This negativity may be a composite of both N1 and MMN activity while that following the small deviant is probably a 'true' MMN. The large deviant continued to elicit a DRN in relaxed wakefulness (eyes closed) and Stages 1 and 2 of sleep, although it was much reduced in amplitude. A significant MMN was recorded for the small deviant only in alert wakefulness. The failure to observe an MMN to small deviance and the attenuation of the DRN to large deviance at sleep onset therefore is probably not due to a decay of sensory memory. It is more likely that cortical encoding of both the standard and deviant is weakened during sleep onset because of prior thalamic inhibition of sensory input.     

Event-related potential indices of auditory vowel processing in 3-year-old children

Objective: To determine the ERP characteristics and ERP indices of central speech sound encoding and discrimination in young children.Methods: Auditory sensory event-related potentials (ERPs) and the ERP index of auditory sensory discrimination (the mismatch negativity, MMN) were elicited by vowel stimuli in 3-year-old children. In an oddball paradigm, the standard stimulus was vowel /a/, one deviant stimulus was vowel /o/ (the across-category change), and the other was nasalized vowel /a/ (within-category change). In addition, the ERP changes occurring during the 14 min uninterrupted recording were examined.Results: As indexed by the sensory P1, N2, and N4 peaks, the 3-year-old children's transient neural encoding of vowels was comparable to that earlier registered in 1-year-old children but also showed vowel-specific characteristics observed in school-age children. The 3-year-old's MMN was comparable in amplitude to the school-age children's MMN and appeared to be sensitive to the across-category aspects of vowel changes. However, its latency was longer in the 3-year-olds than in school-age children. Among the sensory ERPs, only the N4 peak showed significant diminution during the experiment. The across-category change MMN diminished after 10 min of the recording, however, over the frontal areas only.Conclusions: In the 3-year-old children, the sensory processing of vowels exhibited transitional characteristics between those observed in infants and school-age children. The auditory sensory discrimination in the 3-year-olds appeared to be sensitive to the phonemic aspects of stimulus change. The frontally-predominant MMN diminution during the experiment might indicate the greater refractoriness of its frontal-lobe generators. In general, the auditory sensory ERPs show distinct maturational profiles from that of the MMN.     

Auditory processing in visual brain areas of the early blind: evidence from event-related potentials

Auditory event-related potentials (ERPs) were recorded in early blind subjects and sighted controls when they attended to stimuli delivered to a designated ear under dichotic conditions. The scalp distribution of the processing negativity (PN), the endogenous negativity elicited by attended stimuli, was in the blind posterior to that in the sighted. This suggests that posterior brain areas normally involved in vision participate in auditory selective attention in the early blind. Furthermore, occasional higher-frequency tones in the to-be-ignored ear elicited a negativity (presumably the mismatch negativity; MMN) that had a posterior scalp distribution in the blind as compared to controls. This suggests that the posterior brain areas of the blind also participate in processing of auditory stimulus changes occuring outside the focus of attention.