Electrophysiological evidence for the hierarchical organization of auditory change detection in the human brain

Auditory change detection has been associated with mismatch negativity (MMN), an event‐related potential (ERP) occurring at 100–250 ms after the onset of an acoustic change. Yet, single‐unit recordings in animals suggest much faster novelty‐specific responses in the auditory system. To investigate change detection in a corresponding early time range in humans, we measured the Middle Latency Response (MLR) and MMN during a controlled frequency oddball paradigm. In addition to MMN, an early effect of change detection was observed at about 40 ms after change onset reflected in an enhancement of the Nb component of the MLR. Both MMN and the Nb effect were shown to be free from confounding influences such as differences in refractoriness. This finding implies that early change detection processes exist in humans upstream of MMN generation, which supports the emerging view of a hierarchical organization of change detection expanding along multiple levels of the auditory pathway.     

Deviance Detection Based on Regularity Encoding Along the Auditory Hierarchy: Electrophysiological Evidence in Humans

Detection of changes in the acoustic environment is critical for survival, as it prevents missing potentially relevant events outside the focus of attention. In humans, deviance detection based on acoustic regularity encoding has been associated with a brain response derived from the human EEG, the mismatch negativity (MMN) auditory evoked potential, peaking at about 100–200 ms from deviance onset. By its long latency and cerebral generators, the cortical nature of both the processes of regularity encoding and deviance detection has been assumed. Yet, intracellular, extracellular, single-unit and local-field potential recordings in rats and cats have shown much earlier (circa 20–30 ms) and hierarchically lower (primary auditory cortex, medial geniculate body, inferior colliculus) deviance-related responses. Here, we review the recent evidence obtained with the complex auditory brainstem response (cABR), the middle latency response (MLR) and magnetoencephalography (MEG) demonstrating that human auditory deviance detection based on regularity encoding—rather than on refractoriness—occurs at latencies and in neural networks comparable to those revealed in animals. Specifically, encoding of simple acoustic-feature regularities and detection of corresponding deviance, such as an infrequent change in frequency or location, occur in the latency range of the MLR, in separate auditory cortical regions from those generating the MMN, and even at the level of human auditory brainstem. In contrast, violations of more complex regularities, such as those defined by the alternation of two different tones or by feature conjunctions (i.e., frequency and location) fail to elicit MLR correlates but elicit sizable MMNs. Altogether, these findings support the emerging view that deviance detection is a basic principle of the functional organization of the auditory system, and that regularity encoding and deviance detection is organized in ascending levels of complexity along the auditory pathway expanding from the brainstem up to higher-order areas of the cerebral cortex.     

Effect of deviance direction and calculation method on duration and frequency mismatch negativity (MMN)

The mismatch negativity (MMN) component of the auditory event-related potential (ERP) reflects the process of change detection in the auditory system. The present study investigated the effect of deviance direction (increment vs. decrement) and calculation method (traditional vs. same-stimulus) on the amplitude of MMN. MMN was recorded for increments and decrements in frequency and duration in 20 adults. The stimuli (standard/deviant) were 250 Hz/350 Hz (frequency MMN) and 200 ms/300 ms (duration MMN) for increment MMN and vice versa for decrement MMN. Amplitude of MMN was calculated in two ways: the traditional method (subtracting ERP to the standard from the deviant presented in the same block) and the same-stimulus method (subtracting ERP to identical stimuli presented as standard in one block and deviant in another block). We found that increments in frequency produced higher MMN amplitudes compared to decrements for both methods of calculation. For duration deviance, the decrement MMN was absent in the traditional method, while the decrement and increment MMN did not differ for the same-stimulus method. These findings suggest that the brain processes frequency increments and decrements in different ways. The results also suggest the use of same-stimulus method for the calculation of duration MMN when long duration stimuli are used.     

Cortico-cortical phase synchrony in auditory mismatch processing

Previous studies have shown a reduced MMN in patients with lesions in the temporal or frontal lobes, suggesting a temporal-frontal involvement in change detection. However, how the temporal lobe interacts with other brain areas in responding to unexpected deviant stimuli remains unclear. This study aimed to evaluate the functional connectivity between cerebral regions by measuring the phase synchrony of magnetoencephalographic (MEG) signals elicited by regular simple tones and their duration-deviants in an oddball paradigm. We measured MEG responses to deviant (1000-Hz frequency, 50-ms duration, probability of 15%) and standard (1000-Hz frequency, 100-ms duration) sounds in 10 healthy adults. By using the Morlet wavelet-based analysis, relative phase synchronization values of 4-40 Hz MEG responses at 150-300 ms after stimulus onset were calculated with respect to a reference channel from the temporal region. Phase synchronization was clearly identified between the temporal and ipsilateral frontal region in the auditory evoked responses. This temporal-frontal synchronization was significantly larger in deviants-elicited than standards-elicited activation at 4-25 Hz in the left hemisphere (p < 0.05), and at 4-8 Hz in the right hemisphere (p < 0.01). Also, temporal-temporal and temporal-parietal phase synchronies were found in deviants-evoked 4-8 Hz responses.The present results suggest an involvement of temporal-temporal, temporal-frontal, and temporal-parietal neuronal network in detecting auditory change. Phase synchronization analysis may provide a useful window to further understanding of the cerebral reactivity during the processing of auditory deviants.     

Early phase of spatial mismatch negativity is localized to a posterior “where” auditory pathway

The auditory mismatch negativity (MMN) is an event-related potential that reflects early processing of changes in acoustic stimulus features. Although the MMN has been well characterized by previous work, the number, roles, and anatomical locations of its cortical generators remain unresolved. Here, we report that the MMN elicited by occasional deviations in sound location is comprised of two temporally and anatomically distinct phases: an early phase with a generator posterior to auditory cortex and contralateral to the deviant stimulus, and a later phase with generators that are more frontal and bilaterally symmetric. The posterior location of the early-phase generator suggests the engagement of neurons within a putative “where” pathway for processing spatial auditory information.     

Auditory event-related potentials and alpha oscillations in the psychosis prodrome: Neuronal generator patterns during a novelty oddball task

Prior research suggests that event-related potentials (ERP) obtained during active and passive auditory paradigms, which have demonstrated abnormal neurocognitive function in schizophrenia, may provide helpful tools in predicting transition to psychosis. In addition to ERP measures, reduced modulations of EEG alpha, reflecting top-down control required to inhibit irrelevant information, have revealed attentional deficits in schizophrenia and its prodromal stage. Employing a three-stimulus novelty oddball task, nose-referenced 48-channel ERPs were recorded from 22 clinical high-risk (CHR) patients and 20 healthy controls detecting target tones (12% probability, 500 Hz; button press) among nontargets (76%, 350 Hz) and novel sounds (12%). After current source density (CSD) transformation of EEG epochs (− 200 to 1000 ms), event-related spectral perturbations were obtained for each site up to 30 Hz and 800 ms after stimulus onset, and simplified by unrestricted time–frequency (TF) principal components analysis (PCA). Alpha event-related desynchronization (ERD) as measured by TF factor 610–9 (spectral peak latency at 610 ms and 9 Hz; 31.9% variance) was prominent over right posterior regions for targets, and markedly reduced in CHR patients compared to controls, particularly in three patients who later developed psychosis. In contrast, low-frequency event-related synchronization (ERS) distinctly linked to novels (260–1; 16.0%; mid-frontal) and N1 sink across conditions (130–1; 3.4%; centro-temporoparietal) did not differ between groups. Analogous time-domain CSD-ERP measures (temporal PCA), consisting of N1 sink, novelty mismatch negativity (MMN), novelty vertex source, novelty P3, P3b, and frontal response negativity, were robust and closely comparable between groups. Novelty MMN at FCz was, however, absent in the three converters. In agreement with prior findings, alpha ERD and MMN may hold particular promise for predicting transition to psychosis among CHR patients.     

Distraction in a continuous-stimulation detection task

Event-related potential (ERP) correlates of distraction are usually investigated in the oddball paradigm following a discrete, trial-by-trial stimulation protocol. In this design, participants perform a discrimination task while oddball stimuli deviate in a task-irrelevant stimulus feature. In our experiment, participants detected gaps in a continuous tone while infrequent frequency glides served as distracting events.Glides preceding a gap by 150 ms delayed the response to the gap and elicited the ERP sequence of N1, probably MMN, P3a, and reorienting negativity, suggesting that these responses reflect distraction-related processes which are neither task- nor stimulation-specific.When participants watched a silent movie and the auditory stimulation was task-irrelevant, glides preceding a gap by 150 ms enhanced the amplitude of the gap-elicited N1. However, when the auditory stimulation was task-relevant, the gap-elicited N1 was attenuated. These results show that the glides drew attention away from the ongoing task, both from watching the silent movie and from detecting gaps.     

Is fast auditory change detection feature specific? An electrophysiological study in humans

Recent oddball studies showed that auditory change detection responses exist in the first 50 ms after sound onset, upstream of mismatch negativity (MMN). We examined if these early responses could be elicited by feature-specific changes, meaning changes in the value of one attribute of a stimulus, regardless of whether other attributes of the stimulus are changing or not. We used a multifeature paradigm with four types of deviants: frequency, duration, intensity, and interaural time difference. In the middle latency range, only frequency deviants led to an enhanced Nb response. All four feature changes generated significant MMNs. Our results indicate that human brain is capable of detecting a feature-specific change for frequency attributes in the middle latency. The different levels of information being encoded in two separate event-related potential time ranges support the notion of a hierarchical organization of auditory deviance detection.     

Mismatch negativity on the cone of confusion

Localization of sounds by the auditory system is based on the analysis of three sources of information: interaural level differences (ILD, caused by an attenuation of the sound as it travels to the more distant ear), interaural time differences (ITD, caused by the additional amount of time it takes for the sound to arrive at the more distant ear), and spectral cues (caused by direction-specific spectral filter properties of the pinnae). Although in a number of psychophysiological studies cortical processes of ITD and ILD analysis were investigated, there is hitherto no evidence on the cortical processing of spectral cues for sound localization. The objective of the present experiment was to test whether it is possible to observe electrophysiological correlates of sound localization based on spectral cues. In an auditory oddball experiment, 80 ms of broadband noise from varying free field locations were presented to inattentive participants. Mismatch negativities (MMNs) were observed for pairs of standards and location deviants located symmetrically with respect to the interaural axis. As interaural time and level differences are identical for such pairs of sounds, the observed MMNs most likely reflect cognitive processes of sound localization utilizing the spectral filter properties of the pinnae. MMN latencies suggest that sound localization based on spectral cues is slower than ITD- or ILD-based localization.     

The temporal window of integration in elderly and young adults

Integration of information across time is an essential part of auditory processing. Evidence from a variety of experiments support the notion of an approximately 200-ms long time window following the onset of a sound, during which a unitary sound representation is formed (the temporal window of integration, TWI). The temporal resolution in the auditory system is assumed to decrease with aging suggesting that the duration of the TWI may be longer in elderly than young adults. The TWI duration was assessed in young and elderly adults using the oddball paradigm in which a regular auditory event (standard) is occasionally exchanged for a different event (deviant). Previous studies showed that when the stimulus onset asynchrony (SOA) exceeds the duration of the TWI, two successive deviations occurring infrequently in a repetitive sound sequence elicit two separate mismatch negativity (MMN) event-related brain potentials. However, only one MMN is elicited when the SOA is shorter than the TWI. Experiment 1 tested MMN elicitation for the second of two successive deviant sounds as a function of the SOA. Experiment 2 used the sound omission paradigm, in which MMN is only elicited by omissions when the SOA is shorter than the TWI. Again, MMN elicitation was tested by infrequent tone omissions as a function of the SOA. Results showed no significant differences between elderly and younger participants as a function of SOA. This suggests that the duration of the TWI is approximately between 200 and 250 ms in both groups of subjects. On the other hand, the lower MMN amplitudes elicited by frequency deviation in the elderly compared with the younger participants suggest that the specificity of frequency representation deteriorates with aging.     

The utility of brief,spectrally rich,dynamic sounds in the passive oddball paradigm

Experiments investigating auditory processing often utilize spectrally rich, dynamic stimuli to simulate an ecologically valid auditory environment in the laboratory. Often, however, these stimuli do not allow for a strict control of the timing of auditory sensory information which may be distributed over the whole duration of a given sound. In the present study, brief (20 ms long), dynamic, spectrally rich sounds were presented in the context of a passive oddball paradigm to young adults. The short duration made certain that the sensory information was delivered entirely within a 20 ms interval. Two sounds were presented as standards (45–45% probability), other two as deviants (5–5% probability) in random sequences, with a stimulus onset asynchrony (SOA) of 1500 ms. Deviants elicited the mismatch negativity and late difference negativity (LDN) event-related potential components. No N1-effect was produced by deviants, which suggests that the acoustic energy is spread over many different features due to the dynamic spectral properties, which, combined with the brief duration, causes insignificant refractoriness-effects at the present SOA. These results support the usefulness of brief natural sounds in auditory research. The elicitation of LDN in an adult group was an unexpected finding, because LDN is mostly found in children, but not in adults. This result might indicate that LDN elicitation depends on stimulation complexity: stimulus settings in which an LDN is registered in children but not in adults may be perceived as more complex by children than by adults.     

Deviance detection in congruent audiovisual speech: Evidence for implicit integrated audiovisual memory representations

Detection of deviant speech syllables embedded in continuous noise was investigated in an oddball paradigm. Behavioral results showed improvement of detecting and identifying the syllables when congruent visual speech accompanied the utterances. A centrally maximal negative ERP difference wave peaking at approximately 290 ms post-stimulus was elicited by audiovisual but not by auditory- or visual-only task-irrelevant deviant syllables. Whereas the circumstances of the elicitation of this ERP response are similar to those of the mismatch negativity component (MMN and its visual counterpart, vMMN), its scalp distribution differs from that of both unimodal MMNs. Elicitation of an MMN-like ERP response (termed here as the audiovisual MMN: avMMN) suggests that detection of the audiovisual deviants involved integrated audiovisual memory representations. The pattern of behavioral and ERP results suggest that the formation of such cross-modal memory representation does not require voluntary operations and may even proceed for stimuli outside the focus of attention.     

Differential processing of terminal tone parts within structured and non-structured tones

Recent studies utilizing the mismatch negativity (MMN) event-related potential (ERP) revealed that when a repetitive sequence of sinusoidal tones is presented, the occasional insertion of a short deviation into some of the tones leads to the elicitation of an MMN only if it occurs during the initial 300 ms, but not beyond. In contrast, deviations occurring in speech sounds elicit MMN even beyond 300 ms. We conducted two experiments to resolve this conflict. We hypothesised that an additional transient within an otherwise unstructured tone may overcome this limitation. First, we tested for MMN to a deviance at the terminal part of a 650 ms tone which did or did not contain a gap. Only when the tone included the gap, MMN was obtained. Second, we compared the gap condition with two noise conditions, in which the gap was replaced by modulated white noise. The noise conditions differed with respect to the saliency of the perceived interruption of the tone. In all three conditions, MMN was elicited. These results demonstrate that structuring a sinusoidal tone by a gap or a noise interval is sufficient to regain MMN. It is suggested that the introduction of an additional transient triggers a new integration window overcoming the temporal constraints of automatic tone representation. This resolves the seeming contradiction between MMN studies using tonal and speech sounds.     

Detecting violations of temporal regularities in waking and sleeping two-month-old infants

Correctly processing rapid sequences of sounds is essential for developmental milestones, such as language acquisition. We investigated the sensitivity of two-month-old infants to violations of a temporal regularity, by recording event-related brain potentials (ERPs) in an auditory oddball paradigm from 36 waking and 40 sleeping infants. Standard tones were presented at a regular 300 ms inter-stimulus interval (ISI). One deviant, otherwise identical to the standard, was preceded by a 100 ms ISI. Two other deviants, presented with the standard ISI, differed from the standard in their spectral makeup. We found significant differences between ERP responses elicited by the standard and each of the deviant sounds. The results suggest that the ability to extract both temporal and spectral regularities from a sound sequence is already functional within the first few months of life. The scalp distribution of all three deviant-stimulus responses was influenced by the infants’ state of alertness.     

Emotionally negative pictures increase attention to a subsequent auditory stimulus

Emotionally negative stimuli serve as a mechanism of biological preparedness to enhance attention. We hypothesized that emotionally negative stimuli would also serve as motivational priming to increase attention resources for subsequent stimuli. To that end, we tested 11 participants in a dual sensory modality task, wherein emotionally negative pictures were contrasted with emotionally neutral pictures and each picture was followed 600 ms later by a tone in an auditory oddball paradigm. Each trial began with a picture displayed for 200 ms; half of the trials began with an emotionally negative picture and half of the trials began with an emotionally neutral picture; 600 ms following picture presentation, the participants heard either an oddball tone or a standard tone. At the end of each trial (picture followed by tone), the participants categorized, with a button press, the picture and tone combination. As expected, and consistent with previous studies, we found an enhanced visual late positive potential (latency range = 300-700 ms) to the negative picture stimuli. We further found that compared to neutral pictures, negative pictures resulted in early attention and orienting effects to subsequent tones (measured through an enhanced N1 and N2) and sustained attention effects only to the subsequent oddball tones (measured through late processing negativity, latency range = 400-700 ms). Number pad responses to both the picture and tone category showed the shortest response latencies and greatest percentage of correct picture-tone categorization on the negative picture followed by oddball tone trials. Consistent with previous work on natural selective attention, our results support the idea that emotional stimuli can alter attention resource allocation. This finding has broad implications for human attention and performance as it specifically shows the conditions in which an emotionally negative stimulus can result in extended stimulus evaluation.     

Automatic integration of auditory and visual information is not simultaneous in Chinese

ERP studies have revealed that for alphabetic languages letter/speech-sound integration develops with brain maturation and reading instruction over a relatively long period of time. Experienced adult readers associate letters and speech sounds automatically, as indexed by enhanced mismatch negativity (MMN) to simultaneously presented stimuli, but reveal attenuated MMN when the speech sound stimuli are presented with a delay. Chinese as a logographic language differs substantially from alphabetic languages and therefore integration processes might be characterized by unique timing mechanisms. In the present study, MMN was used to investigate the timing and automaticity of association between characters and lexical tones in adult native Chinese speakers. A character was presented simultaneously with a lexical tone, or with either a 100 ms stimulus onset asynchrony (SOA) or a 200 ms SOA in separate conditions. MMN was enhanced when the character and the lexical tone were presented with 100 ms SOA, while no significant MMN enhancement was observed with simultaneous presentation or with 200 ms SOA. These results suggest that the automatic association of characters and lexical tones in experienced Chinese adult readers requires more processing time than for alphabetic languages. These results highlight critical differences between fundamental reading processes across different writing systems. The neural differences between alphabetic and logographic languages for letter/character and speech-sound/tone integration need to be taken into consideration when considering past and future research on reading processes in these languages and especially for investigations of reading disorders, such as developmental dyslexia.     

Effects of inter-stimulus interval (ISI) duration on the N1 and P2 components of the auditory event-related potential

The N1 and P2 components of the event-related potential are relevant markers in the processing of auditory information, indicating the presence of several acoustic phenomena, such as pure tones or speech sounds. In addition, the expression of these components seems to be sensitive to diverse experimental variations. The main purpose of the present investigation was to explore the role of inter-stimulus interval (ISI) on the N1 and P2 responses, considering two widely used experimental paradigms: a single tone task (1000 Hz sound repeated in a fixed rhythm) and an auditory oddball (80% of the stimuli were equal to the sound used in the single tone and the remaining were a 1500 Hz tone). Both tasks had four different conditions, and each one tested a fixed value of ISI (600, 1000, 3000, or 6000 ms). A sample of 22 participants performed these tasks, while an EEG was recorded, in order to examine the maximum amplitude of the N1 and P2 components. Analysis of the stimuli in the single tone task and the frequent tones in the oddball task revealed a similar outcome for both tasks and for both components: N1 and P2 amplitudes were enhanced in conditions with longer ISIs regardless of task. This response pattern emphasizes the dependence of both the N1 and P2 components on the ISI, especially in a scenario of repetitive and regular stimulation. The absence of task effects suggests that the ISI effect reported may depend on refractory mechanisms rather than being due to habituation effects.     

Regularity encoding and deviance detection of frequency modulated sweeps: Human middle‐ and long‐latency auditory evoked potentials

Fast encoding of frequency modulated (FM) sweeps is crucial for communication. In humans, FM sweeps deviating from the acoustic regularity elicit the mismatch negativity (MMN) evoked potential. Yet, direction sensitivity to FM sweeps is found in animals' primary auditory cortex, upstream of MMN sources found in humans. Here, we were interested in whether direction deviants of complex FM sweeps modulated brain responses earlier than MMN. We used a controlled oddball paradigm, and measured the middle latency responses (MLRs) and the MMN. Our results showed a repetition enhancement to the standards at the Pa component of the MLR and a genuine MMN in the later response range. These results show that, early in the cortical hierarchy, the system is sensitive to the physical characteristics of the repetitive stimuli, but a higher‐order mechanism is needed to detect violations of the acoustic regularity.     

Theta oscillation during auditory change detection: An MEG study

To study the phase and power characteristics of brain oscillations during the preattentive detection of auditory deviance, we recorded magnetoencephalographic responses in 10 healthy subjects with an oddball paradigm. As the subject was watching a silent movie, standard tones (1000-Hz frequency, 100-ms duration) and their duration deviants (50-ms duration, probability of 15%) were randomly delivered binaurally. In addition to localizing the magnetic counterpart of mismatch negativity (MMNm) with equivalent current dipole modeling, we also measured the phase-locking value (PLV) and power change of the oscillatory responses to standard and deviant stimuli by employing the Morlet wavelet-based analysis. The MMNm signals peaking at 150-200 ms after the onset of deviant were localized in bilateral temporal regions with larger amplitudes in the right hemisphere. Then 50 ms after the onset of either standard or deviant stimuli, we observed an increase of PLV and power of theta and alpha oscillations in bilateral temporal regions. PLVs of theta and alpha activities to deviant stimuli were significantly larger in the right than left hemisphere (P < 0.001). Compared with standard stimuli, deviants elicited a larger theta PLV (P < 0.001) at 150-300 ms and a larger theta power change (P < 0.05) at 50-300 ms for the responses in the right temporal region. In addition, a prominent theta phase-locking of deviant-elicited responses was found in the right frontal area at 110-250 ms (P < 0.01). Our current data suggest that a pronounced phase and power modulation on sound-elicited theta oscillations might characterize the change detection processing in the temporo-frontal network as reflected by the mismatch negativity.     

Does mismatch negativity show differences in reading-disabled children compared to normal children and children with attention deficit?

An auditory event-related potential (ERP) component called mismatch negativity (MMN) was examined in three groups of children (n = 63) aged 8-14 years. A control group comprised healthy children in second or sixth grade of comprehensive school (n = 21). The two clinical groups included children with reading disability (RD) (n = 21) and children with attention deficit (AD) (n = 21). MMN was elicited in a passive oddball paradigm by duration changes in a continuous sound, consisting of two alternating (600 and 800 Hz) 100 msec tones. The deviant tones were either 30 or 50 msec in duration. Both deviants elicited a clear MMN in all groups. Statistical analyses showed no systematic difference in the MMN peak latencies or amplitudes between the groups. A significant difference between the RD group and the control group was observed in the lateralization of the MMN peak amplitudes.