Magnetoencephalographic gamma-band responses to illusory triangles in humans

Electroencephalography studies have suggested that the perception of illusory figures is associated with increases in gamma-band activity putatively reflecting the formation of synchronously firing neuronal assemblies. Here we assessed magnetoencephalographic gamma-band activity, which has been shown to be topographically more focal than in electroencephalogram. In line with functional brain imaging findings, we hypothesized gamma-band activity over ventral visual stream areas. In addition, we expected that the analysis of oscillatory activity would provide information on the time courses and connectivity patterns of these activations. Following a paradigm previously assessed with electroencephalography, 16 adults were presented four types of stimuli at equal probabilities: illusory (Kanizsa) triangles, real triangles, no-triangle stimuli with rotated inducer disks, and curved illusory triangles serving as targets that subjects had to respond to. Induced oscillatory responses were compared between illusory triangles and no-triangle stimuli and between illusory and real triangles using a statistical probability mapping method. Illusory triangles were distinguished from no-triangles by increased activity at around 70 Hz over midline occipital cortex peaking at 240 ms after stimulus onset. This was followed by activations over bilateral lateral occipital areas at 430 ms. Illusory triangles differed from real triangles by increased spectral activity at 90 Hz over posterior parietal cortex between 100 and 450 ms after stimulus onset, suggesting an involvement of visual dorsal stream regions. Coherence analysis showed increased connectivity between posterior parietal and lateral occipital cortex. These findings suggest that illusory triangles are encoded in parallel by networks along the visual ventral and dorsal streams.     

Neural restoration of degraded audiovisual speech

When speech is interrupted by noise, listeners often perceptually "fill-in" the degraded signal, giving an illusion of continuity and improving intelligibility. This phenomenon involves a neural process in which the auditory cortex (AC) response to onsets and offsets of acoustic interruptions is suppressed. Since meaningful visual cues behaviorally enhance this illusory filling-in, we hypothesized that during the illusion, lip movements congruent with acoustic speech should elicit a weaker AC response to interruptions relative to static (no movements) or incongruent visual speech. AC response to interruptions was measured as the power and inter-trial phase consistency of the auditory evoked theta band (4-8 Hz) activity of the electroencephalogram (EEG) and the N1 and P2 auditory evoked potentials (AEPs). A reduction in the N1 and P2 amplitudes and in theta phase-consistency reflected the perceptual illusion at the onset and/or offset of interruptions regardless of visual condition. These results suggest that the brain engages filling-in mechanisms throughout the interruption, which repairs degraded speech lasting up to ~250 ms following the onset of the degradation. Behaviorally, participants perceived speech continuity over longer interruptions for congruent compared to incongruent or static audiovisual streams. However, this specific behavioral profile was not mirrored in the neural markers of interest. We conclude that lip-reading enhances illusory perception of degraded speech not by altering the quality of the AC response, but by delaying it during degradations so that longer interruptions can be tolerated.     

Human brain activity associated with audiovisual perception and attention

Coherent perception of objects in our environment often requires perceptual integration of auditory and visual information. Recent behavioral data suggest that audiovisual integration depends on attention. The current study investigated the neural basis of audiovisual integration using 3-Tesla functional magnetic resonance imaging (fMRI) in 12 healthy volunteers during attention to auditory or visual features, or audiovisual feature combinations of abstract stimuli (simultaneous harmonic sounds and colored circles). Audiovisual attention was found to modulate activity in the same frontal, temporal, parietal and occipital cortical regions as auditory and visual attention. In addition, attention to audiovisual feature combinations produced stronger activity in the superior temporal cortices than attention to only auditory or visual features. These modality-specific areas might be involved in attention-dependent perceptual binding of synchronous auditory and visual events into coherent audiovisual objects. Furthermore, the modality-specific temporal auditory and occipital visual cortical areas showed attention-related modulations during both auditory and visual attention tasks. This result supports the proposal that attention to stimuli in one modality can spread to encompass synchronously presented stimuli in another modality.     

Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming

An important step in perceptual processing is the integration of information from different sensory modalities into a coherent percept. It has been suggested that such crossmodal binding might be achieved by transient synchronization of neurons from different modalities in the gamma-frequency range (> 30 Hz). Here we employed a crossmodal priming paradigm, modulating the semantic congruency between visual–auditory natural object stimulus pairs, during the recording of the high density electroencephalogram (EEG). Subjects performed a semantic categorization task. Analysis of the behavioral data showed a crossmodal priming effect (facilitated auditory object recognition) in response to semantically congruent stimuli. Differences in event-related potentials (ERP) were found between 250 and 350 ms, which were localized to left middle temporal gyrus (BA 21) using a distributed linear source model. Early gamma-band activity (40–50 Hz) was increased between 120 ms and 180 ms following auditory stimulus onset for semantically congruent stimulus pairs. Source reconstruction for this gamma-band response revealed a maximal increase in left middle temporal gyrus (BA 21), an area known to be related to the processing of both complex auditory stimuli and multisensory processing. The data support the hypothesis that oscillatory activity in the gamma-band reflects crossmodal semantic-matching processes in multisensory convergence sites.     

Processing of audiovisual speech in Broca's area

We investigated cerebral processing of audiovisual speech stimuli in humans using functional magnetic resonance imaging (fMRI). Ten healthy volunteers were scanned with a 'clustered volume acquisition' paradigm at 3 T during observation of phonetically matching (e.g., visual and acoustic /y/) and conflicting (e.g., visual /a/ and acoustic /y/) audiovisual vowels. Both stimuli activated the sensory-specific auditory and visual cortices, along with the superior temporal, inferior frontal (Broca's area), premotor, and visual-parietal regions bilaterally. Phonetically conflicting vowels, contrasted with matching ones, specifically increased activity in Broca's area. Activity during phonetically matching stimuli, contrasted with conflicting ones, was not enhanced in any brain region. We suggest that the increased activity in Broca's area reflects processing of conflicting visual and acoustic phonetic inputs in partly disparate neuron populations. On the other hand, matching acoustic and visual inputs would converge on the same neurons.     

Temporal dynamics of stimulus-specific gamma-band activity components during auditory short-term memory

Recently we have demonstrated that during auditory short-term memory maintenance, gamma-band activity (GBA) components can be identified which are specific to the retained stimulus. These activations peaked in the middle of the delay phase between sample and test stimuli, and their magnitude during the final part of this period correlated with performance. However, using a constant delay duration did not allow to answer the question whether stimulus-specific GBA components represented responses to sample sounds or anticipatory activations preceding test stimuli. Here we addressed this unresolved issue by investigating the temporal dynamics of stimulus-specific GBA during two delay durations. Magnetoencephalogram was recorded in 18 adults during an auditory spatial short-term memory task involving lateralized sample stimuli presented with two different interaural time delays. Subjects had to decide whether test stimuli presented after retention phases of 800 or 1200 ms had the same lateralization as sample sounds. Statistical probability mapping served to identify oscillatory activations differentiating between the two sample sounds. We found stimulus-specific GBA components over posterior cortex peaking about 400 ms prior to the onset of test stimuli regardless of delay duration. Their magnitude correlated with task performance. In summary, stimulus-specific GBA components with a predictive power for short-term memory performance were observed in anticipation of test stimuli. They may reflect the preparatory activation of memory representations or the shifting of attention to the specific expected location of the test stimulus.     

Task- and performance-related modulation of domain-specific auditory short-term memory representations in the gamma-band

The short-term retention of information has been related to oscillatory activity in the gamma-band. In recent auditory spatial short-term memory studies we have found stimulus-specific components of parieto-occipital gamma-band activity (GBA) which might reflect the activation of local networks tuned to task-relevant stimulus features. The present magnetoencephalography study (N = 22) tested this interpretation by assessing whether the topography of stimulus-specific GBA depends on task demands. Sample sounds were characterized by both a variable interaural time delay and a variable central frequency. In separate task blocks, either the lateralization or the frequency of the same stimuli had to be maintained. Statistical probability mapping of differences in oscillatory responses to the retention of sample sounds replicated the contralateral posterior topography for GBA components distinguishing between medial and lateral sounds in the spatial memory task. In contrast, lower- and higher-frequency stimuli were accompanied by frontal GBA components in the frequency task. Memory for lateralization versus frequency selectively enhanced oscillatory activity for these posterior versus frontal components, directly demonstrating their modulation by task demands. Incorrect “non-match” responses were negatively correlated with delay-phase GBA to the relevant feature, whereas incorrect “match” responses correlated positively with GBA to the irrelevant feature. In summary, the topography of stimulus-specific GBA to identical stimuli reflected the selective representation of task-relevant features. Task performance was predicted by both enhanced stimulus-specific GBA for the task-relevant stimulus attribute and reduced gamma-band representations of the task-irrelevant stimulus feature. Stimulus-specific GBA may reflect the memory representation that is used in subsequent recognition.     

Music training leads to the development of timbre-specific gamma band activity

Oscillatory gamma band activity (GBA, 30-100 Hz) has been shown to correlate with perceptual and cognitive phenomena including feature binding, template matching, and learning and memory formation. We hypothesized that if GBA reflects highly learned perceptual template matching, we should observe its development in musicians specific to the timbre of their instrument of practice. EEG was recorded in adult professional violinists and amateur pianists as well as in 4- and 5-year-old children studying piano in the Suzuki method before they commenced music lessons and 1 year later. The adult musicians showed robust enhancement of induced (non-time-locked) GBA, specifically to their instrument of practice, with the strongest effect in professional violinists. Consistent with this result, the children receiving piano lessons exhibited increased power of induced GBA for piano tones with 1 year of training, while children not taking lessons showed no effect. In comparison to induced GBA, evoked (time-locked) gamma band activity (30-90 Hz, approximately 80 ms latency) was present only in adult groups. Evoked GBA was more pronounced in musicians than non-musicians, with synchronization equally exhibited for violin and piano tones but enhanced for these tones compared to pure tones. Evoked gamma activity may index the physical properties of a sound and is modulated by acoustical training, while induced GBA may reflect higher perceptual learning and is shaped by specific auditory experiences.     

Cortical activation to illusory shapes as measured with magnetoencephalography

Spatiotemporal patterns of cortical activation during the perceptual grouping of elements to form illusory shapes were estimated using anatomically constrained magnetoencephalography. Subjects were shown an array of Kanizsa-style figures which were either aligned to form illusory squares or misaligned so that no illusory contour or shape was perceived. Differential activity is more pronounced in the right hemisphere. After a weakly significant modulation at approximately 110 ms in the occipital pole, a prominent peak appears at approximately 155 ms in the lateral occipital cortex. Modulation then appears to spread back from this location toward the occipital pole, as well as ventrally to involve ventral occipital and temporal cortices for the next 180 ms, eventually involving ventral orbitofrontal cortex at 325 ms. The prominent lateral occipital response is consistent with fMRI studies with similar stimuli which found activation in that region as well as in V3A, V4v, V7, and V8. Furthermore, the timing of this activation, after the occipital pole but before ventral temporal, is consistent with a putative role for this region in midlevel vision. The late ventral temporal response (235 ms) is centered in the lingual and fusiform areas implicated in object identification. The V1/V2 modulation at this time may reflect top-down modulation by lateral occipitotemporal and ventral temporal areas.     

Reciprocal modulation of neuromagnetic induced gamma activity by attention in the human visual and auditory cortex

For attentional control of behavior, the brain permanently resolves a competition between the impressions supplied by different senses. Here, using a dual-modality temporal order detection task, we studied attentional modulation of oscillatory neuromagnetic activity in the human cerebral cortex. On each trial, after simultaneous exposure to visual and auditory noise, subjects were presented with an asynchronous pair of a visual and an auditory stimulus. Either of the two stimuli could occur first equally often, their order was not cued. Subjects had to determine the leading stimulus in a pair and attentively monitor it to respond upon its offset. With the attended visual or auditory stimuli, spectral power analysis revealed marked enhancements of induced gamma activity within 250 ms post-stimulus onset over the modality-specific cortices (occipital at 64 Hz, right temporal at 53 Hz). When unattended, however, the stimuli led to a significantly decreased (beneath baseline) gamma response in these cortical regions. The gamma decreases occurred at lower frequencies ( approximately 30 Hz) than did the gamma increases. An increase in the gamma power and frequency for the attended modality and their decrease for the unattended modality suggest that attentional regulation of multisensory processing involves reciprocal changes in synchronization of respective cortical networks. We assume that the gamma decrease reflects an active suppression of the task-irrelevant sensory input. This suppression occurs at lower frequencies, suggesting an involvement of larger scale cell assemblies.     

Early multisensory interactions affect the competition among multiple visual objects

In dynamic cluttered environments, audition and vision may benefit from each other in determining what deserves further attention and what does not. We investigated the underlying neural mechanisms responsible for attentional guidance by audiovisual stimuli in such an environment. Event-related potentials (ERPs) were measured during visual search through dynamic displays consisting of line elements that randomly changed orientation. Search accuracy improved when a target orientation change was synchronized with an auditory signal as compared to when the auditory signal was absent or synchronized with a distractor orientation change. The ERP data show that behavioral benefits were related to an early multisensory interaction over left parieto-occipital cortex (50-60 ms post-stimulus onset), which was followed by an early positive modulation (80-100 ms) over occipital and temporal areas contralateral to the audiovisual event, an enhanced N2pc (210-250 ms), and a contralateral negative slow wave (CNSW). The early multisensory interaction was correlated with behavioral search benefits, indicating that participants with a strong multisensory interaction benefited the most from the synchronized auditory signal. We suggest that an auditory signal enhances the neural response to a synchronized visual event, which increases the chances of selection in a multiple object environment.     

Gamma-band activity dissociates between matching and nonmatching stimulus pairs in an auditory delayed matching-to-sample task

Electro- and magnetoencephalography studies have suggested that increased gamma-band activity (GBA) is a correlate of activated neural stimulus representations. In this study, a delayed matching-to-sample paradigm for auditory spatial information was employed to investigate the role of magnetoencephalographic gamma-band activity in the differentiation between matching and nonmatching stimulus pairs. Twelve subjects made same-different judgments about the lateralization angle of pairs of filtered noise stimuli (S1 and S2) presented with 0.8-s delays. One half of the subjects had to respond to matching stimulus pairs, the other half to nonmatching stimulus pairs. Cortical oscillatory activity in the memory task was compared to a control task requiring the detection of background noise intensity changes. Memory-related GBA increases were revealed over midline parietal areas in the middle of the delay phase and during the presentation of S2 and over frontocentral areas at the end of the delay phase. This replicated previous findings. In addition, nonmatching trials were associated with increased GBA over right parietal areas in response to S2. The midline parietal GBA increase during S2 in the memory condition may have reflected the representation of S1 needed for a comparison between S1 and S2. When S1 and S2 were identical, no further representation was required. In contrast, for nonmatching pairs, a second representation was activated over right parietal areas.     

Sensory gating of auditory evoked and induced gamma band activity in intracranial recordings

Oscillatory activity in the gamma band range (30-50 Hz) and its functional relation to auditory evoked potentials (AEPs) is yet poorly understood. In the current study, we capitalized on the advantage of intracranial recordings and studied gamma band activity (GBA) in an auditory sensory gating experiment. Recordings were obtained from the lateral surface of the temporal lobe in 34 epileptic patients undergoing presurgical evaluation. Two kinds of activity were differentiated: evoked (phase locked) and induced (not phase locked) GBA. In 18 patients, an intracranial P50 was observed. At electrodes with maximal P50, evoked GBA occurred with a similar peak latency as the P50. However, the intensities of P50 and evoked GBA were only modestly correlated, suggesting that the intracranial P50 does not represent a subset of evoked GBA. The peak frequency of the intracranial evoked GBA was on average relatively low (approximately 25 Hz) and is, therefore, probably not equivalent to extracranially recorded GBA which has normally a peak frequency of approximately 40 Hz. Induced GBA was detected in 10 subjects, nearly exclusively in the region of the superior temporal lobe. The induced GBA was increased after stimulation for several hundred milliseconds and encompassed frequencies up to 200 Hz. Single-trial analysis revealed that induced GBA occurred in relatively short bursts (mostly <100 ms), indicating that the duration of the induced GBA in the averages originates from summation effects. Both types of gamma band activity showed a clear attenuation with stimulus repetition.     

Time-frequency analysis of target detection reveals an early interface between bottom-up and top-down processes in the gamma-band

The early visual gamma-band response is an oscillatory signal evoked approximately 100 ms after stimulation. While some studies have found effects of various cognitive processes on this signal, such effects could not be replicated in other studies. Accordingly, some authors have claimed that evoked gamma-band activity reflects merely sensory functions. To resolve these conflicting positions, we conducted a target detection experiment in which the feature that defined the target could be distributed over a large or a small part of the entire stimulus. Only targets covering a larger area of the entire stimulus evoked stronger gamma-band activity than standards although the over-all stimulus size was identical for all stimuli. This increase in evoked activity resulted from stronger oscillatory power and not exclusively from stronger phase-locking. In contrast, N1 and P3 amplitudes were larger for target stimuli irrespective of the distribution of the relevant stimulus feature. These results are consistent with the notion that early gamma-band activity is generated by feature-selective neural assemblies the activity of which can in fact be modulated by top-down processes. This interaction, however, may be only detectable in scalp-recorded EEG if it affects a sufficient number of neural assemblies.     

Mind's ear in a musician: where and when in the brain

The temporospatial pattern of brain activity during auditory imagery was studied using magnetoencephalography. Trained musicians were presented with visual notes and instructed to imagine the corresponding sounds. Brain activity specific to the auditory imagery task was observed, first as enhanced activity of left and right occipital areas (average onset 120-150 ms after the onset of the visual stimulus) and then spreading to the midline parietal cortex (precuneus) and to such extraoccipital areas that were not activated during the visual control condition (e.g., the left temporal auditory association cortex and the left and right premotor cortices). The latest activations, with average onset latencies of 270-400 ms clearly separate from the earliest ones, occurred in the left sensorimotor cortex and the right inferotemporal visual association cortex. These data imply a complex temporospatial activation sequence of multiple cortical areas when musicians recall firmly established audiovisual associations.     

Enhancing cognitive control through neurofeedback: A role of gamma-band activity in managing episodic retrieval

Neural synchronization has been proposed to be the underlying mechanism for exchanging and integrating anatomically distributed information and has been associated with a myriad of cognitive domains, including visual feature binding, top–down control, and long-term memory. Moreover, it seems that separate frequency bands have different functions in these cognitive processes. Here we studied whether neurofeedback training designed either to increase local gamma band activity (GBA+; 36–44 Hz), or local beta band activity (BBA+; 12–20 Hz), would have an impact on performance of behavioral tasks measuring short-term and long-term episodic binding. Our results show that GBA-enhancing neurofeedback training increased occipital GBA within sessions, and occipital and frontal GBA across sessions. Both groups showed an increase of GBA coherence between frontal and occipital areas, but the BBA+ group increased BBA coherence between these areas as well. Neurofeedback training had profound effects on behavior. First, we replicated earlier findings that enhancing GBA led to greater flexibility in handling (selectively retrieving) episodic bindings, which points to a role of GBA in top–down control of memory retrieval. Moreover, the long-term memory task revealed a double dissociation: GBA-targeted training improved recollection, whereas BBA-targeted training improved familiarity memory. We conclude that GBA is important for controlling and organizing memory traces of relational information in both short-term binding and long-term memory, while frontal–occipital coherence in the beta band may facilitate familiarity processes.     

Dynamics of gamma-band activity in human magnetoencephalogram during auditory pattern working memory

Both electrophysiological research in animals and human brain imaging studies have suggested that, similar to the visual system, separate cortical ventral "what" and dorsal "where" processing streams may also exist in the auditory domain. Recently we have shown enhanced gamma-band activity (GBA) over posterior parietal cortex belonging to the putative auditory dorsal pathway during a sound location working memory task. Using a similar methodological approach, the present study assessed whether GBA would be increased over auditory ventral stream areas during an auditory pattern memory task. Whole-head magnetoencephalogram was recorded from N = 12 subjects while they performed a working memory task requiring same-different judgments about pairs of syllables S1 and S2 presented with 0.8-s delays. S1 and S2 could differ either in voice onset time or in formant structure. This was compared with a control task involving the detection of possible spatial displacements in the background sound presented instead of S2. Under the memory condition, induced GBA was enhanced over left inferior frontal/anterior temporal regions during the delay phase and in response to S2 and over prefrontal cortex at the end of the delay period. gamma-Band coherence between left frontotemporal and prefrontal sensors was increased throughout the delay period of the memory task. In summary, the memorization of syllables was associated with synchronously oscillating networks both in frontotemporal cortex, supporting a role of these areas as parts of the putative auditory ventral stream, and in prefrontal, possible executive regions. Moreover, corticocortical connectivity was increased between these structures.     

Spatial and temporal factors during processing of audiovisual speech: a PET study

Speech perception can use not only auditory signals, but also visual information from seeing the speaker's mouth. The relative timing and relative location of auditory and visual inputs are both known to influence crossmodal integration psychologically, but previous imaging studies of audiovisual speech focused primarily on just temporal aspects. Here we used Positron Emission Tomography (PET) during audiovisual speech processing to study how temporal and spatial factors might jointly affect brain activations. In agreement with previous work, synchronous versus asynchronous audiovisual speech yielded increased activity in multisensory association areas (e.g., superior temporal sulcus [STS]), plus in some unimodal visual areas. Our orthogonal manipulation of relative stimulus position (auditory and visual stimuli presented at same location vs. opposite sides) and stimulus synchrony showed that (i) ventral occipital areas and superior temporal sulcus were unaffected by relative location; (ii) lateral and dorsal occipital areas were selectively activated for synchronous bimodal stimulation at the same external location; (iii) right inferior parietal lobule was activated for synchronous auditory and visual stimuli at different locations, that is, in the condition classically associated with the 'ventriloquism effect' (shift of perceived auditory position toward the visual location). Thus, different brain regions are involved in different aspects of audiovisual integration. While ventral areas appear more affected by audiovisual synchrony (which can influence speech identification), more dorsal areas appear to be associated with spatial multisensory interactions.     

Oscillatory MEG gamma band activity dissociates perceptual and conceptual aspects of visual object processing: a combined repetition/conceptual priming study

We used a combined repetition/conceptual priming task to investigate attenuations of induced gamma-band activity (iGBA) due to prior experience. We hypothesized that distinguishable iGBA suppression effects can be related to the processing of (a) perceptual aspects, and (b) conceptual aspects of cortical object representations. Participants were asked to perform a semantic classification task with pictures of real world objects and their semantically corresponding words, using a design that isolated distinct levels of the neural suppression effect. By means of volumetric source analysis we located stimulus domain-specific iGBA repetition suppression effects (60-90 Hz) in temporal, parietal, and occipital areas of the human cortex. In contrast, domain-unspecific iGBA repetition suppression, corresponding to conceptual priming, was restricted to left temporal brain regions. We propose that the selective involvement of left temporal areas points to the activation of conceptual representations, whereas more posterior temporal, parietal, and occipital areas probably reflect perceptual aspects of higher-order visual object processing.     

Sound alters activity in human V1 in association with illusory visual perception

When a single brief visual flash is accompanied by two auditory bleeps, it is frequently perceived incorrectly as two flashes. Here, we used high field functional MRI in humans to examine the neural basis of this multisensory perceptual illusion. We show that activity in retinotopic visual cortex is increased by the presence of concurrent auditory stimulation, irrespective of any illusory perception. However, when concurrent auditory stimulation gave rise to illusory visual perception, activity in V1 was enhanced, despite auditory and visual stimulation being unchanged. These findings confirm that responses in human V1 can be altered by sound and show that they reflect subjective perception rather than the physically present visual stimulus. Moreover, as the right superior temporal sulcus and superior colliculus were also activated by illusory visual perception, together with V1, they provide a potential neural substrate for the generation of this multisensory illusion.