Data-based functional template for sorting independent components of fMRI activity

In human brain imaging with naturalistic stimuli, hemodynamic responses are difficult to predict and thus data-driven approaches, such as independent component analysis (ICA), may be beneficial. Here we propose inter-subject correlation (ISC) maps as stimulus-sensitive functional templates for sorting the independent components (ICs) to identify the most stimulus-related networks without stimulus-dependent temporal covariates. We collected 3-T functional magnetic resonance imaging (fMRI) data during perception of continuous audiovisual speech. Ten adults viewed a video, in which speech intelligibility was varied by altering the sound level. Five ICs with strongest overlap with the ISC map comprised auditory and visual cortices, and the sixth was a left-hemisphere-dominant network (left posterior superior temporal sulcus, inferior frontal gyrus, anterior superior temporal pole, supplementary motor cortex, and right angular gyrus) that was activated stronger during soft than loud speech. Corresponding temporal-model-based analysis revealed only temporal- and parietal-lobe activations without involvement of the anterior areas. The performance of the ISC-based IC selection was confirmed with fMRI data collected during free viewing of movie. Since ISC–ICA requires no predetermined temporal models on stimulus timing, it seems feasible for fMRI studies where hemodynamic variations are difficult to model because of the complex temporal structure of the naturalistic stimulation.     

Analysis of fMRI data by blind separation of data in a tiny spatial domain into independent temporal component

Independent Component Analysis (ICA) is a promising tool for the analysis of functional magnetic resonance imaging (fMRI) time series. In these studies, mostly assumed is a spatially independent component map of fMRI data (spatial ICA). In this paper, we assume that the temporal courses of the signal and noises are independent within a Tiny spatial domain (temporal ICA). Then with fast-ICA algorithm, spatially neighboring fMRI data were blindly separated into several temporal courses and were preassumed to be formed by a signal time course and several noise time courses where the signal has the largest correlation coefficient with the reference signal. The final functional imaging was completed for the signals obtained from each voxel. Simulations showed that compared with the spatial ICA method, the new temporal ICA method is more effective than the spatial ICA in detecting weak signal in a fMRI dataset. As background noise, the simulations include simulated Gaussian noise and fMRI data without stimulation. Finally, vivo fMRI tests showed that the excited areas evoked by a visual stimuli are mainly in the region of the primary visual cortex and that evoked by auditory stimuli are mainly in the region of the primary temporal cortex.     

Data-based functional template for sorting independent components of fMRI activity

In human brain imaging with naturalistic stimuli, hemodynamic responses are difficult to predict and thus data-driven approaches, such as independent component analysis (ICA), may be beneficial. Here we propose inter-subject correlation (ISC) maps as stimulus-sensitive functional templates for sorting the independent components (ICs) to identify the most stimulus-related networks without stimulus-dependent temporal covariates. We collected 3-T functional magnetic resonance imaging (fMRI) data during perception of continuous audiovisual speech. Ten adults viewed a video, in which speech intelligibility was varied by altering the sound level. Five ICs with strongest overlap with the ISC map comprised auditory and visual cortices, and the sixth was a left-hemisphere-dominant network (left posterior superior temporal sulcus, inferior frontal gyrus, anterior superior temporal pole, supplementary motor cortex, and right angular gyrus) that was activated stronger during soft than loud speech. Corresponding temporal-model-based analysis revealed only temporal- and parietal-lobe activations without involvement of the anterior areas. The performance of the ISC-based IC selection was confirmed with fMRI data collected during free viewing of movie. Since ISC-ICA requires no predetermined temporal models on stimulus timing, it seems feasible for fMRI studies where hemodynamic variations are difficult to model because of the complex temporal structure of the naturalistic stimulation.     

Audiovisual Non-Verbal Dynamic Faces Elicit Converging fMRI and ERP Responses

In an everyday social interaction we automatically integrate another’s facial movements and vocalizations, be they linguistic or otherwise. This requires audiovisual integration of a continual barrage of sensory input—a phenomenon previously well-studied with human audiovisual speech, but not with non-verbal vocalizations. Using both fMRI and ERPs, we assessed neural activity to viewing and listening to an animated female face producing non-verbal, human vocalizations (i.e. coughing, sneezing) under audio-only (AUD), visual-only (VIS) and audiovisual (AV) stimulus conditions, alternating with Rest (R). Underadditive effects occurred in regions dominant for sensory processing, which showed AV activation greater than the dominant modality alone. Right posterior temporal and parietal regions showed an AV maximum in which AV activation was greater than either modality alone, but not greater than the sum of the unisensory conditions. Other frontal and parietal regions showed Common-activation in which AV activation was the same as one or both unisensory conditions. ERP data showed an early superadditive effect (AV > AUD + VIS, no rest), mid-range underadditive effects for auditory N140 and face-sensitive N170, and late AV maximum and common-activation effects. Based on convergence between fMRI and ERP data, we propose a mechanism where a multisensory stimulus may be signaled or facilitated as early as 60 ms and facilitated in sensory-specific regions by increasing processing speed (at N170) and efficiency (decreasing amplitude in auditory and face-sensitive cortical activation and ERPs). Finally, higher-order processes are also altered, but in a more complex fashion.     

一种基于独立成份分析的fMRI时-空模型数据处理方法

独立成份分析(ICA)是信号处理领域中斯近发展起来的一种很有应用前景的方法，而脑功能磁共振(fMRI)信号的有效分离与识别是一个正在研究和试验之中的技术领域。因此，发展基于ICA的fMRI数据处理方法具有明显的理论价值和应用前景。本文首先介绍了ICA原理，分析了现行ICA—fMRI方法采用的信号与噪声的空域分布相互独立的信号模型所存在的明显不足，然后提出了微域中的信号与噪声的时域过程相互独立的fMRI信号模型，从而建立了一种新的fMRI数据处理方法：邻域独立成份相关法。合理的fMRI实验数据处理结果验证了新方法的合理性。     

Time course of multisensory interactions during audiovisual speech perception in humans: a magnetoencephalographic study

During social interaction speech is perceived simultaneously by audition and vision. We studied interactions in the processing of auditory (A) and visual (V) speech signals in the human brain by comparing neuromagnetic responses to phonetically congruent audiovisual (AV) syllables with the arithmetic sum of responses to A and V syllables. Differences between AV and A+V responses were found bilaterally in the auditory cortices 150-200 ms and in the right superior temporal sulcus (STS) 250-600 ms after stimulus onset, showing that both sensory-specific and multisensory regions of the human temporal cortices are involved in AV speech processing. Importantly, our results suggest that AV interaction in the auditory cortex precedes that in the multisensory STS region.     

空间ICA在fMRI数据上的应用与分析

独立成分分析(ICA)技术试图将多维数据分解成若干个相互统计独立的分量。时间ICA和空间ICA都可以用于分析功能核磁共振成像(fMRI)数据。但由于fMRI数据空间维数远远大于时间维数，为计算方便，在分析fMRI数据时。则更多的使用空间ICA方法。本文在单任务激励实验中，利用ICA方法从fMRI数据中分离出若干个与任务相关的独立分量，其中包括与任务相关的恒定分量(CTR)和与任务相关的暂态分量(TTR)；通过将这些独立分量进行空间映射，得到了与任务相关的脑部激活区域。将此结果与SPM的分析比较，得到了一致的结果。在对结果的分析中，我们进一步指出了ICA方法的特点和局限性。     

Multisensory integration for orienting responses in humans requires the activation of the superior colliculus

Animal studies have shown that the superior colliculus (SC) is important for synthesising information from multiple senses into a unified map of space. Here, we tested whether the SC is a critical neural substrate for multisensory spatial integration in humans. To do so, we took advantage of neurophysiological findings revealing that the SC does not receive direct projections from short-wavelength-sensitive S cones. In a simple reaction-time task, participants responded more quickly to concurrent peripheral (extra-foveal) audiovisual (AV) stimuli than to an auditory or visual stimulus alone, a phenomenon known as the redundant target effect (RTE). We show that the nature of this RTE was dependent on the colour of the visual stimulus. When using purple short-wavelength stimuli, to which the SC is blind, RTE was simply explained by probability summation, indicating that the redundant auditory and visual channels are independent. Conversely, with red long-wavelength stimuli, visible to the SC, the RTE was related to nonlinear neural summation, which constitutes evidence of integration of different sensory information. We also demonstrate that when AV stimuli were presented at fixation, so that the spatial orienting component of the task was reduced, neural summation was possible regardless of stimulus colour. Together, these findings provide support for a pivotal role of the SC in mediating multisensory spatial integration in humans, when behaviour involves spatial orienting responses.     

Independent components of color natural scenes resemble V1 neurons in their spatial and color tuning

It has been hypothesized that mammalian sensory systems are efficient because they reduce the redundancy of natural sensory input. If correct, this theory could unify our understanding of sensory coding; here, we test its predictions for color coding in the primate primary visual cortex (V1). We apply independent component analysis (ICA) to simulated cone responses to natural scenes, obtaining a set of colored independent component (IC) filters that form a redundancy-reducing visual code. We compare IC filters with physiologically measured V1 neurons, and find great spatial similarity between IC filters and V1 simple cells. On cursory inspection, there is little chromatic similarity; however, we find that many apparent differences result from biases in the physiological measurements and ICA analysis. After correcting these biases, we find that the chromatic tuning of IC filters does indeed resemble the population of V1 neurons, supporting the redundancy-reduction hypothesis.     

Localization of Latent Epileptic Activities Using Spatio-Temporal Independent Component Analysis of fMRI Data

Summary Localizing interictal epileptic activities is a difficult problem in clinical practice. We report a novel noninvasive technique, resting functional magnetic resonance imaging (fMRI) with spatio-temporal independent component analysis (ICA), for localizing interictal epileptic activities. First, the fMRI data is separated into independent spatial patterns by spatial-ICA, and the patterns with Z-values larger than a threshold are selected as the potential spatial patterns of the epileptic activities. Second, the temporal series of the active points in the selected patterns are separated by temporal-ICA, and the component with the biggest Gaussian deviation (kurtosis) is selected as the representative of the epileptic discharge activity in a sub-region. Finally, those spatial sub-regions, which have distinct epileptic discharge activities confirmed by temporal–ICA are considered as the epileptic foci. This method was applied to fMRI data of six epileptic patients, and the results are consistent with the clinical assessment. Though more studies are required to validate this technique, the above preliminary results demonstrate the potential of using the resting fMRI with spatio-temporal ICA to detect and localize latent epileptic activities. An erratum to this article can be found at     

Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study

Using high-field (3 Tesla) functional magnetic resonance imaging (fMRI), we demonstrate that auditory and somatosensory inputs converge in a subregion of human auditory cortex along the superior temporal gyrus. Further, simultaneous stimulation in both sensory modalities resulted in activity exceeding that predicted by summing the responses to the unisensory inputs, thereby showing multisensory integration in this convergence region. Recently, intracranial recordings in macaque monkeys have shown similar auditory-somatosensory convergence in a subregion of auditory cortex directly caudomedial to primary auditory cortex (area CM). The multisensory region identified in the present investigation may be the human homologue of CM. Our finding of auditory-somatosensory convergence in early auditory cortices contributes to mounting evidence for multisensory integration early in the cortical processing hierarchy, in brain regions that were previously assumed to be unisensory.     

Multimodal Integration of fMRI and EEG Data for High Spatial and Temporal Resolution Analysis of Brain Networks

Two major non-invasive brain mapping techniques, electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), have complementary advantages with regard to their spatial and temporal resolution. We propose an approach based on the integration of EEG and fMRI, enabling the EEG temporal dynamics of information processing to be characterized within spatially well-defined fMRI large-scale networks. First, the fMRI data are decomposed into networks by means of spatial independent component analysis (sICA), and those associated with intrinsic activity and/or responding to task performance are selected using information from the related time-courses. Next, the EEG data over all sensors are averaged with respect to event timing, thus calculating event-related potentials (ERPs). The ERPs are subjected to temporal ICA (tICA), and the resulting components are localized with the weighted minimum norm (WMNLS) algorithm using the task-related fMRI networks as priors. Finally, the temporal contribution of each ERP component in the areas belonging to the fMRI large-scale networks is estimated. The proposed approach has been evaluated on visual target detection data. Our results confirm that two different components, commonly observed in EEG when presenting novel and salient stimuli, respectively, are related to the neuronal activation in large-scale networks, operating at different latencies and associated with different functional processes.     

Electrophysiological correlates of facial configuration and audio�Cvisual congruency: evidence that face processing is a visual rather than a multisensory task

Behavioural, neuroimaging and lesion studies show that face processing has a special role in human perception. The purpose of this EEG study was to explore whether auditory information influences visual face perception. We employed a 2?×?2 factorial design and presented subjects with visual stimuli that could be cartoon faces or scrambled faces where size changes of one of the components, the mouth in the face condition, was either congruent or incongruent with the amplitude modulation of a simultaneously presented auditory signal. Our data show a significant main effect for signal congruence at an ERP peak around 135?ms and a significant main effect of face configuration at around 200?ms. The timing and scalp topology of both effects corresponds well to previously reported data on the integration of non-redundant audio-visual stimuli and face-selective processing. Our analysis did not show any significant statistical interactions. This double disassociation suggests that the early component, at 135?ms, is sensitive to auditory-visual congruency but not to facial configuration and that the later component is sensitive to facial configuration but not to AV congruency. We conclude that facial configurational processing is not influenced by the congruence of simultaneous auditory signals and is independent from featural processing where we see evidence for multisensory integration.     

The claustrum/insula region integrates conceptually related sounds and pictures

The brain is able to create coherent percepts from multisensory input. This phenomenon, known as multisensory integration (MSI), is a ubiquitous feature of everyday life and has been found to be essential for a reliable interaction with the environment. Recent functional neuroimaging studies suggest that several different networks are engaged in various forms of MSI depending on the nature of information being integrated. However, little is known about the neural basis of a fundamental form of MSI in natural conditions; integration of common auditory and visual objects which are conceptually related, such as when we look at a cat and hear a meowing sound. Here we used event-related fMRI to compare the brain response to conceptually related and unrelated pairs of audio-visual stimuli denoting common objects. Our protocol was designed to preclude contamination of the results by cognitive processes additional to those needed for MSI. The results indicate that higher-order temporal and occipital areas respond to coincident sounds and pictures regardless of their semantic relationship; whereas, the right claustrum/insula region is differentially activated in association with multisensory integration of conceptually related common objects. This observation has important implications for understanding how multimodal information about common objects is represented in the brain.     

Multisensory processing and oscillatory gamma responses: effects of spatial selective attention

Here we describe an EEG study investigating the interactions between multisensory (audio-visual) integration and spatial attention, using oscillatory gamma-band responses (GBRs). The results include a comparison with previously reported event-related potential (ERP) findings from the same paradigm. Unisensory-auditory (A), unisensory-visual (V), and multisensory (AV) stimuli were presented to the left and right hemispaces while subjects attended to a designated side to detect deviant target stimuli in either sensory modality. For attended multisensory stimuli we observed larger evoked GBRs approximately 40–50 ms post-stimulus over medial-frontal brain areas compared with those same multisensory stimuli when unattended. Further analysis indicated that the integration effect and its attentional enhancement may be caused in part by a stimulus-triggered phase resetting of ongoing gamma-band responses. Interestingly, no such early interaction effects (<90 ms) could be found in the ERP waveforms, suggesting that oscillatory GBRs may be more sensitive than ERPs to these early latency attention effects. Moreover, no GBR attention effects could be found for the unisensory auditory or unisensory visual stimuli, suggesting that attention particularly affects the integrative processing of audiovisual stimuli at these early latencies.     

Integrated MEG/fMRI Model Validated Using Real Auditory Data

The main objective of this paper is to present methods and results for the estimation of parameters of our proposed integrated magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) model. We use real auditory MEG and fMRI datasets from 7 normal subjects to estimate the parameters of the model. The MEG and fMRI data were acquired at different times, but the stimulus profile was the same for both techniques. We use independent component analysis (ICA) to extract activation-related signal from the MEG data. The stimulus-correlated ICA component is used to estimate MEG parameters of the model. The temporal and spatial information of the fMRI datasets are used to estimate fMRI parameters of the model. The estimated parameters have reasonable means and standard deviations for all subjects. Goodness of fit of the real data to our model shows the possibility of using the proposed model to simulate realistic datasets for evaluation of integrated MEG/fMRI analysis methods.     

Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach

Neuroimaging techniques have provided ample evidence for multisensory integration in humans. However, it is not clear whether this integration occurs at the neuronal level or whether it reflects areal convergence without such integration. To examine this issue as regards visuo-tactile object integration we used the repetition suppression effect, also known as the fMRI-based adaptation paradigm (fMR-A). Under some assumptions, fMR-A can tag specific neuronal populations within an area and investigate their characteristics. This technique has been used extensively in unisensory studies. Here we applied it for the first time to study multisensory integration and identified a network of occipital (LOtv and calcarine sulcus), parietal (aIPS), and prefrontal (precentral sulcus and the insula) areas all showing a clear crossmodal repetition suppression effect. These results provide a crucial first insight into the neuronal basis of visuo-haptic integration of objects in humans and highlight the power of using fMR-A to study multisensory integration using non-invasinve neuroimaging techniques.     

Mapping causal interregional influences with concurrent TMS–fMRI

Transcranial magnetic stimulation (TMS) produces a direct causal effect on brain activity that can now be studied by new approaches that simultaneously combine TMS with neuroimaging methods, such as functional magnetic resonance imaging (fMRI). In this review we highlight recent concurrent TMS–fMRI studies that illustrate how this novel combined technique may provide unique insights into causal interactions among brain regions in humans. We show how fMRI can detect the spatial topography of local and remote TMS effects and how these may vary with psychological factors such as task-state. Concurrent TMS–fMRI may furthermore reveal how the brain adapts to so-called virtual lesions induced by TMS, and the distributed activity changes that may underlie the behavioural consequences often observed during cortical stimulation with TMS. We argue that combining TMS with neuroimaging techniques allows a further step in understanding the physiological underpinnings of TMS, as well as the neural correlated of TMS-evoked consequences on perception and behaviour. This can provide powerful new insights about causal interactions among brain regions in both health and disease that may ultimately lead to developing more efficient protocols for basic research and therapeutic TMS applications. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Two distinct regions of secondary somatosensory cortex in the rat: topographical organization and multisensory responses

In rodents, as in other species, regions of secondary somatosensory cortex (SII) may be distinguished from primary cortex (SI) both anatomically and electrophysiologically. However, the number of rodent SII subregions, their somatotopic organization, and their function are poorly understood. The presence of multisensory responsive neurons in some areas of SII suggests that one of its roles may be in the integration of somatosensory information with information from other sensory modalities. In this study, we used auditory, somatosensory, or combined auditory/somatosensory stimuli, and high-resolution epipial-evoked potential maps of rat SII to identify the number of spatially discrete subregions, estimate their somatotopic organization, and delineate regions with multisensory response properties. Maps revealed two distinct subregions within SII, one rostral and the other caudal, which were situated lateral to the posteromedial barrel subfield. Distinct somatotopies were evident at both SII loci, and analysis of evoked responses within both areas indicated multisensory interactions. These data are consistent with the presence of classically defined rostral SII regions and provide functional evidence for a lesser known, but distinct, caudal SII area. Furthermore, evidence for multisensory interactions within SII suggests that both secondary areas may process features specifically associated with multisensory integration in parallel with unimodal processing in primary areas.