Reduced multisensory integration in patients with schizophrenia on a target detection task

A growing body of literature demonstrates impaired multisensory integration (MSI) in patients with schizophrenia compared to non-psychiatric individuals. One of the most basic measures of MSI is intersensory facilitation of reaction times (RTs), in which bimodal targets, with cues from two sensory modalities, are detected faster than unimodal targets. This RT speeding is generally attributed to super-additive processing of multisensory targets. In order to test whether patients with schizophrenia are impaired on this basic measure of MSI, we assessed the degree of intersensory facilitation for a sample of 20 patients compared to 20 non-psychiatric individuals using a very simple target detection task. RTs were recorded for participants to detect targets that were either unimodal (auditory alone, A; visual alone, V) or bimodal (auditory + visual, AV). RT distributions to detect bimodal targets were compared with predicted RT distributions based on the summed probability distribution of each participant's RTs to visual alone and auditory alone targets. Patients with schizophrenia showed less RT facilitation when detecting bimodal targets relative to non-psychiatric individuals, even when groups were matched for unimodal RTs. Within the schizophrenia group, RT benefit was correlated with negative symptoms, such that patients with greater negative symptoms showed the least RT facilitation (r² = 0.20, p < 0.05). Additionally, schizophrenia patients who experienced both auditory and visual hallucinations showed less multisensory benefit compared to patients who experienced only auditory hallucinations, indicating that the presence of hallucinations in two modalities may more strongly impair MSI compared to hallucinations in only one modality.     

Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion

Seeing a speaker's facial articulatory gestures powerfully affects speech perception, helping us overcome noisy acoustical environments. One particularly dramatic illustration of visual influences on speech perception is the "McGurk illusion", where dubbing an auditory phoneme onto video of an incongruent articulatory movement can often lead to illusory auditory percepts. This illusion is so strong that even in the absence of any real change in auditory stimulation, it activates the automatic auditory change-detection system, as indexed by the mismatch negativity (MMN) component of the auditory event-related potential (ERP). We investigated the putative left hemispheric dominance of McGurk-MMN using high-density ERPs in an oddball paradigm. Topographic mapping of the initial McGurk-MMN response showed a highly lateralized left hemisphere distribution, beginning at 175 ms. Subsequently, scalp activity was also observed over bilateral fronto-central scalp with a maximal amplitude at approximately 290 ms, suggesting later recruitment of right temporal cortices. Strong left hemisphere dominance was again observed during the last phase of the McGurk-MMN waveform (350-400 ms). Source analysis indicated bilateral sources in the temporal lobe just posterior to primary auditory cortex. While a single source in the right superior temporal gyrus (STG) accounted for the right hemisphere activity, two separate sources were required, one in the left transverse gyrus and the other in STG, to account for left hemisphere activity. These findings support the notion that visually driven multisensory illusory phonetic percepts produce an auditory-MMN cortical response and that left hemisphere temporal cortex plays a crucial role in this process.     

Attentional modulation of early-stage visual processing in schizophrenia

This study shows that paying attention to the color of a visual stimulus is manifested by an early endogenous scalp-positive event-related brain potential (ERP) component, referred to as "selection positivity", that emerges within the first 100 ms after stimulus onset in healthy observers. In contrast, recently ill and chronically ill schizophrenia patients as well as patients at high risk for schizophrenia all failed to show this early ERP component while attending to color. These results suggest that a relatively early stage of visual-selective processing in posterior extrastriate cortex is disrupted in schizophrenia.     

Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study

Integration of information from multiple senses is fundamental to perception and cognition, but when and where this is accomplished in the brain is not well understood. This study examined the timing and topography of cortical auditory-visual interactions using high-density event-related potentials (ERPs) during a simple reaction-time (RT) task. Visual and auditory stimuli were presented alone and simultaneously. ERPs elicited by the auditory and visual stimuli when presented alone were summed ('sum' ERP) and compared to the ERP elicited when they were presented simultaneously ('simultaneous' ERP). Divergence between the 'simultaneous' and 'sum' ERP indicated auditory-visual (AV) neural response interactions. There was a surprisingly early right parieto-occipital AV interaction, consistent with the finding of an earlier study [J. Cogn. Neurosci. 11 (1999) 473]. The timing of onset of this effect (46 ms) was essentially simultaneous with the onset of visual cortical processing, as indexed by the onset of the visual C1 component, which is thought to represent the earliest cortical visual evoked potential. The coincident timing of the early AV interaction and C1 strongly suggests that AV interactions can affect early visual sensory processing. Additional AV interactions were found within the time course of sensory processing (up to 200 ms post stimulus onset). In total, this system of AV effects over the scalp was suggestive of both activity unique to multisensory processing, and the modulation of 'unisensory' activity. RTs to the stimuli when presented simultaneously were significantly faster than when they were presented alone. This RT facilitation could not be accounted for by probability summation, as evidenced by violation of the 'race' model, providing compelling evidence that auditory-visual neural interactions give rise to this RT effect.     

The role of cingulate cortex in the detection of errors with and without awareness: a high-density electrical mapping study

Error-processing research has demonstrated that the brain uses a specialized neural network to detect errors during task performance but the brain regions necessary for conscious awareness of an error are poorly understood. In the present study we show that two well-known error-related event-related potential (ERP) components, the error-related negativity (ERN) and error positivity (Pe) have a differential relationship with awareness during performance of a manual response inhibition task optimized to examine error awareness. While the ERN was unaffected by the participants' conscious experience of errors, the Pe was only seen when participants were aware of committing an error. Source localization of these components indicated that the ERN was generated by a caudal region of the anterior cingulate cortex (ACC) while the Pe was associated with contributions from a more anterior ACC region and the posterior cingulate-precuneus. Tonic EEG measures of cortical arousal were correlated with individual rates of error awareness and showed a specific relationship with the amplitude of the Pe. The latter finding is consistent with evidence that the Pe represents a P3-like facilitation of information processing modulated by subcortical arousal systems. Our data suggest that the ACC might participate in both preconscious and conscious error detection and that cortical arousal provides a necessary setting condition for error awareness. These findings may be particularly important in the context of clinical studies in which a proper understanding of self-monitoring deficits requires an explicit measurement of error awareness.     

Early visual processing deficits in schizophrenia: impaired P1 generation revealed by high-density electrical mapping.

Integrity of early visual sensory processing in schizophrenia was assessed using the well characterized P1 and N1 components of the visual evoked potential (VEP) as our dependent measures. VEPs were recorded in response to successively less fragmented line drawings of common objects. P1 amplitudes were significantly reduced across all stimulus conditions for patients versus controls. Further, this decrement was relatively greater at parieto-occipital than occipito-temporal electrode sites. No differences in N1 amplitude were found. The finding of P1 deficits in patients, particularly over dorsal scalp, supports the view that schizophrenia is associated with impairment of early dorsal visual stream processing. On the other hand, the finding of normal N1 amplitudes in patients suggests that early stages of ventral stream processing may be relatively more intact. These results imply that the cognitive impairment seen in schizophrenia is not just due to deficits in higher order aspects of cognition but also encompasses significant deficits in early sensory processing.     

Brain mapping biomarkers of socio-emotional processing in schizophrenia

The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) initiative has formed with the expressed intent of identifying constructs and paradigms that would identify biomarkers of psychosis. The manipulation of these biomarkers would serve as targets for treatment interventions. The second phase of CNTRICS consisted of critical discussions evaluating brain mapping (functional neuroimaging and brain electrical activity) paradigms as biomarkers to measure specific constructs. Among the constructs identified in, CNTRICS I was socio-emotional processing, specifically focused on affect recognition. Here, we provide a critical appraisal of the ability of candidate socio-emotional tasks to identify putative biomarkers and recommendations for future directions in this rapidly moving research domain.     

Visual and audio emotion processing training for outpatients with schizophrenia: an integrated multisensory approach

Deficits in emotion processing (EP) represent a target of rehabilitation in schizophrenia, as they have been related to poor personal and social functioning. To date neither the relationship between these deficits and the generalised cognitive impairment, nor the involvement of specific mechanisms of perception (visual or auditory) are fully comprehended. We developed two treatments targeting EP, through visual or auditory channels, with the aim of disentangling possible differences and/or interactions between the two modalities in schizophrenia-related impairments, also taking into account the role of cognition and social functioning. Thirty five outpatients with schizophrenia were assessed for neurocognition, social functioning and EP (visual and auditory channel) and participated in either visual or auditory EP training or in an active control group. Results showed a significant improvement in EP through the specific channel trained for both groups, with an extended effect also on vocal stimuli for the visual training group. Positive correlations were found between working memory, social functioning and EP. Our findings help to shed light on the possible different involvement of perceptual channels in schizophrenia, as well as supporting previous evidence that emotion recognition may be inter-related but does not overlap with neurocognition and can be specifically trained.     

Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study

The timecourse and scalp topography of interactions between neural responses to stimuli in different visual quadrants, straddling either the vertical or horizontal meridian, were studied in 15 subjects. Visual evoked potentials (VEPs) were recorded from 64 electrodes during a simple reaction time (RT) task. VEPs to single stimuli displayed in different quadrants were summed ('sum') and compared to the VEP response from simultaneous stimulation of the same two quadrants ('pair'). These responses would be equivalent if the neural responses to the single stimuli were independent. Divergence between the 'pair' and 'sum' VEPs indicates a neural response interaction. In each visual field, interactions occurred within 72-86 ms post-stimulus over parieto-occipital brain regions. Independent of visual quadrant, RTs were faster for stimulus pairs than single stimuli. This replicates the redundant target effect (RTE) observed for bilateral stimulus pairs and generalizes the RTE to unilateral stimulus pairs. Using Miller's 'race' model inequality (Miller J. Divided attention: evidence for coactivation with redundant signals, Cognitive Psychology 1982;14:247-79), we found that probability summation could fully account for the RTE in each visual field. Although measurements from voltage waveforms replicated the observation of earlier peak P1 latencies for the 'pair' versus 'sum' comparison (Miniussi C, Girelli M, Marzi CA. Neural site of the redundant target effect: electrophysiological evidence. Journal of Cognitive Neuroscience 1998;10:216-30), this did not hold with measurements taken from second derivative (scalp current density) waveforms. Since interaction effects for bilateral stimulus pairs occurred within 86 ms and require interhemispheric transfer, transcallosal volleys must arrive within 86 ms, which is earlier than previously calculated. Interaction effects for bilateral conditions were delayed by approximately 10 ms versus unilateral conditions, consistent with current estimates of interhemispheric transmission time. Interaction effects place an upper limit on the time required for neuronal ensembles to combine inputs from different quadrants of visual space ( approximately 72 ms for unilateral and approximately 82 ms for bilateral conditions).     

Omission mismatch negativity of speech sounds reveals a functionally impaired temporal window of integration in schizophrenia

ObjectiveWe hypothesized that sensory memory associated with the temporal window of integration (TWI) would be impaired in patients with schizophrenia, an issue that had not been evaluated using omission mismatch negativity (MMN) of complex speech sounds. We aimed to assess the functional changes in auditory sensory memory associated with the TWI in patients with schizophrenia by investigating the effect of omission of complex speech stimuli on the MMN.MethodsIn total, 17 patients with schizophrenia and 15 control individuals participated in the study. The MMN in response to omission deviants of complex speech sounds was recorded, while the participants were instructed to ignore the series of speech sounds.ResultsThe MMN latency in patients with schizophrenia was significantly prolonged by deviant stimuli to omissions corresponding to the early and late parts of the temporal TWI. There were no significant group differences in the amplitude of the MMN to omissions at different time points across the TWI.ConclusionsOur results suggested that sensory tracing function in patients with schizophrenia is impaired in the early and the later half of the TWI.SignificanceWe showed that certain MMN abnormalities in patients with schizophrenia may be caused by an impaired TWI.     

Activation time course of responses to illusory contours and salient region: a high-density electrical mapping comparison

Until recently, early visual areas (V1/V2) were believed to respond mainly to illusory contours (ICs). At present, however, functional neuroimaging suggests that the human lateral occipital complex (LOC), a higher tier than V2, responds strongly to ICs and that IC-related activation in V1/V2 in fact might be driven by feedback input from the LOC. When Kanizsa-type ICs are modified by rounding the corners of the inducers and misaligning them slightly, the impression of an enclosed salient region (SR) remains, although ICs no longer are perceived. Stanley and Rubin (Stanley, D.A., Rubin, N., 2003. fMRI activation in response to illusory contours and salient regions in the human lateral occipital complex. Neuron, 37, 323-331) found that the LOC responded to SR, suggesting that the LOC subserves a rapid but crude region-based segmentation process preceding boundary completion in V1/V2. The present study compared the time course of cortical responses to ICs with those to SR using high-density (74-channel) event-related potentials (ERPs). Scalp mapping and statistical analysis indicated that shared negative modulation for ICs and SR was distributed bilaterally over the lateral occipital scalp at a latency of 70 to 180 ms. Slightly later, a weak negative modulation occurred with ICs but not SR at the occipital pole scalp from 170 to 180 ms. Dipoles for early and late modulations were fitted optimally in the LOC and occipital pole, respectively. The present results suggested that IC-related cortical activation could be separated into region-based segmentation and subsequent boundary completion.     

Neuromodulation of multisensory perception: a tDCS study of the sound-induced flash illusion

This study explores whether brain polarization could be effective in modulating multisensory audiovisual interactions in the human brain, as measured by the ‘sound-induced flash illusion’ (Shams et al., 2000). In different sessions, healthy participants performed the task while receiving anodal, cathodal, or sham tDCS (2 mA, 8 min) to the occipital, temporal, or posterior parietal cortices. We found that up- or down-regulating cortical excitability by tDCS can facilitate or reduce audiovisual illusions, depending on the current polarity, the targeted area, and the illusory percept. Specifically, the perceptual ‘fission’ of a single flash, due to multiple beeps, was increased after anodal tDCS of the temporal cortex, and decreased after anodal stimulation of the occipital cortex. A reversal of such effects was induced by cathodal tDCS. Conversely, the perceptual ‘fusion’ of multiple flashes due to a single beep was unaffected by tDCS.This evidence adds novel clues on the cortical substrate of the generation of the sound-flash illusion, and opens new attractive possibilities for modulating multisensory perception in humans: tDCS appears to be an effective tool to modulate the conscious visual experience associated with multisensory interactions, by noninvasively shifting cortical excitability within occipital or temporal areas.     

Neurocomputational approaches to modelling multisensory integration in the brain: A review

The Brain’s ability to integrate information from different modalities (multisensory integration) is fundamental for accurate sensory experience and efficient interaction with the environment: it enhances detection of external stimuli, disambiguates conflict situations, speeds up responsiveness, facilitates processes of memory retrieval and object recognition. Multisensory integration operates at several brain levels: in subcortical structures (especially the Superior Colliculus), in higher-level associative cortices (e.g., posterior parietal regions), and even in early cortical areas (such as primary cortices) traditionally considered to be purely unisensory. Because of complex non-linear mechanisms of brain integrative phenomena, a key tool for their understanding is represented by neurocomputational models. This review examines different modelling principles and architectures, distinguishing the models on the basis of their aims: (i) Bayesian models based on probabilities and realizing optimal estimator of external cues; (ii) biologically inspired models of multisensory integration in the Superior Colliculus and in the Cortex, both at level of single neuron and network of neurons, with emphasis on physiological mechanisms and architectural schemes; among the latter, some models exhibit synaptic plasticity and reproduce development of integrative capabilities via Hebbian-learning rules or self-organizing maps; (iii) models of semantic memory that implement object meaning as a fusion between sensory–motor features (embodied cognition). This overview paves the way to future challenges, such as reconciling neurophysiological and Bayesian models into a unifying theory, and stimulates upcoming research in both theoretical and applicative domains.     

精神分裂症患者认知异常的事件相关电位研究

目的探讨精神分裂症患者失匹配负波（mismatch negativity,MMN）的特征及药物对MMN的影响。方法对31例精神分裂症患者（研究组）和30名健康被试（对照组）行MMN的检测,并比较首发未用药组与已用药组MMN潜伏期、波幅的差异。结果（1）研究组和对照组Fz,Cz,Pz点均可见较明显MMN波形,精神分裂症组波形欠规则。（2）研究组Fz、Cz、Pz点的MMN潜伏期较对照组差异无统计学意义（t=0．74,0．09,0．63;P〉0．05）;三点波幅降低,较对照组比较差异具有统计学意义（t=5．04,5．83,4．47;P〈0．05）。（3）首发未用药病患组12例和已接受药物治疗的病患组19例,在Fz、Cz、Pz点MMN潜伏期、波幅,差异均未见统计学差异（t=0．59,1．17,1．73,1．88,1．12,0．90;P〉0．05）。结论精神分裂症患者认知功能受损,且MMN是一个稳定的素质性指标,不受药物影响。     

Audiovisual integration of speech is disturbed in schizophrenia: An fMRI study

Speech perception is an essential part of social interaction. Visual information (lip movements, facial expression) may supplement auditory information in particular under inadvertent listening situations. Schizophrenia patients have been shown to have a deficit in integrating articulatory motions with the auditory speech input. The goal of this study was to investigate the neural basis of this deficit in audiovisual speech processing in schizophrenia patients by using fMRI. Disyllabic nouns were presented in congruent (audio matches visual information) and incongruent conditions in a slow event related fMRI design. Schizophrenia patients (n = 15) were compared to age and gender matched control participants. The statistical examination was conducted by analysis of variance with main factors: audiovisual congruency and group membership. The patients' brain activity differed from the control group as evidenced by congruency by group interaction effects. The pertinent brain sites were located predominantly in the right hemisphere and comprised the pars opercularis, middle frontal sulcus, and superior temporal gyrus. In addition, we observed interactions bilaterally in the fusiform gyrus and the nucleus accumbens. We suggest that schizophrenia patients' deficits in audiovisual integration during speech perception are due to a dysfunction of the speech motor system in the right hemisphere. Furthermore the results can be also seen as a reflection of reduced lateralization of language functions to the left hemisphere in schizophrenia.     

Brightening prospects for early cortical coding of perceived luminance: a high-density electrical mapping study

Establishing the computational rules and neural substrates of brightness coding is a topic of both historical and contemporary interest. Two major classes of explanations for brightness illusions, such as brightness contrast, can be traced to Hering and Helmholtz. Hering's legacy is a low-level account in which brightness contrast results from obligatory lateral inhibitory interactions occurring at some level(s) in the visual system. Helmholtz offered a high-level account, positing a causal role for factors such as perceptual grouping, inferred illumination, and the extraction of surface properties such as orientation and reflectance. The tension between these theoretical viewpoints persists unabated to date. Intracranial electrophysiological recordings have revealed that brightness is represented in the firing rates of striate neurons, a fact consistent with low-level explanations. However, since the time-course of brightness-related responses relative to the onset of striate activity is undisclosed, it remains possible that striate activation might be temporally and causally secondary to higher-level computational processes. Knowledge of the timing of brightness-related neural activity is thus crucial to both constrain and adjudicate between these competing theories. We utilize high-density electrophysiological recording and a tachistoscopic brightness discrimination task to measure the time-course and scalp topography of brightness-related electrical potentials in human observers. Brightness perception is correlated with electrical activity at the earliest stages of visual cortical processing. These findings are interpreted to support Hering's low-level account of brightness for White's effect, and the results are discussed in the context of current theories of brightness perception.     

Cannabis and schizophrenia: results of a follow-up study

A total of 39 schizophrenic patients with a history of current cannabis abuse at index admission was compared with a control group of schizophrenics without substance abuse matched for age, gender, and year of admission. At follow-up after 68.7 ± 28.3 months, 27/ 39 cases and 26/39 controls could be investigated. 8/27 cases (30%) had continued cannabis abuse, 6/27 (22%) had become alcohol abusers. Only one patient of the control group had started abusing alcohol. Patients with previous cannabis abuse had significantly more rehospitalizations, tended to worse psychosocial functioning, and scored significantly higher on the psychopathological syndromes “thought disturbance” (BPRS) and “hostility” (AMDP). These results confirm the major impact of cannabis abuse on the long-term outcome of schizophrenic patients. Received: 16 December 1997 / Accepted: 19 November 1998     

Dyskinesia and soft neurological signs in schizophrenia: a comparative study

Objective. Several neurological abnormalities can be found at a greater frequency in patients with schizophrenia, including neurological soft signs (NSS) and signs of the “pyramidal” and “extrapyramidal” systems. We aimed to explore the frequency of movement disorders in patients with antipsychotic naïve schizophrenia and to compare and contrast with antipsychotic-treated patients and healthy controls. Methods. Twenty-two antipsychotic naive schizophrenic patients, 22 antipsychotic treated patients and 22 healthy control subjects were assessed by Neurological Evaluation (NES), Abnormal Involuntary Movements (AIMS), and Positive and Negative Syndrome (PANSS) Scales. Results. The NES scores of the never-medicated schizophrenic group were significantly higher than those of normal controls but did not differ significantly from the medicated group. Dyskinesia rates in the both schizophrenic groups were higher than in healthy controls. Medicated and non-medicated schizophrenic patient scores did not differ in AIMS with regard to facial and oral movements, but medicated patients scored higher than non-medicated subjects with respect to extremity movements. Conclusion. Our data suggest that: soft neurological signs and abnormal involuntary movements in the facial region are more prevalent in patients with schizophrenia, whether they are medicated or antipsychotic naïve. On the contrary, abnormal involuntary movements in the trunk and the extremities seem to be associated with medication.     

EEG correlates of face recognition in patients with schizophrenia spectrum disorders: A systematic review

ObjectiveTo systematically assess EEG studies and evaluate neuropsychological changes of face recognition in the context of neutral face stimuli in individuals with schizophrenia spectrum disorders.MethodsA literature search was conducted using the PubMed database from inception to March 2018. Studies included in the review measured any event-related potentials, neural oscillations, or phase synchrony.ResultsA total of 113 articles were identified. Twenty-nine studies were included for the review. The majority of the studies focused on the N170 component. Smaller N170 amplitudes were consistently reported in schizophrenia patients compared to the healthy control group. Significant correlations between N170 amplitudes and social functioning scales were reported. Other results were quite inconsistent; however, group differences were more prominent for tasks with specific conditions.ConclusionsOlder patients with longer disease duration show more consistent neuropsychological correlations. Alterations of event-related potentials are likely to be linked to higher severity of symptoms. The N170 component seems to be the most promising event-related potential to be used for evaluation of present status and for dynamic control of cognitive impairments, social functioning, and rehabilitation effectiveness.SignificanceThis systematic review provides evidence of (1) neuropsychological alterations of face processing outside an emotional context, and (2) a potential role of N170 as a diagnostic and treatment monitoring biomarker.