Audiovisual temporal fusion in 6-month-old infants

The aim of this study was to investigate neural dynamics of audiovisual temporal fusion processes in 6-month-old infants using event-related brain potentials (ERPs). In a habituation-test paradigm, infants did not show any behavioral signs of discrimination of an audiovisual asynchrony of 200 ms, indicating perceptual fusion. In a subsequent EEG experiment, audiovisual synchronous stimuli and stimuli with a visual delay of 200 ms were presented in random order. In contrast to the behavioral data, brain activity differed significantly between the two conditions. Critically, N1 and P2 latency delays were not observed between synchronous and fused items, contrary to previously observed N1 and P2 latency delays between synchrony and perceived asynchrony. Hence, temporal interaction processes in the infant brain between the two sensory modalities varied as a function of perceptual fusion versus asynchrony perception. The visual recognition components Pb and Nc were modulated prior to sound onset, emphasizing the importance of anticipatory visual events for the prediction of auditory signals. Results suggest mechanisms by which young infants predictively adjust their ongoing neural activity to the temporal synchrony relations to be expected between vision and audition.     

My face in yours: Visuo-tactile facial stimulation influences sense of identity

Abstract

Self-face recognition is crucial for sense of identity and self-awareness. Finding self-face recognition disorders mainly in neurological and psychiatric diseases suggests that modifying sense of identity in a simple, rapid way remains a “holy grail” for cognitive neuroscience. By touching the face of subjects who were viewing simultaneous touches on a partner's face, we induced a novel illusion of personal identity that we call “enfacement”: The partner's facial features became incorporated into the representation of the participant's own face. Subjects reported that morphed images of themselves and their partner contained more self than other only after synchronous, but not asynchronous, stroking. Therefore, we modified self-face recognition by means of a simple psychophysical manipulation. While accommodating gradual change in one's own face is an important form of representational plasticity that may help maintaining identity over time, the surprisingly rapid changes induced by our procedure suggest that sense of facial identity may be more malleable than previously believed. “Enfacement” correlated positively with the participant's empathic traits and with the physical attractiveness the participants attributed to their partners. Thus, personality variables modulate enfacement, which may represent a marker of the tendency to be social and may be absent in subjects with defective empathy.     

Parkinson's disease may reduce sensitivity to visual-tactile asynchrony irrespective of dopaminergic treatment: Evidence from the rubber hand illusion

BackgroundThe feeling of body ownership relies on the binding of multisensory body-related signals. Various sensory abnormalities have been described in Parkinson's disease (PD).ObjectiveTo assess the rubber hand illusion (RHI) in patients with PD (PwPD) and age-matched healthy controls (CTRL). To evaluate the influence of the dopaminergic system in a PwPD subgroup OFF medication.MethodsThe RHI paradigm was applied to 42 PwPD and 48 CTRL. In this experimental setup, stroking a visible plastic hand simultaneously with the covered real hand elicits the feeling of ownership over the seen hand. Asynchronous stroking served as a control condition. Proprioceptive bias and an illusion score based on a questionnaire were used as measures of the RHI. Seventeen PwPD additionally underwent the experiments “OFF medication”.ResultsCompared to CTRL, PwPD showed higher proprioceptive bias independent of the stroking condition (p = 0.015), and had higher illusion scores in the asynchronous condition (p < 0.05). In PwPD, there were no significant differences between ON- and OFF-medication state.ConclusionIn PwPD, responses to the RHI are less specific with respect to the degree of synchronicity of brushstrokes. This might be attributed to a less stable body representation, internal “noise” during multisensory integration, or a blur of temporal discrimination in PD. The fact that RHI measures did not differ between ON- and OFF-medication states indicates an involvement of non-dopaminergic transmitter systems in this finding.     

How cognitive aging affects multisensory integration of navigational cues

Entorhinal grid cells and hippocampal place cells are key systems for mammalian navigation. By combining information from different sensory modalities, they provide abstract representations of space. Given that both structures are among the earliest to undergo age-related neurodegenerative changes, we asked whether age-related navigational impairments are related to deficient integration of navigational cues. Younger and older adults performed a homing task that required using visual landmarks, self-motion information, or a combination of both. Further, a conflict between cues assessed the influence of each sensory domain. Our findings revealed performance impairments in the older adults, suggestive of a higher noise in the underlying spatial representations. In addition, even though both groups integrated visual and self-motion information to become more accurate and precise, older adults did not place as much influence on visual information as would have been optimal. As these findings were unrelated to potential changes in balance or spatial working memory, this study provides the first evidence that increasing noise and a suboptimal weighting of navigational cues might contribute to the common problems with spatial representations experienced by many older adults. These findings are discussed in the context of the known age-related changes in the entorhinal-hippocampal network.     

Improving therapeutic outcomes in autism spectrum disorders: Enhancing social communication and sensory processing through the use of interactive robots

For children with autism spectrum disorders (ASDs), social robots are increasingly utilized as therapeutic tools in order to enhance social skills and communication. Robots have been shown to generate a number of social and behavioral benefits in children with ASD including heightened engagement, increased attention, and decreased social anxiety. Although social robots appear to be effective social reinforcement tools in assistive therapies, the perceptual mechanism underlying these benefits remains unknown. To date, social robot studies have primarily relied on expertise in fields such as engineering and clinical psychology, with measures of social robot efficacy principally limited to qualitative observational assessments of children's interactions with robots. In this review, we examine a range of socially interactive robots that currently have the most widespread use as well as the utility of these robots and their therapeutic effects. In addition, given that social interactions rely on audiovisual communication, we discuss how enhanced sensory processing and integration of robotic social cues may underlie the perceptual and behavioral benefits that social robots confer. Although overall multisensory processing (including audiovisual integration) is impaired in individuals with ASD, social robot interactions may provide therapeutic benefits by allowing audiovisual social cues to be experienced through a simplified version of a human interaction. By applying systems neuroscience tools to identify, analyze, and extend the multisensory perceptual substrates that may underlie the therapeutic benefits of social robots, future studies have the potential to strengthen the clinical utility of social robots for individuals with ASD.     

Multisensory integration across exteroceptive and interoceptive domains modulates self-experience in the rubber-hand illusion

Identifying with a body is central to being a conscious self. The now classic “rubber hand illusion” demonstrates that the experience of body-ownership can be modulated by manipulating the timing of exteroceptive (visual and tactile) body-related feedback. Moreover, the strength of this modulation is related to individual differences in sensitivity to internal bodily signals (interoception). However the interaction of exteroceptive and interoceptive signals in determining the experience of body-ownership within an individual remains poorly understood. Here, we demonstrate that this depends on the online integration of exteroceptive and interoceptive signals by implementing an innovative “cardiac rubber hand illusion” that combined computer-generated augmented-reality with feedback of interoceptive (cardiac) information. We show that both subjective and objective measures of virtual-hand ownership are enhanced by cardio-visual feedback in-time with the actual heartbeat, as compared to asynchronous feedback. We further show that these measures correlate with individual differences in interoceptive sensitivity, and are also modulated by the integration of proprioceptive signals instantiated using real-time visual remapping of finger movements to the virtual hand. Our results demonstrate that interoceptive signals directly influence the experience of body ownership via multisensory integration, and they lend support to models of conscious selfhood based on interoceptive predictive coding.     

Perception based method for the investigation of audiovisual integration of speech

Speech comprehension is significantly improved by visual input on the speaker's mouth movements. Audiovisual integration underlying this phenomenon is often studied in EEG experiments in which the event related brain potential (ERP) elicited by a bimodal stimulus is compared to the sum of ERPs triggered by auditory and visual signals of the same source. However, this method leads to spurious results in time ranges when ERP components common to all these stimulus types are present. A method that aims to filter out such common early anticipatory potentials is data high-pass filtering. In the present study, first, we demonstrated that subtle changes in filter cut-off frequency lead to remarkably different results on the interaction effect so that no reliable conclusion on the spatial distribution of the interaction could be drawn. Second, we suggested a different approach for the investigation of ERP correlates of audiovisual integration: bimodal syllables modified by light temporal asynchrony were presented to subjects and ERPs correlating with the fused and unfused perceptions were compared. We found that components corresponding to both auditory N1 and P2 waves were smaller in case of the fused perception, supporting the view that N1 and P2 generator activities are suppressed during multimodal speech perception. The N1 effect showed a clearly right hemisphere dominance while the effect around the P2 peak was most pronounced on centroparietal electrodes and dominated over the left hemisphere.