Meaningful gestures: electrophysiological indices of iconic gesture comprehension

To assess semantic processing of iconic gestures, EEG (29 scalp sites) was recorded as adults watched cartoon segments paired with soundless videos of congruous and incongruous gestures followed by probe words. Event-related potentials time-locked to the onset of gestures and probe words were measured in two experiments. In Experiment 1, participants judged the congruency between gestures and cartoons. Gestures elicited an N400-like component (gesture N450) that was larger for incongruent than congruent items, as well as a late positivity that was larger for congruent items. In Experiment 2, participants assessed the relatedness between probe words and preceding cartoon-gesture pairs. N450 effects to gestures were observed without overlapping positivity. These findings suggest that iconic gestures are subject to semantic processes analogous to those evoked by other meaningful representations, such as pictures and words.     

Grammatical class in lexical production and morhpological processing: Evidence from a case of fluent aphasia

We present the case of a fluent aphasic patient who is impaired at producing nouns relative to verbs in picture naming, sentence completion, and sentence generation tasks, but is better at both producing and comprehending concrete nouns than abstract nouns. Moreover, he displays a selective difficulty in producing the plural forms of some nouns and pseudowords presented as nouns, but was able to produce the phonologically identical third-person singular forms of corresponding verb homonyms and of the same pseudowords presented as verbs. This pattern of performance casts doubt on the hypothesis that grammatical class effects are always epiphenomena of more general semantic impairments that affect the naming of actions or of concrete objects, and suggests that these effects may arise instead from damage to syntactic processes pertaining specifically to the grammatical properties of words. We also discuss the implications of such damage for models of morphological processing.     

Neural correlates of individual differences related to appetite

Using neuroimaging technologies to compare normal weight and obese individuals can reveal much about the pathophysiological state of obesity but such comparisons tell us little about what makes some normal weight individuals susceptible to obesity or about important individual differences amongst obese individuals. The current review therefore reviews neuroimaging research on individual difference measures that can illuminate these important topics. After introducing three neuropsychological models of the nature of motivation to approach rewarding stimuli, neuroimaging research on measures of impulsivity, craving, binge eating, restrained eating and disinhibited eating is reviewed. Although neuroimaging research on individual differences measures of brain activity related to appetite is in its infancy, existing studies suggest that such research could enrich the understanding, prevention and treatment of disordered eating and obesity.     

Neural substrates in color processing: a comparison between painting majors and non-majors

Recent studies implicate regions in the frontal, temporal and occipital cortices of the brain in audio-visual (AV) integration of familiar objects. It remains unclear, however, which brain regions contribute to the creation of object-related AV memories, and whether activation of these regions is affected by crossmodal congruency. Here we used event-related functional MRI in a subsequent memory paradigm to investigate the neural substrates of successful encoding of semantically congruent and incongruent AV memories. Creation of both types of memories activated a region in the left inferior frontal gyrus (IFG). In addition, successful encoding of semantically related and unrelated AV pairs was correlated with increased activity in regions within the right lateral occipital cortex and bilateral lateral temporal cortex, respectively. These results may highlight a common role of IFG in retrieval of semantic information during encoding and suggest that the occipital and temporal cortices differentially process perceptual versus conceptual associations of AV memories.     

Neural mechanisms for the effect of prior knowledge on audiovisual integration

Converging evidence indicates that prior knowledge plays an important role in multisensory integration. However, the neural mechanisms underlying the processes with which prior knowledge is integrated with current sensory information remains unknown. In this study, we measured event-related potentials (ERPs) while manipulating prior knowledge using a novel visual letter recognition task in which auditory information was always presented simultaneously. The color of the letters was assigned to a particular probability of being associated with audiovisual congruency (e.g., green = high probability (HP) and blue = low probability (LP)). Results demonstrate that this prior began affecting reaction times to the congruent audiovisual stimuli at about the 900th trial. Consequently, the ERP data was analyzed in two phases: the “early phase” (<trial 600) and the “late phase” (>trial 900). The effects of prior knowledge were revealed through difference waveforms generated by subtracting the ERPs for the congruent audiovisual stimuli in the LP condition from those in the HP condition. A frontal-central probability effect (90-120 ms) was observed in the early phase. A right parietal-occipital probability effect (40-96 ms) and a frontal-central probability effect (170-200 ms) were observed in the late phase. The results suggest that during the initial acquisition of the knowledge about the probability of congruency, the brain assigned more attention to audiovisual stimuli for the LP condition. Following the acquisition of this prior knowledge, it was then used during early stages of visual processing and modulated the activity of multisensory cortical areas.     

Neural correlates of conflict processing

In this study we examined the neural correlates of conflict processing in the Stroop, counting, and digit-location tasks using event-related brain potentials (ERPs). The behavioral data revealed robust interference in response time and accuracy for all tasks. The interference effect for response time was greater in the Stroop task than the other tasks; in contrast, the interference effect for response accuracy was greater in the counting tasks than the other tasks. The N450 and sustained potential (SP) were elicited in each task. Partial least-squares (PLS) analysis was used to examine the structural relationships between the ERPs, task design, and behavior. TaskPLS analysis revealed that the N450 and SP were associated with a single latent variable leading to the suggestion that a common set of neural generators was recruited during conflict processing across the tasks and that there were differences between ERPs related to early processing across the three tasks. BehavioralPLS analysis revealed that the amplitude of the SP was positively correlated with response time and accuracy, indicating that this modulation of the ERPs may be related to response selection rather than to conflict resolution.     

Spatial disparity affects visual-auditory interactions in human sensorimotor processing

Information from the auditory and visual systems converges in the nervous system with physiological and behavioral consequences. Most of our knowledge about the rules governing such convergence has been obtained in experiments where the strength or the timing of the individual auditory and visual stimuli has been varied. Relatively little attention has been paid to the spatial relationship between different modalities of stimuli in multisensory experiments. We studied saccadic reaction times of human subjects to bimodal auditory and visual stimulus presentations under two conditions: first, with the targets spatially coincident and, second, with various degrees of spatial separation or disparity. In the first experiment, we found that the saccadic reaction times were consistently shorter than would be predicted by independent processing of information about the visual and auditory targets. These results suggest convergence of multimodal information at one or more loci within the nervous system. In the second experiment, we found that saccadic latency gradually increased as spatial distance between the auditory and visual targets increased. Evidence for neural summation was found over a wide range of spatial disparities. These results suggest that multisensory information can be integrated and have significant influences on behavior over a surprisingly large range of spatial disparity. Received: 12 January 1998 / Accepted: 23 April 1998     

Emotion in voice matters: Neural correlates of emotional prosody perception

The ability to perceive emotions is imperative for successful interpersonal functioning. The present study examined the neural characteristics of emotional prosody perception with an exploratory event-related potential analysis. Participants were 59 healthy individuals who completed a discrimination task presenting 120 semantically neutral word pairs from five prosody conditions (happy/happy, angry/angry, neutral/neutral, angry/happy, happy/angry). The task required participants to determine whether words in the pair were spoken in same or different emotional prosody. Reflective of an initial processing stage, the word 1 N1 component was found to have greatest amplitude in parietal regions of the hemispheres, and was largest for emotional compared to neutral stimuli, indicating detection of emotion features. A second processing stage, represented by word 1 P2, showed similar topographic effects; however, amplitude was largest for happy in the left hemisphere while angry was largest in the right, illustrating differentiation of emotions. At the third processing stage, word 1 N3 amplitude was largest in frontal regions, indicating later cognitive processing occurs in the frontal cortex. N3 was largest for happy, which had lowest accuracy compared to angry and neutral. The present results support Schirmer and Kotz's (2006) model of vocal emotion perception because they elucidated the function and ERP components by reflecting three primary stages of emotional prosody perception, controlling for semantic influence.     

Neural correlates of attention bias to threat in post-traumatic stress disorder

Attentional biases have been proposed to contribute to symptom maintenance in posttraumatic stress disorder (PTSD), although the neural correlates of these processes have not been well defined; this was the goal of the present study. We administered an attention bias task, the dot probe, to a sample of 37 (19 control, 18 PTSD+) traumatized African-American adults during fMRI. Compared to controls, PTSD+ participants demonstrated increased activation in the dorsolateral prefrontal cortex (dlPFC) in response to threat cue trials. In addition, attentional avoidance of threat corresponded with increased ventrolateral prefrontal cortex (vlPFC) and dorsal anterior cingulate cortex (dACC) activation in the PTSD group, a pattern that was not observed in controls. These data provide evidence to suggest that relative increases in dlPFC, dACC and vlPFC activation represent neural markers of attentional bias for threat in individuals with PTSD, reflecting selective disruptions in attentional control and emotion processing networks in this disorder.     

Development of multisensory reweighting for posture control in children

Reweighting to multisensory inputs adaptively contributes to stable and flexible upright stance control. However, few studies have examined how early a child develops multisensory reweighting ability, or how this ability develops through childhood. The purpose of the study was to characterize a developmental landscape of multisensory reweighting for upright postural control in children 4–10 years of age. Children were presented with simultaneous small-amplitude somatosensory and visual environmental movement at 0.28 and 0.2 Hz, respectively, within five conditions that independently varied the amplitude of the stimuli. The primary measure was body sway amplitude relative to each stimulus: touch gain and vision gain. We found that children can reweight to multisensory inputs from 4 years on. Specifically, intra-modal reweighting was exhibited by children as young as 4 years of age; however, inter-modal reweighting was only observed in the older children. The amount of reweighting increased with age indicating development of a better adaptive ability. Our results rigorously demonstrate the development of simultaneous reweighting to two sensory inputs for postural control in children. The present results provide further evidence that the development of multisensory reweighting contributes to more stable and flexible control of upright stance, which ultimately serves as the foundation for functional behaviors such as locomotion and reaching.     

Transfer of short-term adaptation in human saccadic eye movements

Controversy exists as to the extent to which the saccadic system, adapted in the so-called gain-shortening paradigm for a particular target configuration, transfers the resulting change in saccade metrics to saccades elicited under different circumstances. In order to further assess this problem, we investigated the properties of human saccadic eye movements after visually induced short-term adaptation under a variety of conditions. We observed that saccades both during and after the adaptation did not significantly change their main sequence properties with respect to the pre-adaptation baseline. Saccade velocity profiles remained normal throughout the experiment, and we obtained no evidence that correction saccades were gradually absorbed in the primary saccade. We found that the effect of the short-term adaptation on saccade metrics is not confined to the particular combination of initial eye position and spatial position of the visual target used to induce the adaptation response. Saccades elicited from different initial positions towards targets with the same retinotopic coordinates as in the adaptation phase yield the same level of adaptation. However, our findings indicate that adaptation is confined to a limited range of saccade vectors around the oculocentric coordinates of the adaptation target (restricted adaptation field). Smaller and larger saccades are endowed with significantly lower adaptation values. Moreover, two further experiments showed that a retinal stimulus is not a prerequisite for adaptation to express itself: First, in a double-step experiment, we dissociated the retinal stimulus vector from the required oculomotor response. Second, we also investigated the effect of visually induced adaptation on auditory evoked saccades. In both tasks the adaptation was transferred to the required motor response. Based on our findings, we conclude that short-term adaptation is expressed at a multisensory stage, where saccadic eye movements are represented as desired eye displacement vectors (motor error). Possible neurophysiological implications are discussed.     

'Prior entry' for pain: attention speeds the perceptual processing of painful stimuli

We investigated whether the perception of simultaneity for pairs of nociceptive and visual stimuli was dependent upon the focus of participants' attention to a particular sensory modality (either pain or vision). Two stimuli (one painful and the other visual) were presented randomly at different stimulus onset asynchronies (SOAs) using the method of constant stimuli. Participants made unspeeded verbal responses as to which stimulus they perceived as having been presented first, or else responded that the two stimuli were presented simultaneously. This temporal discrimination task was repeated under three different attention conditions (blocks): divided attention, attend pain, and attend vision. The results showed that under conditions of divided attention, nociceptive stimuli had to be presented before visual stimuli in order for the two to be perceived as simultaneous. A comparison of the two focused attention conditions revealed that the painful stimulus was perceived as occurring earlier in time (relative to the visual stimulus) when attention was directed toward pain than when it was directed toward vision. These results provide the first empirical demonstration that attention can modulate the temporal perception of painful stimuli.     

Neural correlates of local contextual processing deficits in schizophrenic patients

Deficits in processing contextual information are one of the main features of cognitive dysfunction in schizophrenia, but the neurophysiologic substrate underlying this dysfunction is poorly understood. We used ERPs to investigate local contextual processing in schizophrenic patients. Local context was defined as the occurrence of a short predictive series of stimuli occurring before delivery of a target event. Response times of predicted targets were faster in controls compared to patients. Schizophrenia patients failed to generate the P3b latency shift between predicted and random targets that was observed in controls and demonstrated a prominent reduction of the peak of an early latency context dependent positivity. The current study provides evidence of contextual processing deficits in schizophrenia patients by demonstrating alteration in the behavioral and neural correlates of local contextual processing.     

Spatial memory enhances the precision of angular self-motion updating

Humans are typically able to keep track of brief changes in their head and body orientation, even when visual and auditory cues are temporarily unavailable. Determining the magnitude of one’s displacement from a known location is one form of self-motion updating. Most research on self-motion updating during body rotations has focused on the role of a restricted set of sensory signals (primarily vestibular) available during self-motion. However, humans can and do internally represent spatial aspects of the environment, and little is known about how remembered spatial frameworks may impact angular self-motion updating. Here, we describe an experiment addressing this issue. Participants estimated the magnitude of passive, non-visual body rotations (40°–130°), using non-visual manual pointing. Prior to each rotation, participants were either allowed full vision of the testing environment, or remained blindfolded. Within-subject response precision was dramatically enhanced when the body rotations were preceded by a visual preview of the surrounding environment; constant (signed) and absolute (unsigned) error were much less affected. These results are informative for future perceptual, cognitive, and neuropsychological studies, and demonstrate the powerful role of stored spatial representations for improving the precision of angular self-motion updating.

Neurophysiological correlates of perceptual learning in the human brain

Summary Rapid learning processes are crucial for human object recognition. We report here on the alterations in neurophysiological activity in the human brain induced by repeated presentation of visual stimuli. In psychophysical experiments the percentage of correct responses increased significantly within less than 30 minutes in untrained observers. This stimulus-specific improvement was not carried over to differently oriented stimuli. Similar learning effects were observed in component latencies of evoked potential field distributions. The occurrence of specific potential field configurations reflected perceptual learning. A spatio-temporal activation pattern with steep gradients over the primary visual cortex appeared to be correlated with plasticity in the human visual system.Supported by the German Research Foundation, DFG Sk 26/5-1, Fa 119/5-2, and Zr 9/1-9. The technical assistance of K. Laschke and B. Tutsch during part of the experiments is gratefully acknowledged.     

The development of a dialogue between cortex and midbrain to integrate multisensory information

The anterior ectosylvian (AES) and rostral lateral suprasylvian (rLS) sulci send critical signals to multisensory superior colliculus (SC) neurons that enable them to integrate information from different senses. When either of these areas is temporarily deactivated in adult animals, the ability of SC neurons to integrate multisensory information and, thereby, enhance their responses to cross-modal stimuli is temporarily compromised. As a consequence, the ability to use cross-modal stimuli to enhance SC-mediated behavioral performance is also compromised. In contrast, removal of either one of these areas during early life has little effect on the development of multisensory processes in the SC or on SC-mediated multisensory behaviors and these animals seem very similar to normal controls. These observations suggest that there is considerable plasticity in these cortico-collicular systems during early life, with each area able to compensate for the early loss of the other. However, when both AES and rLS are removed early in life, there appears to be no compensation. The SC neurons now deal with sensory stimuli, even those embedded in multisensory complexes, as if they were there alone, precluding any SC-mediated behavioral benefit to cross-modal stimuli.     

Neural correlates of auditory feedback control in human

Auditory feedback plays an important role in natural speech production. We conducted a functional magnetic resonance imaging (fMRI) experiment using a transformed auditory feedback (TAF) method to delineate the neural mechanism for auditory feedback control of pitch. Twelve right-handed subjects were required to vocalize /a/ for 5 s, while hearing their own voice through headphones. In the TAF condition, the pitch of the feedback voice was randomly shifted either up or down from the original pitch two or three times in each trial. The subjects were required to hold the pitch of the feedback voice constant by changing the pitch of original voice. In non-TAF condition, the pitch of the feedback voice was not modulated and the subjects just vocalized /a/ continuously. The contrast between TAF and non-TAF conditions revealed significant activations; the supramarginal gyrus, the prefrontal area, the anterior insula, the superior temporal area and the intraparietal sulcus in the right hemisphere, but only the premotor area in the left hemisphere. This result suggests that auditory feedback control of pitch is mainly supported by the right hemispheric network.     

Neural correlates of error processing reflect individual differences in interoceptive sensitivity

Although self-monitoring is an important process for adaptive behaviors in multiple domains, the exact relationship among different internal monitoring systems is unclear. Here, we aimed to determine whether and how physiological monitoring (interoception) and behavioral monitoring (error processing) are related to each other. To this end we examined within-subject correlations among measures representing each function. Score on the heartbeat counting task (HCT) was used as a measure of interoceptive awareness. The amplitude of two event-related potentials (error-related negativity [ERN] and error-positivity [Pe]) elicited in error trials of a choice-reaction task (Simon task) were used as measures of error processing. The Simon task presented three types of stimuli (objects, faces showing disgust, and happy faces) to further examine how emotional context might affect inter-domain associations. Results showed that HCT score was robustly correlated with Pe amplitude (the later portion of error-related neural activity), irrespective of stimulus condition. In contrast, HCT score was correlated with ERN amplitude (the early component) only when participants were presented with disgust-faces as stimuli, which may have automatically elicited a physiological response. Behavioral data showed that HCT score was associated with the degree to which reaction times slowed after committing errors in the object condition. Cardiac activity measures indicated that vigilance level would not explain these correlations. These results suggest a relationship between physiological and behavioral monitoring. Furthermore, the degree to which behavioral monitoring relies on physiological monitoring appears to be flexible and depend on the situation.     

Evoked potential correlates of human information processing.

The late positive component (P3) of the feedback evoked potential was investigated in two tasks: the detection of near-threshold auditory stimuli, and the estimation of 1 s time intervals. When the intensity or probability of the threshold stimuli was varied in the detection task, it became apparent that the feedback P3 was related to the 'contingent probability' of confirming or disconfirming feedback given a particular response. In the time-estimation task the relative probabiliti-s of confirming and disconfirming feedback were altered by changing the time-window wherein a response was judged correct. In these experiments it was found that the feedback P3 component was highly dependent upon the probability of the feedback, and relatively independent of its confirming or disconfirming meaning. With decreasing probability of the feedback stimuli, the P3 component became larger, later, and somewhat more frontal.     

ERP correlates of processing the auditory consequences of own versus observed actions

Research has so far focused on neural mechanisms that allow us to predict the sensory consequences of our own actions, thus also contributing to ascribing them to ourselves as agents. Less attention has been devoted to processing the sensory consequences of observed actions ascribed to another human agent. Focusing on audition, there is consistent evidence of a reduction of the auditory N1 ERP for self‐ versus externally generated sounds, while ERP correlates of processing sensory consequences of observed actions are mainly unexplored. In a between‐groups ERP study, we compared sounds generated by self‐performed (self group) or observed (observation group) button presses with externally generated sounds, which were presented either intermixed with action‐generated sounds or in a separate condition. Results revealed an overall reduction of the N1 amplitude for processing action‐ versus externally generated sounds in both the intermixed and the separate condition, with no difference between the groups. Further analyses, however, suggested that an N1 attenuation effect relative to the intermixed condition at frontal electrode sites might exist only for the self but not for the observation group. For both groups, we found a reduction of the P2 amplitude for processing action‐ versus all externally generated sounds. We discuss whether the N1 and the P2 reduction can be interpreted in terms of predictive mechanisms for both action execution and observation, and to what extent these components might reflect also the feeling of (self) agency and the judgment of agency (i.e., ascribing agency either to the self or to others).