Spatial conceptual associations between music and pictures as revealed by N400 effect

The processing of extramusical meaning can be reflected in N400 effects of the ERP. However, how conceptual representations can be activated in music still needs to be specified. We investigated the activation of iconic meaningful representations in music by using a cross‐modal semantic priming paradigm with an implicit task. Pictures of spatial scenes were semantically congruent or incongruent to preceding music in three stimulus onset asynchrony (SOA) conditions. The results revealed that the semantically incongruent target pictures elicited larger N400 amplitude than the congruent target pictures. Moreover, the semantic priming effect was modulated by the SOAs. The N400 effect was observed in the 200‐ms and 800‐ms SOA conditions, but not in the 1,200‐ms SOA condition. These results suggest that extramusical meaning purely due to iconic sign quality can be activated, and that the conceptual activation in music can be rapid and automatic.     

Imitation of hand and tool actions is effector-independent

Following the theoretical notion that tools often extend one’s body, in the present study, we investigated whether imitation of hand or tool actions is modulated by effector-specific information. Subjects performed grasping actions toward an object with either a handheld tool or their right hand. Actions were initiated in response to pictures representing a grip at an object that could be congruent or incongruent with the required action (grip-type congruency). Importantly, actions could be cued by means of a tool cue, a hand cue, and a symbolic cue (effector-type congruency). For both hand and tool actions, an action congruency effect was observed, reflected in faster reaction times if the observed grip type was congruent with the required movement. However, neither hand actions nor tool actions were differentially affected by the effector represented in the picture (i.e., when performing a tool action, the action congruency effect was similar for tool cues and hand cues). This finding suggests that imitation of hand and tool actions is effector-independent and thereby supports generalist rather than specialist theories of imitation.     

Automatic imitation is automatic, but less so for narcissists

Imitation is a fundamentally important human capability and has been the topic of considerable research in the behavioural sciences. One paradigm for investigating the basic nature of imitation is the “automatic imitation” paradigm. In this paradigm, participants are symbolically cued to make a particular response, whilst being incidentally exposed to a congruent or incongruent motor action performed by another person. The robust finding is that when the incidental action is incongruent with the cued action, participants are slower to respond than when it is congruent. Despite the name given to this paradigm, the extent to which the imitative tendency involved is actually automatic remains unclear. Here, we manipulated the probability of congruent and incongruent trials within blocks to assess the effects of expectation on the imitative process. In addition, we determined whether an individual difference variable related to how people process others’ behaviour—narcissism—affected the automaticity of imitation. Our results confirm that imitation as observed in this paradigm is robust in the face of expectation. However, the degree to which expectation modulates automatic imitation was enhanced for individuals who scored higher on a narcissism inventory. Together, these results suggest that imitation in the automatic imitation paradigm is indeed largely automatic, but that individual differences in narcissism can change the extent to which imitative behaviour manifests.     

Temporal aspects of the visuotactile congruency effect

We report an experiment designed to investigate the temporal dynamics of the visuotactile crossmodal congruency effect. Vibrotactile targets were presented randomly to the index finger (top side of a hand-held cube) or thumb (bottom side) of either hand while visual distractors were presented randomly from one of the same four possible locations. The stimulus onset asynchrony (SOA) between the vibrotactile target and the visual distractor was varied on a trial-by-trial basis. Participants made speeded discrimination responses regarding the elevation of the vibrotactile targets (i.e., upper versus lower) while trying to ignore the visual distractors. The largest crossmodal congruency effects (defined as the difference in performance between incongruent and congruent elevation distractor trials) were obtained when the visual distractor preceded the vibrotactile target by 50-100 ms, although significant effects were also reported when the distractor followed the target by as much as 100 ms. These results are discussed in terms of the conjoint influence of response competition, crossmodal perceptual interactions (i.e., the ventriloquism effect), and exogenous spatial attention on the crossmodal congruency effect. The distinct temporal signatures of each of these effects are also highlighted.     

Collision error avoidance: influence of proportion congruency and sensorimotor memory on open-loop grasp control

Grasping behaviour involves the integration of current and historical knowledge about an object, a process that can be influenced by sensory uncertainty. In the present study, participants simultaneously interacted with a visual cue and a haptic cue before reaching to grasp a target object. The visual cue was either congruent (equal in size to haptic cue and target) or incongruent (larger than haptic cue and target). To enhance sensory uncertainty, we manipulated the proportion of congruent trials to be either 80 or 20%. We compared grasp kinematics and forces between congruent and incongruent trials and between the 20 and 80% proportion congruency groups. We also studied the effects of trial history by comparing the performance of congruent and incongruent trials preceded by either the same or opposite trial type. Proportion congruency did not affect temporal kinematics but did affect maximum grip aperture (MGA) as the 80% proportion congruency group used a greater MGA, regardless of trial type. For grasping forces, an interaction effect showed that the 20% proportion congruency group used a greater peak load force on congruent trials. Incongruent trials that followed congruent trials had decreased movement time, increased MGA and increased grasping forces, relative to those that followed incongruent trials. We interpret the data to suggest that the grasp control system integrates multisensory information using flexible, yet specific criteria regarding task constraints. The prevention of collision error (i.e., an inadequate MGA when contacting the target) may be one guiding principle in the control process.     

Skilled readers begin processing sub-phonemic features by 80 ms during visual word recognition: evidence from ERPs

Two masked priming experiments investigated the time-course of the activation of sub-phonemic information during visual word recognition. EEG was recorded as participants read targets with voiced and unvoiced final consonants (e.g., fad and fat), preceded by nonword primes that were incongruent or congruent in voicing and vowel duration (e.g., fap or faz). Experiment 1 used a long duration mask (100 ms) between prime and target, whereas Experiment 2 used a short mask (22 ms). Phonological feature congruency began modulating the amplitude of brain potentials by 80 ms; the feature incongruent condition evoked greater negativity than the feature congruent condition in both experiments. The early onset of the congruency effect indicates that skilled readers initially activate sub-phonemic feature information during word identification. Congruency effects also appeared in the middle and late periods of word recognition, suggesting that readers use phonological representations in multiple aspects of visual word recognition.     

The role of action representations in visual object recognition

It is typically assumed that perception for action and object recognition are subserved by functionally and neuroanatomically distinct processing streams in the brain. However, recent evidence challenges this classical view and suggests an interaction between both visual processing streams. While previous studies showed an influence of object perception on action-related tasks, we investigated whether action representations facilitate visual object recognition. In order to address this question, two briefly displayed masked objects were sequentially presented, either affording congruent or incongruent motor interactions. We found superior naming accuracy for object pairs with congruent as compared to incongruent motor interactions (Experiment 1). This action priming effect indicates that action representations can facilitate object recognition. We further investigated the nature of the representations underlying this action priming effect. The effect was absent when the prime stimulus was presented as a word (Experiment 2). Thus, the action priming effect seems to rely on action representations specified by visual object information. Our findings suggest that processes of object-directed action influence object recognition.     

Imitation behavior is sensitive to visual perspective of the model: an fMRI study

Imitation behavior and accompanying brain activity can be affected by the perspective of the model adopted. The present study was designed to understand the effect of a model’s perspective in terms of the view (1st person vs. 3rd person) and the anatomical congruency of the limb between the model and the performer (congruent vs. incongruent). Eighteen young participants observed video clips of a model’s finger-lifting behavior and lifted the same finger on their right hand as quickly as possible. Half of the video clips were filmed from the view of the participant (the 1st person view), whereas the other half were filmed from the perspective of facing a mirror (the 3rd person view). Each video clip depicted the finger lifting of the model’s right (congruent) or left (incongruent) hand. Comparisons of the latency to imitate among the four perspective conditions showed significantly shorter latency for the 1st person-congruent and 3rd person-incongruent conditions. Hemodynamic measurements with functional magnetic resonance imaging showed that shorter latency was explained with less involvement of the brain areas that are activated when a task is relatively complex. The brain areas considered to be a part of neural substrates of imitation were significantly activated under the 1st person view conditions regardless of the hand congruency. These findings suggest that, although the latency to imitate finger lifting was determined by the complexity of the task induced with the model’s perspective, imitation behavior seemed to be more effectively guided with the models filmed from the 1st person view.     

Differential influence of attention on gaze and head movements

A salient peripheral cue can capture attention, influencing subsequent responses to a target. Attentional cueing effects have been studied for head-restrained saccades; however, under natural conditions, the head contributes to gaze shifts. We asked whether attention influences head movements in combined eye-head gaze shifts and, if so, whether this influence is different for the eye and head components. Subjects made combined eye-head gaze shifts to horizontal visual targets. Prior to target onset, a behaviorally irrelevant cue was flashed at the same (congruent) or opposite (incongruent) location at various stimulus-onset asynchrony (SOA) times. We measured eye and head movements and neck muscle electromyographic signals. Reaction times for the eye and head were highly correlated; both showed significantly shorter latencies (attentional facilitation) for congruent compared with incongruent cues at the two shortest SOAs and the opposite pattern (inhibition of return) at the longer SOAs, consistent with attentional modulation of a common eye-head gaze drive. Interestingly, we also found that the head latency relative to saccade onset was significantly shorter for congruent than that for incongruent cues. This suggests an effect of attention on the head separate from that on the eyes.     

Cross-modal links between vision and touch in tactile discrimination of line orientation

This study examined visuotactile cross-modal interactions in the identification of tactile stimuli. In Experiment 1, participants were required to discriminate the orientation of tactile target stimuli (line segments with left and right diagonal orientations) presented at the tip of the participant's left index finger via an Optacon, while trying to ignore a task-irrelevant visual diagonal line presented with a 200-ms or 700-ms stimulus onset asynchrony (SOA). The results show that the participants responded faster when the orientation of the tactile stimuli were congruent with the orientation of the visual stimuli than when they were incongruent in the 200-ms SOA condition. To assess at what level of processing the reaction time advantage obtained with congruent orientation trials takes place, Experiment 2 served as a replication of Experiment 1 with somatosensory event-related brain potential (ERP) recordings. ERP effects of orientation congruency were observed between 200 and 350 ms post-stimulus, suggesting that visuotactile cross-modal interactions may occur at intermediate perceptual-cognitive stages rather than at early sensory-specific or late motor-related stages.     

How and when auditory action effects impair motor performance

Music performance is characterized by complex cross-modal interactions, offering a remarkable window into training-induced long-term plasticity and multimodal integration processes. Previous research with pianists has shown that playing a musical score is affected by the concurrent presentation of musical tones. We investigated the nature of this audio-motor coupling by evaluating how congruent and incongruent cross-modal auditory cues affect motor performance at different time intervals. We found facilitation if a congruent sound preceded motor planning with a large Stimulus Onset Asynchrony (SOA ?300 and ?200 ms), whereas we observed interference when an incongruent sound was presented with shorter SOAs (?200, ?100 and 0 ms). Interference and facilitation, instead of developing through time as opposite effects of the same mechanism, showed dissociable time-courses suggesting their derivation from distinct processes. It seems that the motor preparation induced by the auditory cue has different consequences on motor performance according to the congruency with the future motor state the system is planning and the degree of asynchrony between the motor act and the sound presentation. The temporal dissociation we found contributes to the understanding of how perception meets action in the context of audio-motor integration.     

Cortical oscillations related to processing congruent and incongruent grapheme-phoneme pairs

In this study, we investigated changes in cortical oscillations following congruent and incongruent grapheme-phoneme stimuli. Hiragana graphemes and phonemes were simultaneously presented as congruent or incongruent audiovisual stimuli to native Japanese-speaking participants. The discriminative reaction time was 57 ms shorter for congruent than incongruent stimuli. Analysis of MEG responses using synthetic aperture magnetometry (SAM) revealed that congruent stimuli evoked larger 2-10 Hz activity in the left auditory cortex within the first 250 ms after stimulus onset, and smaller 2-16 Hz activity in bilateral visual cortices between 250 and 500 ms. These results indicate that congruent visual input can modify cortical activity in the left auditory cortex.     

The effects of illusory line motion on incongruent saccades: implications for saccadic eye movements and visual attention

A complex neural problem must be solved before a voluntary eye movement is triggered away from a stimulus (antisaccade). The location code activated by a stimulus must be internally translated into an appropriate signal to direct the eyes into the opposite visual field, while the reflexive tendency to look directly at the stimulus must be suppressed. No doubt these extra processes contribute to the ubiquitous slowing of antisaccades. However, there is no consensus on the cognitive mechanisms that contribute to the antisaccade programme. Visual attention is closely associated with the generation of saccadic eye movements and it has been shown that attention will track an illusion of line motion. A series of experiments combined this illusion with a saccadic eye movement that was congruent (i.e. directed towards), or incongruent with (i.e. direct away from), a peripheral target. Experiment 1 showed that congruent saccades had faster reaction times than incongruent saccades. In contrast, Experiments 2 and 3 demonstrated that, with illusory line motion, incongruent saccades now had faster reaction times than congruent saccades. These findings demonstrate that an illusory phenomenon can accelerate the processing of an incongruent relative to a congruent saccade.     

Hemispheric laterality in contextual recognition

Hemispheric dominance in contextual recognition is now under discussion. Auditory event-related potentials (ERPs) with a contextual decision task were recorded to study hemispheric dominance by using the wave energy and amplitudes of N400 for eight native volunteers. Two types of 80 sentences with four clauses were used as the structure: [subject] + [object or complement] + [object] + [two-syllable verb]. One type was congruent and the other deviated at the terminal clause as an incongruent sentence. Twenty-one channel ERPs were triggered at the verbs and recorded during -200 and 1,000 ms, with a noncephalic reference from 0.05 to 30 Hz. The wave energy was calculated as mean square potentials and was normalized with the background activity (pretrigger 200 ms). The amplitudes of N400 showed significant differences between the incongruent and congruent sentences, but the hemispheric dominance for the amplitudes of N400 was not significant between the incongruent and congruent sentences. Reaction times with the index finger to respond to the deviated verbs were 470.0 (mean) +/- 42.5 (SD) ms. The maximum normalized wave energy within the mean reaction time showed significant differences between the incongruent and congruent sentences, and the left hemispheric energy was shown to be significantly higher than the right one.     

Integration of visual and tactile stimuli: top-down influences require time

In a visuotactile congruency task, a distracting flash of light presented near a tactile target can influence speeded judgments of tactile location. Localization of the tactile target is more rapid when the elevation of the visual distractor is congruent with the tactile stimulus than when it is incongruent. The goal of the present study was to examine the degree of control that can be exerted on the process proposed to integrate the visual and tactile stimuli. To this end, the proportion of spatially congruent items was manipulated across blocks of trials. A robust congruency effect was observed across three experiments. There was no effect of proportion congruency (varied between 75 and 11% congruent) when the visual event was presented only 30 ms prior to the tactile event. When this lead-time was increased to 100 ms there was a significant increase in the congruency effect, for errors, in the high proportion congruent conditions (experiment 3). We conclude that with sufficient lead-time, top-down influence can be exerted in this task, however, when presented at near simultaneity, visuotactile integration is independent of top-down effects.These data were previously presented at the fifth IMRF in Sitges, Spain, June 2–5, 2004 .     

The effect of odour priming on cortical EEG and visual ERP responses.

Two studies are reported that were designed to examine the way in which subjects, when primed with an olfactory stimulus, make perceptual judgements regarding subsequent congruent/incongruent information. The paradigm used was based on a modified version of a standard ERP semantic priming procedure. Subjects watched a computer screen and made a decision as to whether an image was congruent or incongruent with a priming olfactory stimulus. Results of the two studies showed significant effects in the N400 component of the visual ERP between congruent and incongruent stimuli are reported when a complex malodour accord was used as a prime but not when a complex pleasant odour accord was used as a prime.     

Movement interference during action observation as emergent coordination

Previous research has demonstrated that when an actor coordinates with spatially incompatible movements of another individual that motor interference occurs-the rhythmic arm movements of the actor exhibit increased movement variability in the plane orthogonal (non-instructed) to the instructed plane of motion. Here we examine whether this motor contagion reflects not error but the spontaneous recruitment of additional task-specific movement degrees of freedom employed to withstand increasing task difficulty. Participants coordinated congruent and incongruent forearm movements with a confederate moving at a fast, moderate, and slow target frequency. Examining the variability in the non-instructed plane revealed oscillatory non-instructed plane movements that were coordinated with the instructed plane movements of the confederate. The results suggest motor interference during incongruent coordination can be understood as an emergent, task-specific property of the coordination goal.     

Kinematic and electromyographic responses to perturbation of a rapid grasp

Kinematic and electromyographic responses of the thumb and index finger to load-induced extension of the thumb during a rapid precision grasp of the thumb and finger were studied in four human subjects. Angular position of the index finger metacarpophalangeal (MP) and proximal interphalangeal (PIP) joints was recorded along with the linear position of the thumb tip (TH). Myoelectric activity was recorded from flexor pollicis longus, flexor digitorum superficialis, first dorsal interosseous, and extensor digitorum communis. Loads that extended the thumb were applied ranging from 125 ms prior to movement onset to movement onset. For loads applied earlier than 50 ms before movement onset, the amount of flexion of TH was reduced compared with control trials. Contact between the finger and thumb was attained nonetheless due to an altered trajectory of the fingertip that was generated by reduced flexion of PIP and increased flexion of MP. These finger responses appear to be functional in that contact of the finger pulp on the more distal pulp of the thumb was preserved. With loads delivered near onset of the grasp, there was increased PIP flexion, rather than reduced PIP flexion. These responses to later loads occurred despite greatly reduced magnitudes of TH flexion compared with loads delivered well before onset of the gasp movements. Thus reduced PIP flexion observed with early loads was not simply the result of finger biomechanics. The thumb flexor muscle increased activity 45-55 ms after onset of the load, whereas responses of the finger flexors began 65 ms after load onset. Response magnitudes decreased as loads were introduced nearer to movement onset. Measured reaction times of the finger muscles to thumb extension stimuli averaged 154 ms, which indicated that the responses of the finger muscles were not voluntary responses to the thumb extension. Afferent information generated by perturbation of the thumb during a grasp movement can influence the activity of intrinsic and extrinsic muscles to yield apparently functional compensations in the closing movements. However, temporal limitations exist that appear to offer greater constraints on the use of afferent signals for controlling rapid movements than for sustained grasp.     

Effects of irrelevant stimulus orientation on visually guided grasping movements

The present study investigated the effects of irrelevant stimulus orientation on visually guided grasping movements. Participants had to grasp a rectangular object at either the ends or the sides, depending on the color of a visual stimulus. In this task, correspondence between stimulus orientation and object orientation (stimulus-object congruency) and correspondence between stimulus orientation and hand orientation (stimulus-hand congruency) varied independently. Two experiments, with different sets of object orientations, revealed a consistent pattern of results. In particular, there were significant effects of stimulus-hand congruency, suggesting that perceiving an object activates congruently oriented hand movements. However, stimulus-object congruency had no effects, indicating that participants did not benefit from a preactivation of object orientation in the present task. The pattern of congruency effects implies that the cognitive representation, which is affected by irrelevant visual information, entails only those object or response features that are needed to select and control a response.