Crossmodal interaction in saccadic reaction time: separating multisensory from warning effects in the time window of integration model

In a focused attention task saccadic reaction time (SRT) to a visual target stimulus (LED) was measured with an auditory (white noise burst) or tactile (vibration applied to palm) non-target presented in ipsi- or contralateral position to the target. Crossmodal facilitation of SRT was observed under all configurations and stimulus onset asynchrony (SOA) values ranging from  −500 (non-target prior to target) to 0 ms, but the effect was larger for ipsi- than for contralateral presentation within an SOA range from  −200 ms to 0. The time-window-of-integration (TWIN) model (Colonius and Diederich in J Cogn Neurosci 16:1000, 2004) is extended here to separate the effect of a spatially unspecific warning effect of the non-target from a spatially specific and genuine multisensory integration effect.

The dog’s meow: asymmetrical interaction in cross-modal object recognition

Little is known on cross-modal interaction in complex object recognition. The factors influencing this interaction were investigated using simultaneous presentation of pictures and vocalizations of animals. In separate blocks, the task was to identify either the visual or the auditory stimulus, ignoring the other modality. The pictures and the sounds were congruent (same animal), incongruent (different animals) or neutral (animal with meaningless stimulus). Performance in congruent trials was better than in incongruent trials, regardless of whether subjects attended the visual or the auditory stimuli, but the effect was larger in the latter case. This asymmetry persisted with addition of a long delay after the stimulus and before the response. Thus, the asymmetry cannot be explained by a lack of processing time for the auditory stimulus. However, the asymmetry was eliminated when low-contrast visual stimuli were used. These findings suggest that when visual stimulation is highly informative, it affects auditory recognition more than auditory stimulation affects visual recognition. Nevertheless, this modality dominance is not rigid; it is highly influenced by the quality of the presented information. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.

Crossmodal exogenous orienting improves the accuracy of temporal order judgments

Although many studies have demonstrated that crossmodal exogenous orienting can lead to a facilitation of reaction times, the issue of whether exogenous spatial orienting also affects the accuracy of perceptual judgments has proved to be much more controversial. Here, we examined whether or not exogenous spatial attentional orienting would affect sensitivity in a temporal discrimination task. Participants judged which of the two target letters, presented on either the same or opposite sides, had been presented first. A spatially non-predictive tone was presented 200 ms prior to the onset of the first visual stimulus. In two experiments, we observed improved performance (i.e., a decrease in the just noticeable difference) when the target letters were presented on opposite sides and the auditory cue was presented on the side of the first visual stimulus, even when central fixation was monitored ("Experiment 2"). A shift in the point of subjective simultaneity was also observed in both experiments, indicating ‘prior entry’ for cued as compared to uncued first target trials. No such JND or PSS effects were observed when the auditory tone was presented after the second visual stimulus ("Experiment 3"), thus confirming the attentional nature of the effects observed. These findings clearly show that the crossmodal exogenous orienting of spatial attention can affect the accuracy of temporal judgments.

Involuntary inhibition of movement initiation alters oculomotor competition resolution

Identifying a stimulus as the target for a goal-directed movement involves inhibiting competing responses. Separable inhibitory interconnections bias local competition to ensure only one stimulus is selected and to alter movement initiation. Behavioural evidence of these inhibitory processes comes from the effects of distracters on oculomotor landing positions and saccade latencies. Here, we investigate the relationship between these two sources of inhibition. Targets were presented with or without close and remote distracters. In separate experiments the possible position and identity of the target and distracters were manipulated. In all cases saccade landing position was found to be less affected by the presence of the close distracter when remote distracters were also present. The involuntary increase in the latency of saccade initiation caused by the presence of the remote distracters alters the state of competitive processes involved in selecting the saccade target thus changing its landing position.

Sensory information in perceptual-motor sequence learning: visual and/or tactile stimuli

Sequence learning in serial reaction time (SRT) tasks has been investigated mostly with unimodal stimulus presentation. This approach disregards the possibility that sequence acquisition may be guided by multiple sources of sensory information simultaneously. In the current study we trained participants in a SRT task with visual only, tactile only, or bimodal (visual and tactile) stimulus presentation. Sequence performance for the bimodal and visual only training groups was similar, while both performed better than the tactile only training group. In a subsequent transfer phase, participants from all three training groups were tested in conditions with visual, tactile, and bimodal stimulus presentation. Sequence performance between the visual only and bimodal training groups again was highly similar across these identical stimulus conditions, indicating that the addition of tactile stimuli did not benefit the bimodal training group. Additionally, comparing across identical stimulus conditions in the transfer phase showed that the lesser sequence performance from the tactile only group during training probably did not reflect a difference in sequence learning but rather just a difference in expression of the sequence knowledge.

Working memory capacity,intelligence, and the magnitude of the attentional blink revisited

The attentional blink (AB) is a well-established phenomenon in the study of attention. This deficit in reporting the second of two targets presented in rapid serial visual presentation when it occurs 200–500 ms after the first is considered to reflect a fundamental limitation in attentional processing. However, we recently reported that some individuals do not show an AB, and presented psychophysiological evidence that target processing differs between blinkers and non-blinkers. One possibility is that non-blinkers may have a larger WM capacity, allowing better attentional control. Here we explore the relation between the magnitude of the AB, general intelligence, and different measures of working memory (WM) and short-term memory (STM) capacity. Surprisingly, no correlation was found between memory capacity measures and AB magnitude, raising doubts about the generalizability of earlier findings of such a relationship.

Adaptations of lateral hand movements to early and late visual occlusion in catching

Recent studies suggested that the control of hand movements in catching involves continuous vision-based adjustments. More insight into these adjustments may be gained by examining the effects of occluding different parts of the ball trajectory. Here, we examined the effects of such occlusion on lateral hand movements when catching balls approaching from different directions, with the occlusion conditions presented in blocks or in randomized order. The analyses showed that late occlusion only had an effect during the blocked presentation, and early occlusion only during the randomized presentation. During the randomized presentation movement biases were more leftward if the preceding trial was an early occlusion trial. The effect of early occlusion during the randomized presentation suggests that the observed leftward movement bias relates to the rightward visual acceleration inherent to the ball trajectories used, while its absence during the blocked presentation seems to reflect trial-by-trial adaptations in the visuomotor gain, reminiscent of dynamic gain control in the smooth pursuit system. The movement biases during the late occlusion block were interpreted in terms of an incomplete motion extrapolation—a reduction of the velocity gain—caused by the fact that participants never saw the to-be-extrapolated part of the ball trajectory. These results underscore that continuous movement adjustments for catching do not only depend on visual information, but also on visuomotor adaptations based on non-visual information.

Is visual dominance modulated by the threat value of visual and auditory stimuli?

The simultaneous presentation of a visual and an auditory stimulus can lead to a decrease in people’s ability to perceive or respond to the auditory stimulus. In this study, we investigate the effect that threat has upon this phenomenon, known as the Colavita visual dominance effect. Participants performed two blocks of trials containing 40% visual, 40% auditory, and 20% bimodal trials. The first block of trials was identical for all participants, while in the second block, either the visual stimulus (visual threat condition), auditory stimulus (auditory threat condition), or neither stimulus (control condition) was fear-conditioned using aversive electrocutaneous stimuli. We predicted that, when compared with the control condition, this visual dominance effect would increase in the visual threat condition and decrease in the auditory threat condition. This hypothesis was partially supported by the data. In particular, the results showed that the fear-conditioning of the visual stimulus significantly increased the visual dominance effect relative to the control condition. However, the fear-conditioning of the auditory stimulus did not reduce the visual dominance effect but instead increased it slightly. These findings are discussed in terms of the role that attention and arousal play in the dominance of vision over audition.

The role of binocular vision in grasping: a small stimulus-set distorts results

The role of binocular vision in grasping has frequently been assessed by measuring the effects on grasp kinematics of covering one eye. These studies have typically used three or fewer objects presented at three or fewer distances, raising the possibility that participants learn the properties of the stimulus set. If so, even relatively poor visual information may be sufficient to identify which object/distance configuration is presented on a given trial, in effect providing an additional source of depth information. Here we show that the availability of this uncontrolled cue leads to an underestimate of the effects of removing binocular information, and therefore to an overestimate of the effectiveness of the remaining cues. We measured the effects of removing binocular cues on visually open-loop grasps using (1) a conventional small stimulus-set, and (2) a large, pseudo-randomised stimulus set, which could not be learned. Removing binocular cues resulted in a significant change in grip aperture scaling in both conditions: peak grip apertures were larger (when reaching to small objects), and scaled less with increases in object size. However, this effect was significantly larger with the randomised stimulus set. These results confirm that binocular information makes a significant contribution to grasp planning. Moreover, they suggest that learned stimulus information can contribute to grasping in typical experiments, and so the contribution of information from binocular vision (and from other depth cues) may not have been measured accurately. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.

Randomizing visual feedback in manual aiming: reminiscence of the previous trial condition and prior knowledge of feedback availability

A trial-by-trial analysis was used to systematically examine the influence of switching visual conditions on visual feedback utilization for a manual aiming movement. In experiment one, vision was randomly manipulated from trial to trial with no more than four consecutive trials in the same visual condition. In experiment two, participants were provided with certainty of visual feedback availability prior to every trial. Results of both studies revealed that movement endpoint variability was most associated with visual feedback availability on the previous trial. Furthermore, correlation analyses comparing movement trajectory at 25, 50 and 75% with movement end (i.e. 100%) revealed that the efficiency of online corrections also depends on the availability of visual feedback on the previous trial. These results suggest that the accuracy of an aiming movement is highly dependent on processing of offline visual information from the preceding trial. This study was supported by a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC) awarded to Luc Tremblay.

---

Response preparation changes during practice of an asynchronous bimanual movement

For synchronous bimanual movements, we have shown that a different amplitude can be prepared for each limb in advance and this preparation improves with practice (Maslovat et al. 2008). In the present study, we tested whether an asynchronous bimanual movement can also be prepared in advance and be improved with practice. Participants practiced (160 trials) a discrete bimanual movement in which the right arm led the left by 100 ms in response to an auditory “go” signal (either 80 dB control stimulus or 124 dB startle stimulus). The startle stimulus was used to gauge whether inter-limb timing could be pre-programed. During startle trials, the asynchronous bimanual movement was triggered at early latency suggesting the entire movement could be prepared in advance. However, the triggered movement had a shorter between-arm delay and a temporally compressed within-arm EMG pattern, results that we attribute to increased neural activation caused by the startling stimulus. However, as both startle and control trials improved over time, it does appear response preparation of interval timing can improve with practice.

Effects of seeing and hearing speech on speech production: a response time study

Research demonstrates that listening to and viewing speech excites tongue and lip motor areas involved in speech production. This perceptual-motor relationship was investigated behaviourally by presenting video clips of a speaker producing vowel-consonant-vowel syllables in three conditions: visual-only, audio-only, and audiovisual. Participants identified target letters that were flashed over the mouth during the video, either manually or verbally as quickly as possible. Verbal responses were fastest when the target matched the speech stimuli in all modality conditions, yet optimal facilitation was observed when participants were presented with visual-only stimuli. Critically, no such facilitation occurred when participants were asked to identify the target manually. Our findings support previous research suggesting a close relationship between speech perception and production by demonstrating that viewing speech can ‘prime’ our motor system for subsequent speech production.

Attention modulates the specificity of automatic imitation to human actors

The perception of actions performed by others activates one’s own motor system. Recent studies disagree as to whether this effect is specific to actions performed by other humans, an issue complicated by differences in perceptual salience between human and non-human stimuli. We addressed this issue by examining the automatic imitation of actions stimulated by viewing a virtual, computer-generated, hand. This stimulus was held constant across conditions, but participants’ attention to the virtualness of the hand was manipulated by informing some participants during instructions that they would see a “computer-generated model of a hand,” while making no mention of this to others. In spite of this attentional manipulation, participants in both conditions were generally aware of the virtualness of the hand. Nevertheless, automatic imitation of the virtual hand was significantly reduced––but not eliminated––when participants were told they would see a virtual hand. These results demonstrate that attention modulates the “human bias” of automatic imitation to non-human actors.

The effect of altered gravity states on the perception of orientation

We measured the effect of the orientation of the visual background on the perceptual upright (PU) under different levels of gravity. Brief periods of micro- and hypergravity conditions were created using two series of parabolic flights. Control measures were taken in the laboratory under normal gravity with subjects upright, right side down and supine. Participants viewed a polarized, natural scene presented at various orientations on a laptop viewed through a hood which occluded all other visual cues. Superimposed on the screen was a character the identity of which depended on its orientation. The orientations at which the character was maximally ambiguous were measured and the perceptual upright was defined as half way between these orientations. The visual background affected the orientation of the PU less when in microgravity than when upright in normal gravity and more when supine than when upright in normal gravity. A weighted vector sum model was used to quantify the relative influence of the orientations of gravity, vision and the body in determining the perceptual upright.

Methodological problems undermine tests of the ideo-motor conjecture

Recent behavioural research has investigated whether viewing someone perform an action results in activation of that action by the observer. Postulated empirical support for this ‘ideo-motor (IM) conjecture’ typically rests upon two types of experimental paradigm (reaction time and movement tracking tasks). These paradigms purport to show movement facilitation when compatible movements are observed and vice versa, but only for biological stimuli. Unfortunately, these paradigms often contain confounding (and unavoidable) generic stimulus–response compatibility effects that are not restricted to observed human movement. The current study demonstrates in three experiments that equivalent compatibility effects can be produced by non-biological stimuli. These results suggest that existing empirical paradigms may not, and perhaps cannot, support the IM-conjecture.

The effect of sound intensity on the audiotactile crossmodal dynamic capture effect

We investigated the effect of varying sound intensity on the audiotactile crossmodal dynamic capture effect. Participants had to discriminate the direction of a target stream (tactile, Experiment 1; auditory, Experiment 2) while trying to ignore the direction of a distractor stream presented in a different modality (auditory, Experiment 1; tactile, Experiment 2). The distractor streams could either be spatiotemporally congruent or incongruent with respect to the target stream. In half of the trials, the participants were presented with auditory stimuli at 75 dB(A) while in the other half of the trials they were presented with auditory stimuli at 82 dB(A). Participants’ performance on both tasks was significantly affected by the intensity of the sounds. Namely, the crossmodal capture of tactile motion by audition was stronger with the more intense (vs. less intense) auditory distractors (Experiment 1), whereas the capture effect exerted by the tactile distractors was stronger for less intense (than for more intense) auditory targets (Experiment 2). The crossmodal dynamic capture was larger in Experiment 1 than in Experiment 2, with a stronger congruency effect when the target streams were presented in the tactile (vs. auditory) modality. Two explanations are put forward to account for these results: an attentional biasing toward the more intense auditory stimuli, and a modulation induced by the relative perceptual weight of, respectively, the auditory and the tactile signals.

Non-Newtonian Bile Flow in Elastic Cystic Duct: One- and Three-Dimensional Modeling

Bile flow is thought to play an essential role in the pathophysiological genesis of cholelithiasis (gallstone formation) and in gallbladder pain. In this paper, we extend our previous study of the human biliary system (Li et al., 2007, J. Biomech. Eng., 129:164–173) to include two important factors: the non-Newtonian properties of bile, and elastic deformation of the cystic duct. A one-dimensional (1D) model is analyzed and compared with three-dimensional (3D) fluid–structure interaction simulations. It is found that non-Newtonian bile raises resistance to the flow of bile, which can be augmented significantly by the elastic deformation (collapse) of the cystic duct. We also show that the 1D model predicts the pressure drop of the cystic duct flow well for all cases considered (Newtonian or non-Newtonian flow, rigid or elastic ducts), when compared with the full 3D simulations.

Semantic congruency and the Colavita visual dominance effect

Participants presented with auditory, visual, or bimodal audiovisual stimuli in a speeded discrimination task, fail to respond to the auditory component of bimodal targets significantly more often than to the visual component, a phenomenon known as the Colavita visual dominance effect. Given that spatial and temporal factors have recently been shown to modulate the Colavita effect, the aim of the present study, was to investigate whether semantic congruency also modulates the effect. In the three experiments reported here, participants were presented with a version of the Colavita task in which the stimulus congruency between the auditory and visual components of the bimodal targets was manipulated. That is, the auditory and visual stimuli could refer to the same or different object (in Experiments 1 and 2) or audiovisual speech event (Experiment 3). Surprisingly, semantic/stimulus congruency had no effect on the magnitude of the Colavita effect in any of the experiments, although it exerted a significant effect on certain other aspects of participants’ performance. This finding contrasts with the results of other recent studies showing that semantic/stimulus congruency can affect certain multisensory interactions.

Evidence for an attentional component in saccadic inhibition of return

After presentation of a peripheral cue, facilitation at the cued location is followed by inhibition of return (IOR). It has been recently proposed that IOR may originate at different processing stages for manual and ocular responses, with manual IOR resulting from inhibited attentional orienting, and ocular IOR resulting form inhibited motor preparation. Contrary to this interpretation, we found an effect of target contrast on saccadic IOR. The effect of contrast decreased with increasing reaction times (RTs) for saccades, but not for manual key-press responses. This may have masked the effect of contrast on IOR with saccades in previous studies (Hunt and Kingstone in J Exp Psychol Hum Percept Perform 29:1068–1074, 2003) because only mean RTs were considered. We also found that background luminance strongly influenced the effects of gap and target contrast on IOR.

The relationship between poor sleep and inhibitory functions indicated by event-related potentials

The present study focused on the relationship between normal variations of sleep and inhibitory functions as reflected in event-related potentials. For this reason one night of 21 healthy participants was analysed. After waking up all participants completed a visual Go/Nogo task. On the basis of a sleep disturbance index (SDI) the participants were separated into 8 SDI-good and 13 SDI-poor sleepers using a cluster analysis. The results showed that Nogo-N2 amplitude was smaller and Nogo-P3 latency longer in SDI-poor sleepers. Moreover, Go-P3 amplitude was smaller in SDI-poor sleepers. Performance parameters were not influenced by poor sleep. We concluded that poor sleep specifically affects the intensity of pre-motor inhibitory processes (Nogo-N2 amplitude), the speed to inhibit a motor response (Nogo-P3 latency) and the intensity of task-relevant information processing (Go-P3 amplitude). In further studies, it should be explored under which conditions such subliminal deficits also become relevant for overt behaviour.