Flexible use of post-saccadic visual feedback in oculomotor learning

Saccadic eye movements bring objects of interest onto our fovea. These gaze shifts are essential for visual perception of our environment and the interaction with the objects within it. They precede our actions and are thus modulated by current goals. It is assumed that saccadic adaptation, a recalibration process that restores saccade accuracy in case of error, is mainly based on an implicit comparison of expected and actual post-saccadic position of the target on the retina. However, there is increasing evidence that task demands modulate saccade adaptation and that errors in task performance may be sufficient to induce changes to saccade amplitude. We investigated if human participants are able to flexibly use different information sources within the post-saccadic visual feedback in task-dependent fashion. Using intra-saccadic manipulation of the visual input, participants were either presented with congruent post-saccadic information, indicating the saccade target unambiguously, or incongruent post-saccadic information, creating conflict between two possible target objects. Using different task instructions, we found that participants were able to modify their saccade behavior such that they achieved the goal of the task. They succeeded in decreasing saccade gain or maintaining it, depending on what was necessary for the task, irrespective of whether the post-saccadic feedback was congruent or incongruent. It appears that action intentions prime task-relevant feature dimensions and thereby facilitated the selection of the relevant information within the post-saccadic image. Thus, participants use post-saccadic feedback flexibly, depending on their intentions and pending actions.     

Transsaccadic integration relies on a limited memory resource

Saccadic eye movements cause large-scale transformations of the image falling on the retina. Rather than starting visual processing anew after each saccade, the visual system combines post-saccadic information with visual input from before the saccade. Crucially, the relative contribution of each source of information is weighted according to its precision, consistent with principles of optimal integration. We reasoned that, if pre-saccadic input is maintained in a resource-limited store, such as visual working memory, its precision will depend on the number of items stored, as well as their attentional priority. Observers estimated the color of stimuli that changed imperceptibly during a saccade, and we examined where reports fell on the continuum between pre- and post-saccadic values. Bias toward the post-saccadic color increased with the set size of the pre-saccadic display, consistent with an increased weighting of the post-saccadic input as precision of the pre-saccadic representation declined. In a second experiment, we investigated if transsaccadic memory resources are preferentially allocated to attentionally prioritized items. An arrow cue indicated one pre-saccadic item as more likely to be chosen for report. As predicted, valid cues increased response precision and biased responses toward the pre-saccadic color. We conclude that transsaccadic integration relies on a limited memory resource that is flexibly distributed between pre-saccadic stimuli.     

Post-saccadic changes disrupt attended pre-saccadic object memory

Trans-saccadic memory consists of keeping track of objects’ locations and features across saccades; pre-saccadic information is remembered and compared with post-saccadic information. It has been shown to have limited resources and involve attention with respect to the selection of objects and features. In support, a previous study showed that recognition of distinct post-saccadic objects in the visual scene is impaired when pre-saccadic objects are relevant and thus already encoded in memory (Poth, Herwig, Schneider, 2015). Here, we investigated the inverse (i.e. how the memory of pre-saccadic objects is affected by abrupt but irrelevant changes in the post-saccadic visual scene). We also modulated the amount of attention to the relevant pre-saccadic object by having participants either make a saccade to it or elsewhere and observed that pre-saccadic attentional facilitation affected how much post-saccadic changes disrupted trans-saccadic memory of pre-saccadic objects.Participants identified a flashed symbol (d, b, p, or q, among distracters), at one of six placeholders (figures “8”) arranged in circle around fixation while planning a saccade to one of them. They reported the identity of the symbol after the saccade. We changed the post-saccadic scene in Experiment one by removing the entire scene, only the placeholder where the pre-saccadic symbol was presented, or all other placeholders except this one. We observed reduced identification performance when only the saccade-target placeholder disappeared after the saccade. In Experiment two, we changed one placeholder location (inward/outward shift or rotation re. saccade vector) after the saccade and observed that identification performance decreased with increased shift/rotation of the saccade-target placeholder. We conclude that pre-saccadic memory is disrupted by abrupt attention-capturing post-saccadic changes of visual scene, particularly when these changes involve the object prioritized by being the goal of a saccade. These findings support the notion that limited trans-saccadic memory resources are disrupted when object correspondence at saccadic goal is broken through removal or location change.     

Distorted object perception following whole-field adaptation of saccadic eye movements

The adaptation of an observer's saccadic eye movements to artificial post-saccadic visual error can lead to perceptual mislocalization of individual, transient visual stimuli. In this study, we demonstrate that simultaneous saccadic adaptation to a consistent error pattern across a large number of saccade vectors is accompanied by corresponding spatial distortions in the perception of persistent objects. To induce this adaptation, we artificially introduced several post-saccadic error patterns, which led to a systematic distortion in participants' oculomotor space and a corresponding distortion in their perception of the relative dimensions of a cross-figure. The results indicate a tight coupling between the oculomotor and visual-perceptual spaces that is not limited to misperception of individual visual locations but also affects metrics in the visual-perceptual space. This coupling suggests that our visual perception is continuously recalibrated by the post-saccadic error signal.     

Disconjugate Adaptation of the Vertical Oculomotor System

Conjugate post-saccadic eye drift can be induced in normal humans if a visual pattern is made to drift after every saccade. This study examines the ability of normal humans to create disconjugate vertical post-saccadic drift. Identical fuseable patterns were presented dichoptically, one to each eye. At the end of each vertical saccade one pattern drifted up and the other down, by 5% of the saccade amplitude. Five subjects were trained for 2–3 hr. Eye movements were recorded with eye coils. Normal vertical saccades along the midline were remarkably conjugate and post-saccadic drift was minimal. Training produced only small disconjugate post-saccadic drift (0.14 deg) but substantial saccade amplitude disconjugacy (0.70 deg). For several subjects, the induced disconjugacies persisted even for saccades in the dark indicating that adaptive changes occurred in the binocular coordination of vertical saccades. Apparently vertical disparate post-saccadic retinal slip is not sufficient to stimulate significantly the saccade pulse-step matching mechanism which is believed to control post-saccadic eye drift. The changes we observed aimed to reduce position disparity and not retinal slip in each eye. Copyright © 1996 Elsevier Science Ltd.     

Saccade adaptation is unhampered by distractors

Saccade adaptation has been extensively studied using a paradigm in which a target is displaced during the saccade, inducing an adjustment in saccade amplitude or direction. These changes in saccade amplitude are widely considered to be controlled by the post-saccadic position of the target relative to the fovea. However, because such experiments generally employ only a single target on an otherwise blank screen, the question remains whether the same adaptation could occur if both the target and a similar distractor were present when the saccade landed. To investigate this issue, three experiments were conducted, in which the post-saccadic locations of the target and distractor were varied. Results showed that decreased amplitude adaptation, increased amplitude adaptation, and recovery from adaptation were controlled by the post-saccadic position of the target rather than the distractor. These results imply that target selection is critical to saccade adaptation.     

Oculomotor corollary discharge signaling is related to repetitive behavior in children with autism spectrum disorder

Corollary discharge (CD) signals are “copies” of motor signals sent to sensory regions that allow animals to adjust sensory consequences of self-generated actions. Autism spectrum disorder (ASD) is characterized by sensory and motor deficits, which may be underpinned by altered CD signaling. We evaluated oculomotor CD using the blanking task, which measures the influence of saccades on visual perception, in 30 children with ASD and 35 typically developing (TD) children. Participants were instructed to make a saccade to a visual target. Upon saccade initiation, the presaccadic target disappeared and reappeared to the left or right of the original position. Participants indicated the direction of the jump. With intact CD, participants can make accurate perceptual judgements. Otherwise, participants may use saccade landing site as a proxy of the presaccadic target and use it to inform perception. We used multilevel modeling to examine the influence of saccade landing site on trans-saccadic perceptual judgements. We found that, compared with TD participants, children with ASD were more sensitive to target displacement and less reliant on saccade landing site when spatial uncertainty of the post-saccadic target was high. This pattern was driven by ASD participants with less severe restricted and repetitive behaviors. These results suggest a relationship between altered CD signaling and core ASD symptoms.     

Object recognition during foveating eye movements

We studied how saccadic and smooth pursuit eye movements affect the recognition of briefly presented letters appearing within the eye movement target. First we compared the recognition performance during steady-state pursuit and during fixation. Single letters were presented for seven different durations ranging from 10 to 400 ms and four contrast levels ranging from 5% to 40%. For both types of eye movements the recognition rates increased with duration and contrast, but they were on average 11% lower during pursuit. In daily life humans use a combination of saccadic and smooth pursuit eye movements to foveate a peripheral moving object. To investigate this more natural situation, we presented a peripheral target that was either stationary or moving horizontally, above or below the fixation spot. Participants were asked to saccade to the target and to keep it foveated. The letters were presented at different times relative to the first target directed saccade. As would be expected from retinal masking and motion blur during saccades, the discrimination performance increased with increasing post-saccadic delay. If the target moved and the saccade was followed by pursuit, letter recognition performance was on average 16% lower than if the target was stationary and the saccade was followed by fixation.     

Movement perception during voluntary saccadic eye movements

In order to elucidate the stabilization mechanism of the visual perception during voluntary eye movements the perception of a moving object during a eye saccade was investigated in human subjects. The results were analyzed in terms of velocity and displacement perception channels. The experimental results indicate that during voluntary saccades there is no suppression of movement perception, and, on the other hand, that in darkness the direction of the perceived movement is that of the saccade whereas in light the direction of the objective movement prevails. The latter discrepancy can be explained by assuming that the perceptual analysis of movement during the saccade occurs in the displacement channel and that this channel is provided with a channel evaluation mechanism controlled by the oculomotor centers.     

Post-saccadic oscillations in eye movement data recorded with pupil-based eye trackers reflect motion of the pupil inside the iris

Current video eye trackers use information about the pupil center to estimate orientation and movement of the eye. While dual Purkinje eye trackers suffer from lens wobble and scleral search coils may be influenced by contact lens slippage directly after saccades, it is not known whether pupil-based eye trackers produces similar artifacts in the data. We recorded eye movements from participants making repetitive, horizontal saccades and compared the movement in the data with pupil- and iris movements extracted from the eye images. Results showed that post-saccadic instabilities clearly exist in data recorded with a pupil-based eye tracker. They also exhibit a high degree of reproducibility across saccades and within participants. While the recorded eye movement data correlated well with the movement of the pupil center, the iris center showed only little post-saccadic movement. This means that the pupil moves relative to the iris during post-saccadic eye movements, and that the eye movement data reflect pupil movement rather than eyeball rotation. Besides introducing inaccuracies and additional variability in the data, the pupil movement inside the eyeball influences the decision of when a saccade should end and the subsequent fixation should begin, and consequently higher order analyses based on fixations and saccades.     

Mislocalization after inhibition of saccadic adaptation

Saccadic eye movements are often imprecise and result in an error between expected and actual retinal target location after the saccade. Repeated experience of this error produces changes in saccade amplitude to reduce the error and concomitant changes in apparent visual location. We investigated the relationship between these two plastic processes in a series of experiments. Following a recent paradigm of inhibition of saccadic adaptation, in which participants are instructed to look at the initial target position and to continue to look at that position even if the target were to move again, our participants nevertheless perceived a visual probe presented near the saccade target to be shifted in direction of the target error. The location percept of the target gradually shifted and diverged over time from the executed saccade. Our findings indicate that changes in perceived location can be the same even when changes in saccade amplitude differ according to instruction and can develop even when the amplitude of the saccades executed during the adaptation procedure does not change. There are two possible explanations for this divergence between the adaptation states of saccade amplitude and perceived location. Either the intrasaccadic target step might trigger updating of the association between pre- and post-saccadic target positions, causing the localization shift, or the saccade motor command adjusts together with the perceived location at a common adaptation site, downstream from which voluntary control is exerted upon the executed eye movement only.     

Saccadic suppression of displacement in face of saccade adaptation

Saccades challenge visual perception since they induce large shifts of the image on the retina. Nevertheless, we perceive the outer world as being stable. The saccadic system also can rapidly adapt to changes in the environment (saccadic adaptation). In such case, a dissociation is introduced between a driving visual signal (the original saccade target) and a motor output (the adapted saccade vector). The question arises, how saccadic adaptation interferes with perceptual visual stability. In order to answer this question, we engaged human subjects in a saccade adaptation paradigm and interspersed trials in which the saccade target was displaced perisaccadically to a random position. In these trials subjects had to report on their perception of displacements of the saccade target. Subjects were tested in two conditions. In the ‘blank’ condition, the saccade target was briefly blanked after the end of the saccade. In the ‘no-blank’ condition the target was permanently visible. Confirming previous findings, the visual system was rather insensitive to displacements of the saccade target in an unadapted state, an effect termed saccadic suppression of displacement (SSD). In all adaptation conditions, we found spatial perception to correlate with the adaptive changes in saccade landing site. In contrast, small changes in saccade amplitude that occurred on a trial by trial basis did not correlate with perception. In the ‘no-blank’ condition we observed a prominent increase in suppression strength during backward adaptation. We discuss our findings in the context of existing theories on transsaccadic perceptual stability and its neural basis.     

Poor binocular coordination of saccades in dyslexic children

Aim To examine the quality of binocular coordination of saccades in dyslexic children in single word reading and in a task requiring fixation of single LED. Methods Eighteen children with dyslexia (11.4 ± 2 years old) and 13 non-dyslexic children of matched age were studied. Horizontal saccades from both eyes were recorded with a photoelectric system (Oculomotor-Bouis). Results Binocular coordination during and after the saccade in dyslexics is worse than that of non-dyslexic children; the disconjugacy does not depend on the condition. Moreover, dyslexics do not show the stereotyped pattern of disconjugacy (divergence during the saccade and convergence after the saccade). The conjugate post-saccadic drift is larger in dyslexics for both conditions. Conclusion Poor quality of binocular coordination of saccades and drift of the eyes after the saccade, regardless of the task, indicates an intrinsic ocular motor deficiency. Such a deficiency could be related to immaturity of the normal ocular motor learning mechanisms via which ocular motor coordination and stable fixation are achieved. Learning could be based on the interaction between the saccade and vergence subsystems. The cerebellum, but also cortical areas of the magnocellular stream such as the parietal cortex, could be the sites of ocular motor learning.     

Comparison of performance on memory-guided saccade and delayed spatial match-to-sample tasks in monkeys

To investigate the sources of spatial error in memory-guided saccades (MGS), we have trained monkeys on two different tasks: a MGS task and a delayed spatial match-to-sample (MTS) task. We first tested the effect of introducing a post-saccadic visual feedback on the accuracy of MGS. We found that visual feedback had a pronounced effect on the systematic saccade error, but less of an effect on the variable error. Visual feedback can improve the accuracy of saccadic eye movements over several days, while feedback removal can decrease accuracy in a reversible way. These effects also depend both on target eccentricity and the duration of the memory delay. To test whether saccade error is due to the accuracy of spatial memory storage or arises downstream from that memory, we measured behavioral performance on a spatial MTS task both before and after training with visual feedback. The results showed no significant difference in performance of the MTS task before and after feedback training despite significant changes in MGS accuracy. The results suggest that the accuracy of spatial memory is not the source of the systematic errors that accompany MGS.     

The spatial distribution of receptive field changes in a model of peri-saccadic perception: Predictive remapping and shifts towards the saccade target

At the time of an impending saccade receptive fields (RFs) undergo dynamic changes, that is, their spatial profile is altered. This phenomenon has been observed in several monkey visual areas. Although their link to eye movements is obvious, neither the exact pattern nor their function is fully clear. Several RF shifts have been interpreted in terms of predictive remapping mediating visual stability. In particular, even prior to saccade onset some cells become responsive to stimuli presented in their future, post-saccadic RF. In visual area V4, however, the overall effect of RF dynamics consists of a shrinkage and shift of RFs towards the saccade target. These observations have been linked to a pre-saccadically enhanced processing of the future fixation. In order to better understand these seemingly different outcomes, we analyzed the RF shifts predicted by a recently proposed computational model of peri-saccadic perception (Hamker, Zirnsak, Calow, & Lappe, 2008). This model unifies peri-saccadic compression, pre-saccadic attention shifts, and peri-saccadic receptive field dynamics in a common framework of oculomotor reentry signals in extrastriate visual cortical maps. According to the simulations that we present in the current paper, a spatially selective oculomotor feedback signal leads to RF dynamics which are both consistent with the observations made in studies aiming to investigate predictive remapping and saccade target shifts. Thus, the seemingly distinct experimental observations could be grounded in the same neural mechanism leading to different RF dynamics dependent on the location of the RF in visual space.     

Transsaccadic integration of visual information is predictive,attention-based,and spatially precise

Eye movements produce shifts in the positions of objects in the retinal image, but observers are able to integrate these shifting retinal images into a coherent representation of visual space. This ability is thought to be mediated by attention-dependent saccade-related neural activity that is used by the visual system to anticipate the retinal consequences of impending eye movements. Previous investigations of the perceptual consequences of this predictive activity typically infer attentional allocation using indirect measures such as accuracy or reaction time. Here, we investigated the perceptual consequences of saccades using an objective measure of attentional allocation, reverse correlation. Human observers executed a saccade while monitoring a flickering target object flanked by flickering distractors and reported whether the average luminance of the target was lighter or darker than the background. Successful task performance required subjects to integrate visual information across the saccade. A reverse correlation analysis yielded a spatiotemporal “psychophysical kernel” characterizing how different parts of the stimulus contributed to the luminance decision throughout each trial. Just before the saccade, observers integrated luminance information from a distractor located at the post-saccadic retinal position of the target, indicating a predictive perceptual updating of the target. Observers did not integrate information from distractors placed in alternative locations, even when they were nearer to the target object. We also observed simultaneous predictive perceptual updating for two spatially distinct targets. These findings suggest both that shifting neural representations mediate the coherent representation of visual space, and that these shifts have significant consequences for transsaccadic perception.     

Predictability of spatial and non-spatial target properties improves perception in the pre-saccadic interval

In a dual-task paradigm with a perceptual discrimination task and a concurrent saccade task, we examined participants’ ability to make use of prior knowledge of a critical property of the perceptual target to improve discrimination. Previous research suggests that during a short time window before a saccade, covert attention is imperatively directed towards the saccade target location. Consequently, discrimination of perceptual targets at the saccade target location is better than at other locations. We asked whether the obligatory pre-saccadic attention shift prevents perceptual benefits arising for perceptual target stimuli with predictable as opposed to non-predictable properties. We compared conditions in which the color or location of the perceptual target was constant to conditions in which those properties varied randomly across trials. In addition to the expected improvements of perception at the saccade target location, we found perception to be better with constant than with random properties of the perceptual target. Thus, color or location information about an upcoming perceptual target facilitates perception even while spatial attention is shifted to the saccade target. The improvement occurred irrespective of the saccade target location, which suggests that the underlying mechanism is independent of the pre-saccadic attention shift, but alternative interpretations are discussed as well.     

Continuous perception of motion and shape across saccadic eye movements

Although our na?ve experience of visual perception is that it is smooth and coherent, the actual input from the retina involves brief and discrete fixations separated by saccadic eye movements. This raises the question of whether our impression of stable and continuous vision is merely an illusion. To test this, we examined whether motion perception can "bridge" a saccade in a two-frame apparent motion display in which the two frames were separated by a saccade. We found that transformational apparent motion, in which an object is seen to change shape and even move in three dimensions during the motion trajectory, continues across saccades. Moreover, participants preferred an interpretation of motion in spatial, rather than retinal, coordinates. The strength of the motion percept depended on the temporal delay between the two motion frames and was sufficient to give rise to a motion-from-shape aftereffect, even when the motion was defined by a second-order shape cue ("phantom transformational apparent motion"). These findings suggest that motion and shape information are integrated across saccades into a single, coherent percept of a moving object.     

Saccadic omission: Why we do not see a grey-out during a saccadic eye movement

We investigated why we have no perception of a smeared image resulting from the reduction in contrast (grey-out) occurring at the retina during saccadic eye movements. By turning on light in the experimental room only during the eye movement, we were able to show that this grey-out was perceived as a smeared image of the visual scene. However, when the experimental room was illuminated before and/or after the saccade as well as during the saccade, perception of the grey-out was obliterated. During a period of fixation, perception of a blank image comparable in duration to an eye movement could also be eliminated by a preceding or following clear image. We conclude that lack of perception during saccadic eye movements made in normal contoured environments results primarily from the visual “masking” effect of a clear image before and/or after the eye movement acting on the grey-out during the eye movement. This “saccadic omission” is entirely a visual phenomenon and is far more powerful than the usually studied elevation of visual threshold for detection of a flash, “saccadic suppression.”     

Revisiting corrective saccades: Role of visual feedback

To clarify the role of visual feedback in the generation of corrective movements after inaccurate primary saccades, we used a visually-triggered saccade task in which we varied how long the target was visible. The target was on for only 100 ms (OFF_100ms), on until the start of the primary saccade (OFF_onset) or on for 2 s (ON). We found that the tolerance for the post-saccadic error was small (−2%) with a visual signal (ON) but greater (−6%) without visual feedback (OFF_100ms). Saccades with an error of −10%, however, were likely to be followed by corrective saccades regardless of whether or not visual feedback was present. Corrective saccades were generally generated earlier when visual error information was available; their latency was related to the size of the error. The LATER (Linear Approach to Threshold with Ergodic Rate) model analysis also showed a comparable small population of short latency corrective saccades irrespective of the target visibility. Finally, we found, in the absence of visual feedback, the accuracy of corrective saccades across subjects was related to the latency of the primary saccade. Our findings provide new insights into the mechanisms underlying the programming of corrective saccades: (1) the preparation of corrective saccades begins along with the preparation of the primary saccades, (2) the accuracy of corrective saccades depends on the reaction time of the primary saccades and (3) if visual feedback is available after the initiation of the primary saccade, the prepared correction can be updated.