Parametric integration of visual form across saccades

Through saccadic eye movements, the retinal projection of an extrafoveally glimpsed object can be brought into foveal vision quickly. We investigated what influence visual detail collected before the saccade exerts on the postsaccadic percept. Participants were instructed to saccade towards a peripheral stimulus, and to indicate on a continuum of ellipses with varying aspect ratios which exact shape they had perceived to be present after saccade landing. Compared to both an identical ellipse preview and a qualitatively different square preview, a quantitatively different ellipse preview was observed to shift the mean postsaccadic percept towards the presaccadic aspect ratio parameter value. This integration of subtly different form information was accompanied by an integration of the identity of both stimuli presented: In the great majority of these trials, subjects indicated that they had not noticed the occurrence of a change to the stimulus. When a blank screen preceded the postsaccadic stimulus onset the influence of presaccadic stimulus information on postsaccadic perception was weaker. An immediate postsaccadic mask on the other hand abolished the effect entirely. We conclude that integration of parametric visual form information occurs across saccades, but that it relies on a quickly decaying and maskable visual memory.     

Tracking an occluded visual target with sequences of saccades

Gaze behavior during visual tracking consists of a combination of pursuit and saccadic movements. When the tracked object is intermittently occluded, the role of smooth pursuit is reduced, with a corresponding increase in the role of saccades. However, studies of visual tracking during occlusion have focused only on the first few saccades, usually with occlusion periods of less than 1 second in duration. We investigated tracking on a circular trajectory with random occlusions and found that an occluded object can be tracked reliably for up to several seconds with mainly anticipatory saccades and very little smooth pursuit. Furthermore, we investigated the accumulation of uncertainty in prediction and found that prediction errors seem to accumulate faster when an absolute reference frame is not available during tracking. We suggest that the observed saccadic tracking reflects the use of a time-based internal estimate of object position that is anchored to the environment via fixations.     

Judging relative positions across saccades

When components of a shape are presented asynchronously during smooth pursuit, the retinal image determines the perceived shape, as if the parts belong to the moving object that the eyes are pursuing. Saccades normally shift our gaze between structures of interest, so there is no reason to expect anything to have moved with the eyes. We therefore decided to examine how people judge the separation between a target flashed before and another flashed after a saccade. Subjects tracked a jumping dot with their eyes. Targets were flashed at predetermined retinal positions, with a 67-242 ms interval between the flashes. After each trial subjects indicated where they had seen the targets. We selected the trials on which subjects made a complete saccade between the presentations of the two targets. For short inter-target intervals, subjects' judgements depended almost exclusively on the retinal separation, even when there were conspicuous visual references nearby. Even for the longest intervals, only part of the change in eye orientation was taken into consideration. These findings cannot simply be accounted for on the basis of the mislocalisation of individual targets or a compression of space near saccades. We conclude that the retinal separation determines the perceived separation between targets presented with a short interval between them, irrespective of any intervening eye movements.     

Asymmetries in the direction of saccades during perception of scenes and fractals: Effects of image type and image features

The direction in which people tend to move their eyes when inspecting images can reveal the different influences on eye guidance in scene perception, and their time course. We investigated biases in saccade direction during a memory-encoding task with natural scenes and computer-generated fractals. Images were rotated to disentangle egocentric and image-based guidance. Saccades in fractals were more likely to be horizontal, regardless of orientation. In scenes, the first saccade often moved down and subsequent eye movements were predominantly vertical, relative to the scene. These biases were modulated by the distribution of visual features (saliency and clutter) in the scene. The results suggest that image orientation, visual features and the scene frame-of-reference have a rapid effect on eye guidance.     

Differential detection of global luminance and contrast changes across saccades and flickers during active scene perception

How sensitive are viewers to changes in global image properties across saccades during active real-world scene perception? This question was investigated by globally increasing and/or decreasing luminance or contrast in photographs of real-world scenes across saccadic eye movements or during matched brief interruptions in a flicker paradigm. The results from two experiments demonstrated very poor sensitivity to global image changes in both the saccade-contingent and flicker paradigms, suggesting that the specific values of basic sensory properties do not contribute to the perception of stability across saccades during complex scene perception. In addition, overall sensitivity was significantly worse in the saccade-contingent change paradigm than the flicker paradigm, suggesting that the flicker paradigm is an imperfect simulation of transsaccadic vision.     

Spatiotopic updating across saccades in the absence of awareness

Despite the continuously changing visual inputs caused by eye movements, our perceptual representation of the visual world remains remarkably stable. Visual stability has been a major area of interest within the field of visual neuroscience. The early visual cortical areas are retinotopic-organized, and presumably there is a retinotopic to spatiotopic transformation process that supports the stable representation of the visual world. In this study, we used a cross-saccadic adaptation paradigm to show that both the orientation adaptation and face gender adaptation could still be observed at the same spatiotopic (but different retinotopic) locations even when the adapting stimuli were rendered invisible. These results suggest that awareness of a visual object is not required for its transformation from the retinotopic to the spatiotopic reference frame.     

Landmarks facilitate visual space constancy across saccades and during fixation

It has been demonstrated that visual objects that are present after saccadic eye movements act as landmarks for the localization of stimuli across saccades, facilitating space constancy (Deubel, 2004). We here study the temporal conditions under which landmark effects occur after saccadic eye movements, and during fixation. Two small objects were presented 6° in the periphery, one above the other. Observers saccaded to the space between them. One of the objects disappeared during the saccade and reappeared with a variable delay during or after the saccade. At the same time either that object or the continuously present one jumped by 1°. The observer’s task was to decide which object had moved. The results revealed a strong bias to assign movement to the object that was blanked, regardless of which actually moved. If both objects were blanked, the one that was blanked for a shorter time tended to be seen as stable. The effects were stronger as the onset asynchrony between the stimuli increased. Surprisingly, analogous though weaker effects occurred during visual fixation, suggesting that similar visual mechanisms relying on visual landmarks operate both across saccades and during fixation.     

Motion thresholds of briefly visible stimuli increase asymmetrically with age

During a pursuit eye movement made across a stationary stimulus, that stimulus is often perceived as moving slightly in the direction opposite to the eyes (Filehne illusion). The illusion is generally thought to increase in strength when the stimulus is made visible only briefly. In two experiments the illusion was indeed observed with young subjects. However, with older subjects brief stimulus presentations yielded a strong inverted Filehne illusion (the stimulus appeared to move in the same direction as the eyes). This age dependency of the Filehne illusion is caused by an increase of only the threshold for stimulus motion in the direction opposite to the eyes. No such effect happens with the threshold for stimulus motion in the same direction as the eyes. These findings can be explained if we assume that with increasing age it takes more time to properly register retinal image velocity within the perceptual system.     

Adaptive deployment of spatial and feature-based attention before saccades

What you see depends not only on where you are looking but also on where you will look next. The pre-saccadic attention shift is an automatic enhancement of visual sensitivity at the target of the next saccade. We investigated whether and how perceptual factors independent of the oculomotor plan modulate pre-saccadic attention within and across trials. Observers made saccades to one (the target) of six patches of moving dots and discriminated a brief luminance pulse (the probe) that appeared at an unpredictable location. Sensitivity to the probe was always higher at the target’s location (spatial attention), and this attention effect was stronger if the previous probe appeared at the previous target’s location. Furthermore, sensitivity was higher for probes moving in directions similar to the target’s direction (feature-based attention), but only when the previous probe moved in the same direction as the previous target. Therefore, implicit cognitive processes permeate pre-saccadic attention, so that–contingent on recent experience–it flexibly distributes resources to potentially relevant locations and features.     

The effect of three different visual monitoring strategies on the accuracy of leg‐before‐wicket decisions by cricket umpires

Background: Leg‐before‐wicket (LBW) dismissals in cricket require exacting visual judgements by umpires, a task complicated by the requirement to also adjudicate on where the bowler's front foot is relative to the popping crease line about the time the ball leaves the bowler's hand. The umpire must then judge as soon as possible afterwards whether the direction of the flight of the ball is in line with the batsman's stumps. This study investigated whether the accuracy of cricket umpires' decision‐making in leg‐before‐wicket dismissals is affected by varying the method by which umpires monitor bowlers' feet at the point of delivery of the ball. Methods: Four umpires officiating under simulated match conditions reported their judgements of whether each delivery they observed pitched in line with the stumps. They did so under one of three conditions: watching the bowler's front foot, watching the bowler's back foot prior to the delivery of the ball by the bowler or not monitoring the bowler's feet (‘no foot’ condition). Video recording, assisted by the use of superimposed stump‐to‐stump lines, was used to assess the accuracy of the umpires' responses. Results: There was no statistically significant difference in umpires' performance when comparing the front foot condition to the back foot condition but performance for the ‘no foot’ condition was significantly better than for the front foot condition. Conclusions: These results suggest that umpires' performance judging LBW dismissals would be improved if they did not have to monitor bowlers' feet to adjudicate ‘no‐ball’ deliveries but there would be no benefit from a reversion from the current ‘front foot’ no‐ball law to the previously used back foot law.     

Localization of speed differences of context stimuli during fixation and smooth pursuit eye movements

The visual system can detect speed changes of moving objects only by means of alterations of retinal image motion, which is also subject to changes induced by head or eye movements. Here we investigated whether smooth pursuit eye movements affect the ability to localize short speed perturbations of large context stimuli. Psychophysical thresholds for localization, discrimination and detection of speed perturbations in one of two context stimuli were measured under two main conditions: in fixation trials subjects fixated a central stationary spot, in pursuit trials they followed a horizontally moving target with their eyes. Context stimuli were vertically oriented sine wave gratings moving simultaneously above and below the fixation or pursuit target for one second in the same direction at the same or a different speed as the pursuit target. During the movement one of the gratings suddenly changed its speed for 500 ms and returned to its original speed. Observers were asked to discern the location of the speed change (two-alternative spatial forced choice task). While detection (two-interval forced choice) and discrimination thresholds for the kind of speed perturbation were in the normal range of Weber fractions of 10–15%, thresholds for the location of the speed perturbation were dramatically increased to 30–50%. Localization thresholds were particularly high when the retinal motion was mainly due to the context movements as during fixation or slow pursuit and significantly reduced when the retinal motion was mainly due to pursuit. This result indicates that the origin of retinal motion, whether it is caused by object motion or by voluntary pursuit is important. We conclude that the localization of speed perturbations affecting one of two peripheral moving objects is exceedingly complicated for the visual system probably due to the dominance of relative motion. During smooth pursuit the ability to localize speed perturbations of non-foveated objects seems to be improved by additional information gained from pursuit such as corollary discharge.     

The urgency to look: prompt saccades to the benefit of perception

Researchers have shown that the promptness to initiate a saccade is modulated by countless factors pertaining to the visual context and the task. However, experiments on saccadic eye movements are usually designed in such a way that oculomotor performance is dissociated from the natural role of saccades, namely that of making crucial perceptual information rapidly available for high-resolution, foveal analysis. Here, we demonstrate that the requirement to perform a difficult perceptual judgment at the saccade landing location can reduce saccadic latency (by >15%) and increase saccadic peak velocity. Importantly, the effect cannot be explained in terms of arousal, as latency changes are specific to the location where the perceptual judgement is required. These results indicate that mechanisms for voluntary saccade initiation are under the powerful indirect control of perceptual goals.     

Object recognition during foveating eye movements

We studied how saccadic and smooth pursuit eye movements affect the recognition of briefly presented letters appearing within the eye movement target. First we compared the recognition performance during steady-state pursuit and during fixation. Single letters were presented for seven different durations ranging from 10 to 400 ms and four contrast levels ranging from 5% to 40%. For both types of eye movements the recognition rates increased with duration and contrast, but they were on average 11% lower during pursuit. In daily life humans use a combination of saccadic and smooth pursuit eye movements to foveate a peripheral moving object. To investigate this more natural situation, we presented a peripheral target that was either stationary or moving horizontally, above or below the fixation spot. Participants were asked to saccade to the target and to keep it foveated. The letters were presented at different times relative to the first target directed saccade. As would be expected from retinal masking and motion blur during saccades, the discrimination performance increased with increasing post-saccadic delay. If the target moved and the saccade was followed by pursuit, letter recognition performance was on average 16% lower than if the target was stationary and the saccade was followed by fixation.     

Mislocalization after inhibition of saccadic adaptation

Saccadic eye movements are often imprecise and result in an error between expected and actual retinal target location after the saccade. Repeated experience of this error produces changes in saccade amplitude to reduce the error and concomitant changes in apparent visual location. We investigated the relationship between these two plastic processes in a series of experiments. Following a recent paradigm of inhibition of saccadic adaptation, in which participants are instructed to look at the initial target position and to continue to look at that position even if the target were to move again, our participants nevertheless perceived a visual probe presented near the saccade target to be shifted in direction of the target error. The location percept of the target gradually shifted and diverged over time from the executed saccade. Our findings indicate that changes in perceived location can be the same even when changes in saccade amplitude differ according to instruction and can develop even when the amplitude of the saccades executed during the adaptation procedure does not change. There are two possible explanations for this divergence between the adaptation states of saccade amplitude and perceived location. Either the intrasaccadic target step might trigger updating of the association between pre- and post-saccadic target positions, causing the localization shift, or the saccade motor command adjusts together with the perceived location at a common adaptation site, downstream from which voluntary control is exerted upon the executed eye movement only.     

Eye fixation scanpaths of younger and older drivers in a hazard perception task

Our previous research has shown that observing patterns of eye fixations is a successful method of establishing differences in underlying cognitive processes between groups of drivers. Eye movements recorded from drivers in a laboratory while they watch film clips recorded from a driver's perspective can be used to identify scanpaths and search patterns that reveal ability differences. In the present study 12 older subjects (60-75 years) and 12 younger subjects (30-45 years) watched clips for potential hazards such as other road users appearing on an intersecting trajectory. Acuity and visual field differences between the two groups were eliminated through screening, so that only age-related differences would emerge. Eye fixations were analysed on a frame-by-frame basis to generate sequences of codes representing the location and object of the viewer's interest, before and during the appearance of a hazard. These codes were analysed for the existence of two fixation scanpaths using Markov Matrices. Unique scanpaths were identified for each group of drivers before and during the hazard. Evidence from the inspection of different objects and from the spread of the search indicated that both groups of driver were sensitive to attentional capture by the appearance of the hazard. Detection of the hazards - both speed and accuracy - was similar for older and younger drivers, although the older drivers perceived the films as being more hazardous in general. There is little evidence in this study of an age-related decline in the search of the scene when detecting hazards.     

“Monocular rivalry” of a complex waveform

A perceptual instability which has been termed “monocular rivalry” occurs when scanning a stimulus which is produced by adding sinusoidal gratings consisting of a fundamental frequency and its third harmonic. In the first experiment of the present study of this phenomenon, subjects reported perceptual changes on two dimensions during free scanning as a function of the phase between the two components of the stimulus. These dimensions were the apparent relative contrast of the two components and the apparent waveform shape (based on similarity to a square- or triangular-wave). F + 3F phase-specific effects were found only for apparent changes in the waveform shape. In a second experiment it was found that a shift of the stimulus with contrast fixation (proximally, roughly equivalent to changing fixation) was usually sufficient to produce an apparent change of waveform. The perceptual instability of these waveforms may be due to the interaction of an after-effect of the previous fixation with the signal arising from the current fixation.     

Semantic object-scene inconsistencies affect eye movements,but not in the way predicted by contextualized meaning maps

Semantic information is important in eye movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the scene attract more fixations than consistent objects. One interpretation of this effect is that fixations are driven toward inconsistent objects because they are semantically more informative. We tested this explanation using contextualized meaning maps, a method that is based on crowd-sourced ratings to quantify the spatial distribution of context-sensitive “meaning” in images. In Experiment 1, we compared gaze data and contextualized meaning maps for images, in which objects-scene consistency was manipulated. Observers fixated more on inconsistent versus consistent objects. However, contextualized meaning maps did not assign higher meaning to image regions that contained semantic inconsistencies. In Experiment 2, a large number of raters evaluated image-regions, which were deliberately selected for their content and expected meaningfulness. The results suggest that the same scene locations were experienced as slightly less meaningful when they contained inconsistent compared to consistent objects. In summary, we demonstrated that — in the context of our rating task — semantically inconsistent objects are experienced as less meaningful than their consistent counterparts and that contextualized meaning maps do not capture prototypical influences of image meaning on gaze guidance.     

The extrafoveal preview paradigm as a measure of predictive,active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated “passive” behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.