Differential detection of global luminance and contrast changes across saccades and flickers during active scene perception

How sensitive are viewers to changes in global image properties across saccades during active real-world scene perception? This question was investigated by globally increasing and/or decreasing luminance or contrast in photographs of real-world scenes across saccadic eye movements or during matched brief interruptions in a flicker paradigm. The results from two experiments demonstrated very poor sensitivity to global image changes in both the saccade-contingent and flicker paradigms, suggesting that the specific values of basic sensory properties do not contribute to the perception of stability across saccades during complex scene perception. In addition, overall sensitivity was significantly worse in the saccade-contingent change paradigm than the flicker paradigm, suggesting that the flicker paradigm is an imperfect simulation of transsaccadic vision.     

Selective attention and the active remapping of object features in trans-saccadic perception

When the same object is attended both before and after a saccadic eye movement, its visual features may be remapped to the new retinal position of the object. To further investigate the role of selective attention in trans-saccadic perception, the magnitude of the cross-saccadic tilt aftereffect was measured for both attended and unattended objects. The results show that both selective attention and saccadic eye movements influenced the magnitude of the tilt aftereffect, but in different ways. Dividing attention among multiple objects lead to a general decrease in the tilt aftereffect, independent of whether or not a saccade occurred. Making a saccade also resulted in a consistent reduction of the aftereffect, but this was due to incomplete transfer of form adaptation to the new retinal position. The influences of selective attention and saccadic remapping on the tilt aftereffect were independent and additive. These findings suggest that trans-saccadic perception is not limited to a single object but instead depends on the allocation of selective attention. Overall, the results are consistent with the hypothesis that the role of attention is to select salient objects, with trans-saccadic perception mechanisms acting to maintain information about those salient objects across eye movements.     

The where, what and when of gaze allocation in the lab and the natural environment

How do people distribute their visual attention in the natural environment? We and our colleagues have usually addressed this question by showing pictures, photographs or videos of natural scenes under controlled conditions and recording participants’ eye movements as they view them. In the present study, we investigated whether people distribute their gaze in the same way when they are immersed and moving in the world compared to when they view video clips taken from the perspective of a walker. Participants wore a mobile eye tracker while walking to buy a coffee, a trip that required a short walk outdoors through the university campus. They subsequently watched first-person videos of the walk in the lab. Our results focused on where people directed their eyes and their head, what objects were gazed at and when attention-grabbing items were selected. Eye movements were more centralised in the real world, and locations around the horizon were selected with head movements. Other pedestrians, the path, and objects in the distance were looked at often in both the lab and the real world. However, there were some subtle differences in how and when these items were selected. For example, pedestrians close to the walker were fixated more often when viewed on video than in the real world. These results provide a crucial test of the relationship between real behaviour and eye movements measured in the lab.     

Saliency does not account for fixations to eyes within social scenes

We assessed the role of saliency in driving observers to fixate the eyes in social scenes. Saliency maps (Itti & Koch, 2000) were computed for the scenes from three previous studies. Saliency provided a poor account of the data. The saliency values for the first-fixated locations were extremely low and no greater than what would be expected by chance. In addition, the saliency values for the eye regions were low. Furthermore, whereas saliency was no better at predicting early saccades than late saccades, the average latency to fixate social areas of the scene (e.g., the eyes) was very fast (within 200 ms). Thus, visual saliency does not account for observers’ bias to select the eyes within complex social scenes, nor does it account for fixation behavior in general. Instead, it appears that observers’ fixations are driven largely by their default interest in social information.     

Turning the world around: patterns in saccade direction vary with picture orientation

The eye movements made by viewers of natural images often feature a predominance of horizontal saccades. Can this behaviour be explained by the distribution of saliency around the horizon, low-level oculomotor factors, top-down control or laboratory artefacts? Two experiments explored this bias by recording saccades whilst subjects viewed photographs rotated to varying extents, but within a constant square frame. The findings show that the dominant saccade direction follows the orientation of the scene, though this pattern varies in interiors and during recognition of previously seen pictures. This demonstrates that a horizon bias is robust and affected by both the distribution of features and more global representations of the scene layout.     

Spatial perception during pursuit initiation

Spatial perception is modulated by eye movements. During smooth pursuit, perceived locations are shifted in the direction of the eye movement. During active fixation, visual space is perceptually compressed towards the fovea. In our present study, we were interested to determine the time course of spatial localization during pursuit initiation, i.e. the transition period from fixation to steady-state pursuit. Human observers had to localize briefly flashed targets around the time of pursuit initiation. Our data clearly show that pursuit-like mislocalization starts well before the onset of the eye movement. Our results point towards corollary-discharge as neural source for the observed perceptual effect.     

Parametric integration of visual form across saccades

Through saccadic eye movements, the retinal projection of an extrafoveally glimpsed object can be brought into foveal vision quickly. We investigated what influence visual detail collected before the saccade exerts on the postsaccadic percept. Participants were instructed to saccade towards a peripheral stimulus, and to indicate on a continuum of ellipses with varying aspect ratios which exact shape they had perceived to be present after saccade landing. Compared to both an identical ellipse preview and a qualitatively different square preview, a quantitatively different ellipse preview was observed to shift the mean postsaccadic percept towards the presaccadic aspect ratio parameter value. This integration of subtly different form information was accompanied by an integration of the identity of both stimuli presented: In the great majority of these trials, subjects indicated that they had not noticed the occurrence of a change to the stimulus. When a blank screen preceded the postsaccadic stimulus onset the influence of presaccadic stimulus information on postsaccadic perception was weaker. An immediate postsaccadic mask on the other hand abolished the effect entirely. We conclude that integration of parametric visual form information occurs across saccades, but that it relies on a quickly decaying and maskable visual memory.     

Fixations in natural scenes: interaction of image structure and image content

Explorative eye movements specifically target some parts of a scene while ignoring others. Here, we investigate how local image structure--defined by spatial frequency contrast--and informative image content--defined by higher order image statistics-are weighted for the selection of fixation points. We measured eye movements of macaque monkeys freely viewing a set of natural and manipulated images outside a particular task. To probe the effect of scene content, we locally introduced patches of pink noise into natural images, and to probe the interaction with image structure, we altered the contrast of the noise. We found that fixations specifically targeted the natural image parts and spared the uninformative noise patches. However, both increasing and decreasing the contrast of the noise attracted more fixations, and, in the extreme cases, compensated the effect of missing content. Introducing patches from another natural image led to similar results. In all paradigms tested, the interaction between scene structure and informative scene content was the same in any of the first six fixations on an image, demonstrating that the weighting of these factors is constant during viewing of an image. These results question theories, which suggest that initial fixations are driven by stimulus structure whereas later fixations are determined by informative scene content.     

Semantic guidance of eye movements in real-world scenes

The perception of objects in our visual world is influenced by not only their low-level visual features such as shape and color, but also their high-level features such as meaning and semantic relations among them. While it has been shown that low-level features in real-world scenes guide eye movements during scene inspection and search, the influence of semantic similarity among scene objects on eye movements in such situations has not been investigated. Here we study guidance of eye movements by semantic similarity among objects during real-world scene inspection and search. By selecting scenes from the LabelMe object-annotated image database and applying latent semantic analysis (LSA) to the object labels, we generated semantic saliency maps of real-world scenes based on the semantic similarity of scene objects to the currently fixated object or the search target. An ROC analysis of these maps as predictors of subjects’ gaze transitions between objects during scene inspection revealed a preference for transitions to objects that were semantically similar to the currently inspected one. Furthermore, during the course of a scene search, subjects’ eye movements were progressively guided toward objects that were semantically similar to the search target. These findings demonstrate substantial semantic guidance of eye movements in real-world scenes and show its importance for understanding real-world attentional control.     

Eye movements during information processing tasks: individual differences and cultural effects

The eye movements of native English speakers, native Chinese speakers, and bilingual Chinese/English speakers who were either born in China (and moved to the US at an early age) or in the US were recorded during six tasks: (1) reading, (2) face processing, (3) scene perception, (4) visual search, (5) counting Chinese characters in a passage of text, and (6) visual search for Chinese characters. Across the different groups, there was a strong tendency for consistency in eye movement behavior; if fixation durations of a given viewer were long on one task, they tended to be long on other tasks (and the same tended to be true for saccade size). Some tasks, notably reading, did not conform to this pattern. Furthermore, experience with a given writing system had a large impact on fixation durations and saccade lengths. With respect to cultural differences, there was little evidence that Chinese participants spent more time looking at the background information (and, conversely less time looking at the foreground information) than the American participants. Also, Chinese participants' fixations were more numerous and of shorter duration than those of their American counterparts while viewing faces and scenes, and counting Chinese characters in text.     

Optokinetic potential and the perception of head-centred speed

Extra-retinal information about eye velocity is thought to play an important role in compensating the retinal motion experienced during an eye movement. Evidently this compensation process is prone to error, since stimulus properties such as contrast and spatial frequency have marked effect on perceived motion with respect to the head. Here we investigate the suggestion, that 'optokinetic potential' [Perception 14 (1985) 631] may contribute to an explanation of these errors. First, we measured the optokinetic nystagmus induced by each stimulus so as to determine the optokinetic potential. Second, we determined the speed match between two patches of Gaussian blobs presented sequentially. Observers pursued the first pattern and kept their eyes stationary when viewing the second. For stimuli with identical contrast or spatial frequency, the pursued pattern was perceived to move slower than the non-pursued pattern (the Aubert-Fleischl phenomenon). Lowering the contrast or the spatial frequency of the non-pursued pattern resulted in a systematic decrease of its perceived speed. A further condition in which the contrast or spatial frequency of the pursued pattern was varied, resulted in no change to its perceived speed. Pursuit eye movements were recorded and found to be independent of stimulus properties. The results cast doubt on the idea that changing contrast or spatial frequency affects perceived head-centred speed by altering optokinetic potential.     

The role of gist in scene recognition

Studies of change blindness suggest that we bring only a few attended features of a scene, plus a gist, from one visual fixation to the next. We examine the role of gist by substituting an original image with a second image in which a substitution of one object changes the gist, compared with a third image in which a substitution of that object does not change the gist. Small perceptual changes that affect gist were more rapidly detected than perceptual changes that do not affect gist. When the images were scrambled to remove meaning, this difference disappeared for seven of the nine sets, indicating that gist and not image features dominated the result. In a final experiment a natural image was masked with an 8 × 8 checker pattern, and progressively substituted by squares of a new natural image of the same gist. Spatial jitter prevented fixation on the same square for the sequence of 12 changes. Observers detected a change in an average of 2.1 out of 7 sequences, indicating strong change blindness for images of the same gist but completely different local features. We conclude that gist is automatically encoded, separately from specific features.     

Asymmetries of optokinetic nystagmus in amblyopia: the effect of selected retinal stimulation

Open loop optokinetic eye movements were measured in response to monocular nasalward and temporalward visual field movement presented at four selected retinal sites in 6 strabismic amblyopes and 4 normal observers. Stimulus sites included the central retina (10 X 10 deg), a large field (40 X 32 deg), a large peripheral field with the center (10 X 10 deg) blocked out and a hemiretinal field (15 X 32 deg) excluding the fovea. We found directional preferences of OKN in amblyopia to nasalward stimulus movement for the foveal and concentric peripheral stimuli. The results of peripheral hemiretinal optokinetic stimulation of amblyopic subjects revealed a deficient OKN slow phase response from the temporal hemiretina, particularly for temporalward stimulus movement. There was no marked asymmetry of OKN in the normal group for the concentric stimuli. A normal preference was found for nasalward and temporalward stimulus field movement imaged on the nasal and temporal hemiretinae respectively. These results are interpreted in terms of a model of cortical and subcortical pathways for OKN derived from comparative studies of cat and monkey.     

Adaptive deployment of spatial and feature-based attention before saccades

What you see depends not only on where you are looking but also on where you will look next. The pre-saccadic attention shift is an automatic enhancement of visual sensitivity at the target of the next saccade. We investigated whether and how perceptual factors independent of the oculomotor plan modulate pre-saccadic attention within and across trials. Observers made saccades to one (the target) of six patches of moving dots and discriminated a brief luminance pulse (the probe) that appeared at an unpredictable location. Sensitivity to the probe was always higher at the target’s location (spatial attention), and this attention effect was stronger if the previous probe appeared at the previous target’s location. Furthermore, sensitivity was higher for probes moving in directions similar to the target’s direction (feature-based attention), but only when the previous probe moved in the same direction as the previous target. Therefore, implicit cognitive processes permeate pre-saccadic attention, so that–contingent on recent experience–it flexibly distributes resources to potentially relevant locations and features.     

Transient torsion during and after saccades

In five normal subjects, we analyzed uncalled for torsion (blips) during and after horizontal and vertical saccades. Torsion was defined as movement out of Listing's plane. During horizontal saccades in downward gaze the abducting eye extorted and the adducting eye intorted. The direction of the blips reversed in upward gaze. Peak torsional amplitudes (up to 1–2 deg) were always reached during saccades; drifts back to Listing's plane outlasted the saccades. Torsion of the extorting eye was larger than that of the intorting eye, producing a transient positive cyclovergence. Torsion and cyclovergence evoked by vertical saccades were also stereotyped in each eye, but showed idiosyncratic differences among subjects. We conclude that Listing's law is violated during saccades. Transient saccade-evoked torsion might reflect properties of the three-dimensional velocity-to-position integrator and/or the ocular plant.     

Attention during sequences of saccades along marked and memorized paths

Natural scenes are explored by combinations of saccadic eye movements and shifts of attention. The mechanisms that coordinate attention and saccades during ordinary viewing are not well understood because studies linking saccades and attention have focused mainly on single saccades made in isolation. This study used an orientation discrimination task to examine attention during sequences of saccades made through an array of targets and distractors. Perceptual measures showed that attention was distributed along saccadic paths when the paths were marked by color cues. When paths were followed from memory, attention rarely spread beyond the goal of the upcoming saccade. These different distributions of attention suggest the involvement of separate processes of attentional control during saccadic planning, one triggered by top-down selection of the saccadic target, and the other by activation linked to visual mechanisms not tied directly to saccadic planning. The concurrent activity of both processes extends the effective attentional field without compromising the accuracy, precision, or timing of saccades.     

Landmarks facilitate visual space constancy across saccades and during fixation

It has been demonstrated that visual objects that are present after saccadic eye movements act as landmarks for the localization of stimuli across saccades, facilitating space constancy (Deubel, 2004). We here study the temporal conditions under which landmark effects occur after saccadic eye movements, and during fixation. Two small objects were presented 6° in the periphery, one above the other. Observers saccaded to the space between them. One of the objects disappeared during the saccade and reappeared with a variable delay during or after the saccade. At the same time either that object or the continuously present one jumped by 1°. The observer’s task was to decide which object had moved. The results revealed a strong bias to assign movement to the object that was blanked, regardless of which actually moved. If both objects were blanked, the one that was blanked for a shorter time tended to be seen as stable. The effects were stronger as the onset asynchrony between the stimuli increased. Surprisingly, analogous though weaker effects occurred during visual fixation, suggesting that similar visual mechanisms relying on visual landmarks operate both across saccades and during fixation.     

The size and direction of saccadic curvatures during reading

Eye movements during the reading of multi-line pages of texts were analyzed to determine the trajectory of reading saccades. The results of two experiments showed that the trajectory of the majority of forward-directed saccades was negatively biased, i.e., the trajectory fell below the start and end location of the saccadic movement. This is attributed to a global top-to-bottom orienting of attention. The curvature size and the proportion of negative trajectories were diminished when linguistic processing demands were high and when the beginning lines of a page were read. Longer pre-saccadic fixations also yielded smaller saccadic curvatures, and they resulted in fewer negatively curved forward-directed saccades in Experiment 1 although not in Experiment 2. These findings indicate that the top-to-bottom pull of saccadic trajectories is modulated by processing demands and processing opportunities. The results are in general agreement with a time-locked attraction-inhibition hypothesis, according to which the horizontal movement component of a saccade is initially subject to an automatic top-to-bottom orienting of attention that is subsequently inhibited.     

Saccadic selectivity in complex visual search displays

Visual search is a fundamental and routine task of everyday life. Studying visual search promises to shed light on the basic attentional mechanisms that facilitate visual processing. To investigate visual attention during search processes, numerous studies measured the selectivity of observers' saccadic eye movements for local display features. These experiments almost entirely relied on simple, artificial displays with discrete search items and features. The present study employed complex search displays and targets to examine task-driven (top-down) visual guidance by low-level features under more natural conditions. Significant guidance by local intensity, contrast, spatial frequency, and orientation was found, and its properties such as magnitude and resolution were analyzed across dimensions. Moreover, feature-ratio effects were detected, which correspond to distractor-ratio effects in simple search displays. These results point out the limitations of current purely stimulus-driven (bottom-up) models of attention during scene perception.     

The time course of vertical, horizontal and oblique saccade trajectories: Evidence for greater distractor interference during vertical saccades

The present study aimed to characterize the effect of a nearby distractor on vertical, horizontal, and oblique saccade curvature under normal saccade preparation times. Consistent with previous findings, longer-latency vertical saccades showed greater curvature away from a distractor than did oblique or horizontal saccades. At short latencies, vertical saccades also showed greater curvature towards the distractor. A neural explanation for why vertical saccades show greater interference from a distractor is theorized.