Saccade target selection in visual search: the effect of information from the previous fixation

This paper reports an analysis of saccades made during a task of visual search for a colour shape conjunction. The analysis concentrates on the saccade following the first saccade, thus complementing an earlier paper where the first saccades were analysed. The further analysis addresses the issue of what information might be held in trans-saccadic memory. As with the first saccade, incorrect second saccades tend to fall on distractors sharing one feature with the target. The proximity of the target to the fixation location immediately prior to the saccade is a very significant determinant of whether the saccade will reach the target. The results lead to the conclusion that in the majority of cases, choice of saccade destination is made afresh during each fixation with no carry-over from the previous fixation. However, in a small number of cases, second saccades are made after extremely brief fixation intervals. Although these saccades show a similar probability of reaching the target as those following longer fixations, it is argued that this sub-set of saccades are pre-programmed at the time of the preceding saccade.     

Dynamics of saccade target selection: race model analysis of double step and search step saccade production in human and macaque

We investigated how saccade target selection by humans and macaque monkeys reacts to unexpected changes of the image. This was explored using double step and search step tasks in which a target, presented alone or as a singleton in a visual search array, steps to a different location on infrequent, random trials. We report that human and macaque monkey performance are qualitatively indistinguishable. Performance is stochastic with the probability of producing a compensated saccade to the final target location decreasing with the delay of the step. Compensated saccades to the final target location are produced with latencies relative to the step that are comparable to or less than the average latency of saccades on trials with no target step. Noncompensated errors to the initial target location are produced with latencies less than the average latency of saccades on trials with no target step. Noncompensated saccades to the initial target location are followed by corrective saccades to the final target location following an intersaccade interval that decreases with the interval between the target step and the initiation of the noncompensated saccade. We show that this pattern of results cannot be accounted for by a race between two stochastically independent processes producing the saccade to the initial target location and another process producing the saccade to the final target location. However, performance can be accounted for by a race between three stochastically independent processes--a GO process producing the saccade to the initial target location, a STOP process interrupting that GO process, and another GO process producing the saccade to the final target location. Furthermore, if the STOP process and second GO process start at the same time, then the model can account for the incidence and latency of mid-flight corrections and rapid corrective saccades. This model provides a computational account of saccade production when the image changes unexpectedly.     

Orientation of noisy texture affects saccade direction during free viewing

We redirect our eye approximately three times per second to bring a new part of our environment on to our fovea (Findlay & Gilchrist, 2003). How a scanning path is planned is still an unsolved matter. Most research to date has focused on the question of target selection: how is the next fixation location, or saccade target, selected. Here we investigated the direction of spontaneous saccades, rather than fixation locations per se. We measured eye movements, while observers were freely viewing noisy textures: oriented gabors embedded in either pink (1/f) noise or pixel noise, of which they later had to report their orientation. Our results show that a significant percentage of the spontaneous saccades were directed along the orientation of the stimulus. These results suggest that observers may have used an underlying eye movement strategy involving the search for contour endings.     

The target velocity integration function for saccades

Interacting with a dynamic environment calls for close coordination between the timing and direction of motor behaviors. Accurate motor behavior requires the system to predict where the target for action will be, both when action planning is complete and when the action is executed. In the current study, we investigate the time course of velocity information accrual in the period leading up to a saccade toward a moving object. In two experiments, observers were asked to generate saccades to one of two moving targets. Experiment 1 looks at the accuracy of saccades to targets that have trial-by-trial variations in velocity. We show that the pattern of errors in saccade landing position is best explained by proposing that trial-by-trial target velocity is taken into account in saccade planning. In Experiment 2, target velocity stepped up or down after a variable interval after the movement cue. The extent to which the movement endpoint reflects pre- or post-step velocity can be used to identify the temporal velocity integration window; we show that the system takes a temporally blurred snapshot of target velocity centered ～200 ms before saccade onset. This estimate is used to generate a dynamically updated prediction of the target's likely future location.     

Scene context guides eye movements during visual search

How does scene context guide search behavior to likely target locations? We had observers search for scene-constrained and scene-unconstrained targets, and found that scene-constrained targets were detected faster and with fewer eye movements. Observers also directed more initial saccades to target-consistent scene regions and devoted more time to searching these regions. However, final checking fixations on target-inconsistent regions were common in target-absent trials, suggesting that scene context does not strictly confine search to likely target locations. We interpret these data as evidence for a rapid top-down biasing of search behavior by scene context to the target-consistent regions of a scene.     

An anti-Hick's effect in monkey and human saccade reaction times

In order to execute movements to targets in the environment, we must first select a target in which to move, generally from an array of alternatives. Hick's Law states that reaction time (RT) increases as the number of response alternatives increases. Violations of this law, however, generally in the form of the absence of a relationship between response alternatives and RT have been reported in the literature. K. Kveraga, L. Boucher, and H. C. Hughes (2002), for example, found that saccades to visual targets violate Hick's Law. To examine this violation further, we measured saccade RTs in monkeys and humans and found that saccade RTs actually decreased as the number of potential target locations increased. We hypothesize that this arises because subjects must actively inhibit premature saccades, and that the required inhibition increases as the certainty of a movement to a particular location increases. With increased inhibition, saccade onset is delayed, resulting in an anti-Hick's effect.     

Coarse-to-fine eye movement strategy in visual search

Oculomotor behavior contributes importantly to visual search. Saccadic eye movements can direct the fovea to potentially interesting parts of the visual field. Ensuing stable fixations enables the visual system to analyze those parts. The visual system may use fixation duration and saccadic amplitude as optimizers for visual search performance. Here we investigate whether the time courses of fixation duration and saccade amplitude depend on the subject's knowledge of the search stimulus, in particular target conspicuity. We analyzed 65,000 saccades and fixations in a search experiment for (possibly camouflaged) military vehicles of unknown type and size. Mean saccade amplitude decreased and mean fixation duration increased gradually as a function of the ordinal saccade and fixation number. In addition we analyzed 162,000 saccades and fixations recorded during a search experiment in which the location of the target was the only unknown. Whether target conspicuity was constant or varied appeared to have minor influence on the time courses of fixation duration and saccade amplitude. We hypothesize an intrinsic coarse-to-fine strategy for visual search that is even used when such a strategy is not optimal.     

Remapping attention in multiple object tracking

Which coordinate system do we use to track moving objects? In a previous study using smooth pursuit eye movements, we argued that targets are tracked in both retinal (retinotopic) and scene-centered (allocentric) coordinates (Howe, Pinto, & Horowitz, 2010). However, multiple object tracking typically also elicits saccadic eye movements, which may change how object locations are represented. Observers fixated a cross while tracking three targets out of six identical disks confined to move within an imaginary square. The fixation cross alternated between two locations, requiring observers to make repeated saccades. By moving (or not moving) the imaginary square in sync with the fixation cross, we could disrupt either (or both) coordinate systems. Surprisingly, tracking performance was much worse when the objects moved with the fixation cross, although this manipulation preserved the retinal image across saccades, thereby avoiding the visual disruptions normally associated with saccades. Instead, tracking performance was best when the allocentric coordinate system was preserved, suggesting that targets locations are maintained in that coordinate system across saccades. This is consistent with a theoretical framework in which the positions of a small set of attentional pointers are predictively updated in advance of a saccade.     

Express averaging saccades in monkeys

When monkeys are presented simultaneously with multiple stimuli, they can make one of two types of response. Either they make averaging saccades, that land at intermediate locations between the targets, or target-directed saccades, that land close to one of the targets. The two types of saccades occur at different latencies and are thought to reflect different processes; fast reflexive averaging and slower target selection. We investigated the latency of averaging saccades in five monkeys, with particular emphasis on 'express' latency saccades, which are thought to be inhibited by target selection. Express averaging saccades were made prolifically by the two monkeys that made both express and regular latency saccades, but only when no specific instruction was given regarding the saccade target. When these monkeys had to choose one of the targets, on the basis of its color, they still made averaging saccades. However, the endpoints formed two distributions close to the targets as opposed to one single distribution centered between the targets, as was the case when targets were identical; also, express saccades were almost entirely absent. We conclude that express averaging saccades are a form of spatial and temporal optimization of gaze shifting.     

Frequency and metrics of square-wave jerks: influences of task-demand characteristics

PURPOSE: Square-wave jerks (SWJs) during visual fixation and pursuit tracking of targets of varying speed and predictability were investigated in the present study. METHODS: SWJs were measured in 91 subjects as they fixated a target or a remembered target location and tracked targets that varied in velocity and predictability. RESULTS: Percentages of subjects making SWJ and mean SWJ frequency per minute in the high- and low-predictability conditions were 99% and 9.34 and 91% and 2.78, respectively. SWJ rates were significantly lower when observers fixated remembered target locations rather than visual targets and during tracking of faster-moving and less predictable targets. Differences in task conditions cannot be explained by volitional influences to control the first saccade in the SWJ. There was also no influence of age on SWJ rate. CONCLUSIONS: Reduced SWJ rates after manipulations that increased task demands on visual pursuit of targets suggest an inverse relationship between current demands imposed by visual tasks and rates of intrusive saccades. These findings suggest that signals from cortical attentional systems may suppress inappropriate saccades that would divert the eyes from objects of interest during conditions imposing high task demands on the visual system.     

The long and the short of it: spatial statistics at fixation vary with saccade amplitude and task

We recorded over 90,000 saccades while observers viewed a diverse collection of natural images and measured low level visual features at fixation. The features that discriminated between where observers fixated and where they did not varied considerably with task, and the length of the preceding saccade. Short saccades (<8 degrees) are image feature dependent, long are less so. For free viewing, short saccades target high frequency information, long saccades are scale-invariant. When searching for luminance targets, saccades of all lengths are scale-invariant. We argue that models of saccade behaviour must account not only for task but also for saccade length and that long and short saccades are targeted differently.     

The amplitude and angle of saccades to double-step target displacements

Two experiments examined the magnitude and direction of the initial saccade to a target that underwent two displacements within 200 msec. When the amplitude of the two target displacements was held constant at 10 deg but the angle of the displacements differed by 45 deg, a small but significant number of intermediate-angle saccades occurred. These intermediate-angle saccades were directed to locations between the two targets, thereby generating an angle transition function, and their amplitude was 10-20% less than the amplitude on single-step displacements. These intermediate-angle saccades were not simply the result of programming an oblique saccade because amplitude transition functions virtually identical to those reported by Becker and Jürgens [Vision Res. 19, 967-983 (1979)] for horizontal saccades were obtained for double-step target displacements limited to oblique saccades. Finally, when both target amplitude and target angle were varied in double-step displacements, it became clear that the timing of the amplitude transition function and the angle transition function was not coincident. Across conditions, the angle transition function occurred at a consistent time prior to the initial saccade, whereas the amplitude transition function occurred at a variable time prior to the initial saccade. Because these amplitude and angle transition functions appeared to be dissociated, a modified model of the saccadic programming system for double-step displacements was proposed.     

Eye movements during visual search: the costs of choosing the optimal path.

Saccadic eye movements are usually assumed to be directed to locations containing important or useful information, but such assumptions fail to take into account that planning saccades to such locations might be too costly in terms of effort or attention required. To investigate costs of saccadic planning, subjects searched for a target letter that was contained in either one of two clusters located on either side of a central fixation target. A target was present on each trial and was more likely (probability=0.8) to appear in one cluster than the other. Probabilities were disclosed by differences in cluster intensities. The distance between each cluster and central fixation varied (60'-300'). The presentation time was limited (500 ms) to ensure that a successful search would require a wisely chosen saccadic plan. The best chance of finding the target would be to direct the first saccade to the high-probability location, but only one of the six subjects tested followed this strategy consistently. The rest (to varying degrees) preferred to aim the first saccade to the closer location, often followed by an attempted search of the remaining location. Two-location searches were unsuccessful; performance at both locations was poor due to insufficient time. Preferences for such ineffective strategies were surprising. They suggest that saccadic plans were influenced by attempts to minimize the cognitive and attentional load attached to planning and to maximize the number of new foveal views that can be acquired in a limited period of time. These strategies, though disastrous in our task, may be crucial in natural scanning, when many cognitive operations are performed at once, and the risk attached to a few errant glances at unimportant places is small.     

The influence of attending to multiple locations on eye movements

The present paper reports results of a dual task study in which two locations were endogenously cued as possible target locations, while only one eye movement had to be executed. During the cue period, letters were briefly presented at the saccade goals and at no-saccade goals. Results show that performance was better for letters presented at any of the saccade goals than for letters presented at the no-saccade locations. Furthermore saccades deviated away from the non-saccaded target location, suggesting inhibition of the location to which the eyes should not go. The results indicate that the premotor theory also holds for conditions in which attention is allocated to multiple locations.     

Manipulating saccadic decision-rate distributions in visual search

The Gaussian shape of reciprocal latency distributions typically found in single saccade tasks supports the idea of a race-to-threshold process underlying the decision when to saccade (R. H. Carpenter & M. L. Williams, 1995). However, second and later saccades in a visual search task revealed decision-rate (=reciprocal latency) distributions that were skewed Gamma-like (E. M. Van Loon, I. T. Hooge, & A. V. Van den Berg, 2002). Here we consider a related family of Beta-prime distributions that follows from strong competition with a signal to stop the sequence, and is described by two parameters: a fixate and saccade threshold. In three saccadic search experiments, we tried to manipulate the two thresholds independently, thereby expecting change in shape and mean of the reciprocal latency distribution. Interestingly, rate distributions for later saccades were significantly better fit by Beta-prime than by Gamma functions. Increases in the distribution's skew were found with higher display density, but only for second and later saccades. First saccade rate distributions were not altered by the expected target location or by visual information presented prior to the search, but making pre-search saccades did influence both thresholds. The mean rate remained a stereotyped function of ordinal position in the saccade sequence. Our results support strong competition between two decision signals underlying the timing of saccades.     

Dynamics of attentional deployment during saccadic programming

The dynamics of attentional deployment before saccade execution was studied with a dual-task paradigm. Observers made a horizontal saccade whose direction was indicated by a symbolic precue and had to discriminate the orientation of a Gabor patch displayed at different delays after the precue (but before saccade onset). The patch location relative to the saccadic target was indicated to observers before each block. Therefore, on each trial, observers were informed simultaneously about the respective absolute locations of the saccadic and perceptual targets. The main result is that orientational acuity improved over a period of 150-200 ms after the precue onset at the saccadic target location, where overall performance is best, and at distant locations. This effect is due to attentional factors rather than to an alerting effect. It is also dependent on the efficiency of the temporal masks displayed before and after the Gabor patches.     

Concurrent processing of saccades in visual search

We provide evidence that the saccadic system can simultaneously program two saccades to different goals. We presented subjects with simple visual search displays in which they were required to make a saccade to an odd-colored target embedded in an array of distractors. When there was strong competition between target and distractor stimuli (due to color priming from previous trials), subjects were more likely to make a saccade to a distractor. Such error saccades were often followed, after a very short inter-saccadic interval ( approximately 10-100 ms), by a second saccade to the target. The brevity of these inter-saccadic intervals suggests that the programming of the two saccades (one to a distractor and one to the target) overlapped in time. Using a saccade-contingent change in the search display, we show that new visual information presented during the initial saccade does not change the goal of the second saccade. This supports the idea that, by the end of the first saccade, programming of the second saccade is already well underway. We also elicited two-saccade responses (similar to those seen in search) using a double-step task, with the first saccade directed to the initial target step and the second saccade directed to the second target step. If the two saccades are programmed in parallel and programming of each saccade is triggered by one of the two target steps, the second saccade should occur at a relatively fixed time after the onset of the second target step, regardless of the timing of the initial saccade. This prediction was confirmed, supporting the idea that the two saccades are programmed in parallel. Finally, we observed that the shortest inter-saccadic intervals typically followed hypometric initial saccades, suggesting that the initial saccade may have been interrupted by the impending second saccade. Using predictions from physiological studies of interrupted saccades, we tested this hypothesis and found that the hypometric initial saccades did not appear to be interrupted in mid-flight. We discuss the significance of our findings for models of the saccadic system.     

Saccade control without a fovea

Prior research has suggested that two types of fast eye movements (FEMs) can be distinguished behaviorally. Foveating saccades respond to salient peripheral targets by directing the target image to the fovea. Non-foveating saccades include other FEMs such as nystagmus quick phase, saccades without visual stimuli and visually-directed saccades that direct target images to eccentric retina. Foveating saccades have a shorter initiation latency and are faster than non-foveating saccades. Following adaptation to central scotoma, patients tend to use preferred retinal loci for fixation (PRL). If PRL acquire the foveal characteristic of a retino-motor center then visually guided saccades would acquire the properties of foveating saccades. Using an objectively-calibrated 2-dimensional search coil, we measured saccades in response to salient, unpredictable targets. The saccades of normal observers were compared to the saccades of patients with long-standing macular scotomas. Although the saccades of patients consistently directed images to PRL, the saccades still had the latency and dynamic characteristics of non-foveating saccades. Moreover, the non-foveating saccades of patients were found to be less accurate than foveating saccades, showing a range effect (larger saccades undershoot with greater error than do smaller saccades). Apparently, patients with macular scotoma suppress rather than adapt a foveating saccade mechanism.     

Effect of previously fixated locations on saccade trajectory during free visual search

Recent studies have shown that the saccade trajectory often curved away from an object that was previously attended but irrelevant to the current saccade goal. We investigated whether such curved saccades occur during serial visual search, which requires sequential saccades possibly controlled by inhibition to multiple locations. The results show that the saccade trajectories were affected by at least three previous fixations. Furthermore, the effect of the previous fixations on saccade trajectories decreased exponentially with time or the number of intervening saccades. The relationship between the curved saccade trajectory and inhibition of return during serial visual search was discussed.     

Transsaccadic integration of visual information is predictive,attention-based,and spatially precise

Eye movements produce shifts in the positions of objects in the retinal image, but observers are able to integrate these shifting retinal images into a coherent representation of visual space. This ability is thought to be mediated by attention-dependent saccade-related neural activity that is used by the visual system to anticipate the retinal consequences of impending eye movements. Previous investigations of the perceptual consequences of this predictive activity typically infer attentional allocation using indirect measures such as accuracy or reaction time. Here, we investigated the perceptual consequences of saccades using an objective measure of attentional allocation, reverse correlation. Human observers executed a saccade while monitoring a flickering target object flanked by flickering distractors and reported whether the average luminance of the target was lighter or darker than the background. Successful task performance required subjects to integrate visual information across the saccade. A reverse correlation analysis yielded a spatiotemporal “psychophysical kernel” characterizing how different parts of the stimulus contributed to the luminance decision throughout each trial. Just before the saccade, observers integrated luminance information from a distractor located at the post-saccadic retinal position of the target, indicating a predictive perceptual updating of the target. Observers did not integrate information from distractors placed in alternative locations, even when they were nearer to the target object. We also observed simultaneous predictive perceptual updating for two spatially distinct targets. These findings suggest both that shifting neural representations mediate the coherent representation of visual space, and that these shifts have significant consequences for transsaccadic perception.