Location and identity memory of saccade targets

While the memory of objects’ identity and of their spatiotopic location may sustain transsaccadic spatial constancy, the memory of their retinotopic location may hamper it. Is it then true that saccades perturb retinotopic but not spatiotopic memory? We address this issue by assessing localization performances of the last and of the penultimate saccade target in a series of 2-6 saccades. Upon fixation, nine letter-pairs, eight black and one white, were displayed at 3° eccentricity around fixation within a 20° × 20° grey frame, and subjects were instructed to saccade to the white letter-pair; the cycle was then repeated. Identical conditions were run with the eyes maintaining fixation throughout the trial but with the grey frame moving so as to mimic its retinal displacement when the eyes moved. At the end of a trial, subjects reported the identity and/or the location of the target in either retinotopic (relative to the current fixation dot) or frame-based¹ (relative to the grey frame) coordinates. Saccades degraded target’s retinotopic location memory but not its frame-based location or its identity memory. Results are compatible with the notion that spatiotopic representation takes over retinotopic representation during eye movements thereby contributing to the stability of the visual world as its retinal projection jumps on our retina from saccade to saccade.     

Spatiotopic updating across saccades in the absence of awareness

Despite the continuously changing visual inputs caused by eye movements, our perceptual representation of the visual world remains remarkably stable. Visual stability has been a major area of interest within the field of visual neuroscience. The early visual cortical areas are retinotopic-organized, and presumably there is a retinotopic to spatiotopic transformation process that supports the stable representation of the visual world. In this study, we used a cross-saccadic adaptation paradigm to show that both the orientation adaptation and face gender adaptation could still be observed at the same spatiotopic (but different retinotopic) locations even when the adapting stimuli were rendered invisible. These results suggest that awareness of a visual object is not required for its transformation from the retinotopic to the spatiotopic reference frame.     

Negative feedback control model of proximal convergence and accommodation

A comprehensive model has been developed to illustrate the interactions between the observer and the surrounding environment in the control of oculomotor responses to distance or 3-D space. Accommodation and vergence respond to both spatiotopic (body reference) proximal percepts and retinotopic (eye referenced) physical stimuli of blur and disparity. Both spatiotopic and retinotopic stimuli are derived respectively from perceptual and physical correlates of negative feedback for eye position. The spatiotopic and retinotopic stimulus errors are combined in the feedforward path and drive a common oculomotor controller which has a phasic-tonic organization. Spatiotopic and retinotopic stimuli are shown to be effective over complementary operating ranges. Perceptual spatiotopic errors of gaze provide optimal stimuli for near responses to large depth intervals whereas physical-retinotopic cues of blur and disparity provide quantitative information about small binocular fixation errors. Small dynamic variations of target distance are sensed both spatiotopically and retinotopically. Coarse and fine spatiotopic errors of gaze are processed differently. Large spatiotopic errors are sampled intermittently at the beginning of the near response, whereas small retinotopic position errors and spatiotopic velocity errors are sampled continuously throughout the near response. Former reports of empirically observed higher velocity of vergence responses to very large depth intervals is explained in terms of stimulus sampling modes rather than in terms of separate oculomotor control mechanisms. The model demonstrates a complementary function of top-down spatiotopic cues, which are used to initiate the near response, and bottom-up retinotopic cues, which are used to refine and complete the near response. Cross-couplings by vergence-accommodation and accommodative-vergence serve to coordinate the components of the near response when feedback from sensed response of one motor system (i.e. vergence) is more accurate than that of the other motor system (i.e. accommodation). The model presented here is concerned primarily with the near response mediated by accommodation and disjunctive eye movements and not by the independent vergence mediated by non-conjugate or yoked saccades of unequal amplitude.     

Retinotopic and non-retinotopic stimulus encoding in binocular rivalry and the involvement of feedback

Adaptation is one of the key constituents of the perceptual alternation process during binocular rivalry, as it has been shown that preadapting one of the rivaling pairs before rivalry onset biases perception away from the adapted stimulus during rivalry. We investigated the influence of retinotopic and spatiotopic preadaptation on binocular rivalry. We show that for grating stimuli, preadaptation only influences rivalry when adaptation and rivalry locations are retinotopically matched. With more complex house and face stimuli, effects of preadaptation are found for both retinotopic and spatiotopic preadaptation, showing the importance of spatiotopic encoding in binocular rivalry. We show, furthermore, that adaptation to phase-scrambled faces results in retinotopic effects only, demonstrating the importance of form content for spatiotopic adaptation effects, as opposed to spatial frequency content. Are the spatiotopic adaptation influences on rivalry caused by direct spatiotopic stimulus interactions, or instead are they due to altered feedback from the adapted spatiotopic representations to the retinotopic representations that are involved in rivalry? By using rivaling face and grating stimuli that minimize rivalry between spatiotopic representations while still engaging these representations in stimulus encoding, we show that at least part of the preadaptation effects with face stimuli depend on feedback information.     

Neuronal mechanisms of visual stability

Human vision is stable and continuous in spite of the incessant interruptions produced by saccadic eye movements. These rapid eye movements serve vision by directing the high resolution fovea rapidly from one part of the visual scene to another. They should detract from vision because they generate two major problems: displacement of the retinal image with each saccade and blurring of the image during the saccade. This review considers the substantial advances in understanding the neuronal mechanisms underlying this visual stability derived primarily from neuronal recording and inactivation studies in the monkey, an excellent model for systems in the human brain. For the first problem, saccadic displacement, two neuronal candidates are salient. First are the neurons in frontal and parietal cortex with shifting receptive fields that provide anticipatory activity with each saccade and are driven by a corollary discharge. These could provide the mechanism for a retinotopic hypothesis of visual stability and possibly for a transsaccadic memory hypothesis, The second neuronal mechanism is provided by neurons whose visual response is modulated by eye position (gain field neurons) or are largely independent of eye position (real position neurons), and these neurons could provide the basis for a spatiotopic hypothesis. For the second problem, saccadic suppression, visual masking and corollary discharge are well established mechanisms, and possible neuronal correlates have been identified for each.     

Object-based anisotropic mislocalization by retinotopic motion signals

The relative visual positions of briefly flashed stimuli are systematically modified in the presence of motion signals. We have recently shown that the perceived position of a spatially extended flash stimulus is anisotropically shifted toward a single convergent point back along the trajectory of a moving object without a significant change in the perceived shape of the flash [Watanabe, K., & Yokoi, K. (2006). Object-based anisotropies in the flash-lag effect. Psychological Science, 17, 728-735]. In the previous experiment, the moving stimulus moved in both retinotopic and environmental coordinates. In the present study, we examined whether the anisotropic mislocalization depends on retinotopic or object motion signals. When the retinal image of a moving stimulus was rendered stationary by smooth pursuit, the anisotropic pattern of mislocalization was not observed. In contrast, when the retinal image of a stationary stimulus was moved by eye movements, anisotropic mislocalization was observed, with the magnitude of the mislocalization comparable to that in the previous study. In both cases, there was little indication of shape distortion of the flash stimulus. These results demonstrate a clear case of object-based mislocalization by retinotopic motion signals; retinotopic--not object--motion signals distort the perceived positions of visual objects after the shape representations are established.     

Retinotopic adaptation-based visual duration compression

Eye movements present the visual system with the challenge of providing the experience of a stable world. This appears to require the location of objects to be mapped from retinal to head and body referenced coordinates. Following D. Burr, A. Tozzi, and M. C. Morrone (2007), here we address the issue of whether adaptation-based duration compression (A. Johnston, D. H. Arnold, & S. Nishida, 2006) takes place in a retinocentric or head-centric frame of reference. Duration compression may be associated with shifts in apparent temporal frequency. However, using an adaptation schedule that minimizes any effect of adaptation on apparent temporal frequency, we still find substantial apparent duration compression. Duration compression remains when the adaptor continuously translates in head-centered coordinates but is fixed on the retina, isolating retinal adaptation. Apparent duration was also measured after a change in gaze direction-a strategy which allows eye-centered and head-centered components of adaptation-induced duration compression to be distinguished. In two different paradigms, we found significant compression was elicited by retinotopic adaptation, with no significant change in apparent duration following spatiotopic adaptation. We also observed no interocular transfer of adaptation. These findings point to an early locus for the adaptation-based duration compression effect.     

Centering biases in heterochromatic brightness matching

When the method of constant stimuli is used to measure heterochromatic brightness matches, the resulting matches can be strongly biased toward the center of the range of test luminances used. In the present paper, we investigate the source of this centering bias. The stimuli were 2 degrees red squares presented in a gray surround. In the main experiments, two ranges of stimulus luminance were presented in separate physical locations on a video monitor, but with test trials interleaved in time. Subjects either fixated a fixation cross (fixation condition), creating different retinotopic locations for the two luminance ranges, or foveated each stimulus as it appeared (foveation condition), creating identical retinotopic locations for both ranges. In the fixation condition, the two different stimulus sets resulted in a simultaneous centering bias--two different brightness matches at two different retinotopic locations at the same time. This effect was essentially eliminated in the foveation condition. A dichoptic foveation condition also revealed no centering bias. The results suggest that under the conditions tested, the centering bias is caused by a process located at a post-retinal but still retinotopically organized level of the visual system, rather than by either a retinal process or a more central, spatiotopically organized one.     

Non-retinotopic feature processing in the absence of retinotopic spatial layout and the construction of perceptual space from motion

The spatial representation of a visual scene in the early visual system is well known. The optics of the eye map the three-dimensional environment onto two-dimensional images on the retina. These retinotopic representations are preserved in the early visual system. Retinotopic representations and processing are among the most prevalent concepts in visual neuroscience. However, it has long been known that a retinotopic representation of the stimulus is neither sufficient nor necessary for perception. Saccadic Stimulus Presentation Paradigm and the Ternus–Pikler displays have been used to investigate non-retinotopic processes with and without eye movements, respectively. However, neither of these paradigms eliminates the retinotopic representation of the spatial layout of the stimulus. Here, we investigated how stimulus features are processed in the absence of a retinotopic layout and in the presence of retinotopic conflict. We used anorthoscopic viewing (slit viewing) and pitted a retinotopic feature-processing hypothesis against a non-retinotopic feature-processing hypothesis. Our results support the predictions of the non-retinotopic feature-processing hypothesis and demonstrate the ability of the visual system to operate non-retinotopically at a fine feature processing level in the absence of a retinotopic spatial layout. Our results suggest that perceptual space is actively constructed from the perceptual dimension of motion. The implications of these findings for normal ecological viewing conditions are discussed.     

Retinotopic encoding of the direction aftereffect

Kohn and Movshon [Kohn, A., & Movshon, J. (2003). Neuronal adaptation to visual motion in area MT of the macaque. Neuron, 39, 681-691; Kohn, A., & Movshon, J. A. (2004). Adaptation changes the direction tuning of macaque MT neurons. Nature Neuroscience, 7(7), 764-772] measured the contrast response functions of single neurons in MT (V5) before and after adaptation to high contrast gratings. They found that when gratings were smaller than the MT receptive field, so that adapting and test regions could be either co-localised or non-overlapping, adaptation was spatially specific. This led to the hypothesis that grating adaptation occurs in V1, where receptive fields are small and retinotopically organized, and that MT merely inherits this adaptation. We predicted that spatial specificity would be less for dot stimuli that probably adapt MT cells directly. Also, given recent contradictory claims that hMT primarily exhibits both spatiotopy [d'Avossa, G., Tosetti, M., Crespi, S., Biagi, L., Burr, D., & Morrone, M. (2006). Spatiotopic selectivity of BOLD responses to visual motion in human area MT. Nature Neuroscience, 10, 249-255] and retinotopy [Gardner, J. L., Merriam, E. P., Movshon, J. A., & Heeger, D. J. (2008). Maps of visual space in human occipital cortex are retinotopic, not spatiotopic. The Journal of Neuroscience, 28, 3988-3999], we were interested in producing relevant psychophysical evidence using the direction aftereffect. In three experiments, we measured direction aftereffects (DAEs) induced and tested either with drifting gratings or drifting dots when stimulus location was changed both retinotopically and spatiotopically between adaptation and test; when retinotopic location only was changed; and when spatiotopic location only was changed. We predicted and found that spatial specificity was greater for gratings than for dots. We also found very small spatiotopic effects that call into question some recent claims that area MT exhibits a high degree of spatiotopicity.     

Remapped visual masking

Cells in saccade control areas respond if a saccade is about to bring a target into their receptive fields (J. R. Duhamel, C. L. Colby, & M. R. Goldberg, 1992). This remapping process should shift the retinal location from which attention selects target information (P. Cavanagh, A. R. Hunt, S. R. Afraz, & M. Rolfs, 2010). We examined this attention shift in a masking experiment where target and mask were presented just before an eye movement. In a control condition with no eye movement, masks interfered with target identification only when they spatially overlapped. Just before a saccade, however, a mask overlapping the target had less effect, whereas a mask placed in the target's remapped location was quite effective. The remapped location is the retinal position the target will have after the upcoming saccade, which corresponds to neither the retinotopic nor spatiotopic location of the target before the saccade. Both effects are consistent with a pre-saccadic shift in the location from which attention selects target information. In the case of retinally aligned target and mask, the shift of attention away from the target location reduces masking, but when the mask appears at the target's remapped location, attention's shift to that location brings in mask information that interferes with the target identification.     

Reference frames in early motion detection

To perceive the real motion of objects in the world while moving the eyes, retinal motion signals must be compensated by information about eye movements. Here we study when this compensation takes place in the course of visual processing, and whether uncompensated motion signals are ever available. We used a paradigm based on asymmetry in motion detection: Fast-moving objects are found easier among slow-moving distractors than are slow objects among fast distractors. By coupling object motion to eye motion, we created stimuli that moved fast on the retina but slowly in an eye-independent reference frame, or vice versa. In the 100 ms after stimulus onset, motion detection is dominated by retinal motion, uncompensated for eye movements. As early as 130 ms, compensated signals become available: objects that move slowly on the retina but fast in an eye-independent frame are detected as easily as those that move fast on the retina.     

Statistical modelling of the central 10-degree visual field in short-wavelength automated perimetry

BACKGROUND: Reports of short-wavelength pathway dysfunction in retinal eye disease suggest that short-wavelength automated perimetry may be a useful technique for the investigation of central visual function. The aim of this study was to adapt existing statistical procedures used for the investigation of 30-2 short-wavelength automated perimetry to the 10-2 program of the Humphrey Field Analyser. METHODS: A four- or six-point linear interpolation procedure was used to calculate normal visual field sensitivity for each of the 68 stimulus locations of the 10-2 program using empirical normal data from 51 normal subjects examined using the 30-2 program. Prediction limits for normality were derived at each stimulus location, enabling the calculation of age-corrected global perimetric indices and construction of probability maps for diffuse and focal visual field loss. The normal database was validated by empirical data from five normal subjects, stratified for age. RESULTS: The pointwise distribution of normal sensitivity exhibited a Gaussian distribution at the majority of stimulus locations. The pointwise coefficient of variation did not vary significantly across the visual field. Examples of diabetic pseudophakic patients and a patient with age-related macular degeneration are presented to illustrate the effectiveness of SWAP at detecting visual field abnormality in the central visual field. CONCLUSION: Ten-degree SWAP is able to effectively detect focal visual field loss in central retinal eye disease which may precede those found using conventional perimetry. SWAP may prove to be an invaluable technique for the investigation of central retinal eye disease.     

Sensorimotor transformation during eye movements to remembered visual targets.

For eye movements made to visual targets, the brain must transform the retinotopic coordinate frame of the visual system to that of the oculomotor plant. Ideally, responses should exactly match target demands. However, during eye movements to remembered targets, responses are spatially distorted. The transformation does not retain accurate retinotopic registration, having both constant and variable components of error. Generally, the constant pattern of distortion appears as a hypermetria for upward saccades and a hypometria for downward movements. Most of the error accumulates during the first 800 msec of memory-contingent delay. The results are interpreted with respect to theories of how spatial information may be coded and transformed.     

Mobile computation: spatiotemporal integration of the properties of objects in motion

We demonstrate that, as an object moves, color and motion signals from successive, widely spaced locations are integrated, but letter and digit shapes are not. The features that integrate as an object moves match those that integrate when the eyes move but the object is stationary (spatiotopic integration). We suggest that this integration is mediated by large receptive fields gated by attention and that it occurs for surface features (motion and color) that can be summed without precise alignment but not shape features (letters or digits) that require such alignment. Rapidly alternating pairs of colors and motions were presented at several locations around a circle centered at fixation. The same two stimuli alternated at each location with the phase of the alternation reversing from one location to the next. When observers attended to only one location, the stimuli alternated in both retinal coordinates and in the attended stream: feature identification was poor. When the observer's attention shifted around the circle in synchrony with the alternation, the stimuli still alternated at each location in retinal coordinates, but now attention always selected the same color and motion, with the stimulus appearing as a single unchanging object stepping across the locations. The maximum presentation rate at which the color and motion could be reported was twice that for stationary attention, suggesting (as control experiments confirmed) object-based integration of these features. In contrast, the identification of a letter or digit alternating with a mask showed no advantage for moving attention despite the fact that moving attention accessed (within the limits of precision for attentional selection) only the target and never the mask. The masking apparently leaves partial information that cannot be integrated across locations, and we speculate that for spatially defined patterns like letters, integration across large shifts in location may be limited by problems in aligning successive samples. Our results also suggest that as attention moves, the selection of any given location (dwell time) can be as short as 50 ms, far shorter than the typical dwell time for stationary attention. Moving attention can therefore sample a brief instant of a rapidly changing stream if it passes quickly through, giving access to events that are otherwise not seen.     

Critical flicker frequency and M-scaling of stimulus size and retinal illuminance

Using various stimulus areas and luminances we measured monocular critical flicker frequency (CFF) as a function of eccentricity in the temporal visual field. With constant stimulus area and luminance, CFF was not independent of visual field location. When stimulus area was scaled by the magnification factor of the human striate cortex to produce equal cortical stimulus areas from different retinal locations, CFF increased monotonically with increasing eccentricity. Hence, CFF cannot be made independent of visual field location by spatial M-scaling. However, when also retinal illuminance was M-scaled by reducing stimulus luminance in inverse proportion to Ricco's area at each eccentricity, CFF became independent of visual field location.     

Normal retinotopic mapping in human strabismus with anomalous retinal correspondence

Burian proposed that a functional retinotopic remapping of the deviated eye on striate visual cortex may be the physiologic basis for the perceptual phenomenon of anomalous retinal correspondence (ARC) in human strabismus. This investigation searched for this type of retinotopic remapping in five esotropes and one exotrope with ARC by means of visual evoked potential (VEP) topographic mapping. Uniocular stimulation of the foveas (corresponding points) during binocular vision in a normal subject yielded identical VEP scalp topographies from each eye. Stimulation of anomalously corresponding points produced different VEP scalp topographies from each eye in the six strabismic subjects. Uniocular stimulation of the anatomic foveas of each eye (noncorresponding points) in a strabismic subject during binocular vision produced identical VEP scalp topographies. These results suggest that there is no significant functional binocular realignment of retinotopic mapping in the visual cortex of human strabismics with ARC.     

Visual field defects for vergence eye movements and for stereomotion perception

An objective visual field can be mapped in terms of stimulus-induced eye movement. The authors used the scleral coil technique to record vergence and conjugate eye movements while stimulating different visual field locations with a 3 X 3 deg target whose image vergence was oscillated. For each of three subjects tested there was a visual field location where vergence eye movements were much weaker than in a control location of equal retinal eccentricity. On the other hand, conjugate eye movements driven from these two locations by lateral motion were similar. Field defects for ocular vergence coincided with regions in which oscillating retinal disparity failed to produce a sensation of motion in depth, although visual responses to static disparity were normal, and psychophysical thresholds for lateral motion showed no defect with either binocular or monocular viewing. It was concluded, therefore, that the perceptual stereomotion scotomata were not due to a monocular loss, but to a defective binocular interaction between motion signals from the left and right eyes, and that this defective interaction was specific for opposed rather than parallel motion in the two eyes. Furthermore, the visual loss was specific for motion rather than for position. The correlation between the field defects for ocular vergence and stereomotion perception leads the authors to suggest that the same defect in binocular interaction is responsible for both the eye movement and sensory abnormalities. Two candidate hypotheses are proposed: one is framed in terms of a single population, and the other in terms of two populations of cortical neurons.     

Viewed actions are mapped in retinotopic coordinates in the human visual pathways

Viewed object-oriented actions elicit widespread fMRI activation in the dorsal and ventral visual pathways. This activation is typically stronger in the hemisphere contralateral to the visual field in which action is seen. However, since in previous studies participants kept fixation at the same screen position throughout the scan, it was impossible to infer if the viewed actions are represented in retina-based coordinates or in a more elaborated coordinate system. Here, participants changed their gaze between experimental conditions, such that some conditions shared the same retinotopic coordinates (but differed in their screen position), while other pairs of conditions shared the opposite trait. The degree of similarity between the patterns of activation elicited by the various conditions was assessed using multivoxel pattern analysis methods. Regions of interest, showing robust overall activation, included the intraparietal sulcus (IPS) and the occipitotemporal cortex. In these areas, the correlation between activation patterns for conditions sharing the same retinotopic coordinates was significantly higher than that of those having different retinotopic coordinates. In contrast, the correlations between activation patterns for conditions with the same spatiotopic coordinates were not significantly greater than for non-spatiotopic conditions. These results suggest that viewed object-oriented actions are likely to be maintained in retinotopic-framed coordinates.     

Hemiretinal differences in speed of light detection in esotropic amblyopes

Nine esotropic amblyopes were tested monocularly in a simple reaction time (RT) paradigm with brief suprathreshold flashes of light presented at various eccentricities along the horizontal meridian of the nasal or temporal hemiretinae. All were clinically amblyopic in one eye only. RT was significantly longer in the amblyopic than in the other eye at 1, 5 and 10 deg but not at 25 and 35 deg from the fovea. Another clearcut finding concerned hemiretinal differences: in the non-amblyopic eye, as in control subjects, RT was faster in the nasal than in the temporal hemiretina and such a difference increased with the eccentricity of stimulus presentation. In the amblyopic eye, however, the only significant hemiretinal effect was at 10 deg with a temporal retina advantage and at 35 deg with a nasal retinal advantage. Furthermore, unlike in normal control subjects and the non-amblyopic eye of our esotropes, in the amblyopic eye there was no increase in RT with the eccentricity of stimulus presentation, except for the most peripheral visual field positions. It can be concluded that esotropic amblyopia affects the speed of suprathreshold light detection in the most central 10 deg of visual field and that the nasal hemiretina is clearly more impaired than the temporal hemiretina.