Anisotropies in judging the direction of moving natural scenes

Although visual systems are optimized to deal with the natural visual environment, our understanding of human motion perception is in large part based on the use of artificial stimuli. Here, we assessed observers' ability to estimate the direction of translating natural images and fractals by having them adjust the orientation of a subsequently viewed line. A system of interleaved staircases, driven by observers' direction estimates, ensured that stimuli were presented near one of 16 reference directions. The resulting error distributions (i.e., the differences between reported and true directions) reveal several anisotropies in global motion processing. First, observers' estimates are biased away from cardinal directions (reference repulsion). Second, the standard deviations of estimates show an "oblique effect" being ～45% lower around cardinal directions. Third, errors around cardinal directions are more likely (～22%) to approach zero than would be consistent with Gaussian-distributed errors, suggesting that motion processing minimizes the number as well as magnitude of errors. Fourth, errors are similar for natural scenes and fractals, indicating that observers do not use top-down information to improve performance. Finally, adaptation to unidirectional motion modifies observers' bias by amplifying existing repulsion (e.g., around cardinal directions). This bias change can improve direction discrimination but is not due to a reduction in variability.     

Centrifugal propagation of motion adaptation effects across visual space

Perceptual distortions induced by adaptation (aftereffects) arise through selective changes in the response properties of discrete subpopulations of neurons tuned to particular image features at the adapted spatial location. The systematic and well-documented increase of cortical receptive field sizes with eccentricity dictates that visual aftereffects ought to become less tightly tuned for location as stimuli are moved away from fixation. Here, we demonstrate that while this pattern holds for archetypal orientation and spatial frequency aftereffects, the effects of motion adaptation are characterized by precisely the opposite relationship. Surprisingly, adaptation to translational motion close to fixation induces distortions of perceived position and dynamic motion aftereffects that propagate centrifugally across visual space, resulting in a lack of location specificity. In contrast, motion adaptation in more peripheral locations produces aftereffects that are largely limited to the adapted spatial region. These findings suggest that central motion adaptation has the unique capacity to influence the response state of spatially distant neural populations that do not themselves encode the adapting stimulus.     

Convergent evidence for the visual analysis of optic flow through anisotropic attenuation of high spatial frequencies

Photoreceptors strongly attenuate high temporal frequencies. Hence when an image moves, high spatial frequency components are lost if their direction of modulation coincides with the direction of movement, but not if it is orthogonal. The power spectra of natural images are remarkably consistent in having a 1/f 2 falloff in power in all directions. For moving images, the spatial power spectra will be distorted by becoming steeper in the direction corresponding to modulation in the direction of motion, and the contours of equal power will tend to become elliptical. This study demonstrates that the mammalian visual system is specifically sensitive to such anisotropic changes of the local power spectrum, and it is suggested that these distortions are used to determine patterns of optic flow. Convergent evidence from work on Glass figures, motion streaks, and sensitivity to non-Cartesian gratings is called on in support of this interpretation, which has been foreshadowed in several recent publications.     

Spatiotemporal dynamics of perisaccadic remapping in humans revealed by classification images

We actively scan our environment with fast ballistic movements called saccades, which create large and rapid displacements of the image on the retina. At the time of saccades, vision becomes transiently distorted in many ways: Briefly flashed stimuli are displaced in space and in time, and spatial and temporal intervals appear compressed. Here we apply the psychophysical technique of classification images to study the spatiotemporal dynamics of visual mechanisms during saccades. We show that saccades cause gross distortions of the classification images. Before the onset of saccadic eye movements, the positive lobes of the images become enlarged in both space and in time and also shifted in a systematic manner toward the pre-saccadic fixation (in space) and anticipated in time by about 50 ms. The transient reorganization creates a spatiotemporal organization oriented in the direction of saccadic-induced motion at the time of saccades, providing a potential mechanism for integrating stimuli across saccades, facilitating stable and continuous vision in the face of constant eye movements.     

Electrophysiological evidence for independent speed channels in human motion processing

A variety of psychophysical studies suggests that motion perception in humans is mediated by at least two speed-tuned channels. To study the neurophysiological underpinnings of these channels in the human visual cortex, we recorded visual evoked potentials (VEPs) to motion onset. We applied an adaptation paradigm that allowed us (a) to isolate and extract direction-specific cortical responses and (b) to assess cross-adaptation in the speed domain. VEPs resulting from the onset of left- or rightward motion at either low or high speeds were recorded from three occipital recording sites in 11 subjects. For each of these test stimuli, responses were collected after adaptation to one of five different conditions: a static adaptation pattern (baseline), adaptation to low-speed motion (3.5 degrees/s) either in the same or in the opposite direction as the test, or adaptation to high-speed motion (32 degrees/s) either in the same or in the opposite direction as the test. We report considerable direction-specific adaptation for same adaptation and test speeds (by 28-37% of baseline response; p <.002), whereas there was no direction-specific adaptation across speeds. We supplement these electrophysiological data with corresponding psychophysical results. The lack of direction-specific cross-adaptation in the speed domain demonstrated with physiological and psychophysical techniques supports models of at least two speed-tuned channels in the human motion system.     

Towards the cortical representation of form and motion stimuli generated by a retina implant

Background A retina implant for restoring basic visual perception in patients who are blind due to photoreceptor loss should not only evoke focal phosphenes at high resolution, but should also generate cortical representations of form and motion. We are currently exploring these potential capabilities in anaesthetised cats.Methods Fibre electrodes were inserted through a small scleral incision onto the retinal surface for stimulation. For the recording of cortical population activities we placed up to 16 fibre electrodes in areas 17 and/or 18. Retinal and cortical electrodes were adjusted to corresponding sites, i.e., overlapping receptive fields. Electrical stimuli were charge-balanced impulses (200 µs, 10–100 µA). Basic form stimuli were generated by the selective and synchronous activation of some of the seven retinal stimulation electrodes. Movement stimuli were generated by spatio-temporal shifting of form stimuli. From multiple microelectrode recordings we computed stimulus-related spatio-temporal cortical activation profiles. We used these profiles to estimate the relations between stimulation distance and spatial resolution (form) and between stimulus velocity and spatio-temporal resolution (movement). Influences by the retino-cortical pathway were assessed by comparing cortical activations evoked by true form or motion stimuli with synthetic responses composed by superpositioning of responses to appropriate subsets of form and motion stimuli. In addition, we compared cortical responses to form and motion stimuli by a receptive-field-based backprojection of cortical activities.Results We confirmed our previous finding that electrical retina stimulation may yield a spatial resolution of 1–5° visual angle and a temporal resolution of about 20 ms. We found that the spatio-temporal cortical activation profiles are commonly related to retinal form and motion stimuli. Cortical activity analyses showed that for two-point form stimuli the neuronal interaction depends on the stimulation electrodes' distance and that local cortical group activities can exhibit some tuning to the directions or the velocities of moving electrical bars'. Projections of cortical activations to visual space were consistent with electrical form and motion stimulation of the retina.Conclusions Our data indicate that retinal stimulation with electrical form and motion stimuli can lead to spatio-temporally related cortical activations. However, the selective activation of single cortical neurones with specific visual tuning properties by electrical retina stimulation and the potential adaptation of the visual system to long-term stimulation with retina implants should be addressed in future work.This study was supported by the German Federal Ministry for Education, Science, Research and Technology, BMBF, grant 01 KP 0006.     

Large shifts in perceived motion direction reveal multiple global motion solutions

Moving objects are thought to be decomposed into one-dimensional motion components by early cortical visual processing. Two rules describing how these components might be re-combined to produce coherent object motion are the intersection of constraints and the vector average rules. Using stimuli for which these combination rules predict different directional solutions, we found that adapting one of the solutions through motion adaptation switched perceived direction to the other solution. The effects were symmetrical: shifts from IOC to VA, and from VA to IOC, were observed following adaptation. These large shifts indicate that multiple solutions to global motion processing coexist and compete to determine perceived motion direction.     

A method for the real-time rendering of formless dot field structure-from-motion stimuli

The perception of visual motion relies on different computations and different neural substrates than the perception of static form. It is therefore useful to have psychophysical stimuli that carry mostly or entirely motion information, conveying little or nothing about form in any single frame. Structure-from-motion stimuli can sometimes achieve this dissociation, with some examples in studies of biological motion using point-light walkers. It is, however, generally not trivial to provide motion information without also providing static form information. The problem becomes more computationally difficult when the structures and the motions in question are complex. Here we present a technique by which an animated three-dimensional scene can be rendered in real-time as a pattern of dots. Each dot follows the trajectory of the underlying object in the animation, but each static frame of the animation appears to be a uniform random field of dots. The resulting stimuli capture motion vectors across arbitrary complex scenes, while providing virtually no instantaneous information about the structure of that scene. We also present the results of a psychophysical experiment demonstrating the efficacy and the limitations of the technique. The ability to create such stimuli on the fly allows for interactive adjustment and control of the stimuli, real-time parametric variations of structure and motion, and the creation of large libraries of actions without the need to pre-render a prohibitive number of movies. This technique provides a powerful tool for the dissociation of complex motion from static form.     

Second-order motion conveys depth-order information

Psychophysical and neurophysiological studies have revealed that the visual system is sensitive to both "first-order" motion, in which moving features are defined by luminance cues, and "second-order" motion, in which motion is defined by nonluminance cues, such as contrast or flicker. Here we show psychophysically that common types of second-order stimuli provide potent cues to depth order. Although motion defined exclusively by nonluminance cues may be relatively rare in natural scenes, the depth-order cues offered by second-order stimuli arise ubiquitously as a result of occlusion of one moving object by another. Our results thus shed new light on the ecological importance of second-order motion. Furthermore, our results imply that visual cortical areas that have been shown to be responsive to second-order motion may be extracting information not just about object motion as has been assumed, but also about the relative depth of objects.     

Perception of dynamic Glass patterns

In the mammalian brain, form and motion are processed through two distinct pathways at early stages of visual processing. However, recent evidence suggests that these two pathways may interact. Here we used dynamic Glass patterns, which have been previously shown to create the perception of coherent motion in humans, despite containing no motion coherence. Glass patterns are static stimuli that consist of randomly positioned dot pairs that are integrated spatially to create the perception of a global form, whereas dynamic Glass patterns consist of several independently generated static Glass patterns presented sequentially. In the current study, we measured the detection threshold of five types of dynamic Glass patterns and compared the rank order of the detection thresholds with those found for static Glass patterns and real motion patterns (using random dot stimuli). With both the static Glass patterns and dynamic Glass patterns, detection thresholds were lowest for concentric and radial patterns and highest for horizontal patterns. We also found that vertical patterns were better detected than horizontal patterns, consistent with prior evidence of a “horizontal effect” in the perception of natural scene images. With real motion, detection thresholds were equivalent across all patterns, with the exception of higher thresholds for spiral patterns. Our results suggest that dynamic Glass patterns are processed primarily as form prior to input into the motion system.     

Auditory modulation of visual apparent motion with short spatial and temporal intervals

Recently, E. Freeman and J. Driver (2008) reported a cross-modal temporal interaction in which brief sounds drive the perceived direction of visual apparent-motion, an effect they attributed to "temporal capture" of the visual stimuli by the sounds (S. Morein-Zamir, S. Soto-Faraco, & A. Kingstone, 2003). Freeman and Driver used "long-range" visual motion stimuli, which travel over long spatial and temporal intervals and engage high-order cortical areas (K. G. Claeys, D. T. Lindsey, E. De Schutter, & G. A. Orban, 2003; Y. Zhuo et al., 2003). We asked whether Freeman and Driver's temporal effects extended to the short-range apparent-motion stimuli that engage cortical area MT, a lower-order area with well-established spatiotemporal selectivity for visual motion (e.g. A. Mikami, 1991, 1992; A. Mikami, W. T. Newsome, & R. H. Wurtz, 1986a, 1986b; W. T. Newsome, A. Mikami, & R. H. Wurtz, 1986). Consistent with a temporal-capture account, we found that static sounds bias the perception of both the direction (Experiment 1) and the speed (Experiment 2) of short-range motion. Our results suggest that auditory timing may interact with visual spatiotemporal processing as early as cortical area MT. Examination of the neuronal responses of this well-studied area to the stimuli used in this study would provide a test and might provide insight into the neuronal representation of time.     

Contrast gain control in natural scenes

Behavioral and electrophysiological studies of visual processing routinely employ sine wave grating stimuli, an approach that has led to the development of models in which the first stage of cortical visual processing acts as a bank of narrowband local filters whose responses vary with the contrast of preferred structure falling within their receptive fields. The relevance of this approach to natural vision is currently being challenged. We examine the contrast response of the human visual system to natural scenes. The results support a narrowband approach to visual processing but require its elaboration. Unlike grating patterns, the contrast response to natural scenes depends on the phase structure at remote spatial scales, but over a limited spatial region. The results suggest that contrast gain control acts within, but not across, cortical hypercolumns and serves to reduce the difference between the responses of detectors in regions of high and low contrast. This process tends to normalize the response of the visual system across natural scenes, which contain uneven contrast distributions.     

The posterior cingulate cortex and planum temporale/parietal operculum are activated by coherent visual motion

The posterior cingulate cortex (PCC) is involved in higher order sensory and sensory-motor integration while the planum temporale/parietal operculum (PT/PO) junction takes part in auditory motion and vestibular processing. Both regions are activated during different types of visual stimulation. Here, we describe the response characteristics of the PCC and PT/PO to basic types of visual motion stimuli of different complexity (complex and simple coherent as well as incoherent motion). Functional magnetic resonance imaging (fMRI) was performed in 10 healthy subjects at 3 Tesla, whereby different moving dot stimuli (vertical, horizontal, rotational, radial, and random) were contrasted against a static dot pattern. All motion stimuli activated a distributed cortical network, including previously described motion-sensitive striate and extrastriate visual areas. Bilateral activations in the dorsal region of the PCC (dPCC) were evoked using coherent motion stimuli, irrespective of motion direction (vertical, horizontal, rotational, radial) with increasing activity and with higher complexity of the stimulus. In contrast, the PT/PO responded equally well to all of the different coherent motion types. Incoherent (random) motion yielded significantly less activation both in the dPCC and in the PT/PO area. These results suggest that the dPCC and the PT/PO take part in the processing of basic types of visual motion. However, in dPCC a possible effect of attentional modulation resulting in the higher activity evoked by the complex stimuli should also be considered. Further studies are warranted to incorporate these regions into the current model of the cortical motion processing network.     

Contrast conservation in human vision

Visual experience, which is defined by brief saccadic sampling of complex scenes at high contrast, has typically been studied with static gratings at threshold contrast. To investigate how suprathreshold visual processing is related to threshold vision, we tested the temporal integration of contrast in the presence of large, sudden changes in the stimuli such occur during saccades under natural conditions. We observed completely different effects under threshold and suprathreshold viewing conditions. The threshold contrast of successively presented gratings that were either perpendicularly oriented or of inverted phase showed probability summation, implying no detectable interaction between independent visual detectors. However, at suprathreshold levels we found complete algebraic summation of contrast for stimuli longer than 53 ms. The same results were obtained during sudden changes between random noise patterns and between natural scenes. These results cannot be explained by traditional contrast gain-control mechanisms or the effect of contrast constancy. Rather, at suprathreshold levels, the visual system seems to conserve the contrast information from recently viewed images, perhaps for the efficient assessment of the contrast of the visual scene while the eye saccades from place to place.     

Image statistics and the perception of surface gloss and lightness

Despite previous data demonstrating the critical importance of 3D surface geometry in the perception of gloss and lightness, I. Motoyoshi, S. Nishida, L. Sharan, and E. H. Adelson (2007) recently proposed that a simple image statistic--histogram or sub-band skew--is computed by the visual system to infer the gloss and albedo of surfaces. One key source of evidence used to support this claim was an experiment in which adaptation to skewed image statistics resulted in opponent aftereffects in observers' judgments of gloss and lightness. We report a series of adaptation experiments that were designed to assess the cause of these aftereffects. We replicated their original aftereffects in gloss but found no consistent aftereffect in lightness. We report that adaptation to zero-skew adaptors produced similar aftereffects as positively skewed adaptors, and that negatively skewed adaptors induced no reliable aftereffects. We further find that the adaptation effect observed with positively skewed adaptors is not robust to changes in mean luminance that diminish the intensity of the luminance extrema. Finally, we show that adaptation to positive skew reduces (rather than increases) the apparent lightness of light pigmentation on non-uniform albedo surfaces. These results challenge the view that the adaptation results reported by Motoyoshi et al. (2007) provide evidence that skew is explicitly computed by the visual system.     

Gaze modulation of visual aftereffects

Physiological studies of non-human primates have suggested that the direction of gaze can modulate the gain of neuronal responses to visual stimuli in many cortical areas including V1. The neural gaze modulation is suggested to subserve the conversion from gaze-independent (eye-centered) to dependent (e.g., head-centered) representations. However, it has not been established whether the gaze modulation has significant influences on human visual perception. Here we show that gaze direction modestly but significantly modulates the magnitudes of the motion aftereffect, the tilt aftereffect and the size aftereffect. These aftereffects were stronger when the adaptation and test patterns were presented in the same gaze direction, than when they were presented in different gaze directions, even though the patterns always stimulated the same retinal location. The gaze modulation effect was not statistically significant for the post-adaptation elevation of contrast detection thresholds. The gaze modulation of visual aftereffects provides a useful psychophysical tool to analyze human cortical processes for coordinate transformations of visual space.     

The motion aftereffect of transparent motion: two temporal channels account for perceived direction

Adaptation to orthogonal transparent patterns drifting at the same speed produces a unidirectional motion aftereffect (MAE) whose direction is opposite the average adaptation direction. If the patterns move at different speeds, MAE direction can be predicted by an inverse vector average, using the observer's motion sensitivity to each individual pattern as vector magnitudes. These weights are well approximated by the duration of each pattern's MAE, as measured with static test patterns. However, previous efforts to use the inverse-vector-average rule with dynamic test patterns have failed. Generally, these studies have used spatially and temporally broadband test stimuli. Here, in order to gain insight into the possible contribution of temporal channels, we filtered our test pattern in the temporal domain to produce five ideal, octave-width pass-bands. MAE durations were measured for single-component stimuli drifting at various adaptation speeds and tested at a range of temporal frequencies. Then, two components with orthogonal directions and different speeds were combined and the direction of the resulting MAE was measured. The key findings are that: (i) for a given adaptation speed, the duration of a single component's MAE is dependent on test temporal frequency; (ii) the direction of MAEs produced by transparent motion (i.e., bivectorial adaptation) also varies strongly as a function test temporal frequency (by up to 90 degrees for some speed pairings); and (iii) the inverse-vector-average rule predicts the direction of the transparent MAE provided the MAE durations used to weight the vector combination were obtained from stimuli matched in adaptation speed and test temporal frequency. These results are discussed in terms of the number and shape of temporal channels in our visual system.     

Direction anisotropy of human motion perception depends on stimulus speed

A number of previous studies have extensively investigated directional anisotropy in motion perception. However, consensus has not been reached regarding the nature of motion directional anisotropies in human vision. In this study, we investigated the directional anisotropy of human motion perception by moving random-dot stimuli in the peripheral upper visual field. Our findings show that the degree of directional anisotropy depends on the stimulus speed. Furthermore, the high and low speed conditions have preferred directions that are opposite. This may reflect differences in the directional information among temporal frequencies in natural scenes. These differences are thought to have crucial roles in the detection of motion direction.     

Adaptation to apparent motion in crowding condition

Visual adaptation has been successfully used for studying the neural activity of different cortical areas in response to visual stimuli when observers do not have explicit conscious access to those stimuli. We compared the orientation selective adaptation to apparent motion and its effect on the perception of stimuli with bistable apparent motion in crowded and non-crowded conditions. In the crowding paradigm conscious access to a visual stimulus is severely impaired when it is flanked by other similar stimuli in the peripheral visual field. As expected, adaptation to the target stimulus occurred in the non-crowded condition in all of the individual subjects (n=4; P<0.001). Although in the crowded condition subjects were not able to discriminate the target stimulus, adaptation to that stimulus was still preserved (P<0.001). There was no significant difference between the adaptations in the two conditions of the apparent motion (P>0.05). Imaging studies have shown that V5 cortex is the earliest visual area that specifically responds to apparent motion. Our results suggest that in certain conditions V5 may be activated while there is no explicit conscious access to the apparent motion.     

Adaptation to temporal modulation can enhance differential speed sensitivity

During adaptation to a moving pattern, perceived speed decreases. Thus we know that the adapted visual system does not simply code the absolute speed of a stimulus. We hypothesised that adaptation to a moving stimulus serves to optimise coding of changes in speed at the expense of maintaining an accurate representation of absolute speed. In this case we would expect discrimination of speeds around the adapted level to be preserved or enhanced by motion adaptation. Speed discrimination thresholds were measured for sinusoidal gratings (1.25 cpd; 12.5 Hz; 40% contrast) with and without prior adaptation to moving, static, and flickering stimuli. After adaptation to motion in the same direction as the test, seven of eight subjects showed a reduction of perceived speed in the adapted region, and seven showed enhanced discrimination. Similar effects were found for adaptation to motion in the opposite direction to the test and to counter-phase flicker, suggesting that adaptation is driven by temporal modulation rather than by motion per se. We conclude that motion adaptation preserves or enhances differential speed sensitivity at the expense of an accurate representation of absolute speed.