Neural coding of global form in the human visual cortex

Extensive psychophysical and computational work proposes that the perception of coherent and meaningful structures in natural images relies on neural processes that convert information about local edges in primary visual cortex to complex object features represented in the temporal cortex. However, the neural basis of these mid-level vision mechanisms in the human brain remains largely unknown. Here, we examine functional MRI (fMRI) selectivity for global forms in the human visual pathways using sensitive multivariate analysis methods that take advantage of information across brain activation patterns. We use Glass patterns, parametrically varying the perceived global form (concentric, radial, translational) while ensuring that the local statistics remain similar. Our findings show a continuum of integration processes that convert selectivity for local signals (orientation, position) in early visual areas to selectivity for global form structure in higher occipitotemporal areas. Interestingly, higher occipitotemporal areas discern differences in global form structure rather than low-level stimulus properties with higher accuracy than early visual areas while relying on information from smaller but more selective neural populations (smaller voxel pattern size), consistent with global pooling mechanisms of local orientation signals. These findings suggest that the human visual system uses a code of increasing efficiency across stages of analysis that is critical for the successful detection and recognition of objects in complex environments.     

Hierarchy of direction-tuned motion adaptation in human visual cortex

Prolonged exposure to a single direction of motion alters perception of subsequent static or dynamic stimuli and induces substantial changes in behaviors of motion-sensitive neurons, but the origin of neural adaptation and neural correlates of perceptual consequences of motion adaptation in human brain remain unclear. Using functional magnetic resonance imaging, we measured motion adaptation tuning curves in a fine scale by probing changes in cortical activity after adaptation for a range of directions relative to the adapted direction. We found a clear dichotomy in tuning curve shape: cortical responses in early-tier visual areas reduced at around both the adapted and opposite direction, resulting in a bidirectional tuning curve, whereas response reduction in high-tier areas occurred only at around the adapted direction, resulting in a unidirectional tuning curve. We also found that the psychophysically measured adaptation tuning curves were unidirectional and best matched the cortical adaptation tuning curves in the middle temporal area (MT) and the medial superior temporal area (MST). Our findings are compatible with, but not limited to, an interpretation in which direct impacts of motion adaptation occur in both unidirectional and bidirectional units in early visual areas, but the perceptual consequences of motion adaptation are manifested in the population activity in MT and MST, which may inherit those direct impacts of adaptation from the directionally selective units.     

Predictive coding for motion stimuli in human early visual cortex

The current study investigates if early visual cortical areas, V1, V2 and V3, use predictive coding to process motion information. Previous studies have reported biased visual motion responses at locations where novel visual information was presented (i.e., the motion trailing edge), which is plausibly linked to the predictability of visual input. Using high-field functional magnetic resonance imaging (fMRI), we measured brain activation during predictable versus unpreceded motion-induced contrast changes during several motion stimuli. We found that unpreceded moving dots appearing at the trailing edge gave rise to enhanced BOLD responses, whereas predictable moving dots at the leading edge resulted in suppressed BOLD responses. Furthermore, we excluded biases in directional sensitivity, shifts in cortical stimulus representation, visuo-spatial attention and classical receptive field effects as viable alternative explanations. The results clearly indicate the presence of predictive coding mechanisms in early visual cortex for visual motion processing, underlying the construction of stable percepts out of highly dynamic visual input.     

Remapping in human visual cortex

With each eye movement, stationary objects in the world change position on the retina, yet we perceive the world as stable. Spatial updating, or remapping, is one neural mechanism by which the brain compensates for shifts in the retinal image caused by voluntary eye movements. Remapping of a visual representation is believed to arise from a widespread neural circuit including parietal and frontal cortex. The current experiment tests the hypothesis that extrastriate visual areas in human cortex have access to remapped spatial information. We tested this hypothesis using functional magnetic resonance imaging (fMRI). We first identified the borders of several occipital lobe visual areas using standard retinotopic techniques. We then tested subjects while they performed a single-step saccade task analogous to the task used in neurophysiological studies in monkeys, and two conditions that control for visual and motor effects. We analyzed the fMRI time series data with a nonlinear, fully Bayesian hierarchical statistical model. We identified remapping as activity in the single-step task that could not be attributed to purely visual or oculomotor effects. The strength of remapping was roughly monotonic with position in the visual hierarchy: remapped responses were largest in areas V3A and hV4 and smallest in V1 and V2. These results demonstrate that updated visual representations are present in cortical areas that are directly linked to visual perception.     

Implied motion perception from a still image in infancy

Visual motion perception can arise from non-directional visual stimuli, such as still images (implied motion, cf. Kourtzi, Trends Cogn Sci 8:47–49, 2004). We tested 5- to 8-month-old infants’ implied motion perception with two experiments using the forced-choice preferential looking method. Our results indicated that a still image of a person running toward either the left or right side significantly enhanced infants’ visual preference for a visual target that consistently appeared on the same side as the running direction (the run condition in Experiment 1). Such enhanced visual preference disappeared in response to an image of the same person standing and facing the left/right side (the stand condition in Experiment 1), an image of the running figure covered with a set of opaque rectangles (the block condition in Experiment 2) (Gervais et al. in Atten Percept Psychophys 72:1437–1443, 2010), and an image of the inverted running figure (the inversion condition in Experiment 3). These results suggest that only the figure that implied dynamic body motion shifted the infants’ visual preference to the same direction as the implied running action. These findings demonstrate that even infants as young as 5 to 8 months old are sensitive to the implied motion of static figures.     

Cross-orientation suppression in human visual cortex

Cross-orientation suppression was measured in human primary visual cortex (V1) to test the normalization model. Subjects viewed vertical target gratings (of varying contrasts) with or without a superimposed horizontal mask grating (fixed contrast). We used functional magnetic resonance imaging (fMRI) to measure the activity in each of several hypothetical channels (corresponding to subpopulations of neurons) with different orientation tunings and fit these orientation-selective responses with the normalization model. For the V1 channel maximally tuned to the target orientation, responses increased with target contrast but were suppressed when the horizontal mask was added, evident as a shift in the contrast gain of this channel's responses. For the channel maximally tuned to the mask orientation, a constant baseline response was evoked for all target contrasts when the mask was absent; responses decreased with increasing target contrast when the mask was present. The normalization model provided a good fit to the contrast-response functions with and without the mask. In a control experiment, the target and mask presentations were temporally interleaved, and we found no shift in contrast gain, i.e., no evidence for suppression. We conclude that the normalization model can explain cross-orientation suppression in human visual cortex. The approach adopted here can be applied broadly to infer, simultaneously, the responses of several subpopulations of neurons in the human brain that span particular stimulus or feature spaces, and characterize their interactions. In addition, it allows us to investigate how stimuli are represented by the inferred activity of entire neural populations.     

Visual motion perception from stimulation of the human medial parieto-occipital cortex

Summary Visual phenomena evoked by direct electrical stimulation of extrastriate cortex were observed in 30 epileptic patients as part of a presurgical investigation. An incremental sequence of low-level bipolar stimulation trains was delivered at medial and lateral pairs of contacts of stereotaxically-implanted multilead intracerebral electrodes in parietal, occipital and posterior temporal regions. Diffusion of stimulus afterdischarges was monitored by electrodes in temporal and frontal lobes and by the non-stimulated contacts of the stimulated electrode. Localized stimulations evoked few visual phenomena. The strongest anatomo-perceptual correlation was found for stimulation in the medial parieto-occipital fissure which evoked visual motion phenomena in all three patients stimulated in that region. The evoked motion perceptions were not associated with eye movements or any particular localization of the epileptic focus. These perceptions were only evoked once outside of the medial PO region at the 61 sites examined. The results suggest that the medial parieto-occipital region is closely linked to the human visual motion processing system.     

Uncertainty and invariance in the human visual cortex

The way in which input noise perturbs the behavior of a system depends on the internal processing structure of the system. In visual psychophysics, there is a long tradition of using external noise methods (i.e., adding noise to visual stimuli) as tools for system identification. Here, we demonstrate that external noise affects processing of visual scenes at different cortical areas along the human ventral visual pathway, from retinotopic regions to higher occipitotemporal areas implicated in visual shape processing. We found that when the contrast of the stimulus was held constant, the further away from the retinal input a cortical area was the more its activity, as measured with functional magnetic resonance imaging (fMRI), depended on the signal-to-noise ratio (SNR) of the visual stimulus. A similar pattern of results was observed when trials with correct and incorrect responses were analyzed separately. We interpret these findings by extending signal detection theory to fMRI data analysis. This approach reveals the sequential ordering of decision stages in the cortex by exploiting the relation between fMRI response and stimulus SNR. In particular, our findings provide novel evidence that occipitotemporal areas in the ventral visual pathway form a cascade of decision stages with increasing degree of signal uncertainty and feature invariance.     

Predicting the orientation of invisible stimuli from activity in human primary visual cortex

Humans can experience aftereffects from oriented stimuli that are not consciously perceived, suggesting that such stimuli receive cortical processing. Determining the physiological substrate of such effects has proven elusive owing to the low spatial resolution of conventional human neuroimaging techniques compared to the size of orientation columns in visual cortex. Here we show that even at conventional resolutions it is possible to use fMRI to obtain a direct measure of orientation-selective processing in V1. We found that many parts of V1 show subtle but reproducible biases to oriented stimuli, and that we could accumulate this information across the whole of V1 using multivariate pattern recognition. Using this information, we could then successfully predict which one of two oriented stimuli a participant was viewing, even when masking rendered that stimulus invisible. Our findings show that conventional fMRI can be used to reveal feature-selective processing in human cortex, even for invisible stimuli.     

Pattern-motion responses in human visual cortex.

Physiological models of visual motion processing posit that 'pattern-motion cells' represent the direction of moving objects independent of their particular spatial pattern. We performed fMRI experiments to identify neuronal activity in the human brain selective for pattern motion. A protocol using adaptation to moving 'plaid' stimuli allowed us to separate pattern-motion responses from other types of motion-related activity within the same brain structures, and revealed strong pattern-motion selectivity in human visual area MT+. Reducing the perceptual coherence of the plaids yielded a corresponding decrease in pattern-motion responsivity, providing evidence that percepts of coherent motion are closely linked to the activity of pattern-motion cells in human MT+.     

Orientation-tuned FMRI adaptation in human visual cortex

Adaptation is a general property of almost all neural systems and has been a longstanding tool of psychophysics because of its power to isolate and temporarily reduce the contribution of specific neural populations. Recently, adaptation designs have been extensively applied in functional MRI (fMRI) studies to infer neural selectivity in specific cortical areas. However, there has been considerable variability in the duration of adaptation used in these experiments. In particular, although long-term adaptation has been solidly established in psychophysical and neurophysiological studies, it has been incorporated into few fMRI studies. Furthermore, there has been little validation of fMRI adaptation using stimulus dimensions with well-known adaptive properties (e.g., orientation) and in better understood regions of cortex (e.g., primary visual cortex, V1). We used an event-related fMRI experiment to study long-term orientation adaptation in the human visual cortex. After long-term adaptation to an oriented pattern, the fMRI response in V1, V2, V3/VP, V3A, and V4 to a test stimulus was proportional to the angular difference between the adapting and test stimuli. However, only V3A and V4 showed this response pattern with short-term adaptation. In a separate experiment, we measured behavioral contrast detection thresholds after adaptation and found that the fMRI signal in V1 closely matched the psychophysically derived contrast detection thresholds. Similar to the fMRI results, adaptation induced threshold changes strongly depended on the duration of adaptation. In addition to supporting the existence of adaptable orientation-tuned neurons in human visual cortex, our results show the importance of considering timing parameters in fMRI adaptation experiments.     

Multiple spatial-frequency tuning of electrical responses from human visual cortex

Summary Human occipital potentials evoked by stimulation with a counterphase flickering grating were recorded by a digital narrowband filter technique. The data showed a surprising degree of narrow tuning to particular spatial frequencies in addition to the expected narrow temporal frequency tuning. At each temporal frequency, there could be two or more peaks of response to different spatial frequencies, each distinct from the whole field flicker response. Variations in this multiple spatial frequency tuning were investigated as a function of luminance, electrode location, and temporal frequency for several observers. The results are interpreted in terms of many stimulus-specific resonant neural circuits within the brain, and suggest that it is possible to make a highly detailed exploration of the responses of neural circuits to visual stimulation.Supported by NIH Grants No. 5 R01 EY01582, No. 1 R01 EY02124, NIH General Research Grant No. 5 S01 RR05566 and The Smith-Kettlewell Eye Research Foundation     

3D shape perception from combined depth cues in human visual cortex

Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.     

An oblique effect in human primary visual cortex

Visual perception critically depends on orientation-specific signals that arise early in visual processing. Humans show greater behavioral sensitivity to gratings with horizontal or vertical (0 degrees /90 degrees; 'cardinal') orientations than to other, 'oblique' orientations. Here we used functional magnetic resonance imaging (fMRI) to measure an asymmetry in the responses of human primary visual cortex (V1) to oriented stimuli. We found that neural responses in V1 were larger for cardinal stimuli than for oblique (45 degrees /135 degrees ) stimuli. Thus the fMRI pattern in V1 closely resembled subjects' behavioral judgments; responses in V1 were greater for those orientations that yielded better perceptual performance.     

Task-dependent activation latency in human visual extrastriate cortex

Event related potentials (ERPs) were recorded from subjects who had to perform either an identification task or a simple detection task on moving visual stimuli. Results showed that the amplitude of the so-called visual "N1" component was larger for identification than for mere detection, replicating previous data obtained with static stimuli. However, we also found that: (i) the onset, peak and offset latencies of the visual N1 to dynamic stimuli were significantly earlier in the detection task than in the identification task, and (ii) in both conditions, the coordinates of the equivalent current dipoles best explaining the visual N1 component were consistent with those of the human motion visual area MT+/V5 in the extrastriate cortex. Altogether, these results indicate that dynamic stimuli may activate (at least partly) different pathways and processes in extrastriate cortex according to the nature of the task required on these stimuli.     

Striate cortex in humans demonstrates the relationship between activation and variations in visual form

Electrophysiologic and functional imaging studies have shown that the visual cortex produces differential responses to the presence or absence of structure within visual textures. To further define and characterize regions involved in the analysis of form, functional magnetic resonance imaging (fMRI) was used to detect changes in activation during the viewing of four levels of isodipole textures. The texture levels systematically differed in the density of visual features such as extended contours and blocks of solid color present within the images. A linear relationship between activation level and density of structure was observed in the striate cortex of human subjects. This finding suggests that a special subpopulation of striate cortical neurons participates in the ability to extract and process structural continuity within visual stimuli. Received: 8 July 1998 / Accepted: 25 May 1999     

Decoding conjunctions of direction-of-motion and binocular disparity from human visual cortex

Motion and binocular disparity are two features in our environment that share a common correspondence problem. Decades of psychophysical research dedicated to understanding stereopsis suggest that these features interact early in human visual processing to disambiguate depth. Single-unit recordings in the monkey also provide evidence for the joint encoding of motion and disparity across much of the dorsal visual stream. Here, we used functional MRI and multivariate pattern analysis to examine where in the human brain conjunctions of motion and disparity are encoded. Subjects sequentially viewed two stimuli that could be distinguished only by their conjunctions of motion and disparity. Specifically, each stimulus contained the same feature information (leftward and rightward motion and crossed and uncrossed disparity) but differed exclusively in the way these features were paired. Our results revealed that a linear classifier could accurately decode which stimulus a subject was viewing based on voxel activation patterns throughout the dorsal visual areas and as early as V2. This decoding success was conditional on some voxels being individually sensitive to the unique conjunctions comprising each stimulus, thus a classifier could not rely on independent information about motion and binocular disparity to distinguish these conjunctions. This study expands on evidence that disparity and motion interact at many levels of human visual processing, particularly within the dorsal stream. It also lends support to the idea that stereopsis is subserved by early mechanisms also tuned to direction of motion.     

Seeing speech: visual information from lip movements modifies activity in the human auditory cortex

Neuromagnetic responses were recorded over the left hemisphere to find out in which cortical area the heard and seen speech are integrated. Auditory stimuli were Finnish/pa/syllables presented together with a videotaped face articulating either the concordant syllable/pa/(84% of stimuli, V = A) or the discordant syllable/ka/(16%, V not equal to A). In some subjects the probabilities were reversed. The subjects heard V not equal to A stimuli as/ta/ or ka. The magnetic responses to infrequent perceptions elicited a specific waveform which could be explained by activity in the supratemporal auditory cortex. The results show that visual information from articulatory movements has an entry into the auditory cortex.