FMRI adaptation reveals separate mechanisms for first-order and second-order motion

A key unresolved debate in human vision concerns whether we have two different low-level mechanisms for encoding image motion. Separate neural mechanisms have been suggested for first-order (luminance modulation) and second-order (e.g., contrast modulation) motion in the retinal image but a single mechanism could handle both. Human functional magnetic resonance imaging (fMRI) has not so far convincingly revealed separate anatomical substrates. To examine whether two separate but co-localized mechanisms might exist, we used the technique of fast fMRI adaptation. We found direction-selective adaptation independently for each type of motion in the motion area V5/MT+ of the human brain. However, there was a total absence of cross-adaptation between first-order and second-order motion stimuli. This was true in both of the two subcomponents of MT+ (MT and MST) and similar results were found in V3A. This pattern of adaptation was consistent with psychophysical measurements of detection thresholds in similar stimulus sequences. The results provide strong evidence for separate neural populations that are responsible for detecting first- and second-order motion.     

Delayed discrimination of spatial frequency for gratings of different orientation: behavioral and fMRI evidence for low-level perceptual memory stores in early visual cortex

The concept of perceptual memory refers to the neural and cognitive processes underlying the storage of specific stimulus features such as spatial frequency, orientation, shape, contrast, and color. Psychophysical studies of perceptual memory indicate that observers can retain visual information about the spatial frequency of Gabor patterns independent of the orientation with which they are presented. Compared to discrimination of gratings with the same orientation, reaction times to orthogonally oriented gratings, however, increase suggesting additional processing. Using event-related fMRI we examined the pattern of neural activation evoked when subjects discriminated the spatial frequency of Gabors presented with the same or orthogonal orientation. Blood-oxygen level dependent BOLD fMRI revealed significantly elevated bilateral activity in visual areas (V1, V2) when the gratings to be compared had an orthogonal orientation, compared to when they had the same orientation. These findings suggest that a change in an irrelevant stimulus dimension requires additional processing in primary and secondary visual areas. The finding that the task-irrelevant stimulus property (orientation) had no significant effect on the prefrontal and intraparietal cortex supports a model of working memory in which discrimination and retention of basic stimulus dimensions is based on low-level perceptual memory stores that are located at an early stage in the visual process. Our findings suggest that accessing different stores requires time and has higher metabolic costs. Supported by: BMBF Project “Visuospatial Cognition” and Norwegian Research Council.     

Neuroimaging of direction-selective mechanisms for second-order motion

Psychophysical findings have revealed a functional segregation of processing for 1st-order motion (movement of luminance modulation) and 2nd-order motion (e.g., movement of contrast modulation). However neural correlates of this psychophysical distinction remain controversial. To test for a corresponding anatomical segregation, we conducted a new functional magnetic resonance imaging (fMRI) study to localize direction-selective cortical mechanisms for 1st- and 2nd-order motion stimuli, by measuring direction-contingent response changes induced by motion adaptation, with deliberate control of attention. The 2nd-order motion stimulus generated direction-selective adaptation in a wide range of visual cortical areas, including areas V1, V2, V3, VP, V3A, V4v, and MT+. Moreover, the pattern of activity was similar to that obtained with 1st-order motion stimuli. Contrary to expectations from psychophysics, these results suggest that in the human visual cortex, the direction of 2nd-order motion is represented as early as V1. In addition, we found no obvious anatomical segregation in the neural substrates for 1st- and 2nd-order motion processing that can be resolved using standard fMRI.     

Cross-orientation suppression in human visual cortex

Cross-orientation suppression was measured in human primary visual cortex (V1) to test the normalization model. Subjects viewed vertical target gratings (of varying contrasts) with or without a superimposed horizontal mask grating (fixed contrast). We used functional magnetic resonance imaging (fMRI) to measure the activity in each of several hypothetical channels (corresponding to subpopulations of neurons) with different orientation tunings and fit these orientation-selective responses with the normalization model. For the V1 channel maximally tuned to the target orientation, responses increased with target contrast but were suppressed when the horizontal mask was added, evident as a shift in the contrast gain of this channel's responses. For the channel maximally tuned to the mask orientation, a constant baseline response was evoked for all target contrasts when the mask was absent; responses decreased with increasing target contrast when the mask was present. The normalization model provided a good fit to the contrast-response functions with and without the mask. In a control experiment, the target and mask presentations were temporally interleaved, and we found no shift in contrast gain, i.e., no evidence for suppression. We conclude that the normalization model can explain cross-orientation suppression in human visual cortex. The approach adopted here can be applied broadly to infer, simultaneously, the responses of several subpopulations of neurons in the human brain that span particular stimulus or feature spaces, and characterize their interactions. In addition, it allows us to investigate how stimuli are represented by the inferred activity of entire neural populations.     

Spatially specific FMRI repetition effects in human visual cortex

The functional MRI (fMRI) response to a pair of identical, successively presented stimuli can result in a smaller signal than the presentation of two nonidentical stimuli. This "repetition effect" has become a frequently used tool to make inferences about neural selectivity in specific cortical areas. However, little is known about the mechanism(s) underlying the effect. In particular, despite many successful applications of the technique in higher visual areas, repetition effects in lower visual areas [e.g., primary visual cortex (V1)] have been more difficult to characterize. One property that is well understood in early visual areas is the mapping of visual field locations to specific areas of the cortex (i.e., retinotopy). We used the retinotopic organization of V1 to activate progressively different populations of neurons in a rapid fMRI experimental design. We observed a repetition effect (reduced signal) when localized stimulus elements were repeated in identical locations. We show that this effect is spatially tuned and largely independent of both interstimulus interval (100-800 ms) and the focus of attention. Using the same timing parameters for which we observed a large effect of spatial position, we also examined the response to orientation changes and observed no effect of an orientation change on the response to repeated stimuli in V1 but significant effects in other retinotopic areas. Given these results, we discuss the possible causes of these repetition effects as well as the implications for interpreting other experiments that use this potentially powerful imaging technique.     

Spatial frequency-specific contrast adaptation originates in the primary visual cortex

Adaptation to a high-contrast grating stimulus causes reduced sensitivity to subsequent presentation of a visual stimulus with similar spatial characteristics. This behavioral finding has been attributed by neurophysiological studies to processes within the visual cortex. However, some evidence indicates that contrast adaptation phenomena are also found in early visual pathways. Adaptation effects have been reported in retina and lateral geniculation nucleus (LGN). It is possible that these early pathways could be the physiological origin of the cortical adaptation effect. To study this, we recorded from single neurons in the cat's LGN. We find that contrast adaptation in the LGN, unlike that in the visual cortex, is not spatial frequency specific, i.e., adaptation effects apply to a broad range of spatial frequencies. In addition, aside from the amplitude attenuation, the shape of spatial frequency tuning curves of LGN cells is not affected by contrast adaptation. Again, these findings are unlike those found for cells in the visual cortex. Together, these results demonstrate that pattern specific contrast adaptation is a cortical process.     

Orientation-selective adaptation to first- and second-order patterns in human visual cortex

Second-order textures-patterns that cannot be detected by mechanisms sensitive only to luminance changes-are ubiquitous in visual scenes, but the neuronal mechanisms mediating perception of such stimuli are not well understood. We used an adaptation protocol to measure neural activity in the human brain selective for the orientation of second-order textures. Functional MRI (fMRI) responses were measured in three subjects to presentations of first- and second-order probe gratings after adapting to a high-contrast first- or second-order grating that was either parallel or orthogonal to the probe gratings. First-order (LM) stimuli were generated by modulating the stimulus luminance. Second-order stimuli were generated by modulating the contrast (CM) or orientation (OM) of a first-order carrier. We used four combinations of adapter and probe stimuli: LM:LM, CM:CM, OM:OM, and LM:OM. The fourth condition tested for cross-modal adaptation with first-order adapter and second-order probe stimuli. Attention was diverted from the stimulus by a demanding task at fixation. Both first- and second-order stimuli elicited orientation-selective adaptation in multiple cortical visual areas, including V1, V2, V3, V3A/B, a newly identified visual area anterior to dorsal V3 that we have termed LO1, hV4, and VO1. For first-order stimuli (condition LM:LM), the adaptation was no larger in extrastriate areas than in V1, implying that the orientation-selective first-order (luminance) adaptation originated in V1. For second-order stimuli (conditions CM:CM and OM:OM), the magnitude of adaptation, relative to the absolute response magnitude, was significantly larger in VO1 (and for condition CM:CM, also in V3A/B and LO1) than in V1, suggesting that second-order stimulus orientation was extracted by additional processing after V1. There was little difference in the amplitude of adaptation between the second-order conditions. No consistent effect of adaptation was found in the cross-modal condition LM:OM, in agreement with psychophysical evidence for weak interactions between first- and second-order stimuli and computational models of separate mechanisms for first- and second-order visual processing.     

Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex

The tilt aftereffect (TAE) is a visual illusion in which prolonged adaptation to an oriented stimulus causes shifts in subsequent perceived orientations. Historically, neural models of the TAE have explained it as the outcome of response suppression of neurons tuned to the adapting orientation. Recent physiological studies of neurons in primary visual cortex (V1) have confirmed that such response suppression exists. However, it was also found that the preferred orientations of neurons shift away from the adapting orientation. Here we show that adding this second factor to a population coding model of V1 improves the correspondence between neurophysiological data and TAE measurements. According to our model, the shifts in preferred orientation have the opposite effect as response suppression, reducing the magnitude of the TAE.     

The effect of orientation adaptation on responses of lateral geniculate nucleus neurons with high orientation bias in cats

Adaptation to stimulus orientation is assumed to have a cortical basis, but few studies have addressed whether it affects the activity of subcortical neurons. Using single-unit recording, we studied the effects of orientation adaptation on the responses of lateral geniculate nucleus (LGN) neurons with high orientation bias (OB) in anesthetized and paralyzed cats. Following adaptation to one stimulus orientation, the response at the adapting orientation was decreased, and the preferred orientation was shifted away from the adapting orientation. This phenomenon was similar to the effects observed for orientation adaptation in the primary visual cortex (V1), and was obvious when the adapting orientation was at an appropriate location relative to the original preferred orientation. Moreover, when the V1 was inactivated, the response at the adapting orientation was also decreased but the preferred orientation did not show a systematic shift after orientation adaptation in LGN. This result indicates that cortical feedback contributes to the effect of orientation adaptation on LGN neurons, which have a high OB. These data provide an example of how the corticothalamic loop modulates the processing of visual information, and suggest that the LGN is not only a simply passive relay but also a modulator of visual information.     

Relationship between contrast adaptation and orientation tuning in V1 and V2 of cat visual cortex

Previous studies investigating the response properties of neurons in the primary visual cortex of cats and primates have shown that prolonged exposure to optimally oriented, high-contrast gratings leads to a reduction in responsiveness to subsequently presented test stimuli. We recorded from 119 neurons in cat V1 and V2 and found that in a high proportion of cells contrast adaptation also occurs for gratings oriented orthogonal to a neuron's preferred orientation, even though this stimulus did not elicit significant increases in spiking activity. Approximately 20% of neurons adapted equally to all orientations tested and a further 46% showed at least some adaptation to orthogonally oriented gratings, whereas 20% of neurons did not adapt to orthogonal gratings. The magnitude of contrast adaptation was positively correlated with adapting contrast, but was not related to the spiking activity of the cells. Highly direction selective neurons produced stronger adaptation to orthogonally oriented gratings than other neurons. Orientation-related adaptation was correlated with the rate of change of orientation tuning in consecutive cells along electrode penetrations that traveled parallel to the cortical layers. Nonoriented adaptation was most common in areas where orientation preference changed rapidly, whereas orientation-selective adaptation was most common in areas where orientation preference changed slowly. A minority of neurons did not show contrast adaptation (14%). No major differences were found between units in different cortical layers, V1 and V2, or between complex and simple cells. The relevance of these findings to the current understanding of adaptation within the context of orientation column architecture is discussed.     

Functional imaging of the human lateral geniculate nucleus and pulvinar

In the human brain, little is known about the functional anatomy and response properties of subcortical nuclei containing visual maps such as the lateral geniculate nucleus (LGN) and the pulvinar. Using functional magnetic resonance imaging (fMRI) at 3 tesla (T), collective responses of neural populations in the LGN were measured as a function of stimulus contrast and flicker reversal rate and compared with those obtained in visual cortex. Flickering checkerboard stimuli presented in alternation to the right and left hemifields reliably activated the LGN. The peak of the LGN activation was found to be on average within +/-2 mm of the anatomical location of the LGN, as identified on high-resolution structural images. In all visual areas except the middle temporal (MT), fMRI responses increased monotonically with stimulus contrast. In the LGN, the dynamic response range of the contrast function was larger and contrast gain was lower than in the cortex. Contrast sensitivity was lowest in the LGN and V1 and increased gradually in extrastriate cortex. In area MT, responses were saturated at 4% contrast. Response modulation by changes in flicker rate was similar in the LGN and V1 and occurred mainly in the frequency range between 0.5 and 7.5 Hz; in contrast, in extrastriate areas V4, V3A, and MT, responses were modulated mainly in the frequency range between 7.5 and 20 Hz. In the human pulvinar, no activations were obtained with the experimental designs used to probe response properties of the LGN. However, regions in the mediodorsal right and left pulvinar were found to be consistently activated by bilaterally presented flickering checkerboard stimuli, when subjects attended to the stimuli. Taken together, our results demonstrate that fMRI at 3 T can be used effectively to study thalamocortical circuits in the human brain.     

Decoding the visual and subjective contents of the human brain

The potential for human neuroimaging to read out the detailed contents of a person's mental state has yet to be fully explored. We investigated whether the perception of edge orientation, a fundamental visual feature, can be decoded from human brain activity measured with functional magnetic resonance imaging (fMRI). Using statistical algorithms to classify brain states, we found that ensemble fMRI signals in early visual areas could reliably predict on individual trials which of eight stimulus orientations the subject was seeing. Moreover, when subjects had to attend to one of two overlapping orthogonal gratings, feature-based attention strongly biased ensemble activity toward the attended orientation. These results demonstrate that fMRI activity patterns in early visual areas, including primary visual cortex (V1), contain detailed orientation information that can reliably predict subjective perception. Our approach provides a framework for the readout of fine-tuned representations in the human brain and their subjective contents.     

Neural correlates of stimulus spatial frequency-dependent contrast detection

Psychophysical studies on human and non-human vertebrate species have shown that visual contrast sensitivity function (CSF) peaks at a certain stimulus spatial frequency and declines in both lower and higher spatial frequencies. The underlying neural substrate and mechanisms remain in debate. Here, we investigated the role of primary visual cortex (V1: area 17) in spatial frequency-dependent contrast detection in cats. Perceptual CSFs of three cats were measured using a two-alternative forced-choice task. The responses of V1 neurons to their optimal visual stimuli in a range of luminance contrast levels (from 0 to 1.0) were recorded subsequently using in vivo extracellular single-unit recording techniques. The contrast sensitivity of each neuron was determined. The neuronal CSF for each cat was constructed from the mean contrast sensitivity of neurons with different preferred stimulus spatial frequencies. Result: (1) The perceptual and neuronal CSFs of each of the three cats exhibited a similar shape with peak amplitude near 0.4 cpd. (2) The neuronal CSF of each cat was highly correlated with its perceptual CSF. (3) V1 neurons with different preferred stimulus spatial frequencies had different contrast gains. Conclusion: (1) Contrast detection of visual stimuli with different spatial frequencies may likely involve population coding of V1 neurons with different preferred stimulus spatial frequencies. (2) Difference in contrast gain may underlie the observed contrast sensitivity variation of V1 neurons with different preferred stimulus spatial frequencies, possibly from either evolution or postnatal visual experiences.     

Modulation of V1 activity by shape: image-statistics or shape-based perception?

It is current dogma that neurons in primary visual cortex extract local edges from the scene from which later visual areas reconstruct more meaningful shapes. Recent neuroimaging studies, however, have shown V1 modulations by the degree of structure in the image (shape). These V1 modulations due to the level of shape coherence have been explained in one of two possible ways: due to changes in image statistics or shape-based perceptual influences from higher visual areas. Here we compare both hypotheses using stimuli composed of Gabor arrays constructed to form circular shapes that can be successively degraded by manipulating the orientations of individual Gabors while maintaining local and global statistics. In a first experiment, we confirm that V1 responses are inversely correlated with the degree of structure in the image. In a second experiment, stimulus predictions are compared based on the degree of circular shape or change in the image statistic varied (orientation variance) in the image. We find that these V1 modulations to shape change are correlated with low-level changes in orientation contrast rather than shape perception per se.     

Neural correlates of sustained spatial attention in human early visual cortex

Attention is thought to enhance perceptual performance at attended locations through top-down attention signals that modulate activity in visual cortex. Here, we show that activity in early visual cortex is sustained during maintenance of attention in the absence of visual stimulation. We used functional magnetic resonance imaging (fMRI) to measure activity in visual cortex while human subjects performed a visual detection task in which a variable-duration delay period preceded target presentation. Portions of cortical areas V1, V2, and V3 representing the attended part of the visual field exhibited sustained increases in activity throughout the delay period. Portions of these cortical areas representing peripheral, unattended parts of the visual field displayed sustained decreases in activity. The data were well fit by a model that assumed the sustained neural activity was constant in amplitude over a time period equal to that of the actual delay period for each trial. These results demonstrate that sustained attention responses are present in early visual cortex (including primary visual cortex), in the absence of a visual stimulus, and that these responses correlate with the allocation of visuospatial attention in both the spatial and temporal domains.     

Effects of Stimulus Size and Contrast on the Initial Primary Visual Cortical Response in Humans

Decades of intracranial electrophysiological investigation into the primary visual cortex (V1) have produced many fundamental insights into the computations carried out in low-level visual circuits of the brain. Some of the most important work has been simply concerned with the precise measurement of neural response variations as a function of elementary stimulus attributes such as contrast and size. Surprisingly, such simple but fundamental characterization of V1 responses has not been carried out in human electrophysiology. Here we report such a detailed characterization for the initial “C1” component of the scalp-recorded visual evoked potential (VEP). The C1 is known to be dominantly generated by initial afferent activation in V1, but is difficult to record reliably due to interindividual anatomical variability. We used pattern-pulse multifocal VEP mapping to identify a stimulus position that activates the left lower calcarine bank in each individual, and afterwards measured robust negative C1s over posterior midline scalp to gratings presented sequentially at that location. We found clear and systematic increases in C1 peak amplitude and decreases in peak latency with increasing size as well as with increasing contrast. With a sample of 15 subjects and ~180 trials per condition, reliable C1 amplitudes of ?0.46 µV were evoked at as low a contrast as 3.13% and as large as ?4.82 µV at 100% contrast, using stimuli of 3.33° diameter. A practical implication is that by placing sufficiently-sized stimuli to target favorable calcarine cortical loci, robust V1 responses can be measured at contrasts close to perceptual thresholds, which could greatly facilitate principled studies of early visual perception and attention.     

Dynamic properties of recurrent inhibition in primary visual cortex: contrast and orientation dependence of contextual effects

A fundamental feature of neural circuitry in the primary visual cortex (V1) is the existence of recurrent excitatory connections between spiny neurons, recurrent inhibitory connections between smooth neurons, and local connections between excitatory and inhibitory neurons. We modeled the dynamic behavior of intermixed excitatory and inhibitory populations of cells in V1 that receive input from the classical receptive field (the receptive field center) through feedforward thalamocortical afferents, as well as input from outside the classical receptive field (the receptive field surround) via long-range intracortical connections. A counterintuitive result is that the response of oriented cells can be facilitated beyond optimal levels when the surround stimulus is cross-oriented with respect to the center and suppressed when the surround stimulus is iso-oriented. This effect is primarily due to changes in recurrent inhibition within a local circuit. Cross-oriented surround stimulation leads to a reduction of presynaptic inhibition and a supraoptimal response, whereas iso-oriented surround stimulation has the opposite effect. This mechanism is used to explain the orientation and contrast dependence of contextual interactions in primary visual cortex: responses to a center stimulus can be both strongly suppressed and supraoptimally facilitated as a function of surround orientation, and these effects diminish as stimulus contrast decreases.     

Predictive coding for motion stimuli in human early visual cortex

The current study investigates if early visual cortical areas, V1, V2 and V3, use predictive coding to process motion information. Previous studies have reported biased visual motion responses at locations where novel visual information was presented (i.e., the motion trailing edge), which is plausibly linked to the predictability of visual input. Using high-field functional magnetic resonance imaging (fMRI), we measured brain activation during predictable versus unpreceded motion-induced contrast changes during several motion stimuli. We found that unpreceded moving dots appearing at the trailing edge gave rise to enhanced BOLD responses, whereas predictable moving dots at the leading edge resulted in suppressed BOLD responses. Furthermore, we excluded biases in directional sensitivity, shifts in cortical stimulus representation, visuo-spatial attention and classical receptive field effects as viable alternative explanations. The results clearly indicate the presence of predictive coding mechanisms in early visual cortex for visual motion processing, underlying the construction of stable percepts out of highly dynamic visual input.     

Neuronal correlates of perception in early visual cortex

We used functional magnetic resonance imaging (fMRI) to measure activity in human early visual cortex (areas V1, V2 and V3) during a challenging contrast-detection task. Subjects attempted to detect the presence of slight contrast increments added to two kinds of background patterns. Behavioral responses were recorded so that the corresponding cortical activity could be grouped into the usual signal detection categories: hits, false alarms, misses and correct rejects. For both kinds of background patterns, the measured cortical activity was retinotopically specific. Hits and false alarms were associated with significantly more cortical activity than were correct rejects and misses. That false alarms evoked more activity than misses indicates that activity in early visual cortex corresponded to the subjects' percepts, rather than to the physically presented stimulus.