Position selectivity in scene- and object-responsive occipitotemporal regions

Complex visual scenes preferentially activate several areas of the human brain, including the parahippocampal place area (PPA), the retrosplenial complex (RSC), and the transverse occipital sulcus (TOS). The sensitivity of neurons in these regions to the retinal position of stimuli is unknown, but could provide insight into their roles in scene perception and navigation. To address this issue, we used functional magnetic resonance imaging (fMRI) to measure neural responses evoked by sequences of scenes and objects confined to either the left or right visual hemifields. We also measured the level of adaptation produced when stimuli were either presented first in one hemifield and then repeated in the opposite hemifield or repeated in the same hemifield. Although overall responses in the PPA, RSC, and TOS tended to be higher for contralateral stimuli than for ipsilateral stimuli, all three regions exhibited position-invariant adaptation, insofar as the magnitude of adaptation did not depend on whether stimuli were repeated in the same or opposite hemifields. In contrast, object-selective regions showed significantly greater adaptation when objects were repeated in the same hemifield. These results suggest that neuronal receptive fields (RFs) in scene-selective regions span the vertical meridian, whereas RFs in object-selective regions do not. The PPA, RSC, and TOS may support scene perception and navigation by maintaining stable representations of large-scale features of the visual environment that are insensitive to the shifts in retinal stimulation that occur frequently during natural vision.     

Two kinds of FMRI repetition suppression? Evidence for dissociable neural mechanisms

Repetition suppression (RS) is a reduction of neural response that is often observed when stimuli are presented more than once. Many functional magnetic resonance imaging (fMRI) studies have exploited RS to probe the sensitivity of cortical regions to variations in different stimulus dimensions; however, the neural mechanisms underlying fMRI-RS are not fully understood. Here we test the hypothesis that long-interval (between-trial) and short-interval (within-trial) RS effects are caused by distinct and independent neural mechanisms. Subjects were scanned while viewing visual scenes that were repeated over both long and short intervals. Within the parahippocampal place area (PPA) and other brain regions, suppression effects relating to both long- and short-interval repetition were observed. Critically, two sources of evidence indicated that these effects were engendered by different underlying mechanisms. First, long- and short-interval RS effects were entirely noninteractive even although they were measured within the same set of trials during which subjects performed a constant behavioral task, thus fulfilling the formal requirements for a process dissociation. Second, long- and short-interval RS were differentially sensitive to viewpoint: short-interval RS was only significant when scenes were repeated from the same viewpoint while long-interval RS less viewpoint-dependent. Taken together, these results indicate that long- and short-interval fMRI-RS are mediated by different neural mechanisms that independently modulate the overall fMRI signal. These findings have important implications for understanding the results of studies that use fMRI-RS to explore representational spaces.     

Constructing scenes from objects in human occipitotemporal cortex

We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.     

Cortical activation to indoor versus outdoor scenes: an fMRI study

Prior studies identify two cortical areas, posterior parahippocampal cortex and retrosplenial cortex, that preferentially activate to images of real-world scenes compared to images of other meaningful visual stimuli such as objects and faces. Behavioral and computational studies suggest that sub-categories of real-world scenes differ in their visual and semantic properties. It is presently unknown whether the cortical areas that have been implicated in scene analysis similarly activate differentially to behaviorally relevant scene sub-categories. To examine this issue, we directly compared cortical activation to indoor and outdoor scenes in an fMRI study with a large number of non-repeated images in each condition. Activation in posterior parahippocampal cortex, including parahippocampal place area, was significantly greater for indoor than outdoor scenes. In contrast, no such difference was observed in retrosplenial cortex, though this region preferentially activated to scenes over faces. These findings suggest differences in function in these two areas. The results are consistent with the view that posterior parahippocampal cortex is functional in processing local space.     

Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming

We conducted two event-related functional magnetic resonance imaging (fMRI) experiments to investigate the neural substrates of visual object recognition in humans. We used a repetition-priming method with visual stimuli recurring at unpredictable intervals, either with the same appearance or with changes in size, viewpoint or exemplar. Lateral occipital and posterior inferior temporal cortex showed lower activity for repetitions of both real and non-sense objects; fusiform and left inferior frontal regions showed decreases for repetitions of only real objects. Repetition of different exemplars with the same name affected only the left inferior frontal cortex. Crucially, priming-induced decreases in activity of the right fusiform cortex depended on whether the three-dimensional objects were repeated with the same viewpoint, regardless of whether retinal image size changed; left fusiform decreases were independent of both viewpoint and size. These data show that dissociable subsystems in ventral visual cortex maintain distinct view-dependent and view-invariant object representations.     

Visual,haptic and crossmodal recognition of scenes

Real-world scene perception can often involve more than one sensory modality. Here we investigated the visual, haptic and crossmodal recognition of scenes of familiar objects. In three experiments participants first learned a scene of objects arranged in random positions on a platform. After learning, the experimenter swapped the position of two objects in the scene and the task for the participant was to identify the two swapped objects. In experiment 1, we found a cost in scene recognition performance when there was a change in sensory modality and scene orientation between learning and test. The cost in crossmodal performance was not due to the participants verbally encoding the objects (experiment 2) or by differences between serial and parallel encoding of the objects during haptic and visual learning, respectively (experiment 3). Instead, our findings suggest that differences between visual and haptic representations of space may affect the recognition of scenes of objects across these modalities.     

Spatially specific FMRI repetition effects in human visual cortex

The functional MRI (fMRI) response to a pair of identical, successively presented stimuli can result in a smaller signal than the presentation of two nonidentical stimuli. This "repetition effect" has become a frequently used tool to make inferences about neural selectivity in specific cortical areas. However, little is known about the mechanism(s) underlying the effect. In particular, despite many successful applications of the technique in higher visual areas, repetition effects in lower visual areas [e.g., primary visual cortex (V1)] have been more difficult to characterize. One property that is well understood in early visual areas is the mapping of visual field locations to specific areas of the cortex (i.e., retinotopy). We used the retinotopic organization of V1 to activate progressively different populations of neurons in a rapid fMRI experimental design. We observed a repetition effect (reduced signal) when localized stimulus elements were repeated in identical locations. We show that this effect is spatially tuned and largely independent of both interstimulus interval (100-800 ms) and the focus of attention. Using the same timing parameters for which we observed a large effect of spatial position, we also examined the response to orientation changes and observed no effect of an orientation change on the response to repeated stimuli in V1 but significant effects in other retinotopic areas. Given these results, we discuss the possible causes of these repetition effects as well as the implications for interpreting other experiments that use this potentially powerful imaging technique.     

Hemodynamic and electrocortical reactivity to specific scene contents in emotional perception

Emotional scene perception is characterized by enhanced neural activity across broad regions of visual cortex, the frontoparietal network, and anterior corticolimbic structures. In human fMRI and electrocortical experiments, activation enhancement is strongly related to self‐reported emotional arousal evoked by scene stimuli. However, an additional bias in reaction to pleasant scenes has been reported in a subset of emotion‐enhanced brain regions. Human fMRI and primate electrophysiological studies show biased frontoparietal network activity in response to rewarding cues. In addition, activation in lateral occipital regions may show a bias in pleasant scene perception, as shown in fMRI and in the early posterior negativity (EPN) ERP component. To define this potential pleasure bias, we presented a balanced set of naturalistic scenes to participants during separate fMRI and ERP recording sessions. Consistent with past work, the amplitude of the slow‐wave late positive potential (LPP), as well as hemodynamic activity in fusiform gyrus and amygdala, showed equivalent enhancement across highly arousing pleasant and unpleasant, relative to neutral scenes. In addition to this emotional enhancement, the EPN component, as well as hemodynamic activity in lateral occipital cortex and frontoparietal network, showed greater reactivity during highly arousing pleasant relative to unpleasant scenes, consistent with a pleasure bias. The interpretation of this pattern of reactivity is discussed with respect to selective and evolved attention mechanisms.     

Perirhinal cortex ablation in rats selectively impairs object identification in a simultaneous visual comparison task

In Experiment 1, rats discriminated among computer-generated visual displays (scenes) comprising 3 different shapes (objects). One constant scene (unrewarded) appeared on every trial together with a trial-unique variable scene (rewarded). Four types of variable scene were intermingled: (a) unfamiliar objects in different positions from the constant; (b) unfamiliar objects in same positions as the constant; (c) same objects as the constant in different positions; (d) same objects and positions, recombined. Aspiration lesions of perirhinal cortex impaired performance with type (b) only. Experiment 2 tested spatial delayed nonmatching-to-sample. The perirhinal group were impaired nonsignificantly, and less than fornix-transected rats in an earlier study. Rats' perirhinal cortex, like monkeys', subserves object identification in the absence of memory requirement but does not contribute substantially to hippocampal system spatial memory function.     

Uncertainty and invariance in the human visual cortex

The way in which input noise perturbs the behavior of a system depends on the internal processing structure of the system. In visual psychophysics, there is a long tradition of using external noise methods (i.e., adding noise to visual stimuli) as tools for system identification. Here, we demonstrate that external noise affects processing of visual scenes at different cortical areas along the human ventral visual pathway, from retinotopic regions to higher occipitotemporal areas implicated in visual shape processing. We found that when the contrast of the stimulus was held constant, the further away from the retinal input a cortical area was the more its activity, as measured with functional magnetic resonance imaging (fMRI), depended on the signal-to-noise ratio (SNR) of the visual stimulus. A similar pattern of results was observed when trials with correct and incorrect responses were analyzed separately. We interpret these findings by extending signal detection theory to fMRI data analysis. This approach reveals the sequential ordering of decision stages in the cortex by exploiting the relation between fMRI response and stimulus SNR. In particular, our findings provide novel evidence that occipitotemporal areas in the ventral visual pathway form a cascade of decision stages with increasing degree of signal uncertainty and feature invariance.     

Human non-phase-locked gamma oscillations in experience-based perception of visual scenes

Perception results from ongoing interactions between stimulus-driven visual processes and cognitive context. These reciprocal relations are emphasized when a visual stimulus is degraded, forcing the perceiver to fill the missing information in, based on internal representations. The neural mechanisms by which internal representations facilitate visual perception are still unclear. Here we investigated the role of EEG oscillations in the gamma band, thought to reflect the elaboration of internal visual representations, in the experience-based perception of visual scenes. Twelve subjects were trained with degraded images of natural scenes. EEG was recorded while they performed a detection task on trained and untrained degraded stimuli. Non-phase-locked gamma band responses in a large frequency spectrum (55-85 Hz) were observed around 200 ms post-stimulus onset at posterior sites, and were larger when subjects reported an accurate perception based on previous experience. These results suggest that mid-latency gamma oscillations in the visual cortex underlie the experience-based perception of visual scenes.     

Neuronal correlates of repetition priming of frequently presented objects: insights from induced gamma band responses

The activation of cortical object representations requires the integration of dispersed cortical areas, signified by induced oscillatory bursts of activity > 20 Hz (induced Gamma Band Responses; iGBRs) at approximately 300 ms after stimulus onset. A well established marker of the functional dynamics within such cell assemblies is the suppression of iGBR amplitudes after the repetition of familiar stimuli. This effect is commonly interpreted as a signature of 'sharpening' processes within conceptual networks, which are behaviourally mirrored in repetition priming effects. Furthermore, it was demonstrated that the repetition of unfamiliar stimuli leads to iGBR increases indicating the 'formation' of a new cell assembly. A limitation of previous experiments was that only small numbers of repetitions were used. Thus, in the present EEG study we presented familiar and unfamiliar stimuli 10 times. We were able to replicate sharpening effects within conceptual networks representing familiar stimuli. Furthermore, we observed a gradual increase of iGBRs elicited by repeated unfamiliar stimuli. Interestingly, this formation effect did not turn into a sharpening effect after many repetitions (i.e. after an unfamiliar stimulus became familiar). Thus, we conclude that sharpening and formation effects rely on qualitatively different networks representing familiar and unfamiliar material.     

Neuropsychological evidence for a topographical learning mechanism in parahippocampal cortex

The Parahippocampal Place Area (PPA; Epstein & Kanwisher, 1998) is a region within posterior parahippocampal cortex that responds selectively to visual stimuli that convey information about the layout of local space. Here we describe two patients who suffered damage to the PPA after vascular incidents. Both subsequently exhibited memory problems for topographical materials and were unable to navigate unassisted in unfamiliar environments. Performance on a continuous n-back visual memory test was significantly lower for novel scene-like stimuli than for novel object-like stimuli. In contrast, performance was normal on a famous landmark recognition task and on two perceptual tasks that required on-line analysis of scene geometry. Both patients were able to produce accurate maps of premorbidly learned places but were unable to produce accurate maps of new places. These results converge with previous neuroimaging work to demonstrate that the PPA (1) is selectively involved in processing information about the geometry of surrounding space, and (2) may play a more critical role in the encoding of this information into memory than in the initial perceptual processing, recognition, or recall of this information.     

Neuropsychological evidence for a topographical learning mechanism in parahippocampal cortex

The Parahippocampal Place Area (PPA; Epstein & Kanwisher, 1998) is a region within posterior parahippocampal cortex that responds selectively to visual stimuli that convey information about the layout of local space. Here we describe two patients who suffered damage to the PPA after vascular incidents. Both subsequently exhibited memory problems for topographical materials and were unable to navigate unassisted in unfamiliar environments. Performance on a continuous n-back visual memory test was significantly lower for novel scene-like stimuli than for novel object-like stimuli. In contrast, performance was normal on a famous landmark recognition task and on two perceptual tasks that required on-line analysis of scene geometry. Both patients were able to produce accurate maps of premorbidly learned places but were unable to produce accurate maps of new places. These results converge with previous neuroimaging work to demonstrate that the PPA (1) is selectively involved in processing information about the geometry of surrounding space, and (2) may play a more critical role in the encoding of this information into memory than in the initial perceptual processing, recognition, or recall of this information.     

Neural activation and memory for natural scenes: Explicit and spontaneous retrieval

Stimulus repetition elicits either enhancement or suppression in neural activity, and a recent fMRI meta‐analysis of repetition effects for visual stimuli (Kim, 2017) reported cross‐stimulus repetition enhancement in medial and lateral parietal cortex, as well as regions of prefrontal, temporal, and posterior cingulate cortex. Repetition enhancement was assessed here for repeated and novel scenes presented in the context of either an explicit episodic recognition task or an implicit judgment task, in order to study the role of spontaneous retrieval of episodic memories. Regardless of whether episodic memory was explicitly probed or not, repetition enhancement was found in medial posterior parietal (precuneus/cuneus), lateral parietal cortex (angular gyrus), as well as in medial prefrontal cortex (frontopolar), which did not differ by task. Enhancement effects in the posterior cingulate cortex were significantly larger during explicit compared to implicit task, primarily due to a lack of functional activity for new scenes. Taken together, the data are consistent with an interpretation that medial and (ventral) lateral parietal cortex are associated with spontaneous episodic retrieval, whereas posterior cingulate cortical regions may reflect task or decision processes.     

Center-periphery organization of human object areas

The organizing principles that govern the layout of human object-related areas are largely unknown. Here we propose a new organizing principle in which object representations are arranged according to a central versus peripheral visual field bias. The proposal is based on the finding that building-related regions overlap periphery-biased visual field representations, whereas face-related regions are associated with center-biased representations. Furthermore, the eccentricity maps encompass essentially the entire extent of object-related occipito-temporal cortex, indicating that most object representations are organized with respect to retinal eccentricity. A control experiment ruled out the possibility that the results are due exclusively to unequal feature distribution in these images. We hypothesize that brain regions representing object categories that rely on detailed central scrutiny (such as faces) are more strongly associated with processing of central information, compared to representations of objects that may be recognized by more peripheral information (such as buildings or scenes).     

Neuronal activity in the primary visual cortex of the cat freely viewing natural images

Many studies have now demonstrated that neurons in the visual cortex of cats and monkeys change their activity when stimuli are presented beyond their classical receptive field, and that these responses are not readily apparent from their receptive field properties. However few studies have been conducted to investigate the discharge properties of neurons in the visual cortex of animals when they are allow to freely view natural images. We employ tetrodes, which enable simultaneous and separable recordings of small numbers of neighboring neurons, to record 102 single units from 59 sites from areas 17 and 18 of two alert cats. While the animals viewed either natural images or black screens, they made frequent saccadic eye movements and gaze fixations. Fixations onto an image's location increased neuronal firing peaking at 80-100 ms after the fixation onset, to then decrease steadily with time despite continuous fixation. Saccades trigger a fast decrease in firing rate for both images and darkness. When we examined the incidence of correlated firing, we observed significant synchrony during the initial phases of visual fixations when the animals viewed natural scenes. Such synchrony was absent during saccadic eye movements and during eye movements in darkness. Our data revealed that scanning of natural scenes is associated with a rapid succession of distinct fixation-related activation patterns that included transient rate changes and excess coincident firing. The transient nature of these synchronization phenomena suggests a fast acting mechanism, which is in good agreement with the evidence that basic operations of scene analysis must be accomplished within a few tens of milliseconds in primary visual cortex.     

Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry

During binocular rivalry, two incompatible monocular images compete for perceptual dominance, with one pattern temporarily suppressed from conscious awareness. We measured fMRI signals in early visual cortex while subjects viewed rival dichoptic images of two different contrasts; the contrast difference served as a 'tag' for the neuronal representations of the two monocular images. Activity in primary visual cortex (V1) increased when subjects perceived the higher contrast pattern and decreased when subjects perceived the lower contrast pattern. These fluctuations in V1 activity during rivalry were about 55% as large as those evoked by alternately presenting the two monocular images without rivalry. The rivalry-related fluctuations in V1 activity were roughly equal to those observed in other visual areas (V2, V3, V3a and V4v). These results challenge the view that the neuronal mechanisms responsible for binocular rivalry occur primarily in later visual areas.     

What's special about personally familiar faces? A multimodal approach

Dual-route models of face recognition suggest separate cognitive and affective routes. The predictions of these models were assessed in recognition tasks with unfamiliar, famous, and personally familiar faces. Whereas larger autonomic responses were only triggered for personally familiar faces, priming effects in reaction times to these faces, presumably reflecting cognitive recognition processes, were equal to those of famous faces. Activation of stored structural representations of familiar faces (face recognition units) was assessed by recording the N250r component in event-related brain potentials. Face recognition unit activation increased from unfamiliar over famous to personally familiar faces, suggesting that there are stronger representations for personally familiar than for famous faces. Because the topographies of the N250r for personally and famous faces were indistinguishable, a similar network of face recognition units can be assumed for both types of faces.     

Contrast sensitivity in human visual areas and its relationship to object recognition

An important characteristic of visual perception is the fact that object recognition is largely immune to changes in viewing conditions. This invariance is obtained within a sequence of ventral stream visual areas beginning in area V1 and ending in high order occipito-temporal object areas (the lateral occipital complex, LOC). Here we studied whether this transformation could be observed in the contrast response of these areas. Subjects were presented with line drawings of common objects and faces in five different contrast levels (0, 4, 6, 10, and 100%). Our results show that indeed there was a gradual trend of increasing contrast invariance moving from area V1, which manifested high sensitivity to contrast changes, to the LOC, which showed a significantly higher degree of invariance at suprathreshold contrasts (from 10 to 100%). The trend toward increased invariance could be observed for both face and object images; however, it was more complete for the face images, while object images still manifested substantial sensitivity to contrast changes. Control experiments ruled out the involvement of attention effects or hemodynamic "ceiling" in producing the contrast invariance. The transition from V1 to LOC was gradual with areas along the ventral stream becoming increasingly contrast-invariant. These results further stress the hierarchical and gradual nature of the transition from early retinotopic areas to high order ones, in the build-up of abstract object representations.