Visual scene processing in familiar and unfamiliar environments

Humans and animals use information obtained from the local visual scene to orient themselves in the wider world. Although neural systems involved in scene perception have been identified, the extent to which processing in these systems is affected by previous experience is unclear. We addressed this issue by scanning subjects with functional magnetic resonance imaging (fMRI) while they viewed photographs of familiar and unfamiliar locations. Scene-selective regions in parahippocampal cortex (the parahippocampal place area, or PPA), retrosplenial cortex (RSC), and the transverse occipital sulcus (TOS) responded more strongly to images of familiar locations than to images of unfamiliar locations with the strongest effects (>50% increase) in RSC. Examination of fMRI repetition suppression (RS) effects indicated that images of familiar and unfamiliar locations were processed with the same degree of viewpoint specificity; however, increased viewpoint invariance was observed as individual scenes became more familiar over the course of a scan session. Surprisingly, these within-scan-session viewpoint-invariant RS effects were only observed when scenes were repeated across different trials but not when scenes were repeated within a trial, suggesting that within- and between-trial RS effects may index different aspects of visual scene processing. The sensitivity to environmental familiarity observed in the PPA, RSC, and TOS supports earlier claims that these regions mediate the extraction of navigationally relevant spatial information from visual scenes. As locations become familiar, the neural representations of these locations become enriched, but the viewpoint invariance of these representations does not change.     

Illusory conjunctions in visual short‐term memory: Individual differences in corpus callosum connectivity and splitting attention between the two hemifields

Overloading the capacity of visual attention can result in mistakenly combining the various features of an object, that is, illusory conjunctions. We hypothesize that if the two hemispheres separately process visual information by splitting attention, connectivity of corpus callosum—a brain structure integrating the two hemispheres—would predict the degree of illusory conjunctions. In the current study, we assessed two types of illusory conjunctions using a memory‐scanning paradigm; the features were either presented across the two opposite hemifields or within the same hemifield. Four objects, each with two visual features, were briefly presented together followed by a probe‐recognition and a confidence rating for the recognition accuracy. MRI scans were also obtained. Results indicated that successful recollection during probe recognition was better for across hemifields conjunctions compared to within hemifield conjunctions, lending support to the bilateral advantage of the two hemispheres in visual short‐term memory. Age‐related differences regarding the underlying mechanisms of the bilateral advantage indicated greater reliance on recollection‐based processing in young and on familiarity‐based processing in old. Moreover, the integrity of the posterior corpus callosum was more predictive of opposite hemifield illusory conjunctions compared to within hemifield illusory conjunctions, even after controlling for age. That is, individuals with lesser posterior corpus callosum connectivity had better recognition for objects when their features were recombined from the opposite hemifields than from the same hemifield. This study is the first to investigate the role of the corpus callosum in splitting attention between versus within hemifields.     

On-line attentional selection from competing stimuli in opposite visual fields: effects on human visual cortex and control processes

We used fMRI to investigate competition and on-line attentional selection between targets and distractors in opposite visual hemifields. Displays comprised a high-contrast square-wave grating, defined as target by its orientation, presented alone (unilateral) or with a similar distractor of orthogonal orientation in the opposite hemifield (bilateral displays). The target appeared unpredictably on the left or right, precluding anticipatory attention to one side. We found greater activation in target-contralateral superior occipital gyrus for unilateral than for bilateral displays, indicating suppression of the target's visual representation by distractor presence despite the competing distractor projecting to a different occipital hemisphere. Several frontal and parietal regions showed greater activation for bilateral than unilateral trials, suggesting involvement in on-line attentional selection. This was particularly pronounced for regions in bilateral intraparietal sulcus (IPS), which also showed greater functional coupling with occipital cortex specifically on bilateral trials that required selection plus some repetition-suppression effects when target side was repeated, but again only on bilateral trials requiring selection. Our results indicate that competition between visual stimuli in opposite hemifields can influence occipital cortex, and implicate IPS in resolution of this competition by selection.     

Sustained division of spatial attention to multiple locations within one hemifield

Attending to a location in space significantly improves stimulus perception at that location. Everyday experience requires the deployment of attention to multiple objects at different locations. Recent empirical evidence suggests that the "beam" of attention can be divided between non-contiguous areas of the visual field. Whether this is only possible when stimuli are presented in different hemifields and harder, if not impossible, when stimuli are in the same hemifield is an ongoing debate. Here we use an electrophysiological measure of sustained attentional resource allocation (the steady-state visual evoked potential, SSVEP) to address this question. In combination with behavioural data we demonstrate that splitting the attentional "beam" is in principle possible within one hemifield. However, results showed that task performance was in general lower for same-hemifield presentation as opposed to our previous study with different-hemifield presentation [M.M. Müller, P. Malinowski, T. Gruber, S.A. Hillyard, Sustained division of the attentional spotlight, Nature 424 (2003) 309-312]. SSVEP amplitude showed a mixed pattern of results for stimuli presented in the upper versus lower quadrant of the left visual hemifield under conditions of attending to two separated locations. Results are discussed in the light of the bilateral distribution advantage hypothesis and differences in stimulus salience between the upper and lower visual field.     

Constructing scenes from objects in human occipitotemporal cortex

We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.     

Interhemispheric visual integration in three cases of familial callosal agenesis

Three cases of callosal agenesis (a 39-year-old woman and her 11- and 12-year-old daughters) were tested on their ability to integrate visual information between the visual hemifields. They were all able to name colors and digits in either hemifield with high accuracy and were able to decide whether letters or digits in opposite hemifields were the same or different. They had greater difficulty deciding whether colors in opposite hemifields were the same or different. When shown 6-letter words made up of pairs of 3-letter words that straddled the midline (e.g., MANAGE, ROTATE), they responded to them as whole words and never as 3-letter words, suggesting perceptual continuity across the midline, at least for verbal material. The most likely interpretation is that the integration of form, but not color, is achieved through the intact anterior commissure in these participants.     

一种视觉假体的视觉显著图提取新方法

目的 一幅图像或场景的显著性区域代表它们的主要内容（显著目标）.由于视觉假体可植入电极的数量有限,只有低分辨率的图像对其才有用,因此提取显著性区域有助于视觉假体捕捉到场景中的显著目标.方法 Itti模型是一个显著性检测模型,它检测到的显著性区域与人的视觉感知有差异,显著目标的边界不明确.笔者去除了Itti模型提取的方向和色彩特征,将红（R）、绿（G）、蓝（B）三基色（RGB）图像转换到对应于HSI颜色空间上的色调（H）、纯度（S）、亮度（I）3个新特征分量,对hti模型进行优化改进.在显著图中,将落在显著目标内的显著点面积与总显著点面积的比值定义为显著图精确度;以显著图精确度为提取显著图方法的测度,对改进前后2种方法进行比较.结果 利用改进方法提取的显著图比Itti模型显著图精确度提高了约20％;在检测显著性区域时所用时间减少近50％.结论 提出了一种用于人工视觉的获取显著目标的方法,本算法可以得到更加精确的显著性结果,且可缩短运行时间.     

Both striate cortex and superior colliculus contribute to visual properties of neurons in superior temporal polysensory area of macaque monkey

Although the tectofugal system projects to the primate cerebral cortex by way of the pulvinar, previous studies have failed to find any physiological evidence that the superior colliculus influences visual activity in the cortex. We studied the relative contributions of the tectofugal and geniculostriate systems to the visual properties of neurons in the superior temporal polysensory area (STP) by comparing the effects of unilateral removal of striate cortex, the superior colliculus, or of both structures. In the intact monkey, STP neurons have large, bilateral receptive fields. Complete unilateral removal of striate cortex did not eliminate visual responses of STP neurons in the contralateral visual hemifield; rather, nearly half the cells still responded to visual stimuli in the hemifield contralateral to the lesion. Thus the visual properties of STP neurons are not completely dependent on the geniculostriate system. Unilateral striate lesions did affect the response properties of STP neurons in three ways. Whereas most STP neurons in the intact monkey respond similarly to stimuli in the two visual hemifields, responses to stimuli in the hemifield contralateral to the striate lesion were usually weaker than responses in the ipsilateral hemifield. Whereas the responses of many STP neurons in the intact monkey were selective for the direction of stimulus motion or for stimulus form, responses in the hemifield contralateral to the striate lesion were not selective for either motion or form. Whereas the median receptive field in the intact monkey extended 80 degrees into the contralateral visual field, the receptive fields of cells with responses in the contralateral field that survived the striate lesions had a median border that extended only 50 degrees into the contralateral visual field. Removal of both striate cortex and the superior colliculus in the same hemisphere abolished the responses of STP neurons to visual stimuli in the hemifield contralateral to the combined lesion. Nearly 80% of the cells still responded to visual stimuli in the hemifield ipsilateral to the lesion. Unilateral removal of the superior colliculus alone had only small effects on visual responses in STP. Receptive-field size and visual response strength were slightly reduced in the hemifield contralateral to the collicular lesion. As in the intact monkey, selectivity for stimulus motion or form were similar in the two visual hemifields. We conclude that both striate cortex and the superior colliculus contribute to the visual responses of STP neurons. Striate cortex is crucial for the movement and stimulus specificity of neurons in STP.(ABSTRACT TRUNCATED AT 400 WORDS)     

Interhemispheric comparisons in a man with complete forebrain commissurotomy

J. Sergent (1991) claimed that split-brained people are highly accurate in judging which is the larger of 2 circles in opposite visual hemifields but are relatively poor at judging whether circles in the 2 hemifields are of the same size. The discrepancy could be due, at least in part, to an artifact. A split-brained man, L.B., was markedly worse than normals at judging which was the larger of 2 circles or the longer of 2 horizontal lines in opposite hemifields, and his performance could be largely accounted for without assuming any interhemispheric transfer. L.B. also judged whether a single flashed line extended further into the left or right hemifield and, as in a previous study (M. C. Corballis, 1995), was strongly biased to respond "right longer." This bias was not observed in the judgments about the circles or the separated lines, suggesting that it is not due to a compression of perceived space in the left hemifield.     

Visual,haptic and crossmodal recognition of scenes

Real-world scene perception can often involve more than one sensory modality. Here we investigated the visual, haptic and crossmodal recognition of scenes of familiar objects. In three experiments participants first learned a scene of objects arranged in random positions on a platform. After learning, the experimenter swapped the position of two objects in the scene and the task for the participant was to identify the two swapped objects. In experiment 1, we found a cost in scene recognition performance when there was a change in sensory modality and scene orientation between learning and test. The cost in crossmodal performance was not due to the participants verbally encoding the objects (experiment 2) or by differences between serial and parallel encoding of the objects during haptic and visual learning, respectively (experiment 3). Instead, our findings suggest that differences between visual and haptic representations of space may affect the recognition of scenes of objects across these modalities.     

Temporal limitations in object processing across the human ventral visual pathway

Behavioral studies have shown that object recognition becomes severely impaired at fast presentation rates, indicating a limitation in temporal processing capacity. Here, we studied whether this behavioral limit in object recognition reflects limitations in the temporal processing capacity of early visual areas tuned to basic features or high-level areas tuned to complex objects. We used functional MRI (fMRI) to measure the temporal processing capacity of multiple areas along the ventral visual pathway progressing from the primary visual cortex (V1) to high-level object-selective regions, specifically the fusiform face area (FFA) and parahippocampal place area (PPA). Subjects viewed successive images of faces or houses at presentation rates varying from 2.3 to 37.5 items/s while performing an object discrimination task. Measures of the temporal frequency response profile of each visual area revealed a systematic decline in peak tuning across the visual hierarchy. Areas V1-V3 showed peak activity at rapid presentation rates of 18-25 items/s, area V4v peaked at intermediate rates (9 items/s), and the FFA and PPA peaked at the slowest temporal rates (4-5 items/s). Our results reveal a progressive loss in the temporal processing capacity of the human visual system as information is transferred from early visual areas to higher areas. These data suggest that temporal limitations in object recognition likely result from the limited processing capacity of high-level object-selective areas rather than that of earlier stages of visual processing.     

The role of temporo-parietal junction (TPJ) in global Gestalt perception

Grouping processes enable the coherent perception of our environment. A number of brain areas has been suggested to be involved in the integration of elements into objects including early and higher visual areas along the ventral visual pathway as well as motion-processing areas of the dorsal visual pathway. However, integration not only is required for the cortical representation of individual objects, but is also essential for the perception of more complex visual scenes consisting of several different objects and/or shapes. The present fMRI experiments aimed to address such integration processes. We investigated the neural correlates underlying the global Gestalt perception of hierarchically organized stimuli that allowed parametrical degrading of the object at the global level. The comparison of intact versus disturbed perception of the global Gestalt revealed a network of cortical areas including the temporo-parietal junction (TPJ), anterior cingulate cortex and the precuneus. The TPJ location corresponds well with the areas known to be typically lesioned in stroke patients with simultanagnosia following bilateral brain damage. These patients typically show a deficit in identifying the global Gestalt of a visual scene. Further, we found the closest relation between behavioral performance and fMRI activation for the TPJ. Our data thus argue for a significant role of the TPJ in human global Gestalt perception.     

Object-selective responses in the human motion area MT/MST.

The perception of moving objects and our successful interaction with them entail that the visual system integrates shape and motion information about objects. However, neuroimaging studies have implicated different human brain regions in the analysis of visual motion (medial temporal cortex; MT/MST) and shape (lateral occipital complex; LOC), consistent with traditional approaches in visual processing that attribute shape and motion processing to anatomically and functionally separable neural mechanisms. Here we demonstrate object-selective fMRI responses (higher responses for intact than for scrambled images of objects) in MT/MST, and especially in a ventral subregion of MT/MST, suggesting that human brain regions involved mainly in the processing of visual motion are also engaged in the analysis of object shape.     

Study of target and non-target interplay in spatial attention task

Selective visual attention is the ability to selectively pay attention to the targets while inhibiting the distractors. This paper aims to study the targets and non-targets interplay in spatial attention task while subject attends to the target object present in one visual hemifield and ignores the distractor present in another visual hemifield. This paper performs the averaged evoked response potential (ERP) analysis and time-frequency analysis. ERP analysis agrees to the left hemisphere superiority over late potentials for the targets present in right visual hemifield. Time-frequency analysis performed suggests two parameters i.e. event-related spectral perturbation (ERSP) and inter-trial coherence (ITC). These parameters show the same properties for the target present in either of the visual hemifields but show the difference while comparing the activity corresponding to the targets and non-targets. In this way, this study helps to visualise the difference between targets present in the left and right visual hemifields and, also the targets and non-targets present in the left and right visual hemifields. These results could be utilised to monitor subjects’ performance in brain–computer interface (BCI) and neurorehabilitation.     

Hemodynamic and electrocortical reactivity to specific scene contents in emotional perception

Emotional scene perception is characterized by enhanced neural activity across broad regions of visual cortex, the frontoparietal network, and anterior corticolimbic structures. In human fMRI and electrocortical experiments, activation enhancement is strongly related to self‐reported emotional arousal evoked by scene stimuli. However, an additional bias in reaction to pleasant scenes has been reported in a subset of emotion‐enhanced brain regions. Human fMRI and primate electrophysiological studies show biased frontoparietal network activity in response to rewarding cues. In addition, activation in lateral occipital regions may show a bias in pleasant scene perception, as shown in fMRI and in the early posterior negativity (EPN) ERP component. To define this potential pleasure bias, we presented a balanced set of naturalistic scenes to participants during separate fMRI and ERP recording sessions. Consistent with past work, the amplitude of the slow‐wave late positive potential (LPP), as well as hemodynamic activity in fusiform gyrus and amygdala, showed equivalent enhancement across highly arousing pleasant and unpleasant, relative to neutral scenes. In addition to this emotional enhancement, the EPN component, as well as hemodynamic activity in lateral occipital cortex and frontoparietal network, showed greater reactivity during highly arousing pleasant relative to unpleasant scenes, consistent with a pleasure bias. The interpretation of this pattern of reactivity is discussed with respect to selective and evolved attention mechanisms.     

Looming sounds enhance orientation sensitivity for visual stimuli on the same side as such sounds

Several recent multisensory studies show that sounds can influence visual processing. Some visual judgments can be enhanced for visual stimuli near a sound occurring around the same time. A recent TMS study (Romei et al. 2009) indicates looming sounds might influence visual cortex particularly strongly. But unlike most previous behavioral studies of possible audio-visual exogenous effects, TMS phosphene thresholds rather than judgments of external visual stimuli were measured. Moreover, the visual hemifield assessed relative to the hemifield of the sound was not varied. Here, we compared the impact of looming sounds to receding or "static" sounds, using auditory stimuli adapted from Romei et al. (2009), but now assessing any influence on visual orientation discrimination for Gabor patches (well-known to involve early visual cortex) when appearing in the same hemifield as the sound or on the opposite side. The looming sounds that were effective in Romei et al. (2009) enhanced visual orientation sensitivity (d') here on the side of the sound, but not for the opposite hemifield. This crossmodal, spatially specific effect was stronger for looming than receding or static sounds. Similarly to Romei et al. (2009), the differential effect for looming sounds was eliminated when using white noise rather than structured sounds. Our new results show that looming structured sounds can specifically benefit visual orientation sensitivity in the hemifield of the sound, even when the sound provides no information about visual orientation itself.     

Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming

We conducted two event-related functional magnetic resonance imaging (fMRI) experiments to investigate the neural substrates of visual object recognition in humans. We used a repetition-priming method with visual stimuli recurring at unpredictable intervals, either with the same appearance or with changes in size, viewpoint or exemplar. Lateral occipital and posterior inferior temporal cortex showed lower activity for repetitions of both real and non-sense objects; fusiform and left inferior frontal regions showed decreases for repetitions of only real objects. Repetition of different exemplars with the same name affected only the left inferior frontal cortex. Crucially, priming-induced decreases in activity of the right fusiform cortex depended on whether the three-dimensional objects were repeated with the same viewpoint, regardless of whether retinal image size changed; left fusiform decreases were independent of both viewpoint and size. These data show that dissociable subsystems in ventral visual cortex maintain distinct view-dependent and view-invariant object representations.     

Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey

1. We examined the sensory properties of cells in the anterior bank of the caudal part of the superior temporal sulcus (caudal STS) in anesthetized, paralyzed monkeys to visual, auditory, and somesthetic stimuli. 2. In the anterior bank of the caudal STS, there were three regions distinguishable from each other and also from the middle temporal area (MT) in the floor of the STS and area Tpt in the superior temporal gyrus. The three regions were located approximately in the respective inner, middle, and outer thirds of the anterior bank of the caudal STS. These three regions are referred to, from the inner to the outer, as the medial superior temporal region (MST), the mostly unresponsive region, and the caudal STS polysensory region (cSTP), respectively. 3. The extent of MST and its response properties agreed with previous studies. Cells in MST responded exclusively to visual stimuli, had large visual receptive fields (RFs), and nearly all (91%) showed directional selectivity. 4. In the mostly unresponsive region, three quarters of cells were unresponsive to any stimulus used in this study. A quarter of the cells responded to only visual stimuli and most did not show directional selectivity for moving stimuli. Several directionally selective cells responded to movements of three-dimensional objects, but not of projected stimuli. 5. The response properties of cells in the superficial cortex of the caudal superior temporal gyrus, a part of area Tpt, external to cSTP were different from those of cells in the three regions in the anterior bank of the STS. Cells in Tpt were exclusively auditory, and had much larger auditory RFs (mean = 271 degrees) than those of acoustically-driven cSTP cells (mean = 138 degrees). 6. The cSTP contained unimodal visual, auditory, and somesthetic cells as well as multimodal cells of two or all three modalities. The sensory properties of cSTP cells were as follows. 1) Out of 200 cells recorded, 102 (51%) cells were unimodal (59 visual, 33 auditory, and 10 somesthetic), 36 (18%) cells were bimodal (21 visual+auditory, 7 visual+somesthetic, and 8 auditory+somesthetic), and four (2%) cells were trimodal. Visual and auditory responses were more frequent than somesthetic responses: the ratio of the population of cells driven by visual: auditory: somesthetic stimuli was 3:2:1. 2) Visual RFs were large (mean diameter, 59 degrees), but two-thirds were limited to the contralateral visual hemifield. About half the cells showed directional selectivity for moving visual stimuli. None showed selectivity for particular visual shapes.(ABSTRACT TRUNCATED AT 400 WORDS)     

Relating retinotopic and object-selective responses in human lateral occipital cortex

What is the relationship between retinotopy and object selectivity in human lateral occipital (LO) cortex? We used functional magnetic resonance imaging (fMRI) to examine sensitivity to retinal position and category in LO, an object-selective region positioned posterior to MT along the lateral cortical surface. Six subjects participated in phase-encoded retinotopic mapping experiments as well as block-design experiments in which objects from six different categories were presented at six distinct positions in the visual field. We found substantial position modulation in LO using standard nonobject retinotopic mapping stimuli; this modulation extended beyond the boundaries of visual field maps LO-1 and LO-2. Further, LO showed a pronounced lower visual field bias: more LO voxels represented the lower contralateral visual field, and the mean LO response was higher to objects presented below fixation than above fixation. However, eccentricity effects produced by retinotopic mapping stimuli and objects differed. Whereas LO voxels preferred a range of eccentricities lying mostly outside the fovea in the retinotopic mapping experiment, LO responses were strongest to foveally presented objects. Finally, we found a stronger effect of position than category on both the mean LO response, as well as the distributed response across voxels. Overall these results demonstrate that retinal position exhibits strong effects on neural response in LO and indicates that these position effects may be explained by retinotopic organization.     

The attentional blink within and across the hemispheres: Evidence from a patient with a complete section of the corpus callosum

The attentional blink (AB) refers to an impairment in the report of a second target (T2) if it closely follows the presentation of a first target (T1) in a rapid serial visual presentation (RSVP), when both targets must be reported. In the present study, a modified AB paradigm was used in which targets could appear in any of four simultaneous RSVP streams, one in each quadrant of the visual field. In half of the trials, T1 and T2 were displayed in the same visual hemifield (either left or right) and, in the other half, T1 and T2 were displayed in different visual hemifields. Using this paradigm with both neurologically intact individuals and a split-brain patient, we sought to investigate (1) possible hemispheric asymmetries in attentional processes, and (2) whether the AB would be reduced when targets are displayed in different visual hemifields. A comparable AB was found for both neurologically intact individuals and the split-brain patient, with no significant variations due to whether targets were displayed in the same or in different hemifields. A left hemisphere advantage in the processing of same and different hemifield targets was observed only in the split-brain patient.