Spatially Specific Working Memory Activity in the Human Superior Colliculus

Theoretically, working memory (WM) representations are encoded by population activity of neurons with distributed tuning across the stored feature. Here, we leverage computational neuroimaging approaches to map the topographic organization of human superior colliculus (SC) and model how population activity in SC encodes WM representations. We first modeled receptive field properties of voxels in SC, deriving a detailed topographic organization resembling that of the primate SC. Neural activity within human (5 male and 1 female) SC persisted throughout a retention interval of several types of modified memory-guided saccade tasks. Assuming an underlying neural architecture of the SC based on its retinotopic organization, we used an encoding model to show that the pattern of activity in human SC represents locations stored in WM. Our tasks and models allowed us to dissociate the locations of visual targets and the motor metrics of memory-guided saccades from the spatial locations stored in WM, thus confirming that human SC represents true WM information. These data have several important implications. They add the SC to a growing number of cortical and subcortical brain areas that form distributed networks supporting WM functions. Moreover, they specify a clear neural mechanism by which topographically organized SC encodes WM representations.SIGNIFICANCE STATEMENT Using computational neuroimaging approaches, we mapped the topographic organization of human superior colliculus (SC) and modeled how population activity in SC encodes working memory (WM) representations, rather than simpler visual or motor properties that have been traditionally associated with the laminar maps in the primate SC. Together, these data both position the human SC into a distributed network of brain areas supporting WM and elucidate the neural mechanisms by which the SC supports WM.     

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Processing capabilities for many low-level visual features are experientially malleable, aiding sighted organisms in adapting to dynamic environments. Explicit instructions to attend a specific visual field location influence retinotopic visuocortical activity, amplifying responses to stimuli appearing at cued spatial positions. It remains undetermined both how such prioritization affects surrounding nonprioritized locations, and if a given retinotopic spatial position can attain enhanced cortical representation through experience rather than instruction. The current report examined visuocortical response changes as human observers (N = 51, 19 male) learned, through differential classical conditioning, to associate specific screen locations with aversive outcomes. Using dense-array EEG and pupillometry, we tested the preregistered hypotheses of either sharpening or generalization around an aversively associated location following a single conditioning session. Competing hypotheses tested whether mean response changes would take the form of a Gaussian (generalization) or difference-of-Gaussian (sharpening) distribution over spatial positions, peaking at the viewing location paired with a noxious noise. Occipital 15 Hz steady-state visual evoked potential responses were selectively heightened when viewing aversively paired locations and displayed a nonlinear, difference-of-Gaussian profile across neighboring locations, consistent with suppressive surround modulation of nonprioritized positions. Measures of alpha-band (8–12 Hz) activity were differentially altered in anterior versus posterior locations, while pupil diameter exhibited selectively heightened responses to noise-paired locations but did not evince differences across the nonpaired locations. These results indicate that visuocortical spatial representations are sharpened in response to location-specific aversive conditioning, while top-down influences indexed by alpha-power reduction exhibit posterior generalization and anterior sharpening.SIGNIFICANCE STATEMENT It is increasingly recognized that early visual cortex is not a static processor of physical features, but is instead constantly shaped by perceptual experience. It remains unclear, however, to what extent the cortical representation of many fundamental features, including visual field location, is malleable by experience. Using EEG and an aversive classical conditioning paradigm, we observed sharpening of visuocortical responses to stimuli appearing at aversively associated locations along with location-selective facilitation of response systems indexed by pupil diameter and EEG alpha power. These findings highlight the experience-dependent flexibility of retinotopic spatial representations in visual cortex, opening avenues toward novel treatment targets in disorders of attention and spatial cognition.     

Hemifield effects of spatial attention in early human visual cortex

Early visual areas (V1, V2, V3/VP, V4v) contain representations of the contralateral hemifield within each hemisphere. Little is known about the role of the visual hemifields along the visuo-spatial attention processing hierarchy. It is hypothesized that attentional information processing is more efficient across the hemifields (known as bilateral field advantage) and that the integration of information is greater within one hemifield as compared with across the hemifields. Using functional magnetic resonance imaging we examined the effect of distance and hemifield on parallel attentional processing in the early visual areas (V1-V4v) at individually mapped retinotopic locations aligned adjacently or separately within or across the hemifields. We found that the bilateral field advantage in parallel attentional processing over separated attended locations can be assigned, at least partly, to differences in distractor position integration in early visual areas. These results provide evidence for a greater integration of locations between two attended locations within one hemifield than across both hemifields. This nicely correlates with behavioral findings of a bilateral field advantage in parallel attentional processing (when distractors in between cannot be excluded) and a unilateral field advantage if attention has to be shifted across separated locations (when locations in between were integrated).     

fMRI reveals greater within- than between-hemifield integration in the human lateral occipital cortex

Early visual areas within each hemisphere (V1, V2, V3/VP, V4v) contain distinct representations of the upper and lower quadrants of the contralateral hemifield. As receptive field size increases, the retinotopy in higher-tier visual areas becomes progressively less distinct. Using functional magnetic resonance imaging (fMRI) to map the visual fields, we found that an intermediate level visual area, the lateral occipital region (LO), contains retinotopic maps with a contralateral bias, but with a combined representation of the upper and lower visual field. Moreover, we used the technique of fMRI adaptation to determine whether neurons in LO code for both the upper and lower contralateral quadrants. We found that even when visual stimulus locations are equivalent across comparisons, the LO was more sensitive to location changes that crossed hemifields than location changes within a hemifield. These results suggested that within high-tier visual areas the increasing integration of visual field information is a two-stage process. The upper and lower visual representations are combined first, in LO, then the left and right representations. Furthermore, these results provided evidence for a neural mechanism to explain behavioral findings of greater integration within than between hemifields.     

From knowing what to knowing where: modeling object-based attention with feedback disinhibition of activation

We propose a neural model of visual object-based attention in which the identity of an object is used to select its location in an array of objects. The model is based on neural activity observed in visual search tasks performed by monkeys. In the model, the identity of the object (target) is selected in the higher areas of the ventral stream by means of a cue. Feedback activation from these higher areas carries information about the identity of the target to the (lower) retinotopic areas of the ventral stream. In these areas, the feedback activation interacts with feedforward activation produced by the object array. The interaction occurs in local microcircuits, and results in a selective activation on locations in the retinotopic areas of the visual stream that correspond to the location of the target in the object array. The selective activation consists of a form of gain control, produced by disinhibition. Transmitted to the dorsal stream, this activation directs spatial attention to the location of the target. In this way, an action directed at the target can be generated.     

Self-correction mechanism for path integration in a modular navigation system on the basis of an egocentric spatial map.

Classical Computer Science approaches to navigation by autonomous robots continue to make good progress. However, we have only a limited understanding of how navigation is implemented in the neural networks of animals, which still perform very much better in navigational tasks than robots. In this paper we explore the implementation of neural network based navigation in a simple robot. We use a modular navigation system that contains separate representations of visual input and the path integration process. These representations are combined to influence the behavior of a robot. Both representations are encoded within recurrent neuronal networks. The outputs of the representations are vectors of polar values that encode the location of the nearest object, or of a specific place in the environment. The robot manoeuvres in relation to these attended locations, in the context of its egocentric spatial map. During ego-motion towards a goal, the network representation of the goal moves in a counter-movement due to applied motor feedback. The robot's position is continuously compared against its visual input, and mismatches between the visually perceived goal position and its spatial representation are corrected.     

Decoding the internal focus of attention

The significance of the recent introduction to cognitive neuroscience of multivariate pattern analysis (MVPA) is that, unlike univariate approaches which are limited to identifying magnitudes of activity in localized parts of the brain, it affords the detection and characterization of patterns of activity distributed within and across multiple brain regions. This technique supports stronger inferences because it captures neural representations that have markedly higher selectivity than do univariate activation peaks. Recently, we used MVPA to assess the neural consequences of dissociating the internal focus of attention from short-term memory (STM), finding that the information represented in delay-period activity corresponds only to the former (Lewis-Peacock, Drysdale, Oberauer, & Postle, in press). Here we report several additional analyses of these data in which we directly compared the results generated by MVPA vs. those generated by univariate analyses. The sensitivity of MVPA to subtle variations in patterns of distributed brain activity revealed a novel insight: although overall activity remains elevated in category-selective brain regions corresponding to unattended STM items, the multivariate patterns of activity within these regions reflect the representation of a different category, i.e., the one that is currently being attended to. In addition, MVPA was able to dissociate attended from unattended STM items in brain regions whose univariate activity did not appear to be sensitive to the task. These findings highlight the fallacy of the assumption of homogeneity of representation within putative category-selective regions. They affirm the view that neural representations in STM are highly distributed and overlapping, and they demonstrate the necessity of multivariate analysis for dissociating such representations.     

Brainstem control of orienting movements: intrinsic coordinate systems and underlying circuitry.

A fundamental issue in the understanding of how the nervous system processes information is the way in which sensory information is used to initiate and guide movements. Recent progress has been made by taking an information processing approach in which information--for example, the spatial location of an object towards which an animal will orient--is tracked through the nervous system from sensory to motor levels. In this approach, neurally encoded information is characterized in terms of its representation within a neural or intrinsic coordinate system or set of neural coding parameters. For example, the retina codes spatial location in terms of the location of activity on the retinal surface, whereas motoneurons code spatial location in terms of the pulling directions of the muscles they activate. In between these two peripheral stages, the information passes through intermediate coordinate systems. These intermediate coordinate systems can be characterized by recording or altering the activity of small groups of neurons while an animal is performing a well-defined sensorimotor task. Spatial location information is used to guide orienting movements, those movements made by the eyes, ears, head, or body which function to center an object of interest in the animal's visual field. The optic tectum and forebrain, their connections to the medial mesencephalic and rhombencephalic brainstem tegmental cell groups, and subsequent connections to brainstem motor nuclei and spinal cord are employed to control fundamental aspects of this behavior. Studies reviewed herein indicate that following the retinotopic coding of spatial location in the retina and tectum, spatial location information appears to enter a different coordinate system at tegmental levels in which spatial aspects of orienting movement are coded in terms of their discrete horizontal and vertical components. This Cartesian coordinate system is an example of an abstract neural coordinate system, in that it is a simple, low-dimensional representation of spatial location which differs greatly from both sensory and motor representations. Also, this Cartesian representation may be common to many orienting movements, yet it appears to differ from the coordinate systems controlling other movement types such as stabilization or phasic movements. This suggests an hypothesis in which coordinate systems, especially at intermediate levels of processing, may be organized according to behavioral task as opposed to being determined by the particular sensory or motor system involved in the behavior. Understanding the evolutionary heritage and computational function of abstract neural coordinate systems, and the relation between different coordinate systems and behavioral tasks may be useful in understanding general aspects of sensory information processing and motor control.     

The topography of visuospatial attention as revealed by a novel visual field mapping technique

Previously, we and others have shown that attention can enhance visual processing in a spatially specific manner that is retinotopically mapped in the occipital cortex. However, it is difficult to appreciate the functional significance of the spatial pattern of cortical activation just by examining the brain maps. In this study, we visualize the neural representation of the "spotlight" of attention using a back-projection of attention-related brain activation onto a diagram of the visual field. In the two main experiments, we examine the topography of attentional activation in the occipital and parietal cortices. In retinotopic areas, attentional enhancement is strongest at the locations of the attended target, but also spreads to nearby locations and even weakly to restricted locations in the opposite visual field. The dispersion of attentional effects around an attended site increases with the eccentricity of the target in a manner that roughly corresponds to a constant area of spread within the cortex. When averaged across multiple observers, these patterns appear consistent with a gradient model of spatial attention. However, individual observers exhibit complex variations that are unique but reproducible. Overall, these results suggest that the topography of visual attention for each individual is composed of a common theme plus a personal variation that may reflect their own unique "attentional style."     

Explicit processing of verbal and spatial features during letter-location binding modulates oscillatory activity of a fronto-parietal network

The present study investigated the binding of verbal and spatial features in immediate memory. In a recent study, we demonstrated incidental and asymmetrical letter-location binding effects when participants attended to letter features (but not when they attended to location features) that were associated with greater oscillatory activity over prefrontal and posterior regions during the retention period. We were interested to investigate whether the patterns of brain activity associated with the incidental binding of letters and locations observed when only the verbal feature is attended differ from those reflecting the binding resulting from the controlled/explicit processing of both verbal and spatial features. To achieve this, neural activity was recorded using magnetoencephalography (MEG) while participants performed two working memory tasks. Both tasks were identical in terms of their perceptual characteristics and only differed with respect to the task instructions. One of the tasks required participants to process both letters and locations. In the other, participants were instructed to memorize only the letters, regardless of their location. Time-frequency representation of MEG data based on the wavelet transform of the signals was calculated on a single trial basis during the maintenance period of both tasks. Critically, despite equivalent behavioural binding effects in both tasks, single and dual feature encoding relied on different neuroanatomical and neural oscillatory correlates. We propose that enhanced activation of an anterior-posterior dorsal network observed in the task requiring the processing of both features reflects the necessity for allocating greater resources to intentionally process verbal and spatial features in this task.     

Volitional covert orienting to a peripheral cue does not suppress cue-induced inhibition of return

Detection reaction time (RT) at an extrafoveal location can be increased by noninformative precues presented at that location or ipsilaterally to it. This cue-induced inhibition is called inhibition of return or ipsilateral inhibition. We measured detection RT to simple light targets at extrafoveal locations that could be designated for covert orienting by local or distant cues. We found that cue-induced inhibition cooccurred in an additive fashion with the direct effects of covert orienting, i.e., it detracted from facilitation at attended locations and increased the disadvantage for unattended locations. Thus, cue-induced inhibition cannot be suppressed by a volitional covert orienting to the cued location; the co-occurrence of different facilitatory and inhibitory effects confirms the simultaneous operation of multiple independent attentional mechanisms during covert orienting.     

Attention and sensory gain control: a peripheral visual process?

Attention-related sensory gain control in human extrastriate cortex is believed to improve the acuity of visual perception. Yet given wide variance in the spatial resolution of vision across the retina, it remains unclear whether sensory gain operates homogenously between foveal and nonfoveal retinotopic locations. To address this issue, we used event-related potentials (ERPs) in a variant of the canonical spatial attention task. Participants were cued to expect targets at either fixation (foveal targets) or at a location several degrees above fixation (parafoveal targets). At both target locations, manual reaction times were shorter for cued relative to uncued targets, indicating that attention was consistently oriented to the cued location. Nevertheless, attention-related increases in sensory-evoked cortical activity were only observed at the parafoveal target location, as measured by the amplitude of the lateral occipital P1 ERP component. A second experiment replicated this data pattern using targets with lower stimulus contrast, indicating that the absence of a P1 effect for foveal targets could not be attributed to a saturated P1 response under higher-contrast stimulus conditions. When considered in light of retinogeniculate projections to cortex showing systematic changes in their physiological organization beginning within a degree of visual angle of the fovea, our findings support the proposal that the strategic functions of visual attention may vary with the retinotopic location involved.     

Modulation of alpha and gamma oscillations related to retrospectively orienting attention within working memory

Selective attention mechanisms allow us to focus on information that is relevant to the current behavior and, equally important, ignore irrelevant information. An influential model proposes that oscillatory neural activity in the alpha band serves as an active functional inhibitory mechanism. Recent studies have shown that, in the same way that attention can be selectively oriented to bias sensory processing in favor of relevant stimuli in perceptual tasks, it is also possible to retrospectively orient attention to internal representations held in working memory. However, these studies have not explored the associated oscillatory phenomena. In the current study, we analysed the patterns of neural oscillatory activity recorded with magnetoencephalography while participants performed a change detection task, in which a spatial retro‐cue was presented during the maintenance period, indicating which item or items were relevant for subsequent retrieval. Participants benefited from retro‐cues in terms of accuracy and reaction time. Retro‐cues also modulated oscillatory activity in the alpha and gamma frequency bands. We observed greater alpha activity in a ventral visual region ipsilateral to the attended hemifield, thus supporting its suppressive role, i.e. a functional disengagement of task‐irrelevant regions. Accompanying this modulation, we found an increase in gamma activity contralateral to the attended hemifield, which could reflect attentional orienting and selective processing. These findings suggest that the oscillatory mechanisms underlying attentional orienting to representations held in working memory are similar to those engaged when attention is oriented in the perceptual space.     

Neural Encoding of Active Multi-Sensing Enhances Perceptual Decision-Making via a Synergistic Cross-Modal Interaction

Most perceptual decisions rely on the active acquisition of evidence from the environment involving stimulation from multiple senses. However, our understanding of the neural mechanisms underlying this process is limited. Crucially, it remains elusive how different sensory representations interact in the formation of perceptual decisions. To answer these questions, we used an active sensing paradigm coupled with neuroimaging, multivariate analysis, and computational modeling to probe how the human brain processes multisensory information to make perceptual judgments. Participants of both sexes actively sensed to discriminate two texture stimuli using visual (V) or haptic (H) information or the two sensory cues together (VH). Crucially, information acquisition was under the participants'' control, who could choose where to sample information from and for how long on each trial. To understand the neural underpinnings of this process, we first characterized where and when active sensory experience (movement patterns) is encoded in human brain activity (EEG) in the three sensory conditions. Then, to offer a neurocomputational account of active multisensory decision formation, we used these neural representations of active sensing to inform a drift diffusion model of decision-making behavior. This revealed a multisensory enhancement of the neural representation of active sensing, which led to faster and more accurate multisensory decisions. We then dissected the interactions between the V, H, and VH representations using a novel information-theoretic methodology. Ultimately, we identified a synergistic neural interaction between the two unisensory (V, H) representations over contralateral somatosensory and motor locations that predicted multisensory (VH) decision-making performance.SIGNIFICANCE STATEMENT In real-world settings, perceptual decisions are made during active behaviors, such as crossing the road on a rainy night, and include information from different senses (e.g., car lights, slippery ground). Critically, it remains largely unknown how sensory evidence is combined and translated into perceptual decisions in such active scenarios. Here we address this knowledge gap. First, we show that the simultaneous exploration of information across senses (multi-sensing) enhances the neural encoding of active sensing movements. Second, the neural representation of active sensing modulates the evidence available for decision; and importantly, multi-sensing yields faster evidence accumulation. Finally, we identify a cross-modal interaction in the human brain that correlates with multisensory performance, constituting a putative neural mechanism for forging active multisensory perception.     

The retinotopic organization of area 17 (striate cortex) in the cat.

The location and retinotopic organization of visual areas in the cat cortex were determined by systematically mapping visual cortex in over 100 cats. The positions of the receptive fields of single neurons or small clusters of neurons were related to the locations of the corresponding recording sites in the cortex to determine the representations of the visual field in these cortical areas. In this report, the first of a series, we describe the organization of area 17. A single representation of the cat's entire visual field corresponds closely to the cytoarchitectonically defined area 17. This area has the largest cortical surface area (380 mm2) and the highest cortical magnification factor (3.6 mm2/degree2 at area centralis) of all the cortical areas we have studied. There was perfect agreement between the borders of area 17 determined electrophysiologically and cytoarchitecturally. This area contains a first order transformation of the visual hemifield in which every adjacent point in the visual field is represented as an adjacent point in the cortex. Some variability exists among cats in the extent and retinotopic representation of the visual field in area 17.     

Attending to a location in three-dimensional space modulates early ERPs

It has been reported that attending to a particular location can modulate incoming sensory signals, as reflected by the stimulus-evoked P1 and N1 components of the visual event-related potential (ERPs) in a two-dimensional (2D) display [Attention, Space, and Action: Studies in Cognitive Neuroscience, Oxford University Press, New York, 1999, p. 31]. In contrast, in this study we examined the effect of attention in 3D space using a stereoscopic display. Stimuli were presented randomly, one at a time, in an orthogonal combination of two depths (near, far) and two 2D locations (left, right) relative to the fixation point. The task was to attend selectively to one of these four positions and to respond to a target stimulus defined by shape in the attended 3D location. The effect of 2D location selection on the P1 amplitude was greater for stimuli in the near than the far depth plane, and the amplitude of N1 increased in response to stimuli in the attended combination of 2D location and depth. These results suggest that the effect of early spatial selection on the visual ERP is not simply based on retinotopic organization of the visual field, but also on intermediate stages that construct a 3D spatial representation of the external world.     

Attentional modulation of repetition attenuation is anatomically dissociable for scenes and faces

Repeating a stimulus generally leads to a decreased response in neural activity compared to that for novel items. This neural attenuation provides a marker for stimulus-specific perceptual encoding and memory that can be detected using functional magnetic resonance imaging (fMRI). Although previously assumed to occur automatically whenever a stimulus is repeated, recent studies have begun to show that the repetition attenuation effect is task-specific and modulated by attention. Here, we demonstrate that attention is crucial for obtaining neural attenuation even after extensive stimulus repetitions. Furthermore, the effect of attention on attenuation is anatomically dissociable for stimuli that have relatively segregated neural representations in high-level perceptual cortex. To manipulate attention, we used overlapping scene and face images, and asked subjects to attend to either category. In a scene-sensitive cortical region known as the parahippocampal place area (PPA), significant attenuation in the fMRI BOLD signal was observed for the attended repeated scenes (relative to attended novel scenes), while no attenuation was observed for ignored repeated scenes or attended repeated faces against their respective novel image baselines. Conversely, in the face-sensitive region known as the fusiform face area (FFA), significant attenuation was observed for attended repeated faces, but not for ignored repeated faces or attended repeated scenes. An additional control experiment ruled out alternative explanations based on global signal level reductions due to inattention. Thus, attention actively governed when neuronal activity was attenuated to repeated perceptual input, and such attenuation was specific to the cortical regions that actively represent the attended category of stimuli.     

Saccades to a remembered location elicit spatially specific activation in human retinotopic visual cortex

The possible impact upon human visual cortex from saccades to remembered target locations was investigated using functional magnetic resonance imaging (fMRI). A specific location in the upper-right or upper-left visual quadrant served as the saccadic target. After a delay of 2,400 msec, an auditory signal indicated whether to execute a saccade to that location (go trial) or to cancel the saccade and remain centrally fixated (no-go). Group fMRI analysis revealed activation specific to the remembered target location for executed saccades, in the contralateral lingual gyrus. No-go trials produced similar, albeit significantly reduced, effects. Individual retinotopic mapping confirmed that on go trials, quadrant-specific activations arose in those parts of ventral V1, V2, and V3 that coded the target location for the saccade, whereas on no-go trials, only the corresponding parts of V2 and V3 were significantly activated. These results indicate that a spatial-motor saccadic task (i.e., making an eye movement to a remembered location) is sufficient to activate retinotopic visual cortex spatially corresponding to the target location, and that this activation is also present (though reduced) when no saccade is executed. We discuss the implications of finding that saccades to remembered locations can affect early visual cortex, not just those structures conventionally associated with eye movements, in relation to recent ideas about attention, spatial working memory, and the notion that recently activated representations can be "refreshed" when needed.     

A neural model for visual selection of grouped spatial arrays

Psychophysical and electrophysiological studies indicate that visual attention operates on early retinotopic maps and selects spatially grouped arrays of locations which correspond to objects or perceptual groups. A neural model is proposed which is able to select an array of locations labelled by the same activity level and suppress all other regions that are not in the focus of attention. The model uses self-recurrent dendritic inhibition to compute the maximum activity level in the input and suppress a feedforward flow of activity from input to selection layer at unattended locations. Computer simulations also illustrate the model's ability to detect abrupt onsets of new objects in the visual scene, perform visual search and track moving objects.