Predicting visual acuity from the structure of visual cortex

Three decades ago, Rockel et al. proposed that neuronal surface densities (number of neurons under a square millimeter of surface) of primary visual cortices (V1s) in primates is 2.5 times higher than the neuronal density of V1s in nonprimates or many other cortical regions in primates and nonprimates. This claim has remained controversial and much debated. We replicated the study of Rockel et al. with attention to modern stereological precepts and show that indeed primate V1 is 2.5 times denser (number of neurons per square millimeter) than many other cortical regions and nonprimate V1s; we also show that V2 is 1.7 times as dense. As primate V1s are denser, they have more neurons and thus more pinwheels than similar-sized nonprimate V1s, which explains why primates have better visual acuity.Rockel et al. (1), in an influential and controversial article entitled “The basic uniformity in structure of the neocortex,” reported that the number of neurons underneath a square millimeter of neocortical surface is constant for six cortical areas and five species with one exception: primate primary visual cortex (V1) has a surface density of about 250,000 neurons/mm², around two and a half times the usual density for other areas studied.The Rockel et al. paper has, for a third of a century, continued to generate controversy for two reasons. One reason stems from its implications for an equally energetic debate among neuroscientist “lumpers” and “splitters.” Cortical uniformity supports a theory of neocortical processing wherein different cortical areas are subserved by the same canonical circuit, a view favored by lumpers. Splitters, however, believe each cortical area to be different and doubt the paper’s claims (2). The second reason is that studies from various laboratories using different measurement methods over the last three decades have alternately agreed and disagreed with Rockel et al.’s results (2).Notably, however, Rockel et al’s studies have never been directly replicated. We set out to repeat the observations for the same areas and species Rockel et al. used. We used Rockel et al.’s counting techniques but with attention to the precepts of modern stereology (2). Our goal was to simply determine if Rockel et al.’s observations are repeatable rather than address the larger question of numerical uniformity of neocortex across species and areas. In an earlier publication (2), we confirmed Rockel et al.’s conclusions for nonvisual areas. Here we focus on the primary V1 for the same species used in the original report of Rockel et al.V1 is part of the visual circuit from the retina to the cortex, which is retinotopically organized, and the 2D image of the world that is mapped onto the retina is recreated in V1 (3). Cells within the retina capture visual information for each image location or pixel such as color and light intensity and convey it to structures in V1 (4, 5), which perform computations that contribute to visual abilities. An especially well-studied V1 structure is a pinwheel, which comprises orientation columns that extend vertically down from the cortical surface, containing cells with receptive fields responsive to lines or edges at a particular angle. Columns within a pinwheel are organized so that the angle of orientation increases/decreases smoothly as one radially traverses the cortical surface and stays constant as one moves along a spoke of the pinwheel (5, 6). We wondered whether evolution might have designed primate V1 to be denser to increase the number of cells and pinwheels (7) and thus the computational power of V1 and visual abilities (8).We estimated the neuronal surface density (the number of neurons under a square millimeter of neocortical surface) of V1 for mouse, rat, cat, and monkey (rhesus macaque) and confirmed Rockel et al.’s original report: the first three species have a surface density of about 10⁵ neurons/mm² and monkey V1 has about 2.5 × 10⁵ neurons/mm². We also found that monkey V2 has about 1.7 × 10⁵ neurons/mm².     

Modulation of neuronal responses during covert search for visual feature conjunctions

While searching for an object in a visual scene, an observer''s attentional focus and eye movements are often guided by information about object features and spatial locations. Both spatial and feature-specific attention are known to modulate neuronal responses in visual cortex, but little is known of the dynamics and interplay of these mechanisms as visual search progresses. To address this issue, we recorded from directionally selective cells in visual area MT of monkeys trained to covertly search for targets defined by a unique conjunction of color and motion features and to signal target detection with an eye movement to the putative target. Two patterns of response modulation were observed. One pattern consisted of enhanced responses to targets presented in the receptive field (RF). These modulations occurred at the end-stage of search and were more potent during correct target identification than during erroneous saccades to a distractor in RF, thus suggesting that this modulation is not a mere presaccadic enhancement. A second pattern of modulation was observed when RF stimuli were nontargets that shared a feature with the target. The latter effect was observed during early stages of search and is consistent with a global feature-specific mechanism. This effect often terminated before target identification, thus suggesting that it interacts with spatial attention. This modulation was exhibited not only for motion but also for color cue, although MT neurons are known to be insensitive to color. Such cue-invariant attentional effects may contribute to a feature binding mechanism acting across visual dimensions.     

Temporal dynamics of neuronal modulation during exogenous and endogenous shifts of visual attention in macaque area MT

Dynamically shifting attention between behaviorally relevant stimuli in the environment is a key condition for successful adaptive behavior. Here, we investigated how exogenous (reflexive) and endogenous (voluntary) shifts of visual spatial attention interact to modulate activity of single neurons in extrastriate area MT. We used a double-cueing paradigm, in which the first cue instructed two macaque monkeys to covertly attend to one of three moving random dot patterns until a second cue, whose unpredictable onset exogenously captured attention, either signaled to shift or maintain the current focus of attention. The neuronal activity revealed correlates of both exogenous and endogenous attention, which could be well distinguished by their characteristic temporal dynamics. The earliest effect was a transient interruption of the focus of endogenous attention by the onset of the second cue. The neuronal signature of this exogenous capture of attention was a short-latency decrease of responses to the stimulus attended so far. About 70 ms later, the influence of exogenous attention leveled off, which was reflected in two concurrent processes: responses to the newly cued stimulus continuously increased because of allocation of endogenous attention, while, surprisingly, there was also a gradual rebound of attentional enhancement of the previously relevant stimulus. Only after an additional 110 ms did endogenous disengagement of attention from this previously relevant stimulus become evident. These patterns of attentional modulation can be most parsimoniously explained by assuming two distinct attentional mechanisms drawing on the same capacity-limited system, with exogenous attention having a much faster time course than endogenous attention.     

Effects of cholinergic deafferentation of the rhinal cortex on visual recognition memory in monkeys

Excitotoxic lesion studies have confirmed that the rhinal cortex is essential for visual recognition ability in monkeys. To evaluate the mnemonic role of cholinergic inputs to this cortical region, we compared the visual recognition performance of monkeys given rhinal cortex infusions of a selective cholinergic immunotoxin, ME20.4-SAP, with the performance of monkeys given control infusions into this same tissue. The immunotoxin, which leads to selective cholinergic deafferentation of the infused cortex, yielded recognition deficits of the same magnitude as those produced by excitotoxic lesions of this region, providing the most direct demonstration to date that cholinergic activation of the rhinal cortex is essential for storing the representations of new visual stimuli and thereby enabling their later recognition.     

Novel stimuli evoke excess activity in the mouse primary visual cortex

To explore how neural circuits represent novel versus familiar inputs, we presented mice with repeated sets of images with novel images sparsely substituted. Using two-photon calcium imaging to record from layer 2/3 neurons in the mouse primary visual cortex, we found that novel images evoked excess activity in the majority of neurons. This novelty response rapidly emerged, arising with a time constant of 2.6 ± 0.9 s. When a new image set was repeatedly presented, a majority of neurons had similarly elevated activity for the first few presentations, which decayed to steady state with a time constant of 1.4 ± 0.4 s. When we increased the number of images in the set, the novelty response’s amplitude decreased, defining a capacity to store ∼15 familiar images under our conditions. These results could be explained quantitatively using an adaptive subunit model in which presynaptic neurons have individual tuning and gain control. This result shows that local neural circuits can create different representations for novel versus familiar inputs using generic, widely available mechanisms.

Because the behavioral consequences of a sensory stimulus can depend on whether that stimulus is novel or familiar, sensory systems can benefit from employing different representations of novel versus familiar stimuli. At the level of human psychophysics, stimulus novelty can enhance salience and capture attention (1–3), while familiarity can speed visual search (4). Novelty also affects aversive conditioning (5–7) and fear conditioning (8, 9). In human brain imaging, novel stimuli have been shown to generate the mismatch negativity (MMN) (10, 11) while repeated stimuli lead to repetition suppression (12). Explicit representation of novelty has been shown at higher stages of the sensory hierarchy, such as in the hippocampus (13) and inferotemporal cortex (14–16), and has been interpreted as a possible substrate of recognition memory (17). Lower in sensory hierarchies, the representation of novelty can be enhanced by stimulus-specific adaptation (SSA) (18–21) as well as by gain control (22, 23). Novelty signals are also prominently present in midbrain dopamine neurons (24).Explicit representation of stimulus novelty is also related to theories of predictive coding, in which neural circuits carry out computations that emphasize novel or surprising information. Theories of predictive coding have had a long history, starting with ideas about how the receptive field structure of retinal ganglion cells more efficiently encodes natural visual scenes by removing redundant data (25–28) and including the idea that active adaptation may aid in this process (18). Theories of predictive coding in the neocortex have typically focused on the idea that feedback from higher cortical areas encodes a prediction about lower-level sensory data (29) that is subtracted from the lower-level representation, so that the signals traveling up the cortical hierarchy represent surprise or novelty (30, 31). However, a recent study failed to find these signatures of predictive coding (32).Here, we investigate novelty processing in the mouse primary visual cortex. We repeatedly presented a set of images, each composed of a random superposition of Gabor functions, and then occasionally presented novel images drawn from the same ensemble. Using two-photon imaging of the Ca²⁺ sensor GCaMP6f to measure neural activity in layer 2/3 of awake, head-fixed mice (33), we found that the majority of neurons exhibited excess activity in response to a novel image. This distinction between novel versus familiar images was quickly reached, emerging with a time constant of 2.6 ± 0.9 s. Similarly, when we began presenting a new set of images, a majority of the neurons exhibited elevated firing that relaxed to a steady state with a time constant of 1.4 ± 0.4 s. When we presented novel images within larger image sets, the amplitude of novelty response decreased, defining a capacity of the system to encode ∼15 familiar images. All of these findings could be explained qualitatively using an adaptive subunit model in which neurons presynaptic to a recorded neuron have both individual tuning to visual stimuli and adaptive gain control.     

Experience-dependent functional plasticity and visual response selectivity of surviving subplate neurons in the mouse visual cortex

Subplate neurons are early-born cortical neurons that transiently form neural circuits during perinatal development and guide cortical maturation. Thereafter, most subplate neurons undergo cell death, while some survive and renew their target areas for synaptic connections. However, the functional properties of the surviving subplate neurons remain largely unknown. This study aimed to characterize the visual responses and experience-dependent functional plasticity of layer 6b (L6b) neurons, the remnants of subplate neurons, in the primary visual cortex (V1). Two-photon Ca²⁺ imaging was performed in V1 of awake juvenile mice. L6b neurons showed broader tunings for orientation, direction, and spatial frequency than did layer 2/3 (L2/3) and L6a neurons. In addition, L6b neurons showed lower matching of preferred orientation between the left and right eyes compared with other layers. Post hoc 3D immunohistochemistry confirmed that the majority of recorded L6b neurons expressed connective tissue growth factor (CTGF), a subplate neuron marker. Moreover, chronic two-photon imaging showed that L6b neurons exhibited ocular dominance (OD) plasticity by monocular deprivation during critical periods. The OD shift to the open eye depended on the response strength to the stimulation of the eye to be deprived before starting monocular deprivation. There were no significant differences in visual response selectivity prior to monocular deprivation between the OD changed and unchanged neuron groups, suggesting that OD plasticity can occur in L6b neurons showing any response features. In conclusion, our results provide strong evidence that surviving subplate neurons exhibit sensory responses and experience-dependent plasticity at a relatively late stage of cortical development.

The mammalian cerebral cortex consists of six layers, with distinct roles in information processing (1, 2). At the bottom of the neocortex, on the boundary between the gray matter and white matter, there is a thin sheet of neurons called layer 6b (L6b) (3). Layer 6b neurons are thought to be remnants of subplate neurons based on their location and cell-type marker expression (4). During prenatal and early postnatal periods, subplate neurons form transient neuronal circuits that play key roles in cortical maturation (5–7). In the embryonic cortex, subplate neurons form short-lived synapses with early immature neurons to regulate radial migration (8). During perinatal development, subplate neurons transiently receive inputs from ingrowing thalamic axons and innervate layer 4 (L4) to guide thalamic inputs to the eventual target, L4 (5, 6). Thus, the circuits formed by subplate neurons at the perinatal developmental stage are essential to establish basic neuronal circuits before starting experience-dependent refinements (5–7). Subsequently, subplate neurons largely disappear due to programmed cell death, but some survive and reside in L6b (5, 6). In the adult cortex, L6b neurons form neuronal circuits with local and long-distance neurons, which are different from those formed during early development (9–12). Therefore, surviving subplate neurons may acquire a role in information processing after remodeling of neuronal connections. A recent study using three-photon Ca²⁺ imaging demonstrated that L6b neurons show visual responses with broad orientation/direction tuning in the adult mouse primary visual cortex (V1) (13). However, comparable evidence for L6b response properties with other layer neurons in V1 is lacking (14–20). Moreover, L6b neurons have diverse morphology and molecular expression (21–24). Neurons born during subplate neurogenesis show the different expression patterns of subplate markers in postnatal L6b (4). However, the response properties in each subtype of L6b neurons remain unknown.The sensory responsiveness of cortical neurons is considerably refined by sensory experience relatively late in development, referred to as the critical period (25, 26). Previous studies have demonstrated that sensory activities before the onset of the critical period affect the arrangement of subplate neuron neurites in the barrel cortex and local subplate circuits in the auditory cortex (27, 28). However, there is no direct evidence that the sensory responses of surviving subplate neurons are modified by sensory experience during the critical period. If experience-dependent plasticity occurs in subplate neuron responses, they will contribute to the experience-dependent development of sensory functions and possibly to the functions in the mature cortex. Ocular dominance (OD) plasticity in V1 is a canonical model used to examine experience-dependent refinement of sensory responses (25, 26, 29, 30). If one eye is occluded for several days during the critical period, neurons in V1 lose their response to the deprived eye. OD plasticity is robustly preserved across species and cell types. Therefore, OD plasticity is suitable for evaluating experience-dependent plasticity in L6b neurons.This study aimed to characterize the visual responses and OD plasticity of L6b neurons in V1. Toward this goal, two-photon Ca²⁺ imaging was performed in awake juvenile mice, followed by 3D immunohistochemistry with a subplate neuronal marker, connective tissue growth factor (CTGF) (4, 31). L6b neurons showed broader tuning to visual stimuli and lower binocular matching of orientation preference than did layer 2/3 (L2/3) and L6a neurons. Chronic two-photon imaging revealed significant OD plasticity in individual L6b neurons during the critical period. Our results provide strong evidence that L6b neurons, presumed to be subplate neuron remnants, exhibit sensory responses and experience-dependent functional plasticity at a relatively late stage of cortical development.     

Large-scale dissociations between views of objects,scenes, and reachable-scale environments in visual cortex

Space-related processing recruits a network of brain regions separate from those recruited in object processing. This dissociation has largely been explored by contrasting views of navigable-scale spaces to views of close-up, isolated objects. However, in naturalistic visual experience, we encounter spaces intermediate to these extremes, like the tops of desks and kitchen counters, which are not navigable but typically contain multiple objects. How are such reachable-scale views represented in the brain? In three human functional neuroimaging experiments, we find evidence for a large-scale dissociation of reachable-scale views from both navigable scene views and close-up object views. Three brain regions were identified that showed a systematic response preference to reachable views, located in the posterior collateral sulcus, the inferior parietal sulcus, and superior parietal lobule. Subsequent analyses suggest that these three regions may be especially sensitive to the presence of multiple objects. Further, in all classic scene and object regions, reachable-scale views dissociated from both objects and scenes with an intermediate response magnitude. Taken together, these results establish that reachable-scale environments have a distinct representational signature from both scene and object views in visual cortex.     

Task context impacts visual object processing differentially across the cortex

Perception reflects an integration of “bottom-up” (sensory-driven) and “top-down” (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer’s goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.Perception reflects not only the external world but also our internal goals and biases. Even the simplest actions and decisions about visual objects require a complex integration between “top-down” (internally generated) and “bottom-up” (sensory-driven) signals (1). For example, the information used for object categorization depends on top-down signals arising from the spatial (2) or conceptual (3) context in which the object appears, the prior experience of the observers (4, 5), and the specific task (6, 7). Despite such strong behavioral evidence, the neural correlates of this integration remain unclear, both in terms of the cortical regions involved and the extent of the integration within those regions. Here, we investigate the impact of diverse behavioral goals on the neural architecture that supports object processing.Object recognition is known to depend on the ventral visual pathway, a set of interconnected cortical regions extending from early visual areas (e.g., V1/V2) into the anterior inferotemporal cortex (8). It has been argued that object processing along this pathway can largely be captured in feed-forward hierarchical frameworks without the need for top-down signals (9–12). For example, in the HMAX model (10), the integration of top-down signals is largely constrained to the extrinsic targets of the pathway (8), and in particular the lateral prefrontal cortex (LPFC) (13–17). However, there is strong evidence that top-down signals, such as attention and task, modulate the magnitude of response to simple visual stimuli (e.g., gratings) in early visual areas (18–23) and the response to objects in extrastriate regions (24–30).Although these prior studies provide evidence of an effect of top-down signals on object processing, they afford only limited insight because they tested only the modulation of overall activity and not the impact of top-down signals on fine-grained object information available in the response. The importance of this distinction between gross modulation versus fine-grained information is apparent in functional MRI (fMRI) investigations of working memory, where not all regions that evidence activity modulations contain information about the maintained objects (31, 32). Crucially, quantifying object information allows for a direct test of whether object representations are task-independent (equivalent information within and across tasks) or task-dependent (reduced information across compared with within tasks). Without this test, it remains unclear whether top-down signals, such as task, fundamentally alter the representations of objects or simply scale the response to them.To investigate the full range of task effects, we presented a broad set of objects in six separate tasks, half of which probed physical properties of the stimulus (e.g., color: red/blue) and half its conceptual properties (e.g., content: manmade/natural). This paradigm overcomes a limitation of previous studies, which often treated task and stimulus as simple dichotomous variables (26–30), making it difficult to generalize beyond the limited range of tasks and objects tested. Furthermore, previous studies often manipulated only whether an object was attended or not, and therefore could not establish how different types of information are extracted from the same attended stimuli. In contrast, by presenting an identical set of object images under multiple tasks, all requiring attention to the images, and extracting the response to each combination of task and object, we were able to directly test the effect of task on object responses.Our results revealed that task context has a pervasive effect on visual representations throughout the early visual cortex (EVC), the ventral visual pathway, and the LPFC. Task modulated both response magnitude and multivariate response patterns throughout these regions. Critically, responses in the ventral object-selective as well as the LPFC, were task-dependent, with reduced object information across tasks compared with within task. In contrast, object information in the EVC was task-independent, despite large task-related activity modulations. Together, these findings demonstrate that top-down signals directly contribute to and constrain visual object representations in the ventral object-selective and LPFC. Such effects strongly support a recurrent, highly interactive view of visual object processing within the ventral visual pathway that contrasts with many primarily bottom-up frameworks (9, 10).     

A theory of direction selectivity for macaque primary visual cortex

This paper offers a theory for the origin of direction selectivity (DS) in the macaque primary visual cortex, V1. DS is essential for the perception of motion and control of pursuit eye movements. In the macaque visual pathway, neurons with DS first appear in V1, in the Simple cell population of the Magnocellular input layer 4Cα. The lateral geniculate nucleus (LGN) cells that project to these cortical neurons, however, are not direction selective. We hypothesize that DS is initiated in feed-forward LGN input, in the summed responses of LGN cells afferent to a cortical cell, and it is achieved through the interplay of 1) different visual response dynamics of ON and OFF LGN cells and 2) the wiring of ON and OFF LGN neurons to cortex. We identify specific temporal differences in the ON/OFF pathways that, together with item 2, produce distinct response time courses in separated subregions; analysis and simulations confirm the efficacy of the mechanisms proposed. To constrain the theory, we present data on Simple cells in layer 4Cα in response to drifting gratings. About half of the cells were found to have high DS, and the DS was broadband in spatial and temporal frequency (SF and TF). The proposed theory includes a complete analysis of how stimulus features such as SF and TF interact with ON/OFF dynamics and LGN-to-cortex wiring to determine the preferred direction and magnitude of DS.

This paper proposes a solution to a longstanding question in visual neuroscience, namely, the origin of direction selectivity (DS) in the visual cortex of macaque monkeys. Motion perception is a vital visual capability well developed in primates. As perceiving motion requires perceiving the direction in which a target moves, DS, the ability of visual neurons to sense the direction of movement, is essential for motion perception (1) and for the control of pursuit eye movements (2). For these reasons, understanding DS is an important first step toward understanding how the cortex processes motion signals.DS in cortical neurons was first documented in the cat (3). Since then, it has been found in neurons all along the visual dorsal stream (an area associated with motion processing) in primates like macaque monkeys (4–7), whose vision is like that of humans. Neurons with DS are, in fact, present across species; they are widespread among visual mammals, an experimental fact that testifies to their biological significance.In the visual pathway of macaques, DS appears first in the primary visual cortex (V1), in the Simple cell population of the input layer 4Cα (8). These neurons provide feed-forward direction-selective signals to subsequent cortical layers and brain regions in the dorsal pathway. Thus, to discover the origin of DS, one is led to examining how neurons in layer 4Cα acquire their DS—and that is where it gets interesting: The neurons that provide visual signals to layer 4Cα, the Magnocellular cells in the lateral geniculate nucleus (LGN), are not direction selective (9–12). Yet many of the cells in the input layer of V1 to which they project are direction selective. A fundamental scientific question, therefore, is how 4Cα neurons acquire their DS. That is the question we would like to answer in this paper.Although many papers have been written on DS since its discovery over half a century ago, and there is continued interest in the subject (13–16), no satisfactory mechanistic explanation for the origin of DS in primate cortex has been proposed before now: Early conceptual models of how DS may arise, such as the Reichardt multiplier (17) or the motion energy model (18), were not concerned with biological mechanisms. Later work proposed neural mechanisms for the motion energy model (19), but they are not sufficient for explaining DS in primate cortex. See Discussion for comparisons of different model mechanisms.It is widely accepted that the DS computation requires spatiotemporal inseparability (STI); that is, different subregions of the receptive field have different time courses of response (18, 20, 21). What were lacking were biological mechanisms that could produce STI, and a clear understanding of how DS depends on the interaction between STI and the spatial and temporal character of the visual stimulus. These are the issues we address in this paper.We hypothesize that a plausible biological mechanism is the interplay between 1) the different dynamics of ON and OFF LGN cells and 2) the specific wiring that connects ON and OFF cells to V1. Item 2 refers here to the well-known fact that OFF and ON LGN cells are wired to segregated V1 receptive field subregions (3, 22, 23). Our main contribution is item 1: We identify, in Results, dynamic differences in the ON/OFF pathways that, together with item 2, produce distinct response time courses in separated receptive-field subregions. The mechanisms we propose are biologically grounded, and, as we show, they are sufficient for initiating DS in the feed-forward LGN input to cortical cells.To constrain our theory, we present experimental results on the responses of macaque 4Cα Simple cells to drifting gratings. Most Simple cells we recorded in 4Cα were unambiguously direction selective, preferring, consistently, the same direction over their entire visible ranges of spatial frequency (SF) and temporal frequency (TF); about half of the cells had high DS. Our data reveal also an important characteristic of neurons with DS, namely, the approximate invariance of DS with SF and TF. Explaining the broadband character of DS (in TF and SF) is a challenge for all previous theories. Our theory includes a complete analysis of how stimulus features like SF and TF interact with ON/OFF dynamics and LGN-to-cortex wiring to explain the broadband character of DS. The theoretical predictions are in good agreement with data.With regard to broader implications, although the theory as described in this paper is specifically about DS, an important message is that, when combining information from multiple channels, slight biases in their temporal filters can greatly enhance the capability of a system. Thus, it may be possible to exploit the temporal axis further in the processing of biological and nonbiological signals, especially in the neural processing of sensory inputs and, possibly, in computer vision.     

Shape-specific preparatory activity mediates attention to targets in human visual cortex

The mechanisms of attention prioritize sensory input for efficient perceptual processing. Influential theories suggest that attentional biases are mediated via preparatory activation of task-relevant perceptual representations in visual cortex, but the neural evidence for a preparatory coding model of attention remains incomplete. In this experiment, we tested core assumptions underlying a preparatory coding model for attentional bias. Exploiting multivoxel pattern analysis of functional neuroimaging data obtained during a non-spatial attention task, we examined the locus, time-course, and functional significance of shape-specific preparatory attention in the human brain. Following an attentional cue, yet before the onset of a visual target, we observed selective activation of target-specific neural subpopulations within shape-processing visual cortex (lateral occipital complex). Target-specific modulation of baseline activity was sustained throughout the duration of the attention trial and the degree of target specificity that characterized preparatory activation patterns correlated with perceptual performance. We conclude that top-down attention selectively activates target-specific neural codes, providing a competitive bias favoring task-relevant representations over competing representations distributed within the same subregion of visual cortex.     

From the Cover: Reconstructing representations of dynamic visual objects in early visual cortex

As raw sensory data are partial, our visual system extensively fills in missing details, creating enriched percepts based on incomplete bottom-up information. Despite evidence for internally generated representations at early stages of cortical processing, it is not known whether these representations include missing information of dynamically transforming objects. Long-range apparent motion (AM) provides a unique test case because objects in AM can undergo changes both in position and in features. Using fMRI and encoding methods, we found that the “intermediate” orientation of an apparently rotating grating, never presented in the retinal input but interpolated during AM, is reconstructed in population-level, feature-selective tuning responses in the region of early visual cortex (V1) that corresponds to the retinotopic location of the AM path. This neural representation is absent when AM inducers are presented simultaneously and when AM is visually imagined. Our results demonstrate dynamic filling-in in V1 for object features that are interpolated during kinetic transformations.Contrary to our seamless and unobstructed perception of visual objects, raw sensory data are often partial and impoverished. Thus, our visual system regularly fills in extensive details to create enriched representations of visual objects (1, 2). A growing body of evidence suggests that “filled-in” visual features of an object are represented at early stages of cortical processing where physical input is nonexistent. For example, increased activity in early visual cortex (V1) was found in retinotopic locations corresponding to nonstimulated regions of the visual field during the perception of illusory contours (3, 4) and color filling-in (5). Furthermore, recent functional magnetic resonance imaging (fMRI) studies using multivoxel pattern analysis (MVPA) methods show how regions of V1 lacking stimulus input can contain information regarding objects or scenes presented at other locations in the visual field (6, 7), held in visual working memory (8, 9), or used in mental imagery (10–13).Although these studies have found evidence for internally generated representations of static stimuli in early cortical processing, the critical question remains of whether and how interpolated visual feature representations are reconstructed in early cortical processing while objects undergo kinetic transformations, a situation that is more prevalent in our day-to-day perception.To address this question, we examined the phenomenon of long-range apparent motion (AM): when a static stimulus appears at two different locations in succession, a smooth transition of the stimulus across the two locations is perceived (14–16). Previous behavioral studies have shown that subjects perceive illusory representations along the AM trajectory (14, 17) and that these representations can interfere with the perception of physically presented stimuli on the AM path (18–21). In line with this behavioral evidence, it was found that the perception of AM leads to increased blood oxygen level-dependent (BOLD) response in the region of V1 retinotopically mapped to the AM path (22–25), suggesting the involvement of early cortical processing. This activation increase induced by the illusory motion trace was also confirmed in neurophysiological investigations on ferrets and mice using voltage-sensitive dye (VSD) imaging (26, 27). Despite these findings, however, a crucial question about the information content of the AM-induced signal remains unsolved: whether and how visual features of an object engaged in AM are reconstructed in early retinotopic cortex.Using fMRI and a forward-encoding model (28–31), we examined whether content-specific representations of the intermediate state of a dynamic object engaged in apparent rotation could be reconstructed from the large-scale, population-level, feature-tuning responses in the nonstimulated region of early retinotopic cortex representing the AM path. To dissociate signals linked to high-level interpretations of the stimulus (illusory object features interpolated in motion) from those associated with the bottom-up stimulus input (no retinal input on the path) generating the perception of motion, we used rotational AM, which produces intermediate features that are different from the features of the physically present AM-inducing stimuli (transitional AM). We further probed the nature of such AM-induced feature representations by comparing feature-tuning profiles of the AM path in V1 with those evoked when visually imagining the AM stimuli. Our findings suggest intermediate visual features of dynamic objects, which are not present anywhere in the retinal input, are reconstructed in V1 during kinetic transformations via feedback processing. This result indicates, for the first time to our knowledge, that internally reconstructed representations of dynamic objects in motion are instantiated by retinotopically organized population-level, feature-tuning responses in V1.     

From the Cover: Orientation saliency without visual cortex and target selection in archer fish

Our visual attention is attracted by salient stimuli in our environment and affected by primitive features such as orientation, color, and motion. Perceptual saliency due to orientation contrast has been extensively demonstrated in behavioral experiments with humans and other primates and is believed to be facilitated by the functional organization of the primary visual cortex. In behavioral experiments with the archer fish, a proficient hunter with remarkable visual abilities, we found an orientation saliency effect similar to that observed in human subjects. Given the enormous evolutionary distance between humans and archer fish, our findings suggest that orientation-based saliency constitutes a fundamental building block for efficient visual information processing.     

Spatiotemporal object continuity in human ventral visual cortex

Coherent visual experience requires that objects be represented as the same persisting individuals over time and motion. Cognitive science research has identified a powerful principle that guides such processing: Objects must trace continuous paths through space and time. Little is known, however, about how neural representations of objects, typically defined by visual features, are influenced by spatiotemporal continuity. Here, we report the consequences of spatiotemporally continuous vs. discontinuous motion on perceptual representations in human ventral visual cortex. In experiments using both dynamic occlusion and apparent motion, face-selective cortical regions exhibited significantly less activation when faces were repeated in continuous vs. discontinuous trajectories, suggesting that discontinuity caused featurally identical objects to be represented as different individuals. These results indicate that spatiotemporal continuity modulates neural representations of object identity, influencing judgments of object persistence even in the most staunchly "featural" areas of ventral visual cortex.     

Top-down attention switches coupling between low-level and high-level areas of human visual cortex

Top-down attention is an essential cognitive ability, allowing our finite brains to process complex natural environments by prioritizing information relevant to our goals. Previous evidence suggests that top-down attention operates by modulating stimulus-evoked neural activity within visual areas specialized for processing goal-relevant information. We show that top-down attention also has a separate influence on the background coupling between visual areas: adopting different attentional goals resulted in specific patterns of noise correlations in the visual system, whereby intrinsic activity in the same set of low-level areas was shared with only those high-level areas relevant to the current goal. These changes occurred independently of evoked activity, persisted without visual stimulation, and predicted behavioral success in deploying attention better than the modulation of evoked activity. This attentional switching of background connectivity suggests that attention may help synchronize different levels of the visual processing hierarchy, forming state-dependent functional pathways in human visual cortex to prioritize goal-relevant information.     

APP17肽对糖尿病大鼠脑枕叶视区皮层神经元退行性变的影响

目的 探讨APP17肽对糖尿病(DM)大鼠脑内视觉传导通路枕叶视区皮层神经元退行性变的影响。方法 用链脲佐菌素(STZ)腹腔注射诱发DM模型，10周后取脑组织行免疫组化染色观察神经元内凋亡因子Bax、Bcl-2、Cyto C的表达。同时取枕叶视区皮层处脑组织作电镜观察。结果 DM组枕叶视区皮层17区内Bax、CytoC阳性反应神经元数目多、染色深，Bcl-2较正常组神经元数目少，APP17肽组Bax、Bcl-2、CytoC的表达均接近正常组。DM组大鼠枕叶视区皮层神经元超微结构出现明显损害，给予APPl7肽后病变有改善。结论 APP17肽可改善DM大鼠枕叶视区皮层神经元的退行性改变。     

Visual grouping in human parietal cortex

To efficiently extract visual information from complex visual scenes to guide behavior and thought, visual input needs to be organized into discrete units that can be selectively attended and processed. One important such selection unit is visual objects. A crucial factor determining object-based selection is the grouping between visual elements. Although human lesion data have pointed to the importance of the parietal cortex in object-based representations, our understanding of these parietal mechanisms in normal human observers remains largely incomplete. Here we show that grouped shapes elicited lower functional MRI (fMRI) responses than ungrouped shapes in inferior intraparietal sulcus (IPS) even when grouping was task-irrelevant. This relative ease of representing grouped shapes allowed more shape information to be passed onto later stages of visual processing, such as information storage in superior IPS, and may explain why grouped visual elements are easier to perceive than ungrouped ones after parietal brain lesions. These results are discussed within a neural object file framework, which argues for distinctive neural mechanisms supporting object individuation and identification in visual perception.     

Spine dynamics of PSD-95-deficient neurons in the visual cortex link silent synapses to structural cortical plasticity

Critical periods (CPs) are time windows of heightened brain plasticity during which experience refines synaptic connections to achieve mature functionality. At glutamatergic synapses on dendritic spines of principal cortical neurons, the maturation is largely governed by postsynaptic density protein-95 (PSD-95)-dependent synaptic incorporation of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors into nascent AMPA-receptor silent synapses. Consequently, in mouse primary visual cortex (V1), impaired silent synapse maturation in PSD-95-deficient neurons prevents the closure of the CP for juvenile ocular dominance plasticity (jODP). A structural hallmark of jODP is increased spine elimination, induced by brief monocular deprivation (MD). However, it is unknown whether impaired silent synapse maturation facilitates spine elimination and also preserves juvenile structural plasticity. Using two-photon microscopy, we assessed spine dynamics in apical dendrites of layer 2/3 pyramidal neurons (PNs) in binocular V1 during ODP in awake adult mice. Under basal conditions, spine formation and elimination ratios were similar between PSD-95 knockout (KO) and wild-type (WT) mice. However, a brief MD affected spine dynamics only in KO mice, where MD doubled spine elimination, primarily affecting newly formed spines, and caused a net reduction in spine density similar to what has been observed during jODP in WT mice. A similar increase in spine elimination after MD occurred if PSD-95 was knocked down in single PNs of layer 2/3. Thus, structural plasticity is dictated cell autonomously by PSD-95 in vivo in awake mice. Loss of PSD-95 preserves hallmark features of spine dynamics in jODP into adulthood, revealing a functional link of PSD-95 for experience-dependent synapse maturation and stabilization during CPs.

Early life of an animal is characterized by time windows of functionally and structurally enhanced brain plasticity known as critical periods (CPs), which have been described initially in the primary visual cortex (V1) of kittens (1). During CPs, experience refines the connectivity of principal excitatory neurons to establish the mature functionality of neural networks. This refinement is governed by the constant generation and elimination of nascent synapses on dendritic spines that sample favorable connections to be consolidated and unfavorable ones to be eliminated (2–5). A fraction of nascent synapses is or becomes α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA)-receptor silent, expressing N-methyl-D-aspartate (NMDA) receptors only (6–8). At eye opening, silent synapses are abundant in the primary visual cortex (V1) (9, 10) and mature during CPs by stable AMPA receptor incorporation (11–14). The pace of silent synapse maturation is governed by the opposing yet cooperative function of postsynaptic density protein of 95 kDa (PSD-95) and its paralog PSD-93, two signaling scaffolds of the postsynaptic density of excitatory synapses (12, 13). However, whether silent synapses are preferential substrates for spine elimination during CPs remains to be investigated.In juvenile mice (postnatal days [P] 20 to 35), a brief monocular deprivation (MD) of the dominant contralateral eye results in a shift of the ocular dominance (OD) of binocular neurons in V1 toward the open eye, mediated by a reduction of responses to visual stimulation of the deprived eye (15–17). Structurally, MD induces an increase in spine elimination in apical dendrites of layer (L) 2/3 and L5 pyramidal neurons (PNs) which is only observed during the CP and constitutes a hallmark of juvenile OD plasticity (jODP) (18–20). After CP closure, cortical plasticity declines progressively, and in standard cage-raised mice beyond P40, a 4-d MD no longer induces the functional nor anatomical changes associated with jODP (21–24).At least three different mechanisms involved in experience-dependent maturation of cortical neural networks have been described, but the molecular and cellular mechanisms that cause CP closure remain highly debated (18, 25, 26). First, plasticity of local inhibitory neurons, such as increased inhibitory tone or a reduction of release probability by experience-dependent endocannabinoid receptor 1 (CB1R) activation was reported to close the critical period in rodent V1 (27–29). Second, the expression of so-called “plasticity brakes,” such as extracellular matrix (ECM), Nogo receptor 1 (NgR1), paired immunoglobulin-like receptor B (PirB), and Lynx1 were correlated with the end of critical periods (30–33). Experimentally decreasing the inhibitory tone or absence of plasticity brakes enhanced ODP expression in various knockout (KO) mouse models (32, 34, 35), among which only Lynx1 KO mice were shown to exhibit functional hallmarks of jODP, such as selective deprived eye depression after a short MD (36). Structurally, Lynx1 KO mice exhibited elevated spine dynamics at baseline; however, MD induced a reduction in spine elimination in apical dendrites of L5 PNs, whereas in L2/3 PNs there was no change (37). Thus, the effects of removing plasticity brakes on structural plasticity are variable, and it remains unclear to what extend manipulating the plasticity brakes can reinstate cellular signatures of CP plasticity in the adult wild-type (WT) brain (38). Third, the progressive maturation of AMPAR-silent synapses was correlated with the closure of the CP for jODP (12, 13). Consequently, in PSD-95 KO mice, the maturation of silent synapses is impaired; their fraction remains at the eye opening level, and jODP is preserved lifelong (13). Furthermore, visual cortex-specific knockdown (KD) of PSD-95 in the adult brain reinstated jODP. In contrast, in PSD-93 KO mice, silent synapses mature precociously and the CP for jODP closes precociously (12), correlating the presence of silent synapses with functional plasticity during CPs.While these three mechanisms of CP closure are not mutually exclusive in regulating cortical plasticity (26), it remains elusive whether CP-like structural plasticity can be expressed in the adult brain and whether silent synapses might be substrates for it. Here, we performed chronic two-photon imaging of dendrites of L2/3 pyramidal neurons in binocular V1 of PSD-95 KO (and KD) and WT mice, tracking the same dendritic spines longitudinally before, during, and after a 4-d period of MD. As previous studies have reported anesthesia effects on spine dynamics (39–41), we performed our experiments in awake mice, thoroughly trained for head fixation under the two-photon microscope. Our chronic spine imaging experiments revealed that in adult PSD-95 KO and KD mice, a brief MD indeed increased spine elimination about twofold, while adult WT mice did not display experience-dependent changes in spine elimination or spine formation. Thus, the loss of PSD-95 led to a high number of AMPAR-silent synapses which were correlated with jODP after MD, and with juvenile-like structural plasticity even in the adult brain, underscoring the importance of silent synapses for CP-timing and network maturation and stabilization.     

Learning reward timing in cortex through reward dependent expression of synaptic plasticity

The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.