Hippocampal BOLD response during category learning predicts subsequent performance on transfer generalization

To test a prediction of our previous computational model of cortico‐hippocampal interaction (Gluck and Myers [1993, 2001]) for characterizing individual differences in category learning, we studied young healthy subjects using an fMRI‐adapted category‐learning task that has two phases, an initial phase in which associations are learned through trial‐and‐error feedback followed by a generalization phase in which previously learned rules can be applied to novel associations (Myers et al. [2003]). As expected by our model, we found a negative correlation between learning‐related hippocampal responses and accuracy during transfer, demonstrating that hippocampal adaptation during learning is associated with better behavioral scores during transfer generalization. In addition, we found an inverse relationship between Blood Oxygenation Level Dependent (BOLD) activity in the striatum and that in the hippocampal formation and the orbitofrontal cortex during the initial learning phase. Conversely, activity in the dorsolateral prefrontal cortex, orbitofrontal cortex and parietal lobes dominated over that of the hippocampal formation during the generalization phase. These findings provide evidence in support of theories of the neural substrates of category learning which argue that the hippocampal region plays a critical role during learning for appropriately encoding and representing newly learned information so that that this learning can be successfully applied and generalized to subsequent novel task demands. Hum Brain Mapp 35:3122–3131, 2014. © 2013 Wiley Periodicals, Inc .     

Increasing dopamine levels in the brain improves feedback-based procedural learning in healthy participants: An artificial-grammar-learning experiment

Recently, an increasing number of studies have suggested a role for the basal ganglia and related dopamine inputs in procedural learning, specifically when learning occurs through trial-by-trial feedback (Shohamy, Myers, Kalanithi, & Gluck. (2008). Basal ganglia and dopamine contributions to probabilistic category learning. Neuroscience and Biobehavioral Reviews, 32, 219-236). A necessary relationship has however only been demonstrated in patient studies. In the present study, we show for the first time that increasing dopamine levels in the brain improves the gradual acquisition of complex information in healthy participants. We implemented two artificial-grammar-learning tasks, one with and one without performance feedback. Learning was improved after levodopa intake for the feedback-based learning task only, suggesting that dopamine plays a specific role in trial-by-trial feedback-based learning. This provides promising directions for future studies on dopaminergic modulation of cognitive functioning.     

Hippocampal mediation of stimulus representation: A computational theory

The authors propose a computational theory of the hippocampal region's function in mediating stimulus representations. The theory assumes that the hippocampal region develops new stimulus representations that enhance the discriminability of differentially predictive cues while compressing the representation of redundant cues. Other brain regions, including cerebral and cerebellar cortices, are presumed to use these hippocampal representations to recode their own stimulus representations. In the absence of an intact hippocmpal region, the theory implies that other brain regions will attempt to learn associations using previously established fixed representations. Instantiated as a connectionist network model, the theory provides a simple and unified interpretation of the functional role of the hippocampal region in a wide range of conditioning paradigms, including stimulus discrimination, reversal learning, stimulus generalization, latent inhibition, sensory preconditioning, and contextual sensitivity. The theory makes novel predictions regarding the effects of hippocampal lesions on easy-hard transfer and compound preexposure. Several prior qualitative characterizations of hippocampal function–including stimulus selection, chunking, cue configuration, and cotextual coding–are identified as task-specific special cases derivable from this more general theory. The theory suggests that a profitable direction for future empirical and theoretical research will be the study of learning tasks in which both intact and lesioned animals exhibit similar initial learning behaviors but differ on subsequent transfer and generalization tasks.     

Selectively impaired associative learning in older people with cognitive decline

Older people with declining cognitive function typically display deficits in declarative memory processes, often most evident on tests of associative learning (AL). The hippocampal formation (HF) is thought to be critically involved in the encoding and retrieval of such associations, consistent with neuroimaging findings that the HF is damaged in early stages of neurodegenerative disease and in older people with AL impairments. In the clinic, older people with cognitive decline commonly report difficulties associating names with faces. However, we have observed that such people are particularly impaired on tests requiring the association of novel stimuli. In Experiment 1, a series of AL tasks were administered to older people with cognitive decline to determine whether they were impaired at simply making associations, or at making associations between novel stimuli. In Experiment 2, we measured HF function in these subjects by administering an AL task designed to differentiate between HF-damaged and HF-intact individuals. Our experimental protocols were guided by a computational model of HF function in AL described by Gluck and Myers (1997). Older people with cognitive decline displayed impaired performance on tasks designed to be highly dependent upon intact HF function, including a task in which novel patterns and spatial locations were to be associated. These results suggest that the AL impairments observed in older people with cognitive decline may be due to HF dysfunction.     

Thalamic afferents to prefrontal cortices from ventral motor nuclei in decision‐making

The focus of this literature review is on the three interacting brain areas that participate in decision‐making: basal ganglia, ventral motor thalamic nuclei, and medial prefrontal cortex, with an emphasis on the participation of the ventromedial and ventral anterior motor thalamic nuclei in prefrontal cortical function. Apart from a defining input from the mediodorsal thalamus, the prefrontal cortex receives inputs from ventral motor thalamic nuclei that combine to mediate typical prefrontal functions such as associative learning, action selection, and decision‐making. Motor, somatosensory and medial prefrontal cortices are mainly contacted in layer 1 by the ventral motor thalamic nuclei and in layer 3 by thalamocortical input from mediodorsal thalamus. We will review anatomical, electrophysiological, and behavioral evidence for the proposed participation of ventral motor thalamic nuclei and medial prefrontal cortex in rat and mouse motor decision‐making.     

Computational models of the hippocampal region: implications for prediction of risk for Alzheimer's disease in non-demented elderly

We have pursued an interdisciplinary research program to develop novel behavioral assessment tools for evaluating specific memory impairments following damage to the medial temporal lobe, including the hippocampus and associated structures that show pathology early in the course of Alzheimer's disease (AD). Our approach uses computational models to identify the functional consequences of hippocampal-region damage, leading to testable predictions in both rodents and humans. Our modeling argues that hippocampal-region dysfunction may selectively impair the ability to generalize when familiar information is presented in novel recombinations. Previous research has shown that specific reductions in hippocampal volume in non-demented elderly individuals correlate with future development of AD. In two previous studies, we tested non-demented elderly with and without mild hippocampal atrophy (HA) on stimulus-response learning tasks. Individuals with and without HA could learn the initial information, but the HA group was selectively impaired on transfer tests where familiar features and objects were recombined. This suggests that such generalization deficits may be behavioral markers of HA, and an early indicator of risk for subsequent cognitive decline. Converging support for the relevance of these tasks to aging and Alzheimer's disease comes from our recent fMRI studies of individuals with mild cognitive impairment (MCI). Activity in the hippocampus declines with progressive training on these tasks, suggesting that the hippocampus is important for learning new stimulus representations that support subsequent transfer. Individuals with HA may be able to learn, but in a more hippocampal-independent fashion that does not support later transfer. Ultimately, this line of research could lead to a novel battery of behavioral tests sensitive to very mild hippocampal atrophy and risk for decline to AD, allowing early diagnosis and also allowing researchers to test new Alzheimer's drugs that target individuals in the earliest stages of the disease - before significant cognitive decline. A new mouse version of one of our tasks shows promise for translating these paradigms into rodents, allowing for future studies of therapeutic interventions in transgenic mouse models of AD.     

Dynamic representations and generative models of brain function

The main point made in this article is that the representational capacity and inherent function of any neuron, neuronal population or cortical area is dynamic and context-sensitive. This adaptive and contextual specialisation is mediated by functional integration or interactions among brain systems with a special emphasis on backwards or top-down connections. The critical notion is that neuronal responses, in any given cortical area, can represent different things at different times. Our argument is developed under the perspective of generative models of functional brain architectures, where higher-level systems provide a prediction of the inputs to lower-level regions. Conflict between the two is resolved by changes in the higher-level representations, driven by the resulting error in lower regions, until the mismatch is 'cancelled'. In this model the specialisation of any region is determined both by bottom-up driving inputs and by top-down predictions. Specialisation is therefore not an intrinsic property of any region but depends on both forward and backward connections with other areas. Because these other areas have access to the context in which the inputs are generated they are in a position to modulate the selectivity or specialisation of lower areas. The implications for 'classical' models (e.g., classical receptive fields in electrophysiology, classical specialisation in neuroimaging and connectionism in cognitive models) are severe and suggest these models provide incomplete accounts of real brain architectures. Generative models represent a far more plausible framework for understanding selective neurophysiological responses and how representations are constructed in the brain.     

Correlates of reward‐predictive value in learning‐related hippocampal neural activity

Temporal difference learning (TD) is a popular algorithm in machine learning. Two learning signals that are derived from this algorithm, the predictive value and the prediction error, have been shown to explain changes in neural activity and behavior during learning across species. Here, the predictive value signal is used to explain the time course of learning‐related changes in the activity of hippocampal neurons in monkeys performing an associative learning task. The TD algorithm serves as the centerpiece of a joint probability model for the learning‐related neural activity and the behavioral responses recorded during the task. The neural component of the model consists of spiking neurons that compete and learn the reward‐predictive value of task‐relevant input signals. The predictive‐value signaled by these neurons influences the behavioral response generated by a stochastic decision stage, which constitutes the behavioral component of the model. It is shown that the time course of the changes in neural activity and behavioral performance generated by the model exhibits key features of the experimental data. The results suggest that information about correct associations may be expressed in the hippocampus before it is detected in the behavior of a subject. In this way, the hippocampus may be among the earliest brain areas to express learning and drive the behavioral changes associated with learning. Correlates of reward‐predictive value may be expressed in the hippocampus through rate remapping within spatial memory representations, they may represent reward‐related aspects of a declarative or explicit relational memory representation of task contingencies, or they may correspond to reward‐related components of episodic memory representations. These potential functions are discussed in connection with hippocampal cell assembly sequences and their reverse reactivation during the awake state. The results provide further support for the proposal that neural processes underlying learning may be implementing a temporal difference‐like algorithm. © 2009 Wiley‐Liss, Inc.     

Medial temporal lobe activation during encoding and retrieval of novel face-name pairs

The human medial temporal lobe (MTL) is known to be involved in declarative memory, yet the exact contributions of the various MTL structures are not well understood. In particular, the data as to whether the hippocampal region is preferentially involved in the encoding and/or retrieval of associative memory have not allowed for a consensus concerning its specific role. To investigate the role of the hippocampal region and the nearby MTL cortical areas in encoding and retrieval of associative versus non-associative memories, we used functional magnetic resonance imaging (fMRI) to measure brain activity during learning and later recognition testing of novel face-name pairs. We show that there is greater activity for successful encoding of associative information than for non-associative information in the right hippocampal region, as well as in the left amygdala and right parahippocampal cortex. Activity for retrieval of associative information was greater than for non-associative information in the right hippocampal region also, as well as in the left perirhinal cortex, right entorhinal cortex, and right parahippocampal cortex. The implications of these data for a clear functional distinction between the hippocampal region and the MTL cortical structures are discussed.     

The role of the dentate gyrus in the formation of contextual representations

The hippocampus is involved in encoding and integrating contextual information. Recently, it has been suggested that the dorsal dentate gyrus (dDG) hippocampal subregion may mediate the formation of contextual representations of the spatial environment through a conjunctive encoding process whereby incoming multimodal information is integrated into a single higher‐order representation. Despite anatomical evidence in support of this claim, behavioral evidence is limited. Therefore, a contextual associative learning paradigm was used to determine whether the dDG supports the formation of integrated contextual representations. Male Long‐Evans rats were randomly assigned as controls or to receive bilateral intracranial infusions of colchicine into the dDG. Following recovery from surgery, each rat was tested on an appetitive task that required animals to form an association between a cue (odor) and a context to receive a food reward. Each rat received 10 trials per day and was tested for 10 consecutive days. Upon completion of testing, animals were tested on an additional two‐choice olfactory and contextual discrimination task. The testing order was counterbalanced across animals. Results showed that control animals successfully acquired the contextual associative learning task for olfactory stimuli as indicated by improved performance across the 10 testing days. In contrast, animals with dDG lesions were impaired in the ability to acquire the odor‐context associations. Results from follow‐up odor and context discrimination tests indicated that both groups acquired the discriminations at similar rates. Therefore, it is unlikely that deficits in performance on the contextual associative learning task were due to an inability to discriminate between odors or contexts. The present findings provide further support for DG involvement in the formation of conjunctive contextual representations. © 2012 Wiley Periodicals, Inc.     

Hippocampal network alterations in Alzheimer's disease and Down syndrome: from structure to therapy

Hippocampal structural and functional alterations in Alzheimer's disease (AD), detected by advanced imaging methods, have been linked to significant abnormalities in multiple internal and external networks in this critical brain region. Uncovering the temporal and anatomical pattern of these network alterations would provide important clues into understanding the pathophysiology of AD and suggest new therapeutic strategies for this multi-system and prevalent disorder. Over the last decade, we have focused on studying brain structures that provide major projections to the hippocampus (HC) and the pattern of de-afferentation of this area in mouse models of AD and a related neurodegenerative disorder, i.e. Down syndrome (DS). Our studies have revealed that major inputs into the hippocampal structure undergo significant age-dependent alterations. Studying locus coeruleus (LC), the sole source of noradrenergic terminals for the HC, it has been shown that these neurons show significant age-dependent degeneration in both mouse models of DS and AD. Furthermore, increasing noradrenergic signaling was able to restore cognitive function by improving synaptic plasticity, and possibly promoting microglia recruitment, and amyloid β (Aβ) clearance in transgenic (tg) mouse models of AD. Here, we re-examine the effects of alterations in major inputs to the hippocampal region and their structural and functional consequences in mouse models of neurodegenerative disorders. We will conclude that improving the function of major hippocampal inputs could lead to a significant improvement in cognitive function in both AD and DS.     

Cognitive demands induce selective hippocampal reorganization: Arc expression in a place and response task

Place cells in the hippocampus can maintain multiple representations of a single environment and respond to physical and/or trajectory changes by remapping. Within the hippocampus there are anatomical, electrophysiological, and behavioral dissociations between the dorsal and ventral hippocampus and within dorsal CA1. Arc expression was used to measure the recruitment of ensembles across different hippocampal subregions in rats trained to utilize two different cognitive strategies while traversing an identical trajectory. This behavioral paradigm allowed for the measurement of remapping in the absence of changes in external cues, trajectory traversed (future/past), running speed, motivation, or different stages of learning. Changes in task demands induced remapping in only some hippocampal regions: reorganization of cell ensembles was observed in dorsal CA1 but not in dorsal CA3. Moreover, a gradient was found in the degree of remapping within dorsal CA1 that corresponds to entorhinal connectivity to this region. Remapping was not seen in the ventral hippocampus: neither ventral CA1 nor CA3 exhibited ensemble changes with different cognitive demands. This contrasts with findings of remapping in both the dorsal and ventral dentate gyrus using this task. The results suggest that the dorsal pole of the hippocampus is more sensitive to changes in task demands. © 2012 Wiley Periodicals, Inc.     

Medial temporal contributions to successful face-name learning

The brain mechanisms that enable us to form durable associations between different types of information are not completely understood. Although the hippocampus is widely thought to play a substantial role in forming associations, the role of surrounding cortical regions in the medial temporal lobe, including perirhinal and parahippocampal cortex, is controversial. Using anatomically constrained functional magnetic resonance imaging, we assessed medial temporal contributions to learning arbitrary associations between faces and names. By sorting learning trials based on subsequent performance in associative and item-specific memory tests, we characterized brain activity associated with successful face-name associative learning. We found that right hippocampal activity was greater when corresponding face-name associations were subsequently remembered than when only a face or a name, but not both, were remembered, or when single-item information or associative information was not remembered. Neither perirhinal nor parahippocampal cortex encoding activity differed across these same conditions. Furthermore, right hippocampal activity during successful face-name association learning was strongly correlated with activity in cortical regions involved in multimodal integration, supporting the idea that interactions between the hippocampus and neocortex contribute to associative memory. These results specifically implicate the hippocampus in associative memory formation, in keeping with theoretical formulations in which contributions to across-domain binding differ among brain structures in the medial temporal region.     

Decoding the formation of reward predictions across learning

The predicted reward of different behavioral options plays an important role in guiding decisions. Previous research has identified reward predictions in prefrontal and striatal brain regions. Moreover, it has been shown that the neural representation of a predicted reward is similar to the neural representation of the actual reward outcome. However, it has remained unknown how these representations emerge over the course of learning and how they relate to decision making. Here, we sought to investigate learning of predicted reward representations using functional magnetic resonance imaging and multivariate pattern classification. Using a pavlovian conditioning procedure, human subjects learned multiple novel cue-outcome associations in each scanning run. We demonstrate that across learning activity patterns in the orbitofrontal cortex, the dorsolateral prefrontal cortex (DLPFC), and the dorsal striatum, coding the value of predicted rewards become similar to the patterns coding the value of actual reward outcomes. Furthermore, we provide evidence that predicted reward representations in the striatum precede those in prefrontal regions and that representations in the DLPFC are linked to subsequent value-based choices. Our results show that different brain regions represent outcome predictions by eliciting the neural representation of the actual outcome. Furthermore, they suggest that reward predictions in the DLPFC are directly related to value-based choices.     

Neural Markers of Event Boundaries

The continuous flow of sensorimotor experience is segmented into events that are stored in memory as discrete representations. These events are subsequently available for reconstruction into episodic memories or to be recombined for future thinking, prediction and imagination. Here we briefly review the patterns of brain activity that accompany the processing of events, and the transitions between them, with an aim to identifying signals that would serve as event boundary markers (EBMs). Since many previous studies have highlighted a role for the hippocampus in episodic memory function, consolidation and future thinking, we focus on activity in this region. In particular, we describe the brief bursts of hippocampal activity known as sharp‐wave ripples (SWRs), which tend to occur following the cessation of units of behavior, making them putative EBM candidates. While most current models of SWR function tend to focus on a potential role in memory consolidation or preplay linked to future thinking, here we consider an interpretation that incorporates an EBM component.     

A computational principle for hippocampal learning and neurogenesis

In the three decades since Marr put forward his computational theory of hippocampal coding, many computational models have been built on the same key principles proposed by Marr: sparse representations, rapid Hebbian storage, associative recall and consolidation. Most of these models have focused on either the CA3 or CA1 fields, using "off-the-shelf" learning algorithms such as competitive learning or Hebbian pattern association. Here, we propose a novel coding principle that is common to all hippocampal regions, and from this one principal, we derive learning rules for each of the major pathways within the hippocampus. The learning rules turn out to have much in common with several models of CA3 and CA1 in the literature, and provide a unifying framework in which to view these models. Simulations of the complete circuit confirm that both recognition memory and recall are superior relative to a hippocampally lesioned model, consistent with human data. Further, we propose a functional role for neurogenesis in the dentate gyrus (DG), namely, to create distinct memory traces for highly similar items. Our simulation results support our prediction that memory capacity increases with the number of dentate granule cells, while neuronal turnover with a fixed dentate layer size improves recall, by minimizing interference between highly similar items.     

Dendrites,deep learning,and sequences in the hippocampus

The hippocampus places us both in time and space. It does so over remarkably large spans: milliseconds to years, and centimeters to kilometers. This works for sensory representations, for memory, and for behavioral context. How does it fit in such wide ranges of time and space scales, and keep order among the many dimensions of stimulus context? A key organizing principle for a wide sweep of scales and stimulus dimensions is that of order in time, or sequences. Sequences of neuronal activity are ubiquitous in sensory processing, in motor control, in planning actions, and in memory. Against this strong evidence for the phenomenon, there are currently more models than definite experiments about how the brain generates ordered activity. The flip side of sequence generation is discrimination. Discrimination of sequences has been extensively studied at the behavioral, systems, and modeling level, but again physiological mechanisms are fewer. It is against this backdrop that I discuss two recent developments in neural sequence computation, that at face value share little beyond the label “neural.” These are dendritic sequence discrimination, and deep learning. One derives from channel physiology and molecular signaling, the other from applied neural network theory ‐ apparently extreme ends of the spectrum of neural circuit detail. I suggest that each of these topics has deep lessons about the possible mechanisms, scales, and capabilities of hippocampal sequence computation.     

Hippocampal synaptic enhancement and information storage within a distributed memory system

The hypothesis that the physical substrate of memory in the mammalian brain resides in alterations of synaptic efficacy has been proposed frequently in both neuroscience^1–5 and cognitive science^6–12, and has been widely investigated in behavioural, physiological and theoretical studies. Although this hypothesis remains unproven, considerable evidence suggests that a particular form of synaptic strengthening, induced by electrical stimulation of certain CNS fibre systems, may represent the activation of mechanisms that normally subserve associative memory. This phenomenon is known as long-term potentiation (LTP) or long-term enhancement (LTE)¹. It has been most intensively investigated within the hippocampal formation, a brain structure that plays a crucial role in certain forms of associative memory. Physiological investigation has revealed that LTE exhibits most of the properties implicit in Hebb's original suggestion that associative memory results from a synaptic strengthening that is contingent upon the conjunction of activity in pre- and post-synaptic elements. In this article, we outline a simple neuronal model capable of superimposing multiple memory traces within the same matrix of connections, and consider the correspondence between such models and the properties of LTE in the context of the hippocampal circuitry in which it occurs. Certain predictions are derived from this framework concerning the behavioural consequences of experimental manipulation of LTE, and we conclude by describing experimental evidence that confirms these predictions and suggests that LTE is, in fact, fundamentally involved in memory.     

Learning and inference in the brain.

This article is about how the brain data mines its sensory inputs. There are several architectural principles of functional brain anatomy that have emerged from careful anatomic and physiologic studies over the past century. These principles are considered in the light of representational learning to see if they could have been predicted a priori on the basis of purely theoretical considerations. We first review the organisation of hierarchical sensory cortices, paying special attention to the distinction between forward and backward connections. We then review various approaches to representational learning as special cases of generative models, starting with supervised learning and ending with learning based upon empirical Bayes. The latter predicts many features, such as a hierarchical cortical system, prevalent top-down backward influences and functional asymmetries between forward and backward connections that are seen in the real brain. The key points made in article are: (i). hierarchical generative models enable the learning of empirical priors and eschew prior assumptions about the causes of sensory input that are inherent in non-hierarchical models. These assumptions are necessary for learning schemes based on information theory and efficient or sparse coding, but are not necessary in a hierarchical context. Critically, the anatomical infrastructure that may implement generative models in the brain is hierarchical. Furthermore, learning based on empirical Bayes can proceed in a biologically plausible way. (ii). The second point is that backward connections are essential if the processes generating inputs cannot be inverted, or the inversion cannot be parameterised. Because these processes involve many-to-one mappings, are non-linear and dynamic in nature, they are generally non-invertible. This enforces an explicit parameterisation of generative models (i.e. backward connections) to afford recognition and suggests that forward architectures, on their own, are not sufficient for perception. (iii). Finally, non-linearities in generative models, mediated by backward connections, require these connections to be modulatory, so that representations in higher cortical levels can interact to predict responses in lower levels. This is important in relation to functional asymmetries in forward and backward connections that have been demonstrated empirically.