Intact goal‐directed control in treatment‐seeking drug users indexed by outcome‐devaluation and Pavlovian to instrumental transfer: critique of habit theory

Animal studies have demonstrated that chronic exposure to drugs of abuse impairs goal‐directed control over action selection indexed by the outcome‐devaluation and specific Pavlovian to instrumental transfer procedures, suggesting this impairment might underpin addiction. However, there is currently only weak evidence for impaired goal‐directed control in human drug users. Two experiments were undertaken in which treatment‐seeking drug users and non‐matched normative reference samples (controls) completed outcome‐devaluation and specific Pavlovian to instrumental transfer procedures notionally translatable to animal procedures (Experiment 2 used a more challenging biconditional schedule). The two experiments found significant outcome‐devaluation and specific Pavlovian to instrumental transfer effects overall and there was no significant difference between groups in the magnitude of these effects. Moreover, Bayes factor supported the null hypothesis for these group comparisons. Although limited by non‐matched group comparisons and small sample sizes, the two studies suggest that treatment‐seeking drug users have intact goal‐directed control over action selection, adding uncertainty to already mixed evidence concerning the role of habit learning in human drug dependence. Neuro‐interventions might seek to tackle goal‐directed drug‐seeking rather than habit formation in drug users.     

A novel fMRI paradigm to dissociate the behavioral and neural components of mixed‐strategy decision making from non‐strategic decisions in humans

During competitive interactions, such as predator–prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non‐strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed‐strategy games, how brain activity differs in the context of strategic interactions versus non‐strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed‐strategy interactions from non‐strategic choices, and we used task‐based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed‐strategy game, “matching pennies,” against a dynamic computer opponent designed to exploit predictability in players’ response patterns. Results were contrasted with a non‐strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed‐strategy game was associated with activation of a distributed cortico‐striatal network compared to the non‐strategic task. We propose that choosing in mixed‐strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.     

Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning

The dorsal striatum (DS) has been implicated in instrumental learning but its role in the acquisition of stimulus‐driven behaviour is not clear. To explore the contribution of the DS to both response‐outcome (R‐O) and stimulus‐outcome (S‐O) associative learning, we pharmacologically inactivated subregions (dorsolateral, anterior dorsomedial and posterior dorsomedial) of the DS during acquisition sessions in which subjects acquired two unique, novel R‐O pairs or two unique, novel S‐O pairs. To test whether specific R‐O or S‐O associations were learned under inactivation, rats were tested following selective‐satiety devaluation of one outcome under drug‐free conditions. In the instrumental task, control rats and rats with dorsolateral striatum (DLS) inactivation during learning responded less on the lever that had earned the devalued outcome than on the alternative lever at test, indicating that the DLS is not critical for the formation of R‐O associations. In contrast, rats with inactivation of the medial DS (DMS) (either anterior or posterior) during learning responded indiscriminately, suggesting failure to acquire the novel R‐O associations. In the Pavlovian task, both controls and rats with anterior DMS inactivation during learning responded less in the presence of the stimulus predicting the devalued outcome, whereas rats with DLS or posterior DMS inactivation during learning responded equally to the stimuli, indicating that they had not acquired the novel S‐O associations. These data confirm that the DLS and anterior region DMS mediate different aspects of reward‐related learning, and suggest that the posterior DMS may mediate a function common to both forms of learning (R‐O and S‐O). Finally, we demonstrate that both S‐O and R‐O associations are required for selective Pavlovian‐instrumental transfer.     

Hippocampal neural activity reflects the economy of choices during goal‐directed navigation

Distinguishing spatial contexts is likely essential for the well‐known role of the hippocampus in episodic memory. We studied whether types of hippocampal neural organization thought to underlie context discrimination are impacted by learned economic considerations of choice behavior. Hippocampal place cells and theta activity were recorded as rats performed a maze‐based probability discounting task that involved choosing between a small certain reward or a large probabilistic reward. Different spatial distributions of place fields were observed in response to changes in probability, the outcome of the rats' choice, and whether or not rats were free to make that choice. The degree to which the reward location was represented by place cells scaled with the expected probability of rewards. Theta power increased around the goal location also in proportion to the expected probability of signaled rewards. Furthermore, theta power dynamically varied as specific econometric information was obtained “on the fly” during task performance. Such an economic perspective of memory processing by hippocampal place cells expands our view of the nature of context memories retrieved by hippocampus during adaptive navigation.     

Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

Habits are controlled by antecedent stimuli rather than by goal expectancy. Interval schedules of feedback have been shown to generate habits, as revealed by the insensitivity of behaviour acquired under this schedule to outcome devaluation treatments. Two experiments were conducted to assess the role of the dorsolateral striatum in habit learning. In Experiment 1, sham operated controls and rats with dorsolateral striatum lesions were trained to press a lever for sucrose under interval schedules. After training, the sucrose was devalued by inducing taste aversion to it using lithium chloride, whereas saline injections were given to the controls. Only rats given the devaluation treatment reduced their consumption of sucrose and this reduction was similar in both the sham and the lesioned groups. All rats were then returned to the instrumental chamber for an extinction test, in which the lever was extended but no sucrose was delivered. In contrast to sham operated controls, rats with dorsolateral striatum lesions refrained from pressing the lever if the outcome was devalued. To assess the specificity of the role of dorsolateral striatum in this effect a second experiment was conducted in which a group with lesions of dorsomedial striatum was added. In relation now to both the sham and the dorsomedial lesioned groups, only rats with lesions of dorsolateral striatum significantly reduced responding after outcome devaluation. In conclusion, this study provides direct evidence that the dorsolateral striatum is necessary for habit formation. Furthermore, it suggests that, when the habit system is disrupted, control over instrumental performance reverts to the system controlling the performance of goal-directed instrumental actions.     

Running promotes spatial bias independently of adult neurogenesis

Different memory systems offer distinct advantages to navigational behavior. The hippocampus forms complex associations between environmental stimuli, enabling flexible navigation through space. In contrast, the dorsal striatum associates discrete cues and favorable behavioral responses, enabling habit‐like, automated navigation. While these two systems often complement one another, there are instances where striatal‐dependent responses (e.g. approach a cue) conflict with hippocampal representations of spatial goals. In conflict situations, preference for spatial vs. response strategies varies across individuals and depends on previous experience, plasticity and the integrity of these two memory systems. Here, we investigated the role of adult hippocampal neurogenesis and exercise on mouse search strategies in a water maze task that can be solved with either a hippocampal‐dependent place strategy or a striatal‐dependent cue‐response strategy. We predicted that inhibiting adult neurogenesis would impair hippocampal function and shift behavior towards striatal‐dependent cue responses. However, blocking neurogenesis in a transgenic nestin‐TK mouse did not affect strategy choice. We then investigated whether a pro‐neurogenic stimulus, running, would bias mice towards hippocampal‐dependent spatial strategies. While running indeed promoted spatial strategies, it did so even when neurogenesis was inhibited in nestin‐TK mice. These findings indicate that exercise‐induced increases in neurogenesis are not always required for enhanced cognitive function. Furthermore, our data identify exercise as a potentially useful strategy for promoting flexible, cognitive forms of memory in habit‐related disorders that are characterized by excessive responding to discrete cues.     

A visual, position-independent instrumental reinforcer devaluation task for rats

Flexible goal-directed behavior has been studied across species using reinforcer devaluation tasks, in which subjects form associations between specific stimuli (cues) and specific reinforcer(s). The reinforcer is subsequently devalued by selective satiation or taste aversion. Following devaluation, subjects adjust their responding to the cues reflecting the new value of the reinforcer. Tasks currently used in rats differ in several ways from tasks used in monkeys and this may explain contrasting results between the two species. To address one of the differences, we developed a rat task independent of spatial cues. It employs two visual cues presented simultaneously, changing left and right positions pseudorandomly. Each cue predicts one of two food reinforcers. Rats were trained to lever press in response to the two visual cues. Subsequently, they were satiated on one of the foods followed by an extinction test where in each trial they could choose to respond to one of the two cues, one predicting the devalued reinforcer and the other the non-devalued. This procedure was repeated later with the alternative food devalued. The rats adjusted their responding by choosing the cue predicting the devalued food significantly less (p<0.05) than the alternative cue. These results show that rats can discriminate two visual stimuli presented simultaneously, devalue two different foods by selective satiation, and transfer the new value to the visual cues. Discrimination of the visual cues is not aided by spatial cues, thereby eliminating a major difference between the instrumental tasks used in rats and the task used in monkeys.     

Preventing the stress-induced shift from goal-directed to habit action with a β-adrenergic antagonist

Stress modulates instrumental action in favor of habit processes that encode the association between a response and preceding stimuli and at the expense of goal-directed processes that learn the association between an action and the motivational value of the outcome. Here, we asked whether this stress-induced shift from goal-directed to habit action is dependent on noradrenergic activation and may therefore be blocked by a β-adrenoceptor antagonist. To this end, healthy men and women were administered a placebo or the β-adrenoceptor antagonist propranolol before they underwent a stress or a control procedure. Shortly after the stress or control procedure, participants were trained in two instrumental actions that led to two distinct food outcomes. After training, one of the food outcomes was selectively devalued by feeding participants to satiety with that food. A subsequent extinction test indicated whether instrumental behavior was goal-directed or habitual. As expected, stress after placebo rendered participants' behavior insensitive to the change in the value of the outcome and thus habitual. After propranolol intake, however, stressed participants behaved, same as controls, goal-directed, suggesting that propranolol blocked the stress-induced bias toward habit behavior. Our findings show that the shift from goal-directed to habitual control of instrumental action under stress necessitates noradrenergic activation and could have important clinical implications, particularly for addictive disorders.     

Differential involvement of the basolateral amygdala and orbitofrontal cortex in the formation of sensory-specific associations in conditioned flavor preference and magazine approach paradigms

Four experiments examined the roles of the basolateral amygdala and orbitofrontal cortex in the formation of sensory-specific associations in conditioned flavor preference and conditioned magazine approach paradigms using unconditioned stimulus (US) devaluation and selective Pavlovian-instrumental transfer procedures in Long Evans rats. Experiment 1 found that pre-training amygdala and orbitofrontal cortex lesions had no detectable effect on the formation or flexible use of sensory-specific flavor-nutrient associations in a US devaluation task, where flavor cues were paired either simultaneously or sequentially with nutrient rewards in water-deprived subjects. In Experiment 2, pre-training amygdala and orbitofrontal cortex lesions both attenuated outcome-specific Pavlovian-instrumental transfer. Experiment 3 indicated that amygdala lesions have no effect on the formation of sensory-specific flavor-nutrient associations in a US devaluation task in food-deprived subjects. Finally, Experiment 4 demonstrated that the outcomes used in Experiment 3 were sufficiently motivationally significant to support conditioned flavor preference. These findings suggest that, although both orbitofrontal cortex and amygdala lesions attenuate the acquisition of sensory-specific associations in magazine approach conditioning, neither lesion reduces the ability to appropriately respond to a flavor cue that was paired with a devalued outcome.     

Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats

Two experiments examined the effects of bilateral excitotoxic lesions of either the mediodorsal (MD) or anterior (ANT) thalamic nuclei on instrumental acquisition and performance, sensitivity to changes in the value of the instrumental outcome, and sensitivity to changes in the instrumental contingency. Rats were food deprived and trained to press two levers, each earning a unique food outcome (pellets or sucrose). All rats acquired the instrumental response although ANT lesions appear slightly to increase and MD lesions slightly to suppress instrumental performance. After training, specific satiety-induced devaluation of one of the two instrumental outcomes produced a selective reduction in responding on the lever that in training had earned the now devalued outcome but only in the SHAM and ANT groups. In contrast, MD animals failed to show evidence of a selective devaluation effect when tested in extinction. Additionally, SHAM and ANT animals selectively decreased responding when one action-outcome contingency was degraded, whereas MD animals reduced responding nonselectively on the two levers. Subsequent tests established that an inability to discriminate between either the two actions or the two outcomes cannot account for the lack of selective responding observed in the MD animals. Together these data suggest that MD lesions produce a profound deficit in the ability of rats to utilize specific action-outcome associations and appear to render rats relatively insensitive to the causal consequences of their instrumental actions. In contrast, far from producing a deficit, ANT lesioned rats were as sensitive to the effects of these behavioural manipulations as the sham lesioned controls.     

Neural network modeling of the hippocampal formation spatial signals and their possible role in navigation: A modular approach

Cells throughout the hippocampal formation show striking spatial firing correlates as a rat navigates through space. These cells are thought to play a critical role in orchestrating the navigational abilities of the animals, since damage to the hippocampal formation causes spatial learning deficits. Here, we present a theoretical framework aimed at explaining how the different spatial signals are generated, as well as how they may help guide navigational behavior. Earlier work from our laboratory has presented a simple model for how the location-related signals exhibited by hippocampal place cells could be generated, based on convergent sensory information. Here, the results of this work are combined with two more recent models, to provide a more comprehensive theoretical framework. Specifically, we present 1) A neural network model of head direction cells, based on the idea that the directional signals are generated using a path integration mechanism. Cells which combine directional and angular head velocity information project onto the head direction cells, to “update” the current directional signal. This model reproduces the basic phenomenon of direction-specific firing, as well as the anticipatory nature of this firing, reported for some head direction cells. 2) A network simulation of how the hippocampal spatial signals could be used to orchestrate instrumental learning. Here, place and directional signals converge onto motor cells, each of which are thus driven to fire to specific combinations of location and directional heading. Each active motor cell generates a small leftward or rightward “step” of the simulated animal. When the simulated goal is encountered, recently active synapses are strengthened, so that goal-directed trajectories are “stamped in.” We have found these models useful in helping to clarify our thinking about the proposed theoretical principles, as well as in generating testable predictions. © 1997 Wiley-Liss, Inc.     

Avoidance‐based human Pavlovian‐to‐instrumental transfer

The Pavlovian‐to‐instrumental transfer (PIT) paradigm probes the influence of Pavlovian cues over instrumentally learned behavior. The paradigm has been used extensively to probe basic cognitive and motivational processes in studies of animal learning. More recently, PIT and its underlying neural basis have been extended to investigations in humans. These initial neuroimaging studies of PIT have focused on the influence of appetitively conditioned stimuli on instrumental responses maintained by positive reinforcement, and highlight the involvement of the striatum. In the current study, we sought to understand the neural correlates of PIT in an aversive Pavlovian learning situation when instrumental responding was maintained through negative reinforcement. Participants exhibited specific PIT, wherein selective increases in instrumental responding to conditioned stimuli occurred when the stimulus signaled a specific aversive outcome whose omission negatively reinforced the instrumental response. Additionally, a general PIT effect was observed such that when a stimulus was associated with a different aversive outcome than was used to negatively reinforce instrumental behavior, the presence of that stimulus caused a non‐selective increase in overall instrumental responding. Both specific and general PIT behavioral effects correlated with increased activation in corticostriatal circuitry, particularly in the striatum, a region involved in cognitive and motivational processes. These results suggest that avoidance‐based PIT utilizes a similar neural mechanism to that seen with PIT in an appetitive context, which has implications for understanding mechanisms of drug‐seeking behavior during addiction and relapse.     

Loss of striatal tyrosine‐hydroxylase interneurons impairs instrumental goal‐directed behavior

The striatum mediates a broad range of cognitive and motor functions. Within the striatum, recently discovered tyrosine hydroxylase expressing interneurons (THINs) provide a source of intrastriatal synaptic connectivity that is critical for regulating striatal activity, yet the role of THIN's in behavior remains unknown. Given the important role of the striatum in reward‐based behaviors, we investigated whether loss of striatal THINs would impact instrumental behavior in mice. We selectively ablated striatal THINs in TH‐Cre mice using chemogenetic techniques, and then tested THIN‐lesioned or control mice on three reward‐based striatal‐dependent instrumental tests: (a) progressive ratio test; (b) choice test following selective‐satiety induced outcome devaluation; (c) outcome reinstatement test. Both striatal‐THIN‐lesioned and control mice acquired an instrumental response for flavored food pellets, and their behavior did not differ in the progressive ratio test, suggesting intact effort to obtain rewards. However, striatal THIN lesions markedly impaired choice performance following selective‐satiety induced outcome devaluation. Unlike control mice, THIN‐lesioned mice did not adjust their choice of actions following a change in outcome value. In the outcome reinstatement test THIN‐lesioned and control mice showed response invigoration by outcome presentation, suggesting the incentive properties of outcomes were not disrupted by THIN lesions. Overall, we found that striatal THIN lesions selectively impaired goal‐directed behavior, while preserving motoric and appetitive behaviors. These findings are the first to describe a function of striatal THINs in reward‐based behavior, and further illustrate the important role for intrastriatal interneuronal connectivity in behavioral functions ascribed to the striatum more generally.     

Epigenetic and pharmacological regulation of 5HT3 receptors controls compulsive ethanol seeking in mice

Factors underlying individual vulnerability to develop alcoholism are largely unknown. In humans, the risk for alcoholism is associated with elevated cue reactivity. Recent evidence suggests that in animal models, reactivity to reward‐paired cues is predictive of addictive behaviors. To model cue reactivity in mice, we used a Pavlovian approach (PA) paradigm in which mice were trained to associate a cue with delivery of a food reinforcer. We then investigated the relationship between PA status with habitual and compulsive‐like ethanol seeking. After training mice to respond for 10% ethanol, habitual behavior was investigated using both an outcome devaluation paradigm, in which ethanol was devalued via association with lithium chloride‐induced malaise, and a contingency degradation paradigm in which the relationship between action and outcome was disrupted. Compulsive‐like behavior was investigated in a modified conditioned place preference paradigm in which footshock was paired with the reward‐paired chamber. PA was found to be predictive of habitual and compulsive‐like ethanol seeking. Additionally, innate risk status was related to epigenetic changes in the gene encoding the requisite subunit of the 5HT3 receptor, Htr3a, as well as 5HT3A protein expression in the amygdala. We then used pharmacological tools to demonstrate that risk status determines the ability of a 5HT3 antagonist to reduce compulsive ethanol seeking. These data indicate that risk status can be identified prior to any alcohol exposure by assessment of cue reactivity, and further that this endophenotype may be predictive of response to pharmacological treatment for components of alcoholism.     

Dissociable regulation of instrumental action within mouse prefrontal cortex

Evaluation of the behavioral 'costs', such as effort expenditure relative to the benefits of obtaining reward, is a major determinant of goal-directed action. Neuroimaging evidence suggests that the human medial orbitofrontal cortex (mOFC) is involved in this calculation and thereby guides goal-directed and choice behavior, but this region's functional significance in rodents is unknown despite extensive work characterizing the role of the lateral OFC in cue-related response inhibition processes. We first tested mice with mOFC lesions in an instrumental reversal task lacking discrete cues signaling reinforcement; here, animals were required to shift responding based on the location of the reinforced aperture within the chamber. Mice with mOFC lesions acquired the reversal but failed to inhibit responding on the previously reinforced aperture, while mice with prelimbic prefrontal cortex lesions were unaffected. When tested on a progressive ratio schedule of reinforcement, mice with prelimbic cortical lesions were unable to maintain responding, resulting in declining response levels. Mice with mOFC lesions, by contrast, escalated responding. Neither lesion affected sensitivity to satiety-specific outcome devaluation or non-reinforcement (i.e. extinction), and neither had effects when placed after animals were trained on a progressive ratio response schedule. Lesions of the ventral hippocampus, which projects to the mOFC, resulted in similar response patterns, while lateral OFC and dorsal hippocampus lesions resulted in response acquisition, though not inhibition, deficits in an instrumental reversal. Our findings thus selectively implicate the rodent mOFC in braking reinforced goal-directed action when reinforcement requires the acquisition of novel response contingencies.     

Nonnavigational spatial memory performance is unaffected by hippocampal damage in monkeys

Evidence that the hippocampus is critical for spatial memory in nonnavigational tests is mixed. A recent study reported that temporary hippocampal inactivation impaired spatial memory in the nonnavigational Hamilton Search Task in monkeys. However, several studies have documented no impairment on other nonnavigational spatial memory tests following permanent hippocampal lesions. It was hypothesized that transient, but not permanent, hippocampal disruption produces deficits because monkeys undergoing transient inactivation continue to try to use a hippocampal‐dependent strategy, whereas monkeys with permanent lesions use a nonhippocampal‐dependent strategy. We evaluated this hypothesis by testing five rhesus monkeys with hippocampal lesions and five controls on a computerized analogue of the Hamilton Search Task. On each trial, monkeys saw an array of squares on a touchscreen, each of which “hid” one reward. Retrieving a reward depleted that location and monkeys continued selecting squares until they found all rewards. The optimal strategy is to remember chosen locations and choose each square once. Unlike the inactivation study, monkeys with hippocampal damage were as accurate as controls regardless of retention interval. Critically, we found no evidence that the groups used different strategies, as measured by learning rates, spatial search biases, perseverative win‐stay errors, or inter‐choice distance. This discrepancy between the effect of inactivations and lesions may result from off‐target effects of inactivations or as‐yet‐unidentified differences between the physical and computerized tasks. Combined with previous evidence that hippocampal damage impairs navigational memory in monkeys, this evidence constrains the role of the hippocampus in spatial memory as being critical for navigational tests that likely involve allocentric spatial memory but not nonnavigational tests that likely involve egocentric spatial memory.     

Response selection codes in neurophysiological data predict conjoint effects of controlled and automatic processes during response inhibition

The inhibition of prepotent responses is a requirement for goal‐directed behavior and several factors determine corresponding successful response inhibition processes. One factor relates to the degree of automaticity of pre‐potent response tendencies and another factor relates to the degree of cognitive control that is exerted during response inhibition. However, both factors can conjointly modulate inhibitory control. Cognitive theoretical concepts suggest that codings of stimulus‐response translations may underlie such conjoint effects. Yet, it is unclear in how far such specific codes, as assumed in cognitive psychological concepts, are evident in neurophysiological processes and whether there are specific functional neuroanatomical structures associated with the processing of such codes. Applying a temporal decomposition method of EEG data in combination with source localization methods we show that there are different, intermingled codes (i.e., “stimulus codes” and “response selection codes”) at the neurophysiological level during conjoint effects of “automatic” and “controlled” processes in response inhibition. Importantly, only “response selection codes” predict behavioral performance, and are subject to conjoint modulations by “automatic” and “controlled” processes. These modulations are associated with inferior and superior parietal areas (BA40/BA7), possibly reflecting an updating of internal representations when information is complex and probably difficult to categorize, but essential for behavioral control. Codes proposed by cognitive, psychological concepts seem to have a neurophysiological analogue that fits into current views on functions of inferior and superior parietal regions.     

The role of the dorsomedial striatum in instrumental conditioning

Considerable evidence suggests that, in instrumental conditioning, rats learn the relationship between actions and their specific consequences or outcomes. The present study examined the role of the dorsomedial striatum (DMS) in this type of learning after excitotoxic lesions and reversible, muscimol-induced inactivation. In three experiments, rats were first trained to press two levers for distinct outcomes, and then tested after training using a variety of behavioural assays that have been established to detect action-outcome learning. In Experiment 1, pre-training lesions of the posterior DMS abolished the sensitivity of rats' instrumental performance to both outcome devaluation and contingency degradation when tested in extinction, whereas lesions of the anterior DMS had no effect. In Experiment 2, both pre-training and post-training lesions of the posterior DMS were equally effective in reducing the sensitivity of performance both to devaluation and degradation treatments. In Experiment 3, the infusion of muscimol into the posterior DMS selectively abolished sensitivity of performance to devaluation and contingency degradation without impairing the ability of rats to discriminate either the instrumental actions performed or the identity of the earned outcomes. Taken together, these results suggest that the posterior region of the DMS is a crucial neural substrate for the acquisition and expression of action-outcome associations in instrumental conditioning.     

Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens

Cells in the hippocampal formation show spatial firing correlates thought to be critical to the role played by this structure in spatial learning. Place cells in the hippocampus proper show location-specific activity, whereas cells in the postsubiculum fire as a function of momentary directional heading. One question which has received little attention is how these spatial signals are used by motor structures to actually guide spatial behavior. Here we present a model of how one kind of spatial behavior, instrumental learning in the Morris water maze, could be guided by the spatial information in the hippocampal formation. For this, we concentrate on the hippocampal projection to the nucleus accumbens, which is strongly implicated in instrumental learning. In the model, simulated firing patterns of place cells and head direction cells activate “motor” cells in the “accumbens.” Each motor cell causes a particular locomotor movement in a simulated rat. In this way, the “rat” locomotes through the simulated environment. Each step places the animal in a slightly different location and directional orientation, which, in turn, activates a different set of place and head direction cells, thus causing the next locomotor response, and so on. Connection strengths between cells are initially set randomly. When the animal encounters the reward location, however, connections are altered, so that recently active synapses are strengtheened. Thus, successful moves in a particular locational and directional context are “stamped in.” Simulated rats show rapid learning, similar in many ways to that of actual rats. In particular, they generate efficient routes to the goal after minimal experience, and can do so from somewhat novel starting positions. Consideration of the model architecture shows that (1) combined use of directional and place information is an example of a linearly inseparable problem and that (2) some types of novel route generation, often thought to require a “cognitive mapping” strategy, can be generated from the S-R type model used here. © 1995 Wiley-Liss, Inc.     

At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation

Although it has long been hypothesized that the nucleus accumbens (NAc) acts as an interface between limbic and motor regions, direct evidence for this modulatory role on behavior is lacking. Using a disconnection procedure in rats, we found that basolateral amygdala (BLA) input to the core and medial shell of the NAc separately mediate two distinct incentive processes controlling the performance of goal-directed instrumental actions, respectively: (i) the sensitivity of instrumental responding to changes in the experienced value of the goal or outcome, produced by specific satiety-induced outcome devaluation; and (ii) the effect of reward-related cues on action selection, observed in outcome-specific Pavlovian-instrumental transfer. These results reveal, therefore, that dissociable neural circuits involving BLA inputs to the NAc core and medial shell mediate distinct components of the incentive motivational processes controlling choice and decision-making in instrumental conditioning.