Striatum‐dependent habits are insensitive to both increases and decreases in reinforcer value in mice

The mouse has emerged as an advantageous species for studying the brain circuitry that underlies complex behavior and for modeling neuropsychiatric disease. The transition from flexible, goal‐directed actions to inflexible, habitual responses is argued to be a valid and reliable behavioral model for studying a core aspect of corticostriatal systems that is implicated in certain forms of psychopathology. This transition is thought to correspond to a progression of behavioral control from associative to sensorimotor corticobasal ganglia networks. Habits form following extensive training and are characterized by reduced sensitivity of instrumental responding to reinforcer revaluation; few studies have examined this form of behavioral control in mice. Here we examined the involvement of the dorsolateral and dorsomedial striatum in this transition in the C57BL/6 inbred mouse strain. We provided evidence that damage to the dorsolateral striatum disrupted habitual responding, i.e. it preserved sensitivity to changes in outcome value following either outcome devaluation or, shown for the first time in mice, outcome inflation. Together, these data show that instrumental responding in lesioned mice tracks the current value of a reinforcer and provide evidence that neuroanatomical mechanisms underlying habit learning in rats are preserved in the mouse. This will allow for the genetic and molecular dissection of neural factors involved in decision‐making and mechanisms of aberrant habit formation.     

Measuring reward assessment in a semi-naturalistic context: the effects of selective amygdala, orbital frontal or hippocampal lesions

Studying the neural mechanisms underlying complex goal-directed behaviors, such as social behavior, reward seeking or punishment avoidance, has become increasingly tractable in humans, nonhuman primates and rodents. In most experiments, however, goal-directed behaviors are measured in a laboratory setting, which is vastly different from the context in which these behaviors naturally occur. This study adapted a reward assessment paradigm, previously conducted with rhesus monkeys (Macaca mulatta) in the controlled environment of a Wisconsin General Testing Apparatus (WGTA) [Machado CJ, Bachevalier J (2007) The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates. Eur J Neurosci 25:2885-2904], to a more naturalistic context. We used this new paradigm to examine the effects of bilateral amygdaloid, hippocampal or orbital frontal cortex lesions on established food and nonfood preferences. Behavioral modification following reinforcer devaluation was also measured. Consistent with our previous study, none of the lesions produced changes in preference for palatable foods relative to pre-surgery, but animals with amygdala lesions displayed heightened preference for unpalatable foods that control or other operated animals typically avoided. In contrast to several previous WGTA-based experiments, nonfood preference was not affected by any of the lesions. Finally, animals with orbital frontal cortex lesions continued to select preferred foods after satiation, but those with amygdala, hippocampal or sham lesions altered their foraging behavior appropriately and selected less of the sated food. These findings parallel food devaluation results obtained with these same animals when tested in the WGTA. Overall, this study stresses the importance of testing context when measuring decision-making abilities in nonhuman primates with selective brain lesions.     

Adolescent but not adult Sprague-Dawley rats display goal-directed responding after reward devaluation

Alcohol drinking is typically initiated in adolescence, with use sometimes escalating to problematic levels. Escalation of drinking is often associated with a shift in drinking motives, with goal-directed initial use later transitioning to more habitual behavior. This study assessed whether adolescents are more sensitive than adults to habit formation when indexed via insensitivity to reward devaluation in an operant task for food reward. Adolescent and adult Sprague-Dawley rats were trained on either a random ratio (RR) or random interval (RI) schedule before undergoing devaluation. Adolescent animals on both schedules increased the number of lever presses across all training days. In contrast, adults in the RR group increased the number of lever presses across days whereas RI adults remained relatively stable. In response to pellet devaluation, only adolescents exhibited reduced responding, suggestive of goal-directed behavior, whereas no age differences were evident following control (home cage chow) devaluation. Contrary to our hypothesis, adolescents (but not adults) displayed goal-directed responding indexed via sensitivity to reward devaluation. These findings suggest that adolescents are not necessarily more likely to develop habits than adults, and hence other factors may contribute to the greater propensity of adolescents to engage in and escalate alcohol use.     

Effects of Motivational Downshifts on Specific Pavlovian-Instrumental Transfer in Rats

BackgroundPavlovian stimuli predictive of appetitive outcomes can exert a powerful influence on the selection and initiation of action, a phenomenon termed outcome-selective Pavlovian-instrumental transfer (sPIT). Rodent studies suggest that sPIT is insensitive to motivational downshift induced by outcome devaluation, an effect that is, however, relatively underexplored.MethodsHere we examined in detail the effects of distinct shifts in motivation from hunger to a state of relative satiety on sPIT in rats.ResultsA motivational downshift by outcome-specific devaluation immediately prior to testing markedly reduced overall lever responding and magazine entries but left intact the sPIT effect. A motivational downshift prior testing by (1) giving ad libitum rather than restricted access to maintenance diet in the home cage for 24 hours or by (2) a systemic blockade of hormone secretagogue receptor subtype 1A receptors to inhibit orexigenic actions of ghrelin both reduced overall lever responding and magazine entries. Moreover, these latter motivational downshifts reduced the sPIT effect; however, the sizes of the sPIT effects were still large.ConclusionsCollectively, our rodent findings indicate that major effects of various motivational downshifts are overall inhibition of lever pressing and magazine approach, possibly reflecting reduced general motivation. The observed effects of motivational downshifts on sPIT have implications with regard to the role of general motivating effects in sPIT and to the contribution of Pavlovian-instrumental interactions to excessive food seeking as well as obesity in humans.     

Fractionating the all-or-nothing definition of goal-directed and habitual decision-making

Goal-directed and habitual decision-making are fundamental processes that support the ongoing adaptive behavior. There is a growing interest in examining their disruption in psychiatric disease, often with a focus on a disease shifting control from one process to the other, usually a shift from goal-directed to habitual control. However, several different experimental procedures can be used to probe whether decision-making is under goal-directed or habitual control, including outcome devaluation and contingency degradation. These different experimental procedures may recruit diverse behavioral and neural processes. Thus, there are potentially many opportunities for these disease phenotypes to manifest as alterations to both goal-directed and habitual controls. In this review, we highlight the examples of behavioral and neural circuit divergence and similarity, and suggest that interpretation based on behavioral processes recruited during testing may leave more room for goal-directed and habitual decision-making to coexist. Furthermore, this may improve our understanding of precisely what the involved neural mechanisms underlying aspects of goal-directed and habitual behavior are, as well as how disease affects behavior and these circuits.     

Transient role of the rat prelimbic cortex in goal-directed behaviour

Lesion studies show that goal-directed actions mediated by action-outcome (A-O) associations and habits mediated by stimulus-response (S-R) associations can be dissociated during instrumental training, with the prelimbic region of the medial prefrontal cortex being involved in the former and the infralimbic region in the latter. The present work further investigates the role of the prelimbic region in acquisition vs. expression of goal-directed instrumental behaviour, using reversible neuronal inactivation and outcome devaluation procedures. In a first experiment, inactivating the prelimbic cortex at the time of testing did not alter the sensitivity to devaluation, indicating that this region was not essential for the expression of A-O associations. In a second experiment, the prelimbic cortex was inactivated throughout the training phase. At the time of testing the performance was insensitive to devaluation, indicating that the acquired response was not goal-directed but mediated by an S-R association. These data challenge the view that the habit system replaces the goal-directed system as training progresses. They show that the prelimbic cortex plays a transient but crucial role in the acquisition of goal-directed responding and that the A-O and S-R systems can operate in a competitive fashion early in training.     

The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates

We examined the effects of bilateral amygdaloid, hippocampal or orbital frontal cortex lesions on reward assessment in rhesus monkeys (Macaca mulatta). In Experiment 1, basic preferences for foods and inedible nonfoods were measured pre- and postsurgery. None of the lesions produced changes in animals' preferences for palatable foods or raw meat relative to presurgery, although amygdaloid or hippocampal lesions yielded increased preference for inedible nonfoods postsurgery. When the reinforcement value of each animal's highest-preferred food was decreased by selective satiation, only animals with neurotoxic orbital frontal cortex lesions continued to select the sated food. Experiment 2 measured the impact of each lesion on learning 60 concurrent discrimination problems and, then, on flexibly avoiding objects associated with sated foods in favour of objects associated with nonsated foods. None of the lesions affected concurrent discrimination learning, but animals with neurotoxic amygdala or aspiration orbital frontal lesions could not refrain from displacing items covering devalued foods. Only animals with orbital lesions also selected the devalued food beneath the object. The results indicate a functional dissociation for the amygdala and orbital frontal cortex in reward assessment, depending on the type of the reinforcer available (objects vs. food). Finally, this is the first study indicating that the hippocampal formation is involved in the assessment of familiar nonfoods, but not in judging the current value of unconditioned and conditioned reinforcers.     

Does an unconditioned stimulus memory devaluation procedure decrease disgust memories and conditioned disgust? Results of two laboratory studies

Research has demonstrated that disgust can be installed through classical conditioning by pairing neutral conditioned stimuli (CSs) with disgusting unconditioned stimuli (USs). Disgust has been argued to play an important role in maintaining fear-related disorders. This maintaining role may be explained by conditioned disgust being less sensitive to extinction (i.e., experiencing the CS in the absence of the US). Promising alternatives to extinction training are procedures that focus on the devaluation of US memory representations. In the current study, we investigated whether such devaluation procedures can be successful to counter conditioned disgust. We conducted two laboratory studies (N = 120 and N = 51) in which disgust was conditioned using audio-visual USs. Memory representations of the USs were devalued by having participants recall these USs while they performed a taxing eye-movement task or executed one of several control tasks. The results showed successful conditioned disgust acquisition. However, no strong evidence was obtained that an US memory devaluation procedure modulates disgust memory and diminishes conditioned disgust as indicated by subjective, behavioral, or psychophysiological measures. We discuss the relevance of our results for methodological improvements regarding US memory devaluation procedures and disgust conditioning.     

Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function

Addiction is characterized by persistent drug-seeking despite adverse consequences or outcomes. Such persistent behavior may result from drug-induced brain changes that increase the control of behavior by associations between antecedent cues and responses. However, it is equally plausible that brain changes cause a decrease in the control of behavior by the value of likely outcomes. To test whether drug exposure can cause persistent behavior, and to distinguish between these two accounts of such behavior, we tested cocaine-experienced rats in a Pavlovian 'reinforcer devaluation' task, which provides independent assessments of the control of behavior by antecedent cues and outcome representations. We found that cocaine exposure caused persistent responding in this setting a month after the last drug treatment, and that this deficit resulted from an inability to use representations of outcome value to guide behavior rather than from changes in stimulus-response learning or response inhibition.     

Lithium-induced outcome devaluation in instrumental conditioning: dose-effect analysis

Three experiments with rats assessed the effectiveness of various doses of lithium chloride (LiCl) in producing an aversion to the reinforcer on subsequent instrumental performance in extinction. In Experiments 1A and 1B, subjects were injected with LiCl at a dose of 3.0 and 1.5 meq/kg, respectively. These doses were administered in the form of either an isotonic solution (0.15 M) or a hypertonic solution (0.6 M). While there was an effect of the reinforcer devaluation on the instrumental performance that did not depend on the molarity of the solution with the high LiCl dose, there was no effect of devaluation with the low LiCl dose. Experiment 2 compared the 3.0- and 1.5-meq/kg dose levels directly. A depression of instrumental performance after outcome devaluation was observed in the animals injected with the high LiCl dose independently of the molarity of the solution, an effect that was absent in the low-dose condition. These results were interpreted as evidence that an instrumental reinforcer devaluation effect induced by pairing the reinforcer with illness depends upon the absolute quantity of LiCl injected and not on the concentration of solution.