Preventing the stress-induced shift from goal-directed to habit action with a β-adrenergic antagonist

Stress modulates instrumental action in favor of habit processes that encode the association between a response and preceding stimuli and at the expense of goal-directed processes that learn the association between an action and the motivational value of the outcome. Here, we asked whether this stress-induced shift from goal-directed to habit action is dependent on noradrenergic activation and may therefore be blocked by a β-adrenoceptor antagonist. To this end, healthy men and women were administered a placebo or the β-adrenoceptor antagonist propranolol before they underwent a stress or a control procedure. Shortly after the stress or control procedure, participants were trained in two instrumental actions that led to two distinct food outcomes. After training, one of the food outcomes was selectively devalued by feeding participants to satiety with that food. A subsequent extinction test indicated whether instrumental behavior was goal-directed or habitual. As expected, stress after placebo rendered participants' behavior insensitive to the change in the value of the outcome and thus habitual. After propranolol intake, however, stressed participants behaved, same as controls, goal-directed, suggesting that propranolol blocked the stress-induced bias toward habit behavior. Our findings show that the shift from goal-directed to habitual control of instrumental action under stress necessitates noradrenergic activation and could have important clinical implications, particularly for addictive disorders.     

A specific role for posterior dorsolateral striatum in human habit learning

Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen–globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.     

Stress-induced modulation of instrumental behavior: from goal-directed to habitual control of action

Actions that are directed at achieving pleasant or avoiding unpleasant states are referred to as instrumental. The acquisition of instrumental actions can be controlled by two anatomically and functionally distinct processes: a goal-directed process that is based on the prefrontal cortex and dorsomedial striatum and encodes the causal relationship between an action and the motivational value of the outcome and a dorsolateral striatum-based habit process that learns associations between actions and antecedent stimuli. Here, we review recent research showing that stress modulates the control of instrumental action in a manner that favors habitual over goal-directed action. At the neuroendocrine level, this stress-induced shift towards habit action requires the concerted action of glucocorticoids and noradrenergic arousal and is most likely accompanied by opposite functional changes in the corticostriatal circuits underlying goal-directed and habitual actions. Although generally adaptive, these changes in the control of instrumental action under stress may promote dysfunctional behaviors and the development of psychiatric disorders such as addiction.     

Emotion-induced loss aversion and striatal-amygdala coupling in low-anxious individuals

Adapting behavior to changes in the environment is a crucial ability for survival but such adaptation varies widely across individuals. Here, we asked how humans alter their economic decision-making in response to emotional cues, and whether this is related to trait anxiety. Developing an emotional decision-making task for functional magnetic resonance imaging, in which gambling decisions were preceded by emotional and non-emotional primes, we assessed emotional influences on loss aversion, the tendency to overweigh potential monetary losses relative to gains. Our behavioral results revealed that only low-anxious individuals exhibited increased loss aversion under emotional conditions. This emotional modulation of decision-making was accompanied by a corresponding emotion-elicited increase in amygdala-striatal functional connectivity, which correlated with the behavioral effect across participants. Consistent with prior reports of ‘neural loss aversion’, both amygdala and ventral striatum tracked losses more strongly than gains, and amygdala loss aversion signals were exaggerated by emotion, suggesting a potential role for this structure in integrating value and emotion cues. Increased loss aversion and striatal-amygdala coupling induced by emotional cues may reflect the engagement of adaptive harm-avoidance mechanisms in low-anxious individuals, possibly promoting resilience to psychopathology.     

Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects

Previous functional imaging experiments in humans showed activation increases in the posterior superior temporal gyrus and sulcus during observation of geometrical shapes whose movements appear intentional or goal-directed. We modeled a chase scenario between two objects, in which the chasing object used different strategies to reach the target object: The chaser either followed the target's path or appeared to predict its end position. Activation in the superior temporal gyrus of human observers was greater when the chaser adopted a predict rather than a follow strategy. Attending to the chaser's strategy induced slightly greater activation in the left superior temporal gyrus than attending to the outcome of the chase. These data implicate the superior temporal gyrus in the identification of objects displaying complex goal-directed motion.     

Dissociable brain systems mediate vicarious learning of stimulus-response and action-outcome contingencies

Two distinct strategies have been suggested to support action selection in humans and other animals on the basis of experiential learning: a goal-directed strategy that generates decisions based on the value and causal antecedents of action outcomes, and a habitual strategy that relies on the automatic elicitation of actions by environmental stimuli. In the present study, we investigated whether a similar dichotomy exists for actions that are acquired vicariously, through observation of other individuals rather than through direct experience, and assessed whether these strategies are mediated by distinct brain regions. We scanned participants with functional magnetic resonance imaging while they performed an observational learning task designed to encourage either goal-directed encoding of the consequences of observed actions, or a mapping of observed actions to conditional discriminative cues. Activity in different parts of the action observation network discriminated between the two conditions during observational learning and correlated with the degree of insensitivity to outcome devaluation in subsequent performance. Our findings suggest that, in striking parallel to experiential learning, neural systems mediating the observational acquisition of actions may be dissociated into distinct components: a goal-directed, outcome-sensitive component and a less flexible stimulus-response component.     

Simultaneous glucocorticoid and noradrenergic activity disrupts the neural basis of goal-directed action in the human brain

Stress promotes a shift from goal-directed action-outcome learning toward habitual stimulus-response learning. This shift is mediated by an interaction of noradrenergic activity and glucocorticoid stress hormones. In the present experiment, we examined the neural correlates of the stress (hormone)-induced shift from goal-directed to habit learning in the human brain. Healthy participants were administered hydrocortisone, the α2-adrenoceptor antagonist yohimbine, or both before they were trained in two instrumental actions leading to two distinct food rewards. After training, one of the rewards was devalued by feeding participants to satiety on that food. Finally, participants were presented the two instrumental actions in extinction. We collected functional magnetic resonance images both during instrumental training and during extinction testing. Our behavioral data confirmed that the simultaneous administration of hydrocortisone and yohimbine renders instrumental behavior insensitive to the outcome devaluation (and thus habitual), whereas hydrocortisone or yohimbine alone have no such effect. At the neural level, the combined administration of hydrocortisone and yohimbine reduced the sensitivity of the orbitofrontal and medial prefrontal cortex to changes in outcome value. Brain areas that have been previously implicated in habit learning were not modulated by hydrocortisone and yohimbine. These findings suggest that concurrent glucocorticoid and noradrenergic activity disrupts the neural bases of goal-directed action and thus renders behavior habitual.     

Default mode network connectivity predicts sustained attention deficits after traumatic brain injury

Traumatic brain injury (TBI) frequently produces impairments of attention in humans. These can result in a failure to maintain consistent goal-directed behavior. A predominantly right-lateralized frontoparietal network is often engaged during attentionally demanding tasks. However, lapses of attention have also been associated with increases in activation within the default mode network (DMN). Here, we study TBI patients with sustained attention impairment, defined on the basis of the consistency of their behavioral performance over time. We show that sustained attention impairments in patients are associated with an increase in DMN activation, particularly within the precuneus and posterior cingulate cortex. Furthermore, the interaction of the precuneus with the rest of the DMN at the start of the task, i.e., its functional connectivity, predicts which patients go on to show impairments of attention. Importantly, this predictive information is present before any behavioral evidence of sustained attention impairment, and the relationship is also found in a subgroup of patients without focal brain damage. TBI often results in diffuse axonal injury, which produces cognitive impairment by disconnecting nodes in distributed brain networks. Using diffusion tensor imaging, we demonstrate that structural disconnection within the DMN also correlates with the level of sustained attention. These results show that abnormalities in DMN function are a sensitive marker of impairments of attention and suggest that changes in connectivity within the DMN are central to the development of attentional impairment after TBI.     

Goal-directed behavior under emotional distraction is preserved by enhanced task-specific activation

Despite the distracting effects of emotional stimuli on concurrent task performance, humans are able to uphold goal-directed behavior. Here, we investigated the hypothesis that this effect is due to the enhanced recruitment of task-specific neural resources. In a two-step functional magnetic resonance imaging study, we first localized those areas involved in mental arithmetics by contrasting arithmetic problems with a number detection task. The resulting activation maps were then used as masks in a second experiment that compared the effects of neutral and emotional distracter images on mental arithmetics. We found increased response times in the emotional distracter condition, accompanied by enhanced activation in task-specific areas, including superior parietal cortex, dorsolateral and dorsomedial prefrontal cortex. This activation increase correlated with larger behavioral impairment through emotional distraction. Similar error rates in both conditions indicate that cognitive task performance is preserved through enhanced recruitment of task-specific neural resources when emotional distracter stimuli are present.     

Impairments of behavior, information flow between thalamus and cortex, and prefrontal cortical synaptic plasticity in an animal model of depression

Growing evidence suggests the involvement of stress in the pathophysiology of depression. This study was designed to test behavioral and electrophysiological changes in a stressed model of depression. Rats were randomly divided into control and stressed groups. Chronic unpredictable stress combined with isolation rearing was applied in rats of stressed group for three weeks. Weight and sucrose consumption were measured during the model establishing period. Behavior was measured by Morris water maze. Electroencephalography (EEG) of thalamus and prefrontal cortex was recorded after behavioral tests, followed by recording long-term potentiation (LTP) of the same thalamocortical pathway. Results showed that rats' weight and sucrose intake were significantly lower in stressed group than those in control group. In stressed group, escape latency of reversal training stage in water maze test was significantly prolonged, and platform crossings of reversal probe trials were significantly decreased. EEG test showed that the extent of thalamus driving prefrontal cortex was decreased in stressed group. LTP test showed lower postsynaptic potential amplitude in stressed group as compared to that in control group. In conclusion, chronic stress could cause certain behavioral changes in rats, with possible mechanism of impairing EEG of certain thalamocortical pathway and prefrontal cortical synaptic plasticity.     

The alcoholic brain: neural bases of impaired reward-based decision-making in alcohol use disorders

Neuroeconomics is providing insights into the neural bases of decision-making in normal and pathological conditions. In the neuropsychiatric domain, this discipline investigates how abnormal functioning of neural systems associated with reward processing and cognitive control promotes different disorders, and whether such evidence may inform treatments. This endeavor is crucial when studying different types of addiction, which share a core promoting mechanism in the imbalance between impulsive subcortical neural signals associated with immediate pleasurable outcomes and inhibitory signals mediated by a prefrontal reflective system. The resulting impairment in behavioral control represents a hallmark of alcohol use disorders (AUDs), a chronic relapsing disorder characterized by excessive alcohol consumption despite devastating consequences. This review aims to summarize available magnetic resonance imaging (MRI) evidence on reward-related decision-making alterations in AUDs, and to envision possible future research directions. We review functional MRI (fMRI) studies using tasks involving monetary rewards, as well as MRI studies relating decision-making parameters to neurostructural gray- or white-matter metrics. The available data suggest that excessive alcohol exposure affects neural signaling within brain networks underlying adaptive behavioral learning via the implementation of prediction errors. Namely, weaker ventromedial prefrontal cortex activity and altered connectivity between ventral striatum and dorsolateral prefrontal cortex likely underpin a shift from goal-directed to habitual actions which, in turn, might underpin compulsive alcohol consumption and relapsing episodes despite adverse consequences. Overall, these data highlight abnormal fronto-striatal connectivity as a candidate neurobiological marker of impaired choice in AUDs. Further studies are needed, however, to unveil its implications in the multiple facets of decision-making.     

The influence of emotion regulation on decision-making under risk

Cognitive strategies typically involved in regulating negative emotions have recently been shown to also be effective with positive emotions associated with monetary rewards. However, it is less clear how these strategies influence behavior, such as preferences expressed during decision-making under risk, and the underlying neural circuitry. That is, can the effective use of emotion regulation strategies during presentation of a reward-conditioned stimulus influence decision-making under risk and neural structures involved in reward processing such as the striatum? To investigate this question, we asked participants to engage in imagery-focused regulation strategies during the presentation of a cue that preceded a financial decision-making phase. During the decision phase, participants then made a choice between a risky and a safe monetary lottery. Participants who successfully used cognitive regulation, as assessed by subjective ratings about perceived success and facility in implementation of strategies, made fewer risky choices in comparison with trials where decisions were made in the absence of cognitive regulation. Additionally, BOLD responses in the striatum were attenuated during decision-making as a function of successful emotion regulation. These findings suggest that exerting cognitive control over emotional responses can modulate neural responses associated with reward processing (e.g., striatum) and promote more goal-directed decision-making (e.g., less risky choices), illustrating the potential importance of cognitive strategies in curbing risk-seeking behaviors before they become maladaptive (e.g., substance abuse).     

Functional connectivity with anterior cingulate and orbitofrontal cortices during decision-making

Recent neuroscience research is beginning to discover the brain regions involved in decision-making under uncertainty, but little is known about whether or how these regions functionally interact with each other. Here, we used event-related functional magnetic resonance imaging to examine both changes in overall activity and changes in functional connectivity during risk-taking. Results showed that choosing high-risk over low-risk decisions was associated with increased activity in both anterior cingulate and orbitofrontal cortices. Connectivity analyses revealed that largely distinct, but somewhat overlapping, cortical and subcortical regions exhibited significant functional connectivity with anterior cingulate and orbitofrontal cortices. Additionally, connectivity with the anterior cingulate in some regions, including the orbitofrontal cortex and nucleus accumbens, was modulated by the decision participants chose. These findings (1) elucidate large networks of brain regions that are functionally connected with both anterior cingulate and orbitofrontal cortices during decision-making and (2) demonstrate that the roles of orbitofrontal and anterior cingulate cortices can be functionally differentiated by examining patterns of connectivity.     

Protective and damaging effects of stress mediators: central role of the brain

The mind involves the whole body and two-way communication between the brain and the cardiovascular, immune, and other systems via neural and endocrine mechanisms. Stress is a condition of the mind-body interaction, and a factor in the expression of disease that differs among individuals. It is notjust the dramatic stressful events that exact their toll, but rather the many events of daily life that elevate and sustain activities of physiological systems and cause sleep deprivation, overeating, and other health-damaging behaviors, producing the feeling of being "stressed out." Over time, this results in wear and tear on the body which is called "allostatic load," and it reflects not only the impact of life experiences but also of genetic load, individual lifestyle habits reflecting items such as diet, exercise, and substance abuse, and developmental experiences that set life-long patterns of behavior and physiological reactivity. Hormones associated with stress and allostatic load protect the body in the short run and promote adaptation by the process known as allostasis, but in the long run allostatic load causes changes in the body that can lead to disease. The brain is the key organ of stress, allostasis, and allostatic load, because it determines what is threatening and therefore stressful, and also determines the physiological and behavioral responses. Brain regions such as the hippocampus, amygdala, and prefrontal cortex respond to acute and chronic stress by undergoing structural remodeling, which alters behavioral and physiological responses. Translational studies in humans with structural and functional imaging reveal smaller hippocampal volume in stress-related conditions, such as mild cognitive impairment in aging and prolonged major depressive illness, as well as in individuals with low self-esteem. Alterations in amygdala and prefrontal cortex are also reported. Besides pharmaceuticals, approaches to alleviate chronic stress and reduce allostatic load and the incidence of diseases of modern life include lifestyle change, and policies of government and business that would improve the ability of individuals to reduce their own chronic stress burden.     

Fractionating the all-or-nothing definition of goal-directed and habitual decision-making

Goal-directed and habitual decision-making are fundamental processes that support the ongoing adaptive behavior. There is a growing interest in examining their disruption in psychiatric disease, often with a focus on a disease shifting control from one process to the other, usually a shift from goal-directed to habitual control. However, several different experimental procedures can be used to probe whether decision-making is under goal-directed or habitual control, including outcome devaluation and contingency degradation. These different experimental procedures may recruit diverse behavioral and neural processes. Thus, there are potentially many opportunities for these disease phenotypes to manifest as alterations to both goal-directed and habitual controls. In this review, we highlight the examples of behavioral and neural circuit divergence and similarity, and suggest that interpretation based on behavioral processes recruited during testing may leave more room for goal-directed and habitual decision-making to coexist. Furthermore, this may improve our understanding of precisely what the involved neural mechanisms underlying aspects of goal-directed and habitual behavior are, as well as how disease affects behavior and these circuits.     

Videogame training strategy-induced change in brain function during a complex visuomotor task

Although changes in brain function induced by cognitive training have been examined, functional plasticity associated with specific training strategies is still relatively unexplored. In this study, we examined changes in brain function during a complex visuomotor task following training using the Space Fortress video game. To assess brain function, participants completed functional magnetic resonance imaging (fMRI) before and after 30 h of training with one of two training regimens: Hybrid Variable-Priority Training (HVT), with a focus on improving specific skills and managing task priority, or Full Emphasis Training (FET), in which participants simply practiced the game to obtain the highest overall score. Control participants received only 6 h of FET. Compared to FET, HVT learners reached higher performance on the game and showed less brain activation in areas related to visuo-spatial attention and goal-directed movement after training. Compared to the control group, HVT exhibited less brain activation in right dorsolateral prefrontal cortex (DLPFC), coupled with greater performance improvement. Region-of-interest analysis revealed that the reduction in brain activation was correlated with improved performance on the task. This study sheds light on the neurobiological mechanisms of improved learning from directed training (HVT) over non-directed training (FET), which is related to visuo-spatial attention and goal-directed motor planning, while separating the practice-based benefit, which is related to executive control and rule management.     

Decision making under stress: a selective review

Many decisions must be made under stress, and many decision situations elicit stress responses themselves. Thus, stress and decision making are intricately connected, not only on the behavioral level, but also on the neural level, i.e., the brain regions that underlie intact decision making are regions that are sensitive to stress-induced changes. The purpose of this review is to summarize the findings from studies that investigated the impact of stress on decision making. The review includes those studies that examined decision making under stress in humans and were published between 1985 and October 2011. The reviewed studies were found using PubMed and PsycInfo searches. The review focuses on studies that have examined the influence of acutely induced laboratory stress on decision making and that measured both decision-making performance and stress responses. Additionally, some studies that investigated decision making under naturally occurring stress levels and decision-making abilities in patients who suffer from stress-related disorders are described. The results from the studies that were included in the review support the assumption that stress affects decision making. If stress confers an advantage or disadvantage in terms of outcome depends on the specific task or situation. The results also emphasize the role of mediating and moderating variables. The results are discussed with respect to underlying psychological and neural mechanisms, implications for everyday decision making and future research directions.     

An fMRI study of risk-taking following wins and losses: implications for the gambler's fallacy

Human decision-making involving independent events is often biased and affected by prior outcomes. Using a controlled task that allows us to manipulate prior outcomes, the present study examined the effect of prior outcomes on subsequent decisions in a group of young adults. We found that participants were more risk-seeking after losing a gamble (riskloss) than after winning a gamble (riskwin), a pattern resembling the gambler's fallacy. Functional MRI data revealed that decisions after riskloss were associated with increased activation in the frontoparietal network, but decreased activation in the caudate and ventral striatum. The increased risk-seeking behavior after a loss showed a trend of positive correlation with activation in the frontoparietal network and the left lateral orbitofrontal cortex but a trend of negative correlation with activation in the amgydala and caudate. In addition, there was a trend of positive correlation between feedback-related activation in the left lateral frontal cortex and subsequent increased risk-seeking behavior. These results suggest that a strong cognitive control mechanism but a weak affective decision-making and reinforcement learning mechanism that usually contribute to flexible, goal-directed decisions can lead to decision biases involving random events. This has significant implications for our understanding of the gambler's fallacy and human decision making under risk.     

Error-rate-related caudate and parietal cortex activation during decision making

The role of cortical and subcortical structures in processing success or failure in decision-making situations is unclear. Functional neuroimaging (fMRI) during a two-choice prediction task was used to investigate the relationship between error-rate-related behavioral changes during decision-making and activation patterns in the caudate and parietal cortex. Success-related activation was found in caudate and parietal cortex during a two-choice prediction task. At low error rates, participants utilized success-related behavioral strategies rate by decreasing switching responses and increasing response predictability, which were associated with activation changes in the caudate and parietal cortex. Therefore, less response switching and increased response predictability during decision making can be directly related to the degree of activation in the caudate and posterior parietal cortex.