Reward circuitry dysfunction in psychiatric and neurodevelopmental disorders and genetic syndromes: animal models and clinical findings

This review summarizes evidence of dysregulated reward circuitry function in a range of neurodevelopmental and psychiatric disorders and genetic syndromes. First, the contribution of identifying a core mechanistic process across disparate disorders to disease classification is discussed, followed by a review of the neurobiology of reward circuitry. We next consider preclinical animal models and clinical evidence of reward-pathway dysfunction in a range of disorders, including psychiatric disorders (i.e., substance-use disorders, affective disorders, eating disorders, and obsessive compulsive disorders), neurodevelopmental disorders (i.e., schizophrenia, attention-deficit/hyperactivity disorder, autism spectrum disorders, Tourette’s syndrome, conduct disorder/oppositional defiant disorder), and genetic syndromes (i.e., Fragile X syndrome, Prader–Willi syndrome, Williams syndrome, Angelman syndrome, and Rett syndrome). We also provide brief overviews of effective psychopharmacologic agents that have an effect on the dopamine system in these disorders. This review concludes with methodological considerations for future research designed to more clearly probe reward-circuitry dysfunction, with the ultimate goal of improved intervention strategies.     

How the aging brain translates motivational incentive into action: the role of individual differences in striato-cortical white matter pathways

The anticipation of reward enhances actions that lead to those rewards, but individuals differ in how effectively motivational incentives modulate their actions. Such individual differences are particularly prominent in aging. In order to account for such inter-individual variability among older adults, we approach the neurobiological mechanisms of motivated behavior from an individual differences perspective focusing on white matter pathways in the aging brain. Using analyses of probabilistic tractography seeded in the striatum, we report that the estimated strength of cortico-striatal and intra-striatal white matter pathways among older adults correlated with how effectively motivational incentives modulated their actions. Specifically, individual differences in the extent to which elderly participants utilized reward cues to prepare and perform more efficient antisaccades predicted structural connectivity of the striatum with cortical areas involved in reward anticipation and oculomotor control. These striatal connectivity profiles endow us with a network account for individual differences in motivated behavior among older adults. More generally, the data suggest that capturing individual differences may be crucial to better understand developmental trajectories in motivated behavior.     

Increased Striatal Dopamine Synthesis Capacity in Gambling Addiction

Background

The hypothesis that dopamine plays an important role in the pathophysiology of pathological gambling is pervasive. However, there is little to no direct evidence for a categorical difference between pathological gamblers and healthy control subjects in terms of dopamine transmission in a drug-free state. Here we provide evidence for this hypothesis by comparing dopamine synthesis capacity in the dorsal and ventral parts of the striatum in 13 pathological gamblers and 15 healthy control subjects.

Differential activation of the striatum for decision making and outcomes in a monetary task with gain and loss

We examined brain activation for decision making and for different monetary outcomes in a two-alternative choice task with monetary rewards and punishments. Brain hemodynamic changes were monitored by functional MRI (fMRI) when seventeen healthy volunteers performed a task in which they were required to press one of two buttons (button 1 or 2) which increased or decreased their previously endowed money depending upon accord or disaccord with the number (1 or 2) which was displayed later. The amount of money subjects gained or lost was large when they selected 2 and small when they selected 1. They were informed in advance that the order of the appearance of 1 or 2 at outcome phase was predetermined and random, irrespective of their selection, and that the expected value was mathematically zero for both selecting 1 and 2. Results of fMRI showed that bilateral putamen and nucleus accumbens were more activated when selecting large gain or loss option than selecting small gain or loss option, and the right putamen and nucleus accumbens were more activated for gain outcome than for loss outcome. No significantly different activation of the striatum was found between large gain outcome and small gain outcome, suggesting that activity of the striatum at outcome phase was insensitive for reward magnitude when expectation by subject was already performed. We also found that activations of the bilateral putamen were correlated parametrically with stock amount of money.     

DAT genotype modulates striatal processing and long-term memory for items associated with reward and punishment

Previous studies have shown that appetitive motivation enhances episodic memory formation via a network including the substantia nigra/ventral tegmental area (SN/VTA), striatum and hippocampus. This functional magnetic resonance imaging (fMRI) study now contrasted the impact of aversive and appetitive motivation on episodic long-term memory. Cue pictures predicted monetary reward or punishment in alternating experimental blocks. One day later, episodic memory for the cue pictures was tested. We also investigated how the neural processing of appetitive and aversive motivation and episodic memory were modulated by dopaminergic mechanisms. To that end, participants were selected on the basis of their genotype for a variable number of tandem repeat polymorphism of the dopamine transporter (DAT) gene. The resulting groups were carefully matched for the 5-HTTLPR polymorphism of the serotonin transporter gene. Recognition memory for cues from both motivational categories was enhanced in participants homozygous for the 10-repeat allele of the DAT, the functional effects of which are not known yet, but not in heterozygous subjects. In comparison with heterozygous participants, 10-repeat homozygous participants also showed increased striatal activity for anticipation of motivational outcomes compared to neutral outcomes. In a subsequent memory analysis, encoding activity in striatum and hippocampus was found to be higher for later recognized items in 10-repeat homozygotes compared to 9/10-repeat heterozygotes. These findings suggest that processing of appetitive and aversive motivation in the human striatum involve the dopaminergic system and that dopamine plays a role in memory for both types of motivational information. In accordance with animal studies, these data support the idea that encoding of motivational events depends on dopaminergic processes in the hippocampus.     

Reprint of: DAT genotype modulates striatal processing and long-term memory for items associated with reward and punishment

Previous studies have shown that appetitive motivation enhances episodic memory formation via a network including the substantia nigra/ventral tegmental area (SN/VTA), striatum and hippocampus. This functional magnetic resonance imaging (fMRI) study now contrasted the impact of aversive and appetitive motivation on episodic long-term memory. Cue pictures predicted monetary reward or punishment in alternating experimental blocks. One day later, episodic memory for the cue pictures was tested. We also investigated how the neural processing of appetitive and aversive motivation and episodic memory were modulated by dopaminergic mechanisms. To that end, participants were selected on the basis of their genotype for a variable number of tandem repeat polymorphism of the dopamine transporter (DAT) gene. The resulting groups were carefully matched for the 5-HTTLPR polymorphism of the serotonin transporter gene. Recognition memory for cues from both motivational categories was enhanced in participants homozygous for the 10-repeat allele of the DAT, the functional effects of which are not known yet, but not in heterozygous subjects. In comparison with heterozygous participants, 10-repeat homozygous participants also showed increased striatal activity for anticipation of motivational outcomes compared to neutral outcomes. In a subsequent memory analysis, encoding activity in striatum and hippocampus was found to be higher for later recognized items in 10-repeat homozygotes compared to 9/10-repeat heterozygotes. These findings suggest that processing of appetitive and aversive motivation in the human striatum involve the dopaminergic system and that dopamine plays a role in memory for both types of motivational information. In accordance with animal studies, these data support the idea that encoding of motivational events depends on dopaminergic processes in the hippocampus.     

Striatal responses to negative monetary outcomes differ between temperamentally inhibited and non-inhibited adolescents

The present study compared blood oxygen level dependent (BOLD) response in behaviorally inhibited and behaviorally non-inhibited adolescents to positive and negative feedback following their choice in a reward task. Previous data in these same subjects showed enhanced activation in striatal areas in behaviorally inhibited subjects to cues predicting gain or a loss. However, no analyses had examined responses following actual gains or losses. Relative to non-inhibited subjects, behaviorally inhibited subjects in the current study showed enhanced caudate response to negative but not positive feedback, indicating that striatal sensitivity to feedback may be specific to aversive information. In addition, compared to non-inhibited subjects, behaviorally inhibited subjects exhibited reduced differentiation between positive and negative feedback in ventromedial prefrontal cortex (vmPFC). This suggests a perturbed ability to encode reward value.     

The dopaminergic response to acute stress in health and psychopathology: A systematic review

Previous work in animals has shown that dopamine (DA) in cortex and striatum plays an essential role in stress processing. For the first time, we systematically reviewed the in vivo evidence for DAergic stress processing in health and psychopathology in humans. All studies included (n studies = 25, n observations = 324) utilized DA D2/3 positron emission tomography and measured DAergic activity during an acute stress challenge. The evidence in healthy volunteers (HV) suggests that physiological, but not psychological, stress consistently increases striatal DA release. Instead, increased medial prefrontal cortex (mPFC) DAergic activity in HV was observed during psychological stress. Across brain regions, stress-related DAergic activity was correlated with the physiological and psychological intensity of the stressor. The magnitude of stress-induced DA release was dependent on rearing conditions, personality traits and genetic variations in several SNPs. In psychopathology, preliminary evidence was found for stress-related dorsal striatal DAergic hyperactivity in psychosis spectrum and a blunted response in chronic cannabis use and pain-related disorders, but results were inconsistent. Physiological stress-induced DAergic activity in striatum in HV may reflect somatosensory properties of the stressor and readiness for active fight-or-flight behavior. DAergic activity in HV in the ventral striatum and mPFC may be more related to expectations about the stressor and threat evaluation, respectively. Future studies with increased sample size in HV and psychopathology assessing the functional relevance of stress-induced DAergic activity, the association between cortical and subcortical DAergic activity and the direct comparison of different stressors are necessary to conclusively elucidate the role of the DA system in the stress response.     

Different developmental trajectories for anticipation and receipt of reward during adolescence

Typical adolescent behaviour such as increased risk-taking and novelty-seeking is probably related to developmental changes in the brain reward system. This functional MRI study investigated how brain activation related to two components of reward processing (Reward Anticipation and Reward Outcome) changes with age in a sample of 39 children, adolescents and young adults aged 10–25. Our data revealed age-related changes in brain activity during both components of reward processing. Activation related to Reward Anticipation increased with age, while activation related to Reward Outcome decreased in various regions of the reward network. This shift from outcome to anticipation was confirmed by subsequent analyses showing positive correlations between age and the difference in activation between Reward Anticipation and Reward Outcome. The shift was predominantly present in striatal regions and was accompanied by a significant effect of age on behaviour, with older participants showing more response speeding on potentially rewarding trials than younger participants. This study provides evidence for functional changes in the reward system which may underlie typical adolescent behaviour.     

Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback

A simple working memory model based on recurrent network activation is proposed and its application to selection and reinforcement of an action is demonstrated as a solution to the temporal credit assignment problem. Reactivation of recent salient cue states is generated and maintained as a type of salience gated recurrently active working memory, while lower salience distractors are ignored. Cue reactivation during the action selection period allows the cue to select an action while its reactivation at the reward period allows the reinforcement of the action selected by the reactivated state, which is necessarily the action which led to the reward being found. A down-gating of the external input during the reactivation and maintenance prevents interference. A double winner-take-all system which selects only one cue and only one action allows the targeting of the cue–action allocation to be modified. This targeting works both to reinforce a correct cue–action allocation and to punish the allocation when cue–action allocations change. Here we suggest a firing rate neural network implementation of this system based on the basal ganglia anatomy with input from a cortical association layer where reactivations are generated by signals from the thalamus. Striatum medium spiny neurons represent actions. Auto-catalytic feedback from a dopamine reward signal modulates three-way Hebbian long term potentiation and depression at the cortical–striatal synapses which represent the cue–action associations. The model is illustrated by the numerical simulations of a simple example — that of associating a cue signal to a correct action to obtain reward after a delay period, typical of primate cue reward tasks. Through learning, the model shows a transition from an exploratory phase where actions are generated randomly, to a stable directed phase where the animal always chooses the correct action for each experienced state. When cue–action allocations change, we show that this is noticed by the model, the incorrect cue–action allocations are punished and the correct ones discovered.     

Effect of aging on stimulus-reward association learning

The flexible learning of stimulus-reward associations when required by situational context is essential for everyday behavior. Older adults experience a progressive decline in several cognitive functions and show deficiencies in neuropsychological tasks requiring flexible adaptation to external feedback, which could be related to impairments in reward association learning. To study the effect of aging on stimulus-reward association learning 20 young and 20 older adults performed a probabilistic object reversal task (pORT) along with a battery of tests assessing executive functions and general intellectual abilities. The pORT requires learning and reversing associations between actions and their outcomes. Older participants collected fewer points, needed more trials to reach the learning criterion, and completed less blocks successfully compared to young adults. This difference remained statistically significant after correcting for the age effect of other tests assessing executive functions. This suggests that there is an age-related difference in reward association learning as measured using the pORT, which is not closely related to other executive functions with respect to the age effect. In human aging, structural alterations of reward detecting structures and functional changes of the dopaminergic as well as the serotonergic system might contribute to the deficit in reward association learning observed in this study.     

Neural processing of reward in adolescent rodents

Immaturities in adolescent reward processing are thought to contribute to poor decision making and increased susceptibility to develop addictive and psychiatric disorders. Very little is known; however, about how the adolescent brain processes reward. The current mechanistic theories of reward processing are derived from adult models. Here we review recent research focused on understanding of how the adolescent brain responds to rewards and reward-associated events. A critical aspect of this work is that age-related differences are evident in neuronal processing of reward-related events across multiple brain regions even when adolescent rats demonstrate behavior similar to adults. These include differences in reward processing between adolescent and adult rats in orbitofrontal cortex and dorsal striatum. Surprisingly, minimal age related differences are observed in ventral striatum, which has been a focal point of developmental studies. We go on to discuss the implications of these differences for behavioral traits affected in adolescence, such as impulsivity, risk-taking, and behavioral flexibility. Collectively, this work suggests that reward-evoked neural activity differs as a function of age and that regions such as the dorsal striatum that are not traditionally associated with affective processing in adults may be critical for reward processing and psychiatric vulnerability in adolescents.     

The novelty exploration bonus and its attentional modulation

We hypothesized that novel stimuli represent salient learning signals that can motivate ‘exploration’ in search for potential rewards. In computational theories of reinforcement learning, this is referred to as the novelty ‘exploration bonus’ for rewards. If true, stimulus novelty should enhance the reward anticipation signals in brain areas that are part of dopaminergic circuitry and thereby reduce responses to reward outcomes. We investigated this hypothesis in two fMRI experiments. Images of complex natural scenes predicted monetary reward or a neutral outcome by virtue of depicting either indoor or outdoor scenes. Half of the reward-predicting and neutral images had been familiarized the day before, the other half were novel. In experiment 1, subjects indicated whether images were novel or familiar, whereas in experiment 2, they explicitly decided whether or not images predicted reward by depicting indoor or outdoor scenes. Novelty led to the hypothesized enhancement of mesolimbic reward prediction responses and concomitant reduction of mesolimbic responses to reward outcomes. However, this effect was strongly task-dependent and occurred only in experiment 2, when the reward-predicting property of each image was attended. Recognition memory for the novel and familiar stimuli (after 24 h) was enhanced by reward anticipation in both tasks. These findings are compatible with the proposition that novelty can act as a bonus for rewards under conditions when rewards are explicitly attended, thus biasing the organism towards reward anticipation and providing a motivational signal for exploration.     

In vivo microdialysis studies of age-related alterations in potassium-evoked overflow of dopamine in the dorsal striatum of Fischer 344 rats

Intracerebral microdialysis was used to measure basal levels and potassium (K(+))-stimulated overflow of dopamine (DA), homovanillic acid (HVA) and dihydroxyphenylacetic acid (DOPAC), in the dorsal striatum of young (6 months) and aged (24 months) Fischer 344 (F344) rats. Basal levels of HVA were lower in aged rats whereas basal DA and DOPAC did not differ significantly between the two groups. The administration of three low to moderate doses of K(+) (10, 25, and 50 mM) through the microdialysis probe for one collection period revealed differences between the two age groups of F344 rats. DA overflow increased in a dose-dependent manner in the young but not aged rats. Extracellular levels of DOPAC and HVA decreased during the K(+) stimulation and there was a significant difference in the changes in HVA produced by K(+) stimulation in the young vs aged animals. These data support the hypothesis that low to moderate doses of K(+) may be necessary to demonstrate age-related differences in K(+)-evoked DA overflow, since previous microdialysis studies using higher doses have not reported age-related differences in DA overflow.     

风险框架情境下决策理性的老化研究

研究背景由于框架的影响,人们在决策时会表现出一些非理性选择和决策,但随着年龄的增长是否也存在老龄化效应,值得关注。方法以232例年轻人(青年组)和120例老年人(老年组)为被试进行2×2×2完全被试间试验。其中自变量包括决策情境的个体相关性(高相关性、低相关性)、框架价态(正框架、负框架)和年龄段(青年、老年);控制变量为风险概率,作为被试内变量(低概率为33%、高概率为40%),因变量为决策分数。观察风险框架决策情境下之风险决策理性,以及风险寻求倾向的增龄性变化。结果在风险决策框架情境下,老年组被试的非理性决策趋势更强(框架效应),正框架下决策评分为5.13±2.12、负框架下决策评分为6.55±1.05[F(1,118)=21.470,P=0.000;η2=0.156];青年组正框架下决策评分为3.18±2.49、负框架下决策评分为5.00±2.41[F(1,230)=31.260,P=0.000;η2=0.121]。老年组被试更倾向风险选择[F(1,350)=4.820,P=0.029]。结论情境框架发展过程中风险寻求趋势的增加及理性的轻微退行性改变,可以反映老年人眶额叶皮质及腹内侧前额叶皮质、杏仁核功能的老龄化,以及腹内侧前额叶皮质功能的良好维持。此外,老年人对风险概率的理解并未出现退行性改变,但可预示老年人对期望效应值的敏感性下降。     

Testosterone levels correspond with increased ventral striatum activation in response to monetary rewards in adolescents

Risk taking is an integral part of learning and development, particularly during adolescence the prevalence of risky behaviors peak. It is hypothesized that the tendency to take risks is related to pubertal maturation, where there is interplay between gonadal hormones, the neural mechanisms that underlie affective (e.g., reward) processing, and risky behavior. To test this hypothesis, fifty healthy adolescents (aged 10–16 years; 33 girls, 17 boys) at different stages of puberty performed a gambling task while lying in the MRI scanner, and provided saliva samples for hormone assessment. Gonadal hormone levels were correlated with the neural response to receiving a monetary reward. Results showed that testosterone level correlated positively with activation in the striatum for both boys and girls, suggesting that individual differences in hormones at puberty are related to the way adolescents respond to reward, which can ultimately affect risk-taking behavior.     

DRD2/ANKK1 TaqI A polymorphism affects corticostriatal activity in response to negative affective facial stimuli

DRD2/ANKK1 TaqI A polymorphism has been suggested to be involved in a reward-related psychiatric disorders. However, the effect of Dopamine receptor D2 (DRD2) on emotional processing has not been investigated yet. We investigated the possible relationship between DRD2/ANKK1 TaqI A polymorphism and corticostriatal response to negative facial stimuli using functional magnetic resonance imaging. All participants were genotyped with regard to the DRD2/ANKK1 TaqI A polymorphism. Our results suggest an association between the DRD2/ANKK1 TaqI A polymorphism and activations in the putamen, the anterior cingulate cortex, and amygdala in response to negative facial stimuli. Furthermore, molecular heterosis at the TaqI polymorphism of DRD2/ANKK1 may play an important role in affective regulation by corticostriatal pathway.     

New developments in brain research of internet and gaming disorder

There is evidence that the neural mechanisms underlying Internet Gaming Disorder (IGD) resemble those of drug addiction. Functional Magnetic Resonance Imaging (fMRI) studies of the resting state and measures of gray matter volume have shown that Internet game playing was associated with changes to brain regions responsible for attention and control, impulse control, motor function, emotional regulation, sensory-motor coordination. Furthermore, Internet game playing was associated with lower white matter density in brain regions that are involved in decision-making, behavioral inhibition and emotional regulation. Videogame playing involved changes in reward inhibitory mechanisms and loss of control. Structural brain imaging studies showed alterations in the volume of the ventral striatum that is an important part of the brain's reward mechanisms. Finally, videogame playing was associated with dopamine release similar in magnitude to those of drugs of abuse and lower dopamine transporter and dopamine receptor D₂ occupancy indicating sub-sensitivity of dopamine reward mechanisms.     

Circuit-Based Approaches to Understanding Corticostriatothalamic Dysfunction Across the Psychosis Continuum

Dopamine is known to play a role in the pathogenesis of psychotic symptoms, but the mechanisms driving dopaminergic dysfunction in psychosis remain unclear. Considerable attention has focused on the role of corticostriatothalamic (CST) circuits, given that they regulate and are modulated by the activity of dopaminergic cells in the midbrain. Preclinical studies have proposed multiple models of CST dysfunction in psychosis, each prioritizing different brain regions and pathophysiological mechanisms. A particular challenge is that CST circuits have undergone considerable evolutionary modification across mammals, complicating comparisons across species. Here, we consider preclinical models of CST dysfunction in psychosis and evaluate the degree to which they are supported by evidence from human resting-state functional magnetic resonance imaging studies conducted across the psychosis continuum, ranging from subclinical schizotypy to established schizophrenia. In partial support of some preclinical models, human studies indicate that dorsal CST and hippocampal-striatal functional dysconnectivity are apparent across the psychosis spectrum and may represent a vulnerability marker for psychosis. In contrast, midbrain dysfunction may emerge when symptoms warrant clinical assistance and may thus be a trigger for illness onset. The major difference between clinical and preclinical findings is the strong involvement of the dorsal CST in the former, consistent with an increasing prominence of this circuitry in the primate brain. We close by underscoring the need for high-resolution characterization of phenotypic heterogeneity in psychosis to develop a refined understanding of how the dysfunction of specific circuit elements gives rise to distinct symptom profiles.     

Identification of neuronal programmed cell death in situ in the striatum of normal adult rat brain and its relationship to neuronal death during aging

Apoptotic neurons have been identified in normal adult rat striatum by terminal deoxynucleotidyl transferase mediated dUTP-biotin nick end labeling technique. This observation suggests that neuronal programmed cell death starts at an early stage of adult life and may contribute to the aging associated neuronal loss. In addition, the frequency of apoptotic cells was found to significantly increase in old rats, which implies that aging itself accelerates the process.