Good things come to those who wait: attenuated discounting of delayed rewards in aged Fischer 344 rats

The ability to make advantageous choices among outcomes that differ in magnitude, probability, and delay until their arrival is critical for optimal survival and well-being across the lifespan. Aged individuals are often characterized as less impulsive in their choices than their young adult counterparts, demonstrating an increased ability to forgo immediate in favor of delayed (and often more beneficial) rewards. Such "wisdom" is usually characterized as a consequence of learning and life experience. However, aging is also associated with prefrontal cortical dysfunction and concomitant impairments in advantageous choice behavior. Animal models afford the opportunity to isolate the effects of biological aging on decision-making from experiential factors. To model one critical component of decision-making, young adult and aged Fischer 344 rats were trained on a two-choice delay discounting task in which one choice provided immediate delivery of a small reward and the other provided a large reward delivered after a variable delay period. Whereas young adult rats showed a characteristic pattern of choice behavior (choosing the large reward at short delays and shifting preference to the small reward as delays increased), aged rats maintained a preference for the large reward at all delays (i.e., attenuated "discounting" of delayed rewards). This increased preference for the large reward in aged rats was not due to perceptual, motor, or motivational factors. The data strongly suggest that, independent of life experience, there are underlying neurobiological factors that contribute to age-related changes in decision-making, and particularly the ability to delay gratification.     

Individual differences in offensive aggression in golden hamsters: A model of reactive and impulsive aggression?

In humans, reactive aggression is associated with impulsivity. The purpose of this study is to relate reactive and impulsive aggression in humans with offensive aggression in animals and identify neurobiological correlates associated with certain forms of the behavior. We predicted that individual differences in offensive aggression are associated with individual differences in impulsivity. Adult male hamsters were repeatedly tested for offensive responses and divided into High-Aggression or Low-Aggression groups. They were then trained and tested under a delay-discounting paradigm to assess impulsivity. High-Aggression animals consistently attacked and bit more frequently and faster, and showed highly repetitive behavior, indicated by repeated attacks per contact bout. In addition, these animals engaged in more fragmented and shorter contact bouts. During impulsivity testing, High-Aggression animals preferred immediate smaller rewards over delayed larger rewards. Furthermore, 5-HT and vasopressin (AVP) innervation was compared between the groups. High-Aggression animals showed decreased 5-HT varicosities in several key brain areas involved in aggressive and/or impulsive behavior and decreased AVP fibers in the anterior hypothalamus. Together, these data show a convergence of behavioral phenotypes through individual differences in offensive aggression and impulsivity. As such, this association provides support for an animal model of reactive and impulsive aggression. Furthermore, this behavioral convergence is supported by a concurrent reduction in 5-HT innervation of brain areas controlling aggression and impulsivity, providing a common neural mechanism for this phenotype.     

The neuroscience of impulsive and self-controlled decisions.

Impulsiveness and self-control are two antagonistic choice patterns. Whereas impulsive decisions can be exemplified by the preference for a small, immediate over a large, delayed reward, self-control can be characterised as the opposite preference order. This review focuses on current developments in investigating the neuroscience of impulsiveness and self-control, with particular emphasis on the neuroanatomy, psychopharmacology, and electrophysiology of this class of decision making. The role of the avian forebrain in representing and processing temporal reward discounting - a chief psychological mechanism responsible for producing impulsiveness - is especially highlighted. In addition to its role in impulsive decision making, the avian forebrain also appears to be involved in processing the key functions required for action- and self-control. In particular, recent electrophysiological studies indicate that single forebrain neurons reflect aspects of response omission strategy and the temporal scheduling of response withholding when execution of action needs to be controlled. In conclusion, the significant advances in this field of research may help to explain neuropathologies that are characterised by exaggerated impulsivity, or lack of self-control, as for instance attention deficit disorders, frontal lobe syndrome, drug addiction, or pathological gambling.     

Cocaine exposure causes long-term increases in impulsive choice

In this study, the authors examined the long-term effects of prior exposure to cocaine on a delay-discounting task commonly used to measure impulsive choice. Male Long-Evans rats received daily intraperitoneal injections of 30 mg/kg cocaine HCl or saline for 14 days. Following 3 weeks of withdrawal, rats began training. On each trial, rats were given a choice between 2 levers. A press on 1 lever resulted in immediate delivery of a single 45-mg food pellet, and a press on the other resulted in delivery of 4 pellets after a delay period. Impulsive choice was defined as preference for the small immediate over the large delayed reward. Three months after treatment, cocaine-exposed rats displayed increased impulsive choice behavior. They also showed less anticipatory responding (entries into the food trough) during the delays prior to reward delivery, indicating that the enhanced impulsive choice in these rats may be related to deficits in bridging the delay between response and reward. These data demonstrate that cocaine exposure can cause enduring increases in impulsive choice behavior, consistent with observations in human subjects with drug addictions.     

Inter-individual discount factor differences in reward prediction are topographically associated with caudate activation

In general, humans tend to devalue a delayed reward. Such delay discounting is a theoretical and computational concept in which the discount factor influences the time scale of the trade-off between delay of reward and amount of reward. The discount factor relies on the individual's ability to evaluate the future reward. Using functional magnetic resonance imaging, we investigated brain mechanisms for reward valuation at different individual discount factors in a delayed reward choice task. In the task, participants were required to select small/immediate or large/delayed rewards to maximize the total reward over time. The discount factor for each participant individually was calculated from the behavioral data based on an exponential discounting model. The estimated value of a future reward increases as the expected delivery approaches, so the time course of these estimated values was computed based on each individual's discount factor; each was entered into the regression analysis as an explanatory (independent) variable. After the region of interest was narrowed anatomically to the caudate, a peak coordinate was detected in each individual. A correlation analysis revealed that the location of the peak along the dorsal-ventral axis in the right caudate was positively correlated with the discount factor. This implies that individuals who showed a larger discount factor had peak activations in a more dorsal part of the right caudate associated with future reward prediction. This evidence also suggests that a higher ability to delay reward prediction might be related to activation of the more dorsal caudate.     

Behavioural and physiological correlates of impulsivity in the domestic dog (Canis familiaris)

Impulsivity is a trait related to inhibitory control which is expressed in a range of behaviours. Impulsive individuals show a decreased ability to tolerate delay of reinforcement, and more impulsive behaviour has been linked to decreased levels of serotonin and dopamine in a number of species. In domestic dogs, impulsivity is implicated in problem behaviours that result from a lack of self control, but currently there are no published studies that assess behavioural and physiological measures of impulsivity in relation to this trait. Impulsivity scores were calculated for 41 dogs using an owner-report assessment, the Dog Impulsivity Assessment Scale (DIAS). Twenty-three of these subjects completed an operant choice task based on a delayed reward paradigm, to assess their tolerance to delay of reinforcement. High Pressure Liquid Chromatography (HPLC) with Fluorometric Detection was used to detect levels of the metabolites of serotonin (5-HIAA) and dopamine (HVA) in the urine of 17 of the subjects. Higher impulsivity scores were found to be significantly correlated with more impulsive behaviour (reduced tolerance to delay of reinforcement) in the behaviour tests and lower levels of urinary 5-HIAA and 5-HIAA/HVA ratio. The results demonstrate convergent validity between impulsivity (as assessed by the DIAS) and behavioural and physiological parameters.     

Reward‐related neural dysfunction across depression and impulsivity: A dimensional approach

Recent theoretical models underline reward sensitivity as a potential endophenotype for major depressive disorder. Neural and behavioral evidence reveals depression is associated with reduced reward sensitivity. However, reward dysfunction is not unique to depression, as it is also common across disorders of poor impulse control. We examined the interrelationships of depression (Depression, Anxiety, and Stress Scale [DASS‐21]) and impulsivity (UPPS‐P Impulsive Behavior Scale) with reward sensitivity among a large, representative sample (N = 260). ERPs were recorded to isolate two neural indicators of consummatory reward processing: initial evaluation of rewards in the 250–350 ms time window postonset of feedback (reward positivity [RewP]), and salience to monetary outcomes (P3). Significant interactions were observed between depression and impulsivity facets across these two stages of reward processing: depression and positive urgency predicted RewP amplitude to reward outcomes (win vs. loss); depression and one other impulsivity trait, (lack of) premeditation, predicted P3 amplitude to monetary outcomes. Conversely, high symptoms of depression were related to three biobehavioral profiles: (1) blunted RewP in conjunction with high positive urgency, (2) combination of blunted RewP and low (lack of) premeditation, and (3) blunted P3 to monetary wins/losses, in conjunction with low (lack of) premeditation. Findings illustrate that reward‐related dysfunctions may be optimally conceptualized when examining the interactions between dimensions of internalizing and externalizing psychopathology.     

Reflective and non‐reflective antecedents of health‐related behaviour: Exploring the relative contributions of impulsivity and implicit self‐control to the prediction of dietary behaviour

Objectives. This study (N= 139) explored whether two measures that capture non‐reflective processing (viz. a self‐report measure of impulsivity and a behavioural measure of implicit self‐control) would contribute to the prediction of dietary behaviour over and above cognitive predictors specified by the theory of planned behaviour (TPB). Methods. Four dimensions of impulsivity were measured at Time 1. Implicit self‐control was measured at Time 2, alongside TPB predictors relating to the avoidance of high‐calorie snacks. At Time 3, participants reported their snacking behaviour over the previous 2 weeks. Results. Results revealed that both impulsivity and implicit self‐control significantly contributed to the prediction of snacking behaviour over and above the TPB predictors. Conclusions. It was concluded that the predictive utility of models such as the TPB might be augmented by the inclusion of variables that capture non‐reflective information processing.     

Preserved calibration of persistence based on delay‐timing distribution during sleep deprivation

We frequently encounter decisions where we have to determine whether to wait for a certain reward delayed for an uncertain duration or to move on. The appropriate decision depends upon the underlying temporal distribution of the delay. With some distributions it is best to be completely persistent, whereas in others it is more appropriate to abandon waiting after a certain period of time. The current study examined whether the ability to form temporal expectations and adjust persistence accordingly is compromised by sleep deprivation. Participants performed a willingness‐to‐wait task either in a well‐rested state or after a night of total sleep deprivation. Participants had to decide either to wait for a larger reward or to abandon waiting in favour of a smaller immediate reward. Delays were drawn from either a uniform distribution, where being persistent yields maximal returns, or from a heavy‐tailed distribution, where occasional long delays render full persistence suboptimal. In spite of increased sleepiness and decreased vigilance, sleep‐deprived participants were able to adjust waiting time appropriate to the experienced timing distribution. Additionally, sleep deprivation did not affect the foreperiod effect, indicating intact perception of conditional probability of temporal events and ability to adjust preparation accordingly.     

The role of trait impulsivity in response inhibition: Event-related potentials in a stop-signal task

The study examined the relation between self-reported impulsivity and inhibitory control in normal individuals. We compared stopping performance and neural correlates of stopping on stop-signal task between participants who scored in the top (n = 12) and bottom 25% (n = 12) on Impulsivity Scale from a sample of 305 male adults. Participants scoring high on impulsivity did not show impaired inhibitory control. However, it seems that the high impulsive tended to make more errors of commission and omission. Enhanced N1 amplitudes were found in successful than failed inhibition trials. The high impulsive group had smaller P3 amplitude than the low impulsive group. It appears that the high impulsive group may have a less efficient inhibitory control. Impulsivity Scale non-planning impulsiveness score and inattention score of Adult ADHD Self-Report Scale (ASRS) were negatively correlated with P3 amplitudes on successful inhibition trails, suggesting that impulsivity could have the potential influence on inhibitory control.     

Real and hypothetical rewards in probability discounting

In Experiment 1, undergraduates first made choices hypothetically between an uncertain large reward and a certain small reward, second they made choices with real money, and finally they again made hypothetical choices. The discounting rates in the first hypothetical reward condition were higher than those in the following conditions of real or hypothetical rewards. A significant difference in the discounting rate across conditions was not found when the real reward condition was replaced by the hypothetical reward condition (Experiment 2) or by the condition with probability contingency and hypothetical reward (Experiment 3). The difference in the discounting rate was replicated between the hypothetical and real reward conditions when the participants invested money when they selected an uncertain-large reward (Experiment 4). Johnson & Bickel (2002) found no difference in the discounting rate between hypothetical and real rewards when the real rewards were delivered to participants not during but after the experiment. The present experiments, by contrast, delivered real money immediately after every choice, and demonstrate that real and hypothetical rewards differ in the values of discounting rates.     

The effect of reward magnitude differences on choosing disadvantageous decks in the Iowa Gambling Task

In the Iowa Gambling Task, participants have to develop a long-term profitable monetary scenario in a situation of uncertainty and a conflict between the chance of encountering an immediate large reward (100 US dollars) in two long-term loosing decks (A and B; -250 US dollars per 10 cards) and the chance of encountering an immediate small reward (50 US dollars) in two long-term winning decks (C and D; +250 US dollars per 10 cards). The ratio of the immediate rewards in decks A/B and C/D is thus 2:1. Here, we manipulated these differences in reward magnitude between the advantageous (C/D) and disadvantageous (A/B) decks, while keeping the net gains and losses per 10 cards the same, to assess the impact of the conflict between immediate and distant pay-off on choice behaviour. Participants selected less cards from disadvantageous decks and won more money when the reward magnitude difference was decreased, A/B:C/D=1:1, while they selected more cards from disadvantageous decks and lost more money when reward magnitude differences were increased, A/B:C/D=4:1 and 6:1. This study shows that the outcome in the Iowa Gambling Task is sensitive to differences between the magnitude of immediate rewards in the advantageous and disadvantageous decks.     

Behavioral reactions reflecting differential reward expectations in monkeys

Learning theory emphasizes the importance of expectations in the control of instrumental action. This study investigated the variation of behavioral reactions toward different rewards as an expression of differential expectations of outcomes in primates. We employed several versions of two basic behavioral paradigms, the spatial delayed response task and the delayed reaction task. These tasks are commonly used in neurobiological studies of working memory, movement preparation, and event expectation involving the frontal cortex and basal ganglia. An initial visual instruction stimulus indicated to the animal which one of several food or liquid rewards would be delivered after each correct behavioral response, or whether or not a reward could be obtained. We measured the reaction times of the operantly conditioned arm movement necessary for obtaining the reward, and the durations of anticipatory licking prior to liquid reward delivery as a Pavlovian conditioned response. The results showed that both measures varied depending on the reward predicted by the initial instruction. Arm movements were performed with significantly shorter reaction times for foods or liquids that were more preferred by the animal than for less preferred ones. Still larger differences were observed between rewarded and unrewarded trials. An interesting effect was found in unrewarded trials, in which reaction times were significantly shorter when a highly preferred reward was delivered in the alternative rewarded trials of the same trial block as compared to a less preferred reward. Anticipatory licks preceding the reward were significantly longer when highly preferred rather than less preferred rewards, or no rewards, were predicted. These results demonstrate that behavioral reactions preceding rewards may vary depending on the predicted future reward and suggest that monkeys differentially expect particular outcomes in the presently investigated tasks.     

Individual differences in decisiveness: ERP correlates of response inhibition and error monitoring

The aim of the current study was to examine whether and how self‐reported decisiveness is associated with response inhibition and performance monitoring. We hypothesized that these two cognitive control mechanisms, both of which are often associated with decision making, would differ in individuals varying in decisiveness. We focused on ERP correlates and behavioral measures of inhibition and error processing in the stop‐signal task. We expected a negative relationship between decisiveness and behavioral measures of inhibitory control. We also hypothesized that stop‐signal‐locked N1 and P3 components and response‐locked error‐related negativity (ERN) would be less pronounced when participants self‐reported higher levels of decisiveness. Correlation analysis identified an association between high decisiveness, long stop‐signal reaction time, and low inhibition rate. Analysis with mixed‐effects linear models revealed that stop signals evoked less pronounced N1 and P3 in individuals scoring higher on decisiveness in both successfully and unsuccessfully inhibited trials. Additionally, high decisiveness was linked to reduced error monitoring, as indicated by decreased ERNs. Importantly, we also found positive association between P3 onset latency and decisiveness, suggesting that individuals scoring higher on this measure have relatively less ability to rapidly engage the stopping process. Thus, our findings primarily indicate that decisiveness is negatively associated with the efficiency of both response inhibition and error monitoring. They also suggest that highly decisive people may share some characteristics of diminished cognitive control with impulsive individuals.     

Interaction Between DRD2 C957T Polymorphism and An Acute Psychosocial Stressor on Reward-Related Behavioral Impulsivity

The dopamine D2 receptor (DRD2) C957T polymorphism CC genotype is associated with decreased striatal binding of DRD2 and executive function and working memory impairments in healthy adults. We investigated the relationships between C957T and acute stress with behavioral phenotypes of impulsivity in 72 young adults randomly allocated to either an acute psychosocial stress or relaxation induction condition. Homozygotes for 957C showed increased reward responsiveness after stress induction. They were also quicker when making immediate choices on the delay discounting task when stressed, compared with homozygotes who were not stressed. No effects were found for response inhibition, a dimension of impulsivity not related to extrinsic rewards. These data suggest that C957T is associated with a reward-related impulsivity endophenotype in response to acute psychosocial stress. Future studies should examine whether the greater sensitivity of 957C homozygotes to the effects of stress is mediated through dopamine release. Edited by Deborah Finkel.     

Spontaneous implementation intentions and impulsivity: Can impulsivity moderate the effectiveness of planning strategies?

Objective. Self‐initiated plans relating to when, where, and how behaviour will be performed have been shown to be effective in promoting goal progress. The current study (N =256) explored whether any impact of self‐initiated implementation intentions on the avoidance of snacking was moderated by impulsivity. Design and methods. The study employed a prospective design. At Time 1, participants reported the extent to which they had formed self‐initiated implementation intentions to avoid eating high‐calorie snacks. At Time 2, participants reported their snack consumption over the preceding 2 weeks and completed a measure of impulsivity. Results. Impulsivity (urgency) moderated the effect of self‐initiated implementation intentions on snack consumption. Specifically, self‐initiated implementation intentions benefited the avoidance of snacking most for those low in impulsivity and least for those high in impulsivity. Conclusion. The findings suggest that impulsivity might form an important boundary condition to the effectiveness of self‐initiated implementation intention formation in relation to snacking.     

Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops

Evaluation of both immediate and future outcomes of one's actions is a critical requirement for intelligent behavior. Using functional magnetic resonance imaging (fMRI), we investigated brain mechanisms for reward prediction at different time scales in a Markov decision task. When human subjects learned actions on the basis of immediate rewards, significant activity was seen in the lateral orbitofrontal cortex and the striatum. When subjects learned to act in order to obtain large future rewards while incurring small immediate losses, the dorsolateral prefrontal cortex, inferior parietal cortex, dorsal raphe nucleus and cerebellum were also activated. Computational model-based regression analysis using the predicted future rewards and prediction errors estimated from subjects' performance data revealed graded maps of time scale within the insula and the striatum: ventroanterior regions were involved in predicting immediate rewards and dorsoposterior regions were involved in predicting future rewards. These results suggest differential involvement of the cortico-basal ganglia loops in reward prediction at different time scales.     

Dissociable effects of reward magnitude on fronto-medial theta and FRN during performance monitoring

Reward processing is influenced by reward magnitude, as previous EEG studies showed changes in amplitude of the feedback-related negativity (FRN) and reward positivity (RewP), or power of fronto-medial theta (FMθ). However, it remains unclear whether these changes are driven by increased reward sensitivity, altered reward predictions, enhanced cognitive control, or a combination of these effects. To address this question, we asked 36 participants to perform a simple gambling task where feedback valence (reward vs. no-reward), its magnitude (small vs. large reward), and expectancy (expected vs. unexpected) were manipulated in a factorial design, while 64-channel EEG was recorded concurrently. We performed standard ERP analyses (FRN and RewP) as well as time-frequency decompositions (FMθ) of feedback-locked EEG data. Subjective reports showed that large rewards were more liked and expected than small ones. At the EEG level, increasing magnitude led to a larger RewP irrespective of expectancy, whereas the FRN was not influenced by this manipulation. In comparison, FMθ power was overall increased when reward magnitude was large, except if it was unexpected. These results show dissociable effects of reward magnitude on the RewP and FMθ power. Further, they suggest, that although large reward magnitude boosts reward processing (RewP), it can nonetheless undermine the need for enhanced cognitive control (FMθ) in case reward is unexpected. We discuss these new results in terms of optimistic bias or positive mood resulting from an increased reward magnitude.     

The short (S) allele of the serotonin transporter polymorphism and acute tryptophan depletion both increase impulsivity in men

Reduced serotonergic neurotransmission is implicated in impulsive behavior. We studied the triallelic system of the serotonin transporter gene linked polymorphic region (5-HTTLPR) and acute manipulation of serotonin together to further delineate the mechanisms by which serotonergic neurotransmission affects impulsivity. Fifty-two healthy participants (38 men and 14 women) underwent acute tryptophan depletion (ATD) or placebo in a randomized, double-blind, parallel group experiment. Impulsive response style was measured on two versions of the Continuous Performance Task (CPT), and calculated using signal detection theory. We observed a dose-dependent effect for the short (S′) allele of the 5-HTTLPR on impulsive response style. Individuals who had the S′/S′ genotype were more impulsive than individuals with the L/S′ genotype. Participants with the L/S′ genotype were more impulsive than those with the L/L genotype. ATD increased impulsivity in men, and decreased impulsivity in women. These data demonstrate for the first time that reduced serotonergic tone as a result of either 5-HTTLPR genotype, or experimental ATD, are both independently and additively, associated with elevated impulsive response style in Caucasian men.