Learning-related changes in reward expectancy are reflected in the feedback-related negativity

The feedback-related negativity (FRN) has been hypothesized to be linked to reward-based learning. While many studies have shown that the FRN only occurs in response to unexpected negative outcomes, the relationship between the magnitude of negative prediction errors and FRN amplitude remains a matter of debate. The present study aimed to elucidate this relationship with a new behavioural procedure that allowed subjects to predict precise reward probabilities by learning an explicit rule. Insight into the rule did not only influence subjects' choice behaviour, but also outcome-related event-related potentials. After subjects had learned the rule, the FRN amplitude difference between non-reward and reward mirrored the magnitude of the negative prediction error, i.e. it was larger for less likely negative outcomes. Source analysis linked this effect to the anterior cingulate cortex. P300 amplitude was also modulated by outcome valence and expectancy. It was larger for positive and unexpected outcomes. It remains to be clarified, however, whether the P300 reflects a positive prediction error.     

It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations

The anterior cingulate cortex (ACC) is involved in performance monitoring and in learning from performance feedback. Recent research suggests that the feedback-related negativity (FRN), an event-related potentials (ERP) component reflecting neural activity in the ACC, codes the size of a negative prediction error when reward probabilities are varied. There is as yet no clear evidence that the FRN is also sensitive to violations of reward magnitude expectations. In the present study, 20 healthy young subjects engaged in a learning task in which a coin had to be found on each trial. The value of the coin (the potential reward magnitude) was varied from trial to trial and amounted to 5 cent, 20 cent or 50 cent. Analysis of ERPs revealed that FRN amplitude differences between reward and non-reward were significantly modulated by (potential) reward magnitude. This effect was driven by the neural response to non-reward: the larger the potential reward, the larger was the FRN amplitude in response to non-reward. In contrast, the P300 was larger for positive outcomes and showed an effect of (potential) reward magnitude independent of valence. Together with evidence from previous studies, these results show that the FRN codes negative prediction errors in the context of varying reward probabilities and magnitudes. The findings are in line with recent results based on functional neuroimaging and lend further support to the idea of a key role of the ACC in the integration of information on different aspects of performance outcomes.     

Reward context sensitivity impairment following severe TBI: an event-related potential investigation.

Many rehabilitation protocols following traumatic brain injury (TBI) utilize reinforcement and reward to influence behavior and facilitate recovery; however, previous studies suggest survivors of severe TBI demonstrate impairments in contingency utilization and sensitivity. The precise neurobiological mechanisms underlying these deficits have not been thoroughly explored, but can be examined using the "feedback-related negativity" (FRN)--an event-related potential (ERP) component evoked following performance or response feedback (e.g., whether a monetary reward is obtained) with a larger FRN following unfavorable than favorable outcomes--particularly when unfavorable feedback occurs in the context of high reward probability. We examined ERPs elicited by favorable (monetary gain: "reward") and unfavorable (no monetary gain: "non-reward") feedback during a guessing task where probability of reward outcome was manipulated in survivors of severe TBI and demographically matched healthy participants. Consistent with previous findings, controls showed larger amplitude FRN to non-reward feedback and the largest amplitude FRN following a non-reward when reward probability context was greatest. In contrast, FRN in TBI participants did not significantly differentiate non-reward from reward trials and their FRN was largest to reward trials in the low reward probability context. Findings implicate an electrophysiological marker of impaired reward context sensitivity following severe TBI.     

Inter-trial coherence of medial frontal theta oscillations linked to differential feedback processing in youth and young adults with autism

BackgroundImpairment in prediction and appreciation for choice outcomes could contribute to several core symptoms of ASD. We examined electroencephalography (EEG) oscillations in 27 youth and young adults diagnosed with autism spectrum disorder (ASD) and 22 IQ-matched neurotypical controls while they performed a chance-based reward prediction task.MethodWe re-analyzed our previously published ERP data (Larson et al., 2011) and examined theta band oscillations (4–8 Hz) at frontal midline sites, within a timing window that overlaps with the feedback-related negativity (FRN). We focused on event-related changes after presentation of feedback for reward (WIN) and punitive (LOSE) outcomes, both for spectral power and inter-trial phase coherence.ResultsIn our reward prediction task, for both groups, medial frontal theta power and phase coherence were greater following LOSE compared to WIN feedback. However, compared to controls, inter-trial coherence of medial frontal theta was significantly lower overall (across both feedback types) for individuals with ASD. Our results indicate that while individuals with ASD are sensitive to the valence of reward feedback, comparable to their neurotypical peers, they have reduced synchronization of medial frontal theta activity during feedback processing.ConclusionsThis finding is consistent with previous studies showing neural variability in ASD and suggest that the processes underlying decision-making and reinforcement learning may be atypical and less efficient in ASD.     

Individual differences in reward–prediction–error: extraversion and feedback-related negativity

Medial frontal scalp-recorded negativity occurring ∼200–300 ms post-stimulus [known as feedback-related negativity (FRN)] is attenuated following unpredicted reward and potentiated following unpredicted non-reward. This encourages the view that FRN may partly reflect dopaminergic ‘reward–prediction–error’ signalling. We examined the influence of a putatively dopamine-based personality trait, extraversion (N = 30), and a dopamine-related gene polymorphism, DRD2/ANKK1 (N = 24), on FRN during an associative reward-learning paradigm. FRN was most negative following unpredicted non-reward and least-negative following unpredicted reward. A difference wave contrasting these conditions was significantly more pronounced for extraverted participants than for introverts, with a similar but non-significant trend for participants carrying at least one copy of the A1 allele of the DRD2/ANKK1 gene compared with those without the allele. Extraversion was also significantly higher in A1 allele carriers. Results have broad relevance to neuroscience and personality research concerning reward processing and dopamine function.     

Knowing good from bad: differential activation of human cortical areas by positive and negative outcomes

Previous research has identified a component of the event-related brain potential (ERP), the feedback-related negativity, that is elicited by feedback stimuli associated with unfavourable outcomes. In the present research we used event-related functional magnetic resonance imaging (fMRI) and electroencephalographic (EEG) recordings to test the common hypothesis that this component is generated in the caudal anterior cingulate cortex. The EEG results indicated that our paradigm, a time estimation task with trial-to-trial performance feedback, elicited a large feedback-related negativity (FRN). Nevertheless, the fMRI results did not reveal any area in the caudal anterior cingulate cortex that was differentially activated by positive and negative performance feedback, casting doubt on the notion that the FRN is generated in this brain region. In contrast, we found a number of brain areas outside the posterior medial frontal cortex that were activated more strongly by positive feedback than by negative feedback. These included areas in the rostral anterior cingulate cortex, posterior cingulate cortex, right superior frontal gyrus, and striatum. An anatomically constrained source model assuming equivalent dipole generators in the rostral anterior cingulate, posterior cingulate, and right superior frontal gyrus produced a simulated scalp distribution that corresponded closely to the observed scalp distribution of the FRN. These results support a new hypothesis regarding the neural generators of the FRN, and have important implications for the use of this component as an electrophysiological index of performance monitoring and reward processing.     

Mediofrontal negativities in the absence of responding

The feedback-related negativity (FRN) is an event-related brain potential component that is elicited by feedback stimuli indicating unfavorable outcomes. Until recently, the FRN has been studied primarily using experimental paradigms in which outcomes appeared to be contingent upon the participants' behavior. The present study further addressed the question whether an FRN can be elicited by outcomes that are not contingent on any preceding choice or action. Participants took part in a simple slot-machine task in which they experienced monetary gains and losses in the absence of responses. In addition, they performed a time estimation task often used to study the FRN and a flanker task known to elicit the error-related negativity. Outcomes in the slot-machine task elicited an FRN-like mediofrontal negativity whose amplitude correlated with the amplitude of the FRN associated with negative feedback in the time estimation task. However, the mediofrontal negativity was observed both for (unfavorable) outcomes that averted a gain and for (favorable) outcomes that averted a loss of money. The results are discussed in the framework of current conceptions of the FRN and related electrophysiological components.     

Detecting perceptual conflict by the feedback-related negativity in brain potentials

The feedback-related negativity (FRN) in brain potentials is typically observed for the outcome evaluation concerning one's performance or monetary reward. Using a task in which the participant guesses whether the first stimulus (S1) would have the same color as the subsequently presented second stimulus (S2), this study demonstrates that the FRN to S2, which serves as feedback to the guessing, is also sensitive to the conflict between perceptual representations of S1 and S2 in working memory. The FRN effect for the feedback concerning the correctness of one's performance is modulated by the congruency between perceptual properties of the stimuli. The anterior cingulate cortex, which generates the FRN, is thus a general conflict-monitoring device detecting both response and perceptual conflicts.     

Self-relevant criteria determine the evaluation of outcomes induced by others

The evaluation of whether an outcome is good or bad is reflected in feedback-related negativity (FRN), which is an event-related brain potential (ERP) component that is elicited by bad events. It is still, however, unclear how the FRN reflects the evaluation of outcomes induced by others. We focused on the relationship between self and other, and recorded ERPs during an alternative gambling task performed by two players (participant and virtual player). FRN was elicited not only by the losses of another in a cooperative situation but also by the gains of another in an antagonistic situation. These results indicate that FRN reflects the evaluation on the basis of one's own evaluative criteria, regardless of the monetary outcome for another.     

Errors in reward prediction are reflected in the event-related brain potential

The error-related negativity (ERN) is a negative deflection in the event-related brain potential associated with error processing. A recent theory holds that the ERN is elicited by the impact of a reward prediction error signal carried by the mesencephalic dopamine system on anterior cingulate cortex. The theory predicts that larger ERNs should be elicited by unexpected unfavorable outcomes than by expected unfavorable outcomes. We tested the theory in an experiment in which the frequency of occurrence of reward was varied by condition, reasoning that the system that produces the ERN would come to expect non-reward when rewards were infrequent. Consistent with the theory, we found that larger ERNs were elicited by unexpected absences of reward.     

The near-miss effect in slot-machine gambling: modulation of feedback-related negativity by subjective value

Near-miss plays an important role in the development of gambling addictions. In this study, we measured the neural correlates of the process by which near-miss outcomes are evaluated in simplified, static, slot-machine gambling using event-related potentials. Analysis of event-related potentials revealed that the size of FRN (feedback-related negativity) for a near miss is between the full miss and the win. These results suggest that participants distinguish among near misses, full misses, and wins during the early evaluation phase. The subjective value and objective value of outcome were assessed separately to discuss FRN on outcome evaluation. It is suggested that FRN is mediated not only by the objective value of outcomes but also by the subjective value of feedback.     

Context-based interpersonal relationship modulates social comparison between outcomes: an event-related potential study

Social comparison is a common behavior that largely determines people’s experience of decision outcome. Previous research has showed that interpersonal relationship plays a pivotal role in social comparison. In the current study, we investigated whether the manipulation of context-based relationship would affect participants’ comparison of self-outcome and other-outcome. Participants first finished a trust game with likeable (dislikeable) partner and then they were involved in a gambling task and observed the outcomes for themselves and for partners. According to self-reports, participants were more satisfied with likeable partner’s gains than losses only when they received gains, but they were always more satisfied with dislikeable player’s losses compared to gains. Event-related potentials including the feedback-related negativity (FRN), P3 and late positive component (LPC) were sensitive to context-based relationship. Specifically, the prediction error signal (indexed by the FRN) was largest when participants received losses but dislikeable player received gains. Meanwhile, the P3 indicates that participants had stronger motivation to outperform dislikeable player. Finally, the LPC was larger when participants received the same outcomes with dislikeable players. In general, our results support the key point of the self-evaluation maintenance model that personal closeness modulates subjective sensitivity when drawing a comparison of one’s outcomes with other’s outcomes.     

To do or not to do? Action enlarges the FRN and P300 effects in outcome evaluation

Behavioral studies demonstrate that the outcome following an individual's action evokes stronger emotional responses than the same outcome following inaction. Here we use the event-related potential (ERP) technique to investigate how action affects the brain activity in outcome evaluation. In a gambling task, participants were asked to select a box from three boxes containing monetary reward and then to decide whether they would change their initial choice (i.e., action) or not (i.e., inaction). The feedback-related negativity (FRN), an evoked potential that peaks approximately 250 ms after receipt of feedback information, showed a larger differential effect between loss and win following action than following inaction. Similarly, the P300 showed a larger differential effect following action than following inaction, but now with the responses more positive to the win feedback than to the loss feedback. These results suggest that action may increase the expectancy towards the outcome and/or the motivational/emotional significance of the outcome, and that this action effect can be found in both the FRN and the P300 electrophysiological responses.     

Action‐Monitoring Alterations as Indicators of Predictive Deficits in Schizophrenia

A flexible and dynamically adjustable behavior is crucial to adapt to a continuously changing environment. In order to optimally adapt, we need to learn from the consequences of our behavior. We usually learn through different kinds of prediction errors, which occur when we experience unexpected situations due to false predictions. With this literature review, we intended to contribute to current etiological models that ascribe various positive symptoms (particularly delusions and hallucinations) in patients with schizophrenia to false prediction errors and deficient predictive learning. We discuss alterations in the electrophysiological measure of the error‐related negativity/error negativity (ERN/Ne) as a global deficit and a trait in schizophrenia, as they have been observed in different samples of patients with schizophrenia, in individuals at high‐risk and individuals with subclinical schizotypal traits. As the ERN/Ne can itself be considered the result of predictive processes (evaluation of current action outcomes as worse than expected), we propose that the reported alterations indicate that patients suffering from schizophrenic illnesses fail to adequately classify the outcomes of their actions as better or worse than expected due to a deficit in self‐monitoring. Furthermore, we discuss results in further action‐monitoring components, such as the correct response negativity (CRN)—a smaller negativity elicited by correct responses; and error positivity (Pe)—a later positivity assumed to reflect conscious error processing. The reported results show normal Pe amplitudes and normal post‐error adjustments (adaptations after committed error to improve performance), indicating an intact later and conscious processing. From the results of diminished differences between ERN/Ne and CRN amplitudes, we conclude a general predictive deficit in early aspects of self‐monitoring associated with positive symptoms in patients with schizophrenia.     

Telling good from bad news: ADHD differentially affects processing of positive and negative feedback during guessing

Neuroimaging studies on ADHD suggest abnormalities in brain regions associated with decision-making and reward processing such as the anterior cingulate cortex (ACC) and orbitofrontal cortex. Recently, event-related potential (ERP) studies demonstrated that the ACC is involved in processing feedback signals during guessing and gambling. The resulting negative deflection, the 'feedback-related negativity' (FRN) has been interpreted as reflecting an error in reward prediction. In the present study, ERPs elicited by positive and negative feedback were recorded in children with ADHD and normal controls during guessing. 'Correct' and 'incorrect' guesses resulted in respectively monetary gains and losses. The FRN amplitude to losses was more pronounced in the ADHD group than in normal controls. Positive and negative feedback differentially affected long latency components in the ERP waveforms of normal controls, but not ADHD children. These later deflections might be related to further emotional or strategic processing. The present findings suggest an enhanced sensitivity to unfavourable outcomes in children with ADHD, probably due to abnormalities in mesolimbic reward circuits. In addition, further processing, such as affective evaluation and the assessment of future consequences of the feedback signal seems to be altered in ADHD. These results may further help understanding the neural basis of decision-making deficits in ADHD.     

Non-sympathetic FRN responses to drops in others’ stocks

Previous neuroeconomic studies have observed that people display sympathetic neural responses toward others’ misfortunes. We argue that the reverse emotions, such as gloating or schadenfreude, may also emerge in certain circumstances. To examine this theory, we recorded feedback-related negativity (FRN) toward others’ large or small gains or losses in a stock market context. We adopted the framework of social distance, according to which we hypothesized that because others in the stock market are far away, unidentified, and indistinct, people would show less sympathy or even schadenfreude toward others’ large losses. The results indicated that FRN at Fz was significantly less negative when observing larger decreases in others’ stock, indicating that others’ large losses are not unexpected negative events in the stock market and suggesting the existence of schadenfreude. Our research contributes to the understanding of social neurofinance by demonstrating the schadenfreude effect in relation to the stock market. This study also provides new information regarding the relationship between FRN and the social emotions that form the expectations of gain and loss.     

Baseline EEG theta/beta ratio and punishment sensitivity as biomarkers for feedback-related negativity (FRN) and risk-taking

Objective

Feedback-related negativity (FRN) is associated with reinforcement learning and punishment sensitivity. Furthermore, reinforcement learning proficiency can be predicted from pre-task baseline EEG theta/beta ratio. In this study it was examined whether there was a relation between baseline theta/beta ratio in rest and FRN amplitude during a gambling task, and if such a correlation would be related to theta activity or to beta activity.

Methods

Baseline EEG and a self-report measure of punishment sensitivity (BIS) were obtained from 52 healthy volunteers. FRN was recorded during a gambling task.

Results

FRN amplitude was negatively correlated with theta/beta ratio in high BIS individuals. Furthermore, source localization indicated that baseline theta activity generated in the anterior cingulate cortex (ACC) accounted for this correlation. For low BIS individuals no correlation was found.

Conclusions

An association between high baseline theta/beta ratio with low amplitude FRN and high risk-taking can be found in individuals who score sufficiently high on the BIS scale. This relationship is carried mostly by baseline theta activity, but not by beta activity.

Significance

This link between baseline brain activity, self-report measures and feedback processing may contribute to further understanding the biological basis of conditions that are accompanied by abnormal theta/beta ratio and reward processing, such as attention deficit hyper activity disorder (ADHD).     

Whose loss is it? Human electrophysiological correlates of non-self reward processing

Abstract

To recognize whether another person's action results in a good or bad outcome is imperative for social learning, as well as for understanding the behavior of others in a broad context. Recent studies have reported that a scalp-surface event-related potential (ERP) called medial-frontal negativity (MFN), considered to be an index of negative reward processing, is generated when perceiving not only one's own losses, but also those of others. This suggests that the same neural mechanisms operate in monitoring one's own actions and in perceiving the consequences of the actions of others. To further elucidate the properties of this “observational” MFN, this study examined whether its amplitude differs with different observational targets. In a gambling task, participants observed the performances of non-self agents: a human friend and PC programs. The outcomes of the decisions of these agents were not associated with the participants’ own benefits. ERP results showed that the MFN-like pattern was significantly elicited only when observing the outcomes of decisions made by human agents. Furthermore, self-reported measures of empathy were positively associated with the magnitude of the observational MFN. These findings suggest that the neural activity in non-self reward processing reflects a socioemotional state generated by the target of observation, as well as an empathetic trait of the individual.     

Feedback and reward processing in high-functioning autism

Individuals with high-functioning autism often display deficits in social interactions and high-level cognitive functions. Such deficits may be influenced by poor ability to process feedback and rewards. The feedback-related negativity (FRN) is an event-related potential (ERP) that is more negative following losses than gains. We examined FRN amplitude in 25 individuals with Autism Spectrum Disorder (ASD) and 25 age- and IQ-matched typically developing control participants who completed a guessing task with monetary loss/gain feedback. Both groups demonstrated a robust FRN that was more negative to loss trials than gain trials; however, groups did not differ in FRN amplitude as a function of gain or loss trials. N1 and P300 amplitudes did not differentiate groups. FRN amplitude was positively correlated with age in individuals with ASD, but not measures of intelligence, anxiety, behavioral inhibition, or autism severity. Given previous findings of reduced-amplitude error-related negativity (ERN) in ASD, we propose that individuals with ASD may process external, concrete, feedback similar to typically developing individuals, but have difficulty with internal, more abstract, regulation of performance.     

Asymmetrical effects of frequent gains and frequent losses in a gambling task

Feedback negativity (FN) is elicited when the outcome is worse than expected. Here we report that changes in outcome probabilities affect the reward prediction level reflected in the FN. We used a slot machine-like gambling task in which a button press produced one of the six outcomes: -10, -5, -1, +1, +5, and +10. The outcomes were first presented with equal probability, and subsequently, either +10 or -10 was presented more frequently. The smaller gains elicited an FN when the largest gain occurred frequently, whereas the smaller losses continued eliciting an FN even when the largest loss occurred frequently. The reward prediction level appears to be easy to raise but difficult to diminish. This disposition helps overcome a suboptimal situation.