The authors present a unified account of 2 neural systems concerned with the development and expression of adaptive behaviors: a mesencephalic dopamine system for reinforcement learning and a "generic" error-processing system associated with the anterior cingulate cortex. The existence of the error-processing system has been inferred from the error-related negativity (ERN), a component of the event-related brain potential elicited when human participants commit errors in reaction-time tasks. The authors propose that the ERN is generated when a negative reinforcement learning signal is conveyed to the anterior cingulate cortex via the mesencephalic dopamine system and that this signal is used by the anterior cingulate cortex to modify performance on the task at hand. They provide support for this proposal using both computational modeling and psychophysiological experimentation.Human beings learn from the consequences of their actions. Thorndike (1911Thorndike ( /1970 originally described this phenomenon with his law of effect, which made explicit the commonsense notion that actions that are followed by feelings of satisfaction are more likely to be generated again in the future, whereas actions that are followed by negative outcomes are less likely to reoccur. This fundamental reinforcement learning principle has been developed by the artificial intelligence community into a body of algorithms used to train autonomous systems to operate independently in complex and uncertain environments (Barto & Sutton, 1997;Sutton & Barto, 1998). Research has also evaluated the neural mechanisms underlying reinforcement learning in biological systems, but these mechanisms are still poorly understood.In this article, we provide a framework for understanding the neural basis of reinforcement learning in humans. Our proposal links together two areas of research that have, until now, been considered separately. On the one hand, we have previously inferred the existence of a generic, high-level error-processing system in humans from the error-related negativity (ERN), a negative deflection in the ongoing electroencephalogram (EEG) seen when human participants commit errors in a wide variety of psychological tasks. The ERN appears to be generated in the anterior cingulate cortex. On the other hand, other researchers have argued that the mesencephalic dopamine system conveys reinforcement learning signals to the basal ganglia and frontal cortex, where they are used to facilitate the development of adaptive motor programs. Although the reinforcement learning function attributed to the mesencephalic dopamine system and the error-processing function associated with the ERN appear to be concerned with the same problem-namely, evaluating the appropriateness of ongoing events, and using that information to facilitate the development and expression of adaptive behaviors-a possible relationship between these two systems remains to be explored.In this article, we propose a hypothesis that unifies the two accounts by explicitly linking the gen...

No abstract

The reinforcement learning theory suggests that the feedback negativity should be larger when feedback is unexpected. Two recent studies found, however, that the feedback negativity was unaffected by outcome probability. To further examine this issue, participants in the present studies made reward predictions on each trial of a gambling task where objective reward probability was indicated by a cue. In Study 1, participants made reward predictions following the cue, but prior to their gambling choice; in Study 2, predictions were made following their gambling choice. Predicted and unpredicted outcomes were associated with equivalent feedback negativities in Study 1. In Study 2, however, the feedback negativity was larger for unpredicted outcomes. These data suggest that the magnitude of the feedback negativity is sensitive to violations of reward prediction, but that this effect may depend on the close coupling of prediction and outcome.

No abstract

The N200 and the feedback error-related negativity (fERN) are two components of the event-related brain potential (ERP) that share similar scalp distributions, time courses, morphologies, and functional dependencies, which raises the question as to whether they are actually the same phenomenon. To investigate this issue, we recorded the ERP from participants engaged in two tasks that independently elicited the N200 and fERN. Our results indicate that they are, in fact, the same ERP component and further suggest that positive feedback elicits a positive-going deflection in the time range of the fERN. Taken together, these results indicate that negative feedback elicits a common N200 and that modulation of fERN amplitude results from the superposition on correct trials of a positive-going deflection that we term the feedback correct-related positivity.

The error-related negativity (ERN) is an event-related brain potential observed when subjects receive feedback indicating errors or monetary losses. Evidence suggests that the ERN is larger for unexpected negative feedback. The P300 has also been shown to be enhanced for unexpected feedback, but does not appear to be sensitive to feedback valence. The present study evaluated the role of expectations on the ERN and P300 in two experiments that manipulated the probability of negative feedback (25%, 50%, or 75%) on a trial-by-trial basis in experiment 1, and by varying the frequency of positive and negative feedback across blocks of trials in experiment 2. In both experiments, P300 amplitude was larger for unexpected feedback; however, the ERN was equally large for expected and unexpected negative feedback. These results are discussed in terms of the potential role of expectations in processing errors and negative feedback.

The feedback negativity is a component of the event-related brain potential that is elicited by feedback stimuli associated with unfavorable outcomes. The present research investigated whether this component reflects an evaluation of the valence of experienced outcomes or a process of learning about actions that led to those outcomes. The latter hypothesis predicts that the feedback negativity should be observed only when negative outcomes are experienced in relation to executed actions. Contrary to this prediction, feedback negativities were observed in simple monetary gambling tasks in which participants made no active choices (experiment 1) and no overt actions (experiment 2). However, the amplitude of the component was reduced in these tasks relative to a gambling task in which the outcomes appeared to be contingent upon participants' response choices. This reduction was correlated with changes in participants' subjective ratings of involvement in the tasks, suggesting that the evaluative process indexed by the feedback negativity is sensitive to the motivational significance of ongoing events.

No abstract