Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex

This study investigated how neuronal activity in orbitofrontal cortex related to the expectation of reward changed while monkeys repeatedly learned to associate new instruction pictures with known behavioral reactions and reinforcers. In a delayed go-nogo task with several trial types, an initial picture instructed the animal to execute or withhold a reaching movement and to expect a liquid reward or a conditioned auditory reinforcer. When novel instruction pictures were presented, animals learned according to a trial-and-error strategy. After experience with a large number of novel pictures, learning occurred in a few trials, and correct performance usually exceeded 70% in the first 60-90 trials. About 150 task-related neurons in orbitofrontal cortex were studied in both familiar and learning conditions and showed two major forms of changes during learning. Quantitative changes of responses to the initial instruction were seen as appearance of new responses, increase of existing responses, or decrease or complete disappearance of responses. The changes usually outlasted initial learning trials and persisted during subsequent consolidation. They often modified the trial selectivities of activations. Increases might reflect the increased attention during learning and induce neuronal changes underlying the behavioral adaptations. Decreases might be related to the unreliable reward-predicting value of frequently changing learning instructions. The second form of changes reflected the adaptation of reward expectations during learning. In initial learning trials, animals reacted as if they expected liquid reward in every trial type, although only two of the three trial types were rewarded with liquid. In close correspondence, neuronal activations related to the expectation of reward occurred initially in every trial type. The behavioral indices for reward expectation and their neuronal correlates adapted in parallel during the course of learning and became restricted to rewarded trials. In conclusion, these data support the notion that neurons in orbitofrontal cortex code reward information in a flexible and adaptive manner during behavioral changes after novel stimuli.     

Influence of expectation of different rewards on behavior-related neuronal activity in the striatum

This study investigated how different expected rewards influence behavior-related neuronal activity in the anterior striatum. In a spatial delayed-response task, monkeys reached for a left or right target and obtained a small quantity of one of two juices (apple, grenadine, orange, lemon, black currant, or raspberry). In each trial, an initial instruction picture indicated the behavioral target and predicted the reward. Nonmovement trials served as controls for movement relationships. Consistent preferences in special reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the rewards predicted by the instructions. About 600 of >2,500 neurons in anterior parts of caudate nucleus, putamen, and ventral striatum showed five forms of task-related activations, comprising responses to instructions, spatial or nonspatial activations during the preparation or execution of the movement, and activations preceding or following the rewards. About one-third of the neurons showed different levels of task-related activity depending on which liquid reward was predicted at trial end. Activations were either higher or lower for rewards that were preferred by the animals as compared with nonpreferred rewards. These data suggest that the expectation of an upcoming liquid reward may influence a fraction of task-related neurons in the anterior striatum. Apparently the information about the expected reward is incorporated into the neuronal activity related to the behavioral reaction leading to the reward. The results of this study are in general agreement with an account of goal-directed behavior according to which the outcome should be represented already at the time at which the behavior toward the outcome is performed.     

Neuronal activity in monkey striatum related to the expectation of predictable environmental events.

1. This study investigated neuronal activity in the striatum preceding predictable environmental events and behavioral reactions. Monkeys performed in a delayed go-nogo task that included separate time periods during which animals expected signals of behavioral significance, prepared for execution or inhibition of arm reaching movements, and expected the delivery of reward. In the task, animals were instructed by a green light cue to perform an arm reaching movement when a trigger stimulus came on approximately 3 s later (go situation). Movement was withheld after the same trigger light when the instruction cue had been red (nogo situation). Liquid reward was delivered on correct performance in both situations. 2. A total of 1,173 neurons were studied in the striatum (caudate nucleus and putamen) of 3 animals, of which 615 (52%) showed some change in activity during task performance. This report describes how the activity of 193 task-related neurons increased in advance of at least 1 component of the task, namely the instruction cue, the trigger stimulus, or the delivery of liquid reward. These neurons were found in dorsal and anterior parts of caudate and putamen and were slightly more frequent in the proximity of the internal capsule. 3. The activity of 16 neurons increased in both go and nogo trials before the onset of the instruction and subsided shortly after this signal. These activations may be related to the expectation of the instruction as the first signal in each trial. 4. The activity of 15 neurons increased between the instruction and the trigger stimulus in both go and nogo trials. These activations may be related to the expectation of the trigger stimulus independent of an arm movement. Further 56 neurons showed sustained activations only when the instruction requested a movement reaction. Activations were absent in trials in which the movement was withheld. Twenty-one of these neurons were tested with 2 different movement targets, 5 of which showed activity related to the direction of movement. These activations may be related to the preparation of movement or expectation of the specific movement triggering signal. The activity of an additional 20 neurons was unmodulated before the trigger stimulus in movement trials but increased in the interval between the no-movement instruction and the trigger stimulus for withholding the movement. These activations may be related to the preparation of movement inhibition as specific nogo reaction.(ABSTRACT TRUNCATED AT 400 WORDS)     

Role of primate basal ganglia and frontal cortex in the internal generation of movements

The purpose of these studies was to investigate neuronal activity in the basal ganglia and frontal cortex in relation to the internal generation of goal-directed movements. Monkeys performed goal-directed arm movements at a self-chosen moment in the absence of phasic stimuli providing external temporal reference. They were rewarded with a small morsel of food for each movement, although automatic or repetitive behavior was not reinforced. For reasons of comparison, animals were also trained in a delayed go no-go task in which visual cues instructed them to perform or refrain from an arm movement reaction to a subsequent trigger stimulus. This report describes the activity of neurons in the head of the caudate nucleus and rostral putamen preceding self-initiated arm movements and compares it with instruction-induced preparatory activity preceding movements in the delay task. A total of 497 caudate and 354 putamen neurons were tested in the delay task. Two types of preparatory activity were observed: (1) transient responses to the instruction cue, and (2) sustained activity preceding the trigger stimulus or movement onset. Transient responses were found in 48 caudate and 50 putamen neurons, occurring twice as often in movement ('go') as compared to no-movement ('no-go') trials, but rarely in both. These responses may code the information contained in the instruction relative to the forthcoming behavioral reaction. Sustained activity began after instruction onset and lasted until the trigger stimulus or the arm movement occurred, this being for periods of 2-7 s, 12-35 s, or up to 80 s, depending on the task requirements. This activity was seen in 47 caudate and 45 putamen neurons, was largely confined to go trials, and was unrelated to the preparation of saccadic eye movements. In some cases, this activity began as direct responses to the instruction stimulus, but in the majority of cases developed more gradually before the movement. Thus, both transient and sustained activations appear to be related to the preparation of movements. A total of 390 caudate and 293 putamen neurons were tested during self-initiated movements. Activity preceding earliest movement-related muscle activity was found in 32 caudate and 42 putamen neurons. This premovement activity began 0.5-5.0 s before movement onset (median 1160 ms), increased slowly, reached its peak close to movement onset, and subsided rapidly thereafter. It was unrelated to the preparation of saccadic eye movements. Comparisons between the two tasks were made on 53 neurons.(ABSTRACT TRUNCATED AT 400 WORDS)     

Effects of expectations for different reward magnitudes on neuronal activity in primate striatum

In behavioral science, it is well known that humans and nonhuman animals are highly sensitive to differences in reward magnitude when choosing an outcome from a set of alternatives. We know that a realm of behavioral reactions is altered when animals begin to expect different levels of reward outcome. Our present aim was to investigate how the expectation for different magnitudes of reward influences behavior-related neurophysiology in the anterior striatum. In a spatial delayed response task, different instruction pictures are presented to the monkey. Each image represents a different magnitude of juice. By reaching to the spatial location where an instruction picture was presented, animals could receive the particular liquid amount designated by the stimulus. Reliable preferences in reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the various reward amounts predicted by the instruction cues. A total of 374 of 2,000 neurons in the anterior parts of the caudate nucleus, putamen, and ventral striatum showed five forms of task-related activation during the preparation or execution of movement and activations preceding or following the liquid drop delivery. Approximately one-half of these striatal neurons showed differing response levels dependent on the magnitude of liquid to be received. Results of a linear regression analysis showed that reward magnitude and single cell discharge rate were related in a subset of neurons by a monotonic positive or negative relationship. Overall, these data support the idea that the striatum utilizes expectancies that contain precise information concerning the predicted, forthcoming level of reward in directing general behavioral reactions.     

Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion

In primates, stimuli to sensory systems influence motivational and emotional behavior via neural relays to the orbitofrontal cortex. This article reviews studies on the effects of stimuli from multiple sensory modalities on the brain of humans and some other higher primates. The primate orbitofrontal cortex contains the secondary taste cortex, in which the reward value of taste is represented. It also contains the secondary and tertiary olfactory cortical areas, in which information about the identity and also about the reward value of odors is represented. A somatosensory input is revealed by neurons that respond to the viscosity of food in the mouth, to the texture (mouth feel) of fat in the mouth, and to the temperature of liquids placed into the mouth. The orbitofrontal cortex also receives information about the sight of objects from the temporal lobe cortical visual areas. Information about each of these modalities is represented separately by different neurons, but in addition, other neurons show convergence between different types of sensory input. This convergence occurs by associative learning between the visual or olfactory input and the taste. In that emotions can be defined as states elicited by reinforcers, the neurons that respond to primary reinforcers (such as taste and touch), as well as learn associations to visual and olfactory stimuli that become secondary reinforcers, provide a basis for understanding the functions of the orbitofrontal cortex in emotion. In complementary neuroimaging studies in humans, it is being found that areas of the orbitofrontal cortex are activated by pleasant touch, by painful touch, by taste, by smell, and by more abstract reinforcers such as winning or losing money. Damage to the orbitofrontal cortex in humans can impair the learning and reversal of stimulus-reinforcement associations and thus the correction of behavioral responses when these are no longer appropriate because previous reinforcement contingencies change. It is striking that humans and other catarrhines, being visual specialists like other anthropoids, interface the visual system to other sensory systems (e.g., taste and smell) in the orbitofrontal cortex.     

Responses to reward in monkey dorsal and ventral striatum

Summary The sources of input and the behavioral effects of lesions and drug administration suggest that the striatum participates in motivational processes. We investigated the activity of single striatal neurons of monkeys in response to reward delivered for performing in a go-nogo task. A drop of liquid was given each time the animal correctly executed or withheld an arm movement in reaction to a visual stimulus. Of 1593 neurons, 115 showed increased activity in response to delivery of liquid reward in both go and nogo trials. Responding neurons were predominantly located in dorsal and ventromedial parts of anterior putamen, in dorsal and ventral caudate, and in nucleus accumbens. They were twice as frequent in ventral as compared to dorsal striatal areas. Responses occurred at a median latency of 337 ms and lasted for 525 ms, with insignificant differences between dorsal and ventral striatum. Reward responses differed from activity recorded in the face area of posterior putamen which varied synchronously with individual mouth movements. Responses were directly related to delivery of primary liquid reward and not to auditory stimuli associated with it. Most of them also occurred when reward was delivered outside of the task. These results demonstrate that neurons of dorsal and particularly ventral striatum are involved in processing information concerning the attribution of primary reward.     

Activity of tonically active neurons in the monkey putamen during initiation and withholding of movement

Tonically active neurons (TANs) of the primate striatum are putative interneurons that respond to events of motivational significance, such as primary rewards, and to sensory stimuli that predict such events. Because TANs influence striatal projection neurons, TANs may play a role in the initiation and control of movement. To examine this issue, we recorded from putaminal TANs in macaque monkeys trained to make the same arm movement in two ways--in reaction to an external cue and also after a variable delay without an explicit instruction to move (self-timed movements). On other trials, the animals had to withhold movement following an external cue. The task design ensured that the three types of trials were effectively randomly interleaved, equally frequent, and similar in overall timing. Separately, we presented "playback" trials in which the same sequence of visual stimulation and reward was presented while the animals fixated without making the arm movement. We found that TAN responses were strongly affected by behavioral context. In particular, TAN responses were strikingly stronger when the animals actively withheld movements than on the corresponding playback trials, even though the stimulus sequence and reward timing were identical and no movement was made in either case. Many TANs also became active in the absence of a proximate sensory cue on self-timed movements, suggesting that TANs may reflect internal processes that are specific to self-timed movements. These results suggest that TANs may directly participate in, or monitor the motivational significance of, an animal's actions as well as external events.     

Behavioral reactions reflecting differential reward expectations in monkeys

Learning theory emphasizes the importance of expectations in the control of instrumental action. This study investigated the variation of behavioral reactions toward different rewards as an expression of differential expectations of outcomes in primates. We employed several versions of two basic behavioral paradigms, the spatial delayed response task and the delayed reaction task. These tasks are commonly used in neurobiological studies of working memory, movement preparation, and event expectation involving the frontal cortex and basal ganglia. An initial visual instruction stimulus indicated to the animal which one of several food or liquid rewards would be delivered after each correct behavioral response, or whether or not a reward could be obtained. We measured the reaction times of the operantly conditioned arm movement necessary for obtaining the reward, and the durations of anticipatory licking prior to liquid reward delivery as a Pavlovian conditioned response. The results showed that both measures varied depending on the reward predicted by the initial instruction. Arm movements were performed with significantly shorter reaction times for foods or liquids that were more preferred by the animal than for less preferred ones. Still larger differences were observed between rewarded and unrewarded trials. An interesting effect was found in unrewarded trials, in which reaction times were significantly shorter when a highly preferred reward was delivered in the alternative rewarded trials of the same trial block as compared to a less preferred reward. Anticipatory licks preceding the reward were significantly longer when highly preferred rather than less preferred rewards, or no rewards, were predicted. These results demonstrate that behavioral reactions preceding rewards may vary depending on the predicted future reward and suggest that monkeys differentially expect particular outcomes in the presently investigated tasks.     

Task-related responses of monkey medullary dorsal horn neurons

Medullary dorsal horn neurons with trigeminal sensory properties have been previously shown to have additional responses associated with cues relevant to the successful execution of a behavioral task. These "task-related" responses were evoked by environmental cues but were independent of the specific stimulus parameters. We have examined further the characteristics of task-related responses in medullary dorsal horn neurons of three monkeys. Single-unit activity was recorded while the monkeys were performing behavioral tasks that required them to discriminate thermal or visual stimuli for a liquid reward. Forty-five percent (34/75) of the medullary dorsal horn neurons studied exhibited task-related activity that was significantly correlated with the stereotypical behavioral events that occurred during the tasks. Similar events occurring outside of the task produced no response. In addition to the task-related activity of these medullary dorsal horn neurons, responses to mechanical and/or thermal stimuli presented within the neuron's receptive field were demonstrated in 28 of 34 cases. These sensory responses also were evoked by the same stimuli presented outside of the behavioral task. Fifteen of the neurons with task-related responses could be activated antidromically from thalamic stimulating electrodes. Task-related responses were categorized according to their relationship to the three phases of the behavioral trial: trial initiation, trial continuation, and trial termination. Although an individual task-related response was associated with a single behavioral event, most medullary dorsal horn neurons (30/34) exhibited a reproducible pattern of task-related responses that occurred during more than one phase of the trial. Trial-initiation task-related responses were subdivided depending on their correlation with specific events that occurred within that phase of the trial. One-third of the 18 excitatory trial-initiation responses were associated with the visual stimulus that cued the monkey to begin the trial; the remaining two-thirds were associated with the monkey's press of the button that actually initiated the trial. Trial-continuation task-related responses (observed while the monkey waited for a thermal stimulus that triggered a rewarded motor response) were shown to be independent of the actual temperature of the thermal stimulus. In addition these trial-continuation task-related responses were also noted during trials without a thermal stimulus, in which the trigger cue was the onset of a light (in a visual task).(ABSTRACT TRUNCATED AT 400 WORDS)     

Human cortical responses to water in the mouth,and the effects of thirst

In an event-related functional magnetic resonance imaging (fMRI) study in humans it was shown, first, that water produces activations in cortical taste areas (in particular the frontal operculum/anterior insula which is the primate primary taste cortex, and the caudal orbitofrontal/secondary taste cortex) comparable to those produced by the prototypical tastants salt and glucose. Second, the activations in the frontal operculum/anterior insula produced by water when thirsty were still as large after the subjects had consumed water to satiety. Third, in contrast, the responses to water in the caudal orbitofrontal cortex were modulated by the physiological state of the body, in that responses to the oral delivery of water in this region were not found after the subjects had drunk water to satiety. Fourth, further evidence that the reward value or pleasantness of water is represented in the orbitofrontal cortex was that a positive correlation with the subjective ratings of the pleasantness of the water was found with activations in the caudal and anterior orbitofrontal cortex, and also in the anterior cingulate cortex. Fifth, it was found that a region of the middle part of the insula was also activated by water in the mouth, and further, that this activation only occurred when thirsty. Sixth, analyses comparing pre- and postsatiety periods (i.e., when thirsty and when not thirsty) independently of stimulus delivery revealed higher activity levels in the rostral anterior cingulate cortex. The activity of the rostral anterior cingulate cortex thus appears to reflect the thirst level or motivational state of the subjects.     

Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli

Summary The behavioral relationships of 396 striatum neurons with regular, tonically elevated discharge rates were studied. While monkeys performed a delayed gonogo task, neurons predominantly located in medial putamen responded with phasic depressions (n = 30) or activations (n = 5) to task-specific stimuli. Particularly effective was an instruction light preparing for movement or no-movement reactions, and an auditory signal associated with reward delivery. Stimuli triggering arm or mouth movements were less effective. The data demonstrate that these usually poorly modulated neurons display context-dependent phasic activity in specific behavioral situations.     

Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences

Goal-directed behaviors require the consideration and expenditure of physical effort. The anterior cingulate cortex (ACC) appears to play an important role in evaluating effort and reward and in organizing goal-directed actions. Despite agreement regarding the involvement of the ACC in these processes, the way in which effort-, reward-, and motor-related information is registered by networks of ACC neurons is poorly understood. To contrast ACC responses to effort, reward, and motor behaviors, we trained rats on a reversal task in which the selected paths on a track determined the level of effort or reward. Effort was presented in the form of an obstacle that was climbed to obtain reward. We used single-unit recordings to identify neural correlates of effort- and reward-guided behaviors. During periods of outcome anticipation, 52% of recorded ACC neurons responded to the specific route taken to the reward while 21% responded prospectively to effort and 12% responded prospectively to reward. In addition, effort- and reward-selective neurons typically responded to the route, suggesting that these cells integrated motor-related activity with expectations of future outcomes. Furthermore, the activity of ACC neurons did not discriminate between choice and forced trials or respond to a more generalized measure of outcome value. Nearly all neural responses to effort and reward occurred after path selection and were restricted to discrete temporal/spatial stages of the task. Together, these findings support a role for the ACC in integrating route-specific actions, effort, and reward in the service of sustaining discrete movements through an effortful series of goal-directed actions.     

Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states

In the primate striatum, the tonically discharging neurons respond to conditioned stimuli associated with reward. We investigated whether these neurons respond to the reward itself and how changes in the behavioral context in which the reward is delivered might influence their responsiveness. A total of 286 neurons in the caudate nucleus and putamen were studied in two awake macaque monkeys while liquid reward was delivered in three behavioral situations: (1) an instrumental task, in which reward was delivered upon execution of a visually triggered arm movement; (2) a classically conditioned task, in which reward was delivered 1 s after a visual signal; (3) a free reward situation, in which reward was delivered at irregular time intervals outside of any conditioning task. The monkeys′ uncertainty about the time at which reward will be delivered was assessed by monitoring their mouth movements. A larger proportion of neurons responsive to reward was observed in the free reward situation (86%) than in the classically conditioned (57%) and instrumental tasks (37%). Among the neurons tested in all situations (n = 78), 24% responded to reward regardless of the situation and 65% in only one or two situations. Responses selective for one particular situation occurred exclusively in the free reward situation. When the reward was delivered immediately after the visual signal in the classically conditioned task, most of the neurons reduced or completely lost their responses to reward, and other neurons remained responsive. Conversely, neuronal responses invariably persisted when reward was delivered later than 1 s after the visual signal. This is the first report that tonic striatal neurons might display responses directly to primary rewards. The neuronal responses were strongly influenced by the behavioral context in which the animals received the reward. An important factor appears to be the timing of reward. These neurons might therefore contribute to a general aspect of behavioral reactivity of the subject to relevant stimuli. Received: 16 September 1996 / Accepted: 1 April 1997     

Conditional task-related responses in monkey dorsomedial frontal cortex

Summary Dorsomedial frontal cortex (DMFC) was studied in monkeys trained to make visually guided eye or arm movements. Portions of DMFC are involved in the execution of learned, goal-directed behaviors. Many neurons discharge with both eye and hand movements as well as when motor responses are withheld, provided these behaviors are related to the successful execution of the learned task. Similar movements, when carried out at times unrelated to the task, are not accompanied by neuronal activity. Electrical microstimulation produces either arrest of task-related, but not task-unrelated motor acts, or triggers task-related movements. The nature of stimulation elicited responses depends on the task the animal has been trained on and is altered by new training.     

The orbitofrontal cortex and beyond: from affect to decision-making

The orbitofrontal cortex represents the reward or affective value of primary reinforcers including taste, touch, texture, and face expression. It learns to associate other stimuli with these to produce representations of the expected reward value for visual, auditory, and abstract stimuli including monetary reward value. The orbitofrontal cortex thus plays a key role in emotion, by representing the goals for action. The learning process is stimulus-reinforcer association learning. Negative reward prediction error neurons are related to this affective learning. Activations in the orbitofrontal cortex correlate with the subjective emotional experience of affective stimuli, and damage to the orbitofrontal cortex impairs emotion-related learning, emotional behaviour, and subjective affective state. With an origin from beyond the orbitofrontal cortex, top-down attention to affect modulates orbitofrontal cortex representations, and attention to intensity modulates representations in earlier cortical areas of the physical properties of stimuli. Top-down word-level cognitive inputs can bias affective representations in the orbitofrontal cortex, providing a mechanism for cognition to influence emotion. Whereas the orbitofrontal cortex provides a representation of reward or affective value on a continuous scale, areas beyond the orbitofrontal cortex such as the medial prefrontal cortex area 10 are involved in binary decision-making when a choice must be made. For this decision-making, the orbitofrontal cortex provides a representation of each specific reward in a common currency.     

Contrasting neuronal activity in supplementary and precentral motor cortex of monkeys. I. Responses to instructions determining motor responses to forthcoming signals of different modalities

The present report contrasts neuronal activity in two motor cortical fields after instructions that determine which of two sensory signals will trigger a movement and which will not. The goal of the study was to determine possible differential roles of the two cortical fields in the process of preparing to move in response to one external cue and to ignore another. Single-cell recordings were made from the supplementary motor area (SMA) and the precentral motor area (PCM) of monkeys trained to perform key-press movements in two different modes. In the auditory mode, an instruction signal warned the animal to prepare to start the movement promptly in response to a forthcoming 1,000-Hz tone burst (trigger signal), but to remain motionless if the signal was vibrotactile (nontrigger signal). In the tactile mode, the trigger and nontrigger signals were reversed: a different instruction signal warned the animal to prepare to perform the key-press movement in response to the vibrotactile cue, but to withhold it in response to the 1,000-Hz tone. The instruction signals were auditory tones of 300 Hz for the auditory mode and 100 Hz for the tactile mode. Out of 259 task-related SMA neurons, 128 (49%) responded to instructions. Three types of instruction responses were observed: 1) 95 neurons showed continuous instruction-induced activity changes lasting until the occurrence of the movement-triggering signal, regardless of whether an intervening nontrigger signal occurred. 2) 24 neurons showed increased activity until the occurrence of the nontriggering signal, after which the activity subsided. When there was no nontrigger signal, the activity increased during a period when the nontrigger signal might have been given. 3) Nine neurons responded with a transient, short-latency discharge after the instruction. The responses of SMA neurons to two instructions were often different. Forty-four SMA neurons exhibited a selective response to only one of the two instructions. In 43 neurons the response was differential, with the magnitude of activity increase or decrease being at least three times greater after one instruction than the other. In the remaining 41 neurons the response was nondifferential. Out of 112 task-related PCM neurons, 25 (22%) responded to the instructions. In the majority of them (21 neurons), the instruction response was nondifferential.(ABSTRACT TRUNCATED AT 400 WORDS)     

Activity of neurons of the subthalamic nucleus in relation to motor performance in the cat

The activity of subthalamic nucleus neurons related to motor performance was studied in three unrestrained cats operantly conditioned to perform a lever-release movement. The movement was initiated either rapidly after the trigger stimulus (a brief sound) in a simple reaction-time paradigm or after a delay in trials identified by a tone cue. These paradigms were randomly presented. The activity of 171 neurons was recorded in the contralateral and in the ipsilateral subthalamic nucleus, with respect to the performing limb. The mean spontaneous activity of cells in the ipsilateral side (18.5±13.8 imp/s, mean±SD) was higher than that in the contralateral side (8.5±8.1 imp/s). A total of 145 cells (85%) presented significant changes in activity in relation to the lever-release movement (task-related cells). The remaining 26 cells were either related to other events of the task (n=15; lever-press or reinforcement occurrence) or not related at all to the task performance (n=11). The majority of changes of activity of task-related cells were initial increases in discharge, which started on average, 127 ms before movement onset and lasted several hundreds of milliseconds. These increases in discharge were more frequent in the contralateral side (75 of 80 task-related cells, 94%) than in the ipsilateral side (43 of 65 task-related cells, 66%). The changes in activity, either increases or decreases, occurred early after the trigger stimulus, since 62% of them had a latency of less than 100 ms. Although the mean latency of initial increases was rather similar in both sides (97 ms contralateral versus 104 ms ipsilateral), the contralateral side was characterized by a high proportion of very early responses (less than 20 ms). For most neurons, the early changes in activity described above were absent after the trigger stimulus in the delayed condition. For certain neurons, the changes in activity prior to movement were different in reactiontime condition and in delayed condition, showing that the pattern of activity preceding movement might depend on the temporal requirements for motor initiation. The results suggest that a significant proportion of subthalamic cells are involved in the preparation and the initiation phases of the lever-release movement studied, although other hypotheses (e.g. stimulus-related responses) cannot be definitely ruled out. The timings and patterns of the changes in activity observed in the subthalamic nucleus in the present study, and in the pallidal complex previously, cannot be explained easily by the classical scheme where the external pallidum inhibits the subthalamic nucleus. The results suggest rather that the subthalamic nucleus, driven by a yet-to-be-determined excitatory input, exerts an excitatory influence on the pallidum and plays a crucial role in the control of the basal ganglia output neurons.     

The combination of appetitive and aversive reinforcers and the nature of their interaction during auditory learning

Learned changes in behavior can be elicited by either appetitive or aversive reinforcers. It is, however, not clear whether the two types of motivation, (approaching appetitive stimuli and avoiding aversive stimuli) drive learning in the same or different ways, nor is their interaction understood in situations where the two types are combined in a single experiment. To investigate this question we have developed a novel learning paradigm for Mongolian gerbils, which not only allows rewards and punishments to be presented in isolation or in combination with each other, but also can use these opposite reinforcers to drive the same learned behavior. Specifically, we studied learning of tone-conditioned hurdle crossing in a shuttle box driven by either an appetitive reinforcer (brain stimulation reward) or an aversive reinforcer (electrical footshock), or by a combination of both. Combination of the two reinforcers potentiated speed of acquisition, led to maximum possible performance, and delayed extinction as compared to either reinforcer alone. Additional experiments, using partial reinforcement protocols and experiments in which one of the reinforcers was omitted after the animals had been previously trained with the combination of both reinforcers, indicated that appetitive and aversive reinforcers operated together but acted in different ways: in this particular experimental context, punishment appeared to be more effective for initial acquisition and reward more effective to maintain a high level of conditioned responses (CRs). The results imply that learning mechanisms in problem solving were maximally effective when the initial punishment of mistakes was combined with the subsequent rewarding of correct performance.     

Neurons associated with behavioral context errors in the primary and higher-order gustatory cortices in the monkey

Cue responses of neurons in the taste-related cortex of Japanese macaque monkeys were studied during a NaCl-water discrimination GO-NOGO task, to compare the correct and incorrect responses. Most neurons produced a steady pattern of discharges in response to a given cue at both correct and incorrect responses, presumably responding to the physicochemical nature of the cue. Some neurons showed the discharge pattern for a certain cue changing to that for another cue at task error, presumably representing the subsequent behavioral reaction or behavioral context. These neurons were mainly located in the precentral operculum and orbitofrontal cortex, and rarely in the primary gustatory area, area G.