Using prediction errors to drive saccade adaptation: the implicit double-step task

A prediction-based error signal, neurally computed as the difference between predicted and observed movement outcomes, has been proposed as the driving force for motor learning. This suggests that the generation of predictive saccades to periodically paced targets—whose performance accuracy is actively maintained using this same error signal—invokes the motor-learning network. We examined whether a simple predictive-saccade task (implicit double-step adaptation, in which targets are gradually displaced outward to exaggerate normal hypometric movement errors) can stand in place of a traditional double-step saccade-adaptation task to induce an increase in saccade gain. We find that the implicit double-step adaptation task can induce significant gain-increase adaptation (of comparable magnitude to that of the standard double-step task) in normal control subjects. Unlike control subjects, patients with impaired cerebella are unable to adapt their saccades in response to this paradigm; this implies that the cerebellum is crucial for processing prediction-based error signals for motor learning.     

The role of kinematic redundancy in adaptation of reaching

Although important differences exist between learning a new motor skill and adapting a well-learned skill to new environmental constraints, studies of force field adaptation have been used frequently in recent years to identify processes underlying learning. Most of these studies have been of reaching tasks that were each hand position was specified by a unique combination of joint angles. At the same time, evidence has been provided from a variety of tasks that the central nervous system takes advantage of the redundancy available to it when planning and executing functional movements. The current study attempted to determine whether a change in the use of joint motion redundancy is associated with the adaptation process. Both experimental and control subjects performed 160 trials of reaching in each of four adaptation phases, while holding the handle of a robot manipulandum. During the first and last adaptation phases, the robot motors were turned off. During phases 2 and 3 the motors produced a velocity-dependent force field to which experimental subjects had to adapt to regain relatively straight line hand movements during reaching to a target, while the motors remained off for the control group. The uncontrolled manifold (UCM) method was used to partition the variance of planar clavicle–scapular, shoulder, elbow and wrist joint movements into two orthogonal components, one (V _UCM) that reflected combinations of joint angles that were equivalent with respect to achieving the average hand path and another (V _ORT) that took the hand away from its average path. There was no change in either variance component for the control group performing 640 non-perturbed reaches across four ‘pseudo-adaptation’ phases. The experimental group showed adaptation to reaching in the force field that was accompanied initially by an increase in both components of variance, followed by a smaller decrease of V _UCM than V _ORT during 320 practice reaches in the force field. After initial re-adaptation to reaching to the null field, V _UCM was higher in experimental than in control subjects after performing a comparable number of reaches. V _UCM was also larger in the experimental group after re-adaptation when compared to the 160 null field reaching trials performed prior to initial force field introduction. The results suggest that the central nervous system makes use of kinematic redundancy, or flexibility of motor patterns, to adapt reaching performance to unusual force fields, a fact that has implications for the hypothesis that motor adaptation requires learning of formal models of limb and environmental dynamics.     

Task goals influence online corrections and adaptation of reaching movements

Everyday movements often have multiple solutions. Many of these solutions arise from biomechanical redundancies. Often, however, the goal does not require a unique movement. To examine how people exploit task-related redundancy, we observed as participants produced three-dimensional (3-D) reaching movements, moving to one of two rectangular targets that were diagonally oriented in the frontal (x, y) plane. On most trials, the movement was perturbed by a vertical, velocity-dependent force. Since participants were free to move in 3-D space, online corrections could involve movement along the perturbed, vertical dimension, as well as the nonperturbed, horizontal dimension. If the motor system exploits task redundancies, then corrections along the horizontal dimension should depend on the orientation of the target. Consistent with this prediction, participants modified both the horizontal and vertical coordinates of the trajectory over the course of learning, and the horizontal component was sensitive to the orientation of the target. Furthermore, participants produced online corrections with a horizontal component that brought the hand closer to the target. These results suggest that we not only correct for mismatches between expected and experienced forces but also exploit task-specific redundancies to efficiently improve performance.     

Cerebellar lesions impair context-dependent adaptation of reaching movements in primates

To produce accurate movements when conditions change suddenly, the brain must be capable of learning multiple versions of a given motor task and must be able to access the appropriate program using sensory information linked to the context of the movement. The neural basis for context-dependent motor learning is uncertain, but the cerebellum is thought to play a fundamental role. In this study, we examined the effect of lesions of the dorsal vermal and paravermal cerebellar cortex on the adaptation of reaching movements produced by modified visual feedback and accessed with a visual cue. Two rhesus monkeys were trained to point to targets displayed on a video monitor while viewing monocularly with either eye. During the experimental sessions, visual information received by one eye (the "modified" eye) was displaced horizontally, while the information received by the other ("normal") eye remained unaltered. In the first set of experiments (noncontextual paradigm), the animals pointed to targets while viewing with the modified eye. This paradigm resulted in a gradual improvement in pointing accuracy when viewing with that eye, but also produced a shift in pointing responses of equivalent size when viewing with the normal eye. In the second set of experiments (contextual paradigm), the animals alternated six blocks of reaches while viewing monocularly with the modified eye with six blocks viewing with the normal eye. This paradigm improved the pointing accuracy when viewing with the modified eye, but produced only a small shift in pointing responses when viewing with the normal eye. After the dorsal vermal and paravermal cerebellar cortex were resected, no change occurred in the pattern of adaptation produced by the noncontextual paradigm. The contextual paradigm, however, no longer selectively adapted pointing responses for each eye, but rather produced a pointing shift of equivalent size when viewing with either eye. The results indicate that pointing responses can be differentially adapted for each viewing eye, which is a form of context-dependent motor learning. This capability was lost after focal lesions of the dorsal vermal and paravermal cerebellar cortex, suggesting that these regions of cerebellar cortex are required to learn or store multiple representations of a movement, or to retrieve the appropriate motor program in a given sensory context.     

Long-term adaptation to dynamics of reaching movements: a PET study

Positron emission tomography (PET) was used to examine changes in the cerebellum as subjects learned to make movements with their right arm while holding the handle of a robot that produced a force field. Brain images were acquired during learning and then during recall at 2 and 4 weeks. We also acquired images during a control task where the force field was not learnable and subjects did not show any improvements across sessions. During the 1st day, we observed that motor errors decreased from the control condition to the learning condition. However, regional cerebral blood flow (rCBF) in the posterior region of the right cerebellar cortex initially increased from the control condition and then gradually declined with reductions in motor error. Correspondingly, rCBF in the ipsilateral deep cerebellar nuclei (DCN) initially decreased from the control condition and then increased with reductions in motor error. If measures of rCBF mainly reflect presynaptic activity of neurons, this result predicts that DCN neurons fire with a pattern that starts high in the control task then decreases as learning proceeds. Similarly, Purkinje cells should generally have their lowest activity in the control task. This pattern is consistent with neurophysiological recordings that find that cerebellar activity during early learning of a motor task may mainly reflect changes in coactivation of muscles of the limbs, rather than a learning specific signal. By the end of the first session, motor errors had reached a minimum and no further improvements were observed. However, across the weeks a region in the anterior cerebellar cortex showed gradual decreases in rCBF that could not be attributed to changes in motor performance. Because patterns of rCBF in the cortex and nuclei were highly anti-correlated, we used structural equation modeling to estimate how synaptic activity in the cerebellar cortex influenced synaptic activity in the DCN. We found a negative correlation with a strength that significantly increased during the 4 weeks. This suggests that, during long-term recall, the same input to the cerebellar cortex would produce less synaptic activity at the DCN, possibly because of reduced cerebellar cortex output to the DCN.     

Prism adaptation during walking generalizes to reaching and requires the cerebellum

Adaptation of arm movements to laterally displacing prism glasses is usually highly specific to body part and movement type and is known to require the cerebellum. Here, we show that prism adaptation of walking trajectory generalizes to reaching (a different behavior involving a different body part) and that this adaptation requires the cerebellum. In experiment 1, healthy control subjects adapted to prisms during either reaching or walking and were tested for generalization to the other movement type. We recorded lateral deviations in finger endpoint position and walking direction to measure negative aftereffects and generalization. Results showed that generalization of prism adaptation is asymmetric: walking generalizes extensively to reaching, but reaching does not generalize to walking. In experiment 2, we compared the performance of cerebellar subjects versus healthy controls during the prism walking adaptation. We measured rates of adaptation, aftereffects, and generalization. Cerebellar subjects had reduced adaptation magnitudes, slowed adaptation rates, decreased negative aftereffects, and poor generalization. Based on these experiments, we propose that prism adaptation during whole body movements through space invokes a more general system for visuomotor remapping, involving recalibration of higher-order, effector-independent brain regions. In contrast, prism adaptation during isolated movements of the limbs is probably recalibrated by effector-specific mechanisms. The cerebellum is an essential component in the network for both types of prism adaptation.     

Interaction of visual and proprioceptive feedback during adaptation of human reaching movements

People tend to make straight and smooth hand movements when reaching for an object. These trajectory features are resistant to perturbation, and both proprioceptive as well as visual feedback may guide the adaptive updating of motor commands enforcing this regularity. How is information from the two senses combined to generate a coherent internal representation of how the arm moves? Here we show that eliminating visual feedback of hand-path deviations from the straight-line reach (constraining visual feedback of motion within a virtual, "visual channel") prevents compensation of initial direction errors induced by perturbations. Because adaptive reduction in direction errors occurred with proprioception alone, proprioceptive and visual information are not combined in this reaching task using a fixed, linear weighting scheme as reported for static tasks not requiring arm motion. A computer model can explain these findings, assuming that proprioceptive estimates of initial limb posture are used to select motor commands for a desired reach and visual feedback of hand-path errors brings proprioceptive estimates into registration with a visuocentric representation of limb position relative to its target. Simulations demonstrate that initial configuration estimation errors lead to movement direction errors as observed experimentally. Registration improves movement accuracy when veridical visual feedback is provided but is not invoked when hand-path errors are eliminated. However, the visual channel did not exclude adjustment of terminal movement features maximizing hand-path smoothness. Thus visual and proprioceptive feedback may be combined in fundamentally different ways during trajectory control and final position regulation of reaching movements.     

Differential effect of task conditions on errors of direction and extent of reaching movements

 Invariant patterns in the distribution of the endpoints of reaching movements have been used to suggest that two important movement parameters of reaching movements, direction and extent, are planned by two independent processing channels. This study examined this hypothesis by testing the effect of task conditions on variable errors of direction and extent of reaching movements. Subjects made reaching movements to 25 target locations in a horizontal workspace, in two main task conditions. In task 1, subjects looked directly at the target location on the horizontal workspace before closing their eyes and pointing to it. In task 2, arm movements were made to the same target locations in the same horizontal workspace, but target location was displayed on a vertical screen in front of the subjects. For both tasks, variable errors of movement extent (on-axis error) were greater than for movement direction (off-axis error). As a result, the spatial distributions of endpoints about a given target usually formed an ellipse, with the principal axis oriented in the mean movement direction. Also, both on- and off-axis errors increased with movement amplitude. However, the magnitude of errors, especially on-axis errors, scaled differently with movement amplitude in the two task conditions. This suggests that variable errors of direction and extent can be modified independently by changing the nature of the sensorimotor transformations required to plan the movements. This finding is further evidence that the direction and extent of reaching movements appear to be controlled independently by the motor system. Received: 8 October 1996 / Accepted: 14 January 1997     

Sensory adaptation and incentive contrast as factors affecting NaCl preference

One normal rat and 6 adrenalectomized rats were tested for the effect of predrinking various concentrations of NaCl on the preference for NaCl solutions. Short term variations of the traditional 1-bottle and 2-bottle preference tests were presented to the animals and the behavior measured in terms of number of licks and consistency of choice. The results showed that an equal preference for 0.1 m NaCl and 0.25 m NaCl after predrinking a weak NaCl solution (0.01 m) changed to a strong preference for 0.25 m NaCl over 0.1 m NaCl when the animals predrank a stronger NaCl solution (0.25 m). Control tests indicated that these results could not be explained simply on the basis of sensory adaptation. It was suggested that both sensory adaptation and incentive contrast were the essential factors which controlled the preference behavior in this experiment.     

Sensitivity to prediction error in reach adaptation

It has been proposed that the brain predicts the sensory consequences of a movement and compares it to the actual sensory feedback. When the two differ, an error signal is formed, driving adaptation. How does an error in one trial alter performance in the subsequent trial? Here we show that the sensitivity to error is not constant but declines as a function of error magnitude. That is, one learns relatively less from large errors compared with small errors. We performed an experiment in which humans made reaching movements and randomly experienced an error in both their visual and proprioceptive feedback. Proprioceptive errors were created with force fields, and visual errors were formed by perturbing the cursor trajectory to create a visual error that was smaller, the same size, or larger than the proprioceptive error. We measured single-trial adaptation and calculated sensitivity to error, i.e., the ratio of the trial-to-trial change in motor commands to error size. We found that for both sensory modalities sensitivity decreased with increasing error size. A reanalysis of a number of previously published psychophysical results also exhibited this feature. Finally, we asked how the brain might encode sensitivity to error. We reanalyzed previously published probabilities of cerebellar complex spikes (CSs) and found that this probability declined with increasing error size. From this we posit that a CS may be representative of the sensitivity to error, and not error itself, a hypothesis that may explain conflicting reports about CSs and their relationship to error.     

Features of motor performance that drive adaptation in rapid hand movements

In order to explore how subjects correct for errors in movement and adapt their motor programs, we studied rapid hand movements. Subjects grasped a grooved knob and made brisk turning movements to various targets, similar to tuning a radio dial. A motor attached to the knob shaft was configured to apply a destabilizing negative viscous perturbation. Following a period of practice with no perturbations, the negative viscosity was engaged, which caused a large change in overall kinematics: the peak velocity increased, the movement amplitude was too large, and discrete corrective submovements were generated to bring the pointer back onto the target. After about an hour and nearly 1000 trials, subjects learned to move accurately in the new dynamic environment, returning their overall kinematics near to previous levels. Measures of performance included the endpoint error of the primary movement (the initial movement segment), the frequency and amplitude of corrective submovements, task success rate, mean squared jerk, and deviation from a "normal" angular velocity temporal profile. Both the amplitude and frequency of corrective submovements decreased progressively during adaptation as the subjects made fewer target overshoot errors. These results are consistent with motor learning schemes in which adaptation of the motor controller is driven by an attempt to reduce the endpoint error of the primary movement. While there have been many theories regarding what is being optimized in motor control, in general, biologically plausible mechanisms for implementing these schemes have not been described. A biologically plausible optimization criterion is the minimization of the occurrence and amplitude of corrective submovements, since the latter have been proposed as realistic climbing fiber training signals for adaptive changes in the cerebellum. We postulate that the other criteria that have been proposed are instead secondary to an increased accuracy of the primary movement and a corresponding decrease in the occurrence and amplitude of corrective submovements. Electronic Publication     

Medial prefrontal cell activity signaling prediction errors of action values

To adapt behavior to a changing environment, one must monitor outcomes of executed actions and adjust subsequent actions accordingly. Involvement of the medial frontal cortex in performance monitoring has been suggested, but little is known about neural processes that link performance monitoring to performance adjustment. Here, we recorded from neurons in the medial prefrontal cortex of monkeys learning arbitrary action-outcome contingencies. Some cells preferentially responded to positive visual feedback stimuli and others to negative feedback stimuli. The magnitude of responses to positive feedback stimuli decreased over the course of behavioral adaptation, in correlation with decreases in the amount of prediction error of action values. Therefore, these responses in medial prefrontal cells may signal the direction and amount of error in prediction of values of executed actions to specify the adjustment in subsequent action selections.     

Lateral habenula neurons signal errors in the prediction of reward information

Humans and animals have the ability to predict future events, which they cultivate by continuously searching their environment for sources of predictive information. However, little is known about the neural systems that motivate this behavior. We hypothesized that information-seeking is assigned value by the same circuits that support reward-seeking, such that neural signals encoding reward prediction errors (RPEs) include analogous information prediction errors (IPEs). To test this, we recorded from neurons in the lateral habenula, a nucleus that encodes RPEs, while monkeys chose between cues that provided different chances to view information about upcoming rewards. We found that a subpopulation of lateral habenula neurons transmitted signals resembling IPEs, responding when reward information was unexpectedly cued, delivered or denied. These signals evaluated information sources reliably, even when the monkey's decisions did not. These neurons could provide a common instructive signal for reward-seeking and information-seeking behavior.     

Sensory transduction and adaptation in inner and outer hair cells of the mouse auditory system

Auditory function in the mammalian inner ear is optimized by collaboration of two classes of sensory cells known as inner and outer hair cells. Outer hair cells amplify and tune sound stimuli that are transduced and transmitted by inner hair cells. Although they subserve distinct functions, they share a number of common properties. Here we compare the properties of mechanotransduction and adaptation recorded from inner and outer hair cells of the postnatal mouse cochlea. Rapid outer hair bundle deflections of about 0.5 micron evoked average maximal transduction currents of about 325 pA, whereas inner hair bundle deflections of about 0.9 micron were required to evoke average maximal currents of about 310 pA. The similar amplitude was surprising given the difference in the number of stereocilia, 81 for outer hair cells and 48 for inner hair cells, but may be reconciled by the difference in single-channel conductance. Step deflections of inner and outer hair bundles evoked adaptation that had two components: a fast component that consisted of about 60% of the response occurred over the first few milliseconds and a slow component that consisted of about 40% of the response followed over the subsequent 20-50 ms. The rate of the slow component in both inner and outer hair cells was similar to the rate of slow adaptation in vestibular hair cells. The rate of the fast component was similar to that of auditory hair cells in other organisms and several properties were consistent with a model that proposes calcium-dependent release of tension allows transduction channel closure.     

The relative importance of retinal error and prediction in saccadic adaptation

When saccades systematically miss their visual target, their amplitude adjusts, causing the position errors to be progressively reduced. Conventionally, this adaptation is viewed as driven by retinal error (the distance between primary saccade endpoint and visual target). Recent work suggests that the oculomotor system is informed about where the eye lands; thus not all "retinal error" is unexpected. The present study compared two error signals that may drive saccade adaptation: retinal error and prediction error (the difference between predicted and actual postsaccadic images). Subjects made saccades to a visual target in two successive sessions. In the first session, the target was extinguished during saccade execution if the amplitude was smaller (or, in other experiments, greater) than the running median, thereby modifying the average retinal error subjects experienced without moving the target during the saccade as in conventional adaptation paradigms. In the second session, targets were extinguished at the start of saccades and turned back on at a position that reproduced the trial-by-trial retinal error recorded in the first session. Despite the retinal error in the first and second sessions having been identical, adaptation was severalfold greater in the second session, when the predicted target position had been changed. These results argue that the eye knows where it lands and where it expects the target to be, and that deviations from this prediction drive saccade adaptation more strongly than retinal error alone.