Neural mechanisms of saccade target selection: gated accumulator model of the visual-motor cascade

We review a new computational model developed to understand how evidence about stimulus salience in visual search is translated into a saccade command. The model uses the activity of visually responsive neurons in the frontal eye field as evidence for stimulus salience that is accumulated in a network of stochastic accumulators to produce accurate and timely saccades. We discovered that only when the input to the accumulation process was gated could the model account for the variability in search performance and predict the dynamics of movement neuron discharge rates. This union of cognitive modeling and neurophysiology indicates how the visual-motor transformation can occur, and provides a concrete mapping between neuron function and specific cognitive processes.     

Manual responses and saccades in chronic and recovered hemispatial neglect: a study using visual search

Hemispatial neglect affects both the ability to respond to targets on the contralesional side of space and to programme saccades to such targets. In the current study, we looked in detail at saccade programming and manual reaction times (RTs) in a range of visual search tasks, in which task difficulty was systematically increased by changing the nature of the distractors. In condition 1, the target was presented with no distractors. In the other conditions, displays contained three distractors that were changed across conditions to manipulate similarity to the target and so task difficulty.We tested two neglect patients, one chronic, one recovered along with two RCVA control patients and 12 age-matched controls. Both neglect patients studied could successfully execute saccades into the neglected field when the target was presented alone. However, a dissociation emerged between the two patients when the target was presented with distractor items. Patient ERs first saccade to target performance in the three search conditions revealed clear effects of distractor type. In contrast for the recovered patient AF, the left/right difference was present for all search displays and appeared to be constant regardless of distractor type. This differential pattern of behaviour may reflect the different underlying neural causes of the neglect in these patients. In the current study, the measurement of saccades allowed the task to be fractionated, and thus, reveal the action of multiple mechanisms controlling saccades in search.     

Transfer of adaptation from visually guided saccades to averaging saccades elicited by double visual targets

The adaptive mechanisms that control the amplitude of visually guided saccades (VGS) are only partially elucidated. In this study, we investigated, in six human subjects, the transfer of VGS adaptation to averaging saccades elicited by the simultaneous presentation of two visual targets. The generation of averaging saccades requires the transformation of two representations encoding the desired eye displacement toward each of the two targets into a single representation encoding the averaging saccade (averaging programming site). We aimed to evaluate whether VGS adaptation acts upstream (hypothesis 1) or at/below (hypothesis 2) the level of averaging saccades programming. Using the double-step target paradigm, we simultaneously induced a backward adaptation of 17.5 degrees horizontal VGS and a forward adaptation of 17.5 degrees oblique VGS performed along the +/- 40 degrees directions relative to the azimuth. We measured the effects of this dual adaptation protocol on averaging saccades triggered by two simultaneous targets located at 17.5 degrees along the +/- 40 degrees directions. To increase the yield of averaging saccades, we instructed the subjects to move their eyes as fast as possible to an intermediate position between the two targets. We found that the amplitude of averaging saccades was smaller after VGS adaptation than before and differed significantly from that predicted by hypothesis 1, but not by hypothesis 2, with an adaptation transfer of 50%. These findings indicate that VGS adaptation largely occurs at/below the averaging saccade programming site. Based on current knowledge of the neural substrate of averaging saccades, we suggest that VGS adaptation mainly acts at the level of the superior colliculus or downstream.     

From prediction error to incentive salience: mesolimbic computation of reward motivation

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience.     

Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback

A simple working memory model based on recurrent network activation is proposed and its application to selection and reinforcement of an action is demonstrated as a solution to the temporal credit assignment problem. Reactivation of recent salient cue states is generated and maintained as a type of salience gated recurrently active working memory, while lower salience distractors are ignored. Cue reactivation during the action selection period allows the cue to select an action while its reactivation at the reward period allows the reinforcement of the action selected by the reactivated state, which is necessarily the action which led to the reward being found. A down-gating of the external input during the reactivation and maintenance prevents interference. A double winner-take-all system which selects only one cue and only one action allows the targeting of the cue–action allocation to be modified. This targeting works both to reinforce a correct cue–action allocation and to punish the allocation when cue–action allocations change. Here we suggest a firing rate neural network implementation of this system based on the basal ganglia anatomy with input from a cortical association layer where reactivations are generated by signals from the thalamus. Striatum medium spiny neurons represent actions. Auto-catalytic feedback from a dopamine reward signal modulates three-way Hebbian long term potentiation and depression at the cortical–striatal synapses which represent the cue–action associations. The model is illustrated by the numerical simulations of a simple example — that of associating a cue signal to a correct action to obtain reward after a delay period, typical of primate cue reward tasks. Through learning, the model shows a transition from an exploratory phase where actions are generated randomly, to a stable directed phase where the animal always chooses the correct action for each experienced state. When cue–action allocations change, we show that this is noticed by the model, the incorrect cue–action allocations are punished and the correct ones discovered.     

Spatial remapping of the visual world across saccades

Recent research has identified neurons in the visual system that remap their receptive fields before a saccade. The activity of these neurons may signal a prediction of postsaccadic visual input, derived from an efference copy of saccadic motor output. Such a prediction is often thought to underlie our perception of a stable visual world, by compensating for the shifts in retinal image that accompany each eye movement. Here we review the evidence, and conclude that prediction does not in fact play a significant role in maintaining visual stability. Instead, we consider a novel perspective in which the primary function of spatial remapping is to support three key nonperceptual processes: action control, sensorimotor adaptation and spatial memory.     

Changes in visual perception at the time of saccades

We frequently reposition our gaze by making rapid ballistic eye movements that are called saccades. Saccades pose problems for the visual system, because they generate rapid, large-field motion on the retina and change the relationship between the object position in external space and the image position on the retina. The brain must ignore the one and compensate for the other. Much progress has been made in recent years in understanding the effects of saccades on visual function and elucidating the mechanisms responsible for them. Evidence suggests that saccades trigger two distinct neural processes: (1) a suppression of visual sensitivity, specific to the magnocellular pathway, that dampens the sensation of motion and (2) a gross perceptual distortion of visual space in anticipation of the repositioning of gaze. Neurophysiological findings from several laboratories are beginning to identify the neural substrates involved in these effects.     

What and where information in the caudate tail guides saccades to visual objects

We understand the world by making saccadic eye movements to various objects. However, it is unclear how a saccade can be aimed at a particular object, because two kinds of visual information, what the object is and where it is, are processed separately in the dorsal and ventral visual cortical pathways. Here, we provide evidence suggesting that a basal ganglia circuit through the tail of the monkey caudate nucleus (CDt) guides such object-directed saccades. First, many CDt neurons responded to visual objects depending on where and what the objects were. Second, electrical stimulation in the CDt induced saccades whose directions matched the preferred directions of neurons at the stimulation site. Third, many CDt neurons increased their activity before saccades directed to the preferred objects and directions of the neurons in a free-viewing condition. Our results suggest that CDt neurons receive both "what" and "where" information and guide saccades to visual objects.     

The responses of visual cortical neurons encode differences across saccades

Primate vision consists mostly of periods of stable fixation separated by rapid saccadic eye movements. Each saccade brings a new scene onto the retina, and each new scene results in a burst of activity in the neurons of visual cortex. It might be expected that the activity of these neurons should only represent what is on the retina now, much as a video camera hooked up to a television only displays what the camera is currently pointed at. However, we show here that this is not the case. Recording from 25 primary visual cortical neurons in an awake primate demonstrated that the responses to the saccade-induced presentation of a stimulus within a neuron's receptive field (RF) are typically suppressed by the presence of a stimulus in the RF before the saccade. Flashing stimuli on with the eyes stationary showed, on average, suppressive effects of similar magnitude, suggesting that the mechanism is simple adaptation. However, while the mechanism may be simple, the implications for the operation of the visual system are not. The activity of visual cortical neurons does not represent just the current retinal image, but also the differences between the current retinal image and the previous one. These results suggest that the current approach of studying the visual system, which concentrates on determining the relationship between a single stimulus and a single response, may have to be modified to take into account the timing of retinal image changes that occurs in normal vision.     

Information flow related to visual search assessed using magnetoencephalography

The sequence of neural activation during a visual search task was investigated using magnetoencephalography and the source locations for the activations were analyzed using a single-dipole algorithm. Five components (M1-5) were detected at mean latencies of 110, 146, 196, 250 and 333 ms in both of two different stimulus conditions; a target popped out in one stimulus condition (pop-out), while it did not in the other condition (non-pop-out). Statistical analysis showed that the M3 amplitude was larger and the M5 latency was shorter in the pop-out condition than in the non-pop-out condition, while there was no difference in the other components between the conditions. Neural sources were localized in the calcarine sulcus (M1) and the posterior fusiform gyrus (M2) of the hemisphere contralateral to the stimuli, the intraparietal sulcus and the posterior superior temporal sulcus (M3) in either of the hemispheres, and the calcarine sulcus (M4) of the same hemisphere in which the early processing (M1 and M2) occurred. The criteria for source localization were not satisfied for M5. The results suggest that the processing for pop-out and non-pop-out stimuli share a common mechanism; after early feature processing in the occipital cortex (M1 and M2), visual information is processed in the parietal and temporal regions (M3) and then some of this information is fed back to the occipital cortex (M4).     

A model of active visual search with object-based attention guiding scan paths.

When a monkey searches for a colour and orientation feature conjunction target, the scan path is guided to target coloured locations in preference to locations containing the target orientation [Vision Res. 38 (1998b) 1805]. An active vision model, using biased competition, is able to replicate this behaviour. As object-based attention develops in extrastriate cortex, featural information is passed to posterior parietal cortex (LIP), enabling it to represent behaviourally relevant locations [J. Neurophysiol. 76 (1996) 2841] and guide the scan path. Attention evolves from an early spatial effect to being object-based later in the response of the model neurons, as has been observed in monkey single cell recordings. This is the first model to reproduce these effects with temporal precision and is reported here at the systems level allowing the replication of psychophysical scan paths.     

From intrusive thoughts to obsessions: The role of perceptions of responsibility, salience, and thought suppression

Salkovskis [1985. Obsessional-compulsive problems: A cognitive-behavioural analysis. Behaviour Research and Therapy, 23, 571–583.] hypothesized that intrusive thoughts are more likely to occur if the thought is salient for the individual, triggers feelings of responsibility and if the individual attempts to suppress the thought. The relationship between these three factors (responsibility, salience, thought suppression) and frequency of intrusive thoughts as well as anxiety were examined in the present study. One hundred female college students were led to believe that a snake had escaped from the cage in which it was housed. Half of the participants were led to believe that they were responsible for the snake's escape and half were not (Responsibility/No Responsibility). Half of the participants in each Responsibility condition were instructed to suppress thoughts of snakes during a stream of consciousness exercise and half were not (Thought Suppression vs. No Thought Suppression). All participants then completed a second stream of consciousness without suppression instructions. Salience groups were determined by a median split on a measure of snake fearfulness. Participants for whom the snake was a salient stimulus and who believed that they were responsible for the snake's escape had the highest frequency of snake thoughts during the second stream of consciousness task and also experienced the highest levels of state anxiety. Findings are discussed with respect to Salkovskis' model.     

From numbers to letters: Feedback regularization in visual word recognition

Word reading in alphabetic languages involves letter identification, independently of the format in which these letters are written. This process of letter ‘regularization’ is sensitive to word context, leading to the recognition of a word even when numbers that resemble letters are inserted among other real letters (e.g., M4TERI4L). The present study investigates the electrophysiological correlates of number-to-letter regularization by means of the masked priming paradigm: target words (MATERIAL) were preceded by fully alphabetic primes (MATERIAL), primes with letter-like numbers (M4T3R14L), or primes with unrelated numbers (M7T6R28L). ERPs revealed three subsequent effects. Around 150 ms the unrelated numbers condition elicited a positive effect, compared to the other two conditions, in the occipital electrodes. Then, target words preceded by primes with numbers elicited a more negative N200 in the same electrodes compared to the fully alphabetic condition. Finally, both alphabetic primes and letter-like numbers elicited a posterior positive component peaking around 260 ms compared to unrelated numbers. Source analysis for each electrophysiological effect revealed a similar early increase of activity in the left occipito-temporal pathway for alphabetic primes and primes with letter-like numbers. Around 200 ms, the orthographic interference due to the numerical values correlated with an increase of activity in parietal areas; finally, a recursive effect in the left occipital cortex was found, reflecting abstract letter activation. These results indicate that direct feedback interaction from word units strongly influences the activation of the letter units at a format-independent abstract level.     

Aberrant visual pathway development in albinism: From retina to cortex

Albinism refers to a group of genetic abnormalities in melanogenesis that are associated neuronal misrouting through the optic chiasm. We perform quantitative assessment of visual pathway structure and function in 23 persons with albinism (PWA) and 20 matched controls using optical coherence tomography (OCT), volumetric magnetic resonance imaging (MRI), diffusion tensor imaging and visual evoked potentials (VEP). PWA had a higher streamline decussation index (percentage of total tractography streamlines decussating at the chiasm) compared with controls (Z = ?2.24, p = .025), and streamline decussation index correlated weakly with inter‐hemispheric asymmetry measured using VEP (r = .484, p = .042). For PWA, a significant correlation was found between foveal development index and total number of streamlines (r = .662, p < .001). Significant positive correlations were found between peri‐papillary retinal nerve fibre layer thickness and optic nerve (r = .642, p < .001) and tract (r = .663, p < .001) width. Occipital pole cortical thickness was 6.88% higher (Z = ?4.10, p < .001) in PWA and was related to anterior visual pathway structures including foveal retinal pigment epithelium complex thickness (r = ?.579, p = .005), optic disc (r = .478, p = .021) and rim areas (r = .597, p = .003). We were unable to demonstrate a significant relationship between OCT‐derived foveal or optic nerve measures and MRI‐derived chiasm size or streamline decussation index. Our novel tractographic demonstration of altered chiasmatic decussation in PWA corresponds to VEP measured cortical asymmetry and is consistent with chiasmatic misrouting in albinism. We also demonstrate a significant relationship between retinal pigment epithelium and visual cortex thickness indicating that retinal pigmentation defects in albinism lead to downstream structural reorganisation of the visual cortex.     

Reprint of: Fear-enhanced visual search persists after amygdala lesions

Previous research has indicated that the amygdala is a critical neural substrate of the emotional modulation of attention. However, a recent case study suggests that the amygdala may not be essential for all types of emotion-attention interactions. In order to test this hypothesis, we assessed the visual-search performance of patients with unilateral amygdala lesions, matched controls, and medication-matched epilepsy patients with intact amygdalae. All participants completed a visual-search task consisting of trials in which (1) an emotional target was embedded among neutral distractors, (2) a neutral target was embedded among emotional distractors, or (3) a neutral target was embedded among neutral distractors. All participant groups, including those with amygdala lesions, detected emotional targets more efficiently than neutral targets. These data indicate that the amygdala is not necessary for emotion-guided visual search and suggest that other mechanisms beyond the amygdala help guide attention toward threatening stimuli.     

Fast responses to neglected targets in visual search reflect pre-attentive processes: an exploration of response times in visual neglect

AE is a patient who suffered a right hemisphere stroke resulting in visual neglect symptoms. In the first experiment, AE neglected a single visual target that was present in half of the trials and appeared in variable and unpredictable positions on the computer screen. The contrast of the target to the screen's background was also varied. AE demonstrated severe neglect for left-sided targets, and yet his RTs to targets reported incorrectly as absent were faster than correct rejections and even right-sided hits. AEs fast "neglect" responses seem to indicate that the target was detected but that he remained unaware of its presence. Counter intuitively, his fast misses got faster as the discriminability of the target decreased. The possibility that fast responses to neglected targets reflected a guessing strategy, used proportionally to the degree of uncertainty of a target presence, was examined. AEs fast misses were indeed faster at lower level of contrast of the stimulus, but his error rate did not tend to approach the chance level as the guessing model would predict. In a second experiment, AE searched for the letter Z, present on half of the trials, among variable sets of distractor letters. In one condition the distractors were all O's and therefore differed from the target by an elementary feature. In the other condition, the distractors were various letters that differed from the target by combinations of features. The key finding was that fast responses to neglected targets occurred only in the simple feature search task and not in the complex features (conjunction) task. We interpret these findings as indicating that AEs pre-attentive processing can detect pop-out targets on the left-hand side, but that the attentional search is faulty and is aborted early. Hence, the patient's attentional system has an "early start" when "pop-out" forms are present, but can also fail to "grab" the detected target; consequently, by not attending to a stimulus, the patient remains unaware of its presence and will quickly respond "no" to present targets.     

Hemispatial neglect and visual search: a large scale analysis

Visual search tasks have standardly been divided into two categories: those in which the target is detected through a serial, attention-driven search and those in which the target is detected rapidly in parallel and, apparently, without attentional processing. Several studies have examined this distinction in patients with hemispatial neglect with the clear prediction that the former, but not the latter, should be impaired. These studies, however, have proved inconclusive. We have addressed this issue in a large sample of patients with unilateral hemispheric infarcts to the left or right hemisphere. In addition to measuring the patients' performance on both types of visual search tasks, we documented the presence and severity of neglect and of visual field defects in these same individuals. Patients with brain-damage with or without accompanying neglect were impaired at searching for the contralateral target on both forms of visual search, relative to normal control subjects, although this deficit was magnified in individuals with neglect and was also exacerbated by the presence of hemianopia. This pattern was also more pronounced in individuals with right-than with left-hemisphere lesions. The findings not only clarify the contradictory neuropsychological data but also provide clear evidence for the involvement of attentional processing in all forms of visual search.     

Contribution of the retino-tectal pathway to visually guided saccades after lesion of the primary visual cortex in monkeys

Previous reports on 'blindsight' have shown that some patients with lesions of the primary visual cortex (V1) could localize visual targets in their scotoma with hand and/or eye movements without visual awareness. A role of the retino-tectal pathway on residual vision has been proposed but the direct evidence for this still remains sparse. To examine this possibility, we inactivated the superior colliculus (SC) of unilateral V1-lesioned monkeys using microinjections of muscimol, and analysed the effects on visually guided saccades. Following muscimol injections into the contralesional SC, the monkeys performed the visually guided saccade task with relatively minor deficits. The effects of ipsilesional SC inactivation were more severe. After injections, the monkeys failed to localize the target within the visual field represented at the injection site on the SC map. The effects of ipsilesional SC inactivation may result from sensory deficits, motor deficits or a combination of both. To examine these possibilities, we tested the effects of SC inactivation on the motor system by investigating spontaneous saccades. After inactivation of the ipsilesional SC, spontaneous saccades toward the injection site were not abolished, suggesting that impairment of visually guided saccades following inactivation of the ipsilesional SC could not be explained solely by a motor deficit and was primarily due to a visual deficit, presumably by interfering with processing in the superficial layer. We conclude that the retino-tectal pathway plays an essential role in residual vision after V1 lesion. The results suggest that this pathway may be involved in mediating unconscious vision in blindsight patients.