From knowing what to knowing where: modeling object-based attention with feedback disinhibition of activation

We propose a neural model of visual object-based attention in which the identity of an object is used to select its location in an array of objects. The model is based on neural activity observed in visual search tasks performed by monkeys. In the model, the identity of the object (target) is selected in the higher areas of the ventral stream by means of a cue. Feedback activation from these higher areas carries information about the identity of the target to the (lower) retinotopic areas of the ventral stream. In these areas, the feedback activation interacts with feedforward activation produced by the object array. The interaction occurs in local microcircuits, and results in a selective activation on locations in the retinotopic areas of the visual stream that correspond to the location of the target in the object array. The selective activation consists of a form of gain control, produced by disinhibition. Transmitted to the dorsal stream, this activation directs spatial attention to the location of the target. In this way, an action directed at the target can be generated.     

Neural correlates of object-based attentional selection in human cortex

Humans can attend to different objects independent of their spatial locations. While selecting an object has been shown to modulate object processing in high-level visual areas in occipitotemporal cortex, where/how behavioral importance (i.e., priority) for objects is represented is unknown. Here we examined the patterns of distributed neural activity during an object-based selection task. We measured brain activity with functional magnetic resonance imaging (fMRI), while participants viewed two superimposed, dynamic objects (left- and right-pointing triangles) and were cued to attend to one of the triangle objects. Enhanced fMRI response was observed for the attention conditions compared to a neutral condition, but no significant difference was found in overall response amplitude between two attention conditions. By using multi-voxel pattern classification (MVPC), however, we were able to distinguish the neural patterns associated with attention to different objects in early visual cortex (V1 to hMT+) and lateral occipital complex (LOC). Furthermore, distinct multi-voxel patterns were also observed in frontal and parietal areas. Our results demonstrate that object-based attention has a wide-spread modulation effect along the visual hierarchy and suggest that object-specific priority information is represented by patterned neural activity in the dorsal frontoparietal network.     

The role of early visual cortex in visual integration: a neural model of recurrent interaction

This paper presents a model on the potential functional roles of the early visual cortex in the primate visual system. Our hypothesis is that early visual areas, such as V1, are important for continual interaction among various higher order visual areas during visual processing. The interaction is mediated by recurrent connections between higher order visual areas and V1, manifested in the long-latency context-sensitive activities often observed in neurophysiological experiments, and is responsible for the re-integration of information analysed by the higher visual areas. Specifically, we considered the case of integrating 'what' and 'where' information from the ventral and dorsal streams. We found that such a cortical architecture provides simple solutions and fresh insights into the problems of attentional routing and visual search. The computational viability of this architecture was tested by simulating a large-scale neural dynamical network.     

Integration of distributed cortical systems by reentry: a computer simulation of interactive functionally segregated visual areas

A computer model based on visual cortex has been constructed to analyze how the operations of multiple, functionally segregated cortical areas can be coordinated and integrated to yield a unified perceptual response. We propose that cortical integration arises through the process of reentry--the ongoing, parallel, recursive signaling between separate maps along ordered anatomical connections. To test the efficacy of this reentrant cortical integration (RCI) model, we have carried out detailed computer simulations of 3 interconnected cortical areas in the striate and extrastriate cortex of the macaque. The simulated networks contained a total of over 222,000 units and 8.5 million connections. The 3 modeled areas, called VOR, VOC, and VMO, incorporate major anatomical and physiological properties of cortical areas V1, V3, and V5 but are vastly simplified compared with monkey visual cortex. Simulated area VOR contains both orientation and directionally selective units; simulated area VMO discriminates the direction of motion of arbitrarily oriented objects; and simulated area VOC responds to both luminance and occlusion boundaries in the stimulus. Area VOC is able to respond to illusory contours (Kanizsa, 1979) by means of the same neural architecture used for the discrimination of occlusion boundaries. This architecture also generates responses to structure-from-motion by virtue of reentrant connections from VMO to VOC. The responses of the simulated networks to these illusions are consistent with the perceptual responses of humans and other species presented with these stimuli. The networks also respond in a consistent manner to a novel illusion that combines illusory contours and structure-from-motion. The response synthesized to this combined illusion provides a strong argument supporting the need for a recursive reentrant process in the cortex. Functional integration of the simulated areas in the RCI model were found to depend upon the combined action of 3 reentrant processes: (1) conflicting responses among segregated areas are competitively eliminated, (2) outputs of each area are used by other areas in their own operations, and (3) outputs of an area are "reentered" back to itself (through lower areas) and can thus be used iteratively to synthesize responses to complex or illusory stimuli. Transection of the reentrant connections selectively abolished these integrative processes and led to failure of figural synthesis. The proposed model of reentry suggests a basis for understanding how multiple visual areas as well as other cortical areas may be integrated within a distributed system.     

Deconstructing the architecture of dorsal and ventral attention systems with dynamic causal modeling

Attentional orientation to a spatial cue and reorientation-after invalid cueing-are mediated by two distinct networks in the human brain. A bilateral dorsal frontoparietal network, comprising the intraparietal sulcus (IPS) and the frontal eye fields (FEF), controls the voluntary deployment of attention and may modulate visual cortex in preparation for upcoming stimulation. In contrast, reorienting attention to invalidly cued targets engages a right-lateralized ventral frontoparietal network comprising the temporoparietal junction (TPJ) and ventral frontal cortex. The present fMRI study investigated the functional architecture of these two attentional systems by characterizing effective connectivity during lateralized orienting and reorienting of attention, respectively. Subjects performed a modified version of Posner's location-cueing paradigm. Dynamic causal modeling (DCM) of regional responses in the dorsal and ventral network, identified in a conventional (SPM) whole-brain analysis, was used to compare different functional architectures. Bayesian model selection showed that top-down connections from left and right IPS to left and right visual cortex, respectively, were modulated by the direction of attention. Moreover, model evidence was highest for a model with directed influences from bilateral IPS to FEF, and reciprocal coupling between right and left FEF. Invalid cueing enhanced forward connections from visual areas to right TPJ, and directed influences from right TPJ to right IPS and IFG (inferior frontal gyrus). These findings shed further light on the functional organization of the dorsal and ventral attentional network and support a context-sensitive lateralization in the top-down (backward) mediation of attentional orienting and the bottom-up (forward) effects of invalid cueing.     

Object-based attention and visual area LO

We investigated the neural basis of so-called “object-based attention” by examining patient D.F., who has visual form agnosia caused by bilateral damage to the lateral occipital (LO) area of the ventral visual stream. We tested D.F.’s object-based attention in two ways. In the first experiment, we used a spatial cueing procedure to compare the costs associated with shifting attention within versus between two separate outline figures. D.F. did not show the normal advantage of within-object over between-object attention shifts. In the second experiment, we used a complementary paradigm in which two separate stimuli, presented either on the same or on different objects, have to be identified as the same or different, We found no evidence for the normal pattern of superior performance for within versus between figure comparisons. In a third experiment, we checked that D.F. showed normal shift costs for invalid as opposed to valid cueing in a standard Posner spatial attention task. In a final experiment, we compared horizontal versus vertical attention shifting in group of healthy controls without the presence of outline rectangles, and found that their pattern of shift costs was indistinguishable from that seen in D.F. when the rectangles were present (Experiment 1). We conclude that whilst D.F. has a normal spatial orienting system this is completely uninfluenced by object structure. We suggest that area LO may mediate form processing precisely at the stage where visual representations normally influence the spread of attention.     

Non-homogenous neural networks with chaotic recursive nodes: connectivity and multi-assemblies structures in recursive processing elements architectures.

This paper addresses recurrent neural architectures based on bifurcating nodes that exhibit chaotic dynamics, with local dynamics defined by first order parametric recursions. In the studied architectures, logistic recursive nodes interact through parametric coupling, they self organize, and the network evolves to global spatio-temporal period-2 attractors that encode stored patterns. The performance of associative memories arrangements is measured through the average error in pattern recovery, under several levels of prompting noise. The impact of the synaptic connections magnitude on architecture performance is analyzed in detail, through pattern recovery performance measures and basin of attraction characterization. The importance of a planned choice of the synaptic connections scale in RPEs architectures is shown. A strategy for minimizing pattern recovery degradation when the number of stored patterns increases is developed. Experimental results show the success of such strategy. Mechanisms for allowing the studied associative networks to deal with asynchronous changes in input patterns, and tools for the interconnection between different associative assemblies are developed. Finally, coupling in heterogeneous assemblies with diverse recursive maps is analyzed, and the associated synaptic connections are equated.     

A Neural Dynamic Model Perceptually Grounds Nested Noun Phrases

We present a neural dynamic model that perceptually grounds nested noun phrases, that is, noun phrases that contain further (possibly also nested) noun phrases as parts. The model receives input from the visual array and a representation of a noun phrase from language processing. It organizes a search for the denoted object in the visual scene. The model is a neural dynamic architecture of interacting neural populations which has clear interfaces with perceptual processes. It solves a set of theoretical challenges, including the problem of keeping a nested structure in short-term memory in a way that solves the problem of 2 and massive binding problem emphasized by Jackendoff. The model organizes a search for the objects that are referenced in that structure. We motivate the model, demonstrate simulation results, and discuss how it differs from related models.     

Object recognition by artificial cortical maps.

Object recognition is one of the most important functions of the human visual system, yet one of the least understood, this despite the fact that vision is certainly the most studied function of the brain. We understand relatively well how several processes in the cortical visual areas that support recognition capabilities take place, such as orientation discrimination and color constancy. This paper proposes a model of the development of object recognition capability, based on two main theoretical principles. The first is that recognition does not imply any sort of geometrical reconstruction, it is instead fully driven by the two dimensional view captured by the retina. The second assumption is that all the processing functions involved in recognition are not genetically determined or hardwired in neural circuits, but are the result of interactions between epigenetic influences and basic neural plasticity mechanisms. The model is organized in modules roughly related to the main visual biological areas, and is implemented mainly using the LISSOM architecture, a recent neural self-organizing map model that simulates the effects of intercortical lateral connections. This paper shows how recognition capabilities, similar to those found in brain ventral visual areas, can develop spontaneously by exposure to natural images in an artificial cortical model.     

Neural associative memories for the integration of language,vision and action in an autonomous agent

Language understanding is a long-standing problem in computer science. However, the human brain is capable of processing complex languages with seemingly no difficulties. This paper shows a model for language understanding using biologically plausible neural networks composed of associative memories. The model is able to deal with ambiguities on the single word and grammatical level. The language system is embedded into a robot in order to demonstrate the correct semantical understanding of the input sentences by letting the robot perform corresponding actions. For that purpose, a simple neural action planning system has been combined with neural networks for visual object recognition and visual attention control mechanisms.     

Insula of the old world monkey. III: Efferent cortical output and comments on function

The insula sends neural efferents to cortical areas from which it receives reciprocal afferent projections. A collective consideration of afferents and efferents indicates that the insula has connections with principal sensory areas in the olfactory, gustatory, somesthetic (SI and SII), and auditory AI and AII) modalities. There are additional connections with association areas for the visual (TEm), auditory (supratemporal plane), and somesthetic (posterior parietal cortex) modalities; with parameter cortex (area 6 and perhaps MII); with polymodal association cortex; and with a wide range of paralimbic areas in the orbital, temporopolar, and cingulate areas. The topographic distribution of these connections suggests that the posterodorsal insula is specialized for auditory-somesthetic-skeletomotor function whereas the anteroventral insula is related to olfactory-gustatory-autonomic function. Most of the insula, especially its anteroventral portions, have extensive interconnections with limbic structures. Through its connections with the amygdala, the insula provides a pathway for somatosen-sory, auditory, gustatory, olfactory, and visceral sensations to reach the limbic system. The cortical areas connected with the granular sector of the insula are also granular in architecture whereas virtually all the connections of the agranular insula arise from allocortical, agranular, or dysgranular areas. Thus, there is a correspondence between the architecture of insular sectors and the areas with which they have connections. The insula is heavily interconnected with temporopolar and lateral orbital areas. Furthermore, many cortical connections of the lateral orbital cortex are quite similar to those of the insula. These common connectivity patterns support the conclusion, based on architectonic observations, that the insulo-orbito-tempo-ropolar component of the paralimbic brain should be considered as an integrated unit of cerebral organization.     

The power of connectivity: Identity preserving transformations on visual streams in the spike domain

We investigate neural architectures for identity preserving transformations (IPTs) on visual stimuli in the spike domain. The stimuli are encoded with a population of spiking neurons; the resulting spikes are processed and finally decoded. A number of IPTs are demonstrated including faithful stimulus recovery, as well as simple transformations on the original visual stimulus such as translations, rotations and zoomings. We show that if the set of receptive fields satisfies certain symmetry properties, then IPTs can easily be realized and additionally, the same basic stimulus decoding algorithm can be employed to recover the transformed input stimulus. Using group theoretic methods we advance two different neural encoding architectures and discuss the realization of exact and approximate IPTs. These are realized in the spike domain processing block by a “switching matrix” that regulates the input/output connectivity between the stimulus encoding and decoding blocks. For example, for a particular connectivity setting of the switching matrix, the original stimulus is faithfully recovered. For other settings, translations, rotations and dilations (or combinations of these operations) of the original video stream are obtained. We evaluate our theoretical derivations through extensive simulations on natural video scenes, and discuss implications of our results on the problem of invariant object recognition in the spike domain.     

The contribution of TMS–EEG coregistration in the exploration of the human cortical connectome

Recent developments in neuroscience have emphasised the importance of integrated distributed networks of brain areas for successful cognitive functioning. Our current understanding is that the brain has a modular organisation in which segregated networks supporting specialised processing are linked through a few long-range connections, ensuring processing integration. Although such architecture is structurally stable, it appears to be flexible in its functioning, enabling long-range connections to regulate the information flow and facilitate communication among the relevant modules, depending on the contingent cognitive demands. Here we show how insights brought by the coregistration of transcranial magnetic stimulation and electroencephalography (TMS–EEG) integrate and support recent models of functional brain architecture. Moreover, we will highlight the types of data that can be obtained through TMS–EEG, such as the timing of signal propagation, the excitatory/inhibitory nature of connections and causality. Last, we will discuss recent emerging applications of TMS–EEG in the study of brain disorders.     

Selective Attention and Consciousness: Investigating Their Relation Through Computational Modelling

The present study aimed at investigating the possible connection between conscious awareness and attention through the implementation of a neurocomputational model of visual selective attention. The development of the model was based on recent neurophysiological findings that document the synchronization of neural activity in cortical areas of the brain and the presence of competitive interactions among stimuli at the early stages of visual processing. The model was used to simulate the findings of a behavioural experiment conducted by Naccache et al. in Psychol Sci 13:416–424 (2002), which have sparked a debate on the possible links between attention and consciousness. The model reproduced closely the pattern of the behavioural data while incorporating mechanisms that take into account the neural activity representing the early visual processing of stimuli and the effects of top–down attention. Thus, by adopting a computational approach, we present a possible explanation of the findings at the neural level of information processing. The implications of these findings for the relation between attentional processes and conscious awareness are discussed.     

Cue validity and object-based attention

In a previous study, Egly, Driver, and Rafal (1994) observed both space- and object-based components of visual selective attention. However, the mechanisms underlying these two components and the relationship between them are not well understood. In the present research, with a similar paradigm, these issues were addressed by manipulating cue validity. Behavioral results indicated the presence of both space- and object-based components under high cue validity, similar to the results of Egly et al.'s study. In addition, under low cue validity, the space-based component was absent, whereas the object-based component was maintained. Further event-related potential results demonstrated an object-based effect at a sensory level over the posterior areas of brain, and a space-based effect over the anterior region. The present data suggest that the space- and object-based components reflect mainly voluntary and reflexive mechanisms, respectively.     

Disconnection syndromes of basal ganglia, thalamus, and cerebrocerebellar systems

Disconnection syndromes were originally conceptualized as a disruption of communication between different cerebral cortical areas. Two developments mandate a re-evaluation of this notion. First, we present a synopsis of our anatomical studies in monkey elucidating principles of organization of cerebral cortex. Efferent fibers emanate from every cortical area, and are directed with topographic precision via association fibers to ipsilateral cortical areas, commissural fibers to contralateral cerebral regions, striatal fibers to basal ganglia, and projection subcortical bundles to thalamus, brainstem and/or pontocerebellar system. We note that cortical areas can be defined by their patterns of subcortical and cortical connections. Second, we consider motor, cognitive and neuropsychiatric disorders in patients with lesions restricted to basal ganglia, thalamus, or cerebellum, and recognize that these lesions mimic deficits resulting from cortical lesions, with qualitative differences between the manifestations of lesions in functionally related areas of cortical and subcortical nodes. We consider these findings on the basis of anatomical observations from tract tracing studies in monkey, viewing them as disconnection syndromes reflecting loss of the contribution of subcortical nodes to the distributed neural circuits. We introduce a new theoretical framework for the distributed neural circuits, based on general, and specific, principles of anatomical organization, and on the architecture of the nodes that comprise these systems. We propose that neural architecture determines function, i.e., each architectonically distinct cortical and subcortical area contributes a unique transform, or computation, to information processing; anatomically precise and segregated connections between nodes define behavior; and association fiber tracts that link cerebral cortical areas with each other enable the cross-modal integration required for evolved complex behaviors. This model enables the formulation and testing of future hypotheses in investigations using evolving magnetic resonance imaging techniques in humans, and in clinical studies in patients with cortical and subcortical lesions.     

Sensory suppression during shifts of attention between surfaces in transparent motion

During transparent motion, attention to changes in the direction of one illusory surface will impede recognition of a similar event affecting the other surface if both are close together in time. This is a form of object-based attentional blink (AB). Here, we show that this AB is related to a smaller N200 response to the change in direction and that the response is even smaller for trials on which the subject makes mistakes compared to those with correct responses consistent with signal detection theory models. The variation of N200 associated with the AB can be modeled by an attenuation of current sources estimated in visual extrastriate cortex. These results suggest that the AB in the transparent motion paradigm is due to the suppression of sensory signals in early visual areas.     

Searching with unilateral neglect

We address two longstanding conflicts in the visual search and unilateral neglect literature by studying feature and conjunction search performance of neglect patients using laterally presented search arrays. The first issue relates to whether feature search is performed independently of attention, or rather requires "spread attention." If feature search is "preattentive," it should survive neglect. However, we find neglect effects for both feature and conjunction search, suggesting that feature search, too, has an attentional requirement. The second controversy refers to the space- or object-based nature of neglect following unilateral right-hemisphere parietal lobe damage. If neglect were a purely spatial phenomenon, then we would expect no detriment in performance in the right (nonneglect) field, and diminished performance for the whole left (neglect) field. On the other hand, if neglect were purely object-based, we would expect diminished performance on the left side of the search array, irrespective of its location in the visual field. We now demonstrate a combination of strong object-based and space-based neglect effects for conjunction search with laterally placed element arrays, suggesting that these two mechanisms work in tandem.     

An embodied biologically constrained model of foraging: from classical and operant conditioning to adaptive real-world behavior in DAC-X

Animals successfully forage within new environments by learning, simulating and adapting to their surroundings. The functions behind such goal-oriented behavior can be decomposed into 5 top-level objectives: ‘how’, ‘why’, ‘what’, ‘where’, ‘when’ (H4W). The paradigms of classical and operant conditioning describe some of the behavioral aspects found in foraging. However, it remains unclear how the organization of their underlying neural principles account for these complex behaviors. We address this problem from the perspective of the Distributed Adaptive Control theory of mind and brain (DAC) that interprets these two paradigms as expressing properties of core functional subsystems of a layered architecture. In particular, we propose DAC-X, a novel cognitive architecture that unifies the theoretical principles of DAC with biologically constrained computational models of several areas of the mammalian brain. DAC-X supports complex foraging strategies through the progressive acquisition, retention and expression of task-dependent information and associated shaping of action, from exploration to goal-oriented deliberation. We benchmark DAC-X using a robot-based hoarding task including the main perceptual and cognitive aspects of animal foraging. We show that efficient goal-oriented behavior results from the interaction of parallel learning mechanisms accounting for motor adaptation, spatial encoding and decision-making. Together, our results suggest that the H4W problem can be solved by DAC-X building on the insights from the study of classical and operant conditioning. Finally, we discuss the advantages and limitations of the proposed biologically constrained and embodied approach towards the study of cognition and the relation of DAC-X to other cognitive architectures.