Implicit location encoding via stored representations of familiar objects: neuropsychological evidence

We report data on the visual localisation ability of a patient with Balint's syndrome, GK. We show that, with relatively long exposures of displays, GK is better able to judge the spatial relations between separate objects (a "between-object judgement") than the spatial relations between a part and a whole object (a "within-object judgement") (Experiments 1-3). This dissociation occurred even when the same stimulus was used for both judgements, and the task instructions biased GK to parse the stimulus as either a single or as two separate objects (Experiments 2 and 6). However, when he could use a stored representation to make a within-object judgement, then performance was better than on a comparable spatial judgement of the relations between two separate objects (Experiments 4-7). The data demonstrate that stored representations of objects can support the spatial coding of parts to perceptual wholes. In the absence of stored representations, part-whole relations must be explicitly coded by attention, a process that is impaired in this patient.     

Evidence from unilateral visual neglect

Abstract

We report the case of a patient, JR, who manifests left neglect in reading single words but right neglect in copying and drawing tasks. We show that left neglect is not confined to reading, but is also found in picture naming and in line-bisection tasks. In each of these tasks, JR neglected the right side of multiple stimulus displays. We show that neglect is manifest on the left or right side as a function of whether visual stimuli are encoded as parts of a single perceptual object or as separate perceptual objects. JR's left neglect in reading words can be switched to right neglect by having him read aloud all the letters in the string. He also showed left neglect in a search task in which the visual elements configure into a coherent object, but right neglect in displays using similar elements which did not so configure. JR's case demonstrates that the visual system employs separate representations to encode the spatial relations between the parts of single objects (within-object spatial coding) and between separate perceptual objects (between-object spatial coding). These two forms of spatial representation are encoded independently and in parallel.     

Attending to space within and between objects: Implications from a patient with Balint's syndrome

Neuropsychological conditions such as Balint's syndrome have shown that perceptual organization of parts into a perceptual unit can be dissociated from the ability to localize objects relative to each other. Neural mechanisms that code the spatial structure within individual objects or words may seem to be intact, while between-object structure is compromised. Here we investigate the nature of within-object spatial processing in a patient with Balint's syndrome (RM). We suggest that within-object spatial structure can be determined (a) directly by explicit spatial processing of between-part relations, mediated by the same dorsal pathway as between-object spatial relations; or (b) indirectly by the discrimination of object identities, which may involve implicit processing of between-part relations and which is probably mediated by the ventral system. When this route is ruled out, by testing discrimination of differences in part location that do not change the identity of the object, we find no evidence of explicit within-object spatial coding in a patient without functioning parietal lobes.     

Attending to space within and between objects: Implications from a patient with Balint's syndrome

Neuropsychological conditions such as Balint's syndrome have shown that perceptual organization of parts into a perceptual unit can be dissociated from the ability to localize objects relative to each other. Neural mechanisms that code the spatial structure within individual objects or words may seem to be intact, while between-object structure is compromised. Here we investigate the nature of within-object spatial processing in a patient with Balint's syndrome (RM). We suggest that within-object spatial structure can be determined (a) directly by explicit spatial processing of between-part relations, mediated by the same dorsal pathway as between-object spatial relations; or (b) indirectly by the discrimination of object identities, which may involve implicit processing of between-part relations and which is probably mediated by the ventral system. When this route is ruled out, by testing discrimination of differences in part location that do not change the identity of the object, we find no evidence of explicit within-object spatial coding in a patient without functioning parietal lobes.     

Routes to action: Evidence from apraxia

We report the case of a right-handed patient (CD), with a unilateral lesion of the left parietal lobe, who was unimanually apraxic to visually presented objects. In particular, CD was unable to initiate learned gestures to seen objects with his right hand, even though he was able to perform the same gestures with his left hand (and was often able to perform right-hand gestures subsequently). CD could perform right-handed gestures when the objects were not present and he was given their names, and his performance with seen objects improved when he was given pairs of objects. From CD's performance we argue that gestures to visually presented objects are normally based on co-operation between stored knowledge about the class of gesture to use, and directly computed visual representations that provide the spatial co-ordinates for action. We suggest that CD's problem is due to his having an impaired “route” from vision to action that selectively impairs right-hand actions for him.     

Developmental disorders of verbal short-term memory and their relation to sentence comprehension: A reply to Howard and Butterworth

The effects of lexical knowledge on extinction were examined in a patient with bilateral parietal lesions and left extinction under double simultaneous stimulation: GK. GK was bilaterally presented with two letters that could form either a word or a nonword. In Experiments 1–3, the task was to identify each letter. GK showed better identification of left-side letters in words than in nonwords, whilst the identification of left-side letters in nonwords was worse than that of single letters presented in the same spatial positions (i.e., there was a word superiority effect under conditions in which extinction occurred). This lexical effect on completely correct responses tended to be larger for words with lower-case letters (Experiments 2 and 3) than for words with upper-case letters (Experiment 1). Different results arose when detection was measured. When letters could group by proximity and common contrast polarity, no word superiority effect was apparent. However, a word superiority effect re-emerged when low-level grouping was reduced by using letters with opposite contrast polarity (one white and one black on a grey background). The results are discussed in terms of the impact of different factors on selection in detection and identification tasks, and in terms of the modulatory roles of familiar form and stored knowledge on visual selection.     

Visual selection and action in Balint's syndrome

We examined the relations between attention and action in a patient with Balint's syndrome following bilateral damage involving the parietal lobes. The first two experiments used prolonged stimulus exposures and showed that unlike normals, the patient GK made independent actions to bilateral stimuli, even when explicitly instructed to make coordinated reaches. In contrast, bimanual actions to a single stimulus were coordinated in time. Experiments 3 and 4 employed reduced stimulus exposures to produce visual extinction. We found comparable extinction effects in detection and action, along with improved bimanual movements when the stimuli were grouped by collinearity and surface contrast, plus better coordination with bilateral stimuli proximity. The results are discussed in relation to the common role of visual selection processes in perception and action.     

Visual selection and action in Balint's syndrome

We examined the relations between attention and action in a patient with Balint's syndrome following bilateral damage involving the parietal lobes. The first two experiments used prolonged stimulus exposures and showed that unlike normals, the patient GK made independent actions to bilateral stimuli, even when explicitly instructed to make coordinated reaches. In contrast, bimanual actions to a single stimulus were coordinated in time. Experiments 3 and 4 employed reduced stimulus exposures to produce visual extinction. We found comparable extinction effects in detection and action, along with improved bimanual movements when the stimuli were grouped by collinearity and surface contrast, plus better coordination with bilateral stimuli proximity. The results are discussed in relation to the common role of visual selection processes in perception and action.     

Different patterns of spoken and written word comprehension deficit in aphasic stroke patients

This study presents neuropsychological evidence for differences in the semantic representations underpinning spoken and written word comprehension. Potential modality-based discrepancies in the semantic system were examined by testing whether spoken word (auditory–verbal input) and written word (visual–verbal input) comprehension exhibited the same effect profile on variables typically used to distinguish so-called access and storage disorders (e.g., response consistency, sensitivity to item frequency). The study was based on the premise that damage to a common set of semantic representations should have an equivalent impact upon comprehension performance irrespective of input modality, whereas damage to partially dissociable semantic representations may give rise to different qualities of deficit (access/storage) in the comprehension of stimuli presented in different input modalities (spoken/written). The study involved two patients with global aphasia following left middle cerebral artery stroke (F.B.I. and H.O.P.). The two patients showed matched performance on conventional tests of single word comprehension with clear evidence of semantic impairment for stimuli presented in both the spoken and written input modalities. However, in H.O.P., spoken and written word comprehension was affected in the same way by variations in stimulus category, frequency, and multiple stimulus presentations, whilst in F.B.I., there were clear differences between input modalities with all three variables. More specifically, F.B.I.'s written word comprehension was significantly affected by category (living?>?nonliving) and frequency (high?>?low) but not multiple presentations (single?=?multiple), more consistent with degradation of stored representations (storage deficit). By contrast, his spoken word comprehension was unaffected by category (living?=?nonliving) and frequency (high?=?low) but was affected by multiple presentations (single?>?multiple; serial position effects), more consistent with impaired access to stored representations (access deficit). These spoken/written input modality differences were observed on tasks matched closely for output modality, stimulus identity, and executive control requirements. It is argued that subtle differences in comprehension performance for stimuli presented in different input modalities may reflect damage to multimodal representations, which are intermediate between unimodal and amodal representations on a continuum of convergence within the semantic system. These ideas are discussed in the context of existing “distributed-only”, “distributed-plus-convergence”, or “distributed-plus-hub” models of conceptual knowledge.     

THE PERCEPTION OF SPATIAL RELATIONS IN A PATIENT WITH VISUAL FORM AGNOSIA

It is generally believed that neuropsychological patients presenting with visual agnosia, a deficit on object perception/recognition, have suffered damage to the ventral visual cortical pathway (Milner & Goodale, 1995). Rarely has the ability of such patients to perceive the spatial location of objects been investigated-perhaps because “spatial vision” is thought by some researchers to be mediated exclusively by the dorsal visual cortical pathway. Here we present data on spatial perception in a patient DF, who has a profound visual form agnosia. DF and two control subjects were required to make a copy of the spatial arrangement of a target display of five differently coloured circular tokens using a duplicate set of the same tokens. Spatial performance was analysed in two ways: (1) relative location measured the ability to reconstruct the relative spatial relations between the tokens such as left versus right, above versus below, and nearer versus farther; (2) absolute location measured the exact displacement in millimetres of each token's copied position relative to its true location. DF was able to copy some of relative location relations between the tokens although her abilityto do so was not nearly as accurate as that of the control subjects. Nevertheless, DF's limited appreciation of relative location was enough to enable her to discriminate rather well between spatial patterns of tokens. She could not, however, reconstruct the absolute distance relations between the tokens and showed large displacements of token position compared to the control subjects. Interestingly, although Df was not “normal” in her ability to appreciate the allocentric spatial relations between the locations of the tokens relative to one another, she could accurately process token location egocentrically (i.e. relative to her own body and hand position). Thus, like controls, she was perfectly able to point to and touch all the tokens in an array. These results demonstrate deficits in the ability to perceive spatial relations between objects in a patient with visual form agnosia and suggest that the ventral steam also plays a functional role in spatial vision, particularly allocentric spatial vision.     

Object identification in simultanagnosia: When wholes are not the sum of their parts

We examined object identification in two simultanagnosic patients, ES and GK. We show that the patients tended to identify animate objects more accurately than inanimate objects (Experiments 1 and 4). The patients also showed relatively good identification of objects that could be recognised from their global shape, but not objects whose recognition depended on their internal detail (Experiment 2). Indeed, the presence of local segmentation cues disrupted global identification (Experiment 3). Identification was aided, though, by the presence of surface colour and texture (Experiment 4). We suggest that the patients could derive global representations of objects that served to recognise animate items. In contrast, they were impaired at coding parts-based representations for the identification of inanimate objects.     

Within-object and between-object coding deficits in drawing production

We examine how perceptual grouping influenced drawing errors in two patients with constructional apraxia. The patients copied simple geometric stimuli, formed from right-angled elements of variable size arranged as the corners of a square. Grouping was systematically manipulated by altering the closure between these local parts. The patients also made perceptual judgments about similar figures. Despite relatively intact perceptual discrimination of the stimuli, both patients produced errors in drawings relative to controls. When local elements were arranged as a square, patient ECR produced some shape errors but showed a marked tendency to complete across gaps between elements. In contrast, patient RA made no such completion errors but placed the elements so that they were poorly spatially located. When grouping was reduced, ECR completed fewer gaps and RA showed improved spatial localization. These symptoms suggest impairments of perceptuomotor representations of object relations. ECR's deficit appeared to be in between-object coding—she often failed to separate the individual elements when they formed a strong perceptual whole. In contrast, RA's deficit appeared to be in within-object coding—spatial localization of elements was impaired when they shared strong grouping cues. His performance only improved when the strength of the grouping between the elements was reduced. These findings demonstrate the coding of different forms of spatial relation in the perceptual-motor domain.     

The processing of perceptual similarity with different features or spatial relations as revealed by P2/P300 amplitude

Visual features such as “color” and spatial relations such as “above” or “beside” have complex effects on similarity and difference judgments. We examined the relative impact of features and spatial relations on similarity and difference judgments via ERPs in an S1–S2 paradigm. Subjects were required to compare a remembered geometric shape (S1) with a second one (S2), and made a “high” or “low” judgment of either similarity or difference in separate blocks of trials. We found three main differences that suggest that the processing of features and spatial relations engages distinct neural processes. The first difference is a P2 effect in fronto-central regions which is sensitive to the presence of a feature difference. The second difference is a P300 in centro-parietal regions that is larger for difference judgments than for similarity judgments. Finally, the P300 effect elicited by feature differences was larger relative to spatial relation differences. These results supported the view that similarity judgments involve structural alignment rather than simple feature and relation matches, and furthermore, indicate the similarity judgment could be divided into three phases: feature or relation comparison (P2), structural alignment (P3 at 300–400 ms), and categorization (P3 at 450–550 ms).     

Lexical recovery from extinction: Interactions between visual form and stored knowledge modulate visual selection

The effects of lexical knowledge on extinction were examined in a patient with bilateral parietal lesions and left extinction under double simultaneous stimulation: GK. GK was bilaterally presented with two letters that could form either a word or a nonword. In Experiments 1-3, the task was to identify each letter. GK showed better identification of left-side letters in words than in nonwords, whilst the identification of left-side letters in nonwords was worse than that of single letters presented in the same spatial positions (i.e., there was a word superiority effect under conditions in which extinction occurred). This lexical effect on completely correct responses tended to be larger for words with lower-case letters (Experiments 2 and 3) than for words with upper-case letters (Experiment 1). Different results arose when detection was measured. When letters could group by proximity and common contrast polarity, no word superiority effect was apparent. However, a word superiority effect re-emerged when low-level grouping was reduced by using letters with opposite contrast polarity (one white and one black on a grey background). The results are discussed in terms of the impact of different factors on selection in detection and identification tasks, and in terms of the modulatory roles of familiar form and stored knowledge on visual selection.     

Object identification in simultanagnosia: When wholes are not the sum of their parts

Abstract

We examined object identification in two simultanagnosic patients, ES and GK. We show that the patients tended to identify animate objects more accurately than inanimate objects (Experiments 1 and 4). The patients also showed relatively good identification of objects that could be recognised from their global shape, but not objects whose recognition depended on their internal detail (Experiment 2). Indeed, the presence of local segmentation cues disrupted global identification (Experiment 3). Identification was aided, though, by the presence of surface colour and texture (Experiment 4). We suggest that the patients could derive global representations of objects that served to recognise animate items. In contrast, they were impaired at coding parts-based representations for the identification of inanimate objects.     

Within-object and between-object coding deficits in drawing production

We examine how perceptual grouping influenced drawing errors in two patients with constructional apraxia. The patients copied simple geometric stimuli, formed from right-angled elements of variable size arranged as the corners of a square. Grouping was systematically manipulated by altering the closure between these local parts. The patients also made perceptual judgments about similar figures. Despite relatively intact perceptual discrimination of the stimuli, both patients produced errors in drawings relative to controls. When local elements were arranged as a square, patient ECR produced some shape errors but showed a marked tendency to complete across gaps between elements. In contrast, patient RA made no such completion errors but placed the elements so that they were poorly spatially located. When grouping was reduced, ECR completed fewer gaps and RA showed improved spatial localization. These symptoms suggest impairments of perceptuomotor representations of object relations. ECR's deficit appeared to be in between-object coding-she often failed to separate the individual elements when they formed a strong perceptual whole. In contrast, RA's deficit appeared to be in within-object coding-spatial localization of elements was impaired when they shared strong grouping cues. His performance only improved when the strength of the grouping between the elements was reduced. These findings demonstrate the coding of different forms of spatial relation in the perceptual-motor domain.     

Knowing where but not what: Impaired thematic roles and spatial language

We describe case J.P. who, following a left inferior frontal lesion, made frequent role confusions in comprehension and production (e.g., saying “The boy kicks the girl” for a picture showing a girl kicking a boy). J.P.'s preserved ability to judge the grammaticality of sentences rules out a syntactic deficit as the primary cause of the role confusions. Thematic role assignment is also required with spatial prepositions such as in or above, and J.P.'s thematic role assignment was also severely impaired with spatial prepositions. We capitalized on prior linguistic analyses and behavioural studies to design accurate tests of the semantics of spatial terms, spatial relations, and critical features of objects. Fine-grain semantic tests revealed that the semantics of spatial terms and objects was intact. We hypothesize that J.P.'s role confusions reflected a failure to integrate objects within semantic representations that define the thematic roles. Our data suggest that properties of objects and thematic roles are specified by distinct semantic processes, which have different brain localizations. J.P.'s lesion further suggests that left inferior frontal regions are critical in thematic role assignment, thus contributing to the understanding of the linguistic functions of these regions.     

C ROSSMODAL AGNOSIA FOR FAMILIAR PEOPLE AS A CONSEQUENCE OF RIGHT INFERO POLAR TEMPORAL ATROPHY

A 60-year-old, right-handed woman, with no focal brain lesions, suffered from a progressive impairment in recognising people of personal relevance and public figures familiar to her in the premorbid period. The patient did not suffer from general cognitive deterioration. There was no ecological or clear psychometric evidence of visuoperceptual or visuospatial deficits. Her defective person recognition was not overcome by extra-facial (e.g., observing animated people in their usual surroundings) or extra-visual information (e.g., listening to the voice). Moreover, presenting the correct name in the presence of an unrecognised familiar person failed to prompt her familiarity judgement, or retrieval of the relevant biographical knowledge. The patient also had some recognition difficulties with famous buildings and songs as well as with some common objects. It is argued that the patient's difficulty in identifying familiar people was the consequence of progressive loss of stored exemplars of familiar persons and perhaps also of some other “unique items” (famous songs and monuments) in an independent subsystem of semantics that we term “exemplar semantics.” We discuss the associative (semantic) nature and specificity of the deficit in person knowledge, the possible top-down negative influences of the loss of exemplars in the person recognition system, and the link between the disorders and the right/left temporal lobe.     

Identifying thematic roles from neural representations measured by functional magnetic resonance imaging

ABSTRACT

The generativity and complexity of human thought stem in large part from the ability to represent relations among concepts and form propositions. The current study reveals how a given object such as rabbit is neurally encoded differently and identifiably depending on whether it is an agent (“the rabbit punches the monkey”) or a patient (“the monkey punches the rabbit”). Machine-learning classifiers were trained on functional magnetic resonance imaging (fMRI) data evoked by a set of short videos that conveyed agent–verb–patient propositions. When tested on a held-out video, the classifiers were able to reliably identify the thematic role of an object from its associated fMRI activation pattern. Moreover, when trained on one subset of the study participants, classifiers reliably identified the thematic roles in the data of a left-out participant (mean accuracy?=?.66), indicating that the neural representations of thematic roles were common across individuals.