Distributed category‐specific recognition‐memory signals in human perirhinal cortex

Evidence from a large body of research suggests that perirhinal cortex (PrC), which interfaces the medial temporal lobe with the ventral visual pathway for object identification, plays a critical role in item‐based recognition memory. The precise manner in which PrC codes for the prior occurrence of objects, however, remains poorly understood. In the present functional magnetic resonance imaging (fMRI) study, we used multivoxel pattern analyses to examine whether the prior occurrence of faces is coded by distributed patterns of PrC activity that consist of voxels with decreases as well as increases in signal. We also investigated whether pertinent voxels are preferentially tuned to the specific object category to which judged stimuli belong. We found that, when no a priori constraints were imposed on the direction of signal change, activity patterns that allowed for successful classification of recognition‐memory decisions included some voxels with decreases and others with increases in signal in association with perceived prior occurrence. Moreover, successful classification was obtained in the absence of a mean difference in activity across the set of voxels in these patterns. Critically, we observed a positive relationship between classifier accuracy and behavioral performance across participants. Additional analyses revealed that voxels carrying diagnostic information for classification of memory decisions showed category specificity in their tuning for faces when probed with an independent functional localizer in a nonmnemonic task context. These voxels were spatially distributed in PrC, and extended beyond the contiguous voxel clusters previously described as the anterior temporal face patch. Our findings provide support for proposals, recently raised in the neurophysiological literature, that the prior occurrence of objects is coded by distributed PrC representations. They also suggest that the stimulus category to which an item belongs shapes the organization of these distributed representations. © 2015 Wiley Periodicals, Inc.     

Category specificity in the medial temporal lobe: A systematic review

Theoretical accounts of medial temporal lobe (MTL) function ascribe different functions to subregions of the MTL including perirhinal, entorhinal, parahippocampal cortices, and the hippocampus. Some have suggested that the functional roles of these subregions vary in terms of their category specificity, showing preferential coding for certain stimulus types, but the evidence for this functional organization is mixed. In this systematic review, we evaluate existing evidence for regional specialization in the MTL for three categories of visual stimuli: faces, objects, and scenes. We review and synthesize across univariate and multivariate neuroimaging studies, as well as neuropsychological studies of cases with lesions to the MTL. Neuroimaging evidence suggests that faces activate the perirhinal cortex, entorhinal cortex, and the anterior hippocampus, while scenes engage the parahippocampal cortex and both the anterior and posterior hippocampus, depending on the contrast condition. There is some evidence for object‐related activity in anterior MTL regions when compared to scenes, and in posterior MTL regions when compared to faces, suggesting that aspects of object representations may share similarities with face and scene representations. While neuroimaging evidence suggests some hippocampal specialization for faces and scenes, neuropsychological evidence shows that hippocampal damage leads to impairments in scene memory and perception, but does not entail equivalent impairments for faces in cases where the perirhinal cortex remains intact. Regional specialization based on stimulus categories has implications for understanding the mechanisms of MTL subregions, and highlights the need for the development of theoretical models of MTL function that can accommodate the differential patterns of specificity observed in the MTL.     

Human memory manipulated: dissociating factors contributing to MTL activity, an fMRI study

Memory processes are mainly studied with subjective rating procedures. We used a morphing procedure to objectively manipulate the similarity of target stimuli. While undergoing functional magnetic resonance imaging, nineteen subjects performed a encoding and recognition task on face and scene stimuli, varying the degree of manipulation of previously studied targets at 0%, 20%, 40% or 60%. Analyses were performed with parametric modulations for objective stimulus status (morphing level), subjective memory (confidence rating), and reaction times (RTs). Results showed that medial temporal lobe (MTL) activity can be best explained by a combination of subjective and objective factors. Memory success is associated with activity modulation in the hippocampus both for faces and for scenes. Memory failures correlated with lower hippocampal activity for scenes, but not for faces. Activity changed during retrieval on similar areas activated during encoding. There was a considerable impact of RTs on memory-related areas. Objective perceptual identity correlated with activity in the left MTL, while subjective memory experience correlated with activity in the right MTL for both types of material. Overall, the results indicate that MTL activity is heterogeneous, showing both linear and non-linear activity, depending on the factor analyzed.     

Medial temporal lobe activity during complex discrimination of faces,objects, and scenes: Effects of viewpoint

The medial temporal lobe (MTL), a set of heavily interconnected structures including the hippocampus and underlying entorhinal, perirhinal and parahippocampal cortex, is traditionally believed to be part of a unitary system dedicated to declarative memory. Recent studies, however, demonstrated perceptual impairments in amnesic individuals with MTL damage, with hippocampal lesions causing scene discrimination deficits, and perirhinal lesions causing object and face discrimination deficits. The degree of impairment on these tasks was influenced by the need to process complex conjunctions of features: discriminations requiring the integration of multiple visual features caused deficits, whereas discriminations that could be solved on the basis of a single feature did not. Here, we address these issues with functional neuroimaging in healthy participants as they performed a version of the oddity discrimination task used previously in patients. Three different types of stimuli (faces, scenes, novel objects) were presented from either identical or different viewpoints. Consistent with studies in patients, we observed increased perirhinal activity when participants distinguished between faces and objects presented from different, compared to identical, viewpoints. The posterior hippocampus, by contrast, showed an effect of viewpoint for both faces and scenes. These findings provide convergent evidence that the MTL is involved in processes beyond long‐term declarative memory and suggest a critical role for these structures in integrating complex features of faces, objects, and scenes into view‐invariant, abstract representations. © 2009 Wiley‐Liss, Inc.     

Perception and conception: temporal lobe activity during complex discriminations of familiar and novel faces and objects

Recent studies indicate that medial-temporal lobe (MTL) damage, either from focal lesions or neurodegenerative disease (e.g., semantic dementia), impairs perception as well as long-term declarative memory. Notably, however, these two patient groups show different performance for meaningful versus unfamiliar stimuli. In amnesics with nonprogressive MTL lesions, the use of meaningful stimuli, compared with unfamiliar items, boosted discrimination performance. In semantic dementia, a condition characterized by progressive deterioration of conceptual knowledge in the context of anterolateral temporal lobe damage, performance for meaningful stimuli was equivalent to that for unfamiliar items. To further investigate these findings, we scanned healthy volunteers while they performed odd-one-out discriminations involving familiar (i.e., meaningful/famous) and unfamiliar (i.e., novel) objects and faces and a baseline task of size oddity. Outside the scanner, volunteers' recognition memory was assessed. We found above baseline activity in the perirhinal cortex and hippocampus for all object and face discriminations and above baseline activity in the temporal pole for all face discriminations. The comparison of meaningful, relative to novel, faces and objects, revealed increased activity in the perirhinal cortex and hippocampus. In the temporal pole, we also found activity related to meaningfulness for faces but not for objects. Importantly, these meaningfulness effects were evident even for discriminations that were not subsequently well remembered, suggesting that the difference between meaningful and novel stimuli reflects perceptual or conceptual processes rather than solely incidental encoding into long-term memory. The results provide further evidence that the MTL is recruited during complex perceptual discrimination and additionally suggest that these structures are recruited in semantic processing of objects and faces.     

Memory search for faces and digits in patients with unilateral brain lesions

Abstract

To clarify whether the impairment of right-brain-damaged (RBD) patients in face recognition is related to perceptual or mnestic processing stages, we tested unilaterally lesioned patients and controls in a Sternberg-type memory search task. Subjects had to memorize sets of 1, 2, or 3 faces (or, in a control condition, digits) and were then to recognize these memorized stimuli among new ones by speeded choice reactions. In this task, deficits in stimulus encoding and memory search should show up in increased intercepts and slopes, respectively, of the RT function over memory set size. A face-specific impairment of the RBD patient group, consisting in longer reaction times and higher error rates, was confirmed but could not be unequivocally attributed to either stimulus encoding or memory search. However, inspection of individual data suggested that (1) some RBD patients are virtually unimpaired in face recognition and (2) if impairment after right hemisphere damage is present, it may selectively affect either stimulus encoding or memory search.     

Memory search for faces and digits in patients with unilateral brain lesions.

To clarify whether the impairment of right-brain-damaged (RBD) patients in face recognition is related to perceptual or mnestic processing stages, we tested unilaterally lesioned patients and controls in a Sternberg-type memory search task. Subjects had to memorize sets of 1, 2, or 3 faces (or, in a control condition, digits) and were then to recognize these memorized stimuli among new ones by speeded choice reactions. In this task, deficits in stimulus encoding and memory search should show up in increased intercepts and slopes, respectively, of the RT function over memory set size. A face-specific impairment of the RBD patient group, consisting in longer reaction times and higher error rates, was confirmed but could not be unequivocally attributed to either stimulus encoding or memory search. However, inspection of individual data suggested that (1) some RBD patients are virtually unimpaired in face recognition and (2) if impairment after right hemisphere damage is present, it may selectively affect either stimulus encoding or memory search.     

Material specific lateralization of medial temporal lobe function: An fMRI investigation

The theory of material specific lateralization of memory function posits that left and right MTL regions are asymmetrically involved in mnemonic processing of verbal and nonverbal material respectively. Lesion and functional imaging (fMRI) studies provide robust evidence for a left MTL asymmetry in the verbal memory domain. Evidence for a right MTL/nonverbal asymmetry is not as robust. A handful of fMRI studies have investigated this issue but have generally utilised nonverbal stimuli which are amenable to semantic elaboration. This fMRI study aimed to investigate the neural correlates of recognition memory processing in 20 healthy young adults (mean age = 26 years) for verbal stimuli and nonverbal stimuli that were specifically designed to minimize verbalisation. Analyses revealed that the neural correlates of recognition memory processing for verbal and nonverbal stimuli were differentiable and asymmetrically recruited the left and right MTL respectively. The right perirhinal cortex and hippocampus were preferentially involved in successful recognition memory of items devoid of semantic information. In contrast, the left anterior hippocampus was preferentially involved in successful recognition memory of stimuli which contained semantic meaning. These results suggest that the left MTL is preferentially involved in mnemonic processing of verbal/semantic information. In contrast, the right MTL is preferentially involved in visual/non‐semantic mnemonic processing. We propose that during development, the left MTL becomes specialised for verbal mnemonic processing due to its proximity with left lateralised cortical language processing areas while visual/non‐semantic mnemonic processing gets ‘crowded out’ to become predominantly, but not completely, the domain of the right MTL. Hum Brain Mapp 37:933–941, 2016. © 2015 Wiley Periodicals, Inc .     

The effect of encoding strategy on the neural correlates of memory for faces

Encoding and recognition of unfamiliar faces in young adults were examined using positron emission tomography to determine whether different encoding strategies would lead to encoding/retrieval differences in brain activity. Three types of encoding were compared: a 'deep' task (judging pleasantness/unpleasantness), a 'shallow' task (judging right/left orientation), and an intentional learning task in which subjects were instructed to learn the faces for a subsequent memory test but were not provided with a specific strategy. Memory for all faces was tested with an old/new recognition test. A modest behavioral effect was obtained, with deeply-encoded faces being recognized more accurately than shallowly-encoded or intentionally-learned faces. Regardless of encoding strategy, encoding activated a primarily ventral system including bilateral temporal and fusiform regions and left prefrontal cortices, whereas recognition activated a primarily dorsal set of regions including right prefrontal and parietal areas. Within encoding, the type of strategy produced different brain activity patterns, with deep encoding being characterized by left amygdala and left anterior cingulate activation. There was no effect of encoding strategy on brain activity during the recognition conditions. Posterior fusiform gyrus activation was related to better recognition accuracy in those conditions encouraging perceptual strategies, whereas activity in left frontal and temporal areas correlated with better performance during the 'deep' condition. Results highlight three important aspects of face memory: (1) the effect of encoding strategy was seen only at encoding and not at recognition; (2) left inferior prefrontal cortex was engaged during encoding of faces regardless of strategy; and (3) differential activity in fusiform gyrus was found, suggesting that activity in this area is not only a result of automatic face processing but is modulated by controlled processes.     

Recognition memory for faces and scenes in amnesia: dissociable roles of medial temporal lobe structures

The relative contributions of the hippocampus and the perirhinal cortex to recognition memory are currently the subject of intense debate. Whereas some authors propose that both structures play a similar role in recognition memory, others suggest that the hippocampus might mediate recollective and/or associative aspects of recognition memory, whereas the perirhinal cortex may mediate item memory. Here we investigate an alternative functional demarcation between these structures, following reports of stimulus-specific perceptual deficits in amnesics with medial temporal lobe (MTL) lesions. Using a novel recognition memory test for faces and scenes, participants with broad damage to MTL structures, which included the hippocampus and the perirhinal cortex, were impaired on both face and scene memory. By contrast, participants with damage limited to the hippocampus showed deficits only in memory for scenes. These findings imply that although both the hippocampus and surrounding cortex contribute to recognition memory, their respective roles can be distinguished according to the type of material to be remembered. This interaction between lesion site and stimulus category may explain some of the inconsistencies present in the literature.     

Differential contributions of the parahippocampal place area and the anterior hippocampus to human memory for scenes

Past neuroimaging research has identified a parahippocampal place area (PPA) in the posterior medial temporal lobe (MTL), which responds preferentially to visual scenes and plays a role in episodic memory for this class of stimuli. In the present positron emission tomography study, we examined to what extent the functional characteristics of the PPA resemble those of other, more anterior MTL regions across various learning and recognition-memory tasks. We also determined whether the involvement of the PPA in recognition of previously studied scenes is specific to a particular type of scene information. We found that, like the PPA, anterior hippocampal regions showed a novelty response (higher activation for novel than repeated scenes) and a stimulus-related response (higher activation for scenes than objects) during learning, indicating that MTL structures other than the PPA contribute to the encoding of novel stimulus relationships in scenes. However, these anterior hippocampal regions showed no involvement during recognition of either spatial or nonspatial information contained in scenes. The PPA, by contrast, was consistently involved in recognition of all types of scene details, presumably through interactions with co-activated parietal and occipitotemporal cortices. We suggest that MTL contributions from the PPA are sufficient to support recognition of scenes when the task can be based on a perceptually based familiarity process.     

Early signal from the hippocampus for memory encoding

The mediotemporal lobe (MTL), including the hippocampus, is involved in all stages of episodic memory including memory encoding, consolidation, and retrieval. However, the exact timing of the hippocampus' involvement immediately after stimulus encounter remains unclear. In this study, we used high‐density 156‐channel electroencephalography to study the processing of entirely new stimuli, which had to be encoded, in comparison to highly overlearned stimuli. Sixteen healthy subjects performed a continuous recognition task with meaningful pictures repeated up to four consecutive times. Waveform and topographic cluster analyses of event‐related potentials revealed that new items, in comparison to repetitions, were processed significantly differently at 220–300 ms. Source estimation localized activation for processing new stimuli in the right MTL. Our study demonstrates the occurrence of a transient signal from the MTL in response to new information already at 200–300 ms poststimulus onset, which presumably reflects encoding as an initial step toward memory consolidation.     

The medial temporal lobe functional connectivity patterns associated with forming different mental representations

The medial temporal lobes (MTL), and more specifically the hippocampus, are critical for forming mental representations of past experiences—autobiographical memories—and for forming other “nonexperienced” types of mental representations, such as imagined scenarios. How the MTL coordinate with other brain areas to create these different types of representations is not well understood. To address this issue, we performed a task‐based functional connectivity analysis on a previously published dataset in which fMRI data were collected as participants created different types of mental representations under three conditions. One condition required forming and relating together details from a past event (autobiographical task), another required forming and relating together details of a spatial context (spatial task) and another condition required relating together conceptual/perceptual features of an object (conceptual task). We contrasted the connectivity patterns associated with a functionally defined region in the parahippocampal cortex (PHC) and anatomically defined anterior and posterior hippocampal segments across these tasks. Examining PHC connectivity patterns revealed that the PHC seed was distinctly connected to other MTL structures during the autobiographical task, to posterior parietal regions during the spatial task and to a distributed network of regions for the conceptual task. Examining hippocampal connectivity patterns revealed that the anterior hippocampus was preferentially connected to regions of default mode network during the autobiographical task and to areas implicated in semantic processing for the conceptual task whereas the posterior hippocampus was preferentially connected to medial‐posterior regions of the brain during the spatial task. We interpret our findings as evidence that there are MTL‐guided networks for forming distinct types of mental representations that align with functional distinctions within the hippocampus.     

Spatial and stimulus‐type tuning in the LEC,MEC, POR,PrC, CA1, and CA3 during spontaneous item recognition memory

According to the “two streams” hypothesis, the lateral entorhinal (LEC) and the perirhinal (PrC) cortices process information related to items (a “what” stream), the postrhinal (POR) and the medial entorhinal cortices (MEC) process spatial information (a “where” stream), and both types of information are integrated in the hippocampus (HIP). However, within the framework of memory function, only the HIP is reliably shown to preferentially process spatial information, and the PrC items' features. In contrast, the role of the LEC and MEC in memory is virtually unexplored, and conflicting results emerge for the POR. Moreover, the specific contribution of the hippocampal subfields CA1 and CA3 to spatial and non‐spatial memory is not thoroughly understood. To investigate which of these areas is specifically tuned to spatial demands or stimulus identity (odor or object), we assessed the pattern of activation of these areas during recognition memory by detecting the immediate‐early gene Arc, commonly used as a marker of neuronal activation. We report that all MTL areas were recruited during the spatial and the non‐spatial tasks. However, the LEC, MEC, POR, and CA1 were activated to a comparable level in spatial and non‐spatial tasks, while the PrC was tuned to stimulus‐type, not spatial demands, and CA3 to spatial demands but not stimulus‐type. Results are discussed within the frame of a recent model suggesting that the MTL could be segregated in terms of memory processes, such as recollection and familiarity, rather than information content. © 2013 Wiley Periodicals, Inc.     

Working memory and amnesia: the role of stimulus novelty

Despite the traditional view that damage to the hippocampus and/or surrounding areas of the medial temporal lobe (MTL) does not impair short-term or working memory (WM), recent research has shown MTL amnesics to be impaired on WM tasks that require maintaining a small amount of information over brief retention intervals (e.g., maintenance of a single face for one second). However, the types of tasks that have demonstrated WM impairments in amnesia tend to have involved novel stimuli. We hypothesized that WM may be impaired in amnesia for tasks that require maintaining novel information, but may be preserved for more familiar material, particularly if the material can be easily rehearsed. To test this hypothesis, patient HC, a 22-year-old developmental amnesic with relatively preserved semantic memory and 20 age and education matched controls performed a delayed match-to-sample task that required maintaining a single famous or non-famous face for 1-8s, digit span and reading span tasks, and a modified Brown-Peterson task that required maintaining a single high- or low-frequency word or a non-word for 4-8s. HC's performance was impaired for non-famous faces but preserved for famous faces, impaired for the reading span task but preserved for digit span, and it was impaired for non-words and unfamiliar low-frequency words but preserved for familiar words. These results support the hypothesis that an intact hippocampus is necessary for maintaining a single novel stimulus in WM. However, stimulus familiarity and rehearsal support WM via cortical regions independent of the MTL.     

An event-related brain potential study of explicit face recognition

To determine the time course of face recognition and its links to face-sensitive event-related potential (ERP) components, ERPs elicited by faces of famous individuals and ERPs to non-famous control faces were compared in a task that required explicit judgements of facial identity. As expected, the face-selective N170 component was unaffected by the difference between famous and non-famous faces. In contrast, the occipito-temporal N250 component was linked to face recognition, as it was selectively triggered by famous faces. Importantly, this component was present for famous faces that were judged to be definitely known relative to famous faces that just appeared familiar, demonstrating that it is associated with the explicit identification of a particular face. The N250 is likely to reflect early perceptual stages of face recognition where long-term memory traces of familiar faces in ventral visual cortex are activated by matching on-line face representations. Famous faces also triggered a broadly distributed longer-latency positivity (P600f) that showed a left-hemisphere bias and was larger for definitely known faces, suggesting links between this component and name generation. These results show that successful face recognition is predicted by ERP components over face-specific visual areas that emerge within 230 ms after stimulus onset.     

Material Specificity Drives Medial Temporal Lobe Familiarity But Not Hippocampal Recollection

The specific role of the perirhinal (PRC), entorhinal (ERC) and parahippocampal cortices (PHC) in supporting familiarity‐based recognition remains unknown. An fMRI study explored whether these medial temporal lobe (MTL) structures responded in the same way or differentially to familiarity as a function of stimulus type at recognition. A secondary aim was to explore whether the hippocampus responds in the same way to equally strong familiarity and recollection and whether this is influenced by the kind of stimulus involved. Univariate and multivariate analyses revealed that familiarity responses in the PRC, ERC, PHC and the amygdala are material‐specific. Specifically, the PRC and ERC selectively responded to object familiarity, while the PHC responded to both object and scene familiarity. The amygdala only responded to familiarity memory for faces. The hippocampus did not respond to stimulus familiarity for any of the three types of stimuli, but it did respond to recollection for all three types of stimuli. This was true even when recollection was contrasted to equally accurate familiarity. Overall, the findings suggest that the role of the MTL neocortices and the amygdala in familiarity‐based recognition depends on the kind of stimulus in memory, whereas the role of the hippocampus in recollection is independent of the type of cuing stimulus. © 2016 The Authors Hippocampus Published by Wiley Periodicals, Inc.     

A role for perirhinal cortex in memory for novel object-context associations

It is debated whether functional divisions between structures in the medial temporal lobe (MTL), in particular the perirhinal cortex (PrC) and hippocampus (HC), are best conceptualized according to memory process (Diana et al., 2007; Ranganath, 2010; Wixted et al., 2010) or stimulus category (Graham et al., 2010). In the former account, PrC is critical for item familiarity but not recollection of associations between items and their contexts (which is instead dependent upon the HC; Ranganath et al., 2004). In the latter theory, complex object representations in PrC are capable of supporting memory for objects as well as for object-context associations, particularly when there is a demand to discriminate between highly visually similar objects (Cowell et al., 2010). To adjudicate between these accounts, human participants were scanned while making two different judgments about visually presented objects (is the object common or uncommon, or does the object have more edges or curves). In a subsequent, unscanned, retrieval phase, participants made item (old/new) followed by context (encoding task) judgments about previously seen and novel objects. Neural activity at encoding was separated according to the accuracy of the retrieval judgments. PrC activity predicted successful item-context judgments, a result that remained when item-memory strength was equated across objects for which the context was remembered or forgotten. These data imply that the function of PrC goes beyond processing item-based memory information, contributing additionally to memory for item-context associations when the stimuli are objects (Graham et al., 2010).     

Selective familiarity deficits after left anterior temporal-lobe removal with hippocampal sparing are material specific

Research has firmly established a link between recognition memory and the functional integrity of the medial temporal lobes (MTL). Dual-process models of MTL organization maintain that there is a division of labour within the MTL, with the hippocampus (HC) supporting recollective processes and perirhinal cortex (PRc) supporting familiarity assessment. An older neuropsychological literature suggested a different type of division of labour within the MTL, with left-sided structures playing a critical role in memory for verbal materials and right-sided structures being differentially involved in memory for material that cannot easily be verbalized. Research that has related predictions made by these two accounts to each other is limited. Evidence from research in patients with selective recollection impairments and fMRI data in healthy individuals suggests that lateralization of recollection for verbal materials is not clear-cut. Here we examined lateralization of familiarity processes in the MTL by asking whether selective familiarity impairments after unilateral anterior temporal-lobe removal with hippocampal sparing are material specific. We examined this issue in NB, an individual who was previously shown to exhibit selective familiarity impairments with such a lesion (Bowles et al., 2007). We administered three similar recognition memory tests in combination with the same Remember-Know procedure for three different types of novel stimuli without pre-existing semantic representations. Analyses focused on discrimination and on possible differences in response criterion, and included an ROC based approach as well. We found that NB exhibited a deficit in overall recognition of aurally presented pronounceable non-words that reflected a specific impairment of familiarity assessment with preservation of recollective processes. Examination of recognition memory for visually presented abstract pictures and faces did not reveal any impairment, neither at the level of overall recognition nor, more specifically, at the level of familiarity assessment. These findings suggest that the neural mechanisms that support familiarity assessment in the temporal lobe operate in a manner that is tied to the specific stimulus class being assessed.     

Categorical learning revealed in activity pattern of left fusiform cortex

The brain is organized such that it encodes and maintains category information about thousands of objects. However, how learning shapes these neural representations of object categories is unknown. The present study focuses on faces, examining whether: (1) Enhanced categorical discrimination or (2) Feature analysis enhances face/non‐face categorization in the brain. Stimuli ranged from non‐faces to faces with two‐toned Mooney images used for testing and gray‐scale images used for training. The stimulus set was specifically chosen because it has a true categorical boundary between faces and non‐faces but the stimuli surrounding that boundary have very similar features, making the boundary harder to learn. Brain responses were measured using functional magnetic resonance imaging while participants categorized the stimuli before and after training. Participants were either trained with a categorization task, or with non‐categorical semblance analyzation. Interestingly, when participants were categorically trained, the neural activity pattern in the left fusiform gyrus shifted from a graded representation of the stimuli to a categorical representation. This corresponded with categorical face/non‐face discrimination, critically including both an increase in selectivity to faces and a decrease in false alarm response to non‐faces. By contrast, while activity pattern in the right fusiform cortex correlated with face/non‐face categorization prior to training, it was not affected by learning. Our results reveal the key role of the left fusiform cortex in learning face categorization. Given the known right hemisphere dominance for face‐selective responses, our results suggest a rethink of the relationship between the two hemispheres in face/non‐face categorization. Hum Brain Mapp 38:3648–3658, 2017. © 2017 Wiley Periodicals, Inc.