Neurons in the amygdala of the monkey with responses selective for faces

To investigate the functions of the amygdala in visual information processing and in emotional and social responses, recordings were made from single neurons in the amygdala of the monkey. A population of neurons (40 of more than 1000 recorded in 4 monkeys) was investigated which responded primarily to faces. These neurons typically (1) responded to some human or monkey faces, which were presented to the monkey through a large aperture shutter so that response latencies could be measured, or were simply shown to the monkey, (2) responded to 2-dimensional representations of these faces, as well as to real 3-dimensional faces, (3) had no responses or only small (less than half maximum) responses to gratings, simple geometrical, other complex 3-D stimuli, or to arousing and aversive stimuli, (4) had response latencies of 110-200 ms, (5) were located in the basal accessory nucleus of the amygdala, (6) responded differently to different faces, as shown by measures of d', and could thus over a population of such neurons code information useful for making different responses to different individuals, (7) could in some cases (9/11 tested) respond to parts of faces, and (8) in a few cases (4/19 tested) responded more to a face which produced an emotional response. A comparison made in three monkeys of the responses of these neurons with the responses of 77 neurons with face-selective responses recorded in the cortex of the superior temporal sulcus (STS) showed that the amygdaloid neurons had longer response latencies (110-200 compared to 90-140 ms), and were in some respects more selective in their responses to different faces. It is suggested that the deficits in social and emotional behavior produced by amygdala lesions could be due in part to damage to a neuronal system specialized in utilizing information from faces so that appropriate social and emotional responses can be made to different individuals.     

The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey

Neurophysiological studies have shown that some neurons in the cortex in the superior temporal sulcus and the inferior temporal gyrus of macaque monkeys respond to faces. To determine if facial factors such as expression and identity are encoded independently by face-responsive neurons, 45 neurons were tested on a stimulus set depicting 3 monkeys with 3 expressions each. As tested on a two-way ANOVA, 15 neurons showed response differences to different identities independently of expression, and 9 neurons showed responses to different expressions independently of identity. Three neurons showed significant effects of both factors. Six of the neurons with responses related to expression responded primarily to calm faces, while 2 responded primarily to threat faces. Of a further set of 31 neurons tested on pairs of different expressions, 6 showed strong responses to open-mouth fear or threat expressions, while 2 showed stronger responses to calm faces than threat expressions. Neurons responsive to expression were found primarily in the cortex in the superior temporal sulcus, while neurons responsive to identity were found primarily in the inferior temporal gyrus. The difference in anatomical distribution was statistically significant. This supports the possibility that specific impairments of the recognition of the identity of a face and of its expression in man are due to damage to or disconnection of separate neuronal substrates.     

Stimulus-selective properties of inferior temporal neurons in the macaque

Previous studies have reported that some neurons in the inferior temporal (IT) cortex respond selectively to highly specific complex objects. In the present study, we conducted the first systematic survey of the responses of IT neurons to both simple stimuli, such as edges and bars, and highly complex stimuli, such as models of flowers, snakes, hands, and faces. If a neuron responded to any of these stimuli, we attempted to isolate the critical stimulus features underlying the response. We found that many of the responsive neurons responded well to virtually every stimulus tested. The remaining, stimulus-selective cells were often selective along the dimensions of shape, color, or texture of a stimulus, and this selectivity was maintained throughout a large receptive field. Although most IT neurons do not appear to be "detectors" for complex objects, we did find a separate population of cells that responded selectively to faces. The responses of these cells were dependent on the configuration of specific face features, and their selectivity was maintained over changes in stimulus size and position. A particularly high incidence of such cells was found deep in the superior temporal sulcus. These results indicate that there may be specialized mechanisms for the analysis of faces in IT cortex.     

The representation of information about faces in the temporal and frontal lobes

Neurophysiological evidence is described showing that some neurons in the macaque inferior temporal visual cortex have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning. Which face or object is present is encoded using a distributed representation in which each neuron conveys independent information in its firing rate, with little information evident in the relative time of firing of different neurons. This ensemble encoding has the advantages of maximising the information in the representation useful for discrimination between stimuli using a simple weighted sum of the neuronal firing by the receiving neurons, generalisation and graceful degradation. These invariant representations are ideally suited to provide the inputs to brain regions such as the orbitofrontal cortex and amygdala that learn the reinforcement associations of an individual's face, for then the learning, and the appropriate social and emotional responses, generalise to other views of the same face. A theory is described of how such invariant representations may be produced in a hierarchically organised set of visual cortical areas with convergent connectivity. The theory proposes that neurons in these visual areas use a modified Hebb synaptic modification rule with a short-term memory trace to capture whatever can be captured at each stage that is invariant about objects as the objects change in retinal view, position, size and rotation. Another population of neurons in the cortex in the superior temporal sulcus encodes other aspects of faces such as face expression, eye gaze, face view and whether the head is moving. These neurons thus provide important additional inputs to parts of the brain such as the orbitofrontal cortex and amygdala that are involved in social communication and emotional behaviour. Outputs of these systems reach the amygdala, in which face-selective neurons are found, and also the orbitofrontal cortex, in which some neurons are tuned to face identity and others to face expression. In humans, activation of the orbitofrontal cortex is found when a change of face expression acts as a social signal that behaviour should change; and damage to the orbitofrontal cortex can impair face and voice expression identification, and also the reversal of emotional behaviour that normally occurs when reinforcers are reversed.     

Comparison of differential neuronal responsiveness for different regions of the prefrontal cortex in a conditional visual discrimination task

To investigate neuronal processing during monkeys' performance of a visual conditional discrimination task, recordings were made from four areas of prefrontal cortex (ventromedial, orbitofrontal, dorsolateral and anterior cingulate) where lesions have been shown to produce impairment of such tasks. Of 1911 recorded neurons, 573 (31%) responded to elements of the task. This proportion was less than the 50% previously reported as responsive in temporal cortex under the same conditions, suggesting sparser encoding in prefrontal than temporal cortex. Of the responsive prefrontal neurons, 165 (29%) responded differently on the different types of trial, so signalling various types of information relevant to task performance and cognition. In line with recent lesion findings, in the dorsolateral region the incidence of such differentially responsive neurons was only an eighth that in the other regions. The relatively high incidence of neuronal responses that encoded a potential instruction cue rather than specific individual stimulus arrangements was consistent with the animals solving the task by using such information, though other neuronal responses could have enabled the task to have been solved by rote learning. Compared to temporal neurons, prefrontal responses more frequently coded information relating to the planned behavioural response rather than perceptual aspects of the task. Population differential response latencies were long (> approximately 225 ms) in prefrontal cortex. A comparison of such differential latencies between temporal and prefrontal cortex indicated that potential information flow was likely to be primarily from temporal to prefrontal cortex rather than vice versa.     

Differential neuronal responsiveness in primate perirhinal cortex and hippocampal formation during performance of a conditional visual discrimination task

Neuronal responses in the hippocampal formation, including the entorhinal cortex, have been compared with those in the inferior temporal cortex, including the perirhinal cortex, during performance by monkeys of a visual conditional discrimination task. In the task, the arrangement of three geometric shapes determined the correctness of either a left or right behavioural response according to a conditional rule. Neurons that responded differently to different types of trial were common (50% of the visually responsive neurons) in the entorhinal cortex, perirhinal cortex and area TE of the inferior temporal cortex, but significantly less common in the hippocampus (13%). This differential incidence suggests a more important role for the rhinal cortices and area TE than for the hippocampus in this task. Based on the neuronal responses, arguments are advanced that the animals probably solved the task by a strategy that did not require spatial or hippocampal processing. Thus, of the differential responses, those that would allow the animals to solve the task by using a conditional rule and so avoid spatial processing were twice as common (37%) as those allowing solution to be by selection of a particular spatially directed response to each arrangement of shapes (19%). Moreover, the differential latencies of responses that allowed the task to be solved by a conditional rule were shorter (< approximately 165 ms), and hence processing was faster, than those that provided information about particular individual types of trial ( approximately 195 ms). Even so, hippocampal responsiveness in the conditional task was differentially enhanced when compared with that during a recognition memory task, and the neuronal responses potentially allow the animal to employ a second, alternative strategy that might be expected to depend on hippocampal processing.     

Categorization of complex visual images by rhesus monkeys. Part 2: single-cell study

In order to investigate the neural coding of ordinate-level visual categories, single-cell recordings were made in the anterior temporal cortex of two rhesus monkeys performing a categorization of colour images of trees versus images of other objects. Neurons showed a high average degree of selectivity for these complex colour images. Although most neurons responded to trees and non-trees, about a quarter responded in a category-specific manner, e.g. to trees but not non-trees, and about one-tenth responded almost exclusively to exemplars of the trained category. The responses of these neurons were largely invariant for stimulus transformations, e. g. changes in position or size, and decreased with the degree of image scrambling, mimicking the behavioural results. However, the responses of single neurons were insufficiently stimulus invariant to accommodate the entire range of variability present in the features of exemplars within the same category. This strong within-category selectivity challenges the idea that a prototype is represented at the single neuron level, but suggests that ordinate-level categorization is based on a population of neurons, each selective for a limited set of exemplars.     

Neuronal responses to face‐like stimuli in the monkey pulvinar

The pulvinar nuclei appear to function as the subcortical visual pathway that bypasses the striate cortex, rapidly processing coarse facial information. We investigated responses from monkey pulvinar neurons during a delayed non‐matching‐to‐sample task, in which monkeys were required to discriminate five categories of visual stimuli [photos of faces with different gaze directions, line drawings of faces, face‐like patterns (three dark blobs on a bright oval), eye‐like patterns and simple geometric patterns]. Of 401 neurons recorded, 165 neurons responded differentially to the visual stimuli. These visual responses were suppressed by scrambling the images. Although these neurons exhibited a broad response latency distribution, face‐like patterns elicited responses with the shortest latencies (approximately 50 ms). Multidimensional scaling analysis indicated that the pulvinar neurons could specifically encode face‐like patterns during the first 50‐ms period after stimulus onset and classify the stimuli into one of the five different categories during the next 50‐ms period. The amount of stimulus information conveyed by the pulvinar neurons and the number of stimulus‐differentiating neurons were consistently higher during the second 50‐ms period than during the first 50‐ms period. These results suggest that responsiveness to face‐like patterns during the first 50‐ms period might be attributed to ascending inputs from the superior colliculus or the retina, while responsiveness to the five different stimulus categories during the second 50‐ms period might be mediated by descending inputs from cortical regions. These findings provide neurophysiological evidence for pulvinar involvement in social cognition and, specifically, rapid coarse facial information processing.     

A direct measure of human lateral temporal lobe neurons responsive to face matching

Do the human cerebral hemispheres process faces differently? Clinical lesion observations and primate studies suggest that the right temporal lobe is critical in face processing. Yet, the nature of this relationship remains unclear. Recording from single neurons during a visuospatial (VS)-matching paradigm, we found that 100% of significantly active neurons discriminated matching from perception, bilaterally. Lateralized differences in the nature and timing of responses revealed that the right hemisphere neurons responded earlier, and with uniform frequency reductions. Additional lateralized differences favoring the right hemisphere neurons were found when subjects matched intact faces compared to scrambled faces or complex objects. We conclude that widely distributed neural ensembles are involved in 'lateralized' behaviors, but cerebral specialization of face processing is as much a function of the nature and timing of neuronal activity as anatomic location.     

Functional integration of the posterior superior temporal sulcus correlates with facial expression recognition

Face perception is essential for daily and social activities. Neuroimaging studies have revealed a distributed face network (FN) consisting of multiple regions that exhibit preferential responses to invariant or changeable facial information. However, our understanding about how these regions work collaboratively to facilitate facial information processing is limited. Here, we focused on changeable facial information processing, and investigated how the functional integration of the FN is related to the performance of facial expression recognition. To do so, we first defined the FN as voxels that responded more strongly to faces than objects, and then used a voxel‐based global brain connectivity method based on resting‐state fMRI to characterize the within‐network connectivity (WNC) of each voxel in the FN. By relating the WNC and performance in the “Reading the Mind in the Eyes” Test across participants, we found that individuals with stronger WNC in the right posterior superior temporal sulcus (rpSTS) were better at recognizing facial expressions. Further, the resting‐state functional connectivity (FC) between the rpSTS and right occipital face area (rOFA), early visual cortex (EVC), and bilateral STS were positively correlated with the ability of facial expression recognition, and the FCs of EVC‐pSTS and OFA‐pSTS contributed independently to facial expression recognition. In short, our study highlights the behavioral significance of intrinsic functional integration of the FN in facial expression processing, and provides evidence for the hub‐like role of the rpSTS for facial expression recognition. Hum Brain Mapp 37:1930–1940, 2016. © 2016 Wiley Periodicals, Inc.     

Material Specificity Drives Medial Temporal Lobe Familiarity But Not Hippocampal Recollection

The specific role of the perirhinal (PRC), entorhinal (ERC) and parahippocampal cortices (PHC) in supporting familiarity‐based recognition remains unknown. An fMRI study explored whether these medial temporal lobe (MTL) structures responded in the same way or differentially to familiarity as a function of stimulus type at recognition. A secondary aim was to explore whether the hippocampus responds in the same way to equally strong familiarity and recollection and whether this is influenced by the kind of stimulus involved. Univariate and multivariate analyses revealed that familiarity responses in the PRC, ERC, PHC and the amygdala are material‐specific. Specifically, the PRC and ERC selectively responded to object familiarity, while the PHC responded to both object and scene familiarity. The amygdala only responded to familiarity memory for faces. The hippocampus did not respond to stimulus familiarity for any of the three types of stimuli, but it did respond to recollection for all three types of stimuli. This was true even when recollection was contrasted to equally accurate familiarity. Overall, the findings suggest that the role of the MTL neocortices and the amygdala in familiarity‐based recognition depends on the kind of stimulus in memory, whereas the role of the hippocampus in recollection is independent of the type of cuing stimulus. © 2016 The Authors Hippocampus Published by Wiley Periodicals, Inc.     

Early face processing specificity: it's in the eyes!

Unlike most other objects that are processed analytically, faces are processed configurally. This configural processing is reflected early in visual processing following face inversion and contrast reversal, as an increase in the N170 amplitude, a scalp-recorded event-related potential. Here, we show that these face-specific effects are mediated by the eye region. That is, they occurred only when the eyes were present, but not when eyes were removed from the face. The N170 recorded to inverted and negative faces likely reflects the processing of the eyes. We propose a neural model of face processing in which face- and eye-selective neurons situated in the superior temporal sulcus region of the human brain respond differently to the face configuration and to the eyes depending on the face context. This dynamic response modulation accounts for the N170 variations reported in the literature. The eyes may be central to what makes faces so special.     

Single neuron studies of inferior temporal cortex

This paper reviews our experiments on the response properties of single neurons in inferior temporal (IT) cortex in the monkey that were carried out starting in 1965. It describes situational factors that led us to find neurons sensitive to images of faces and hands and summarizes the basic sensory properties of IT neurons. Subsequent developments on the cognitive properties of IT neurons and on imaging the responses of human temporal cortex to facial images are outlined. Finally, this paper summarizes recent results on fMRI imaging of the responses of temporal cortex to facial images.     

Differential neural activity in the human temporal lobe evoked by faces of family members and friends

In 6 patients, depth electrodes revealed differential evoked responses to familiar versus novel faces. These differential responses were obtained in the amygdala, hippocampus, and temporal neocortex but not in the dorsolateral frontal or cingulate cortex. The limbic and temporal structures that differentiated novel from familiar faces did not respond differentially to variations in luminance. Limbic structures and temporal cortex thus appear to participate in face recognition and in encoding the familiarity of visual experiences.     

Event‐related potential and functional MRI measures of face‐selectivity are highly correlated: A simultaneous ERP‐fMRI investigation

A face‐selective neural signal is reliably found in humans with functional MRI and event‐related potential (ERP) measures, which provide complementary information about the spatial and temporal properties of the neural response. However, because most neuroimaging studies so far have studied ERP and fMRI face‐selective markers separately, the relationship between them is still unknown. Here we simultaneously recorded fMRI and ERP responses to faces and chairs to examine the correlations across subjects between the magnitudes of fMRI and ERP face‐selectivity measures. Findings show that the face‐selective responses in the temporal lobe (i.e., fusiform gyrus—FFA) and superior temporal sulcus (fSTS), but not the face‐selective response in the occipital cortex (OFA), were highly correlated with the face‐selective N170 component. In contrast, the OFA was correlated with earlier ERPs at about 110 ms after stimulus‐onset. Importantly, these correlations reveal a temporal dissociation between the face‐selective area in the occipital lobe and face‐selective areas in the temporal lobe. Despite the very different time‐scale of the fMRI and EEG signals, our data show that a correlation analysis across subjects may be informative with respect to the latency in which different brain regions process information. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.     

Self-face recognition in social context

The concept of "social self" is often described as a representation of the self-reflected in the eyes or minds of others. Although the appearance of one's own face has substantial social significance for humans, neuroimaging studies have failed to link self-face recognition and the likely neural substrate of the social self, the medial prefrontal cortex (MPFC). We assumed that the social self is recruited during self-face recognition under a rich social context where multiple other faces are available for comparison of social values. Using functional magnetic resonance imaging (fMRI), we examined the modulation of neural responses to the faces of the self and of a close friend in a social context. We identified an enhanced response in the ventral MPFC and right occipitoparietal sulcus in the social context specifically for the self-face. Neural response in the right lateral parietal and inferior temporal cortices, previously claimed as self-face-specific, was unaffected for the self-face but unexpectedly enhanced for the friend's face in the social context. Self-face-specific activation in the pars triangularis of the inferior frontal gyrus, and self-face-specific reduction of activation in the left middle temporal gyrus and the right supramarginal gyrus, replicating a previous finding, were not subject to such modulation. Our results thus demonstrated the recruitment of a social self during self-face recognition in the social context. At least three brain networks for self-face-specific activation may be dissociated by different patterns of response-modulation in the social context, suggesting multiple dynamic self-other representations in the human brain.     

Preferential responses to faces in superior temporal and medial prefrontal cortex in three-year-old children

Perceiving faces and understanding emotions are key components of human social cognition. Prior research with adults and infants suggests that these social cognitive functions are supported by superior temporal cortex (STC) and medial prefrontal cortex (MPFC). We used functional near-infrared spectroscopy (fNIRS) to characterize functional responses in these cortical regions to faces in early childhood. Three-year-old children (n = 88, M(SD) = 3.15(.16) years) passively viewed faces that varied in emotional content and valence (happy, angry, fearful, neutral) and, for fearful and angry faces, intensity (100%, 40%), while undergoing fNIRS. Bilateral STC and MPFC showed greater oxygenated hemoglobin concentration values to all faces relative to objects. MPFC additionally responded preferentially to happy faces relative to neutral faces. We did not detect preferential responses to angry or fearful faces, or overall differences in response magnitude by emotional valence (100% happy vs. fearful and angry) or intensity (100% vs. 40% fearful and angry). In exploratory analyses, preferential responses to faces in MPFC were not robustly correlated with performance on tasks of early social cognition. These results link and extend adult and infant research on functional responses to faces in STC and MPFC and contribute to the characterization of the neural correlates of early social cognition.     

Associative recognition in a patient with selective hippocampal lesions and relatively normal item recognition

Previous work (Mayes et al., Hippocampus 12:325-340, 2002) found that patient YR, who suffered a selective bilateral lesion to the hippocampus in 1986, showed relatively preserved verbal and visual item recognition memory in the face of clearly impaired verbal and visual recall. In this study, we found that YR's Yes/No as well as forced-choice recognition of both intra-item associations and associations between items of the same kind was as well preserved as her item recognition memory. In contrast, YR was clearly impaired, and more so than she was on the above kinds of recognition, at recognition of associations between different kinds of information. Thus, her recognition memory for associations between objects and their locations, words and their temporal positions, abstract visual items or words and their temporal order, animal pictures and names of professions, faces and voices, faces and spoken names, words and definitions, and pictures and sounds, was clearly impaired. Several of the different information associative recognition tests at which YR was impaired could be compared with related item or inter-item association recognition tests of similar difficulty that she performed relatively normally around the same time. It is suggested that YR's familiarity memory for items, intra-item associations, and associations between items of the same kind was mediated by her intact medial temporal lobe cortices and was preserved, whereas her hippocampally mediated recall/recollection of these kinds of information was impaired. It is also suggested that the components of associations between different kinds of information are represented in distinct neocortical regions and that initially they only converge for memory processing within the hippocampus. No familiarity memory may exist in normal subjects for such associations, and, if so, YR's often chance recognition occurred because of her severe recall/recollection deficit. Conflicting data and views are discussed, and the way in which recall as well as item and associative recognition need to be systematically explored in patients with apparently selective hippocampal lesions, in order to resolve existing conflicts, is outlined.     

Recognition memory for faces and scenes in amnesia: dissociable roles of medial temporal lobe structures

The relative contributions of the hippocampus and the perirhinal cortex to recognition memory are currently the subject of intense debate. Whereas some authors propose that both structures play a similar role in recognition memory, others suggest that the hippocampus might mediate recollective and/or associative aspects of recognition memory, whereas the perirhinal cortex may mediate item memory. Here we investigate an alternative functional demarcation between these structures, following reports of stimulus-specific perceptual deficits in amnesics with medial temporal lobe (MTL) lesions. Using a novel recognition memory test for faces and scenes, participants with broad damage to MTL structures, which included the hippocampus and the perirhinal cortex, were impaired on both face and scene memory. By contrast, participants with damage limited to the hippocampus showed deficits only in memory for scenes. These findings imply that although both the hippocampus and surrounding cortex contribute to recognition memory, their respective roles can be distinguished according to the type of material to be remembered. This interaction between lesion site and stimulus category may explain some of the inconsistencies present in the literature.     

Single-Unit Recordings Reveal the Selectivity of a Human Face Area

The exquisite capacity of primates to detect and recognize faces is crucial for social interactions. Although disentangling the neural basis of human face recognition remains a key goal in neuroscience, direct evidence at the single-neuron level is limited. We recorded from face-selective neurons in human visual cortex in a region characterized by functional magnetic resonance imaging (fMRI) activations for faces compared with objects. The majority of visually responsive neurons in this fMRI activation showed strong selectivity at short latencies for faces compared with objects. Feature-scrambled faces and face-like objects could also drive these neurons, suggesting that this region is not tightly tuned to the visual attributes that typically define whole human faces. These single-cell recordings within the human face processing system provide vital experimental evidence linking previous imaging studies in humans and invasive studies in animal models.SIGNIFICANCE STATEMENT We present the first recordings of face-selective neurons in or near an fMRI-defined patch in human visual cortex. Our unbiased multielectrode array recordings (i.e., no selection of neurons based on a search strategy) confirmed the validity of the BOLD contrast (faces–objects) in humans, a finding with implications for all human imaging studies. By presenting faces, feature-scrambled faces, and face-pareidolia (perceiving faces in inanimate objects) stimuli, we demonstrate that neurons at this level of the visual hierarchy are broadly tuned to the features of a face, independent of spatial configuration and low-level visual attributes.