Neural Activation during Covert Processing of Positive Emotional Facial Expressions

Lesion studies indicate distinct neural systems for recognition of facial identity and emotion. Split-brain experiments also suggest that emotional evaluation of a stimulus can occur without conscious identification. The present study tested a hypothesis of a differential neural response, independent of explicit conscious mediation, to emotional compared to nonemotional faces. The experimental paradigm involved holding in mind an image of a face across a 45-s delay while regional cerebral blood flow was measured using positron emission tomography. Prior to the delay, a single face was presented with an explicit instruction to match it to one of two faces, photographed at different angles from the target face, presented at the end of the delay. Repeated blood flow measures were obtained while subjects held happy or neutral faces in mind or during a neutral control fixation condition without initial face presentation. The representation of emotional faces over a delay period, compared to either the nonemotional or the fixation condition, was associated with significant activation in the left ventral prefrontal cortex, the left anterior cingulate cortex, and the right fusiform gyrus. The findings support our hypothesis of a differential neural response to facial emotion, independent of conscious mediation, in regions implicated in the processing of faces and of emotions.     

Faces are represented holistically in the human occipito-temporal cortex

Two identical top parts of a face photograph look different if their bottom parts differ. This perceptual illusion, the "face composite effect", is taken as strong evidence that faces are processed as a whole rather than as a collection of independent features. To test the hypothesis that areas responding preferentially to faces in the human brain represent faces holistically, we recorded functional magnetic resonance imaging (fMRI) during an adaptation paradigm with the composite face illusion. In both the middle fusiform gyrus (MFG) and the inferior occipital gyrus (IOG), we observed a significantly larger response to the same top face when it was aligned with different bottom parts than with the same bottom part, with a most robust effect in the right middle fusiform gyrus. This difference was not found when the top and the bottom face parts were spatially misaligned or when the faces were presented upside-down. These findings indicate that facial features are integrated into holistic face representations in areas of the human visual cortex responding preferentially to faces.     

Developmental differences in the neural bases of the face inversion effect show progressive tuning of face-selective regions to the upright orientation

Face inversion hinders face processing in adults, while not affecting children in the same way. This fMRI study examines the neural underpinnings of the behavioral face inversion effect (FIE) from childhood to adulthood, and how face-selective regions in the brain may change with development. Adults, children, and teens performed a facial expression decision on upright and inverted face stimuli. In the right hemisphere (RH) all age groups showed similar profiles of neural activation for upright faces, but important developmental differences occured for inverted faces. For inverted faces, adults, and to a lesser degree teens, exhibited decreased levels of activity in the face-selective, right lateral fusiform gyrus (LFG). However, children exhibited greater activation for inverted than for upright faces in the same region. We found similar, but less robust, developmental trends in the right superior temporal sulcus (STS) and medial fusiform gyrus (MFG). Furthermore, the present study identifies the right LFG as the primary neural correlate of the behavioral FIE, and therefore of face processing expertise, by showing a significant correlation between the behavioral FIE and the neural FIE only in this region. Finally, the present findings shed some light on at least one of the possible mechanisms underlying the development of face processing expertise, by suggesting a progressive tuning of face-selective regions in the right hemisphere to the upright orientation, that extends well into adolescence.     

The functionally defined right occipital and fusiform "face areas" discriminate novel from visually familiar faces

Neuroimaging (PET and fMRI) studies have identified a set of brain areas responding more to faces than to other object categories in the visual extrastriate cortex of humans. This network includes the middle lateral fusiform gyrus (the fusiform face area, or FFA) as well as the inferior occipital gyrus (occipital face area, OFA). The exact functions of these areas in face processing remain unclear although it has been argued that their primary function is to distinguish faces from nonface object categories-"face detection"-or also to discriminate among faces, irrespective of their visual familiarity to the observer. Here, we combined the data from two previous positron emission tomography (PET) studies to show that the functionally defined face areas are involved in the automatic discrimination between unfamiliar faces and familiar faces. Consistent with previous studies, a face localizer contrast (faces-objects) revealed bilateral activation in the middle lateral fusiform gyrus (FFA, BA37) and in the right inferior occipital cortex (OFA, BA19). Within all the regions of the right hemisphere, larger levels of activation were found for unfamiliar as compared to familiar faces. These results suggest that the very same areas involved in categorizing faces at the basic or individual level, play a role in differentiating familiar faces from new faces, showing an overlap between visual and presemantic mnesic representations of faces in the right hemisphere.     

Activation of the fusiform gyrus when individuals with autism spectrum disorder view faces

Prior imaging studies have failed to show activation of the fusiform gyrus in response to emotionally neutral faces in individuals with autism spectrum disorder (ASD) [Critchley et al., Brain 124 (2001) 2059; Schultz et al., Arch. Gen. Psychiatry 57 (2000) 331]. However, individuals with ASD do not typically exhibit the striking behavioral deficits that might be expected to result from fusiform gyrus damage, such as those seen in prosopagnosia, and their deficits appear to extend well beyond face identification to include a wide range of impairments in social perceptual processing. In this study, our goal was to further assess the question of whether individuals with ASD have abnormal fusiform gyrus activation to faces. We used high-field (3 T) functional magnetic resonance imaging to study face perception in 11 adult individuals with autism spectrum disorder (ASD) and 10 normal controls. We used face stimuli, object stimuli, and sensory control stimuli (Fourier scrambled versions of the face and object stimuli) containing a fixation point in the center to ensure that participants were looking at and attending to the images as they were presented. We found that individuals with ASD activated the fusiform face area and other brain areas normally involved in face processing when they viewed faces as compared to non-face stimuli. These data indicate that the face-processing deficits encountered in ASD are not due to a simple dysfunction of the fusiform area, but to more complex anomalies in the distributed network of brain areas involved in social perception and cognition.     

Let's face it: it's a cortical network

Face perception elicits activation within a distributed cortical network in the human brain. The network includes visual ("core") regions, which process invariant facial features, as well as limbic and prefrontal ("extended") regions that process changeable aspects of faces. Analysis of effective connectivity reveals that the major entry node in the "face network" is the lateral fusiform gyrus and that the functional coupling between the core and the extended systems is content-dependent. A model for face perception is proposed, in which the flow of information through the network is shaped by cognitive demands.     

Decoding Face Information in Time, Frequency and Space from Direct Intracranial Recordings of the Human Brain

Faces are processed by a neural system with distributed anatomical components, but the roles of these components remain unclear. A dominant theory of face perception postulates independent representations of invariant aspects of faces (e.g., identity) in ventral temporal cortex including the fusiform gyrus, and changeable aspects of faces (e.g., emotion) in lateral temporal cortex including the superior temporal sulcus. Here we recorded neuronal activity directly from the cortical surface in 9 neurosurgical subjects undergoing epilepsy monitoring while they viewed static and dynamic facial expressions. Applying novel decoding analyses to the power spectrogram of electrocorticograms (ECoG) from over 100 contacts in ventral and lateral temporal cortex, we found better representation of both invariant and changeable aspects of faces in ventral than lateral temporal cortex. Critical information for discriminating faces from geometric patterns was carried by power modulations between 50 to 150 Hz. For both static and dynamic face stimuli, we obtained a higher decoding performance in ventral than lateral temporal cortex. For discriminating fearful from happy expressions, critical information was carried by power modulation between 60–150 Hz and below 30 Hz, and again better decoded in ventral than lateral temporal cortex. Task-relevant attention improved decoding accuracy more than10% across a wide frequency range in ventral but not at all in lateral temporal cortex. Spatial searchlight decoding showed that decoding performance was highest around the middle fusiform gyrus. Finally, we found that the right hemisphere, in general, showed superior decoding to the left hemisphere. Taken together, our results challenge the dominant model for independent face representation of invariant and changeable aspects: information about both face attributes was better decoded from a single region in the middle fusiform gyrus.     

The fusiform face area is tuned for curvilinear patterns with more high-contrasted elements in the upper part

The ability to identify conspecifics from the face is of primary interest for human social behavior. Newborns' visual preference for schematic face-like stimuli has been recently related to a sensitivity for visual patterns with a greater number of elements in the upper compared to the lower part. At the adult level, neuroimaging studies have identified a network of cortical areas devoted to the detection and identification of faces. However, whether and how low-level structural properties of face stimuli contribute to the preferential response to faces in these areas remain to be clarified. Using functional magnetic resonance imaging (fMRI), here we investigated whether the adults' face-sensitive cortical areas show a preference for top-heavy patterns, similarly to newborns' preference. Twelve participants were presented with head-shaped and square patterns with either more elements in the upper or the lower vertical part. In the right fusiform gyrus ('fusiform face area', FFA), an area showing a preference for faces over other visual object categories, there was a larger activation for curvilinear patterns with more high-contrast elements in the upper part, even though these patterns were not perceived as face stimuli. These findings provide direct evidence that the FFA is tuned for geometrical properties fitting best with the structure of faces, a computational mechanism that might drive the automatic detection of faces in the visual world.     

Dynamics of processing invisible faces in the brain: automatic neural encoding of facial expression information

The fusiform face area (FFA) and the superior temporal sulcus (STS) are suggested to process facial identity and facial expression information respectively. We recently demonstrated a functional dissociation between the FFA and the STS as well as correlated sensitivity of the STS and the amygdala to facial expressions using an interocular suppression paradigm [Jiang, Y., He, S., 2006. Cortical responses to invisible faces: dissociating subsystems for facial-information processing. Curr. Biol. 16, 2023-2029.]. In the current event-related brain potential (ERP) study, we investigated the temporal dynamics of facial information processing. Observers viewed neutral, fearful, and scrambled face stimuli, either visibly or rendered invisible through interocular suppression. Relative to scrambled face stimuli, intact visible faces elicited larger positive P1 (110-130 ms) and larger negative N1 or N170 (160-180 ms) potentials at posterior occipital and bilateral occipito-temporal regions respectively, with the N170 amplitude significantly greater for fearful than neutral faces. Invisible intact faces generated a stronger signal than scrambled faces at 140-200 ms over posterior occipital areas whereas invisible fearful faces (compared to neutral and scrambled faces) elicited a significantly larger negative deflection starting at 220 ms along the STS. These results provide further evidence for cortical processing of facial information without awareness and elucidate the temporal sequence of automatic facial expression information extraction.     

Cortical mechanisms of visual self-recognition

Several lines of evidence have suggested that visual self-recognition is supported by a special brain mechanism; however, its functional anatomy is of great controversy. We performed an event-related functional magnetic resonance imaging (fMRI) study to identify brain regions selectively involved in recognition of one's own face. We presented pictures of each subject's own face (SELF) and a prelearned face of an unfamiliar person (CONT), as well as two personally familiar faces with high and low familiarity (HIGH and LOW, respectively) to test selectivity of activation to the SELF face. Compared with the CONT face, activation selective to the SELF face was observed in the right occipito-temporo-parietal junction and frontal operculum, as well as in the left fusiform gyrus. On the contrary, the temporoparietal junction in both the hemispheres and the left anterior temporal cortex, which were activated during recognition of HIGH and/or LOW faces, were not activated during recognition of the SELF face. The results confirmed the partial distinction of the brain mechanism involved in recognition of personally familiar faces and that in recognition of one's own face. The right occipito-temporo-parietal junction and frontal operculum appear to compose a network processing motion-action contingency, a role of which in visual self-recognition has been suggested in previous behavioral studies. Activation of the left fusiform gyrus selective to one's own face was consistent with the results of two previous functional imaging studies and a neuropsychological report, possibly suggesting its relationship with lexical processing.     

Neural responses to Mooney images reveal a modular representation of faces in human visual cortex

The way in which information about objects is represented in visual cortex remains controversial. One model of human object recognition poses that information is processed in modules, highly specialised for different categories of objects; an opposing model appeals to a distributed representation across a large network of visual areas. We addressed this debate by monitoring activity in face- and object-selective areas while human subjects viewed ambiguous face stimuli (Mooney faces). The measured neural response in the face-selective region of the fusiform gyrus was greater when subjects reported seeing a face than when they perceived the image as a collection of blobs. In contrast, there was no difference in magnetic resonance response between face and no-face perceived events in either the face-selective voxels of the superior temporal sulcus or the object-selective voxels of the parahippocampal gyrus and lateral occipital complex. These results challenge the concept that neural representation of faces is distributed and overlapping and suggest that the fusiform gyrus is tightly linked to the awareness of faces.     

Reactivation of visual cortex during memory retrieval: content specificity and emotional modulation

Studies on memory retrieval suggest a reactivation of cortical regions engaged during encoding, such that visual or auditory areas reactivate for visual or auditory memories. The content specificity and any emotion dependency of such reactivations are still unclear. Because distinct visual areas are specialized in processing distinct stimulus categories, we tested for face and word specific reactivations during a memory task using functional magnetic resonance imaging (fMRI). Furthermore, because visual processing and memory are both modulated by emotion, we compared reactivation for stimuli encoded in a neutral or emotionally significant context. In the learning phase, participants studied pairs of stimuli that consisted of either a scene and a face, or a scene and a word. Scenes were either neutral or negative, but did not contain faces or words. In the test phase scenes were presented alone (one in turn), and participants indicated whether it was previously paired with a face, a word, or was new. Results from the test phase showed activation in a functionally defined face-responsive region in the right fusiform gyrus, as well as in a word-responsive region in the left inferior temporal gyrus, for scenes previously paired with faces and words, respectively. Reactivation tended to be larger in both the face- and word-responsive regions when the associated scene was negative as compared to neutral. However, relative to neutral context, the recall of faces and words paired with a negative context produced smaller activations in brain regions associated with social and semantic processing, respectively, as well as poorer memory performance overall. Taken together, these results support the idea of cortical memory reactivations, even at a content-specific level, and further suggest that emotional context may produce opposite effects on reactivations in early sensory areas and more elaborate processing in higher-level cortical areas.     

Seeing John Malkovich: the neural substrates of person categorization

Neuroimaging data have implicated regions of the ventral temporal cortex (e.g., fusiform gyrus) as functionally important in face recognition. Recent evidence, however, suggests that these regions are not face-specific, but rather reflect subordinate-level categorical processing underpinned by perceptual expertise. Moreover, when people possess expertise for a particular class of stimuli (e.g., faces), subordinate-level identification is thought to be an automatic process. To investigate the neural substrates of person construal, we used functional magnetic resonance imaging (fMRI) to contrast brain activity while participants judged faces at different levels of semantic specificity (i.e., identity vs. occupation). The results revealed that participants were quicker to access identity than occupational knowledge. In addition, greater activity was observed in bilateral regions of the fusiform gyrus on identity than occupation trials. Taken together, these findings support the viewpoint that person construal is characterized by the ability to access subordinate-level semantic information about people, a capacity that is underpinned by neural activity in discrete regions of the ventral temporal cortex.     

Effect of familiarity on the processing of human faces

Most brain imaging studies on face perception have investigated the processing of unknown faces and addressed mainly the question of specific face processing in the human brain. The goal of this study was to highlight the effects of familiarity on the visual processing of faces. Using [15O]water 3D Positron Emission Tomography, regional cerebral blood flow distribution was measured in 11 human subjects performing an identical task (gender categorization) on both unknown and known faces. Subjects also performed two control tasks (a face recognition task and a visual pattern discrimination task). They were scanned after a training phase using videotapes during which they had been familiarized with and learned to recognize a set of faces. Two major results were obtained. On the one hand, we found bilateral activations of the fusiform gyri in the three face conditions, including the so-called fusiform-face area, a region in the right fusiform gyrus specifically devoted to face processing. This common activation suggests that different cognitive tasks performed on known and unknown faces require the involvement of this fusiform region. On the other hand, specific regional cerebral blood flow changes were related to the processing of known and unknown faces. The left amygdala, a structure involved in implicit learning of visual representations, was activated by the categorization task on unknown faces. The same task on known faces induced a relative decrease of activity in early visual areas. These differences between the two categorization tasks reveal that the human brain processes known and unknown faces differently.     

Cholinergic enhancement modulates neural correlates of selective attention and emotional processing

Neocortical cholinergic afferents are proposed to influence both selective attention and emotional processing. In a study of healthy adults we used event-related fMRI while orthogonally manipulating attention and emotionality to examine regions showing effects of cholinergic modulation by the anticholinesterase physostigmine. Either face or house pictures appeared at task-relevant locations, with the alternative picture type at irrelevant locations. Faces had either neutral or fearful expressions. Physostigmine increased relative activity within the anterior fusiform gyrus for faces at attended, versus unattended, locations, but decreased relative activity within the posterolateral occipital cortex for houses in attended, versus unattended, locations. A similar pattern of regional differences in the effect of physostigmine on cue-evoked responses was also present in the absence of stimuli. Cholinergic enhancement augmented the relative neuronal response within the middle fusiform gyrus to fearful faces, whether at attended or unattended locations. By contrast, physostigmine influenced responses in the orbitofrontal, intraparietal and cingulate cortices to fearful faces when faces occupied task-irrelevant locations. These findings suggest that acetylcholine may modulate both selective attention and emotional processes through independent, region-specific effects within the extrastriate cortex. Furthermore, cholinergic inputs to the frontoparietal cortex may influence the allocation of attention to emotional information.     

Passive and active recognition of one's own face

Facial identity recognition has been studied mainly with explicit discrimination requirement and faces of social figures in previous human brain imaging studies. We performed a PET activation study with normal volunteers in facial identity recognition tasks using the subject's own face as visual stimulus. Three tasks were designed so that the activation of the visual representation of the face and the effect of sustained attention to the representation could be separately examined: a control-face recognition task (C), a passive own-face recognition task (no explicit discrimination was required) (P), and an active own-face recognition task (explicit discrimination was required) (A). Increased skin conductance responses during recognition of own face were seen in both task P and task A, suggesting the occurrence of psychophysiological changes during recognition of one's own face. The left fusiform gyrus, the right supramarginal gyrus, the left putamen, and the right hypothalamus were activated in tasks P and A compared with task C. The left fusiform gyrus and the right supramarginal gyrus are considered to be involved in the representation of one's own face. The activation in the right supramarginal gyrus may be associated with the representation of one's own face as a part of one's own body. The prefrontal cortices, the right anterior cingulate, the right presupplementary motor area, and the left insula were specifically activated during task A compared with tasks C and P, indicating that these regions may be involved in the sustained attention to the representation of one's own face.     

Seen Gaze-Direction Modulates Fusiform Activity and Its Coupling with Other Brain Areas during Face Processing

Gaze-contact is often a preliminary to social interaction and so constitutes a signal for the allocation of processing resources to the gazing face. We investigated how gaze direction influences face processing in an fMRI study, where seen gaze and head direction could independently be direct or deviated. Direct relative to averted gaze elicited stronger activation for faces in ventral occipitotemporal cortices around the fusiform gyrus, regardless of head orientation. Moreover, direct gaze led to greater correlation between activity in the fusiform and the amygdala, a region associated with emotional responses and stimulus saliency. By contrast, faces with averted gaze (again, regardless of head orientation) yielded increased correlation between activity in the fusiform and the intraparietal sulcus, a region associated with shifting attention to the periphery.     

Neural Correlates of Perceiving Emotional Faces and Bodies in Developmental Prosopagnosia: An Event-Related fMRI-Study

Many people experience transient difficulties in recognizing faces but only a small number of them cannot recognize their family members when meeting them unexpectedly. Such face blindness is associated with serious problems in everyday life. A better understanding of the neuro-functional basis of impaired face recognition may be achieved by a careful comparison with an equally unique object category and by a adding a more realistic setting involving neutral faces as well facial expressions. We used event-related functional magnetic resonance imaging (fMRI) to investigate the neuro-functional basis of perceiving faces and bodies in three developmental prosopagnosics (DP) and matched healthy controls. Our approach involved materials consisting of neutral faces and bodies as well as faces and bodies expressing fear or happiness. The first main result is that the presence of emotional information has a different effect in the patient vs. the control group in the fusiform face area (FFA). Neutral faces trigger lower activation in the DP group, compared to the control group, while activation for facial expressions is the same in both groups. The second main result is that compared to controls, DPs have increased activation for bodies in the inferior occipital gyrus (IOG) and for neutral faces in the extrastriate body area (EBA), indicating that body and face sensitive processes are less categorically segregated in DP. Taken together our study shows the importance of using naturalistic emotional stimuli for a better understanding of developmental face deficits.     

View-independent coding of face identity in frontal and temporal cortices is modulated by familiarity: an event-related fMRI study

Face recognition is a unique visual skill enabling us to recognize a large number of person identities, despite many differences in the visual image from one exposure to another due to changes in viewpoint, illumination, or simply passage of time. Previous familiarity with a face may facilitate recognition when visual changes are important. Using event-related fMRI in 13 healthy observers, we studied the brain systems involved in extracting face identity independent of modifications in visual appearance during a repetition priming paradigm in which two different photographs of the same face (either famous or unfamiliar) were repeated at varying delays. We found that functionally defined face-selective areas in the lateral fusiform cortex showed no repetition effects for faces across changes in image views, irrespective of pre-existing familiarity, suggesting that face representations formed in this region do not generalize across different visual images, even for well-known faces. Repetition of different but easily recognizable views of an unfamiliar face produced selective repetition decreases in a medial portion of the right fusiform gyrus, whereas distinct views of a famous face produced repetition decreases in left middle temporal and left inferior frontal cortex selectively, but no decreases in fusiform cortex. These findings reveal that different views of the same familiar face may not be integrated within a single representation at initial perceptual stages subserved by the fusiform face areas, but rather involve later processing stages where more abstract identity information is accessed.     

Distinct cortical networks for the detection and identification of human body

In the human brain information about bodies and faces is processed in specialized cortical regions named EBA and FBA (extrastriate and fusiform body area) and OFA and FFA (occipital and fusiform face area), respectively. Here we investigate with functional magnetic resonance imaging (fMRI) the cortical areas responsible for the identification of individual bodies and the distinction between ‘self’ and ‘others’. To this end we presented subjects with images of unfamiliar and familiar bodies and their own body. We identified separate coactivation networks for body-detection (processing body related information), body-identification (processing of information relating to individual bodies) and self-identification (distinction of self from others). Body detection involves the EBA in both hemispheres, and in the right hemisphere: the FBA and areas in the IPL (inferior parietal lobe). Body identification involves areas in the inferior frontal gyrus (IFG) of both hemispheres and in the right hemisphere areas in the medial frontal gyrus (MFG), in the cingulate gyrus (CG), in the central (CS) and the post-central sulcus (PCS), in the inferior parietal lobe (IPL) and the FBA. When the recognition of one's own body is contrasted to the identification of familiar bodies, differential activation is observed in areas of the inferior parietal lobe (IPL) and inferior parietal sulcus (IPS) of the right hemisphere, and in the posterior orbital gyrus (pOrbG) and in the lateral occipital gyrus (LOG) of the left hemisphere. Thus, identification of individual bodies and self-other distinction involve in addition to the classical occipito-parietal network a parieto-frontal network. Interestingly, the EBA shows no differential activation for distinctions between familiar or unfamiliar bodies or recognition of one's own body.