Effect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus

We used functional MRI with an event-related design to dissociate the brain activation in the fusiform gyrus (FG) and posterior superior temporal sulcus (STS) for multiple face and gaze orientations. The event-related design allowed for concurrent behavioral analysis, which revealed a significant effect of both head and gaze orientation on the speed of gaze processing, with the face and gaze forward condition showing the fastest reaction times. In conjunction with this behavioral finding, the FG responded with the greatest activation to face and gaze forward, perhaps reflecting the unambiguous social salience of congruent face and gaze directed toward the viewer. Random effects analysis showed greater activation in both the FG and posterior STS when the subjects viewed a direct face compared to an angled face, regardless of gaze direction. Additionally, the FG showed greater activation for forward gaze compared to angled gaze, but only when the face was forward. Together, these findings suggest that head orientation has a significant effect on gaze processing and these effects are manifest not only in the STS, but also the FG.     

Processing social aspects of human gaze: a combined fMRI-DTI study

Human gaze is a critical social cue that can reveal intentions and dispositions of others. The right posterior superior temporal sulcus (pSTS) is thought to be critically involved in processing eye gaze information. We combined diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI) to identify direct neural connections of right pSTS and to determine how these connections are modulated by the social significance of perceived gaze shifts. Participants saw faces with direct or averted gaze during event-related fMRI. Half of these faces remained static, and half displayed a dynamic gaze shift either towards or away from the subject. Social attention (dynamic gaze shifts towards the observer) not only increased activity in right pSTS, but also its functional connectivity with the right anterior insula (aIns) and right fusiform gyrus (FG). However, direct fiber connections from pSTS were demonstrated by DTI for the right aIns, but not the right FG. Moreover, the right FG responded to eye motion irrespective of direction and social significance; whereas the right aIns was selectively sensitive to social significance (i.e. gaze shifts towards the observer), but not generally to eye motion. We conclude that the social aspects of mutual gaze contact are processed by direct fiber pathways between right pSTS and right aIns; whereas increased connectivity with FG could reflect an enhanced perceptual analysis of changing facial features in dynamic gaze conditions and involves indirect fiber pathways with pSTS, perhaps via motion-selective regions in middle temporal (MT) gyrus that exhibited strong white-matter connections with both pSTS and FG and could thus provide inputs to these two areas.     

Increased amygdala activation to averted versus direct gaze in humans is independent of valence of facial expression

A recent study in monkeys showed that averted gaze displayed by conspecifics activates the amygdala regardless of the valence of facial expression (angry, neutral, appeasing). In contrast to this result, previous findings on amygdala activation to gaze information in humans have been mainly interpreted in terms of threat-related processing of facial signals. In the present event-related functional magnetic resonance imaging study with humans, we investigated brain responses to angry, happy, and neutral faces with either direct or averted gaze. Averted versus direct gaze induced increased amygdala responses regardless of facial expression. This finding suggests a critical role of the amygdala in gaze processing independently of the valence of facial expression. Furthermore, the effect of gaze on amygdalar responses seems to be preserved across nonhuman and human primates.     

Duration matters: Dissociating neural correlates of detection and evaluation of social gaze

The interpretation of interpersonal gaze behavior requires the use of complex cognitive processes and guides social interactions. Among a variety of different gaze characteristics, gaze direction and gaze duration modulate crucially the meaning of the “social gaze”. Nevertheless, prior neuroimaging studies disregarded the relevance of gaze duration by focusing on gaze direction only.The present functional magnetic resonance imaging (fMRI) study focused on the differentiation of these two gaze parameters. Therefore direct gaze displayed by virtual characters was contrasted with averted gaze and, additionally, systematically varied with respect to gaze duration (i.e., 1, 2.5 or 4 s). Consistent with prior findings, behavioral data showed that likeability was higher for direct than for averted gaze and increased linearly with increasing direct gaze duration. On the neural level, distinct brain regions were associated with the processing of gaze direction and gaze duration: (i) the comparison between direct and averted gaze revealed activations in bilateral occipito-temporal regions including the posterior superior temporal sulcus (pSTS); (ii) whereas increasing duration of direct gaze evoked differential neural responses in the medial prefrontal cortex (MPFC) including orbitofrontal and paracingulate regions.The results suggest two complementary cognitive processes related to different gaze parameters. On the one hand, the recruitment of multimodal sensory regions in the pSTS indicates detection of gaze direction via complex visual analysis. On the other hand, the involvement of the MPFC associated with outcome monitoring and mentalizing indicates higher-order social cognitive processes related to evaluation of the ongoing communicational input conveyed by direct gaze duration.     

Developmental differences in the neural bases of the face inversion effect show progressive tuning of face-selective regions to the upright orientation

Face inversion hinders face processing in adults, while not affecting children in the same way. This fMRI study examines the neural underpinnings of the behavioral face inversion effect (FIE) from childhood to adulthood, and how face-selective regions in the brain may change with development. Adults, children, and teens performed a facial expression decision on upright and inverted face stimuli. In the right hemisphere (RH) all age groups showed similar profiles of neural activation for upright faces, but important developmental differences occured for inverted faces. For inverted faces, adults, and to a lesser degree teens, exhibited decreased levels of activity in the face-selective, right lateral fusiform gyrus (LFG). However, children exhibited greater activation for inverted than for upright faces in the same region. We found similar, but less robust, developmental trends in the right superior temporal sulcus (STS) and medial fusiform gyrus (MFG). Furthermore, the present study identifies the right LFG as the primary neural correlate of the behavioral FIE, and therefore of face processing expertise, by showing a significant correlation between the behavioral FIE and the neural FIE only in this region. Finally, the present findings shed some light on at least one of the possible mechanisms underlying the development of face processing expertise, by suggesting a progressive tuning of face-selective regions in the right hemisphere to the upright orientation, that extends well into adolescence.     

Effect of familiarity on the processing of human faces

Most brain imaging studies on face perception have investigated the processing of unknown faces and addressed mainly the question of specific face processing in the human brain. The goal of this study was to highlight the effects of familiarity on the visual processing of faces. Using [15O]water 3D Positron Emission Tomography, regional cerebral blood flow distribution was measured in 11 human subjects performing an identical task (gender categorization) on both unknown and known faces. Subjects also performed two control tasks (a face recognition task and a visual pattern discrimination task). They were scanned after a training phase using videotapes during which they had been familiarized with and learned to recognize a set of faces. Two major results were obtained. On the one hand, we found bilateral activations of the fusiform gyri in the three face conditions, including the so-called fusiform-face area, a region in the right fusiform gyrus specifically devoted to face processing. This common activation suggests that different cognitive tasks performed on known and unknown faces require the involvement of this fusiform region. On the other hand, specific regional cerebral blood flow changes were related to the processing of known and unknown faces. The left amygdala, a structure involved in implicit learning of visual representations, was activated by the categorization task on unknown faces. The same task on known faces induced a relative decrease of activity in early visual areas. These differences between the two categorization tasks reveal that the human brain processes known and unknown faces differently.     

Dissociation between overt and unconscious face processing in fusiform face area

The precise role of the fusiform face area (FFA) in face processing remains controversial. In this study, we investigated to what degree FFA activation reflects additional functions beyond face perception. Seven volunteers underwent rapid event-related functional magnetic resonance imaging while they performed a face-encoding and a face-recognition task. During face encoding, activity in the FFA for individual faces predicted whether the individual face was subsequently remembered or forgotten. However, during face recognition, no difference in FFA activity between consciously remembered and forgotten faces was observed, but the activity of FFA differentiated if a face had been seen previously or not. This demonstrated a dissociation between overt recognition and unconscious discrimination of stimuli, suggesting that physiological processes of face recognition can take place, even if not all of its operations are made available to consciousness.     

MEG/EEG sources of the 170-ms response to faces are co-localized in the fusiform gyrus

The 170-ms electrophysiological processing stage (N170 in EEG, M170 in MEG) is considered an important computational step in face processing. Hence its neuronal sources have been modelled in several studies. The current study aimed to specify the relation of the dipolar sources underlying N170 and M170. Whole head EEG and MEG were measured simultaneously during the presentation of unfamiliar faces. An Independent Component Analysis (ICA) was applied to the data prior to localization. N170 and M170 were then modelled with a pair of dipoles in a four-shell ellipse (EEG)/homogeneous sphere (MEG) arranged symmetrically across midline. The dipole locations were projected onto the individual structural MR brain images. Dipoles were localized in fusiform gyri in ten out of eleven individuals for EEG and in seven out of eleven for MEG. N170 and M170 were co-localized in the fusiform gyrus in six individuals. The ICA shifted some of the single-subject dipoles up from cerebellum to fusiform gyrus mainly due to the removal of cardiac activity. The group mean dipole locations were also found in posterior fusiform gyri, and did not differ significantly between EEG and MEG. The result was replicated in a repeated measurement 3 months later.     

The functionally defined right occipital and fusiform "face areas" discriminate novel from visually familiar faces

Neuroimaging (PET and fMRI) studies have identified a set of brain areas responding more to faces than to other object categories in the visual extrastriate cortex of humans. This network includes the middle lateral fusiform gyrus (the fusiform face area, or FFA) as well as the inferior occipital gyrus (occipital face area, OFA). The exact functions of these areas in face processing remain unclear although it has been argued that their primary function is to distinguish faces from nonface object categories-"face detection"-or also to discriminate among faces, irrespective of their visual familiarity to the observer. Here, we combined the data from two previous positron emission tomography (PET) studies to show that the functionally defined face areas are involved in the automatic discrimination between unfamiliar faces and familiar faces. Consistent with previous studies, a face localizer contrast (faces-objects) revealed bilateral activation in the middle lateral fusiform gyrus (FFA, BA37) and in the right inferior occipital cortex (OFA, BA19). Within all the regions of the right hemisphere, larger levels of activation were found for unfamiliar as compared to familiar faces. These results suggest that the very same areas involved in categorizing faces at the basic or individual level, play a role in differentiating familiar faces from new faces, showing an overlap between visual and presemantic mnesic representations of faces in the right hemisphere.     

Differential selectivity for dynamic versus static information in face-selective cortical regions

Neuroimaging studies have identified multiple face-selective regions in human cortex but the functional division of labor between these regions is not yet clear. A central hypothesis, with some empirical support, is that face-selective regions in the superior temporal sulcus (STS) are particularly responsive to dynamic information in faces, whereas the fusiform face area (FFA) computes the static or invariant properties of faces. Here we directly tested this hypothesis by measuring the magnitude of response in each region to both dynamic and static stimuli. Consistent with the hypothesis, we found that the response to movies of faces was not significantly different from the response to static images of faces from these same movies in the right FFA and right occipital face area (OFA). By contrast the face-selective region in the right posterior STS (pSTS) responded nearly three times as strongly to dynamic faces as to static faces, and a face-selective region in the right anterior STS (aSTS) responded to dynamic faces only. Both of these regions also responded more strongly to moving faces than to moving bodies, indicating that they are preferentially engaged in processing dynamic information from faces, not in more general processing of any dynamic social stimuli. The response to dynamic and static faces was not significantly different in a third face-selective region in the posterior continuation of the STS (pcSTS). The strong selectivity of face-selective regions in the pSTS and aSTS, but not the FFA, OFA or pcSTS, for dynamic face information demonstrates a clear functional dissociation between different face-selective regions, and provides further clues into their function.     

The Role of the Fusiform Gyrus in Successful Encoding of Face Stimuli

PET was used to measure regional cerebral blood flow (rCBF) while memorizing pictures of unfamiliar human faces presented one at a time (FaceMemory). Other conditions included: (1) FaceRepeat—memorization of four individual faces presented repeatedly; (2) FaceWatching—viewing passively single faces without overt memory demands; and (3) Scrambled—counting dots superimposed on pictures of scrambled faces. After each FaceMemory condition and after the final FaceWatching condition scan, recall was tested by measuring face recognition. Contrasting FaceMemory and Scrambled conditions revealed several temporal activations: right midfusiform and bilateral anterior fusiform gyri. Contrasting FaceWatching and Scrambled conditions showed bilateral activation in the temporal poles and in the anterior fusiform gyri. No hippocampal activation arose from any contrast. Region of interest analyses on the above areas showed correlations with performance: (1) only rCBF in the right midfusiform correlated positively with encoding during the FaceMemory and FaceWatching conditions; (2) in the right temporal polar cortex rCBF decreased during FaceMemory and correlated positively with performance, whereas rCBF increased during FaceWatching and correlated negatively with incidental performance; and (3) activity in the anterior fusiform gyri remained constant across the conditions of FaceMemory, FaceRepeat, FaceWatching, and Scrambled and was uncorrelated with performance. These data suggest an expanded mnemonic role for the right midfusiform in depth of processing/encoding of face information, temporal polar cortex in face perception and recognition, and anterior fusiform activity in featural visual feature processing.     

Scale invariant adaptation in fusiform face-responsive regions

Several functional neuroimaging studies have observed response adaptation in face-sensitive regions when repeating identical face stimuli. To address whether this was due to low-level stimulus properties or facial identity, we decomposed pictures of faces into pictures preserving only the lower or higher parts of the normal frequency spectrum. In an event-related functional neuroimaging study, pairs of such pictures were sequentially presented that showed the same or different persons in the same or different frequency bands. This factorial design allowed to separate effects related to repetition of personal identity from those related to identical stimulus properties. In a random effects group analysis, activation in the right fusiform region was affected by repetition of personal identity regardless of changing or constant spatial scale. Responses in the more medial and posterior fusiform and lingual regions adapted with repetition of the same frequency band. An analysis in regions of interest determined individually as face responsive showed that repetition decreases for the same faces in fusiform face-responsive regions generalized across spatial frequency bands. Our results therefore point to a role of this area in discriminating individual faces at a level of representation that is invariant to changes in low-level stimulus properties, as spatial scale. The same invariance could not be detected in more posterior occipital face-responsive regions.     

View-independent coding of face identity in frontal and temporal cortices is modulated by familiarity: an event-related fMRI study

Face recognition is a unique visual skill enabling us to recognize a large number of person identities, despite many differences in the visual image from one exposure to another due to changes in viewpoint, illumination, or simply passage of time. Previous familiarity with a face may facilitate recognition when visual changes are important. Using event-related fMRI in 13 healthy observers, we studied the brain systems involved in extracting face identity independent of modifications in visual appearance during a repetition priming paradigm in which two different photographs of the same face (either famous or unfamiliar) were repeated at varying delays. We found that functionally defined face-selective areas in the lateral fusiform cortex showed no repetition effects for faces across changes in image views, irrespective of pre-existing familiarity, suggesting that face representations formed in this region do not generalize across different visual images, even for well-known faces. Repetition of different but easily recognizable views of an unfamiliar face produced selective repetition decreases in a medial portion of the right fusiform gyrus, whereas distinct views of a famous face produced repetition decreases in left middle temporal and left inferior frontal cortex selectively, but no decreases in fusiform cortex. These findings reveal that different views of the same familiar face may not be integrated within a single representation at initial perceptual stages subserved by the fusiform face areas, but rather involve later processing stages where more abstract identity information is accessed.     

Parametric design and correlational analyses help integrating fMRI and electrophysiological data during face processing

Face perception is typically associated with activation in the inferior occipital, superior temporal (STG), and fusiform gyri (FG) and with an occipitotemporal electrophysiological component peaking around 170 ms on the scalp, the N170. However, the relationship between the N170 and the multiple face-sensitive activations observed in neuroimaging is unclear. It has been recently shown that the amplitude of the N170 component monotonically decreases as gaussian noise is added to a picture of a face [Jemel et al., 2003]. To help clarify the sources of the N170 without a priori assumptions regarding their number and locations, ERPs and fMRI were recorded in five subjects in the same experiment, in separate sessions. We used a parametric paradigm in which the amplitude of the N170 was modulated by varying the level of noise in a picture, and identified regions where the percent signal change in fMRI correlated with the ERP data. N170 signals were observed for pictures of both cars and faces but were stronger for faces. A monotonic decrease with added noise was observed for the N170 at right hemisphere sites but was less clear on the left and occipital central sites. Correlations between fMRI signal and N170 amplitudes for faces were highly significant (P < 0.001) in bilateral fusiform gyrus and superior temporal gyrus. For cars, the strongest correlations were observed in the parahippocampal region and in the STG (P < 0.005). Besides contributing to clarify the spatiotemporal course of face processing, this study illustrates how ERP information may be used synergistically in fMRI analyses. Parametric designs may be developed further to provide some timing information on fMRI activity and help identify the generators of ERP signals.     

Activity in the fusiform gyrus predicts conscious perception of Rubin's vase-face illusion

We localized regions in the fusiform gyrus and superior temporal sulcus that were more active when subjects viewed photographs of real faces than when they viewed complex inanimate objects and other areas in the parahippocampal gyrus and the lateral occipital lobe that showed more activity during the presentation of nonface objects. Event-related functional magnetic resonance imaging was then used to monitor activity in these extrastriate visual areas while subjects viewed Rubin's vase-face stimulus and indicated switches in perception. Since the spontaneous shifts in interpretation were too rapid for direct correlation with hemodynamic responses, each reported percept (faces or vase) was prolonged by suddenly adding subtle local contrast gradients (embossing) to one side or the other of the figure-ground boundary, stabilizing the percept. Under these conditions, only face-selective areas in the fusiform gyrus responded more strongly during the perception of faces. To control for effects of the physical change to Rubin's stimulus (i.e., addition of embossing), we compared activity when the face contours were embossed after the subject had just reported the onset of perception of either faces or vase. Activity in the fusiform face area responded more strongly under the first condition, despite the fact that the physical stimulus sequences were identical. Moreover, on a trial-to-trial basis, the activity was statistically predictive of the subjects' responses, suggesting that the conscious perception of faces could be made explicit in this extrastriate visual area.     

Neural correlates of temporal integration in face recognition: an fMRI study

Integration of temporally separated visual inputs is crucial for perception of a unified representation. Here, we show that regions involved in configural processing of faces contribute to temporal integration occurring within a limited time-window using a multivariate analysis (partial least squares, PLS) exploring the relation between brain activity and recognition performance. During fMRI, top and bottom parts of a famous face were presented sequentially with a varying interval (0, 200, or 800 ms) or were misaligned. The 800 ms condition activated several regions implicated in face processing, attention and working memory, relative to the other conditions, suggesting more active maintenance of individual face parts. Analysis of brain-behavior correlations showed that better identification in the 0 and 200 conditions was associated with increased activity in areas considered to be part of a configural face processing network, including right fusiform, middle occipital, bilateral superior temporal areas, anterior/middle cingulate and frontal cortices. In contrast, successful recognition in the 800 and misaligned conditions, which involve analytic and strategic processing, was negatively associated with activation in these regions. Thus, configural processing may involve rapid temporal integration of facial features and their relations. Our finding that regions concerned with configural and analytic processes in the service of face identification opposed each other may explain why it is difficult to apply the two processes concurrently.     

Familiarity enhances invariance of face representations in human ventral visual cortex: fMRI evidence

Face recognition across different viewing conditions is strongly improved by familiarity. In the present study, we tested the hypothesis that the neural basis of this effect is a less view-dependent representation of familiar faces in ventral visual cortex by assessing priming-related fMRI repetition effects. 15 healthy volunteers made male/female judgements on familiar (famous) and unfamiliar (novel) faces preceded by the same image, a different image of the same face, or another (unprimed) face. Reaction times revealed priming by same and different images independent of familiarity and more pronounced for same than different images. In the imaging data, a main effect of prime condition was found in bilateral fusiform and orbitofrontal regions. A right anterior fusiform region expressed stronger response decreases to repetition of familiar than unfamiliar faces. Bilateral mid-fusiform areas showed stronger response decreases to repetition of same than different images. A regions-of-interest analysis focussing specifically on face responsive regions suggested differences in the degree of image dependency across fusiform cortex. Collapsing across familiarity, there was greater image dependency of repetition effects in right than left anterior fusiform, replicating previous imaging findings obtained with common objects. For familiar faces alone, there was greater generalisation of repetition effects over different images in anterior than middle fusiform. This suggests a role of anterior fusiform cortex in coding image-independent representations of familiar faces.     

Activation of the fusiform gyrus when individuals with autism spectrum disorder view faces

Prior imaging studies have failed to show activation of the fusiform gyrus in response to emotionally neutral faces in individuals with autism spectrum disorder (ASD) [Critchley et al., Brain 124 (2001) 2059; Schultz et al., Arch. Gen. Psychiatry 57 (2000) 331]. However, individuals with ASD do not typically exhibit the striking behavioral deficits that might be expected to result from fusiform gyrus damage, such as those seen in prosopagnosia, and their deficits appear to extend well beyond face identification to include a wide range of impairments in social perceptual processing. In this study, our goal was to further assess the question of whether individuals with ASD have abnormal fusiform gyrus activation to faces. We used high-field (3 T) functional magnetic resonance imaging to study face perception in 11 adult individuals with autism spectrum disorder (ASD) and 10 normal controls. We used face stimuli, object stimuli, and sensory control stimuli (Fourier scrambled versions of the face and object stimuli) containing a fixation point in the center to ensure that participants were looking at and attending to the images as they were presented. We found that individuals with ASD activated the fusiform face area and other brain areas normally involved in face processing when they viewed faces as compared to non-face stimuli. These data indicate that the face-processing deficits encountered in ASD are not due to a simple dysfunction of the fusiform area, but to more complex anomalies in the distributed network of brain areas involved in social perception and cognition.     

Neural responses to Mooney images reveal a modular representation of faces in human visual cortex

The way in which information about objects is represented in visual cortex remains controversial. One model of human object recognition poses that information is processed in modules, highly specialised for different categories of objects; an opposing model appeals to a distributed representation across a large network of visual areas. We addressed this debate by monitoring activity in face- and object-selective areas while human subjects viewed ambiguous face stimuli (Mooney faces). The measured neural response in the face-selective region of the fusiform gyrus was greater when subjects reported seeing a face than when they perceived the image as a collection of blobs. In contrast, there was no difference in magnetic resonance response between face and no-face perceived events in either the face-selective voxels of the superior temporal sulcus or the object-selective voxels of the parahippocampal gyrus and lateral occipital complex. These results challenge the concept that neural representation of faces is distributed and overlapping and suggest that the fusiform gyrus is tightly linked to the awareness of faces.     

Sex differences in the neural correlates of child facial resemblance: an event-related fMRI study

Detection of genetic relatedness (i.e. kinship) impacts the social, parental, and sexual behavior of many species. In humans, self-referent phenotype matching based on facial resemblance may indicate kinship. For example, faces that resemble ours are perceived as more trustworthy and attractive. Sex differences in behavioral reactions to facial resemblance among children have also been demonstrated and are consistent with evolutionary theory suggesting that resemblance might serve as a paternity cue. Using event-related fMRI, we show that specific regions of the brain are implicated in processing facial resemblance and a sex difference in cortical response to facial resemblance expressed in children. We found a consistent activation in the fusiform gyrus across all face conditions, which is consistent with previous research on face processing. There were no sex differences in overall response to faces in the fusiform gyrus, and also to faces that did not resemble subjects. When resemblance was not modeled, females showed greater activation to child faces than males. Consistent with parental investment theory and theories of sexual selection, males showed greater cortical activity than females in response to children's faces that resembled them. These data suggest natural selection may have crafted a sexually differentiated neuro-sensory module implicated in detection of facial resemblance that may serve as a kin detection and paternity cue. This process may capitalize on neural substrates involved in self-referent processing and familiarity detection.