Attention to emotion modulates fMRI activity in human right superior temporal sulcus

A parallel neural network has been proposed for processing various types of information conveyed by faces including emotion. Using functional magnetic resonance imaging (fMRI), we tested the effect of the explicit attention to the emotional expression of the faces on the neuronal activity of the face-responsive regions. Delayed match to sample procedure was adopted. Subjects were required to match the visually presented pictures with regard to the contour of the face pictures, facial identity, and emotional expressions by valence (happy and fearful expressions) and arousal (fearful and sad expressions). Contour matching of the non-face scrambled pictures was used as a control condition. The face-responsive regions that responded more to faces than to non-face stimuli were the bilateral lateral fusiform gyrus (LFG), the right superior temporal sulcus (STS), and the bilateral intraparietal sulcus (IPS). In these regions, general attention to the face enhanced the activities of the bilateral LFG, the right STS, and the left IPS compared with attention to the contour of the facial image. Selective attention to facial emotion specifically enhanced the activity of the right STS compared with attention to the face per se. The results suggest that the right STS region plays a special role in facial emotion recognition within distributed face-processing systems. This finding may support the notion that the STS is involved in social perception.     

Dynamic facial expressions evoke distinct activation in the face perception network: a connectivity analysis study

Very little is known about the neural structures involved in the perception of realistic dynamic facial expressions. In the present study, a unique set of naturalistic dynamic facial emotional expressions was created. Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend Haxby et al.'s [Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. The distributed human neural system for face perception. Trends in Cognitive Science, 4, 223-233, 2000] distributed neural system for face perception. This network includes early visual regions, such as the inferior occipital gyrus, which is identified as insensitive to motion or affect but sensitive to the visual stimulus, the STS, identified as specifically sensitive to motion, and the amygdala, recruited to process affect. Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as the inferior occipital gyrus and the STS, along with coupling between the STS and the amygdala, as well as the inferior frontal gyrus. These findings support the presence of a distributed network of cortical regions that mediate the perception of different dynamic facial expressions.     

Recognition memory of newly learned faces

We used event-related fMRI to study recognition memory of newly learned faces. Caucasian subjects memorized unfamiliar, neutral and happy South Korean faces and 4 days later performed a memory retrieval task in the MR scanner. We predicted that previously seen faces would be recognized faster and more accurately and would elicit stronger neural activation than novel faces. Consistent with our hypothesis, novel faces were recognized more slowly and less accurately than previously seen faces. We found activation in a distributed cortical network that included face-responsive regions in the visual cortex, parietal and prefrontal regions, and the hippocampus. Within all regions, correctly recognized, previously seen faces evoked stronger activation than novel faces. Additionally, in parietal and prefrontal cortices, stronger activation was observed during correct than incorrect trials. Finally, in the hippocampus, false alarms to happy faces elicited stronger responses than false alarms to neutral faces. Our findings suggest that face recognition memory is mediated by stimulus-specific representations stored in extrastriate regions; parietal and prefrontal regions where old and new items are classified; and the hippocampus where veridical memory traces are recovered.     

Are face-responsive regions selective only for faces?

To examine the specificity of face-responsive regions for face processing, we used fMRI to measure the response of the fusiform gyrus and the superior temporal sulcus (STS) to pictures of human faces, animals, faceless animals, and houses. Results indicate that faces, animals, and faceless animals all elicited greater activity than houses, and had identical peaks of activation in the lateral fusiform gyrus, bilaterally, and in the right posterior STS. Moreover, within the lateral fusiform gyrus the responses to faces, animals and faceless animals were all greater than the responses to these stimuli in the medial aspect of the fusiform gyrus, a region that responds more strongly to other objects (e.g. houses). These findings suggest that the neural representation of animals in the fusiform gyrus and the posterior STS relies strongly on the same neural substrates that represent faces.     

Neural substrates of facial emotion processing using fMRI

We identified human brain regions involved in the perception of sad, frightened, happy, angry, and neutral facial expressions using functional magnetic resonance imaging (fMRI). Twenty-one healthy right-handed adult volunteers (11 men, 10 women; aged 18-45; mean age 21.6 years) participated in four separate runs, one for each of the four emotions. Participants viewed blocks of emotionally expressive faces alternating with blocks of neutral faces and scrambled images. In comparison with scrambled images, neutral faces activated the fusiform gyri, the right lateral occipital gyrus, the right superior temporal sulcus, the inferior frontal gyri, and the amygdala/entorhinal cortex. In comparisons of emotional and neutral faces, we found that (1) emotional faces elicit increased activation in a subset of cortical regions involved in neutral face processing and in areas not activated by neutral faces; (2) differences in activation as a function of emotion category were most evident in the frontal lobes; (3) men showed a differential neural response depending upon the emotion expressed but women did not.     

Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning)

Can the cortical substrates for the perception of face actions be distinguished when the superficial visual qualities of these actions are very similar? Two fMRI experiments are reported. Compared with watching the face at rest, observing silent speech was associated with bilateral activation in a number of temporal cortical regions, including the superior temporal sulcus (STS). Watching face movements of similar extent and duration, but which could not be construed as speech (gurning; Experiment 1b) was not associated with activation of superior temporal cortex to the same extent, especially in the left hemisphere. Instead, the peak focus of the largest cluster of activation was in the posterior part of the inferior temporal gyrus (right, BA 37). Observing silent speech, but not gurning faces, was also associated with bilateral activation of inferior frontal cortex (BA 44 and 45). In a second study, speechreading and observing gurning faces were compared within a single experiment, using stimuli which comprised the speaker's face and torso (and hence a much smaller image of the speaker's face and facial actions). There was again differential engagement of superior temporal cortex which followed the pattern of Experiment 1. These findings suggest that superior temporal gyrus and neighbouring regions are activated bilaterally when subjects view face actions--at different scales--that can be interpreted as speech. This circuitry is not accessed to the same extent by visually similar, but linguistically meaningless actions. However, some temporal regions, such as the posterior part of the right superior temporal sulcus, appear to be common processing sites for processing both seen speech and gurns.     

Differential modulation of neural activity throughout the distributed neural system for face perception in patients with Social Phobia and healthy subjects

Social Phobia (SP) is a marked and persistent fear of social or performance situations in which the person is exposed to unfamiliar people or to possible scrutiny by others. Faces of others are perceived as threatening by social phobic patients (SPP). To investigate how face processing is altered in the distributed neural system for face perception in Social Phobia, we designed an event-related fMRI study in which Healthy Controls (HC) and SPP were presented with angry, fearful, disgusted, happy and neutral faces and scrambled pictures (visual baseline). As compared to HC, SPP showed increased neural activity not only in regions involved in emotional processing including left amygdala and insula, as expected from previous reports, but also in the bilateral superior temporal sulcus (STS), a part of the core system for face perception that is involved in the evaluation of expression and personal traits. In addition SPP showed a significantly weaker activation in the left fusiform gyrus, left dorsolateral prefrontal cortex, and bilateral intraparietal sulcus as compared to HC. These effects were found not only in response to emotional faces but also to neutral faces as compared to scrambled pictures. Thus, SPP showed enhanced activity in brain areas related to processing of information about emotional expression and personality traits. In contrast, brain activity was decreased in areas for attention and for processing other information from the face, perhaps as a result of a feeling of wariness. These results indicate a differential modulation of neural activity throughout the different parts of the distributed neural system for face perception in SPP as compared to HC.     

Neural substrates underlying intentional empathy

Although empathic responses to stimuli with emotional contents may occur automatically, humans are capable to intentionally empathize with other individuals. Intentional empathy for others is even possible when they do not show emotional expressions. However, little is known about the neuronal mechanisms of this intentionally controlled empathic process. To investigate the neuronal substrates underlying intentional empathy, we scanned 20 healthy Chinese subjects, using fMRI, when they tried to feel inside the emotional states of neutral or angry faces of familiar (Asian) and unfamiliar (Caucasian) models. Skin color evaluation of the same stimuli served as a control task. Compared to a baseline condition, the empathy task revealed a network of established empathy regions, including the anterior cingulate cortex, bilateral inferior frontal cortex and bilateral anterior insula. The contrast of intentional empathy vs skin color evaluation, however, revealed three regions: the bilateral inferior frontal cortex, whose hemodynamic responses were independent of perceived emotion and familiarity and the right-middle temporal gyrus, whose activity was modulated by emotion but not by familiarity. These findings extend our understanding of the role of the inferior frontal cortex and the middle temporal gyrus in empathy by demonstrating their involvement in intentional empathy.     

Neural correlates of impaired emotional discrimination in borderline personality disorder: An fMRI study

A common approach to study neuronal aspects of emotional reactivity of borderline personality disorder (BPD) is to study the brain response to emotional faces with functional magnetic resonance imaging (fMRI). 10 BPD patients and 10 matched controls were submitted to an emotional discrimination task in which subjects had to identify an emotional face from a neutral face while fMRI data was acquired. BPD patients made more mistakes than controls in the discrimination task when negative faces were involved. The emotional discrimination task activated brain areas that are known to participate in processing of emotional faces (fusiform gyrus, insula and amygdala) regardless of the psychiatric condition. Additionally, BPD showed higher activation than controls in the middle and inferior temporal cortical areas, brain areas that participate in the processing of face features that carry emotional value. Furthermore, activity at this site correlated with impulsivity score in the Zuckerman–Kuhlman Personality Questionnaire. Our findings may be related to cognitive impairment that may be characteristic of the disorder.     

Regional brain response to faces of humans and dogs

The extent to which the brain regions associated with face processing are selective for that specific function remains controversial. In addition, little is known regarding the extent to which face-responsive brain regions are selective for human faces. To study regional selectivity of face processing, we used functional magnetic resonance imaging to examine whole brain activation in response to human faces, dog faces, and houses. Fourteen healthy right-handed volunteers participated in a passive viewing, blocked experiment. Results indicate that the lateral fusiform gyrus (Brodmann's area 37) responds maximally to both dog and human faces when compared with other sites, followed by the middle/inferior occipital gyrus (BA 18/19). Sites that were activated by houses versus dog and human faces included the medial fusiform gyrus (BA 19/37), the posterior cingulate (BA 30), and the superior occipital gyrus (BA 19). The only site that displayed significant differences in activation between dog and human faces was the lingual/medial fusiform gyrus. In this site, houses elicited the strongest activation, followed by dog faces, while the response to human faces was negligible and did not differ from fixation. The parahippocampal gyrus/amygdala was the sole site that displayed significant activation to human faces, but not to dog faces or houses.     

Neural substrates for functionally discriminating self-face from personally familiar faces

Understanding the neurobiological substrates of self-recognition yields important insight into socially and clinically critical cognitive functions such as theory of mind. Experimental evidence suggests that right frontal and parietal lobes preferentially process self-referent information. Recognition of one's own face is an important parameter of self-recognition, but well-controlled experimental data on the brain substrates of self-face recognition is limited. The goal of this study was to characterize the activation specific to self-face in comparison with control conditions of two levels of familiarity: unknown unfamiliar face and the more stringent control of a personally familiar face. We studied 12 healthy volunteers who made "unknown," "familiar," and "self" judgments about photographs of three types of faces: six different novel faces, a personally familiar face (participant's fraternity brother), and their own face during an event-related functional MRI (fMRI) experiment. Contrasting unknown faces with baseline showed activation of the inferior occipital lobe, which supports previous findings suggesting the presence of a generalized face-processing area within the inferior occipital-temporal region. Activation in response to a familiar face, when contrasted with an unknown face, invoked insula, middle temporal, inferior parietal, and medial frontal lobe activation, which is consistent with an existing hypothesis suggesting familiar face recognition taps neural substrates that are different from those involved in general facial processing. Brain response to self-face, when contrasted with familiar face, revealed activation in the right superior frontal gyrus, medial frontal and inferior parietal lobes, and left middle temporal gyrus. The contrast familiar vs. self produced activation only in the anterior cingulate gyrus. Our results support the existence of a bilateral network for both perceptual and executive aspects of self-face processing that cannot be accounted for by a simple hemispheric dominance model. This network is similar to those implicated in social cognition, mirror neuron matching, and face-name matching. Our findings also show that some regions of the medial frontal and parietal lobes are specifically activated by familiar faces but not unknown or self-faces, indicating that these regions may serve as markers of face familiarity and that the differences between activation associated with self-face recognition and familiar face recognition are subtle and appear to be localized to lateral frontal, parietal, and temporal regions.     

The representation of parts and wholes in face-selective cortex

Although face perception is often characterized as depending on holistic, rather than part-based, processing, there is behavioral evidence for independent representations of face parts. Recent work has linked "face-selective" regions defined with functional magnetic resonance imaging (fMRI) to holistic processing, but the response of these areas to face parts remains unclear. Here we examine part-based versus holistic processing in "face-selective" visual areas using face stimuli manipulated in binocular disparity to appear either behind or in front of a set of stripes [Nakayama, K., Shimojo, S., & Silverman, G. H. Stereoscopic depth: Its relation to image segmentation, grouping, and the recognition of occluded objects. Perception, 18, 55-68, 1989]. While the first case will be "filled in" by the visual system and perceived holistically, we demonstrate behaviorally that the latter cannot be completed amodally, and thus is perceived as parts. Using these stimuli in fMRI, we found significant responses to both depth manipulations in inferior occipital gyrus and middle fusiform gyrus (MFG) "face-selective" regions, suggesting that neural populations in these areas encode both parts and wholes. In comparison, applying these depth manipulations to control stimuli (alphanumeric characters) elicited much smaller signal changes within face-selective regions, indicating that the part-based representation for faces is separate from that for objects. The combined adaptation data also showed an interaction of depth and familiarity within the right MFG, with greater adaptation in the back (holistic) condition relative to parts for familiar but not unfamiliar faces. Together, these data indicate that face-selective regions of occipito-temporal cortex engage in both part-based and holistic processing. The relative recruitment of such representations may be additionally influenced by external factors such as familiarity.     

Towards a functional neuroanatomy of self processing: effects of faces and words

We studied the neural correlates of self vs. non-self judgements using functional magnetic resonance imaging (fMRI). Individually tailored faces and personality trait words were used as stimuli in three experiments (exp.). In the first two experiments, brain activation was measured while subjects viewed morphed versions of either their own (self face exp.) or their partner's face (partner's face exp.), alternating in blocks with presentation of an unknown face. In the self face exp. right limbic areas (hippocampal formation, insula, anterior cingulate), the right middle temporal lobe, left inferior parietal and left prefrontal regions showed signal changes. In the partner's face exp., only the right insula was activated. In the third exp., subjects made decisions about psychological trait adjectives previously categorized as describing their own attributes. Activation was present in the precuneus, the left parietal lobe, left insula/inferior frontal gyrus and the left anterior cingulate. A reaction time advantage was present when subjects responded to self-relevant words. The main area with signal changes during self-reference processing, regardless of the type of stimulus, was the left fusiform gyrus. The self-relevant stimuli engaged to a differential extent long term and working memory, semantic and emotional processes. We suggest that regions activated by these stimuli are engaged in self-processing.     

Neural correlates of personally familiar faces: parents, partner and own faces

Investigations of the neural correlates of face recognition have typically used old/new paradigms where subjects learn to recognize new faces or identify famous faces. Familiar faces, however, include one's own face, partner's and parents' faces. Using event-related fMRI, we examined the neural correlates of these personally familiar faces. Ten participants were presented with photographs of own, partner, parents, famous and unfamiliar faces and responded to a distinct target. Whole brain, two regions of interest (fusiform gyrus and cingulate gyrus), and multiple linear regression analyses were conducted. Compared with baseline, all familiar faces activated the fusiform gyrus; own faces also activated occipital regions and the precuneus; partner faces activated similar areas, but in addition, the parahippocampal gyrus, middle superior temporal gyri and middle frontal gyrus. Compared with unfamiliar faces, only personally familiar faces activated the cingulate gyrus and the extent of activation varied with face category. Partner faces also activated the insula, amygdala and thalamus. Regions of interest analyses and laterality indices showed anatomical distinctions of processing the personally familiar faces within the fusiform and cingulate gyri. Famous faces were right lateralized whereas personally familiar faces, particularly partner and own faces, elicited bilateral activations. Regression analyses show experiential predictors modulated with neural activity related to own and partner faces. Thus, personally familiar faces activated the core visual areas and extended frontal regions, related to semantic and person knowledge and the extent and areas of activation varied with face type.     

Emotional attention in acquired prosopagnosia

The present study investigated whether emotionally expressive faces guide attention and modulate fMRI activity in fusiform gyrus in acquired prosopagnosia. Patient PS, a pure case of acquired prosopagnosia with intact right middle fusiform gyrus, performed two behavioral experiments and a functional imaging experiment to address these questions. In a visual search task involving face stimuli, PS was faster to select the target face when it was expressing fear or happiness as compared to when it was emotionally neutral. In a change detection task, PS detected significantly more changes when the changed face was fearful as compared to when it was neutral. Finally, an fMRI experiment showed enhanced activation to emotionally expressive faces and bodies in right fusiform gyrus. In addition, PS showed normal body-selective activation in right fusiform gyrus, partially overlapping the fusiform face area. Together these behavioral and neuroimaging results show that attention was preferentially allocated to emotional faces in patient PS, as observed in healthy subjects. We conclude that systems involved in the emotional guidance of attention by facial expression can function normally in acquired prosopagnosia, and can thus be dissociated from systems involved in face identification.     

Letter processing automatically recruits a sensory-motor brain network

Behavioral, neuropsychological and neuroimaging research suggest a distributed network that is recruited when we interact with letters. For the first time, we combine several letter processing tasks in a single experiment to study why letters seem to engage such disparate processing areas. Using fMRI, we investigate how the brain responds to letters using tasks that should recruit systems for letter perception, letter writing, letter copying and letter imagery. We describe a network of five cortical regions including the left fusiform gyrus, two left pre-central areas, left cuneus and the left inferior frontal gyrus that are all selectively engaged during a 1-back matching paradigm with letters. Our results suggest involvement of these regions to different extents in different tasks. However, the regions also form an integrated network such that letter perception also engages motor regions while writing recruits letter-specific visual regions as well. We suggest that this distributed network is a direct result of our sensory–motor interactions with letters.     

Recovery from adaptation to facial identity is larger for upright than inverted faces in the human occipito-temporal cortex

Human faces look more similar to each other when they are presented upside-down, leading to an increase of error rates and response times during individual face discrimination tasks. Here we used functional magnetic resonance imaging (fMRI) to test the hypothesis that this perceived similarity leads to a lower recovery from identity adaptation for inverted faces than for upright faces in face-sensitive areas of the occipito-temporal cortex. Ten subjects were presented with blocks of upright and inverted faces, with the same face identity repeated consecutively in half of the blocks, and different facial identities repeated in the other blocks. When face stimuli were presented upright, the percent signal change in the bilateral middle fusiform gyrus (MFG) was larger for different faces as compared to same faces, replicating previous observations of a recovery from facial identity adaptation in this region. However, there was no significant recovery from adaptation when different inverted faces were presented. Most interestingly, the difference in activation between upright and inverted faces increased progressively during a block when different facial identities were presented. A similar pattern of activation was found in the left middle fusiform gyrus, but was less clear-cut in bilateral face-sensitive areas of the inferior occipital cortex. These findings show that the differential level of activation to upright and inverted faces in the fusiform gyrus is mainly due to a difference in recovery from adaptation, and they explain the discrepancies in the results reported in previous fMRI studies which compared the processing of upright and inverted faces. The lack of recovery from adaptation for inverted faces in the fusiform gyrus may underlie the face inversion effect (FIE), which takes place during perceptual encoding of individual face representations.     

Abnormal activation of the social brain during face perception in autism

ASD involves a fundamental impairment in processing social-communicative information from faces. Several recent studies have challenged earlier findings that individuals with autism spectrum disorder (ASD) have no activation of the fusiform gyrus (fusiform face area, FFA) when viewing faces. In this study, we examined activation to faces in the broader network of face-processing modules that comprise what is known as the social brain. Using 3T functional resonance imaging, we measured BOLD signal changes in 10 ASD subjects and 7 healthy controls passively viewing nonemotional faces. We replicated our original findings of significant activation of face identity-processing areas (FFA and inferior occipital gyrus, IOG) in ASD. However, in addition, we identified hypoactivation in a more widely distributed network of brain areas involved in face processing [including the right amygdala, inferior frontal cortex (IFC), superior temporal sulcus (STS), and face-related somatosensory and premotor cortex]. In ASD, we found functional correlations between a subgroup of areas in the social brain that belong to the mirror neuron system (IFC, STS) and other face-processing areas. The severity of the social symptoms measured by the Autism Diagnostic Observation Schedule was correlated with the right IFC cortical thickness and with functional activation in that area. When viewing faces, adults with ASD show atypical patterns of activation in regions forming the broader face-processing network and social brain, outside the core FFA and IOG regions. These patterns suggest that areas belonging to the mirror neuron system are involved in the face-processing disturbances in ASD.     

Neural basis of prosopagnosia: an fMRI study

Brain imaging research has identified at least two regions in human extrastriate cortex responding selectively to faces. One of these is located in the mid-fusiform gyrus (FFA), the other in the inferior occipital gyrus (IOG). We studied activation of these areas using fMRI in three individuals with severely impaired face recognition (one pure developmental and two childhood prosopagnosics). None of the subjects showed the normal pattern of higher fMRI activity to faces than to objects in the FFA and IOG or elsewhere. Moreover, in two of the patients, faces and objects produced similar activations in the regions corresponding to where the FFA and IOG are found in normal subjects. Our study casts light on the important role of FFA and IOG in the network of areas involved in face recognition, and indicates limits of brain plasticity.     

A functional MRI study of face recognition in patients with prosopagnosia.

An fMRI investigation was conducted to determine whether patients with impaired face recognition, a deficit known as prosopagnosia, would show functional activation in the fusiform gyrus, the neural substrate for face processing, when viewing faces. While the patients did show activation in the fusiform gyrus, with significantly more voxels in posterior areas than their control subjects, this activation was not sufficient for face processing. In one of the patients, the posterior activation was particularly evident in the left hemisphere, which is thought to be involved in feature-based strategies of face perception. We conclude that an increased reliance on feature-based processing in prosopagnosia leads to a recruitment of neurons in posterior regions of the fusiform gyrus, regions that are not ideally suited for processing faces.