Facial expression and emotional face recognition in schizophrenia and depression

Summary Twenty-three acute schizophrenics, 21 acute major depressives (Research Diagnostic Criteria), and 15 normal controls participated in a study on facial expression and emotional face recognition. Under clinical conditions, spontaneous facial expression was assessed according to the affective flattening section of the Scale for the Assessment of Negative Symptoms. Under experimental laboratory conditions involuntary (emotioneliciting interview) and voluntary facial expression (imitation and simulation of six basic emotions) were recorded on videotape, from which a raterbased analysis of intensity or correctness of facial activity was obtained. Emotional face recognition was also assessed under experimental conditions using the same stimulus material. All subjects were assessed twice (within 4 weeks), controlling for change of the psychopathological status in the patient groups. In schizophrenics, neuroleptic drug influence was controlled by random allocation to treatment with either haloperidol or perazine. The main findings were that schizophrenics and depressives are characterized by different quantitative, qualitative, and temporal patterns of affect-related dysfunctions. In particular, schizophrenics demonstrated a trait-like deficit in affect recognition and in their spontaneous and voluntary facial activity, irrespective of medication, drug type and dosage, or extrapyramidal side-effects. In depressives a stable deficit could be demonstrated only in their involuntary expression under emotion-eliciting interview conditions, whereas in the postacute phase a reduction in their voluntary expression became apparent. Differences in patterns of affect-related behavioral deficits may reflect dysfunctions in different underlying psychobiological systems.     

Cross-modal influences of affect across social and non-social domains in individuals with Williams syndrome

The Williams syndrome (WS) cognitive profile is characterized by relative strengths in face processing, an attentional bias towards social stimuli, and an increased affinity and emotional reactivity to music. An audio-visual integration study examined the effects of auditory emotion on visual (social/non-social) affect identification in individuals with WS and typically developing (TD) and developmentally delayed (DD) controls. The social bias in WS was hypothesized to manifest as an increased ability to process social than non-social affect, and a reduced auditory influence in social contexts. The control groups were hypothesized to perform similarly across conditions. The results showed that while participants with WS exhibited indistinguishable performance to TD controls in identifying facial affect, DD controls performed significantly more poorly. The TD group outperformed the WS and DD groups in identifying non-social affect. The results suggest that emotionally evocative music facilitated the ability of participants with WS to process emotional facial expressions. These surprisingly strong facial-processing skills in individuals with WS may have been due to the effects of combining social and music stimuli and to a reduction in anxiety due to the music in particular. Several directions for future research are suggested.     

Impaired processing of relative distances between features and of the eye region in acquired prosopagnosia—Two sides of the same holistic coin?

Acquired prosopagnosia (AP) is characterized by impaired recognition of individual faces following brain damage. The nature of the functional impairment(s) underlying AP remains debated. Recent studies have demonstrated deficient processing of diagnostic information in the region of the eyes (Caldara et al., 2005); other studies suggest that patients fail to judge relative distances between facial features (Barton et al., 2002). We hypothesized that these apparently different observations are related to a common cause. More precisely, we suggest that AP arises due to an impairment of a process that reduces uncertainty about the nature/location of the diagnostic cues for face individualization: the ability to perceive multiple elements of a face as a single global representation (holistic processing). Being impaired at processing individual faces holistically, prosopagnosic patients would tend to perform relatively worse for processing facial areas containing multiple elements (i.e., the eyes), and for elements that are widely spaced apart. Here we tested PS, a single case of AP, at matching unfamiliar faces differing either with respect to local features or inter-feature distances, over the upper and lower areas of the face. A pilot study and Experiment 1 confirmed that PS was extremely poor at using information encompassing the eyes, but was also deficient at perceiving relative distances between features. When uncertainty about the location and nature of the diagnostic cue was removed in Experiment 2, PS' performance remained below normal range, but she improved substantially. Most interestingly, her pattern of performance across the different conditions appeared qualitatively identical to that of normal controls. In line with previous observations of PS and other cases of prosopagnosia, our findings indicate that the reduced reliance on the area of the eyes and on relative distances between features in AP may have a common underlying cause—the disruption of holistic processing of the individual face.     

Brain dysfunctions during facial discrimination in schizophrenia: Selective association to affect decoding

Schizophrenia patients exhibit impaired facial affect perception, yet the exact nature of this impairment remains unclear. We investigated neural activity related to processing facial emotional and non-emotional information and complex images in 12 schizophrenia patients and 15 healthy controls using functional magnetic resonance imaging. All subjects performed a facial information processing task with three conditions: matching facial emotion, matching facial identity, and matching complex visual patterns. Patients and controls showed comparable behavioral performance in all task conditions. The neural activation patterns in schizophrenia patients and healthy controls were distinctly different while processing affect-related facial information but not other non-emotional facial features. During emotion matching, orbital frontal cortex and left amydala activations were found in controls but not in patients. When comparing emotion versus identity matching, controls activated the fusiform and middle temporal gyri, left superior temporal gyrus, and right inferior and middle frontal gyrus, whereas schizophrenia patients only activated the middle and inferior frontal gyri, the frontal operculi and the right insular cortex. Our findings suggest that schizophrenia patients and healthy controls may utilize different neural networks when processing facial emotional information.     

Neural processing of facial expressions of emotion in first onset psychosis

Schizophrenia is characterized by deficits in face and facial emotion processing. This is the first study using event-related potentials (ERPs) to investigate the corresponding neural activation in first onset psychosis. ERPs for 108 first onset psychosis participants and 108 matched healthy controls were recorded while they viewed facial expressions. Group differences on general (neutral) face processing and emotional valence were examined under both unmasked (conscious) and backward-masked (nonconscious implicit) conditions over frontal and temporo-occipital regions. Clinical significance was assessed by comparing diagnoses and correlating ERPs with symptoms. During general face processing, patients showed reduced activation within 70 ms and exaggerated later processing from 160 ms over the frontal region, with a negative shift in voltage over left temporal and occipital regions across the time course. In addition, from 70 ms onwards, patients showed a positive shift in voltage for disgust whereas controls showed a negative shift in voltage for fear and anger (both compared to happy) over temporo-occipital regions. Effects were related to disorganization and depression symptoms and (preliminarily) were apparent across psychotic diagnoses. These results suggest that first onset psychosis is characterized by general as well as emotion-specific face processing impairments from the earliest, automatic processing period.     

Disengagement of attention from facial emotion in unipolar depression

Abnormal processing of facial expressions is assumed to be an important factor mediating the course of depression. The aim of the present study was to investigate the ability to disengage attention from facial emotion in depressed patients in the course of an inpatient treatment program. It was hypothesized that in depression disengagement of visual attention from negative facial expression is delayed, while disengagement from positive facial expression is facilitated. A face-in-the-crowd task using schematic stimuli and crowds of neutral and emotion faces as 'distractor' stimuli was administered to depressed patients and normal controls matched for age, sex, and education. Patients with major depression (n=15) and normal controls (n=15) were tested twice, about 6 weeks apart. From test 1 to test 2, patients' depressivity decreased significantly. Depressives showed higher response latencies and error rates than control subjects. However, depressed patients exhibited the same scanning pattern for facial emotion as healthy individuals across both test sessions. Participants detected a negative face more rapidly in a crowd of faces than a positive face. When displays consisted of repetitions of the same face, subjects were generally slower (and less accurate) when faces were negative relative to positive or neutral. The present data suggest that the ability to disengage attention from facial emotion in visual search is not impaired in depression.     

The role of configural information in facial emotion recognition in schizophrenia

The schizophrenia deficit in facial emotion recognition could be accounted for by a deficit in processing the configural information of the face. The present experiment was designed to further test this hypothesis by studying the face-inversion effect in a facial emotion recognition task. The ability of 26 schizophrenic patients and 26 control participants to recognize facial emotions on upright and upside-down faces was assessed. Participants were told to state whether faces expressed one of six possible emotions (happiness, anger, disgust, fear, sadness, neutrality) in two sessions, one with upright faces and the other with upside-down faces. Discriminability and the decision criterion were computed. The results indicated that the schizophrenic patients were impaired in upright facial emotion discrimination by comparison with the controls. They also exhibited an inversion effect similar to the controls. However, whereas controls tended to adopt a more conservative criterion for all emotions and a liberal criterion for neutrality when the faces were upside-down, schizophrenic patients presented a decision criterion pattern that was similar for the two orientations and similar to controls in upside-down emotion recognition. The lack of a decision criterion shift was associated with positive symptoms such as delusions, hallucinations, and bizarre behavior. Moreover, positive and negative symptoms were associated with inversion effect on discriminability; the more severe the symptoms, the weaker the inversion effect. We conclude that individuals with schizophrenia do process the configural information of the face. However, further investigations are needed to assert whether this information is of good quality in schizophrenia.     

Understanding facial emotion perception in Parkinson's disease: the role of configural processing

Parkinson's disease (PD) has been frequently associated with facial emotion recognition impairments, which could adversely affect the social functioning of those patients. Facial emotion recognition requires processing of the spatial relations between facial features, known as the facial configuration. Few studies, however, have investigated this ability in people with PD. We hypothesized that facial emotion recognition impairments in patients with PD could be accounted for by a deficit in configural processing. To assess this hypothesis, three tasks were proposed to 10 patients with PD and 10 healthy controls (HC): (i) a facial emotion recognition task with upright faces, (ii) a similar task with upside-down faces, to explore the face inversion effect, and (iii) a configural task to assess participants’ abilities to detect configural modifications made on a horizontal or vertical axis. The results showed that when compared with the HC group, the PD group had impaired facial emotion recognition, in particular for faces expressing anger and fear, and exhibited reduced face inversion effect for these emotions. More importantly, the PD group's performance on the configural task to detect vertical modifications was lower than the HC group's. Taken together, these results suggest the presence of a configural processing alteration in patients with PD, especially for vertical, second-order information. Furthermore, configural performance was positively correlated with emotion recognition for anger, disgust, and fear, suggesting that facial emotion recognition could be related, at least partially, to configural processing.     

Facial emotion recognition in agenesis of the corpus callosum

Background

Impaired social functioning is a common symptom of individuals with developmental disruptions in callosal connectivity. Among these developmental conditions, agenesis of the corpus callosum provides the most extreme and clearly identifiable example of callosal disconnection. To date, deficits in nonliteral language comprehension, humor, theory of mind, and social reasoning have been documented in agenesis of the corpus callosum. Here, we examined a basic social ability as yet not investigated in this population: recognition of facial emotion and its association with social gaze.

Methods

Nine individuals with callosal agenesis and nine matched controls completed four tasks involving emotional faces: emotion recognition from upright and inverted faces, gender recognition, and passive viewing. Eye-tracking data were collected concurrently on all four tasks and analyzed according to designated facial regions of interest.

Results

Individuals with callosal agenesis exhibited impairments in recognizing emotions from upright faces, in particular lower accuracy for fear and anger, and these impairments were directly associated with diminished attention to the eye region. The callosal agenesis group exhibited greater consistency in emotion recognition across conditions (upright vs. inverted), with poorest performance for fear identification in both conditions. The callosal agenesis group also had atypical facial scanning (lower fractional dwell time in the eye region) during gender naming and passive viewing of faces, but they did not differ from controls on gender naming performance. The pattern of results did not differ when taking into account full-scale intelligence quotient or presence of autism spectrum symptoms.

Conclusions

Agenesis of the corpus callosum results in a pattern of atypical facial scanning characterized by diminished attention to the eyes. This pattern suggests that reduced callosal connectivity may contribute to the development and maintenance of emotion processing deficits involving reduced attention to others'' eyes.     

Diminished neural sensitivity to irregular facial expression in first‐episode schizophrenia

Introduction: Blunted, inappropriate affective‐social behavior is a hallmark of early schizophrenia, possibly corresponding to reduced ability to recognize and express emotions. It is yet unknown if this affective deficiency relates to disturbed neural sensitivity to facial expressions or to overall face processing. In a previous imaging study, healthy subjects showed less suppression of the fusiform gyrus (FG) to repeated presentation of the same transfigured‐bizarre face relative to regular face. We assumed that the FG in schizophrenia will show reduced repetition related sensitivity to transfigured‐bizarre faces, while having overall normal response to faces. Methods: Ten first‐episode patients with schizophrenia and 10 controls rated the bizarreness of upright and inverted faces. In an fMRI study, another group of 17 first‐episode patients with schizophrenia and 12 controls viewed regular and transfigured‐bizarre faces in blocks. Each block contained regular‐ or transfigured‐bizarre faces of either different or same individual, presented in an upright or inverted orientation. Results: Patients in comparison with controls rated irregular faces as less bizarre. The FG, in patients and controls exhibited similar response to inverted faces, suggesting normal face processing. In contrast, the FG only in patients, showed similar suppression to repeated transfigured‐bizarre and regular faces. Finally, the FG in patients compared with controls showed reduced functional connectivity with the amygdala and prefrontal cortex. Conclusion: Patients with schizophrenia already at first‐episode, showed reduced behavioral and neural sensitivity to bizarre facial expressions. Possibly, this deficiency is related to disturbed modulations of emotion‐related face processing in the FG by the amygdala and prefrontal cortex. Hum Brain Mapp 2009. © 2009 Wiley‐Liss, Inc.     

Facial emotion perception in Chinese patients with schizophrenia and non-psychotic first-degree relatives

Although there is a consensus that patients with schizophrenia have certain deficits in perceiving and expressing facial emotions, previous studies of facial emotion perception in schizophrenia do not present consistent results. The objective of this study was to explore facial emotion perception deficits in Chinese patients with schizophrenia and their non-psychotic first-degree relatives. Sixty-nine patients with schizophrenia, 56 of their first-degree relatives (33 parents and 23 siblings), and 92 healthy controls (67 younger healthy controls matched to the patients and siblings, and 25 older healthy controls matched to the parents) completed a set of facial emotion perception tasks, including facial emotion discrimination, identification, intensity, valence, and corresponding face identification tasks. The results demonstrated that patients with schizophrenia performed significantly worse than their siblings and younger healthy controls in accuracy in a variety of facial emotion perception tasks, whereas the siblings of the patients performed as well as the corresponding younger healthy controls in all of the facial emotion perception tasks. Patients with schizophrenia also showed significantly reduced speed than younger healthy controls, while siblings of patients did not demonstrate significant differences with both patients and younger healthy controls in speed. Meanwhile, we also found that parents of the schizophrenia patients performed significantly worse than the corresponding older healthy controls in accuracy in terms of facial emotion identification, valence, and the composite index of the facial discrimination, identification, intensity and valence tasks. Moreover, no significant differences were found between the parents of patients and older healthy controls in speed after controlling the years of education and IQ. Taken together, the results suggest that facial emotion perception deficits may serve as potential endophenotypes for schizophrenia.     

Using event related potentials to explore stages of facial affect recognition deficits in schizophrenia

Schizophrenia patients show impairments in identifying facial affect; however, it is not known at what stage facial affect processing is impaired. We evaluated 3 event-related potentials (ERPs) to explore stages of facial affect processing in schizophrenia patients. Twenty-six schizophrenia patients and 27 normal controls participated. In separate blocks, subjects identified the gender of a face, the emotion of a face, or if a building had 1 or 2 stories. Three ERPs were examined: (1) P100 to examine basic visual processing, (2) N170 to examine facial feature encoding, and (3) N250 to examine affect decoding. Behavioral performance on each task was also measured. Results showed that schizophrenia patients' P100 was comparable to the controls during all 3 identification tasks. Both patients and controls exhibited a comparable N170 that was largest during processing of faces and smallest during processing of buildings. For both groups, the N250 was largest during the emotion identification task and smallest for the building identification task. However, the patients produced a smaller N250 compared with the controls across the 3 tasks. The groups did not differ in behavioral performance in any of the 3 identification tasks. The pattern of intact P100 and N170 suggest that patients maintain basic visual processing and facial feature encoding abilities. The abnormal N250 suggests that schizophrenia patients are less efficient at decoding facial affect features. Our results imply that abnormalities in the later stage of feature decoding could potentially underlie emotion identification deficits in schizophrenia.     

Temporal Lobe Structures and Facial Emotion Recognition in Schizophrenia Patients and Nonpsychotic Relatives

Temporal lobe abnormalities and emotion recognition deficits are prominent features of schizophrenia and appear related to the diathesis of the disorder. This study investigated whether temporal lobe structural abnormalities were associated with facial emotion recognition deficits in schizophrenia and related to genetic liability for the disorder. Twenty-seven schizophrenia patients, 23 biological family members, and 36 controls participated. Several temporal lobe regions (fusiform, superior temporal, middle temporal, amygdala, and hippocampus) previously associated with face recognition in normative samples and found to be abnormal in schizophrenia were evaluated using volumetric analyses. Participants completed a facial emotion recognition task and an age recognition control task under time-limited and self-paced conditions. Temporal lobe volumes were tested for associations with task performance. Group status explained 23% of the variance in temporal lobe volume. Left fusiform gray matter volume was decreased by 11% in patients and 7% in relatives compared with controls. Schizophrenia patients additionally exhibited smaller hippocampal and middle temporal volumes. Patients were unable to improve facial emotion recognition performance with unlimited time to make a judgment but were able to improve age recognition performance. Patients additionally showed a relationship between reduced temporal lobe gray matter and poor facial emotion recognition. For the middle temporal lobe region, the relationship between greater volume and better task performance was specific to facial emotion recognition and not age recognition. Because schizophrenia patients exhibited a specific deficit in emotion recognition not attributable to a generalized impairment in face perception, impaired emotion recognition may serve as a target for interventions.     

Orienting to social stimuli differentiates social cognitive impairment in autism and schizophrenia

Both autism and schizophrenia feature deficits in aspects of social cognition that may be related to amygdala dysfunction, but it is unclear whether these are similar or different patterns of impairment. We compared the visual scanning patterns and emotion judgments of individuals with autism, individuals with schizophrenia and controls on a task well characterized with respect to amygdala functioning. On this task, eye movements of participants are recorded as they assess emotional content within a series of complex social scenes where faces are either included or digitally erased. Results indicated marked abnormalities in visual scanning for both disorders. Controls increased their gaze on face regions when faces were present to a significantly greater degree than both the autism or schizophrenia groups. While the control and the schizophrenia groups oriented to face regions faster when faces were present compared to when they were absent, the autism group oriented at the same rate in both conditions. The schizophrenia group, meanwhile, exhibited a delay in orienting to face regions across both conditions, although whether anti-psychotic medication contributed to this effect is unclear. These findings suggest that while processing emotional information in social scenes, both individuals with autism and individuals with schizophrenia fixate faces less than controls, although only those with autism fail to orient to faces more rapidly based on the presence of facial information. Autism and schizophrenia may therefore share an abnormality in utilizing facial information for assessing emotional content in social scenes, but differ in the ability to seek out socially relevant cues from complex stimuli. Impairments in social orienting are discussed within the context of evidence suggesting the role of the amygdala in orienting to emotionally meaningful information.     

Event-related potentials to threat-related faces in schizophrenia

Specialised network disturbances such as abnormalities in processing faces, may be associated with functional disturbances of interpersonal communication in schizophrenia. This study focused on the temporal dimension, investigating facial processing deficits in patients with schizophrenia (and non-patient controls) in a passive event-related potential (ERP) paradigm. ERPs invoked to an angry and neutral face stimulus were recorded in 27 patients with schizophrenia and 27 age and sex matched normal controls. Patients with schizophrenia showed a significant generalised delay, and diminished P200 amplitude (primarily frontal) for both stimuli-with more widespread regions of disturbance associated with the angry face. Normal controls, on the other hand, showed relatively reduced posterior P200 amplitude for angry compared to neutral faces, and a lateralised pattern of engagement in response to both stimuli. These findings indicate suboptimal processing of neutral faces in patients with schizophrenia, further exacerbated for affect laden angry faces.     

Facial EMG responses to dynamic emotional facial expressions in boys with disruptive behavior disorders

Based on the assumption that facial mimicry is a key factor in emotional empathy, and clinical observations that children with disruptive behavior disorders (DBD) are weak empathizers, the present study explored whether DBD boys are less facially responsive to facial expressions of emotions than normal controls. Facial electromyographic (EMG) activity in the zygomaticus major and corrugator supercilii muscle regions, and heart rate activity were studied in 22 clinically referred 8-12-year-old DBD boys and 22 age-matched normal controls during exposure to dynamic happy and angry expressions. Dispositional emotional empathy was assessed by a self-report questionnaire for children. The happy and angry facial expressions evoked distinct facial EMG response patterns, with increased zygomaticus muscle activity to happy expressions and increased corrugator muscle activity to angry expressions. The corrugator (but not the zygomaticus) muscle response pattern was less pronounced for DBD boys than the normal controls. Attending to the emotional expressions was associated with equivalent cardiac deceleration in both groups, reflecting a similar orienting/attention response. Lower empathy scores were obtained for DBD boys than for normal controls. In conclusion, facial mimicry responses to angry facial expressions were subnormal in DBD boys, which may be a sign of a deficient early component in the process of emotional empathy, and thus play a role in impaired empathic responding.     

Deficits in facial emotion recognition in schizophrenia: a replication study with korean subjects

Objective

We investigated the deficit in the recognition of facial emotions in a sample of medicated, stable Korean patients with schizophrenia using Korean facial emotion pictures and examined whether the possible impairments would corroborate previous findings.

Methods

Fifty-five patients with schizophrenia and 62 healthy control subjects completed the Facial Affect Identification Test with a new set of 44 colored photographs of Korean faces including the six universal emotions as well as neutral faces.

Results

Korean patients with schizophrenia showed impairments in the recognition of sad, fearful, and angry faces [F(1,114)=6.26, p=0.014; F(1,114)=6.18, p=0.014; F(1,114)=9.28, p=0.003, respectively], but their accuracy was no different from that of controls in the recognition of happy emotions. Higher total and three subscale scores of the Positive and Negative Syndrome Scale (PANSS) correlated with worse performance on both angry and neutral faces. Correct responses on happy stimuli were negatively correlated with negative symptom scores of the PANSS. Patients with schizophrenia also exhibited different patterns of misidentification relative to normal controls.

Conclusion

These findings were consistent with previous studies carried out with different ethnic groups, suggesting cross-cultural similarities in facial recognition impairment in schizophrenia.     

Dissociable face processing impairments after brain injury

Abstract

Matched populations of head-injured patients and normal control subjects completed three “forced-choice” face processing tasks designed to test facial expression recognition, familiar face recognition, and unfamiliar face matching. We hypothesised a significant difference in the performance of the patients and controls on the three tasks, and hoped to observe individual differences in the patients' performance across tasks. As predicted the head-injured patients made significantly more errors than the controls on the forced-choice tasks. Four cases of dissociable impairments affecting only one of the face processing tasks are reported; patient JP impaired only on facial expression recognition, patients AB and HI impaired only on familiar face recognition, and patient VS impaired only on unfamiliar face matching. These dissociable impairments provide further evidence for independent cognitive processing of specific face properties.     

Selective attention to facial emotion and identity in schizophrenia.

The selective attention to facial emotion and identity was investigated in 12 patients with schizophrenia and 12 healthy participants. Both patients and controls were required to perform two classification tasks (according either to identity or emotion). Two separate values for identity (person A/person B) and for emotion (fear/anger) were used. When the classification task was on one dimension, the other dimension was either correlated, constant, or orthogonal (Garner WR. The Processing of Information and Structure. Potomac, MD: Erlbaum, 1974, Garner WR. Interaction of stimulus dimensions in concept and choice processes. Cognitive Psychology 1976;8:98-123). Results indicated that both patients and healthy participants had an asymmetrical pattern of performance: they were able to selectively attend to the identity of the face presented, regardless of the emotion expressed on the face, but variation in identity interfered with the classification of facial emotion. Moreover, a correlational study indicated that the identity interference on emotion classification for schizophrenic patients covaried with the severity of their negative symptoms. The selective attention competencies in schizophrenia and the independence hypothesis of emotion and face recognition are discussed in the framework of current face recognition models.     

Facial expression recognition in people with medicated and unmedicated Parkinson's disease

Recognition of facial expressions of emotion was investigated in people with medicated and unmedicated Parkinson's disease (PD) and matched controls (unmedicated PD, n=16; medicated PD, n=20; controls, n=40). Participants in the medicated group showed some visual impairment (impaired contrast sensitivity) and performed less well on perception of unfamiliar face identity, but did not show significant deficits in the perception of sex, gaze direction, or familiar identity from the face. For both Parkinson's disease groups, there was evidence of impaired recognition of facial expressions in comparison to controls. These deficits were more consistently noted in the unmedicated group, who were also found to perform worse than the medicated group at recognising disgust from prototypical facial expressions, and at recognising anger and disgust in computer-manipulated images. Although both Parkinson's disease groups showed impairments of facial expression recognition, the consistently worse recognition of disgust in the unmedicated group is consistent with the hypothesis from previous studies that brain regions modulated by dopaminergic neurons are involved in the recognition of disgust.