Asymmetry of facial expression in spontaneous emotion

The observation that emotional expressions are more intense on the left side of the face is consistent with other evidence of the importance of the right hemisphere in emotional communication. However, the question has been raised whether it is truly spontaneous emotional expressions or only posed facial displays that show a left-sided asymmetry. We surreptitiously examined facial asymmetry during spontaneous emotional expressions as subjects remembered happy or sad experiences. These were contrasted with the subjects' posed expressions of happy or sad emotions. Both of these procedures resulted in more intense expressions on the left side of the face. The left-sided advantage was stronger during the spontaneous than the posed displays, and was observed for both happy and sad emotions.     

The role of structural facial asymmetry in asymmetry of peak facial expressions

Asymmetric facial expression is generally attributed to asymmetry in movement, but structural asymmetry in the face may also affect asymmetry of expression. Asymmetry in posed expressions was measured using image-based approaches in digitised sequences of facial expression in 55 individuals, N=16 men, N=39 women. Structural asymmetry (at neutral expression) was higher in men than women and accounted for .54, .62, and .66 of the variance in asymmetry at peak expression for joy, anger, and disgust expressions, respectively. Movement asymmetry (measured by change in pixel values over time) was found, but was unrelated to peak asymmetry in joy or anger expressions over the whole face and in facial subregions relevant to the expression. Movement asymmetry was negatively related to peak asymmetry in disgust expressions. Sidedness of movement asymmetry (defined as the ratio of summed movement on the left to movement on the right) was consistent across emotions within individuals. Sidedness was found only for joy expressions, which had significantly more movement on the left. The significant role of structural asymmetry in asymmetry of emotion expression and the exploration of facial expression asymmetry have important implications for evolutionary interpretations of facial signalling and facial expressions in general.     

The role of structural facial asymmetry in asymmetry of peak facial expressions

Asymmetric facial expression is generally attributed to asymmetry in movement, but structural asymmetry in the face may also affect asymmetry of expression. Asymmetry in posed expressions was measured using image-based approaches in digitised sequences of facial expression in 55 individuals, N=16 men, N=39 women. Structural asymmetry (at neutral expression) was higher in men than women and accounted for .54, .62, and .66 of the variance in asymmetry at peak expression for joy, anger, and disgust expressions, respectively. Movement asymmetry (measured by change in pixel values over time) was found, but was unrelated to peak asymmetry in joy or anger expressions over the whole face and in facial subregions relevant to the expression. Movement asymmetry was negatively related to peak asymmetry in disgust expressions. Sidedness of movement asymmetry (defined as the ratio of summed movement on the left to movement on the right) was consistent across emotions within individuals. Sidedness was found only for joy expressions, which had significantly more movement on the left. The significant role of structural asymmetry in asymmetry of emotion expression and the exploration of facial expression asymmetry have important implications for evolutionary interpretations of facial signalling and facial expressions in general.     

Posed emotional expression in unilateral brain damaged patients

The facial expressions of six basic emotions were posed by two groups of right (N = 23) and left (N = 34) brain damaged patients and by a control group of normal subjects (N = 28). The posed expressions were examined by means of the Facial Action Coding System (FACS) which provides analytical and objective scoring, as by a subjective scale of appropriateness of expression. Results indicated no difference between controls and patients with a lesion in the right or left hemisphere. These findings are inconsistent with the hypothesis that the right hemisphere plays a specific role in the control of posed facial expression. No relationship was observed between posed emotional expressions and facial paralysis or the presence of oral apraxia.     

Facial expression and emotional face recognition in schizophrenia and depression

Summary Twenty-three acute schizophrenics, 21 acute major depressives (Research Diagnostic Criteria), and 15 normal controls participated in a study on facial expression and emotional face recognition. Under clinical conditions, spontaneous facial expression was assessed according to the affective flattening section of the Scale for the Assessment of Negative Symptoms. Under experimental laboratory conditions involuntary (emotioneliciting interview) and voluntary facial expression (imitation and simulation of six basic emotions) were recorded on videotape, from which a raterbased analysis of intensity or correctness of facial activity was obtained. Emotional face recognition was also assessed under experimental conditions using the same stimulus material. All subjects were assessed twice (within 4 weeks), controlling for change of the psychopathological status in the patient groups. In schizophrenics, neuroleptic drug influence was controlled by random allocation to treatment with either haloperidol or perazine. The main findings were that schizophrenics and depressives are characterized by different quantitative, qualitative, and temporal patterns of affect-related dysfunctions. In particular, schizophrenics demonstrated a trait-like deficit in affect recognition and in their spontaneous and voluntary facial activity, irrespective of medication, drug type and dosage, or extrapyramidal side-effects. In depressives a stable deficit could be demonstrated only in their involuntary expression under emotion-eliciting interview conditions, whereas in the postacute phase a reduction in their voluntary expression became apparent. Differences in patterns of affect-related behavioral deficits may reflect dysfunctions in different underlying psychobiological systems.     

Human facial expressions are organized functionally across the upper-lower facial axis.

Most clinical research has focused on intensity differences of facial expressions between the right and left hemiface to explore lateralization of emotions in the brain. Observations by social psychologists, however, suggest that control of facial expression is organized predominantly across the upper-lower facial axis because of the phenomena of facial blends: simultaneous display of different emotions on the upper and lower face. Facial blends are related to social emotions and development of display rules that allow individuals to sculpt facial expressions for social and manipulative purposes. We have demonstrated that facial blends of emotion are more easily and accurately posed on the upper-lower than right-left hemiface, and that upper facial emotions are processed preferentially by the right hemisphere whereas lower facial emotions are processed preferentially by the left hemisphere. Based on these results, recent anatomical studies showing separate cortical areas for motor control of upper and lower face and the neurology of posed and spontaneous expressions of emotion, a functional-anatomic model of how the forebrain modulates facial expressions, is presented. The unique human ability to produce facial blends of emotion is, most likely, an adaptive modification linked to the evolution of speech and language.     

Left-sided oral asymmetries in spontaneous but not posed smiles

To investigate possible facial asymmetries during the production of posed and spontaneous smiles, the displacement of various reference points on the mouth were measured as subjects produced both kinds of smiles. Strobe cameras were used in combination with a computer-based analysis to record the smiles of left- and right-handed males and females. The analysis revealed that the left side of the mouth moved more than the right side during spontaneous but not posed smiles, supporting the notion that the right hemisphere may play a special role in emotional expression. This asymmetry was most apparent in left-handed females and right-handed males. These sex and handedness differences are discussed with reference to apparent inconsistencies in previous research on asymmetries in emotional expression.     

Left-handers' smiles: asymmetries in the projection of a posed expression

Asymmetries in the expression of a posed smile and in a relaxed facial expression were observed in 24 left-handers. Neither writing position nor familial sinistrality predicted the variance of the results. Left-handers were found to smile more with their left than right side of face; an asymmetry which had previously been observed in right-handers. If anything, left-handers' smiles were more asymmetric, though in the same direction, than right-handers. When relaxed, however, the left-handers' face was judged more unhappy on its right than left side--a reversal of the direction of asymmetry previously noted in right-handers' relaxed expressions. No single neurological or psychological theory accounts for these results; it is suggested that hand preference may exert a myotonic effect which is reflected in judgements of relaxed facial expressions.     

Facial expressiveness in patients with schizophrenia compared to depressed patients and nonpatient comparison subjects

OBJECTIVE: Blunted affect is a major symptom in schizophrenia, and affective deficits clinically encompass deficits in expressiveness. Emotion research and ethological studies have shown that patients with schizophrenia are impaired in various modalities of expressiveness (posed and spontaneous emotion expressions, coverbal gestures, and smiles). Similar deficits have been described in depression, but comparative studies have brought mixed results. Our aim was to study and compare facial expressive behaviors related to affective deficits in patients with schizophrenia, depressed patients, and nonpatient comparison subjects. METHOD: Fifty-eight nondepressed inpatients with schizophrenia, 25 nonpsychotic inpatients with unipolar depression, and 25 nonpatient comparison subjects were asked to reproduce facial emotional expressions. Then the subjects were asked to speak about a specific emotion for 2 minutes. Each time, six cross-cultural emotions were tested. Facial emotional expressions were rated with the Facial Action Coding System. The number of facial coverbal gestures (facial expressions that are tied to speech) and the number of words were calculated. RESULTS: In relation to nonpatient comparison subjects, both patient groups were impaired for all expressive variables. Few differences were found between schizophrenia and depression: depressed subjects had less spontaneous expressions of other-than-happiness emotions, but overall, they appeared more expressive. Fifteen patients with schizophrenia were tested without and with typical or atypical antipsychotic medications: no differences could be found in study performance. CONCLUSIONS: The patients with schizophrenia and the patients with depression presented similar deficits in various expressive modalities: posed and spontaneous emotional expression, smiling, coverbal gestures, and verbal output.     

Asymmetry of the face at rest: size, area and emotional expression

In a sample of 53 right-handed, young adult males, asymmetry was examined in the size of 11 facial regions, in total area, and in emotional expression of the face at rest. In the complete sample no evidence was obtained of consistent asymmetry in size of facial regions, in facial area, or in emotional expression. There was, however, an interaction between family history of sinistrality and sighting dominance in facial asymmetry for the pleasantness/unpleasantness of facial expression. The findings suggest that previously established asymmetries in phasic emotional displays are not an outcome of expressive or morphological asymmetries of the face at rest.     

What can asymmetry and laterality in EMG tell us about the face and brain?

This paper will review the use of the electromyographic (EMG) technique for studying asymmetry and laterality in facial behavior. My discussion will include several issues: (1) a review of the neurology of facial expression with respect to asymmetry/laterality; (2) a compendium of models of cerebral hemispheric participation in emotion and in facial motor control that imply asymmetry-laterality in facial behavior; (3) a summary of facial asymmetry-laterality findings to date; and (4) uses and abuses of EMG techniques in inferring asymmetry-laterality in facial behavior. I conclude with (5) recommendations for future facial asymmetry-laterality research in general.     

Automated video-based facial expression analysis of neuropsychiatric disorders

Deficits in emotional expression are prominent in several neuropsychiatric disorders, including schizophrenia. Available clinical facial expression evaluations provide subjective and qualitative measurements, which are based on static 2D images that do not capture the temporal dynamics and subtleties of expression changes. Therefore, there is a need for automated, objective and quantitative measurements of facial expressions captured using videos. This paper presents a computational framework that creates probabilistic expression profiles for video data and can potentially help to automatically quantify emotional expression differences between patients with neuropsychiatric disorders and healthy controls. Our method automatically detects and tracks facial landmarks in videos, and then extracts geometric features to characterize facial expression changes. To analyze temporal facial expression changes, we employ probabilistic classifiers that analyze facial expressions in individual frames, and then propagate the probabilities throughout the video to capture the temporal characteristics of facial expressions. The applications of our method to healthy controls and case studies of patients with schizophrenia and Asperger's syndrome demonstrate the capability of the video-based expression analysis method in capturing subtleties of facial expression. Such results can pave the way for a video-based method for quantitative analysis of facial expressions in clinical research of disorders that cause affective deficits.     

Observations on spontaneous facial expression after focal cerebral excisions and after intracarotid injection of sodium amytal

Ethological recording techniques were used to demonstrate that patients with unilateral frontal-lobe lesions exhibit a reduction in spontaneous facial expression, as compared to patients with unilateral parietal- or temporal-lobe lesions. There was no evidence of hemispheric asymmetry on these measures. Observation of patients during preoperative speech testing following an intracarotid injection of sodium Amytal revealed a significant reduction in spontaneous facial expressions, which outlasted any contralateral facial weakness. This result was independent of side of injection, probable lesion site, and hemisphere mediating speech. Subjective assessments of mood change following Amytal testing also showed no consistent relationship between side of injection and direction of mood change.     

Digitizing the moving face during dynamic displays of emotion

Humans typically decode facial signals during dynamic interactions in which the face moves. In this study, we digitized real time video signals in order to examine movement asymmetries across the face during emotional and nonemotional expressions. Forty dextral males were tested. For each expression, a 400 ms video segment was analyzed for changes in signal value (pixel intensity) over consecutive frames. The upper and lower face regions were examined separately due to differences in the cortical enervation of facial muscles in the upper (bilateral) vs lower face (contralateral). Results revealed distinctly different movement asymmetries over the lower and upper hemiface. In the upper face, more movement occurred over the right side for most facial expressions, regardless of emotionality. The latter finding questions the assumption that muscles of the upper face are symmetrical and/or bilaterally enervated in a symmetrical manner. In the lower face, negative expressions linked to fight-flight emotions (i.e. fear, anger) were associated with greater left sided movement, whereas happiness tended to be associated with more right sided movement. No consistent pattern of movement asymmetry occurred for nonemotional expressions. Although the valence-related movement asymmetries in the lower face are consistent with neuropsychological models of emotional expressivity, it remains unclear whether they reflect activation or inhibitory hemispheric mechanisms. Taken together, these data suggest that multiple factors may contribute to expressive movement asymmetries of the face.     

Facial asymmetry while posing positive and negative emotions: support for the right hemisphere hypothesis

The purpose of this report was to examine facial asymmetry during the expression of positive and negative emotions. In addition, methodological factors in the study of facial asymmetry were considered. Subjects were 16 neurologically-healthy right-handed adult males, videotaped while posing eight facial expressions (positive and negative) under two conditions (verbal command and visual imitation). Separate asymmetry ratings of the two sides of the face were made by judges viewing normal or mirror-reversed versions of the videotape. There were no effects of valence, condition, or videotape orientation on the asymmetry ratings, and, in general, expressions were produced significantly more intensely on the left than the right side of the face. This finding is consistent with the hypothesis that the right cerebral hemisphere is dominant for the expression of facial emotion of both valences.     

asymmetry of Facial Expression Related to Handedness,Footedness, and Eyedness: a Quantitative Study

This study investigates the relationship between traditional measures of lateral dominance and facial asymmetry or facedness, i.e., the relative intensity of expression "and muscular involvement" on the two sides of the face. Subjects were 19 left handed and 29 right-handed normal adults coached to produce eight facial expressions of emotion and one expression involving unilateral facial movement. Videotapes of each subject were reliably rated for facedness. a lateral dominance battery yielded preference scores for hand, foot, and eye, and performance scores for accuracy, speed, and strength. Facedness and lateral dominance scores were intercorrelated. For all subjects, facedness for the eight emotional expressions was positively related to eyedness, but unrelated to handedness or footedness. Facial asymmetry furing emotional expression was more intense on the left side of the lower , face and may be a right hemisphere function, regardless of handedness. The one expression requiring deliberate unilateral facial movement was significantly and inversely related to handedness and footedness.     

Similar facial electromyographic responses to faces, voices, and body expressions

Observing facial expressions automatically prompts imitation, as can be seen with facial electromyography. To investigate whether this reaction is driven by automatic mimicry or by recognition of the emotion displayed we recorded electromyograph responses to presentations of facial expressions, face-voice combinations and bodily expressions, which resulted from happy and fearful stimuli. We observed emotion-specific facial muscle activity (zygomaticus for happiness, corrugator for fear) for all three stimulus categories. This indicates that spontaneous facial expression is more akin to an emotional reaction than to facial mimicry and imitation of the seen face stimulus. We suggest that seeing a facial expression, an emotional body expression or hearing an emotional tone of voice all activate the affect program corresponding to the emotion displayed.     

Age determines memory for face identity and expression

Background: The recognition of facial expressions is an important component of emotion processing which contributes to interactional behavior. One of the factors highly associated with potential decline of ability in behavioral tasks is age. Methods: We have investigated age‐related changes in facial identity and expression memory of healthy subjects in three age groups: young adults (20–40 years), elderly adults (60–80 years) and, for the first time in the literature, very old adults (over 80 years of age). Using a picture test, photographs of faces with happy or angry expressions were presented to study participants during the encoding task, and the memory for identity and emotional facial expression was investigated in a subsequent recognition task showing emotionally neutral faces. Half of the faces presented in the recognition task were initially shown in the encoding task. Results: Age interacted with the memory process: the ability to recognize both facial identity and emotional expression declined with advanced age. Happy facial expressions were better recognized in all age groups. Although there was a continuous overall decrease in recognition of both happy and angry expressions with advanced age, the effect favoring happy facial expressions was stable also in very old adults. Other factors such as gender or educational level did not affect the memory process for facial expressions. Conclusions: Our findings suggest that age is a significant determinant of memory for facial identity and emotional expression, and that, similar to younger adults, the recognition process of the elderly favors happy emotional facial expressions.     

Deliberate emotional expressions of socially anxious children and their mothers

The aim of the present study was to investigate whether socially anxious children show deficits in their deliberate facial expression of emotions. To test for potential mother-child transmission effects, the mothers' facial expressions were also assessed. Fifty socially anxious and 25 socially nonanxious children (8-12 years) and their mothers participated in a facial expression posing task. The expressions produced were coded using Ekman and Friesen's (1978) Facial Action Coding System (FACS). In addition, naive raters rated their quality of emotion. FACS analyses indicated that socially anxious children show a reduced general facial activity, have a more restricted facial repertoire and differ qualitatively from controls in their facial expression of emotions. Similarly, the global ratings indicated that the socially anxious children's posed facial expressions are less accurate. For the mothers no differences between groups were found when global ratings were used. However, the FACS data demonstrate that the facial expressions of mothers of socially anxious children are less intense compared to controls. It is possible that the decreased intensity of the mothers' facial expressiveness makes it more difficult for the socially anxious children to learn adequate facial expressions.     

Bradykinesia of posed smiling and voluntary movement of the lower face in Parkinson's disease

ObjectiveImpaired facial expression, including spontaneous and emotional movements such as smiling, has been often reported in Parkinson's disease (PD). There is a general consensus that spontaneous smiling is abnormal in PD. Investigations on posed smiling yield contrasting results. Moreover, no study has yet addressed the relationship between posed smiling and abnormalities of voluntary movements of the lower face, global motor impairment and the effects of dopaminergic medication.MethodsWe investigated the kinematics of posed smiling (mimicking a smile shown in a picture) and those of voluntary movements of the lower face (showing the teeth as fast as possible – voluntary grinning) in 15 patients with PD (ON and OFF therapy) and in 16 healthy controls. Facial movements were recorded using a 3D optoelectronic system and analyzed using dedicated software.ResultsSome kinematic parameters of both posed smiling and voluntary grinning were abnormally lower in PD patients in comparison to healthy subjects. The kinematics of posed smiling correlated with those of voluntary grinning in PD patients but not in healthy controls. Posed smiling and voluntary grinning abnormalities were related to global motor severity but did not significantly improve upon L-dopa administration.ConclusionsThese results suggest that posed smiling and voluntary grinning are both abnormal in PD patients and that they are likely mediated by a common pathophysiological mechanism.