Patterns of Visual Attention to Faces and Objects in Autism Spectrum Disorder

This study used eye-tracking to examine visual attention to faces and objects in adolescents with autism spectrum disorder (ASD) and typical peers. Point of gaze was recorded during passive viewing of images of human faces, inverted human faces, monkey faces, three-dimensional curvilinear objects, and two-dimensional geometric patterns. Individuals with ASD obtained lower scores on measures of face recognition and social-emotional functioning but exhibited similar patterns of visual attention. In individuals with ASD, face recognition performance was associated with social adaptive function. Results highlight heterogeneity in manifestation of social deficits in ASD and suggest that naturalistic assessments are important for quantifying atypicalities in visual attention.     

Reduced binocular depth inversion in schizophrenic patients.

Binocular depth inversion represents an illusion of visual perception, serving to invert the perception of implausible hollow objects, e.g. a hollow face into a normal face. Such inversion occurs frequently, especially when objects with a high degree of familiarity (e.g. photographs of faces) are displayed. Under normal conditions, cognitive factors apparently override the binocular disparity cues of stereopsis. This internal mechanism--a kind of "censorship" of perception balancing "top-down" and "bottom-up" processes of perception--appears to be disturbed in psychotic states. The clinical and neuropsychological performance of schizophrenic patients was assessed using the Brief Psychiatric Rating Scale (BPRS), the Positive And Negative Symptoms Scale (PANSS), the Clinical Global Impression Scale (CGI), the Mehrfach-Wahlwortschatz Intelligence Test (MWT-B) and the binocular depth inversion test (BDIT) using pictures with a high degree of familiarity. In schizophrenic patients, the performance in the BDIT differed significantly from healthy controls and from patients with major depression. The schizophrenic patients were more veridical in their judgements in the BDIT. During antipsychotic treatment, BPRS and PANSS scores improved and the inversed faces were seen as more illusionary, driven by an increase in top-down processing. At the end of treatment, there was no significant difference between the patient group and the healthy controls in the score of binocular depth inversion. These findings suggest that testing of binocular depth inversion can detect specific dysfunctions in visual perception and might be useful as a state-marker for psychotic states.     

Reprint of: Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.     

Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.     

An Event-Related Potential Investigation of Early Visual Processing Deficits During Face Perception in Youth at Clinical High Risk for Psychosis

Impairments in early visual face perception are well documented in patients with schizophrenia. Specifically, event-related potential (ERP) research in patients with schizophrenia has demonstrated deficits in early sensory processing of stimulus properties (P1 component) and the structural encoding of faces (N170 component). However, it is not well understood if similar impairments are present in individuals at clinical high risk (CHR) for psychosis (ie, those in the putative prodromal stage of the illness). Thus, it is unknown if face perception deficits are the result of illness onset or are present in the high-risk period for the illness. The present study used the ERP technique to examine neural activation when viewing facial emotion expressions and objects in 44 CHR and 47 control adolescents and young adults (N = 91). P1 amplitude was similar across groups, indicating that early sensory processing impairments did not substantially contribute to face perception deficits in CHR youth. CHR youth exhibited reduced N170 amplitude compared to controls when viewing faces but not objects, implicating a specific deficit in the structural encoding of faces rather than a general perceptual deficit. Further, whereas controls demonstrated the expected face-selective N170 effect (ie, larger amplitude for faces than objects), CHR youth did not, which suggests that facial emotion expressions do not elicit the expected preferential perceptual processing for critical social information in individuals at CHR for psychosis. Together, these findings provide valuable information regarding the specific impairments contributing to face perception deficits in the high-risk period where treatment stands to aid in preventing illness progression.     

Visual recognition of faces, objects, and words using degraded stimuli: where and when it occurs

We studied time course and cerebral localisation of word, object, and face recognition using event-related potentials (ERPs) and source localisation techniques. To compare activation rates of these three categories, we used degraded images that easily pop out without any change in the physical features of the stimuli, once the meaning is revealed. Comparisons before and after identification show additional periods of activation beginning at 100 msec for faces and at around 200 msec for objects and words. For faces, this activation occurs predominantly in right temporal areas, whereas for objects, the specific time period gives rise to bilateral posterior but right dominant foci. Finally, words show a maximum area of activation in the left temporooccipital area at their specific time period. These results provide unequivocal evidence that when effects of low-level visual features are circumvented, faces, objects, and words are not only distinct in terms of their anatomic routes, but also in terms of their times of processing.     

Visual and cognitive processing of face information in schizophrenia: detection, discrimination and working memory

Face recognition involves several physiological and psychological processes, including those in visual, cognitive and affective domains. Studies have found that schizophrenia patients are deficient at recognizing facial emotions, yet visual and cognitive processing of facial information in this population has not been systematically examined. In this study, we examined visual detection, perceptual discrimination and working memory of faces as well as non-face visual objects in patients. Visual detection was measured by accuracy when detecting the presence of a briefly displayed face, image which contained only the basic configural information of a face. Perceptual discrimination was measured by discriminability scores for individual facial identity images, in which the degree of similarity between images was systematically varied via morphing. Working memory was measured by the discriminability scores when two comparison face images were separated by 3 or 10 s. All measurements were acquired using a psychophysical method (two-alternative forced choice). Relative to controls, patients showed significantly reduced accuracy in visual detection of faces (p=0.003), moderately degraded performance in perceptual discrimination of faces (p=0.065), and significantly impaired performance in working memory of faces (p<0.001 for both 3 and 10 sec conditions). Patients' performance on non-face versions of these tasks, while degraded, was not correlated with performance on face recognition. This pattern of results indicates that greater signal strength is required for visual and cognitive processing of facial information in schizophrenia.     

Attention to social stimuli and facial identity recognition skills in autism spectrum disorder

Background Previous research suggests that individuals with autism spectrum disorder (ASD) have a reduced preference for viewing social stimuli in the environment and impaired facial identity recognition. Methods Here, we directly tested a link between these two phenomena in 13 ASD children and 13 age‐matched typically developing (TD) controls. Eye movements were recorded while participants passively viewed visual scenes containing people and objects. Participants also completed independent matching tasks for faces and objects. Results and Conclusions Behavioural data showed that participants with ASD were impaired on both face‐ and object‐matching tasks relative to TD controls. Eye‐tracking data revealed that both groups showed a strong bias to orient towards people. TD children spent proportionally more time looking at people than objects; however, there was no difference in viewing times between people and objects in the ASD group. In the ASD group, an individual's preference for looking first at the people in scenes was associated with level of face recognition ability. Further research is required to determine whether a causal relationship exists between these factors.     

Mechanisms of face perception in humans: A magneto‐ and electro‐encephalographic study

We have been studying the underlying mechanisms of face perception in humans using magneto‐ (MEG) and electro‐encephalography (EEG) including (1) perception by viewing the static face, (2) differences in perception by viewing the eyes and whole face, (3) the face inversion effect, (4) the effect of gaze direction, (5) perception of eye motion, (6) perception of mouth motion, and (7) the interaction between auditory and visual stimuli related to the vowel sounds. In this review article, we mainly summarize our results obtained on 3, 5, and 6 above. With the presentation of both upright and inverted unfamiliar faces, the inferior temporal cortex (IT) centered on the fusiform gyrus, and the lateral temporal cortex (LT) near the superior temporal sulcus were activated simultaneously, but independently, between 140 and 200 ms post‐stimulus. The right hemisphere IT and LT were both active in all subjects, and those in the left hemisphere in half of the subjects. Latencies with inverted faces relative to those with upright faces were longer in the right hemisphere, and shorter in the left hemisphere. Since the activated regions under upright and those under inverted face stimuli did not show a significant difference, we consider that differences in processing upright versus inverted faces are attributable to temporal processing differences rather than to processing of information by different brain regions. When viewing the motion of the mouth and eyes, a large clear MEG component, 1M (mean peak latency of approximately 160 ms), was elicited to both mouth and eye movement, and was generated mainly in the occipito‐temporal border, at human MT/V5. The 1M to mouth movement and the 1M to eye movement showed no significant difference in amplitude or generator location. Therefore, our results indicate that human MT/V5 is active in the perception of both mouth and eye motion, and that the perception of movement of facial parts is probably processed similarly.     

The effect of gaze direction on three-dimensional face recognition in infant brain activity

In three-dimensional face recognition studies, it is well known that viewing rotating faces enhance face recognition. For infants, our previous study indicated that 8-month-old infants showed recognition of three-dimensional rotating faces with a direct gaze, and they did not learn with an averted gaze. This suggests that gaze direction may affect three-dimensional face recognition in infants. In this experiment, we used near-infrared spectroscopy to measure infants' hemodynamic responses to averted gaze and direct gaze. We hypothesized that infants would show different neural activity for averted and direct gazes. The responses were compared with the baseline activation during the presentation of non-face objects. We found that the concentration of oxyhemoglobin increased in the temporal cortex on both sides only during the presentation of averted gaze compared with that of the baseline period. This is the first study to show that infants' brain activity in three-dimensional face processing is different between averted gaze and direct gaze.     

Dissociating the Neural Correlates of Consciousness and Task Relevance in Face Perception Using Simultaneous EEG-fMRI

Current theories of visual consciousness disagree about whether it emerges during early stages of processing in sensory brain regions or later when a widespread frontoparietal network becomes involved. Moreover, disentangling conscious perception from task-related postperceptual processes (e.g., report) and integrating results across different neuroscientific methods remain ongoing challenges. The present study addressed these problems using simultaneous EEG-fMRI and a specific inattentional blindness paradigm with three physically identical phases in female and male human participants. In phase 1, participants performed a distractor task during which line drawings of faces and control stimuli were presented centrally. While some participants spontaneously noticed the faces in phase 1, others remained inattentionally blind. In phase 2, all participants were made aware of the task-irrelevant faces but continued the distractor task. In phase 3, the faces became task-relevant. Bayesian analysis of brain responses demonstrated that conscious face perception was most strongly associated with activation in fusiform gyrus (fMRI) as well as the N170 and visual awareness negativity (EEG). Smaller awareness effects were revealed in the occipital and prefrontal cortex (fMRI). Task-relevant face processing, on the other hand, led to strong, extensive activation of occipitotemporal, frontoparietal, and attentional networks (fMRI). In EEG, it enhanced early negativities and elicited a pronounced P3b component. Overall, we provide evidence that conscious visual perception is linked with early processing in stimulus-specific sensory brain areas but may additionally involve prefrontal cortex. In contrast, the strong activation of widespread brain networks and the P3b are more likely associated with task-related processes.SIGNIFICANCE STATEMENT How does our brain generate visual consciousness—the subjective experience of what it is like to see, for example, a face? To date, it is hotly debated whether it emerges early in sensory brain regions or later when a widespread frontoparietal network is activated. Here, we use simultaneous fMRI and EEG for high spatial and temporal resolution and demonstrate that conscious face perception is predominantly linked to early and occipitotemporal processes, but also prefrontal activity. Task-related processes (e.g., decision-making), on the other hand, elicit brain-wide activations including late and strong frontoparietal activity. These findings challenge numerous previous studies and highlight the importance of investigating the neural correlates of consciousness in the absence of task relevance.     

Cortical sensitivity to contrast polarity and orientation of faces is modulated by temporal-nasal hemifield asymmetry

Behavioral studies demonstrate that the efficiency of detection of faces is dependent on configural and contrast polarity information characteristic to human faces. Stimulus inversion or contrast polarity reversal can disrupt this process. We investigated whether a face-sensitive event-related potential component, the N170, is modulated by the orientation and contrast polarity of highly degraded schematic face-like patterns (Experiment 1) in the same manner as it is for face photographs (Experiment 2). Inversion and/or contrast reversal delayed and enhanced the N170 for both kinds of stimuli, suggesting that a white oval with three black squares is sufficient to elicit face-sensitive cortical responses. In Experiment 3 we further tested whether the extrageniculate visual pathways modulate early cortical responses to faces. We found that the N170 responses to configural and contrast information are modulated by temporal-nasal visual field asymmetry under monocular viewing conditions, suggesting the involvement of subcortical, extrageniculate visual pathways in face detection. These results are consistent with the idea that an ontogenetically early and primitive bias to orient towards face-like patterns with relevant configural and contrast information influences the early stages of cortical face processing.     

Affective blindsight in the intact brain: neural interhemispheric summation for unseen fearful expressions

The emotional valence of facial expressions can be reliably discriminated even in the absence of conscious visual experience by patients with lesions to the primary visual cortex (affective blindsight). Prior studies in one such patient (GY) also showed that this non-conscious perception can influence conscious recognition of normally seen emotional faces. Here we report a similar online interaction across hemispheres between conscious and non-conscious perception of emotions in normal observers. Fearful and happy facial expressions were presented either unilaterally (to the left or right visual field) or simultaneously to both visual fields. In bilateral displays, conscious perception of one face in a pair was prevented by backward masking after 20 ms, while the opposite expression remained normally visible. The results showed a bidirectional influence of non-conscious fear processing over conscious recognition of happy as well as fearful expressions. Consciously perceived fearful faces were more readily recognized when they were paired with invisible emotionally congruent fearful expressions in the opposite field, as compared to the single presentation of the same unmasked faces. On the other hand, recognition of unmasked happy faces was delayed by the simultaneous presence of a masked fearful face. No such effect was reported for masked happy expressions. These findings show that non-conscious processing of fear may modulate ongoing conscious evaluation of facial expressions via neural interhemispheric summation even in the intact brain.     

Integrating physical and social cues when forming face preferences: Differences among low and high-anxiety individuals

Abstract

This study investigated individual differences in the integration of social (i.e., direction of social interest) and physical (i.e., apparent health) cues in a face preference test. While low-anxiety individuals demonstrated preferences for social engagement from healthy-looking faces, but not from unhealthy-looking faces, high-anxiety individuals preferred social engagement from both healthy and unhealthy individuals. Importantly, anxious individuals were not simply less discriminating in their face preferences generally: anxiety levels were positively related to the strength of overall preferences for positive social interest. Collectively, our findings show that perceptions of gaze and expression can be modulated by aspects of facial appearance and that systematic variation among individuals exists in the extent to which this modulation occurs. Furthermore, since previous studies have demonstrated hypersensitive amygdala responses among anxious individuals when viewing faces, our findings suggest that the amygdala may play an important role in determining how different facial cues are integrated in person perception.     

The robustness of perception

The natural environment around us, which is often crowded, cluttered or even foggy, is subject to a dynamically changing composition of objects and events. The human brain is continuously perceiving, recognizing and evaluating this dynamic scene composition. If the perception of degraded visual objects is important, e.g. in the case of potential threat stimuli, the brain needs to be more sensitive in detecting these objects from the natural environment. It is therefore hypothesized that reacting to the dynamically changing environment involves a robust and quick processing of salient information, which can be either with or without conscious awareness. We investigated the dynamics and robustness of perception using pictures of three salience levels, i.e. fearful faces (most salient), neutral faces (salient) and houses (nonsalient), which appear from dynamically decreasing random visual noise. Stimuli were matched for luminance, contrast, brightness and spatial frequency information. Reaction times show a significantly earlier response for faces than for houses. Fearful faces were significantly more quickly detected than neutral faces. The neural correlates sustaining robust perception were investigated with event-related functional magnetic resonance imaging (fMRI). The amygdala showed a significant perception-related response for faces, as compared to houses, that was further enhanced for fearful faces as compared to neutral faces. Our data indicate that emotionally salient information processing is (i) mediated by the amygdala and (ii) more robust than for nonsalient stimuli as it shows a significantly lower perceptual threshold.     

Visual processing in anorexia nervosa and body dysmorphic disorder: Similarities,differences, and future research directions

Anorexia nervosa (AN) and body dysmorphic disorder (BDD) are psychiatric disorders that involve distortion of the experience of one's physical appearance. In AN, individuals believe that they are overweight, perceive their body as “fat,” and are preoccupied with maintaining a low body weight. In BDD, individuals are preoccupied with misperceived defects in physical appearance, most often of the face. Distorted visual perception may contribute to these cardinal symptoms, and may be a common underlying phenotype. This review surveys the current literature on visual processing in AN and BDD, addressing lower- to higher-order stages of visual information processing and perception. We focus on peer-reviewed studies of AN and BDD that address ophthalmologic abnormalities, basic neural processing of visual input, integration of visual input with other systems, neuropsychological tests of visual processing, and representations of whole percepts (such as images of faces, bodies, and other objects). The literature suggests a pattern in both groups of over-attention to detail, reduced processing of global features, and a tendency to focus on symptom-specific details in their own images (body parts in AN, facial features in BDD), with cognitive strategy at least partially mediating the abnormalities. Visuospatial abnormalities were also evident when viewing images of others and for non-appearance related stimuli. Unfortunately no study has directly compared AN and BDD, and most studies were not designed to disentangle disease-related emotional responses from lower-order visual processing. We make recommendations for future studies to improve the understanding of visual processing abnormalities in AN and BDD.     

Object identification in cerebral visual impairment characterized by gaze behavior and image saliency analysis

Individuals with cerebral visual impairment (CVI) have difficulties identifying common objects, especially when presented as cartoons or abstract images. In this study, participants were shown a series of images of ten common objects, each from five possible categories ranging from abstract black & white line drawings to color photographs. Fifty individuals with CVI and 50 neurotypical controls verbally identified each object and success rates and reaction times were collected. Visual gaze behavior was recorded using an eye tracker to quantify the extent of visual search area explored and number of fixations. A receiver operating characteristic (ROC) analysis was also carried out to compare the degree of alignment between the distribution of individual eye gaze patterns and image saliency features computed by the graph-based visual saliency (GBVS) model. Compared to controls, CVI participants showed significantly lower success rates and longer reaction times when identifying objects. In the CVI group, success rate improved moving from abstract black & white images to color photographs, suggesting that object form (as defined by outlines and contours) and color are important cues for correct identification. Eye tracking data revealed that the CVI group showed significantly greater visual search areas and number of fixations per image, and the distribution of eye gaze patterns in the CVI group was less aligned with the high saliency features of the image compared to controls. These results have important implications in helping to understand the complex profile of visual perceptual difficulties associated with CVI.     

Integrating physical and social cues when forming face preferences: differences among low and high-anxiety individuals

This study investigated individual differences in the integration of social (i.e., direction of social interest) and physical (i.e., apparent health) cues in a face preference test. While low-anxiety individuals demonstrated preferences for social engagement from healthy-looking faces, but not from unhealthy-looking faces, high-anxiety individuals preferred social engagement from both healthy and unhealthy individuals. Importantly, anxious individuals were not simply less discriminating in their face preferences generally: anxiety levels were positively related to the strength of overall preferences for positive social interest. Collectively, our findings show that perceptions of gaze and expression can be modulated by aspects of facial appearance and that systematic variation among individuals exists in the extent to which this modulation occurs. Furthermore, since previous studies have demonstrated hypersensitive amygdala responses among anxious individuals when viewing faces, our findings suggest that the amygdala may play an important role in determining how different facial cues are integrated in person perception.     

Positive and negative gaze perception in autism spectrum conditions

Abstract

Eyes are key social features providing a wealth of information about the attention, interest, emotion, and intention of others. Humans are typically very adept at detecting gaze direction, but there is a large decrement in gaze discrimination ability when eye images change from positive to negative polarity. This is thought to show an expert system for gaze perception that applies a contrast-specific heuristic to determine where someone else is looking. Autism spectrum conditions (ASC) are characterized by social deficits including difficulties in face-processing and in the social use of gaze. People with ASC are thought to have less expertise for gaze processing compared to typical controls, though little research has tested this. We investigated gaze direction perception in typical males and females, and males with ASC using facial stimuli with positive or negative polarity of the eyes. Results showed that the ASC group was worse at judging gaze direction with positive stimuli, and showed less of a decrement in performance when eye stimuli changed from positive to negative polarity. The differences in gaze perception for the ASC group were most evident when information from the eyes was more difficult and ambiguous. Typical females performed better at gaze direction detection with positive polarity than typical males, who in turn performed better than males with ASC. This latter finding is consistent with the extreme male brain theory of autism, and with the idea that people with ASC have less gaze expertise.     

The NIMH Child Emotional Faces Picture Set (NIMH-ChEFS): a new set of children's facial emotion stimuli

With the emergence of new technologies, there has been an explosion of basic and clinical research on the affective and cognitive neuroscience of face processing and emotion perception. Adult emotional face stimuli are commonly used in these studies. For developmental research, there is a need for a validated set of child emotional faces. This paper describes the development of the National Institute of Mental Health Child Emotional Faces Picture Set (NIMH-ChEFS), a relatively large stimulus set with high quality, color images of the emotional faces of children. The set includes 482 photographs of fearful, angry, happy, sad and neutral child faces with two gaze conditions: direct and averted gaze. In this paper we describe the development of the NIMH-ChEFS and data on the set's validity based on ratings by 20 healthy adult raters. Agreement between the a priori emotion designation and the raters' labels was high and comparable with values reported for commonly used adult picture sets. Intensity, representativeness, and composite "goodness" ratings are also presented to guide researchers in their choice of specific stimuli for their studies. These data should give researchers confidence in the NIMH-ChEFS's validity for use in affective and social neuroscience research.