Rapid face-selective adaptation of an early extrastriate component in MEG

Adaptation paradigms are becoming increasingly popular for characterizing visual areas in neuroimaging, but the relation of these results to perception is unclear. Neurophysiological studies have generally reported effects of stimulus repetition starting at 250-300 ms after stimulus onset, well beyond the latencies of components associated with perception (100-200 ms). Here we demonstrate adaptation for earlier evoked components when 2 stimuli (S1 and S2) are presented in close succession. Using magnetoencephalography, we examined the M170, a "face-selective" response at 170 ms after stimulus onset that shows a larger response to faces than to other stimuli. Adaptation of the M170 occurred only when stimuli were presented with relatively short stimulus onset asynchronies (< 800 ms) and was larger for faces preceded by faces than by houses. This face-selective adaptation is not merely low-level habituation to physical stimulus attributes, as photographic, line-drawing, and 2-tone face images produced similar levels of adaptation. Nor does it depend on the amplitude of the S1 response: adaptation remained greater for faces than houses even when the amplitude of the S1 face response was reduced by visual noise. These results indicate that rapid adaptation of early, short-latency responses not only exists but also can be category selective.     

Effects of long-term object familiarity on event-related potentials in the monkey

Although some change in the neural representation of an object must occur as it becomes familiar, the nature of this change is not fully understood. In humans, it has been shown that the N170-an evoked visual potential-is enhanced for classes of objects for which people have visual expertise. In this study, we explored whether monkeys show a similar modulation in event-related potential (ERP) amplitude as a result of long-term familiarity by recording ERPs with chronically implanted electrodes over extended training periods spanning many sessions. In each of 3 experiments, we found larger amplitude visual evoked responses to highly familiar images for the time period of 120-250 ms after stimulus onset. This difference was found when the monkeys were trained in an individual-level discrimination task, in a task that required only color discrimination, and even following a viewing-only task. We thus observed this familiarity effect across several tasks and different object categories and further found that the difference between "familiar" and "novel" became smaller as the animals gained experience with the previously unfamiliar objects across multiple test sessions. These data suggest that changes in visual responses associated with familiarity are evident early in the evoked visual response, are robust, and may be automatic, driven at least in part by repeated object exposure.     

Rapid adaptation of the m170 response: importance of face parts

Face perception is often characterized as depending on configural, rather than part-based, processing. Here we examined the relative contributions of configuration and parts to early "face-selective" processing at the M170, a magnetoencephalographic response approximately 170 ms after stimulus onset, using adaptation. Previously (Harris and Nakayama 2007), we showed that rapid successive presentation of 2 stimuli (stimulus-onset asynchrony < 800 ms) attenuates the M170 response. Such adaptation is face-selective, with greater attenuation when faces are preceded by other faces than by houses. This technique therefore provides an independent method to assess the nature of this early neurophysiological marker. In these experiments, we measured the adapting power of face configurations versus parts using upright and inverted faces (Experiment 1), face-like configurations of black ovals versus scrambled nonface configurations of face parts (Experiment 2), and isolated face parts (Experiment 3). Although face configurations alone do not produce face-selective adaptation, scrambled and even isolated face parts adapt the M170 response to a similar extent as full faces. Thus, at least for the relatively early face-selective M170 response, face parts produce face-selective adaptation but face configurations do not. These results suggest that face parts are important at relatively early stages of face perception.     

Face configuration processing in the human brain: the role of symmetry

Symmetry is an important cue in face perception. We manipulated symmetry and other configurational variables to study their role in face processing in the human brain. We employed 2 types of symmetry: image symmetry (where one part of the image is defined as the mirrored transform of the other part about an axis) and object symmetry (where the spatial relationships among the image components are interpreted as parts of a symmetric 3-dimensional object). We compared blood oxygenation level dependent responses in healthy human observers for upright front-view faces with responses to different symmetry-controlled images. The cortical areas activated by the face images, relative to Fourier-matched scrambled images, were the fusiform (FFA) and occipital (OFA) face areas, the middle occipital gyri (MOG), and areas around the superior temporal and intraoccipital sulci (IOS). Contrasting faces and their image-symmetric scrambled versions showed a similar activation pattern except in the right OFA, suggesting an involvement in facial symmetry processing. The upright versus inverted faces (with the same image symmetry but unfamiliar object identity) showed robust differential activation in the FFA, OFA, MOG, IOS, and precuneus. The response to frontal-view versus 3/4-view faces (having the same object symmetry but disrupted image symmetry) showed little differential activation in the FFA or the OFA but strong responses in the MOG and IOS, suggesting that face processing in the FFA and the OFA is holistic and viewpoint invariant.     

Electrophysiological correlates of visual adaptation to faces and body parts in humans

The existence of facial aftereffects suggests that shape-selective mechanisms at the higher stages of visual object coding -- similarly to the early processing of low-level visual features -- are adaptively recalibrated. Our goal was to uncover the ERP correlates of shape-selective adaptation and to test whether it is also involved in the visual processing of human body parts. We found that prolonged adaptation to female hands -- similarly to adaptation to female faces -- biased the judgements about the subsequently presented hand test stimuli: they were perceived more masculine than in the control conditions. We also showed that these hand aftereffects are size and orientation invariant. However, no aftereffects were found when the adaptor and test stimuli belonged to different categories (i.e. face adaptor and hand test, or vice versa), suggesting that the underlying adaptation mechanisms are category-specific. In accordance with the behavioral results, both adaptation to faces and hands resulted in a strong and category-specific modulation -- reduced amplitude and increased latency -- of the N170 component of ERP responses. Our findings suggest that shape-selective adaptation is a general mechanism of visual object processing and its neural effects are primarily reflected in the N170 component of the ERP responses.     

Electrophysiological evidence for temporal dissociation between spatial attention and sensory competition during human face processing

Scalp event-related potential (ERP) studies in humans indicate that face processes taking place between 130 and 170 ms after stimulus onset at posterior sites (N170) are strongly reduced when another face stimulus is processed concurrently or has been presented shortly before for a prolonged period. These observations suggest that neural representations of individual faces compete in the occipitotemporal cortex as early as 130 ms. Here, we tested the respective role of spatial attention and sensory competition in accounting for the amplitude reduction of the N170 during concurrent face stimulation. ERPs time locked to a lateralized face stimulus were recorded while subjects were fixating either a face or a controlled scrambled-face stimulus (context factor) and were engaged in either a high- or a low-attentional load task at fixation (task factor). The N170 amplitude to the lateralized face stimulus was reduced both when the central stimulus was a face compared with a scrambled face and when the attentional load at fixation was high. However, these effects of context and task factors were largely additive. Most importantly, spatial attention modulated visual processes as early as 80 ms after stimulus onset, whereas sensory competition effects started at about 130 ms. These results provide strong evidence that the N170 in response to faces is modulated by spatial attention, and also that spatial attention and sensory competition do not reflect the same mechanisms of early selection of visual information in the extrastriate cortex.     

Face recognition and cortical responses show similar sensitivity to noise spatial frequency

To find cortical correlates of face recognition, we manipulatedthe recognizability of face images in a parametric manner bymasking them with narrow-band spatial noise. Face recognitionperformance was best at the lowest and highest noise spatialfrequencies (NSFs, 2 and 45 c/image, respectively), and degradedgradually towards central NSFs (11–16 c/image). The strengthof the 130–180 ms neuromagnetic response (M170) in thetemporo-occipital cortex paralleled the recognition performance,whereas the mid-occipital response at 70–120 ms actedin the opposite manner, being strongest for the central NSFs.To noise stimuli without faces, M170 was small and rather insensitiveto NSF, whereas the mid-occipital responses resembled closelythe responses to the combined face and noise stimuli. Theseresults suggest that the 100 ms mid-occipital response is sensitiveto the central spatial frequencies that are critical for facerecognition, whereas the M170 response is sensitive to the visibilityof a face and closely related to face recognition.     

Dissociated neural mechanisms for face detection and configural encoding: evidence from N170 and induced gamma-band oscillation effects

Despite ample research, the structure and the functional characteristics of neural systems involved in human face processing are still a matter of active debate. Here we dissociated between a neural mechanism manifested by the face-sensitive N170 event-related potential effect and a mechanism manifested by induced electroencephalographic oscillations in the gamma band, which have been previously associated with the integration of individually coded features and activation of corresponding neural representations. The amplitude of the N170 was larger in the absence of the face contour but not affected by the configuration of inner components (ICs). Its latency was delayed by scrambling the configuration of the components as well as by the absence of the face contour. Unlike the N170, the amplitude of the induced gamma activity was sensitive to the configuration of ICs but insensitive to their presence within or outside a face contour. This pattern suggests a dual mechanism for early face processing, each utilizing different visual cues, which might indicate their respective roles in face processing. The N170 seems to be associated primarily with the detection and categorization of faces, whereas the gamma oscillations may be involved in the activation of their mental representation.     

How Are Three-Dimensional Objects Represented in the Brain?

In this report we discuss a variety of psychophysical experimentsthat explore different aspects of the problem of object recognitionand representation in human vision. In all experiments, subjectswere presented with realistically rendered images of computer-generated3D objects, with tight control over stimulus shape, surfaceproperties, illumination, and viewpoint, as well as subjects'prior exposure to the stimulus objects. Contrary to the predictionsof the paradigmatic theory of recognition, which holds thatobject representations are viewpoint invariant performance inall experiments was consistently viewpoint dependent, was onlypartially aided by binocular stereo and other depth information,was specific to viewpoints that were familiar, and was systematicallydisrupted by rotation in depth more than by deforming the 2Dimages of the stimuli. The emerging concept of multiple-viewsrepresentation supported by these results is consistent withrecently advanced computational theories of recognition basedon view interpolation. Moreover, in several simulated experimentsemploying the same stimuli used in experiments with human subjects,models based on multiple-views representations replicated manyof the psychophysical results concerning the observed patternof human performance.     

Sub-exemplar shape tuning in human face-related areas

Although human face recognition performance shows high selectivity, even for unfamiliar faces, the neuronal circuitry underlying this high performance is poorly understood. Two extreme alternatives can be considered: either a "labeled-line" principle, in which subtle changes in face images lead to activation of differently tuned neuronal populations, or a coarse coding principle, where the high face selectivity is coded by the relative activation of broadly tuned neurons. In this study, we set to parametrically examine the shape and selectivity profile of face-related visual areas. To that end, we applied the functional magnetic resonance (fMR)-adaptation paradigm. Unfamiliar face stimuli were morphed into sets ranging from identical faces, through subtle morphing, to completely different exemplars. The fusiform face area (FFA) revealed high face sensitivity, so that even facial images perceived as belonging to the same individual (<35%) were sufficient to produce full recovery from adaptation. Interestingly, the psychophysical detectability of facial differences paralleled the release from fMR-adaptation. These results support the labeled-line model where high sensitivity to face changes is paralleled by narrow tuning of neuronal populations selective to each face image, and they suggest that fMR-adaptation is closely related to behavior. The results bear strong implications to the nature of face-related neuronal responses.     

Duration-dependent FMRI adaptation and distributed viewer-centered face representation in human visual cortex

Two functional magnetic resonance imaging (fMRI) face viewpoint adaptation experiments were conducted to investigate whether fMRI adaptation in high-level visual cortex depends on the duration of adaptation and how different views of a face are represented in the human visual system. We found adaptation effects in multiple face-selective areas, which suggest a distributed, viewer-centered representation of faces in the human visual system. However, the nature of the adaptation effects was dependent on the length of adaptation. With long adaptation durations, face-selective areas along the hierarchy of the visual system gradually exhibited viewpoint-tuned adaptation. As the angular difference between the adapter and test stimulus increased, the blood oxygen level-dependent (BOLD) signal evoked by the test stimulus gradually increased as a function of the amount of 3-dimensional (3D) rotation. With short adaptation durations, however, face-selective areas in the ventral pathway, including the lateral occipital cortex and right fusiform area, exhibited viewpoint-sensitive adaptation. These areas showed an increase in the BOLD signal with a 3D rotation, but this signal increase was independent of the amount of rotation. Further, the right superior temporal sulcus showed little or very weak viewpoint adaptation with short adaptation durations. Our findings suggest that long- and short-term fMRI adaptations may reflect selective properties of different neuronal mechanisms.     

Varieties of functional deficits in prosopagnosia.

Prosopagnosia is a neurologically based deficit characterized by the inability to recognize faces of known individuals in the absence of severe intellectual, perceptual, and memory impairments. The nature of the underlying disturbance was investigated in three patients in an attempt to identify the structural and functional levels at which the processing of faces breaks down, the relation between prosopagnosia and associated deficits, and the specificity of the prosopagnosic disturbance. The breakdown of face processing resulted from unilateral damage in different cerebral structures of the right hemisphere in the three patients, and it involved different functional levels of face processing, but all three patients displayed perceptual impairments of unequal severity. In one patient (R.M.), the deficit encompassed all perceptual operations on faces, including matching identical views of the same faces, but it did not extend to all categories of objects characterized by a close similarity among their instances; the second patient (P.M.) exhibited a less severe perceptual impairment but was unable to derive the configurational properties from a facial representation and to extract its physiognomic invariants; the third patient (P.C.) had not lost the capacity to differentiate faces on the basis of their configurations but could not associate a facial representation with its pertinent memories. Associated deficits were present in each patient but differed depending on the anatomofunctional locus of the breakdown, although all patients were impaired at recognizing noncanonical views of objects that they readily recognized when shown from a conventional viewpoint. However, performance dissociation within patients and double dissociation between patients suggest that these associated deficits are not necessary concomitants of prosopagnosia.     

N170 or N1? Spatiotemporal differences between object and face processing using ERPs

The ERP component N170 is face-sensitive, yet its specificity for faces is controversial. We recorded ERPs while subjects viewed upright and inverted faces and seven object categories. Peak, topography and segmentation analyses were performed. N170 was earlier and larger to faces than to all objects. The classic increase in amplitude and latency was found for inverted faces on N170 but also on P1. Segmentation analyses revealed an extra map found only for faces, reflecting an extra cluster of activity compared to objects. While the N1 for objects seems to reflect the return to baseline from the P1, the N170 for faces reflects a supplement activity. The electrophysiological 'specificity' of faces could lie in the involvement of extra generators for face processing compared to objects and the N170 for faces seems qualitatively different from the N1 for objects. Object and face processing also differed as early as 120 ms.     

Context influences early perceptual analysis of faces--an electrophysiological study

Electrophysiological and hemodynamic correlates of processing isolated faces have been investigated extensively over the last decade. A question not addressed thus far is whether the visual scene, which normally surrounds a face or a facial expression, has an influence on how the face is processed. Here we investigated this issue by presenting faces in natural contexts and measuring whether the emotional content of the scene influences processing of a facial expression. Event-related potentials were recorded to faces (fearful/neutral) embedded in scene contexts (fearful/neutral) while participants performed an orientation-decision task (face upright or inverted). Two additional experiments were run, one to examine the effects of context that occur without a face and the other to evaluate the effects of faces isolated from contexts. Faces without any context showed the largest N170 amplitudes. The presence of a face in a fearful context enhances the N170 amplitude over a face in neutral contexts, an effect that is strongest for fearful faces on left occipito-temporal sites. This N170 effect, and the corresponding topographic distribution, was not found for contexts-only, indicating that the increased N170 amplitude results from the combination of face and fearful context. These findings suggest that the context in which a face appears may influence how it is encoded.     

Converging neuronal activity in inferior temporal cortex during the classification of morphed stimuli

How does the brain dynamically convert incoming sensory data into a representation useful for classification? Neurons in inferior temporal (IT) cortex are selective for complex visual stimuli, but their response dynamics during perceptual classification is not well understood. We studied IT dynamics in monkeys performing a classification task. The monkeys were shown visual stimuli that were morphed (interpolated) between pairs of familiar images. Their ability to classify the morphed images depended systematically on the degree of morph. IT neurons were selected that responded more strongly to one of the 2 familiar images (the effective image). The responses tended to peak approximately 120 ms following stimulus onset with an amplitude that depended almost linearly on the degree of morph. The responses then declined, but remained above baseline for several hundred ms. This sustained component remained linearly dependent on morph level for stimuli more similar to the ineffective image but progressively converged to a single response profile, independent of morph level, for stimuli more similar to the effective image. Thus, these neurons represented the dynamic conversion of graded sensory information into a task-relevant classification. Computational models suggest that these dynamics could be produced by attractor states and firing rate adaptation within the population of IT neurons.     

Electrophysiological studies of human face perception. II: Response properties of face-specific potentials generated in occipitotemporal cortex.

In the previous paper the locations and basic response properties of N200 and other face-specific event-related potentials (ERPs) were described. In this paper responsiveness of N200 and related ERPs to the perceptual features of faces and other images was assessed. N200 amplitude did not vary substantially, whether evoked by colored or grayscale faces; normal, blurred or line-drawing faces; or by faces of different sizes. Human hands evoked small N200s at face-specific sites, but evoked hand-specific ERPs at other sites. Cat and dog faces evoked N200s that were 73% as large as to human faces. Hemifield stimulation demonstrated that the right hemisphere is better at processing information about upright faces and transferring it to the left hemisphere, whereas the left hemisphere is better at processing information about inverted faces and transferring it to the right hemisphere. N200 amplitude was largest to full faces and decreased progressively to eyes, face contours, lips and noses viewed in isolation. A region just lateral to face-specific N200 sites was more responsive to internal face parts than to faces, and some sites in ventral occipitotemporal cortex were face-part-specific. Faces with eyes averted or closed evoked larger N200s than those evoked by faces with eyes forward. N200 amplitude and latency were affected by the joint effects of eye and head position in the right but not in the left hemisphere. Full and three-quarter views of faces evoked larger N200s than did profile views. The results are discussed in relation to behavioral studies in humans and single-cell recordings in monkeys.     

Face repetition effects in implicit and explicit memory tests as measured by fMRI.

Recent parallels between neurophysiological and neuroimaging findings suggest that repeated stimulus processing produces decreased responses in brain regions associated with that processing--a 'repetition suppression' effect. In the present study, volunteers performed two tasks on repeated presentation of famous and unfamiliar faces during functional magnetic resonance imaging (fMRI). In the implicit task, they made fame-judgements (regardless of repetition); in the explicit task, they made episodic recognition judgements (regardless of familiarity). Only in the implicit task was repetition suppression observed: for famous faces in a right lateral fusiform region, and for both famous and unfamiliar faces in a left inferior occipital region. Repetition suppression is therefore not an automatic consequence of repeated perceptual processing of stimuli.     

Temporal dynamics of adaptation to natural sounds in the human auditory cortex

We aimed at testing the cortical representation of complex natural sounds within auditory cortex by conducting 2 human magnetoencephalography experiments. To this end, we employed an adaptation paradigm and presented subjects with pairs of complex stimuli, namely, animal vocalizations and spectrally matched noise. In Experiment 1, we presented stimulus pairs of same or different animal vocalizations and same or different noise. Our results suggest a 2-step process of adaptation effects: first, we observed a general item-unspecific reduction of the N1m peak amplitude at 100 ms, followed by an item-specific amplitude reduction of the P2m component at 200 ms after stimulus onset for both animal vocalizations and noise. Multiple dipole source modeling revealed the right lateral Heschl's gyrus and the bilateral superior temporal gyrus as sites of adaptation. In Experiment 2, we tested for cross-adaptation between animal vocalizations and spectrally matched noise sounds, by presenting pairs of an animal vocalization and its corresponding or a different noise sound. We observed cross-adaptation effects for the P2m component within bilateral superior temporal gyrus. Thus, our results suggest selectivity of the evoked magnetic field at 200 ms after stimulus onset in nonprimary auditory cortex for the spectral fine structure of complex sounds rather than their temporal dynamics.     

Configuration‐Based Processing of Phosphene Pattern Recognition for Simulated Prosthetic Vision

Visual prosthesis can elicit phosphenes by stimulating the retina, optic nerve, or visual cortex along the visual pathway. Psychophysical studies have demonstrated that visual function can be partly recovered with phosphene‐based prosthetic vision. This study investigated the cognitive process of prosthetic vision through a face recognition task. Both behavioral response and the face‐specific N170 component of event‐related potential were analyzed in the presence of face and non‐face stimuli with natural and simulated prosthetic vision. Our results showed that: (i) the accuracy of phosphene face recognition was comparable with that of the normal one when phosphene grid increased to 25 × 21 or more; (ii) shorter response time was needed for phosphene face recognition; and (iii) the N170 component was delayed and enhanced under phosphene stimuli. It was suggested that recognition of phosphene patterns employ a configuration‐based holistic processing mechanism with a distinct substage unspecific to faces.     

Effects of familiarity on neural activity in monkey inferior temporal lobe

Long-term familiarity facilitates recognition of visual stimuli. To better understand the neural basis for this effect, we measured the local field potential (LFP) and multiunit spiking activity (MUA) from the inferior temporal (IT) lobe of behaving monkeys in response to novel and familiar images. In general, familiar images evoked larger amplitude LFPs whereas MUA responses were greater for novel images. Familiarity effects were attenuated by image rotations in the picture plane of 45 degrees. Decreasing image contrast led to more pronounced decreases in LFP response magnitude for novel, compared with familiar images, and resulted in more selective MUA response profiles for familiar images. The shape of individual LFP traces could be used for stimulus classification, and classification performance was better for the familiar image category. Recording the visual and auditory evoked LFP at multiple depths showed significant alterations in LFP morphology with distance changes of 2 mm. In summary, IT cortex shows local processing differences for familiar and novel images at a time scale and in a manner consistent with the observed behavioral advantage for classifying familiar images and rapidly detecting novel stimuli.