N170 or N1? Spatiotemporal differences between object and face processing using ERPs

The ERP component N170 is face-sensitive, yet its specificity for faces is controversial. We recorded ERPs while subjects viewed upright and inverted faces and seven object categories. Peak, topography and segmentation analyses were performed. N170 was earlier and larger to faces than to all objects. The classic increase in amplitude and latency was found for inverted faces on N170 but also on P1. Segmentation analyses revealed an extra map found only for faces, reflecting an extra cluster of activity compared to objects. While the N1 for objects seems to reflect the return to baseline from the P1, the N170 for faces reflects a supplement activity. The electrophysiological 'specificity' of faces could lie in the involvement of extra generators for face processing compared to objects and the N170 for faces seems qualitatively different from the N1 for objects. Object and face processing also differed as early as 120 ms.     

Rapid face-selective adaptation of an early extrastriate component in MEG

Adaptation paradigms are becoming increasingly popular for characterizing visual areas in neuroimaging, but the relation of these results to perception is unclear. Neurophysiological studies have generally reported effects of stimulus repetition starting at 250-300 ms after stimulus onset, well beyond the latencies of components associated with perception (100-200 ms). Here we demonstrate adaptation for earlier evoked components when 2 stimuli (S1 and S2) are presented in close succession. Using magnetoencephalography, we examined the M170, a "face-selective" response at 170 ms after stimulus onset that shows a larger response to faces than to other stimuli. Adaptation of the M170 occurred only when stimuli were presented with relatively short stimulus onset asynchronies (< 800 ms) and was larger for faces preceded by faces than by houses. This face-selective adaptation is not merely low-level habituation to physical stimulus attributes, as photographic, line-drawing, and 2-tone face images produced similar levels of adaptation. Nor does it depend on the amplitude of the S1 response: adaptation remained greater for faces than houses even when the amplitude of the S1 face response was reduced by visual noise. These results indicate that rapid adaptation of early, short-latency responses not only exists but also can be category selective.     

Configuration‐Based Processing of Phosphene Pattern Recognition for Simulated Prosthetic Vision

Visual prosthesis can elicit phosphenes by stimulating the retina, optic nerve, or visual cortex along the visual pathway. Psychophysical studies have demonstrated that visual function can be partly recovered with phosphene‐based prosthetic vision. This study investigated the cognitive process of prosthetic vision through a face recognition task. Both behavioral response and the face‐specific N170 component of event‐related potential were analyzed in the presence of face and non‐face stimuli with natural and simulated prosthetic vision. Our results showed that: (i) the accuracy of phosphene face recognition was comparable with that of the normal one when phosphene grid increased to 25 × 21 or more; (ii) shorter response time was needed for phosphene face recognition; and (iii) the N170 component was delayed and enhanced under phosphene stimuli. It was suggested that recognition of phosphene patterns employ a configuration‐based holistic processing mechanism with a distinct substage unspecific to faces.     

Human Extrastriate Visual Cortex and the Perception of Faces, Words, Numbers, and Colors

Electrophysiological correlates of the processing of visualinformation were studied in epileptic patients with electrodeschronically implanted on the surface of striate and extrastriatecortex. In separate experiments patients viewed faces, letterstrings (words and non-words), numbers, and control stimuli.A negative potential, N200, was evoked by faces, letter strings,and numbers, but not by the control stimuli. N200 was recordedbilaterally from discrete regions of the fusiform and inferiortemporal gyri. These category-specific face, letter-string,and number "modules" vary in location. In most cases there wasno overlap in the location of face and letter-string modules,suggesting a mosaic of functionally discrete regions. In somecases letter-string and number N200s were recorded from thesame location, suggesting that these modules may be less spatiallyand functionally discrete. Face N200-like potentials can berecorded from temporal scalp, allowing the possibility of studyingearly face processing in normal subjects. Longer-latency face-specificpotentials were recorded from the inferior surface of the anteriortemporal lobe. Potentials evoked by colored checkerboards wererecorded from a region of the fusiform gyrus posterior to thefusiform region from which category-specific N200s were recorded. These results suggest that there are several processing streamsin inferior extrastriate cortex. In addition to object recognitionsystems previously proposed for faces and words, our preliminaryresults suggest a separate system dealing with numbers. Postulatedsystems dealing with larger manipulable objects and animalshave not been detected.     

Rapid adaptation of the m170 response: importance of face parts

Face perception is often characterized as depending on configural, rather than part-based, processing. Here we examined the relative contributions of configuration and parts to early "face-selective" processing at the M170, a magnetoencephalographic response approximately 170 ms after stimulus onset, using adaptation. Previously (Harris and Nakayama 2007), we showed that rapid successive presentation of 2 stimuli (stimulus-onset asynchrony < 800 ms) attenuates the M170 response. Such adaptation is face-selective, with greater attenuation when faces are preceded by other faces than by houses. This technique therefore provides an independent method to assess the nature of this early neurophysiological marker. In these experiments, we measured the adapting power of face configurations versus parts using upright and inverted faces (Experiment 1), face-like configurations of black ovals versus scrambled nonface configurations of face parts (Experiment 2), and isolated face parts (Experiment 3). Although face configurations alone do not produce face-selective adaptation, scrambled and even isolated face parts adapt the M170 response to a similar extent as full faces. Thus, at least for the relatively early face-selective M170 response, face parts produce face-selective adaptation but face configurations do not. These results suggest that face parts are important at relatively early stages of face perception.     

The M170 reflects a viewpoint-dependent representation for both familiar and unfamiliar faces

The aim of this study was to determine the extent to which the neural representation of faces in visual cortex is viewpoint dependent or viewpoint invariant. Magnetoencephalography was used to measure evoked responses to faces during an adaptation paradigm. Using familiar and unfamiliar faces, we compared the amplitude of the M170 response to repeated images of the same face with images of different faces. We found a reduction in the M170 amplitude to repeated presentations of the same face image compared with images of different faces when shown from the same viewpoint. To establish if this adaptation to the identity of a face was invariant to changes in viewpoint, we varied the viewing angle of the face within a block. We found a reduction in response was no longer evident when images of the same face were shown from different viewpoints. This viewpoint-dependent pattern of results was the same for both familiar and unfamiliar faces. These results imply that either the face-selective M170 response reflects an early stage of face processing or that the computations underlying face recognition depend on a viewpoint-dependent neuronal representation.     

"What" precedes "which": developmental neural tuning in face- and place-related cortex

Although category-specific activation for faces in the ventral visual pathway appears adult-like in adolescence, recognition abilities for individual faces are still immature. We investigated how the ability to represent "individual" faces and houses develops at the neural level. Category-selective regions of interest (ROIs) for faces in the fusiform gyrus (FG) and for places in the parahippocampal place area (PPA) were identified individually in children, adolescents, and adults. Then, using an functional magnetic resonance imaging adaptation paradigm, we measured category selectivity and individual-level adaptation for faces and houses in each ROI. Only adults exhibited both category selectivity and individual-level adaptation bilaterally for faces in the FG and for houses in the PPA. Adolescents showed category selectivity bilaterally for faces in the FG and houses in the PPA. Despite this profile of category selectivity, adolescents only exhibited individual-level adaptation for houses bilaterally in the PPA and for faces in the "left" FG. Children only showed category-selective responses for houses in the PPA, and they failed to exhibit category-selective responses for faces in the FG and individual-level adaptation effects anywhere in the brain. These results indicate that category-level neural tuning develops prior to individual-level neural tuning and that face-related cortex is disproportionately slower in this developmental transition than is place-related cortex.     

Unintentional temporal context-based prediction of emotional faces: an electrophysiological study

The ability to extract sequential regularities embedded in the temporal context or temporal structure of sensory events and to predict upcoming events based on the extracted sequential regularities plays a central role in human cognition. In the present study, we demonstrate that, without any intention, upcoming emotional faces can be predicted based on sequential regularities, by showing that prediction error responses as reflected by visual mismatch negativity (MMN), an event-related brain potential (ERP) component, were evoked in response to emotional faces that violated a regular alternation pattern of 2 emotional faces (fearful and happy faces) under a situation where the emotional faces themselves were unrelated to the participant's task. Face-inversion and negative-bias effects in the visual MMN further indicated the involvement of holistic face representations. In addition, through successive source analyses of the visual MMN, it was revealed that the prediction error responses were composed of activations mainly in the face-responsible visual extrastriate areas and the prefrontal areas. The present results provide primary evidence for the existence of the unintentional temporal context-based prediction of biologically relevant visual stimuli as well as empirical support for the major engagement of the visual and prefrontal areas in unintentional temporal context-based prediction in vision.     

Consequences of magnocellular dysfunction on processing attended information in schizophrenia

Schizophrenia is associated with perceptual and cognitive dysfunction including impairments in visual attention. These impairments may be related to deficits in early stages of sensory/perceptual processing, particularly within the magnocellular/dorsal visual pathway. In the present study, subjects viewed high and low spatial frequency (SF) gratings designed to test functioning of the parvocellular/magnocellular pathways, respectively. Schizophrenia patients and healthy controls attended to either the low SF (magnocellularly biased) or high SF (parvocellularly biased) gratings. Functional magnetic resonance imaging (fMRI) and recordings of event-related potentials (ERPs) were carried out during task performance. Patients were impaired at detecting low-frequency targets. ERP amplitudes to low-frequency gratings were diminished, both for the early sensory-evoked components and for the attend minus unattend difference component (the selection negativity), which is regarded as a neural index of feature-selective attention. Similarly, fMRI revealed that activity in extrastriate visual cortex was reduced in patients during attention to low, but not high, SF. In contrast, activity in frontal and parietal areas, previously implicated in the control of attention, did not differ between patients and controls. These findings suggest that impaired sensory processing of magnocellularly biased stimuli lead to impairments in the effective processing of attended stimuli, even when the attention control systems themselves are intact.     

Category-specific conceptual processing of color and form in left fronto-temporal cortex

To investigate the cortical basis of color and form concepts, we examined event-related functional magnetic resonance imaging (fMRI) responses to matched words related to abstract color and form information. Silent word reading elicited activity in left temporal and frontal cortex, where category-specific activity differences were also observed. Whereas color words preferentially activated anterior parahippocampal gyrus, form words evoked category-specific activity in fusiform and middle temporal gyrus as well as premotor and dorsolateral prefrontal areas in inferior and middle frontal gyri. These results demonstrate that word meanings and concepts are not processed by a unique cortical area, but by different sets of areas, each of which may contribute differentially to conceptual semantic processing. We hypothesize that the anterior parahippocampal activation to color words indexes computation of the visual feature conjunctions and disjunctions necessary for classifying visual stimuli under a color concept. The predominant premotor and prefrontal activation to form words suggests action-related information processing and may reflect the involvement of neuronal elements responding in an either-or fashion to mirror neurons related to adumbrating shapes.     

Population dynamics of face-responsive neurons in the inferior temporal cortex

Neurons in the inferior temporal (IT) cortex of monkeys respond selectively to complex visual stimuli, such as faces. Single neurons in the IT cortex encode different kinds of information about visual stimuli in their temporal firing patterns. To understand the temporal aspects of the information encoded at a population level in the IT cortex, we applied principal component analysis (PCA) to the responses of a population of neurons. The responses of each neuron were recorded while visual stimuli that consisted of geometric shapes and faces of humans and monkeys were presented. We found that global categorization, i.e. human faces versus monkey faces versus shapes, occurred in the earlier part of the population response, and that fine categorization occurred within each member of the global category in the later part of the population response. A cluster analysis, a mixture of Gaussians analysis, confirmed that the clusters in the earlier part of the responses represented the global category. Moreover, the clusters in the earlier part separated into sub-clusters corresponding to either human identity or monkey expression in the later part of the responses, and the global categorization was maintained even after the appearance of the sub-clusters. The results suggest that a hierarchical relationship of the test stimuli is represented temporally by the population response of IT neurons.     

Phosphene Object Perception Employs Holistic Processing During Early Visual Processing Stage

Psychophysical studies have verified the possibility of recovering the visual ability by the form of low‐resolution format of images, that is, phosphene‐based representations. Our previous study has found that early visual processing for phosphene patterns is configuration based. This study further investigated the configural processing mechanisms of prosthetic vision by analyzing the event‐related potential components (P1 and N170) in response to phosphene face and non‐face stimuli. The results reveal that the coarse processing of phosphenes involves phosphene‐specific holistic processing that recovers separated phosphenes into a gestalt; low‐level feature processing of phosphenes is also enhanced compared with that of normal stimuli due to increased contrast borders introduced by phosphenes; while fine processing of phosphene stimuli is impaired reflected by reduced N170 amplitude because of the degraded detailed features in the low‐resolution format representations. Therefore, we suggest that strategies that can facilitate the specific holistic processing stages of prosthetic vision should be considered in order to improve the performance when designing the visual prosthesis system.     

The functional and temporal characteristics of top-down modulation in visual selection

Perceptual load of an attended task influences the processing of irrelevant background stimuli. In a series of behavioral, functional magnetic resonance (fMRI) and electroencephalography (EEG) experiments we examined the influence of working memory (WM) load related to a relevant visual stimulus on the processing of irrelevant backgrounds. We further addressed two open questions about the mechanism of load-dependent modulation: (i) is this modulation dependent on regional activity (i.e. phasic)? (ii) At what processing stage does this modulation take place? Load was manipulated by a WM task and concurrently the processing of irrelevant visual objects was assessed with fMRI and EEG. To examine the dependency of this modulation on intrinsic activity, we varied the activity level of visual areas by presenting objects with different levels of degradation. Activity in the lateral occipital complex (LOC) increased with object visibility and was phasically modulated by WM load. Event related potentials revealed that this phasic modulation occurred approximately 170 ms after stimulus onset, indicative of an early selection under high load. The results indicate a phasic modulatory effect of WM load on visual object processing in the LOC that is comparable to the effects found for perceptual load manipulations.     

Representation of the perceived 3-D object shape in the human lateral occipital complex

We used human functional magnetic resonance imaging (fMRI) to test whether the human lateral occipital complex (LOC), an area known to be involved in the analysis of visual shape, represents the perceived 3-D shape of objects or simply their 2-D contours. We employed an fMRI adaptation paradigm, in which repeated presentation of a stimulus results in decreased responses compared to responses to different stimuli. We found adaptation in the LOC for images of objects with the same perceived 3-D shape structure but different 2-D contours that resulted from small rotations of the objects in the frontal plane or in depth. However, no adaptation was observed in the LOC for images of objects that had the same 2-D contours but differed in their perceived 3-D shape; namely, 2-D silhouettes versus 3-D shaded images of objects, or convex versus concave objects. Differences in the fMRI adaptation responses across subregions in the LOC suggest that different neural populations in the LOC may mediate different mechanisms for the processing of object features.     

Electrophysiological evidence for temporal dissociation between spatial attention and sensory competition during human face processing

Scalp event-related potential (ERP) studies in humans indicate that face processes taking place between 130 and 170 ms after stimulus onset at posterior sites (N170) are strongly reduced when another face stimulus is processed concurrently or has been presented shortly before for a prolonged period. These observations suggest that neural representations of individual faces compete in the occipitotemporal cortex as early as 130 ms. Here, we tested the respective role of spatial attention and sensory competition in accounting for the amplitude reduction of the N170 during concurrent face stimulation. ERPs time locked to a lateralized face stimulus were recorded while subjects were fixating either a face or a controlled scrambled-face stimulus (context factor) and were engaged in either a high- or a low-attentional load task at fixation (task factor). The N170 amplitude to the lateralized face stimulus was reduced both when the central stimulus was a face compared with a scrambled face and when the attentional load at fixation was high. However, these effects of context and task factors were largely additive. Most importantly, spatial attention modulated visual processes as early as 80 ms after stimulus onset, whereas sensory competition effects started at about 130 ms. These results provide strong evidence that the N170 in response to faces is modulated by spatial attention, and also that spatial attention and sensory competition do not reflect the same mechanisms of early selection of visual information in the extrastriate cortex.     

Revisiting the role of the fusiform face area in visual expertise

It has previously been reported (Gauthier et al., 2000, Nat. Neurosci., 3:191-197) in a functional magnetic resonance imaging (fMRI) study that objects of visual expertise (cars and birds) activate the right fusiform face area (FFA) more strongly than non-expertise stimuli, and it was argued that the right FFA is involved in expertise specific rather than face specific visual processing. This expertise effect, however, may be due to experts taking advantage of the 'faceness' of the stimuli: birds have faces and three-quarter frontal views of cars resemble faces. This expertise effect may also be caused by a biased attentional modulation: with a blocked fMRI design, experts may attend more to a block of expertise than a block of non-expertise stimuli. In this study, using both side-view car images that do not resemble faces and bird images in an event-related fMRI design that minimizes attentional modulation, an expertise effect in the right FFA is observed in both car and bird experts (although a baseline bias makes the bird expertise effect less reliable). These results are consistent with those of Gauthier et al., and suggest the involvement of the right FFA in processing non-face expertise visual stimuli.     

Effects of long-term object familiarity on event-related potentials in the monkey

Although some change in the neural representation of an object must occur as it becomes familiar, the nature of this change is not fully understood. In humans, it has been shown that the N170-an evoked visual potential-is enhanced for classes of objects for which people have visual expertise. In this study, we explored whether monkeys show a similar modulation in event-related potential (ERP) amplitude as a result of long-term familiarity by recording ERPs with chronically implanted electrodes over extended training periods spanning many sessions. In each of 3 experiments, we found larger amplitude visual evoked responses to highly familiar images for the time period of 120-250 ms after stimulus onset. This difference was found when the monkeys were trained in an individual-level discrimination task, in a task that required only color discrimination, and even following a viewing-only task. We thus observed this familiarity effect across several tasks and different object categories and further found that the difference between "familiar" and "novel" became smaller as the animals gained experience with the previously unfamiliar objects across multiple test sessions. These data suggest that changes in visual responses associated with familiarity are evident early in the evoked visual response, are robust, and may be automatic, driven at least in part by repeated object exposure.     

Context influences early perceptual analysis of faces--an electrophysiological study

Electrophysiological and hemodynamic correlates of processing isolated faces have been investigated extensively over the last decade. A question not addressed thus far is whether the visual scene, which normally surrounds a face or a facial expression, has an influence on how the face is processed. Here we investigated this issue by presenting faces in natural contexts and measuring whether the emotional content of the scene influences processing of a facial expression. Event-related potentials were recorded to faces (fearful/neutral) embedded in scene contexts (fearful/neutral) while participants performed an orientation-decision task (face upright or inverted). Two additional experiments were run, one to examine the effects of context that occur without a face and the other to evaluate the effects of faces isolated from contexts. Faces without any context showed the largest N170 amplitudes. The presence of a face in a fearful context enhances the N170 amplitude over a face in neutral contexts, an effect that is strongest for fearful faces on left occipito-temporal sites. This N170 effect, and the corresponding topographic distribution, was not found for contexts-only, indicating that the increased N170 amplitude results from the combination of face and fearful context. These findings suggest that the context in which a face appears may influence how it is encoded.     

Large-scale visuomotor integration in the cerebral cortex

Efficient visuomotor behavior depends on integrated processing by the visual and motor systems of the cerebral cortex. Yet, many previous cortical neurophysiology studies have examined the visual and motor modalities in isolation, largely ignoring questions of large-scale cross-modal integration. To address this issue, we analyzed event-related local field potentials simultaneously recorded from multiple visual, motor, and executive cortical sites in monkeys performing a visuomotor pattern discrimination task. The timing and cortical location of four aspects of event-related activities were examined: stimulus-evoked activation onset, stimulus-specific processing, stimulus category-specific processing, and response-specific processing. Activations appeared earliest in striate cortex and rapidly thereafter in other visual areas. Stimulus-specific processing began early in most visual cortical areas, some at activation onset. Early onset latencies were also observed in motor, premotor, and prefrontal areas, some as early as in striate cortex, but these early-activating frontal sites did not show early stimulus-specific processing. Response-specific processing began around 150 ms poststimulus in widespread cortical areas, suggesting that perceptual decision formation and response selection arose through concurrent processes of visual, motor, and executive areas. The occurrence of stimulus-specific and stimulus category-specific differences after the onset of response-specific processing suggests that sensory and motor stages of visuomotor processing overlapped in time.     

Six years of follow-up after bilateral hand replantation

Replantation is the gold standard surgical treatment of amputations of the upper limb; however, this demanding procedure is not always preformed in bilateral limb amputation. The objective of this study was to analyze, six years after surgery, the sensorimotor recovery of both replanted hands. A 21-year-old patient with bilateral hand amputation was benefited from limb replantation. Surgery included debridement of the amputated hands and recipient's stumps, bone fixation, arterial and venous anastomoses, nerve sutures, tendon sutures and skin closure. Rehabilitation program included physiotherapy, electrostimulation and occupational therapy. Sensory and motor evaluation was performed 6 years after replantation. At 6 years, the patient presented a good/satisfactory recovery of range of motion and strength, better at right hand. The patient was able to perform right thumb opposition. Static two-point discrimination was 20 mm, sensitivity to pain and thermal stimuli and ability to sweat were present on both hands. Reinnervation was confirmed by electromyography. Functional recovery was higher at the right hand when compared to the left hand. He was highly satisfied with the result of surgery for right hand and fairly satisfied with the result for left hand. He was able to return to a secretary work 16 months after the accident. A proper functional result can be accomplished with bilateral hand replantation.