Dissociating recollection from familiarity: electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect

In recognition memory research, a tension exists between dual-process and single-process models of episodic retrieval. Dual-process models propose that 'familiarity' assessment and the 'recollection' of contextual information are independent processes, while single-process models claim that one common process supports retrieval. Event-related potentials (ERPs) have been used to show dissociations between the mid frontal and the left parietal ERP old/new effects, which have been associated with familiarity and recollection, respectively. While much ERP evidence favours dual-process theory, Yovel and Paller [Yovel, G., Paller, K.A., 2004. The neural basis of the butcher-on-the-bus phenomenon: when a face seems familiar but is not remembered. NeuroImage 21, 789-800] used faces as retrieval cues to demonstrate that posterior old/new effects index both familiarity and recollection, a finding consistent with single-process models. Here we present evidence supporting Yovel and Paller's claim that a posterior old/new effect indexes familiarity for faces, along with a novel finding that recollection is associated with an anterior old/new effect. Importantly, and in contrast to Yovel and Paller, the old/new effects associated with familiarity and recollection were topographically dissociable, consistent with a dual-process view of recognition memory. The neural correlates of familiarity and recollection identified here for faces appear to be different from those typically observed, suggesting that the ERP old/new effects associated with episodic recognition are not the same under all circumstances.     

The FN400 indexes familiarity-based recognition of faces

Separate event-related brain potential (ERP) components have been hypothesized to index familiarity and recollection processes that support recognition memory. A 300- to 500-ms mid-frontal FN400 old/new difference has been related to familiarity, whereas a 500- to 800-ms parietal old/new difference has been related to recollection. Other recent work has cast doubt on the FN400 familiarity hypothesis, especially its application to familiarity-based recognition of conceptually impoverished stimuli such as novel faces. Here we show that FN400 old/new differences can be observed with novel faces, and as predicted by the familiarity hypothesis, these differences are observed regardless of whether or not recognition is accompanied by the recollection of specific details from the study episode. Furthermore, FN400 differentiation between hits and misses is more consistent with an explicit familiarity process than an implicit memory process.     

Investigating the functional interaction between semantic and episodic memory: convergent behavioral and electrophysiological evidence for the role of familiarity

Throughout our lives we acquire general knowledge about the world (semantic memory) while also retaining memories of specific events (episodic memory). Although these two forms of memory have been dissociated on the basis of neuropsychological data, it is clear that they typically function together during normal cognition. The goal of the present study was to investigate this interaction. One influence of semantic memory on episodic retrieval is 'Levels Of Processing'; recognition is enhanced when stimuli are processed in a semantically meaningful way. Studies examining this semantic processing advantage have largely concluded that semantic memory augments episodic retrieval primarily by enhancing recollection. The present study provides strong evidence for an alternative relationship between semantic and episodic memory. We employed a manipulation of the semantic coherence of to-be-remembered information (semantically related vs. unrelated word pairs) during an associative recognition memory test. Results revealed that associative recognition is significantly enhanced for semantically coherent material, and behavioral estimates (using the process dissociation procedure) demonstrated concomitant changes in the contribution of familiarity to retrieval. Neuroimaging data (event-related potentials recorded at test) also revealed a significant increase in familiarity based retrieval. The electrophysiological correlate of familiarity (the mid-frontal ERP old/new effect) was larger for semantically related compared to unrelated word pairs, but no difference was present in the electrophysiological correlate of recollection (the left parietal old/new effect). We conclude that semantic memory and episodic memory do indeed interact in normal functioning, and not only by modulating recollection, but also by enhancing familiarity.     

Examining ERP correlates of recognition memory: evidence of accurate source recognition without recollection

Recollection is typically associated with high recognition confidence and accurate source memory. However, subjects sometimes make accurate source memory judgments even for items that are not confidently recognized, and it is not known whether these responses are based on recollection or some other memory process. In the current study, we measured event related potentials (ERPs) while subjects made item and source memory confidence judgments in order to determine whether recollection supported accurate source recognition responses for items that were not confidently recognized. In line with previous studies, we found that recognition memory was associated with two ERP effects: an early on-setting FN400 effect, and a later parietal old-new effect [late positive component (LPC)], which have been associated with familiarity and recollection, respectively. The FN400 increased gradually with item recognition confidence, whereas the LPC was only observed for highly confident recognition responses. The LPC was also related to source accuracy, but only for items that had received a high confidence item recognition response; accurate source judgments to items that were less confidently recognized did not exhibit the typical ERP correlate of recollection or familiarity, but rather showed a late, broadly distributed negative ERP difference. The results indicate that accurate source judgments of episodic context can occur even when recollection fails.     

Electrophysiological correlates of recollecting faces of known and unknown individuals

We recorded brain potentials from healthy human subjects during a recognition test in order to monitor neural processing associated with face recollection. Subjects first attempted to memorize 40 faces; half were accompanied by a voice simulating that person speaking (e.g., "I'm Jimmy and I was a roadie for the Grateful Dead") and half were presented in silence. In the test phase, subjects attempted to discriminate both types of old faces (i.e., "named" and "unnamed" faces) from new faces. Recognition averaged 87% correct for named faces, 74% correct for unnamed faces, and 91% correct for new faces. Potentials to old faces were more positive than those to new faces from 300 to 600 ms after face onset. For named faces, the old-new ERP difference was observed at anterior and posterior scalp locations. For unnamed faces, the old-new ERP difference was observed only at posterior scalp locations. Results from a prior experiment suggest that these effects do not reflect perceptual priming of faces. The posterior portion of the old-new ERP difference was thus interpreted as a neural correlate of retrieval of visual face information and the anterior portion as an indication of retrieval of person-specific semantic information.     

Magnetoencephalographic correlates of different levels in subjective recognition memory

In our current study we employed whole-head magnetoencephalography (MEG) to identify neurophysiological correlates (event-related fields, ERFs) of different phenomenologies in human recognition memory. Words which had previously been semantically processed were presented along with previously unstudied words. Via button presses, participants provided subjective indices of three forms of memory: confident recognition, familiarity-based recognition, and misclassification of previously presented items as new (no recognition, misses). Behavioral results revealed a clear distinction between confident recognition (shortest reaction times) and familiarity-based recognition and misses, respectively, and physiological data pointed to bilateral anterior and left anterior/central regions in which magnetic field patterns were directly related to word recognition from approximately 300 ms to 500 ms after word onset. In the context of the prevalent dual process controversy on the roles of familiarity and recollection in recognition memory, we first highlight that two operationalizations of recollection need to be differentiated: We argue that a strategic search for a particular contextual feature stands in clear contrast to the fast and incidental availability of some contextual feature and derive experimental and behavioral indicators for either form of recollection. These indicators are used to select from manifold cognitive neuroscientific work on recognition memory in order to further discuss the neurocognitive characteristics of incidental recollection in contrast to other forms of episodic memory.     

Content dependence of the electrophysiological correlates of recollection

The neural correlates of episodic retrieval ('recollection') have been shown to differ according to the content of retrieved episodes. It has been hypothesized that these content-dependent differences reflect the 'reinstatement' of encoding-related processes or representations at the time of recollection. It remains unclear, however, whether these effects directly reflect the recollection of differential episodic content, as would be predicted by the reinstatement hypothesis, or whether they are instead associated with processes that are contingent on successful recollection. To address this issue, the present study employed event-related potentials (ERPs), permitting the investigation of the temporal dynamics of content-dependent neural effects during retrieval, and in particular, their onset with respect to well-established ERP correlates of recollection, such as the left parietal old-new effect. Subjects studied a series of words that were each presented in the context of one of two encoding tasks. One task required the covert generation of a sentence incorporating each word, whereas the other required imagining the object corresponding to each word within a superimposed scenic picture. Memory for the words was subsequently tested with the 'remember/know' procedure. ERPs elicited by recollected words differed according to the prior encoding history of the word, beginning at approximately 300 ms following word onset. These content-dependent ERP differences were maximal over the anterior scalp and, importantly, onset as early as the left parietal old-new effect. The findings demonstrate that content-dependent neural activity during retrieval can occur in a timeframe that is compatible with a direct role in the recollection and representation of episodic information.     

Left prefrontal cortex control of novel occurrences during recollection: a psychopharmacological study using scopolamine and event-related fMRI

Recollection and familiarity represent two processes involved in episodic memory retrieval. We investigated how scopolamine (an antagonist of acetylcholine muscarinic receptors) influenced brain activity during memory retrieval, using a paradigm that separated recollection and familiarity. Eighteen healthy volunteers were recruited in a randomized, placebo-controlled, double-blind design using event-related fMRI. Participants were required to perform a verbal recognition memory task within the scanner, either under placebo or scopolamine conditions. Depending on the subcondition, participants were required to make a simple recognition decision (old/new items) or base their decision on more specific information related to prior experience (target/non-target/new items). We show a drug modulation in left prefrontal and perirhinal cortex during recollection. Such an effect was specifically driven by novelty and showed an inverse correlation with accuracy performance. Additionally, we show a direct correlation between drug-related signal change in left prefrontal and perirhinal cortices. We discuss the findings in terms of acetylcholine mediation of the familiarity/novelty signal through perirhinal cortex and the control of the relative signal strength through prefrontal cortex.     

Very early modulation of brain responses to neutral faces by a single prior association with an emotional context: evidence from MEG

Recent electrophysiological studies have demonstrated modulations of the very first stages of visual processing (<100 ms) due to prior experience. This indicates an influence of a memory trace on the earliest stages of stimulus processing. Here we investigated if emotional audio-verbal information associated with faces on first encounter can affect the very early responses to those faces on subsequent exposure. We recorded magneto-encephalographic (MEG) responses to neutral faces that had been previously associated with positive (happy), negative (angry) or neutral auditory verbal emotional contexts. Our results revealed a very early (30-60 ms) difference in the brain responses to the neutral faces according to the type of previously associated emotional context, with a clear dissociation between the faces previously associated to positive vs. negative or neutral contexts. Source localization showed that two main regions were involved in this very early association effect: the bilateral ventral occipito-temporal regions and the right anterior medial temporal region. These results provide evidence that the memory trace of a face integrates positive emotional cues present in the context of prior encounter and that this emotional memory can influence the very first stages of face processing. These experimental findings support the idea that face perception can be shaped by experience from its earliest stages and in particular through emotional association effects.     

Emotional context modulates subsequent memory effect

Emotions have been shown to modulate memory processes. However, the neuronal substrate underlying these modulatory effects is largely unknown. Using event-related functional magnetic resonance imaging (fMRI), we investigated whether the context of emotional encoding modulates brain activation predictive for subsequent recall of emotionally neutral material. While inferior frontal activation predicted recall in general, our data show that in a positive encoding context, recall was predicted by activation of right anterior parahippocampal and extrastriate visual brain areas, whereas in a negative encoding context, recall was predicted by activation of the amygdala. Thus, we could demonstrate that successful episodic encoding is differentially modulated by emotional context. These results contribute to the understanding of the interaction of emotion and cognition and moreover are of general relevance for studies of episodic memory.     

Electrophysiological evidence for the effect of interactive imagery on episodic memory: encouraging familiarity for non-unitized stimuli during associative recognition

Episodic memory depends upon multiple processes, including familiarity and recollection. Although associative recognition tasks are traditionally viewed as requiring recollection, recent research suggests a role for familiarity if to-be-remembered stimuli are perceived as unitized. Here we use event-related potentials (ERPs) to examine the relationship between stimulus properties and encoding strategy on the engagement of familiarity during associative recognition. Participants studied word pairs containing an association (e.g. traffic-jam) or an unassociated semantic relationship (e.g. violin-guitar), using either item or interactive imagery. At test, participants were required to recognize if word pairs were presented in the same pairing as study, were rearranged, or new. We hypothesized that adopting a strategy of interactive imagery during encoding (i.e. encouraging unitization) would enhance familiarity for unassociated word pairs but would have no effect on association pairs because they are already perceived as unitized. As expected, overall recognition performance was better for word pairs encoded with interactive imagery, and for association than semantic word pairs. ERPs recorded at test revealed an interaction between encoding strategy and stimulus properties. Association word pairs elicited similar bilateral frontal (familiarity) and left parietal (recollection) old/new effects following item and interactive imagery. By contrast, for semantic word pairs, the left parietal effect was equivalent across conditions, but the bilateral frontal effect was enhanced for the interactive imagery condition. The ERP results suggest that an encoding strategy of interactive imagery can enhance familiarity during associative recognition, but this effect is ultimately dependent on the properties of the stimuli to-be-remembered and the nature of the representations that underlie them.     

The hippocampal region is involved in successful recognition of both remote and recent famous faces

There is currently a debate regarding the precise role of medial temporal regions in memory, in particular regarding the time scale of their involvement in conscious recollection of information stored in long-term memory. Using event-related fMRI, we have attempted to contribute to this debate by identifying brain regions associated with the successful recognition of famous faces from two different periods: "Old" faces of people who became famous in the 1960s-1970s and "Recent" faces of people who became famous in the 1990s. We demonstrate that the hippocampus is involved in the successful recognition of famous faces from both periods and does not appear to distinguish between these two periods. We also highlight a network of brain regions, including the left prefrontal cortex, the retrosplenial cortex, the temporo-parietal junction, the caudate and the right cerebellum, which is activated in association with successful recognition of famous faces. Finally, an analysis of the results obtained during a post hoc episodic recognition task shows the specific involvement of anterior hippocampus in the successful encoding of the unfamiliar faces, which were presented during the fame decision task, suggesting a functional distinction between anterior and posterior parts of the hippocampus, the former being specifically involved in successful episodic encoding and the latter being associated with successful retrieval of semantic information.     

Social and emotional attachment in the neural representation of faces

To dissociate the role of visual familiarity from the role of social and emotional factors in recognizing familiar individuals, we measured neural activity using functional magnetic resonance imaging (fMRI) while subjects viewed (1) faces of personally familiar individuals (i.e. friends and family), (2) faces of famous individuals, and (3) faces of strangers. Personally familiar faces evoked a stronger response than did famous familiar faces and unfamiliar faces in areas that have been associated with 'theory of mind', and a weaker response in the amygdala. These response modulations may reflect the spontaneous activation of social knowledge about the personality and attitudes of close friends and relatives and the less guarded attitude one has around these people. These results suggest that familiarity causes changes in neural response that extend beyond a visual memory for a face.     

Activation reduction in anterior temporal cortices during repeated recognition of faces of personal acquaintances

Repeated recognition of the face of a familiar individual is known to show semantic repetition priming effect. In this study, normal subjects were repeatedly presented faces of their colleagues, and the effect of repetition on the regional cerebral blood flow change was measured using positron emission tomography. They repeated a set of three tasks: the familiar-face detection (F) task, the facial direction discrimination (D) task, and the perceptual control (C) task. During five repetitions of the F task, familiar faces were presented six times from different views in a pseudorandom order. Activation reduction through the repetition of the F tasks was observed in the bilateral anterior (anterolateral to the polar region) temporal cortices which are suggested to be involved in the access to the long-term memory concerning people. The bilateral amygdala, the hypothalamus, and the medial frontal cortices, were constantly activated during the F tasks, and considered to be associated with the behavioral significance of the presented familiar faces. Constant activation was also observed in the bilateral occipitotemporal regions and fusiform gyri and the right medial temporal regions during perception of the faces, and in the left medial temporal regions during the facial familiarity detection task, which are consistent with the results of previous functional brain imaging studies. The results have provided further information about the functional segregation of the anterior temporal regions in face recognition and long-term memory.     

Neural correlates of perceptual contributions to nondeclarative memory for faces

Face priming is a nondeclarative memory phenomenon that can be observed when recognition is facilitated for a recently encountered face. This data-driven form of priming is distinct from conceptually driven priming. Moreover, it includes two dissociable components, the facilitated access to pre-existing representations and facilitation in perceptual processing of faces. In the present study, we measured neural correlates of perceptual contributions to face priming with event-related brain potentials. Faces appeared two times (separated by 7-17 s), while participants discriminated familiar from unfamiliar faces. Half of the initial face stimuli were inverted, thereby disrupting perceptual face processing and making possible an assessment of perceptual contributions to face priming. Whereas none of the brain waves previously linked to perceptual processing of faces showed indications of priming, such effects were observed between 200 and 600 ms at left occipito-parieto-temporal recording sites. This electrical activity was present for both unfamiliar and familiar faces. The scalp topography of this effect was consistent with sources within the temporal and occipital cortices of the left hemisphere (based on a LORETA source localization). These findings suggest that priming of perceptual face processing is subserved by prolonged neural activity from 200 to 600 ms primarily in the left hemisphere. We propose that this priming reflects facilitated selection based on second-order relations among facial features.     

Familiarity enhances invariance of face representations in human ventral visual cortex: fMRI evidence

Face recognition across different viewing conditions is strongly improved by familiarity. In the present study, we tested the hypothesis that the neural basis of this effect is a less view-dependent representation of familiar faces in ventral visual cortex by assessing priming-related fMRI repetition effects. 15 healthy volunteers made male/female judgements on familiar (famous) and unfamiliar (novel) faces preceded by the same image, a different image of the same face, or another (unprimed) face. Reaction times revealed priming by same and different images independent of familiarity and more pronounced for same than different images. In the imaging data, a main effect of prime condition was found in bilateral fusiform and orbitofrontal regions. A right anterior fusiform region expressed stronger response decreases to repetition of familiar than unfamiliar faces. Bilateral mid-fusiform areas showed stronger response decreases to repetition of same than different images. A regions-of-interest analysis focussing specifically on face responsive regions suggested differences in the degree of image dependency across fusiform cortex. Collapsing across familiarity, there was greater image dependency of repetition effects in right than left anterior fusiform, replicating previous imaging findings obtained with common objects. For familiar faces alone, there was greater generalisation of repetition effects over different images in anterior than middle fusiform. This suggests a role of anterior fusiform cortex in coding image-independent representations of familiar faces.     

Neural substrates participating in acquisition of facial familiarity: an fMRI study

The amygdala is related to recognition of faces and emotions, and functional magnetic resonance imaging (fMRI) studies have reported that the amygdala is habituated over time with repetition of facial stimuli. When subjects are presented repeatedly with unfamiliar faces, they come to gradually recognize the unfamiliar faces as familiar. To investigate the brain areas participating in the acquisition of familiarity to repeatedly presented unfamiliar faces, we conducted an fMRI study in 16 healthy subjects. During the task periods, the subjects were instructed to see presented unfamiliar faces repeatedly and to judge whether the face was male or female or whether the face had emotional valences. The experiment consisted of nine sessions. To clarify the brain areas that showed increasing or decreasing activation as the experimental session proceeded, we analyzed the fMRI data using specified linear covariates in the face recognition task from the first session to the ninth session. Imaging data were investigated on a voxel-by-voxel basis for single-group analysis according to the random effect model using Statistical Parametric Mapping. The bilateral posterior cingulate cortices showed significant increases in activity as the experimental sessions proceeded, while the activation in the right amygdala and the left medial fusiform gyrus decreased. Thus, the posterior cingulate cortex may play an important role in the acquisition of facial familiarity.     

Electrophysiological neural mechanisms for detection, configural analysis and recognition of faces

Despite ample explorations the nature of neural mechanisms underlying human expertise in face perception is still undetermined. Here we examined the response of two electrophysiological signals, the N170 ERP and induced gamma-band activity (>20 Hz), to face orientation and familiarity across two blocks, one in which the face identity was task-relevant and one in which it was not. N170 amplitude to inverted faces was higher than to upright faces and was not influenced by face familiarity or its task relevancy. In contrast, induced gamma activity was higher for upright than for inverted faces and for familiar than unfamiliar faces. The effect of face inversion was found in lower gamma frequency band (25-50 Hz), whereas familiarity affected amplitudes in higher gamma frequency band (50-70 Hz). For gamma, the relevance of face identity to the task modulated both inversion and familiarity effects. These findings pinpoint three functionally dissociated neural mechanisms involved in face processing, namely, detection, configural analysis, and recognition.     

Time-dependent neural activations related to recognition of people's names in emotional and neutral face-name associative learning: an fMRI study

Previous data have indicated that the left anterior temporal lobe contributes to the retrieval of familiar people's names, and that the extended network including the bilateral anterior temporal lobe plays an important role in the retrieval of newly learned people's names. However, there has been no direct evidence for time-dependent change in brain activation in face-name associations. In addition, previous studies have demonstrated that emotional information such as emotional faces may contribute to the organization of long-lasting episodic memory. In the present study, we investigated the activations related to the recognition of people's names in the context of emotional and neutral face-name associative learning. Before fMRI scanning, subjects learned face-name associations that included emotionally positive and neutral facial expressions. In immediate (5 min later) and delayed (2 weeks later) recognition with fMRI scanning, subjects were presented with studied faces with two names, and were asked to choose the correct associative name learned previously. Recognition-related activations were identified in the anterior part of bilateral temporal lobe for immediate recognition and only in the left temporal lobe for delayed recognition. Further analysis confirmed the time-dependent change in activation of the right anterior temporal lobe. Activation related to the processing of faces with positive expressions were observed in the left periamygdaloid area and temporal pole, although emotional information did not have an influence on task performance in this study. These findings suggest that the neural network involving the bilateral temporal lobe contributes to the retrieval of newly learned people's names, and that the left temporal lobe has a crucial and stable role in retrieval of people's names from faces, whereas the role of the right temporal lobe in retrieval of people's names may decrease with the time course.     

Neuroanatomical substrates involved in true and false memories for face

We often mistake an unknown person for a familiar person because of the similarities in facial features. This phenomenon, known as false memory, has been investigated mainly using words, pictures, and shapes. Previous neuroimaging studies on false memory have shown that both true and false memories trigger a similar activation in the medial temporal lobe, suggesting that it plays a common role in both. However, no study to date has investigated neural substrates of false memories for faces. In the present fMRI study, we applied a modified version of the standard false memory paradigm, using morphed pictures of faces, to induce false memory in an MRI environment. We found that activity in the amygdala and orbital cortices was associated with the degree of familiarity of items. In particular, false responses to "lure" items evoked a level of activity in the amygdala between that evoked for correct or incorrect responses to "true" items. This indicates a possible role of the amygdala in false memory. A specific region in the anterior cingulate cortex was involved in false recognition; the activity being correlated to reaction times for the response types. These results suggest that the amygdala is involved in determining the relevance of items; therefore, ambiguousness of lure items in terms of familiarity and novelty may be related to decreased activity in the amygdala. The anterior cingulate activity in false memory may be caused not only by increased effort and motor demand but also by higher mnemonic processing of lure items.