Dissociating recollection from familiarity: electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect

In recognition memory research, a tension exists between dual-process and single-process models of episodic retrieval. Dual-process models propose that 'familiarity' assessment and the 'recollection' of contextual information are independent processes, while single-process models claim that one common process supports retrieval. Event-related potentials (ERPs) have been used to show dissociations between the mid frontal and the left parietal ERP old/new effects, which have been associated with familiarity and recollection, respectively. While much ERP evidence favours dual-process theory, Yovel and Paller [Yovel, G., Paller, K.A., 2004. The neural basis of the butcher-on-the-bus phenomenon: when a face seems familiar but is not remembered. NeuroImage 21, 789-800] used faces as retrieval cues to demonstrate that posterior old/new effects index both familiarity and recollection, a finding consistent with single-process models. Here we present evidence supporting Yovel and Paller's claim that a posterior old/new effect indexes familiarity for faces, along with a novel finding that recollection is associated with an anterior old/new effect. Importantly, and in contrast to Yovel and Paller, the old/new effects associated with familiarity and recollection were topographically dissociable, consistent with a dual-process view of recognition memory. The neural correlates of familiarity and recollection identified here for faces appear to be different from those typically observed, suggesting that the ERP old/new effects associated with episodic recognition are not the same under all circumstances.     

Changes in brain electrical activity during extended continuous word recognition

Twenty healthy subjects (10 men, 10 women) participated in an EEG study with an extended continuous recognition memory task, in which each of 30 words was randomly shown 10 times and subjects were required to make old vs. new decisions. Both event-related brain potentials (ERPs) and induced band power (IBP) were investigated. We hypothesized that repeated presentations affect recollection rather than familiarity. For the 300- to 500-ms time window, an 'old/new' ERP effect was found for the first vs. second word presentations. The correct recognition of an 'old' word was associated with a more positive waveform than the correct identification of a new word. The old/new effect was most pronounced at and around the midline parietal electrode position. For the 500- to 800-ms time window, a linear repetition effect was found for multiple word repetitions. Correct recognition after an increasing number of repetitions was associated with increasing positivity. The multiple repetitions effect was most pronounced at the midline central (Cz) and fronto-central (FCz) electrode positions and reflects a graded recollection process: the stronger the memory trace grows, the more positive the ERP in the 500- to 800-ms time window. The ERP results support a dual-processing model, with familiarity being discernable from a more graded recollection state that depends on memory strengths. For IBP, we found 'old/new' effects for the lower-2 alpha, theta, and delta bands, with higher bandpower during 'old' words. The lower-2 alpha 'old/new' effect most probably reflects attentional processes, whereas the theta and delta effects reflect encoding and retrieval processes. Upon repeated word presentations, the magnitude of induced delta power in the 375- to 750-ms time window diminished linearly. Correlation analysis suggests that decreased delta power is moderately associated with faster decision speed and higher accuracy.     

Electrophysiological evidence for the effect of interactive imagery on episodic memory: encouraging familiarity for non-unitized stimuli during associative recognition

Episodic memory depends upon multiple processes, including familiarity and recollection. Although associative recognition tasks are traditionally viewed as requiring recollection, recent research suggests a role for familiarity if to-be-remembered stimuli are perceived as unitized. Here we use event-related potentials (ERPs) to examine the relationship between stimulus properties and encoding strategy on the engagement of familiarity during associative recognition. Participants studied word pairs containing an association (e.g. traffic-jam) or an unassociated semantic relationship (e.g. violin-guitar), using either item or interactive imagery. At test, participants were required to recognize if word pairs were presented in the same pairing as study, were rearranged, or new. We hypothesized that adopting a strategy of interactive imagery during encoding (i.e. encouraging unitization) would enhance familiarity for unassociated word pairs but would have no effect on association pairs because they are already perceived as unitized. As expected, overall recognition performance was better for word pairs encoded with interactive imagery, and for association than semantic word pairs. ERPs recorded at test revealed an interaction between encoding strategy and stimulus properties. Association word pairs elicited similar bilateral frontal (familiarity) and left parietal (recollection) old/new effects following item and interactive imagery. By contrast, for semantic word pairs, the left parietal effect was equivalent across conditions, but the bilateral frontal effect was enhanced for the interactive imagery condition. The ERP results suggest that an encoding strategy of interactive imagery can enhance familiarity during associative recognition, but this effect is ultimately dependent on the properties of the stimuli to-be-remembered and the nature of the representations that underlie them.     

Electrophysiological correlates of recollecting faces of known and unknown individuals

We recorded brain potentials from healthy human subjects during a recognition test in order to monitor neural processing associated with face recollection. Subjects first attempted to memorize 40 faces; half were accompanied by a voice simulating that person speaking (e.g., "I'm Jimmy and I was a roadie for the Grateful Dead") and half were presented in silence. In the test phase, subjects attempted to discriminate both types of old faces (i.e., "named" and "unnamed" faces) from new faces. Recognition averaged 87% correct for named faces, 74% correct for unnamed faces, and 91% correct for new faces. Potentials to old faces were more positive than those to new faces from 300 to 600 ms after face onset. For named faces, the old-new ERP difference was observed at anterior and posterior scalp locations. For unnamed faces, the old-new ERP difference was observed only at posterior scalp locations. Results from a prior experiment suggest that these effects do not reflect perceptual priming of faces. The posterior portion of the old-new ERP difference was thus interpreted as a neural correlate of retrieval of visual face information and the anterior portion as an indication of retrieval of person-specific semantic information.     

Brain and behavioral indices of retrieval mode

In recent event-related potential (ERP) studies of episodic retrieval, ERPs have been acquired in tasks where participants have been cued trial-by-trial to prepare either to make episodic or semantic retrieval judgments. ERPs elicited during this preparatory cue period and separated according to retrieval task have diverged at right frontal scalp electrodes, with a relatively greater positivity associated with preparation for episodic rather than for semantic retrieval. Importantly, this pattern of differences has been observed only on 'stay' trials: those trials where the participant was cued to prepare for the same retrieval task on the previous trial. These findings have provided the basis for the proposal that the ERP modulations index processes that support the adoption or configuration of retrieval mode - a tonic process that can be sustained while recovery of episodic information is required and which facilitates the retrieval process. In these studies, however, the preparatory period on each trial was no more than 2000 ms, raising the possibility that, with more time available, neural correlates of these preparatory processes would not be restricted to stay trials. In this experiment, participants were cued trial-by-trial to complete either an episodic or a semantic retrieval task, and the preparatory period was greater than 4000 ms on the majority of trials. In keeping with previous findings, the ERPs elicited by these two cue types diverged principally on stay trials at right frontal electrode locations, suggesting that time to prepare is not the primary determinant of the onset of task-specific preparatory retrieval processing. In an important addition to previous findings, moreover, the accuracy of episodic memory judgments increased with the number of successive trials of the same task that participants completed, a finding consistent with the view that adopting a retrieval mode successfully can influence the accuracy of episodic memory judgments.     

Examining ERP correlates of recognition memory: evidence of accurate source recognition without recollection

Recollection is typically associated with high recognition confidence and accurate source memory. However, subjects sometimes make accurate source memory judgments even for items that are not confidently recognized, and it is not known whether these responses are based on recollection or some other memory process. In the current study, we measured event related potentials (ERPs) while subjects made item and source memory confidence judgments in order to determine whether recollection supported accurate source recognition responses for items that were not confidently recognized. In line with previous studies, we found that recognition memory was associated with two ERP effects: an early on-setting FN400 effect, and a later parietal old-new effect [late positive component (LPC)], which have been associated with familiarity and recollection, respectively. The FN400 increased gradually with item recognition confidence, whereas the LPC was only observed for highly confident recognition responses. The LPC was also related to source accuracy, but only for items that had received a high confidence item recognition response; accurate source judgments to items that were less confidently recognized did not exhibit the typical ERP correlate of recollection or familiarity, but rather showed a late, broadly distributed negative ERP difference. The results indicate that accurate source judgments of episodic context can occur even when recollection fails.     

Investigating the functional interaction between semantic and episodic memory: convergent behavioral and electrophysiological evidence for the role of familiarity

Throughout our lives we acquire general knowledge about the world (semantic memory) while also retaining memories of specific events (episodic memory). Although these two forms of memory have been dissociated on the basis of neuropsychological data, it is clear that they typically function together during normal cognition. The goal of the present study was to investigate this interaction. One influence of semantic memory on episodic retrieval is 'Levels Of Processing'; recognition is enhanced when stimuli are processed in a semantically meaningful way. Studies examining this semantic processing advantage have largely concluded that semantic memory augments episodic retrieval primarily by enhancing recollection. The present study provides strong evidence for an alternative relationship between semantic and episodic memory. We employed a manipulation of the semantic coherence of to-be-remembered information (semantically related vs. unrelated word pairs) during an associative recognition memory test. Results revealed that associative recognition is significantly enhanced for semantically coherent material, and behavioral estimates (using the process dissociation procedure) demonstrated concomitant changes in the contribution of familiarity to retrieval. Neuroimaging data (event-related potentials recorded at test) also revealed a significant increase in familiarity based retrieval. The electrophysiological correlate of familiarity (the mid-frontal ERP old/new effect) was larger for semantically related compared to unrelated word pairs, but no difference was present in the electrophysiological correlate of recollection (the left parietal old/new effect). We conclude that semantic memory and episodic memory do indeed interact in normal functioning, and not only by modulating recollection, but also by enhancing familiarity.     

The worth of pictures: using high density event-related potentials to understand the memorial power of pictures and the dynamics of recognition memory

To understand the neural correlates of the memorial power of pictures, pictures and words were systematically varied at study and test within subjects, and high-density event-related potentials (ERPs) were recorded at retrieval. Using both conventional and novel methods, the results were presented as ERP waveforms, 50 ms scalp topographies, and video clips, and analyzed using t-statistic topographic maps and nonparametric p-value maps. The authors found that a parietally-based ERP component was enhanced when pictures were presented at study or test, compared to when words were presented. An early frontally-based component was enhanced when words were presented at study compared to pictures. From these data the authors speculate that the memorial power of pictures is related to the ability of pictures to enhance recollection. Familiarity, by contrast, was enhanced when words were presented at study compared to pictures. From these results and the dynamic view of memory afforded by viewing the data as video clips, the authors propose an ERP model of recognition memory.     

Recognition memory for studied words is determined by cortical activation differences at encoding but not during retrieval

Prior work has shown that when responses to incidentally encoded words are sorted, subsequently remembered words elicit greater left prefrontal BOLD signal change relative to forgotten words. Similarly, low-frequency words elicit greater activation than high-frequency words in the same left prefrontal regions, contributing to their better subsequent memorability. This study examined the relative contribution of encoding and retrieval processes to the correct recognition of target words. A mixture of high- and low-frequency words was incidentally encoded. Scanning was performed at encoding as well as during retrieval. During encoding, greater activation in the left prefrontal and anterior cingulate regions predicted a higher proportion of hits for low-frequency words. However, data acquired during recognition showed that word frequency did not modulate activation in any of the areas tracking successful recognition. This result demonstrates that under some circumstances, the recognition of studied words is determined purely by processes that are active during encoding. In contrast to the finding for hits, activation associated with correctly rejected foils was modulated by word frequency, being higher for high-frequency words in the left lateral parietal and anterior prefrontal regions. These findings were replicated in two further experiments, one in which the number of test items at recognition was doubled and another where encoding strength for high-frequency words was varied (once vs. 10 times). These results indicate that word frequency modulates activity in the left lateral parietal and anterior prefrontal regions contingent on whether the item involved is correctly recognized as a target or a foil. This observation is consistent with a dual process account of episodic memory.     

Left temporal and temporoparietal brain activity depends on depth of word encoding: a magnetoencephalographic study in healthy young subjects

Using a 143-channel whole-head magnetoencephalograph (MEG) we recorded the temporal changes of brain activity from 26 healthy young subjects (14 females) related to shallow perceptual and deep semantic word encoding. During subsequent recognition tests, the subjects had to recognize the previously encoded words which were interspersed with new words. The resulting mean memory performances across all subjects clearly mirrored the different levels of encoding. The grand averaged event-related fields (ERFs) associated with perceptual and semantic word encoding differed significantly between 200 and 550 ms after stimulus onset mainly over left superior temporal and left superior parietal sensors. Semantic encoding elicited higher brain activity than perceptual encoding. Source localization procedures revealed that neural populations of the left temporal and temporoparietal brain areas showed different activity strengths across the whole group of subjects depending on depth of word encoding. We suggest that the higher brain activity associated with deep encoding as compared to shallow encoding was due to the involvement of more neural systems during the processing of visually presented words. Deep encoding required more energy than shallow encoding but for all that led to a better memory performance.     

Cognitive training-related changes in hippocampal activity associated with recollection in older adults

Impairments in the ability to recollect specific details of personally experienced events are one of the main cognitive changes associated with aging. Cognitive training can improve older adults' recollection. However, little is currently known regarding the neural correlates of these training-related changes in recollection. Prior research suggests that the hippocampus plays a central role in supporting recollection in young and older adults, and that age-related changes in hippocampal function may lead to age-related changes in recollection. The present study investigated whether cognitive training-related increases in older adults' recollection are associated with changes in their hippocampal activity during memory retrieval. Older adults' hippocampal activity during retrieval was examined before and after they were trained to use semantic encoding strategies to intentionally encode words. Training-related changes in recollection were positively correlated with training-related changes in activity for old words in the hippocampus bilaterally. Positive correlations were also found between training-related changes in activity in prefrontal and left lateral temporal regions associated with self-initiated semantic strategy use during encoding and training-related changes in right hippocampal activity associated with recollection during retrieval. These results suggest that cognitive training-related improvements in older adults' recollection can be supported by changes in their hippocampal activity during retrieval. They also suggest that age differences in cognitive processes engaged during encoding are a significant contributor to age differences in recollection during retrieval.     

On why the elderly have normal semantic retrieval but deficient episodic encoding: a study of left inferior frontal ERP activity

Age-related left inferior prefrontal cortex (LIPFC) blood flow reductions during semantic retrieval are associated with reduced subsequent episodic recognition memory performance but are inconsistent with age-invariant semantic retrieval performance. Therefore, we compared brain activity in young and elderly persons during low- and high-selection versions of a semantic task using ERPs recorded at 62 scalp locations. In an early interval (400-800 ms), both age groups showed more negativity over left inferior frontal scalp while performing the high- compared to the low-selection task. This early semantic selection-related negativity was associated with age-equivalent accuracy and reaction time on the semantic tasks. Further, in the early time interval, the ERPs of the young were more negative than those of the elderly. In addition, only the young showed a late selection-related negativity (1,200--1,400 ms). These age-related differences in left frontal ERP activity were associated with significant decrements in subsequent recognition for the elderly. Moreover, additional analyses revealed that larger amounts of negativity over left inferior frontal scalp locations during semantic retrieval were correlated with enhanced subsequent episodic recognition in both groups. In sum, the data suggest that (1) semantic retrieval and episodic encoding processes overlap temporally in similar brain regions, most likely including the LIPFC, and (2) the failure to recruit these regions in the service of episodic encoding underlies the age-related deficit in episodic memory.     

The role of precuneus and left inferior frontal cortex during source memory episodic retrieval

The posterior medial parietal cortex and left prefrontal cortex (PFC) have both been implicated in the recollection of past episodes. In a previous study, we found the posterior precuneus and left lateral inferior frontal cortex to be activated during episodic source memory retrieval. This study further examines the role of posterior precuneal and left prefrontal activation during episodic source memory retrieval using a similar source memory paradigm but with longer latency between encoding and retrieval. Our results suggest that both the precuneus and the left inferior PFC are important for regeneration of rich episodic contextual associations and that the precuneus activates in tandem with the left inferior PFC during correct source retrieval. Further, results suggest that the left ventro-lateral frontal region/frontal operculum is involved in searching for task-relevant information (BA 47) and subsequent monitoring or scrutiny (BA 44/45) while regions in the dorsal inferior frontal cortex are important for information selection (BA 45/46).     

The neural basis of the butcher-on-the-bus phenomenon: when a face seems familiar but is not remembered

A common distinction in contemporary research on episodic memory is between familiarity, an unsubstantiated impression that an event was experienced previously, and recollection, remembering some information plus the spatiotemporal context of the episode in which it was acquired. The epitome of pure familiarity--the butcher-on-the-bus phenomenon--occurs when one believes that a person is familiar (often upon seeing their face in an atypical context) while failing to recall any information about that person whatsoever. Prior research on familiarity and recollection has relied on verbal material. Whereas word meanings and pronunciations are well learned in advance, here we produced pure familiarity and recollection using photographs of faces never seen before the experiment. When participants recognized a face, recollection was inferred if they also remembered either the occupation associated with that face earlier in the experiment or any other episodic detail. Pure familiarity was inferred when recognition occurred in the absence of any such contextual retrieval. Analyses of brain potentials recorded during initial encoding showed that right-sided neural activity predicted subsequent face familiarity, whereas bilateral potentials predicted subsequent face recollection. Results during memory testing were inconsistent with the popular idea that familiarity is generically indexed by reduced frontal N400-like potentials. Instead, both memory experiences were associated with bilateral, parietal-maximum brain potentials, although with smaller amplitudes and for a shorter duration for familiarity. These similarities between electrophysiological correlates of pure familiarity and recollection suggest that familiarity with faces may arise by virtue of a subset of the neural processing responsible for recollection.     

Activation of human hippocampal formation reflects success in both encoding and cued recall of paired associates

Contemporary theories of hippocampal function suggest that both encoding and retrieval of episodic memories may be accomplished by neural circuitry embedded within the same anatomical structures, but neuroimaging support for this hypothesis has been ambiguous. Recent studies suggest that the best available indicators of hippocampal encoding and retrieval operations are selective activations due to novelty, encoding success, and recall success in a paired associate learning paradigm. In the current study, both encoding and cued recall of paired associate words were conducted during a single session of fMRI scanning. Bilateral activation in the medial temporal lobe was detected for encoding word pairs vs. a fixation baseline and for encoding novel word pairs vs. repeated word pairs. These activations were stronger in subjects who successfully memorized more word pairs. In cued recall, greater responses were seen in higher performing subjects. In lower performing subjects, responses were greater to cue words whose paired associate was correctly recalled than to cue words whose correct associate had been forgotten (or not encoded). The difference between correct and incorrect trials was more pronounced on repeated presentations of the same cue words, but not apparent on their first presentation alone. Overlap of encoding and retrieval effects was maximal in the middle of the longitudinal extent of the right hippocampus, with one additional locus of overlap outside the MTL, in left occipitotemporal cortex. The conjunction of these effects suggests that it is correct to view both encoding and recall of associative memories as functions of an integrated hippocampal system.     

Isolating the retrieval of imagined pictures during episodic memory: activation of the left precuneus and left prefrontal cortex

The posterior medial parietal cortex and the left prefrontal cortex have both been implicated in the recollection of past episodes. In order to clarify their functional significance, we performed this functional magnetic resonance imaging study, which employed event-related source memory and item recognition retrieval of words paired with corresponding imagined or viewed pictures. Our results suggest that episodic source memory is related to a functional network including the posterior precuneus and the left lateral prefrontal cortex. This network is activated during explicit retrieval of imagined pictures and results from the retrieval of item-context associations. This suggests that previously imagined pictures provide a context with which encoded words can be more strongly associated.     

Electrophysiological correlates of retrieval orientation in reality monitoring

Retrieval orientation describes the modulation in the processing of retrieval cues by the nature of the targeted material in memory. Retrieval orientation is usually investigated by analyzing the cortical responses to new (unstudied) material when different memory contents are targeted. This approach avoids confounding effects of retrieval success. We investigated the neural correlates of retrieval orientation in reality monitoring with event-related potentials (ERPs) and assessed the impact of retrieval accuracy on obtained ERP measures. Thirty-two subjects studied visually presented object names that were followed either by a picture of that object (perceived condition) or by the instruction to mentally generate such a picture (imagine condition). Subsequently, subjects had to identify object names of one study condition and reject object names of the second study condition together with newly presented object names. The data analysis showed that object names were more accurately identified when they had been presented in the perceived condition. Two topographically distinct ERP effects of retrieval orientation were revealed: From 600 to 1100 ms after stimulus representation, ERPs were more positive at frontal electrode sites when object names from the imagine condition were targeted. The analysis of response-locked ERP data revealed an additional effect at posterior electrode sites, with more negative ERPs shortly after response onset when items from the imagine condition were targeted. The ERP effect at frontal electrode sites, but not at posterior electrode sites was modulated by relative memory accuracy, with stronger effects in subjects who had lower memory accuracy for items of the imagine condition. The findings are suggestive for a contribution of frontal brain areas to retrieval orientation processes in reality monitoring and indicate that neural correlates of retrieval orientation can be modulated by retrieval effort, with stronger activation of these correlates with increasing task demands.     

Cortical lateralization during verb generation: a combined ERP and fMRI study

Lateralization of scalp-recorded event-related potentials (ERPs) and functional MRI (fMRI) activation was investigated using a verb generation task in 10 healthy right-handed adults. ERPs showed an early transient positivity in the left inferior temporal region (500-1250 ms) following auditory presentation of the stimulus noun. A sustained slow cortical negativity of later onset (1250-3000 ms) was then recorded, most pronounced over left inferior frontal regions. fMRI data were in agreement with both ERP effects, showing left lateralized activation in inferior and superior temporal as well as inferior frontal cortices. Lateralized ERP effects occurred during the verb generation task but not during passive word listening or during word- and nonword repetition. Thus, ERPs and fMRI provided convergent evidence regarding language lateralization, with ERPs revealing the temporal sequence of posterior to anterior cortical activation during semantic retrieval.     

The FN400 is functionally distinct from the N400

The FN400 refers to the early midfrontally-distributed difference between ERPs elicited by old and new items, which operates in a way consistent with a neural marker of familiarity-based recognition. Double dissociations between the FN400 and a later ERP index of recollection provide some of the most compelling evidence in support of dual-process models to date. It has recently been claimed, however, that there is no evidence that the FN400 is functionally distinct from the N400 index of implicit semantic priming (Voss, J., and Federmeier, K., FN400 potentials are functionally identical to N400 potentials and reflect semantic processing during recognition testing, Psychophysiology, 48, 532-546, 2011), challenging inferences made on the basis of this effect. We argue that the design employed to make this claim is flawed because it comprised a semantic priming manipulation embedded within a continuous recognition test which enabled recognition contrasts to be confounded by semantic processes in a number of ways. Here, ERPs were recorded from a design which avoided these confounds by employing a semantic priming paradigm which also served as the encoding phase for a surprise subsequent recognition test phase. An N400 effect elicited in the semantic priming task demonstrated the established centro-parietal maximum, whereas the difference between correctly responded to old and new ERPs in the recognition test was maximal over frontal sites in the same time window. When direct comparisons of the electrophysiological correlates of semantic priming and episodic recognition are recorded in a paradigm in which the two are not confounded, the FN400 reflects a qualitatively distinct effect from the N400.     

Magnetoencephalographic correlates of different levels in subjective recognition memory

In our current study we employed whole-head magnetoencephalography (MEG) to identify neurophysiological correlates (event-related fields, ERFs) of different phenomenologies in human recognition memory. Words which had previously been semantically processed were presented along with previously unstudied words. Via button presses, participants provided subjective indices of three forms of memory: confident recognition, familiarity-based recognition, and misclassification of previously presented items as new (no recognition, misses). Behavioral results revealed a clear distinction between confident recognition (shortest reaction times) and familiarity-based recognition and misses, respectively, and physiological data pointed to bilateral anterior and left anterior/central regions in which magnetic field patterns were directly related to word recognition from approximately 300 ms to 500 ms after word onset. In the context of the prevalent dual process controversy on the roles of familiarity and recollection in recognition memory, we first highlight that two operationalizations of recollection need to be differentiated: We argue that a strategic search for a particular contextual feature stands in clear contrast to the fast and incidental availability of some contextual feature and derive experimental and behavioral indicators for either form of recollection. These indicators are used to select from manifold cognitive neuroscientific work on recognition memory in order to further discuss the neurocognitive characteristics of incidental recollection in contrast to other forms of episodic memory.