Electrophysiological evidence for the effect of interactive imagery on episodic memory: encouraging familiarity for non-unitized stimuli during associative recognition

Episodic memory depends upon multiple processes, including familiarity and recollection. Although associative recognition tasks are traditionally viewed as requiring recollection, recent research suggests a role for familiarity if to-be-remembered stimuli are perceived as unitized. Here we use event-related potentials (ERPs) to examine the relationship between stimulus properties and encoding strategy on the engagement of familiarity during associative recognition. Participants studied word pairs containing an association (e.g. traffic-jam) or an unassociated semantic relationship (e.g. violin-guitar), using either item or interactive imagery. At test, participants were required to recognize if word pairs were presented in the same pairing as study, were rearranged, or new. We hypothesized that adopting a strategy of interactive imagery during encoding (i.e. encouraging unitization) would enhance familiarity for unassociated word pairs but would have no effect on association pairs because they are already perceived as unitized. As expected, overall recognition performance was better for word pairs encoded with interactive imagery, and for association than semantic word pairs. ERPs recorded at test revealed an interaction between encoding strategy and stimulus properties. Association word pairs elicited similar bilateral frontal (familiarity) and left parietal (recollection) old/new effects following item and interactive imagery. By contrast, for semantic word pairs, the left parietal effect was equivalent across conditions, but the bilateral frontal effect was enhanced for the interactive imagery condition. The ERP results suggest that an encoding strategy of interactive imagery can enhance familiarity during associative recognition, but this effect is ultimately dependent on the properties of the stimuli to-be-remembered and the nature of the representations that underlie them.     

Activation of human hippocampal formation reflects success in both encoding and cued recall of paired associates

Contemporary theories of hippocampal function suggest that both encoding and retrieval of episodic memories may be accomplished by neural circuitry embedded within the same anatomical structures, but neuroimaging support for this hypothesis has been ambiguous. Recent studies suggest that the best available indicators of hippocampal encoding and retrieval operations are selective activations due to novelty, encoding success, and recall success in a paired associate learning paradigm. In the current study, both encoding and cued recall of paired associate words were conducted during a single session of fMRI scanning. Bilateral activation in the medial temporal lobe was detected for encoding word pairs vs. a fixation baseline and for encoding novel word pairs vs. repeated word pairs. These activations were stronger in subjects who successfully memorized more word pairs. In cued recall, greater responses were seen in higher performing subjects. In lower performing subjects, responses were greater to cue words whose paired associate was correctly recalled than to cue words whose correct associate had been forgotten (or not encoded). The difference between correct and incorrect trials was more pronounced on repeated presentations of the same cue words, but not apparent on their first presentation alone. Overlap of encoding and retrieval effects was maximal in the middle of the longitudinal extent of the right hippocampus, with one additional locus of overlap outside the MTL, in left occipitotemporal cortex. The conjunction of these effects suggests that it is correct to view both encoding and recall of associative memories as functions of an integrated hippocampal system.     

Neural correlates of successful encoding of semantically and phonologically mediated inter-item associations

This experiment investigated whether the neural correlates of inter-item associative encoding vary according to study task. At study, pairs of unrelated words were subjected to either semantic or phonological relational judgments. Test items comprised studied word pairs (intact), pairs comprised of words belonging to different study pairs (rearranged), and novel pairs. The test requirement was to discriminate between these different classes of test item. fMRI was employed to contrast the neural activity elicited by studied pairs that were correctly endorsed as intact on the later associative recognition test, as opposed to pairs for which associative information was unavailable. In contrast to prior findings for the encoding of single items, there was no evidence that the loci of subsequent associative memory effects varied according to study task. Instead, in both tasks, pairs that were later correctly endorsed as intact elicited enhanced activity in mid- and ventral regions of the left ventrolateral prefrontal cortex (VLPFC). These findings were accompanied by extensive task-invariant reversed subsequent memory effects in medial and lateral parietal and frontal cortices. The findings suggest that the left VLPFC may play a domain-general role in the encoding of item-item associations, and in addition highlight the importance of elucidating the functional significance of reversed subsequent memory effects.     

Prefrontal hemodynamic activity predicts false memory--a near-infrared spectroscopy study

Evidence from lesion studies suggests an important role of the prefrontal cortex (PFC) in the reconstructive processes of episodic memory or memory distortion. Results from functional imaging studies imply PFC involvement during the illusionary recollection of non-experienced events. Here, we used a two-channel near-infrared spectroscopy (NIRS) system and conducted real-time monitoring of PFC hemodynamics, while subjects studied word lists and subsequently recognized unstudied items (false recognition). Bilateral increases in the oxygenated hemoglobin concentration ([oxy-Hb]) were observed during false recognition compared to true recognition, and a left PFC dominant increase of [oxy-Hb] was observed during encoding phases where subjects later claimed that they recognized unstudied words. Traces of semantic processing, reflected primarily in the left PFC activity, could eventually predict whether subjects falsely recognize non-experienced events.     

Encoding activity in anterior medial temporal lobe supports subsequent associative recognition

The ability to bind information together, such as linking a name with a face or a car with a parking space, is a vital process in human episodic memory. To identify the neural bases for this binding process, we measured brain activity during a verbal associative encoding task using event-related functional MRI (fMRI), followed by an associative recognition test for the studied word pairs. Analysis of the encoding data sorted by the associative recognition accuracy allowed us to isolate regions involved in successfully creating associations. We found that encoding activity in bilateral anterior medial temporal lobe (MTL) regions was greater for successfully bound pairs, that is, those later recognized as intact, than for all other pairs. These findings provide evidence that the anterior medial temporal lobes support the successful binding of information in memory.     

Prefrontal and hippocampal contributions to the generation and binding of semantic associations during successful encoding

The ability to form and bind associations between items is an important aspect of successful memory formation. We hypothesize that, during encoding, the left inferior frontal gyrus (IFG) supports generation of associations between items and the hippocampus then binds these associations. This study examined the parametric responses of these regions to varying amounts of generative and relational processing during successful encoding (i.e., for subsequently recognized items). Encoding involved presentation of word triads varying in the number of semantic associations among them (none, one or all); participants judged how many associations were present in each triad. Thus, triads with fewer associations had higher generative load while triads with more associations had higher relational load. Participants later completed a forced-choice recognition test for encoding triads. Successful encoding relative to a control task resulted in activation of bilateral IFG and left hippocampus, and the hippocampus also exhibited a significant subsequent memory effect (hits>misses). Linear parametric analyses revealed that generative load modulated activity in bilateral IFG while relational load correlated with activity in left hippocampus. Although univariate analyses distinguished IFG and hippocampal contributions to the generative and relational stages of encoding, respectively, effective connectivity between these regions did not differ according to condition. Furthermore, this analysis revealed that the left IFG played a pivotal role in coordinating associative encoding processes. Our findings illustrate that modulation of components in a memory network can be independent of patterns of mutual connectivity among those components in mediating successful encoding.     

Putting names to faces: successful encoding of associative memories activates the anterior hippocampal formation

The ability to form associations between previously unrelated items of information, such as names and faces, is an essential aspect of episodic memory function. The neural substrate that determines success vs. failure in learning these associations remains to be elucidated. Using event-related functional MRI during the encoding of novel face-name associations, we found that successfully remembered face-name pairs showed significantly greater activation in the anterior hippocampal formation bilaterally and left inferior prefrontal cortex, compared to pairs that were forgotten. Functional connectivity analyses revealed significant correlated activity between the right and left hippocampus and neocortical regions during successful, but not attempted, encoding. These findings suggest that anterior regions of the hippocampal formation, in particular, are crucial for successful associative encoding and that the degree of coordination between hippocampal and neocortical activity may predict the likelihood of subsequent memory.     

Distinct neural correlates of associative working memory and long-term memory encoding in the medial temporal lobe

Increasing evidence suggests a role for the hippocampus not only in long-term memory (LTM) but also in relational working memory (WM) processes, challenging the view of the hippocampus as being solely involved in episodic LTM. However, hippocampal involvement reported in some neuroimaging studies using "classical" WM tasks may at least partly reflect incidental LTM encoding. To disentangle WM processing and LTM formation we administered a delayed-match-to-sample associative WM task in an event-related fMRI study design. Each trial of the WM task consisted of four pairs of faces and houses, which had to be maintained during a delay of 10s. This was followed by a probe phase consisting of three consecutively presented pairs; for each pair participants were to indicate whether it matched one of the pairs of the encoding phase. After scanning, an unexpected recognition-memory (LTM) task was administered. Brain activity during encoding was analyzed based on WM and LTM performance. Hence, encoding-related activity predicting WM success in the absence of successful LTM formation could be isolated. Furthermore, regions critical for successful LTM formation for pairs previously correctly processed in WM were analyzed. Results showed that the left parahippocampal gyrus including the fusiform gyrus predicted subsequent accuracy on WM decisions. The right anterior hippocampus and left inferior frontal gyrus, in contrast, predicted successful LTM for pairs that were previously correctly classified in the WM task. Our results suggest that brain regions associated with higher-level visuo-perceptual processing are involved in successful associative WM encoding, whereas the anterior hippocampus and left inferior frontal gyrus are involved in successful LTM formation during incidental encoding.     

Effects of semantic relatedness on age-related associative memory deficits: the role of theta oscillations

Growing evidence suggests that age-related deficits in associative memory are alleviated when the to-be-associated items are semantically related. Here we investigate whether this beneficial effect of semantic relatedness is paralleled by spatio-temporal changes in cortical EEG dynamics during incidental encoding. Young and older adults were presented with faces at a particular spatial location preceded by a biographical cue that was either semantically related or unrelated. As expected, automatic encoding of face-location associations benefited from semantic relatedness in the two groups of age. This effect correlated with increased power of theta oscillations over medial and anterior lateral regions of the prefrontal cortex (PFC) and lateral regions of the posterior parietal cortex (PPC) in both groups. But better-performing elders also showed increased brain-behavior correlation in the theta band over the right inferior frontal gyrus (IFG) as compared to young adults. Semantic relatedness was, however, insufficient to fully eliminate age-related differences in associative memory. In line with this finding, poorer-performing elders relative to young adults showed significant reductions of theta power in the left IFG that were further predictive of behavioral impairment in the recognition task. All together, these results suggest that older adults benefit less than young adults from executive processes during encoding mainly due to neural inefficiency over regions of the left ventrolateral prefrontal cortex (VLPFC). But this associative deficit may be partially compensated for by engaging preexistent semantic knowledge, which likely leads to an efficient recruitment of attentional and integration processes supported by the left PPC and left anterior PFC respectively, together with neural compensatory mechanisms governed by the right VLPFC.     

The role of frontopolar cortex in subgoal processing during working memory

Neuroimaging studies have implicated the anterior-most or frontopolar regions of prefrontal cortex (FP-PFC, e.g., Brodmann's Area 10) as playing a central role in higher cognitive functions such as planning, problem solving, reasoning, and episodic memory retrieval. The current functional magnetic resonance imaging (fMRI) study tested the hypothesis that FP-PFC subserves processes related to the monitoring and management of subgoals, while maintaining information in working memory (WM). Subjects were scanned while performing two variants of a simple delayed response WM task. In the control WM condition, subjects monitored for the presence of a specific concrete probe word (LIME) occurring following a specific abstract cue word (FATE). In the subgoal WM condition, subjects monitored for the presence of any concrete probe word immediately following any abstract cue word. Thus, the task required semantic classification of the probe word (the subgoal task), while the cue was simultaneously maintained in WM, so that both pieces of information could be integrated into a target determination. In a second control condition, subjects performed abstract/concrete semantic classification without WM demands. A region within right FP-PFC was identified which showed significant activation during the subgoal WM condition, but no activity in either of the two control conditions. However, this FP-PFC region was not modulated by direct manipulation of active maintenance demands. In contrast, left dorsolateral PFC was affected by active maintenance demands, but the effect did not interact with the presence of a subgoal task. Finally, left ventral PFC regions showed activation in response to semantic classification, but were not affected by WM demands. These results suggest a triple dissociation of function within PFC regions, and further indicate that FP-PFC is selectively engaged by the requirement to monitor and integrate subgoals during WM tasks.     

Neuroimaging a single thought: dorsolateral PFC activity associated with refreshing just-activated information.

Neuroimaging studies of human working memory (WM) show conflicting results regarding whether dorsolateral prefrontal cortex (PFC) contributes to maintaining information in consciousness or is recruited primarily when information must be manipulated. Using functional magnetic resonance imaging (fMRI), we looked at a minimal maintenance process--thinking back to a single, just-seen stimulus (refreshing). We found greater activity in left dorsolateral PFC (BA9) when participants refreshed a word compared to reading a word once or a second time. Furthermore, recognition memory was subsequently more accurate and faster for items that had been refreshed, demonstrating that a single thought that maintains activation can have consequences for long-term memory. Our fMRI results call into question any class of models of the functional organization of PFC and WM that associates simple and/or maintenance processes only with ventrolateral PFC or that associates dorsolateral PFC only with more complex processes such as manipulation.     

Functional network in the prefrontal cortex during episodic memory retrieval

A recent consistent finding in neuroimaging studies of human memory is that the prefrontal cortex (PFC) is activated during episodic memory retrieval. To date, however, there has been no direct evidence to explain how activity in the right and left PFC and in the anterior and posterior PFC are functionally interconnected. The goal of the present study was to obtain such evidence by event-related functional magnetic resonance imaging (MRI) and the functional connectivity method. Subjects were first asked to try to remember a series of associate-word lists outside the MRI scanner in preparation for a later recognition test. In the MRI scanning phase, they were asked to make recognition judgments in regard to old words, semantically related lure words, and unrelated new words. The analysis of functional connectivity revealed that the posterior PFC in each hemisphere had strong functional interconnections with the contralateral posterior PFC, whereas the anterior PFC in each hemisphere had only weak functional interconnections with the contralateral anterior PFC. No strong functional interconnections were found between the anterior and posterior PFC in either hemisphere. These findings support the hypothesis of an associative contribution of the bilateral posterior PFC to episodic memory retrieval and a dissociative contribution of the bilateral anterior PFC.     

A face to remember: emotional expression modulates prefrontal activity during memory formation

Emotion can exert a modulatory role on episodic memory. Several studies have shown that negative stimuli (e.g., words, pictures) are better remembered than neutral ones. Although facial expressions are powerful emotional stimuli and have been shown to influence perception and attention processes, little is known about their effect on memory. We used functional magnetic resonance imaging (fMRI) in humans to investigate the effects of expression (happy, neutral, and fearful) on prefrontal cortex (PFC) activity during the encoding of faces, using a subsequent memory effect paradigm. Our results show that activity in right PFC predicted memory for faces, regardless of expression, while a homotopic region in the left hemisphere was associated with successful encoding only for faces with an emotional expression. These findings are consistent with the proposed role of right dorsolateral PFC in successful encoding of nonverbal material, but also suggest that left DLPFC may be a site where integration of memory and emotional processes occurs. This study sheds new light on the current controversy regarding the hemispheric lateralization of PFC in memory encoding.     

Sex differences in the medial temporal lobe during encoding and recognition of pseudowords and abstract designs

Sex differences in various cognitive abilities have been demonstrated in terms of performance differentials and, more recently, in differences in activation patterns during fMRI. Hemispheric lateralization is sometimes accentuated in sex differences; e.g., women demonstrating greater activation of the left hemisphere than men during verbal tasks. We were interested in whether this phenomenon applies to memory for words and designs (i.e., material specific memory). Using analogous verbal (pseudowords) and nonverbal (abstract designs) encoding and recognition tasks completed back-to-back in a 3T scanner, we found that women tend to show exaggerated left hippocampal activation during certain stages of encoding and recognition of verbal information, compared with men. Likewise (although to a lesser extent), men showed more right hippocampal activation than women did during the abstract design learning task. These results have important implications for the generalization of fMRI memory study results, for example to clinical populations such as patients undergoing epilepsy surgery.     

Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: an event-related fMRI study

Prefrontal cortex (PFC) activity associated with emotional evaluation and subsequent memory was investigated with event-related functional MRI (fMRI). Participants were scanned while rating the pleasantness of emotionally positive, negative, and neutral pictures, and memory for the pictures was tested after scanning. Emotional evaluation was measured by comparing activity during the picture rating task relative to baseline, and successful encoding was measured by comparing activity for subsequently remembered versus forgotten pictures (Dm effect). The effect of arousal on these measures was indicated by greater activity for both positive and negative pictures than for neutral ones, and the effect of valence was indicated by differences in activity between positive and negative pictures. The study yielded three main results. First, consistent with the valence hypothesis, specific regions in left dorsolateral PFC were more activated for positive than for negative picture evaluation, whereas regions in right ventrolateral PFC showed the converse pattern. Second, dorsomedial PFC activity was sensitive to emotional arousal, whereas ventromedial PFC activity was sensitive to positive valence, consistent with evidence linking these regions, respectively, to emotional processing and self-awareness or appetitive behavior. Finally, successful encoding (Dm) activity in left ventrolateral and dorsolateral PFC was greater for arousing than for neutral pictures. This finding suggests that the enhancing effect of emotion on memory formation is partly due to an augmentation of PFC-mediated strategic, semantic, and working memory operations. These results underscore the critical role of PFC in emotional evaluation and memory, and disentangle the effects of arousal and valence across PFC regions associated with different cognitive functions.     

Effective auditory-verbal encoding activates the left prefrontal and the medial temporal lobes: A generalization to illiterate subjects.

Recent event-related FMRI studies indicate that the prefrontal (PFC) and the medial temporal lobe (MTL) regions are more active during effective encoding than during ineffective encoding. The within-subject design and the use of well-educated young college students in these studies makes it important to replicate these results in other study populations. In this PET study, we used an auditory word-pair association cued-recall paradigm and investigated a group of healthy upper middle-aged/older illiterate women. We observed a positive correlation between cued-recall success and the regional cerebral blood flow of the left inferior PFC (BA 47) and the MTLs. Specifically, we used the cued-recall success as a covariate in a general linear model and the results confirmed that the left inferior PFC and the MTL are more active during effective encoding than during ineffective encoding. These effects were observed during encoding of both semantically and phonologically related word pairs, indicating that these effects are robust in the studied population, that is, reproducible within group. These results generalize the results of Brewer et al. (1998, Science 281, 1185-1187) and Wagner et al. (1998, Science 281, 1188-1191) to an upper middle aged/older illiterate population. In addition, the present study indicates that effective relational encoding correlates positively with the activity of the anterior medial temporal lobe regions.     

Hemispheric Activation of Anterior and Inferior Prefrontal Cortex during Verbal Encoding and Recognition: A PET Study of Healthy Volunteers

Evidence of bilateral prefrontal activation during memory encoding and retrieval has increased attention given to anatomical subdivisions within the prefrontal cortex. The current study examined anterior and inferior aspects of the prefrontal cortex to determine their degree of functional and hemispheric overlap during encoding and recognition. Cerebral blood flow of 25 healthy volunteers was measured using PET ¹⁵O-water methods during four conditions: resting baseline, sequential finger movement, word encoding, and word recognition. Resting and motor images were averaged to provide a single reference that was subtracted from encoding and recognition using statistical parametric mapping (SPM96). Memory conditions were also subtracted from each other to identify differences in regional activity. Subjects performed well (86% correct) and had a slightly conservative response bias. Baseline subtraction from encoding revealed focal activation of left inferior prefrontal cortex (area 45) without significant contralateral activation. Recognition minus baseline subtraction produced a focal right anterior prefrontal activation (areas 9 and 10) that was not present in the left hemisphere. Bilateral effects were seen in area 45 during recognition. Subtraction of memory tasks from each other did not reveal any areas of greater activity during encoding. However, the recognition task produced greater activation in right area 9 extending into the anterior cingulate. Greater activity during recognition was also observed in left insula and bilateral visual integration areas. These results are discussed in relation to the prevailing model of prefrontal hemispheric asymmetry during episodic memory.     

II. PET Studies of Memory: Novel versus Practiced Free Recall of Word Lists

Positron emission tomography (PET) with the tracer Ha₂¹⁵O was used to measure regional cerebral blood flow in 13 healthy volunteers while they engaged in free recall of 15-item word lists from the Rey Auditory Verbal Learning task. The study was designed so that recall of well-practiced versus novel material could be compared. One week before the PET study, subjects were trained to perfect recall of List A, while they were exposed to list B only 60s prior to PET data acquisition. As in the companion study of free recall of complex narratives, we observed that practice tended to decrease the size of activations in regions involved in the memory component of the task; we also observed that the novel recall task produced greater activation in left frontal regions, probably due to active encoding. A commonality of other regions observed in this pair of studies, as well as other studies of memory in the literature, suggests that the human brain may contain a distributed multinodal general memory system. Nodes on this network include the frontal, parietal, and temporal cortices, the thalamus, the anterior and posterior cingulate, the precuneus, and the cerebellum. There appears to be a commonality of components across tasks (e.g., retrieval, encoding) that is independent of content, as well as differentiation of some components that may be content-specific or task-specific. In addition, these results support a significant role for the cerebellum in cognitive functions such as memory.     

Task constraints modulate activation in right ventral lateral prefrontal cortex

Lesion data suggest that right prefrontal cortex (PFC) plays a critical role in open-ended problem solving. To test this hypothesis, we scanned fifteen normal subjects with fMRI as they completed three types of anagram problems varying in the level of constraints placed on the search space. On unconstrained trials, they rearranged letters to generate solutions (e.g., Can you make a "Word with ZJAZ?"). On semantically constrained trials, they rearranged letters to generate solutions within particular semantic categories (e.g., Can you make a type of "Music with ZJAZ?"). On baseline trials, they rearranged letters to make specific words (e.g., Can you make the word "JAZZ with ZJAZ?"). As predicted, the critical comparison of unconstrained vs. semantically constrained trials revealed significant activation in right ventral lateral PFC, as well as left superior frontal gyrus, frontopolar cortex, right superior parietal lobe, right post central gyrus, and the occipital-parietal sulcus. Furthermore, activation in right ventral lateral PFC (BA 47) increased as the constraints placed on the anagram search space were reduced. We argue that the activation in right ventral lateral PFC is related to hypothesis generation in unconstrained settings, whereas activation in other structures is related to additional processes linked to anagram problems such as semantic retrieval, semantic categorization, and cognitive monitoring. These results extend the lesion data and imaging studies by demonstrating that a relative absence of constraints on the solution space is sufficient to engage right ventral lateral PFC in hypothesis generation tasks.     

Hippocampal activations during encoding and retrieval in a verbal working memory paradigm

Though the hippocampus has been associated with encoding and retrieval processes in episodic memory, the precise nature of its involvement in working memory has yet to be determined. This functional magnetic resonance imaging (fMRI) study employed a verbal working memory paradigm that allows for the within-subject comparison of functional activations during encoding, maintenance, and retrieval. In each trial, participants were shown 5 target words and, after an 8 s delay, a series of probe words. Probe words consisted of target matches, phonetically or semantically related foils, or foils unrelated to the target words. Both the left and right hippocampi showed higher mean activation amplitudes during encoding than maintenance. In contrast, the right dorsolateral prefrontal cortex (DLPFC) showed greater activation during maintenance than encoding. Both hippocampal and DLPFC regions were more active during retrieval than maintenance. Furthermore, an analysis of retrieval activation separated by probe type showed a trend toward greater bilateral hippocampal activation for probes related (both semantically and phonetically) to the target than for unrelated probes and still greater activation for target matches. This pattern suggests that there may be roles for the hippocampus and DLPFC in working memory that change as function of information processing stage. Additionally, the trend towards increased involvement of the hippocampus with the increase in relatedness of the probe stimuli to the information maintained is interpreted to be consistent with the role of the hippocampus in recollection-based retrieval in long-term memory and may indicate that this role extends to working memory processes.