Rapid onset relational memory effects are evident in eye movement behavior, but not in hippocampal amnesia

Little is known about the mechanisms by which memory for relations is accomplished, or about the time course of the critical processes. Here, eye movement measures were used to examine the time course of subjects' access to and use of relational memory. In four experiments, participants studied faces superimposed on scenic backgrounds and were tested with three-face displays superimposed on the scenes viewed earlier. Participants exhibited disproportionate viewing of the face originally studied with the scene, compared to other equally familiar faces in the test display. When a preview of a previously viewed scene was provided, permitting expectancies about the to-be-presented face to emerge, disproportionate viewing was manifested within 500-750 msec after test display onset, more than a full second in advance of explicit behavioral responses, and occurred even when overt responses were not required. In the absence of preview, the viewing effects were delayed by approximately 1 sec. Relational memory effects were absent in the eye movement behavior of amnesic patients with hippocampal damage, suggesting that these effects depend critically on the hippocampal system. The results provide an index of memory for face-scene relations, indicate the time by which retrieval and identification of these relations occur, and suggest that retrieval and use of relational memory depends critically on the hippocampus and occurs obligatorily, regardless of response requirements.     

Hippocampal‐cortical contributions to strategic exploration during perceptual discrimination

The hippocampus is crucial for long‐term memory; its involvement in short‐term or immediate expressions of memory is more controversial. Rodent hippocampus has been implicated in an expression of memory that occurs on‐line during exploration termed “vicarious trial‐and‐error” (VTE) behavior. VTE occurs when rodents iteratively explore options during perceptual discrimination or at choice points. It is strategic in that it accelerates learning and improves later memory. VTE has been associated with activity of rodent hippocampal neurons, and lesions of hippocampus disrupt VTE and associated learning and memory advantages. Analogous findings of VTE in humans would support the role of hippocampus in active use of short‐term memory to guide strategic behavior. We therefore measured VTE using eye‐movement tracking during perceptual discrimination and identified relevant neural correlates with functional magnetic resonance imaging. A difficult perceptual‐discrimination task was used that required visual information to be maintained during a several second trial, but with no long‐term memory component. VTE accelerated discrimination. Neural correlates of VTE included robust activity of hippocampus and activity of a network of medial prefrontal and lateral parietal regions involved in memory‐guided behavior. This VTE‐related activity was distinct from activity associated with simply viewing visual stimuli and making eye movements during the discrimination task, which occurred in regions frequently associated with visual processing and eye‐movement control. Subjects were mostly unaware of performing VTE, thus further distancing VTE from explicit long‐term memory processing. These findings bridge the rodent and human literatures on neural substrates of memory‐guided behavior, and provide further support for the role of hippocampus and a hippocampal‐centered network of cortical regions in the immediate use of memory in on‐line processing and the guidance of behavior.     

Aerobic fitness enhances relational memory in preadolescent children: The FITKids randomized control trial

It is widely accepted that aerobic exercise enhances hippocampal plasticity. Often, this plasticity co‐occurs with gains in hippocampal‐dependent memory. Cross‐sectional work investigating this relationship in preadolescent children has found behavioral differences in higher versus lower aerobically fit participants for tasks measuring relational memory, which is known to be critically tied to hippocampal structure and function. The present study tested whether similar differences would arise in a clinical intervention setting where a group of preadolescent children were randomly assigned to a 9‐month after school aerobic exercise intervention versus a wait‐list control group. Performance measures included eye‐movements as a measure of memory, based on recent work linking eye‐movement indices of relational memory to the hippocampus. Results indicated that only children in the intervention increased their aerobic fitness. Compared to the control group, those who entered the aerobic exercise program displayed eye‐movement patterns indicative of superior memory for face‐scene relations, with no differences observed in memory for individual faces. The results of this intervention study provide clear support for the proposed linkage among the hippocampus, relational memory, and aerobic fitness, as well as illustrating the sensitivity of eye‐movement measures as a means of assessing memory. © 2012 Wiley Periodicals, Inc.     

Hippocampal gamma‐band Synchrony and pupillary responses index memory during visual search

Memory for scenes is supported by the hippocampus, among other interconnected structures, but the neural mechanisms related to this process are not well understood. To assess the role of the hippocampus in memory‐guided scene search, we recorded local field potentials and multiunit activity from the hippocampus of macaques as they performed goal‐directed search tasks using natural scenes. We additionally measured pupil size during scene presentation, which in humans is modulated by recognition memory. We found that both pupil dilation and search efficiency accompanied scene repetition, thereby indicating memory for scenes. Neural correlates included a brief increase in hippocampal multiunit activity and a sustained synchronization of unit activity to gamma band oscillations (50–70 Hz). The repetition effects on hippocampal gamma synchronization occurred when pupils were most dilated, suggesting an interaction between aroused, attentive processing and hippocampal correlates of recognition memory. These results suggest that the hippocampus may support memory‐guided visual search through enhanced local gamma synchrony. © 2016 Wiley Periodicals, Inc.     

Neural correlates of contextual cueing are modulated by explicit learning

Contextual cueing refers to the facilitated ability to locate a particular visual element in a scene due to prior exposure to the same scene. This facilitation is thought to reflect implicit learning, as it typically occurs without the observer's knowledge that scenes repeat. Unlike most other implicit learning effects, contextual cueing can be impaired following damage to the medial temporal lobe. Here we investigated neural correlates of contextual cueing and explicit scene memory in two participant groups. Only one group was explicitly instructed about scene repetition. Participants viewed a sequence of complex scenes that depicted a landscape with five abstract geometric objects. Superimposed on each object was a letter T or L rotated left or right by 90°. Participants responded according to the target letter (T) orientation. Responses were highly accurate for all scenes. Response speeds were faster for repeated versus novel scenes. The magnitude of this contextual cueing did not differ between the two groups. Also, in both groups repeated scenes yielded reduced hemodynamic activation compared with novel scenes in several regions involved in visual perception and attention, and reductions in some of these areas were correlated with response-time facilitation. In the group given instructions about scene repetition, recognition memory for scenes was superior and was accompanied by medial temporal and more anterior activation. Thus, strategic factors can promote explicit memorization of visual scene information, which appears to engage additional neural processing beyond what is required for implicit learning of object configurations and target locations in a scene.     

The hippocampus uses information just encountered to guide efficient ongoing behavior

Adaptive ongoing behavior requires using immediate sensory input to guide upcoming actions. Using a novel paradigm with volitional exploration of visuo‐spatial scenes, we revealed novel deficits among hippocampal amnesic patients in effective spatial exploration of scenes, indicated by less‐systematic exploration patterns than those of healthy comparison subjects. The disorganized exploration by amnesic patients occurred despite successful retention of individual object locations across the entire exploration period, indicating that exploration impairments were not secondary to rapid decay of scene information. These exploration deficits suggest that amnesic patients are impaired in integrating memory for recent actions, which may include information such as locations just visited and scene content, to plan immediately forthcoming actions. Using a novel task that measured the on‐line links between sensory input and behavior, we observed the critical role of the hippocampus in modulating ongoing behavior. © 2013 Wiley Periodicals, Inc.     

Reactivation during encoding supports the later discrimination of similar episodic memories

Episodic memory is characterized by remembering events as unique combinations of features. Even when some features of events overlap, we are later often able to discriminate among them. Here we ask whether hippocampally mediated reactivation of an earlier event when a similar one occurs supports subsequent memory that two similar but not identical events occurred (mnemonic discrimination). In two experiments, participants viewed objects (Experiment 1) or scenes (Experiment 2) during functional MRI (fMRI). After scanning, participants had to remember whether repeated items had been identical or similar. In Experiment 2, representational similarity between the 1st and 2nd presentation predicted participants' ability to remember that the presentations were different, suggesting that the first item was reactivated while viewing the second. A similar but weaker result was found in Experiment 1 that did not survive correction for multiple comparisons. Furthermore, both experiments yielded evidence that the hippocampus was involved in reactivation; hippocampal pattern similarity (and, in Experiment 2, hippocampal activity during the 2nd presentation) correlated with pattern similarity in several regions of visual cortex. These results provide the first fMRI evidence that hippocampally mediated reactivation contributes to the later memory that two similar, but different events occurred. © 2016 Wiley Periodicals, Inc.     

Eye Movements Show Optimal Average Anticipation with Natural Dynamic Scenes

A less studied component of gaze allocation in dynamic real-world scenes is the time lag of eye movements in responding to dynamic attention-capturing events. Despite the vast amount of research on anticipatory gaze behaviour in natural situations, such as action execution and observation, little is known about the predictive nature of eye movements when viewing different types of natural or realistic scene sequences. In the present study, we quantify the degree of anticipation during the free viewing of dynamic natural scenes. The cross-correlation analysis of image-based saliency maps with an empirical saliency measure derived from eye movement data reveals the existence of predictive mechanisms responsible for a near-zero average lag between dynamic changes of the environment and the responding eye movements. We also show that the degree of anticipation is reduced when moving away from natural scenes by introducing camera motion, jump cuts, and film-editing.     

Decoding representations of scenes in the medial temporal lobes

Recent theoretical perspectives have suggested that the function of the human hippocampus, like its rodent counterpart, may be best characterized in terms of its information processing capacities. In this study, we use a combination of high-resolution functional magnetic resonance imaging, multivariate pattern analysis, and a simple decision making task, to test specific hypotheses concerning the role of the medial temporal lobe (MTL) in scene processing. We observed that while information that enabled two highly similar scenes to be distinguished was widely distributed throughout the MTL, more distinct scene representations were present in the hippocampus, consistent with its role in performing pattern separation. As well as viewing the two similar scenes, during scanning participants also viewed morphed scenes that spanned a continuum between the original two scenes. We found that patterns of hippocampal activity during morph trials, even when perceptual inputs were held entirely constant (i.e., in 50% morph trials), showed a robust relationship with participants' choices in the decision task. Our findings provide evidence for a specific computational role for the hippocampus in sustaining detailed representations of complex scenes, and shed new light on how the information processing capacities of the hippocampus may influence the decision making process.     

Intact CA3 in the hippocampus is only sufficient for contextual behavior based on well‐learned and unaltered visual background

Computational models suggest that the dentate gyrus and CA3 subfields of the hippocampus are responsible for discrete memory representations using pattern separation and pattern completion when a modified external stimulus is recognized as an old memory or encoded as a new memory. Experimental evidence of such computational processes in the hippocampus has been obtained mostly from spatial navigational tasks, and little is known about the proposed computational functions of the hippocampal subfields in “nonspatial” memory tasks. We tested whether rats with major damage in the dentate gyrus induced by colchicine lesions could remember patterned visual scene stimuli presented on LCD screens in the background. Rats responded using a touchscreen to indicate the identity of the visual scene. Performance of the lesion group was normal when tested with familiar visual scenes that had been learned prior to surgery. Lesioned rats exhibited severe deficits in learning novel visual scenes, but eventually reached the same level of performance as controls. However, unlike in controls, novel scene‐associated memories formed in the lesion group were unstable and easily disrupted when ambiguous versions of the novel scenes were presented intermixed with the original stimuli. Our findings confirm that the prior computational models can also be applied to the nonspatial memory domain and suggest that the dentate gyrus is not necessary for the retrieval of learned visual scene‐associated behavioral responses but plays a crucial role in forming novel visual scene‐dependent memory and recognizing altered or ambiguous visual scenes in the background. © 2014 Wiley Periodicals, Inc.     

Immediate memory for “when,where and what”: Short‐delay retrieval using dynamic naturalistic material

We investigated the neural correlates supporting three kinds of memory judgments after very short delays using naturalistic material. In two functional magnetic resonance imaging (fMRI) experiments, subjects watched short movie clips, and after a short retention (1.5–2.5 s), made mnemonic judgments about specific aspects of the clips. In Experiment 1, subjects were presented with two scenes and required to either choose the scene that happened earlier in the clip (“scene‐chronology”), or with a correct spatial arrangement (“scene‐layout”), or that had been shown (“scene‐recognition”). To segregate activity specific to seen versus unseen stimuli, in Experiment 2 only one probe image was presented (either target or foil). Across the two experiments, we replicated three patterns underlying the three specific forms of memory judgment. The precuneus was activated during temporal‐order retrieval, the superior parietal cortex was activated bilaterally for spatial‐related configuration judgments, whereas the medial frontal cortex during scene recognition. Conjunction analyses with a previous study that used analogous retrieval tasks, but a much longer delay (>1 day), demonstrated that this dissociation pattern is independent of retention delay. We conclude that analogous brain regions mediate task‐specific retrieval across vastly different delays, consistent with the proposal of scale‐invariance in episodic memory retrieval. Hum Brain Mapp 36:2495–2513, 2015. © 2015 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.     

Oxytocin increases amygdala reactivity to threatening scenes in females

The neuropeptide oxytocin (OT) is well known for its profound effects on social behavior, which appear to be mediated by an OT-dependent modulation of amygdala activity in the context of social stimuli. In humans, OT decreases amygdala reactivity to threatening faces in males, but enhances amygdala reactivity to similar faces in females, suggesting sex-specific differences in OT-dependent threat-processing. To further explore whether OT generally enhances amygdala-dependent threat-processing in females, we used functional magnetic resonance imaging (fMRI) in a randomized within-subject crossover design to measure amygdala activity in response to threatening and non-threatening scenes in 14 females following intranasal administration of OT or placebo. Participants' eye movements were recorded to investigate whether an OT-dependent modulation of amygdala activity is accompanied by enhanced exploration of salient scene features. Although OT had no effect on participants' gazing behavior, it increased amygdala reactivity to scenes depicting social and non-social threat. In females, OT may, thus, enhance the detection of threatening stimuli in the environment, potentially by interacting with gonadal steroids, such as progesterone and estrogen.     

How emotion leads to selective memory: neuroimaging evidence

Often memory for emotionally arousing items is enhanced relative to neutral items within complex visual scenes, but this enhancement can come at the expense of memory for peripheral background information. This ‘trade-off’ effect has been elicited by a range of stimulus valence and arousal levels, yet the magnitude of the effect has been shown to vary with these factors. Using fMRI, this study investigated the neural mechanisms underlying this selective memory for emotional scenes. Further, we examined how these processes are affected by stimulus dimensions of arousal and valence. The trade-off effect in memory occurred for low to high arousal positive and negative scenes. There was a core emotional memory network associated with the trade-off among all the emotional scene types, however, there were additional regions that were uniquely associated with the trade-off for each individual scene type. These results suggest that there is a common network of regions associated with the emotional memory trade-off effect, but that valence and arousal also independently affect the neural activity underlying the effect.     

Deciding what is possible and impossible following hippocampal damage in humans

There is currently much debate about whether the precise role of the hippocampus in scene processing is predominantly constructive, perceptual, or mnemonic. Here, we developed a novel experimental paradigm designed to control for general perceptual and mnemonic demands, thus enabling us to specifically vary the requirement for constructive processing. We tested the ability of patients with selective bilateral hippocampal damage and matched control participants to detect either semantic (e.g., an elephant with butterflies for ears) or constructive (e.g., an endless staircase) violations in realistic images of scenes. Thus, scenes could be semantically or constructively ‘possible’ or ‘impossible’. Importantly, general perceptual and memory requirements were similar for both types of scene. We found that the patients performed comparably to control participants when deciding whether scenes were semantically possible or impossible, but were selectively impaired at judging if scenes were constructively possible or impossible. Post‐task debriefing indicated that control participants constructed flexible mental representations of the scenes in order to make constructive judgements, whereas the patients were more constrained and typically focused on specific fragments of the scenes, with little indication of having constructed internal scene models. These results suggest that one contribution the hippocampus makes to scene processing is to construct internal representations of spatially coherent scenes, which may be vital for modelling the world during both perception and memory recall. © 2016 The Authors. Hippocampus Published by Wiley Periodicals, Inc.     

Parietal cortex integrates contextual and saliency signals during the encoding of natural scenes in working memory

The Brief presentation of a complex scene entails that only a few objects can be selected, processed indepth, and stored in memory. Both low‐level sensory salience and high‐level context‐related factors (e.g., the conceptual match/mismatch between objects and scene context) contribute to this selection process, but how the interplay between these factors affects memory encoding is largely unexplored. Here, during fMRI we presented participants with pictures of everyday scenes. After a short retention interval, participants judged the position of a target object extracted from the initial scene. The target object could be either congruent or incongruent with the context of the scene, and could be located in a region of the image with maximal or minimal salience. Behaviourally, we found a reduced impact of saliency on visuospatial working memory performance when the target was out‐of‐context. Encoding‐related fMRI results showed that context–congruent targets activated dorsoparietal regions, while context–incongruent targets de‐activated the ventroparietal cortex. Saliency modulated activity both in dorsal and ventral regions, with larger context‐related effects for salient targets. These findings demonstrate the joint contribution of knowledge‐based and saliency‐driven attention for memory encoding, highlighting a dissociation between dorsal and ventral parietal regions. Hum Brain Mapp 36:5003–5017, 2015. © 2015 Wiley Periodicals, Inc .     

A comparison of two fMRI methods for predicting verbal memory decline after left temporal lobectomy: Language lateralization versus hippocampal activation asymmetry

Purpose: Language lateralization measured by preoperative functional magnetic resonance imaging (fMRI) was shown recently to be predictive of verbal memory outcome in patients undergoing left anterior temporal lobe (L‐ATL) resection. The aim of this study was to determine whether language lateralization or functional lateralization in the hippocampus is a better predictor of outcome in this setting. Methods: Thirty L‐ATL patients underwent preoperative language fMRI, preoperative hippocampal fMRI using a scene encoding task, and pre‐ and postoperative neuropsychological testing. A group of 37 right ATL (R‐ATL) surgery patients was included for comparison. Results: Verbal memory decline occurred in roughly half of the L‐ATL patients. Preoperative language lateralization was correlated with postoperative verbal memory change. Hippocampal activation asymmetry was strongly related to side of seizure focus and to Wada memory asymmetry but was unrelated to verbal memory outcome. Discussion: Preoperative hippocampal activation asymmetry elicited by a scene encoding task is not predictive of verbal memory outcome. Risk of verbal memory decline is likely to be related to lateralization of material‐specific verbal memory networks, which are more closely correlated with language lateralization than with overall asymmetry of episodic memory processes.     

Frequency‐specific noninvasive modulation of memory retrieval and its relationship with hippocampal network connectivity

Episodic memory is thought to rely on interactions of the hippocampus with other regions of the distributed hippocampal‐cortical network (HCN) via interregional activity synchrony in the theta frequency band. We sought to causally test this hypothesis using network‐targeted transcranial magnetic stimulation. Healthy human participants completed four experimental sessions, each involving a different stimulation pattern delivered to the same individualized parietal cortex location of the HCN for all sessions. There were three active stimulation conditions, including continuous theta‐burst stimulation, intermittent theta‐burst stimulation, and beta‐frequency (20‐Hz) repetitive stimulation, and one sham condition. Resting‐state fMRI and episodic memory testing were used to assess the impact of stimulation on hippocampal fMRI connectivity related to retrieval success. We hypothesized that theta‐burst stimulation conditions would most strongly influence hippocampal‐HCN fMRI connectivity and retrieval, given the hypothesized relevance of theta‐band activity for HCN memory function. Continuous theta‐burst stimulation improved item retrieval success relative to sham and relative to beta‐frequency stimulation, whereas intermittent theta‐burst stimulation led to numerical but nonsignificant item retrieval improvement. Mean hippocampal fMRI connectivity did not vary for any stimulation conditions, whereas individual differences in retrieval improvements due to continuous theta‐burst stimulation were associated with corresponding increases in fMRI connectivity between the hippocampus and other HCN locations. No such memory‐related connectivity effects were identified for the other stimulation conditions, indicating that only continuous theta‐burst stimulation affected memory‐related hippocampal‐HCN connectivity. Furthermore, these effects were specific to the targeted HCN, with no significant memory‐related fMRI connectivity effects for two distinct control brain networks. These findings support a causal role for fMRI connectivity of the hippocampus with the HCN in episodic memory retrieval and indicate that contributions of this network to retrieval are particularly sensitive to continuous theta‐burst noninvasive stimulation.     

Intracranial EEG correlates of implicit relational inference within the hippocampus

Drawing inferences from past experiences enables adaptive behavior in future situations. Inference has been shown to depend on hippocampal processes. Usually, inference is considered a deliberate and effortful mental act which happens during retrieval, and requires the focus of our awareness. Recent fMRI studies hint at the possibility that some forms of hippocampus‐dependent inference can also occur during encoding and possibly also outside of awareness. Here, we sought to further explore the feasibility of hippocampal implicit inference, and specifically address the temporal evolution of implicit inference using intracranial EEG. Presurgical epilepsy patients with hippocampal depth electrodes viewed a sequence of word pairs, and judged the semantic fit between two words in each pair. Some of the word pairs entailed a common word (e.g., “winter–red,” “red–cat”) such that an indirect relation was established in following word pairs (e.g., “winter–cat”). The behavioral results suggested that drawing inference implicitly from past experience is feasible because indirect relations seemed to foster “fit” judgments while the absence of indirect relations fostered “do not fit” judgments, even though the participants were unaware of the indirect relations. A event‐related potential (ERP) difference emerging 400 ms post‐stimulus was evident in the hippocampus during encoding, suggesting that indirect relations were already established automatically during encoding of the overlapping word pairs. Further ERP differences emerged later post‐stimulus (1,500 ms), were modulated by the participants' responses and were evident during encoding and test. Furthermore, response‐locked ERP effects were evident at test. These ERP effects could hence be a correlate of the interaction of implicit memory with decision‐making. Together, the data map out a time‐course in which the hippocampus automatically integrates memories from discrete but related episodes to implicitly influence future decision making. © 2015 Wiley Periodicals, Inc.     

Scene construction in amnesia: an FMRI study

In recent years, there has been substantial interest in how the human hippocampus not only supports recollection of past experiences, but also the construction of fictitious and future events, and the leverage this might offer for understanding the operating mechanisms of the hippocampus. Evidence that patients with bilateral hippocampal damage and amnesia cannot construct novel or future scenes/events has been influential in driving this line of research forward. There are, however, some patients with hippocampal damage and amnesia who retain the ability to construct novel scenes. This dissociation may indicate that the hippocampus is not required for scene construction, or alternatively, there could be residual function in remnant hippocampal tissue sufficient to support the basic construction of scenes. Resolving this controversy is central to current theoretical debates about the hippocampus. To investigate, we used fMRI and a scene construction task to test patient P01, who has dense amnesia, ～50% bilateral hippocampal volume loss, and intact scene construction. We found that scene construction in P01 was associated with increased activity in a set of brain areas, including medial temporal, retrosplenial, and posterior parietal cortices, that overlapped considerably with the regions engaged in control participants performing the same task. Most notably, the remnant of P01's right hippocampus exhibited increased activity during scene construction. This suggests that the intact scene construction observed in some hippocampal-damaged amnesic patients may be supported by residual function in their lesioned hippocampus, in accordance with theoretical frameworks that ascribe a vital role to the hippocampus in scene construction.     

Functional dissociation of hippocampal mechanism during implicit learning based on the domain of associations

Traditionally, the medial temporal lobe (MTL) was linked to explicit or declarative memory in associative learning. However, recent studies have reported MTL involvement even when volunteers are not consciously aware of the learned contingencies. Therefore, the mechanism of the MTL-related learning process cannot be described sufficiently by the explicit/implicit distinction, and the underlying process in the MTL for associative learning needs a more functional characterization. A possible feature that would allow a functional specification also for implicit learning is the nature of the material that is learned. Given that implicit memory tasks often comprise a combination of perceptual and motor learning, we hypothesized that implicit learning of the perceptual but not the motor component entails MTL activation in these studies. To directly test this hypothesis, we designed a purely perceptual and a purely motor variant of the serial reaction time task. In two groups of human volunteers, behavioral results clearly showed that both variants were learned without awareness. Neuronal recordings using fMRI revealed that bilateral hippocampal activation was observed only for implicit learning of the perceptual sequence, not for the motor sequence. This dissociation clearly shows that the functional role of the hippocampus for learning is determined by the domain of the learned association and that the function of the medial temporal lobe system is the processing of contingencies between perceptual features regardless of the explicit or implicit nature of the ensuing memory.