The functional neuroanatomy of novelty processing: integrating ERP and fMRI results.

Recent research indicates that non-tonal novel events, deviating from an ongoing auditory environment, elicit a positive event-related potential (ERP), the novel P3. Although a variety of studies examined the neural network engaged in novelty detection, there is no complete picture of the underlying brain mechanisms. This experiment investigated these neural mechanisms by combining ERP and functional magnetic resonance imaging (fMRI). Hemodynamic and electrophysiological responses were measured in the same subjects using the same experimental design. The ERP analysis revealed a novel P3, while the fMRI responses showed bilateral foci in the middle part of the superior temporal gyrus. When subjects attended to the novel stimuli only identifiable novel sounds evoked a N4-like negativity. Subjects showing a strong N4-effect had additional fMRI activation in right prefrontal cortex (rPFC) as compared to subjects with a weak N4-effect. This pattern of results suggests that novelty processing not only includes the registration of deviancy but may also lead to a fast access and retrieval of related semantic concepts. The fMRI activation pattern suggests that the superior temporal gyrus is involved in novelty detection, whereas accessing and retrieving semantic concepts related to novel sounds additionally engages the rPFC.     

Brain activity underlying encoding and retrieval of source memory

Neural activity elicited during the encoding and retrieval of source information was investigated with event-related functional magnetic resonance imaging (efMRI). During encoding, 17 subjects performed a natural/artificial judgement on pictures of common objects which were presented randomly in one of the four quadrants of the display. At retrieval, old pictures were mixed with new ones and subjects judged whether each picture was new or old and, if old, indicated in which quadrant it was presented at encoding. During encoding, study items that were later recognized and assigned a correct source judgement elicited greater activity than recognized items given incorrect judgements in a variety of regions, including right lateral occipital and left prefrontal cortex. At retrieval, regions showing greater activity for recognized items given correct versus incorrect source judgements included the right hippocampal formation and the left prefrontal cortex. These findings indicate a role for these regions in the encoding and retrieval of episodic information beyond that required for simple item recognition.     

Task-dependency of the neural correlates of episodic encoding as measured by fMRI.

Using event-related functional magnetic resonance imaging (fMRI), the neural correlates of memory encoding can be studied by contrasting item-related activity elicited in a study task according to whether the items are remembered or forgotten in a subsequent memory test. Previous studies using this approach have implicated the left prefrontal cortex in the successful encoding of verbal material into episodic memory when the study task is semantic in nature. In the current study, we asked whether the neural correlates of episodic encoding differ depending on type of study task. Seventeen volunteers participated in an event-related fMRI experiment in which at study, volunteers were cued to make either animacy or syllable judgements about words. A recognition memory test followed after a delay of approximately 15 min. For the animacy task, words that were subsequently remembered showed greater activation in left and medial prefrontal regions. For the syllable task, by contrast, successful memory for words was associated with activations in bilateral intraparietal sulcus, bilateral fusiform gyrus, right prefrontal cortex and left superior occipital gyrus. These findings suggest that the brain networks supporting episodic encoding differ according to study task.     

Electrophysiological and haemodynamic correlates of face perception,recognition and priming

Face perception, recognition and priming were examined with event-related functional magnetic resonance imaging (fMRI) and scalp event-related potentials (ERPs). Face perception was associated with haemodynamic increases in regions including bilateral fusiform and right superior temporal cortices, and a right posterior negativity (N170), most likely generated in the superior temporal region. Face recognition was associated with haemodynamic increases in fusiform, medial frontal and orbitofrontal cortices, and with a frontocentral positivity from 550 ms poststimulus. Face repetition was associated with a positivity from 400 to 600 ms and behavioural priming. Repetition of familiar faces was also associated with earlier onset of the ERP familiarity effect, and haemodynamic decreases in fusiform cortex. These data support a multi-component model of face-processing, with priming arising from more than one stage.     

Linking hemodynamic and electrophysiological measures of brain activity: evidence from functional MRI and intracranial field potentials

We investigated the relation between electrophysiological and hemodynamic measures of brain activity through comparison of intracranially recorded event-related local field potentials (ERPs) and blood-oxygenation level dependent functional magnetic resonance imaging (BOLD fMRI). We manipulated the duration of visual checkerboard stimuli across trials and measured stimulus-duration-related changes in ERP and BOLD activity in three brain regions: peri-calcarine cortex, the fusiform gyrus and lateral temporal-occipital (LTO) cortex. ERPs were recorded from patients who had indwelling subdural electrodes as part of presurgical testing, while BOLD responses were measured in similar brain regions in a second set of subjects. Similar BOLD responses were measured in peri-calcarine and fusiform regions, with both showing monotonic but non-linear increases in hemodynamic amplitude with stimulus duration. In sharp contrast, very different ERP responses were observed in these same regions, such that calcarine electrodes exhibited onset potentials, sustained activity over the course of stimulus duration and prominent offset potentials, while fusiform electrodes only exhibited onset potentials that did not vary with stimulus duration. No duration-related ERP or BOLD changes were observed in LTO. Additional analyses revealed no consistent changes in the EEG spectrum across different brain sites that correlated with duration-related changes in the BOLD response. We conclude that the relation between ERPs and fMRI differs across brain regions.     

Regional specificity of format-specific priming effects in mirror word reading using functional magnetic resonance imaging

The speed and accuracy with which subjects can read words is enhanced or "primed" by a prior presentation of the same words. Moreover, priming effects are generally larger when the physical form of the words is maintained from the first to the second presentation. We investigated the neural basis of format-specific priming in a mirror word-reading task using event-related functional magnetic resonance imaging (fMRI). Participants read words that were presented either in mirror-image (M) orientation or in normal (N) orientation and were repeated either in the same or the alternate orientation, creating 4 study-test conditions, N-N, M-N, N-M, and M-M. Priming of N words resulted in reductions in fMRI signal in multiple brain regions, even though reading times (RTs) were unchanged. Priming of M words showed a pattern of RTs consistent with format-specific priming, with greater reductions when the prime matched the form of the test word. Priming-related reductions in fMRI activity were evident in all regions involved in mirror-image reading, regardless of the orientation of the prime. Importantly, reductions in several posterior regions, including fusiform, superior parietal, and superior temporal regions were also format specific. That is, signal reductions in these regions were greatest when the visual form of the prime and target matched (M-M compared with N-M). The results indicate that, although there are global neural priming effects due to stimulus repetition, it is also possible to identify regional brain changes that are sensitive to the specific perceptual overlap of primes and targets.     

Temporal and cerebellar brain regions that support both declarative memory formation and retrieval

Using event-related fMRI, we scanned young healthy subjects while they memorized real-world photographs and subsequently tried to recognize them within a series of new photographs. We confirmed that activity in the medial temporal lobe (MTL) and inferior prefrontal cortex correlates with declarative memory formation as defined by the subsequent memory effect, stronger responses to subsequently remembered than forgotten items. Additionally, we confirmed that activity in specific regions within the parietal lobe, anterior prefrontal cortex, anterior cingulate and cerebellum correlate with recognition memory as measured by the conventional old/new effect, stronger responses for recognized old items (hits) than correctly identified new items (correct rejections). To obtain a purer measure of recognition success, we introduced two recognition effects by comparing brain responses to hits and old items misclassified as new (misses). The positive recognition effect (hits > misses) revealed prefrontal, parietal and cerebellar contributions to recognition, and in line with electrophysiological findings, the negative recognition effect (hits < misses) revealed an anterior medial temporal contribution. Finally, by inclusive masking, we identified temporal and cerebellar brain areas that support both declarative memory formation and retrieval. For matching operations during recognition, these areas may re-use representations formed and stored locally during encoding.     

Electrophysiology and functional MRI in post-acute mild traumatic brain injury

Symptoms persisting beyond the acute phase (>2 months) after a mild traumatic brain injury (MTBI) are often reported, but their origin remains controversial. Some investigators evoke dysfunctional cerebral mechanisms, while others ascribe them to the psychological consequences of the injury. We address this controversy by exploring possible cerebral dysfunction with functional magnetic resonance imaging (fMRI) and event-related potentials (ERP) in a group of patients during the post-acute phase. Fourteen MTBI symptomatic patients (5.7±2.9 months post-injury) were tested with fMRI and ERP using a visual externally ordered working memory task, and were compared with 23 control subjects. Attenuated blood oxygen level dependent (BOLD) signal changes in the left and right mid-dorsolateral prefrontal cortex (mid-DLPFC), the putamen, the body of the caudate nucleus, and the right thalamus were found in the MTBI group compared with the control group. Moreover, symptom severity and BOLD signal changes were correlated: patients with more severe symptoms had lower BOLD signal changes in the right mid-DLPFC. For ERP, a group×task interaction was observed for N350 amplitude. A larger amplitude for the working memory task than for the control task was found in control subjects, but not in MTBI subjects, who had weak amplitudes for both tasks. This study confirms that persistent symptoms after MTBI cannot be uniquely explained by psychological factors, such as depression and/or malingering, and indicates that they can be associated with cerebral dysfunction. ERP reveals decreased amplitude of the N350 component, while fMRI demonstrates that the more severe the symptoms, the lower the BOLD signal changes in the mid-DLPFC.     

Isolating rule- versus evidence-based prefrontal activity during episodic and lexical discrimination: a functional magnetic resonance imaging investigation of detection theory distinctions

Dorsolateral and frontopolar prefrontal cortices (PFCs) are often implicated in neuroimaging studies of memory retrieval, with this activity ascribed to controlled monitoring processes indicative of difficult or demanding retrieval. Difficulty, however, is multiply determined, with success rates governed both by the available evidence and by the nature of decision rules applied to that evidence. Using event-related functional magnetic resonance imaging, we isolated these factors by 1) contrasting different decision rules across matched evidence and 2) manipulating the level of evidence within a fixed decision rule. For identically constructed retrieval probes (1 old and 1 new item), same-different (are these different?) compared with forced-choice (which one is old?) decision rules yielded bilateral dorsolateral and right frontopolar PFC increases. However, these regions were unaffected when the available evidence was greatly lowered within forced-choice decisions. Thus, the regions were simultaneously sensitive to the type of decision rule and yet insensitive to the level of evidence supporting those decisions. Analogous lexical tasks yielded similar patterns, demonstrating that the PFC responses were not episodic memory specific. We discuss the mechanistic differences between same-different versus forced-choice decisions and the implications of these data for current theories of PFC activity during episodic remembering and executive control.     

The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving

We sought to map the time course of autobiographical memory retrieval, including brain regions that mediate phenomenological experiences of reliving and emotional intensity. Participants recalled personal memories to auditory word cues during event-related functional magnetic resonance imaging (fMRI). Participants pressed a button when a memory was accessed, maintained and elaborated the memory, and then gave subjective ratings of emotion and reliving. A novel fMRI approach based on timing differences capitalized on the protracted reconstructive process of autobiographical memory to segregate brain areas contributing to initial access and later elaboration and maintenance of episodic memories. The initial period engaged hippocampal, retrosplenial, and medial and right prefrontal activity, whereas the later period recruited visual, precuneus, and left prefrontal activity. Emotional intensity ratings were correlated with activity in several regions, including the amygdala and the hippocampus during the initial period. Reliving ratings were correlated with activity in visual cortex and ventromedial and inferior prefrontal regions during the later period. Frontopolar cortex was the only brain region sensitive to emotional intensity across both periods. Results were confirmed by time-locked averages of the fMRI signal. The findings indicate dynamic recruitment of emotion-, memory-, and sensory-related brain regions during remembering and their dissociable contributions to phenomenological features of the memories.     

A distributed left hemisphere network active during planning of everyday tool use skills

Determining the relationship between mechanisms involved in action planning and/or execution is critical to understanding the neural bases of skilled behaviors, including tool use. Here we report findings from two fMRI studies of healthy, right-handed adults in which an event-related design was used to distinguish regions involved in planning (i.e. identifying, retrieving and preparing actions associated with a familiar tools' uses) versus executing tool use gestures with the dominant right (experiment 1) and non-dominant left (experiment 2) hands. For either limb, planning tool use actions activates a distributed network in the left cerebral hemisphere consisting of: (i) posterior superior temporal sulcus, along with proximal regions of the middle and superior temporal gyri; (ii) inferior frontal and ventral premotor cortices; (iii) two distinct parietal areas, one located in the anterior supramarginal gyrus (SMG) and another in posterior SMG and angular gyrus; and (iv) dorsolateral prefrontal cortex (DLFPC). With the exception of left DLFPC, adjacent and partially overlapping sub-regions of left parietal, frontal and temporal cortex are also engaged during action execution. We suggest that this left lateralized network constitutes a neural substrate for the interaction of semantic and motoric representations upon which meaningful skills depend.     

Source analysis of event-related cortical activity during visuo-spatial attention

Recordings of event-related potentials (ERPs) were combined with structural and functional magnetic resonance imaging (fMRI) to study the spatio-temporal patterns of cortical activity that underlie visual-spatial attention. Small checkerboard stimuli were flashed in random order to the four quadrants of the visual field at a rapid rate while subjects attended to stimuli in one quadrant at a time. Attended stimuli elicited enhanced ERP components in the latency range 80-200 ms that were co-localized with fMRI activations in multiple extrastriate cortical regions. The earliest ERP component (C1 at 50-90 ms) was unaffected by attention and was localized by dipole modeling to calcarine cortex. A longer latency deflection in the 150-225 ms range that was accounted for by this same calcarine source, however, did show consistent modulation with attention. This late attention effect, like the C1, inverted in polarity for upper versus lower field stimuli, consistent with a neural generator in primary visual cortex (area V1). These results provide support to current hypotheses that spatial attention in humans is associated with delayed feedback to area V1 from higher extrastriate areas that may have the function of improving the salience of stimuli at attended locations.     

Repetition suppression and reactivation in auditory-verbal short-term recognition memory

The neural response to stimulus repetition is not uniform across brain regions, stimulus modalities, or task contexts. For instance, it has been observed in many functional magnetic resonance imaging (fMRI) studies that sometimes stimulus repetition leads to a relative reduction in neural activity (repetition suppression), whereas in other cases repetition results in a relative increase in activity (repetition enhancement). In the present study, we hypothesized that in the context of a verbal short-term recognition memory task, repetition-related "increases" should be observed in the same posterior temporal regions that have been previously associated with "persistent activity" in working memory rehearsal paradigms. We used fMRI and a continuous recognition memory paradigm with short lags to examine repetition effects in the posterior and anterior regions of the superior temporal cortex. Results showed that, consistent with our hypothesis, the 2 posterior temporal regions consistently associated with working memory maintenance, also show repetition increases during short-term recognition memory. In contrast, a region in the anterior superior temporal lobe showed repetition suppression effects, consistent with previous research work on perceptual adaptation in the auditory-verbal domain. We interpret these results in light of recent theories of the functional specialization along the anterior and posterior axes of the superior temporal lobe.     

Modulation of neural activity during object naming: effects of time and practice

Repeated exposure to objects improves our ability to identify and name them, even after a long delay. Previous brain imaging studies have demonstrated that this experience-related facilitation of object naming is associated with neural changes in distinct brain regions. We used event-related functional magnetic resonance imaging (fMRI) to examine the modulation of neural activity in the object naming system as a function of experience and time. Pictures of common objects were presented repeatedly for naming at different time intervals (1 h, 6 h and 3 days) before scanning, or at 30 s intervals during scanning. The results revealed that as objects became more familiar with experience, activity in occipitotemporal and left inferior frontal regions decreased while activity in the left insula and basal ganglia increased. In posterior regions, reductions in activity as a result of multiple repetitions did not interact with time, whereas in left inferior frontal cortex larger decreases were observed when repetitions were spaced out over time. This differential modulation of activity in distinct brain regions provides support for the idea that long-lasting object priming is mediated by two neural mechanisms. The first mechanism may involve changes in object-specific representations in occipitotemporal cortices, the second may be a form of procedural learning involving a reorganization in brain circuitry that leads to more efficient name retrieval.     

Neural correlates of true memory, false memory, and deception

We used functional magnetic resonance imaging (fMRI) to determine whether neural activity can differentiate between true memory, false memory, and deception. Subjects heard a series of semantically related words and were later asked to make a recognition judgment of old words, semantically related nonstudied words (lures for false recognition), and unrelated new words. They were also asked to make a deceptive response to half of the old and unrelated new words. There were 3 main findings. First, consistent with the notion that executive function supports deception, 2 types of deception (pretending to know and pretending not to know) recruited prefrontal activity. Second, consistent with the sensory reactivation hypothesis, the difference between true recognition and false recognition was found in the left temporoparietal regions probably engaged in the encoding of auditorily presented words. Third, the left prefrontal cortex was activated during pretending to know relative to correct rejection and false recognition, whereas the right anterior hippocampus was activated during false recognition relative to correct rejection and pretending to know. These findings indicate that fMRI can detect the difference in brain activity between deception and false memory despite the fact that subjects respond with "I know" to novel events in both processes.     

The relationship between aging, performance, and the neural correlates of successful memory encoding

The present functional magnetic resonance imaging (fMRI) study investigated whether age-related differences in the neural correlates of successful memory encoding are modulated by memory performance. Young (mean age 22 years; N = 16) and older (mean age 69 years; N = 32) subjects were scanned while making animacy decisions on visually presented words. Memory for the words was later assessed in a recognition test, allowing fMRI activity elicited by study words to be contrasted according to subsequent memory performance. Young and older adults exhibited equivalent subsequent memory effects (enhanced activity for later remembered items) in an extensive network that included left inferior prefrontal cortex and anterior hippocampus. In posterior cingulate cortex, reversed subsequent memory effects (greater activity for later forgotten items) were of greater magnitude in young subjects. A voxel-of-interest analysis conducted on left and right prefrontal subsequent memory effects revealed that the effects were distributed more bilaterally in older than in young subjects, replicating previous findings. This age-related difference was confined to older subjects with relatively poor recognition performance, who were also the only group to demonstrate statistically significant right prefrontal subsequent memory effects. The findings suggest that relative preservation of memory performance with increasing age does not depend upon right prefrontal "over-recruitment."     

Consequences of magnocellular dysfunction on processing attended information in schizophrenia

Schizophrenia is associated with perceptual and cognitive dysfunction including impairments in visual attention. These impairments may be related to deficits in early stages of sensory/perceptual processing, particularly within the magnocellular/dorsal visual pathway. In the present study, subjects viewed high and low spatial frequency (SF) gratings designed to test functioning of the parvocellular/magnocellular pathways, respectively. Schizophrenia patients and healthy controls attended to either the low SF (magnocellularly biased) or high SF (parvocellularly biased) gratings. Functional magnetic resonance imaging (fMRI) and recordings of event-related potentials (ERPs) were carried out during task performance. Patients were impaired at detecting low-frequency targets. ERP amplitudes to low-frequency gratings were diminished, both for the early sensory-evoked components and for the attend minus unattend difference component (the selection negativity), which is regarded as a neural index of feature-selective attention. Similarly, fMRI revealed that activity in extrastriate visual cortex was reduced in patients during attention to low, but not high, SF. In contrast, activity in frontal and parietal areas, previously implicated in the control of attention, did not differ between patients and controls. These findings suggest that impaired sensory processing of magnocellularly biased stimuli lead to impairments in the effective processing of attended stimuli, even when the attention control systems themselves are intact.     

Nonvisual codes and nonvisual brain areas support visual working memory

Systems models hold working memory to depend on specialized, domain-specific storage buffers. Here, however, we demonstrate that short-term retention of the identity or location of visually presented stimuli is disrupted by nonvisual secondary tasks performed in the dark-passive listening to nouns or endogenous generation of saccades, respectively. This indicates that the short-term retention of visual information relies on multiple mental codes, some of them nonvisual. Event-related functional magnetic resonance imaging (fMRI) reveals the neural correlates of these interference effects to be more complex and more regionally specific than previously described. Although nonspecific dual-task effects produce a generalized decrease of task-evoked fMRI response across many brain regions, the interference-specific effect is a relative increase of activity localized to regions associated with the secondary task in question: left hemisphere perisylvian cortex in the case of passive listening distraction and frontal oculomotor regions in the case of saccadic distraction. Within these regions, the neural interference effects are specific to voxels that show delay-period activity on unfilled memory trials. They also predict individual differences in the magnitude of the behavioral interference effect. These results indicate that nonvisual processes supported by nonvisual brain areas contribute importantly to "visual" working memory performance.     

Learning-related neuronal responses in prefrontal cortex studied with functional neuroimaging

We assessed time-dependent neuronal activity accompanying learning using functional magnetic resonance imaging (fMRI). An artificial grammar learning paradigm enabled us to dissociate activations associated with individual item learning from those involved in learning the underlying grammar system. We show that a localized region of right prefrontal cortex (PFC) is preferentially sensitive to individual item learning during the early stages of the experiment, while the left PFC region is sensitive to grammar learning which occurred across the entire course of the experiment. In addition to dissociating these two types of learning, we were able to characterize the effect of rule acquisition on neuronal responses associated with explicit learning of individual items. This effect was expressed as modulation of the time-dependent right PFC activations such that the early increase in activation associated with item learning was attenuated as the experiment progressed. In a further analysis we used structural equation modelling to explore time-dependent changes in inter-regional connectivity as a function of both item and grammar rule learning. Although there were no significant effects of item learning on the measured path strengths, rule learning was associated with a decrease in right fronto-parietal connectivity and an increase in connectivity between left and right PFC. Further fronto-parietal path strengths were observed to change, with an increase in left fronto-parietal and a decrease in right fronto-parietal connectivity path strength from right PFC to left parietal cortex. We interpret our findings in terms of a left frontal system mediating the semantic analysis of study items and directly influencing a right fronto-parietal system associated with episodic memory retrieval.