Attending to and remembering tactile stimuli: a review of brain imaging data and single-neuron responses.

Clinical and neuroimaging observations of the cortical network implicated in tactile attention have identified foci in parietal somatosensory, posterior parietal, and superior frontal locations. Tasks involving intentional hand-arm movements activate similar or nearby parietal and frontal foci. Visual spatial attention tasks and deliberate visuomotor behavior also activate overlapping posterior parietal and frontal foci. Studies in the visual and somatosensory systems thus support a proposal that attention to the spatial location of an object engages cortical regions responsible for the same coordinate referents used for guiding purposeful motor behavior. Tactile attention also biases processing in the somatosensory cortex through amplification of responses to relevant features of selected stimuli. Psychophysical studies demonstrate retention gradients for tactile stimuli like those reported for visual and auditory stimuli, and suggest analogous neural mechanisms for working memory across modalities. Neuroimaging studies in humans using memory tasks, and anatomic studies in monkeys support the idea that tactile information relayed from the somatosensory cortex is directed ventrally through the insula to the frontal cortex for short-term retention and to structures of the medial temporal lobe for long-term encoding. At the level of single neurons, tactile (such as visual and auditory) short-term memory appears as a persistent response during delay intervals between sampled stimuli.     

Frontal and parietal theta burst TMS impairs working memory for visual-spatial conjunctions

In tasks that selectively probe visual or spatial working memory (WM) frontal and posterior cortical areas show a segregation, with dorsal areas preferentially involved in spatial (e.g. location) WM and ventral areas in visual (e.g. object identity) WM. In a previous fMRI study [1], we showed that right parietal cortex (PC) was more active during WM for orientation, whereas left inferior frontal gyrus (IFG) was more active during colour WM. During WM for colour-orientation conjunctions, activity in these areas was intermediate to the level of activity for the single task preferred and non-preferred information. To examine whether these specialised areas play a critical role in coordinating visual and spatial WM to perform a conjunction task, we used theta burst transcranial magnetic stimulation (TMS) to induce a functional deficit. Compared to sham stimulation, TMS to right PC or left IFG selectively impaired WM for conjunctions but not single features. This is consistent with findings from visual search paradigms, in which frontal and parietal TMS selectively affects search for conjunctions compared to single features, and with combined TMS and functional imaging work suggesting that parietal and frontal regions are functionally coupled in tasks requiring integration of visual and spatial information. Our results thus elucidate mechanisms by which the brain coordinates spatially segregated processing streams and have implications beyond the field of working memory.     

Dynamic links between theta executive functions and alpha storage buffers in auditory and visual working memory

Working memory (WM) tasks require not only distinct functions such as a storage buffer and central executive functions, but also coordination among these functions. Neuroimaging studies have revealed the contributions of different brain regions to different functional roles in WM tasks; however, little is known about the neural mechanism governing their coordination. Electroencephalographic (EEG) rhythms, especially theta and alpha, are known to appear over distributed brain regions during WM tasks, but the rhythms associated with task‐relevant regional coupling have not been obtained thus far. In this study, we conducted time–frequency analyses for EEG data in WM tasks that include manipulation periods and memory storage buffer periods. We used both auditory WM tasks and visual WM tasks. The results successfully demonstrated function‐specific EEG activities. The frontal theta amplitudes increased during the manipulation periods of both tasks. The alpha amplitudes increased during not only the manipulation but also the maintenance periods in the temporal area for the auditory WM and the parietal area for the visual WM. The phase synchronization analyses indicated that, under the relevant task conditions, the temporal and parietal regions show enhanced phase synchronization in the theta bands with the frontal region, whereas phase synchronization between theta and alpha is significantly enhanced only within the individual areas. Our results suggest that WM task‐relevant brain regions are coordinated by distant theta synchronization for central executive functions, by local alpha synchronization for the memory storage buffer, and by theta–alpha coupling for inter‐functional integration.     

Visual spatial and visual pattern working memory: neuropsychological evidence for a differential role of left and right dorsal visual brain

According to neurophysiological, neuroimaging, and behavioural evidence, visual working memory (WM) can be separated into a "what" and a "where" component, reflecting the duality of visual processing. Whereas a wealth of empirical data suggests a right-sided fronto-parietal network critical for the maintenance of spatial information, the cortical structures underlying maintenance of object information have remained controversial. Although visual object processing depends on ventral, inferior temporal areas, recent neuroimaging results suggest that maintenance of visual object information involves a left-sided or bilateral fronto-parietal network. The aim of the present study is to further clarify the role of the left and right parietal lobes for pattern and spatial visual WM. Seven patients with left-sided, seven with right-sided parietal brain injury, and two age-matched healthy control groups performed a delayed-matching-to-sample task using either pattern (shape) or spatial (location) information or both. In addition, eight patients with left-sided injury sparing parietal areas were tested to further examine the specific role of the left parietal cortex in pattern WM. Left parietal injury resulted in pattern WM impairment, only, while right parietal injury was associated with pattern and spatial WM deficits. Non-parietal injury was not associated with comparable deficits. These results suggest that visual spatial WM depends critically on right parietal areas; in contrast, pattern WM depends on both, left and right parietal areas.     

Auditory and visual connectivity gradients in frontoparietal cortex

A frontoparietal network of brain regions is often implicated in both auditory and visual information processing. Although it is possible that the same set of multimodal regions subserves both modalities, there is increasing evidence that there is a differentiation of sensory function within frontoparietal cortex. Magnetic resonance imaging (MRI) in humans was used to investigate whether different frontoparietal regions showed intrinsic biases in connectivity with visual or auditory modalities. Structural connectivity was assessed with diffusion tractography and functional connectivity was tested using functional MRI. A dorsal–ventral gradient of function was observed, where connectivity with visual cortex dominates dorsal frontal and parietal connections, while connectivity with auditory cortex dominates ventral frontal and parietal regions. A gradient was also observed along the posterior–anterior axis, although in opposite directions in prefrontal and parietal cortices. The results suggest that the location of neural activity within frontoparietal cortex may be influenced by these intrinsic biases toward visual and auditory processing. Thus, the location of activity in frontoparietal cortex may be influenced as much by stimulus modality as the cognitive demands of a task. It was concluded that stimulus modality was spatially encoded throughout frontal and parietal cortices, and was speculated that such an arrangement allows for top–down modulation of modality‐specific information to occur within higher‐order cortex. This could provide a potentially faster and more efficient pathway by which top–down selection between sensory modalities could occur, by constraining modulations to within frontal and parietal regions, rather than long‐range connections to sensory cortices. Hum Brain Mapp 38:255–270, 2017. © 2016 Wiley Periodicals, Inc.     

The effect of musicianship on pitch memory in performance matched groups

We compared brain activation patterns between musicians and non-musicians (matched in performance score) while they performed a pitch memory task (using a sparse temporal sampling fMRI method). Both groups showed bilateral activation of the superior temporal, supramarginal, posterior middle and inferior frontal gyrus, and superior parietal lobe. Musicians showed more right temporal and supramarginal gyrus activation while non-musicians had more right primary and left secondary auditory cortex activation. Since both groups' performance were matched, these results probably indicate processing differences between groups that are possibly related to musical training. Non-musicians rely more on brain regions important for pitch discrimination while musicians prefer to use brain regions specialized in short-term memory and recall to perform well in this pitch memory task.     

Perceptual Learning beyond Perception: Mnemonic Representation in Early Visual Cortex and Intraparietal Sulcus

The ability to discriminate between stimuli relies on a chain of neural operations associated with perception, memory and decision-making. Accumulating studies show learning-dependent plasticity in perception or decision-making, yet whether perceptual learning modifies mnemonic processing remains unclear. Here, we trained human participants of both sexes in an orientation discrimination task, while using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) to separately examine training-induced changes in working memory (WM) representation. fMRI decoding revealed orientation-specific neural patterns during the delay period in primary visual cortex (V1) before, but not after, training, whereas neurodisruption of V1 during the delay period led to behavioral deficits in both phases. In contrast, both fMRI decoding and disruptive effect of TMS showed that intraparietal sulcus (IPS) represented WM content after, but not before, training. These results suggest that training does not affect the necessity of sensory area in representing WM information, consistent with the sensory recruitment hypothesis in WM, but likely alters the coding format of the stored stimulus in this region. On the other hand, training can render WM content to be maintained in higher-order parietal areas, complementing sensory area to support more robust maintenance of information.SIGNIFICANCE STATEMENT There has been accumulating progresses regarding experience-dependent plasticity in perception or decision-making, yet how perceptual experience moulds mnemonic processing of visual information remains less explored. Here, we provide novel findings that learning-dependent improvement of discriminability accompanies altered WM representation at different cortical levels. Critically, we suggest a role of training in modulating cortical locus of WM representation, providing a plausible explanation to reconcile the discrepant findings between human and animal studies regarding the recruitment of sensory or higher-order areas in WM.     

Neural substrates associated with the concurrent performance of dual working memory tasks

The neural substrates of the dual working memory (WM) process were investigated using concurrent performance of auditory and visual n-back WM tasks. Based on the pre-fMRI behavioral testing, a lettered 1-back WM paradigm was implemented for an fMRI examination of healthy volunteers who performed (1) auditory, (2) visual, and (3) simultaneous visual and auditory WM tasks. The behavioral performance, as measured by the reaction time, was deteriorated in the dual task condition compared to the single task condition. Group analysis of the fMRI data revealed that the majority of activation identified during each component task was concurrently activated in the dual task condition. However, several neural substrates such as left middle frontal gyrus, left superior parietal lobule, posterior aspect of right inferior temporal gyrus, and bilateral parahippocampal gyri were selectively activated during the dual WM task. These data suggest that new neural networks come into play to assist in the greater load placed on the WM with the incongruent stimulation modality, which may also have implications in crossmodal integrative processes.     

Cortical activation patterns during long-term memory retrieval of visually or haptically encoded objects and locations

The present study used functional magnetic resonance imaging to delineate cortical networks that are activated when objects or spatial locations encoded either visually (visual encoding group, n=10) or haptically (haptic encoding group, n=10) had to be retrieved from long-term memory. Participants learned associations between auditorily presented words and either meaningless objects or locations in a 3-D space. During the retrieval phase one day later, participants had to decide whether two auditorily presented words shared an association with a common object or location. Thus, perceptual stimulation during retrieval was always equivalent, whereas either visually or haptically encoded object or location associations had to be reactivated. Moreover, the number of associations fanning out from each word varied systematically, enabling a parametric increase of the number of reactivated representations. Recall of visual objects predominantly activated the left superior frontal gyrus and the intraparietal cortex, whereas visually learned locations activated the superior parietal cortex of both hemispheres. Retrieval of haptically encoded material activated the left medial frontal gyrus and the intraparietal cortex in the object condition, and the bilateral superior parietal cortex in the location condition. A direct test for modality-specific effects showed that visually encoded material activated more vision-related areas (BA 18/19) and haptically encoded material more motor and somatosensory-related areas. A conjunction analysis identified supramodal and material-unspecific activations within the medial and superior frontal gyrus and the superior parietal lobe including the intraparietal sulcus. These activation patterns strongly support the idea that code-specific representations are consolidated and reactivated within anatomically distributed cell assemblies that comprise sensory and motor processing systems.     

Comparable mechanisms of working memory interference by auditory and visual motion in youth and aging

Intrasensory interference during visual working memory (WM) maintenance by object stimuli (such as faces and scenes), has been shown to negatively impact WM performance, with greater detrimental impacts of interference observed in aging. Here we assessed age-related impacts by intrasensory WM interference from lower-level stimulus features such as visual and auditory motion stimuli. We consistently found that interference in the form of ignored distractions and secondary task interruptions presented during a WM maintenance period, degraded memory accuracy in both the visual and auditory domain. However, in contrast to prior studies assessing WM for visual object stimuli, feature-based interference effects were not observed to be significantly greater in older adults. Analyses of neural oscillations in the alpha frequency band further revealed preserved mechanisms of interference processing in terms of post-stimulus alpha suppression, which was observed maximally for secondary task interruptions in visual and auditory modalities in both younger and older adults. These results suggest that age-related sensitivity of WM to interference may be limited to complex object stimuli, at least at low WM loads.     

Bihemispheric network dynamics coordinating vocal feedback control

Modulation of vocal pitch is a key speech feature that conveys important linguistic and affective information. Auditory feedback is used to monitor and maintain pitch. We examined induced neural high gamma power (HGP) (65–150 Hz) using magnetoencephalography during pitch feedback control. Participants phonated into a microphone while hearing their auditory feedback through headphones. During each phonation, a single real‐time 400 ms pitch shift was applied to the auditory feedback. Participants compensated by rapidly changing their pitch to oppose the pitch shifts. This behavioral change required coordination of the neural speech motor control network, including integration of auditory and somatosensory feedback to initiate change in motor plans. We found increases in HGP across both hemispheres within 200 ms of pitch shifts, covering left sensory and right premotor, parietal, temporal, and frontal regions, involved in sensory detection and processing of the pitch shift. Later responses to pitch shifts (200–300 ms) were right dominant, in parietal, frontal, and temporal regions. Timing of activity in these regions indicates their role in coordinating motor change and detecting and processing of the sensory consequences of this change. Subtracting out cortical responses during passive listening to recordings of the phonations isolated HGP increases specific to speech production, highlighting right parietal and premotor cortex, and left posterior temporal cortex involvement in the motor response. Correlation of HGP with behavioral compensation demonstrated right frontal region involvement in modulating participant's compensatory response. This study highlights the bihemispheric sensorimotor cortical network involvement in auditory feedback‐based control of vocal pitch. Hum Brain Mapp 37:1474‐1485, 2016. © 2016 Wiley Periodicals, Inc.     

Modality‐independent representations of small quantities based on brain activation patterns

Machine learning or MVPA (Multi Voxel Pattern Analysis) studies have shown that the neural representation of quantities of objects can be decoded from fMRI patterns, in cases where the quantities were visually displayed. Here we apply these techniques to investigate whether neural representations of quantities depicted in one modality (say, visual) can be decoded from brain activation patterns evoked by quantities depicted in the other modality (say, auditory). The main finding demonstrated, for the first time, that quantities of dots were decodable by a classifier that was trained on the neural patterns evoked by quantities of auditory tones, and vice‐versa. The representations that were common across modalities were mainly right‐lateralized in frontal and parietal regions. A second finding was that the neural patterns in parietal cortex that represent quantities were common across participants. These findings demonstrate a common neuronal foundation for the representation of quantities across sensory modalities and participants and provide insight into the role of parietal cortex in the representation of quantity information. Hum Brain Mapp 37:1296‐1307, 2016. © 2016 Wiley Periodicals, Inc.     

Ventromedial Prefrontal Cortex Drives the Prioritization of Self-Associated Stimuli in Working Memory

Humans show a pervasive bias for processing self- over other-related information, including in working memory (WM), where people prioritize the maintenance of self- (over other-) associated cues. To elucidate the neural mechanisms underlying this self-bias, we paired a self- versus other-associated spatial WM task with fMRI and transcranial direct current stimulation (tDCS) of human participants of both sexes. Maintaining self- (over other-) associated cues resulted in enhanced activity in classic WM regions (frontoparietal cortex), and in superior multivoxel pattern decoding of the cue locations from visual cortex. Moreover, ventromedial PFC (VMPFC) displayed enhanced functional connectivity with WM regions during maintenance of self-associated cues, which predicted individuals'' behavioral self-prioritization effects. In a follow-up tDCS experiment, we targeted VMPFC with excitatory (anodal), inhibitory (cathodal), or sham tDCS. Cathodal tDCS eliminated the self-prioritization effect. These findings provide strong converging evidence for a causal role of VMPFC in driving self-prioritization effects in WM and provide a unique window into the interaction between social, self-referential processing and high-level cognitive control processes.SIGNIFICANCE STATEMENT People have a strong tendency to attend to self-related stimuli, such as their names. This self-bias extends to the automatic prioritization of arbitrarily self-associated stimuli held in working memory. Since working memory is central to high-level cognition, this bias could influence how we make decisions. It is therefore important to understand the underlying brain mechanisms. Here, we used neuroimaging and noninvasive neurostimulation techniques to show that the source of self-bias in working memory is the ventromedial PFC, which modulates activity in frontoparietal brain regions to produce prioritized representations of self-associated stimuli in sensory cortex. This work thus reveals a brain circuit underlying the socially motivated (self-referential) biasing of high-level cognitive processing.     

Dynamics of a temporo-fronto-parietal network during sustained spatial or spectral auditory processing

Animal and human studies have suggested that posterior temporal, parietal, and frontal regions are specifically involved in auditory spatial (location and motion) processing, forming a putative dorsal "where" pathway. We used scalp EEG and current density mapping to investigate the dynamics of this network in human subjects presented with a varying acoustic stream in a two-factor paradigm: spatial versus pitch variations, focused versus diverted attention. The main findings were: (i) a temporo-parieto-frontal network was activated during the whole duration of the stream in all conditions and modulated by attention; (ii) the left superior temporal cortex was the only region showing different activations for pitch and spatial variations. Therefore, parietal and frontal regions would be involved in task-related processes (attention and motor preparation), whereas the differential processing of acoustic spatial and object-related features seems to take place at the temporal level.     

Cortical oscillatory activity during spatial echoic memory

In human magnetoencephalogram, we have found gamma-band activity (GBA), a putative measure of cortical network synchronization, during both bottom-up and top-down auditory processing. When sound positions had to be retained in short-term memory for 800 ms, enhanced GBA was detected over posterior parietal cortex, possibly reflecting the activation of higher sensory storage systems along the hypothesized auditory dorsal space processing stream. Additional prefrontal GBA increases suggested an involvement of central executive networks in stimulus maintenance. The present study assessed spatial echoic memory with the same stimuli but a shorter memorization interval of 200 ms. Statistical probability mapping revealed posterior parietal GBA increases at 80 Hz near the end of the memory phase and both gamma and theta enhancements in response to the test stimulus. In contrast to the previous short-term memory study, no prefrontal gamma or theta enhancements were detected. This suggests that spatial echoic memory is performed by networks along the putative auditory dorsal stream, without requiring an involvement of prefrontal executive regions.     

Reading fluent speech from talking faces: typical brain networks and individual differences

Listeners are able to extract important linguistic information by viewing the talker's face-a process known as 'speechreading.' Previous studies of speechreading present small closed sets of simple words and their results indicate that visual speech processing engages a wide network of brain regions in the temporal, frontal, and parietal lobes that are likely to underlie multiple stages of the receptive language system. The present study further explored this network in a large group of subjects by presenting naturally spoken sentences which tap the richer complexities of visual speech processing. Four different baselines (blank screen, static face, nonlinguistic facial gurning, and auditory speech) enabled us to determine the hierarchy of neural processing involved in speechreading and to test the claim that visual input reliably accesses sound-based representations in the auditory cortex. In contrast to passively viewing a blank screen, the static-face condition evoked activation bilaterally across the border of the fusiform gyrus and cerebellum, and in the medial superior frontal gyrus and left precentral gyrus (p < .05, whole brain corrected). With the static face as baseline, the gurning face evoked bilateral activation in the motion-sensitive region of the occipital cortex, whereas visual speech additionally engaged the middle temporal gyrus, inferior and middle frontal gyri, and the inferior parietal lobe, particularly in the left hemisphere. These latter regions are implicated in lexical stages of spoken language processing. Although auditory speech generated extensive bilateral activation across both superior and middle temporal gyri, the group-averaged pattern of speechreading activation failed to include any auditory regions along the superior temporal gyrus, suggesting that f luent visual speech does not always involve sound-based coding of the visual input. An important finding from the individual subject analyses was that activation in the superior temporal gyrus did reach significance (p < .001, small-volume corrected) for a subset of the group. Moreover, the extent of the left-sided superior temporal gyrus activity was strongly correlated with speechreading performance. Skilled speechreading was also associated with activations and deactivations in other brain regions, suggesting that individual differences ref lect the efficiency of a circuit linking sensory, perceptual, memory, cognitive, and linguistic processes rather than the operation of a single component process.     

What is common to brain activity evoked by the perception of visual and auditory filled durations? A study with MEG and EEG co-recordings

EEG and MEG scalp data were simultaneously recorded while human participants were performing a duration discrimination task in visual and auditory modality, separately. Short durations were used ranging from 500 to 900 ms, among which participants had to discriminate a previously memorized 700-ms "standard" duration. Behavioral results show accurate but variable performance within and between participants with expected modality effects: the percentage of responses was greater and the mean response time was shorter for auditory than for visual signals. Sustained electric and magnetic activities were obtained correlatively to duration estimation, but with distinct spatiotemporal properties. Electric CNV-like potentials showed fronto-central negativity in both modalities, whereas magnetic sustained fields were distributed with respect to the modality of the interval to be timed. Time courses of these slow brain activities were found to be dependent on stimulus duration but not on its modality nor on the recording signal (EEG or MEG). Source reconstruction demonstrated that these sustained potentials/fields were generated by superimposed contributions from visual and auditory cortices (sustained sensory responses, SSR) and from prefrontal and parietal regions. By using these two complementary techniques, we thus demonstrated the involvement of frontal and parietal cerebral cortex in human timing.     

The neural substrate of orientation working memory

We have used positron emission tomography (PET) to identify the neural substrate of two major cognitive components of working memory (WM), maintenance and manipulation of a single elementary visual attribute, i.e., the orientation of a grating presented in central vision. This approach allowed us to equate difficulty across tasks and prevented subjects from using verbal strategies or vestibular cues. Maintenance of orientations involved a distributed fronto-parietal network, that is, left and right lateral superior frontal sulcus (SFSl), bilateral ventrolateral prefrontal cortex (VLPFC), bilateral precuneus, and right superior parietal lobe (SPL). A more medial superior frontal sulcus region (SFSm) was identified as being instrumental in the manipulative operation of updating orientations retained in the WM. Functional connectivity analysis revealed that orientation WM relies on a coordinated interaction between frontal and parietal regions. In general, the current findings confirm the distinction between maintenance and manipulative processes, highlight the functional heterogeneity in the prefrontal cortex (PFC), and suggest a more dynamic view of WM as a process requiring the coordinated interaction of anatomically distinct brain areas.     

Strategic resource allocation in the human brain supports cognitive coordination of object and spatial working memory

The ability to integrate different types of information (e.g., object identity and spatial orientation) and maintain or manipulate them concurrently in working memory (WM) facilitates the flow of ongoing tasks and is essential for normal human cognition. Research shows that object and spatial information is maintained and manipulated in WM via separate pathways in the brain (object/ventral versus spatial/dorsal). How does the human brain coordinate the activity of different specialized systems to conjoin different types of information? Here we used functional magnetic resonance imaging to investigate conjunction‐ versus single‐task manipulation of object (compute average color blend) and spatial (compute intermediate angle) information in WM. Object WM was associated with ventral (inferior frontal gyrus, occipital cortex), and spatial WM with dorsal (parietal cortex, superior frontal, and temporal sulci) regions. Conjoined object/spatial WM resulted in intermediate activity in these specialized areas, but greater activity in different prefrontal and parietal areas. Unique to our study, we found lower temporo‐occipital activity and greater deactivation in temporal and medial prefrontal cortices for conjunction‐ versus single‐tasks. Using structural equation modeling, we derived a conjunction‐task connectivity model that comprises a frontoparietal network with a bidirectional DLPFC‐VLPFC connection, and a direct parietal‐extrastriate pathway. We suggest that these activation/deactivation patterns reflect efficient resource allocation throughout the brain and propose a new extended version of the biased competition model of WM. Hum Brain Mapp, 2011. © 2010 Wiley‐Liss, Inc.     

Decoding Word Information from Spatiotemporal Activity of Sensory Neurons

Spatiotemporal activity of neurons is ubiquitous in sensory coding in the CNS. It is a fundamental problem for sensory perception to understand how sensory information is decoded from the spatiotemporal activity. However, little is known about the decoding mechanism. To address this issue, we are concerned with auditory system as a model system exhibiting spatiotemporal activity. We present here a model of auditory cortex, which performs a hierarchical processing of auditory information. The model consists of three layers of two-dimensional networks. The first layer represents auditory stimulus as a spatiotemporal activity of neurons. The second layer consists of feature-detecting neurons, which extract the features of phonemes and their overlaps from the spatiotemporal activity of the first layer. The third layer combines information of the sound features encoded by the second layer and decodes word information about the sound stimulus as a temporal sequence of attractors. Using the model, we show how the information of phonemes and words emerge in the hierarchical processing of the auditory cortex. We also show that the overlap between phonemes plays a crucial role in linking the attractors of phonemes. The present study may provide a clue for understanding the mechanism by which word information is decoded from spatiotemporal activity of neurons.