Brain activations during conscious self-monitoring of speech production with delayed auditory feedback: an fMRI study

When a speaker's voice returns to one's own ears with a 200-ms delay, the delay causes the speaker to speak less fluently. This phenomenon is called a delayed auditory feedback (DAF) effect. To investigate neural mechanisms of speech processing through the DAF effect, we conducted a functional magnetic resonance imaging (fMRI) experiment, in which we designed a paradigm to explore the conscious overt-speech processing and the automatic overt-speech processing separately, while reducing articulatory motion artifacts. The subjects were instructed to (1) read aloud visually presented sentences under real-time auditory feedback (NORMAL), (2) read aloud rapidly under real-time auditory feedback (FAST), (3) read aloud slowly under real-time auditory feedback (SLOW), and (4) read aloud under DAF (DELAY). In the contrasts of DELAY-NORMAL, DELAY-FAST, and DELAY-SLOW, the bilateral superior temporal gyrus (STG), the supramarginal gyrus (SMG), and the middle temporal gyrus (MTG) showed significant activation. Moreover, we found that the STG activation was correlated with the degree of DAF effect for all subjects. Because the temporo-parietal regions did not show significant activation in the comparisons among NORMAL, FAST, and SLOW conditions, we can exclude the possibility that its activation is due to speech rates or enhanced attention to altered speech sounds. These results suggest that the temporo-parietal regions function as a conscious self-monitoring system to support an automatic speech production system.     

Exploring consequences of short- and long-term deafness on speech production: A lip-tube perturbation study

Studies have reported strong links between speech production and perception. We aimed to evaluate the role of long- and short-term auditory feedback alteration on speech production. Eleven adults with normal hearing (controls) and 17 cochlear implant (CI) users (7 pre-lingually deaf and 10 post-lingually deaf adults) were recruited. Short-term auditory feedback deprivation was induced by turning off the CI or by providing masking noise. Acoustic and articulatory measures were obtained during the production of /u/, with and without a tube inserted between the lips (perturbation), and with and without auditory feedback. F1 values were significantly different between the implant OFF and ON conditions for the pre-lingually deaf participants. In the absence of auditory feedback, the pre-lingually deaf participants moved the tongue more forward. Thus, a lack of normal auditory experience of speech may affect the internal representation of a vowel.     

Auditory and audio–visual processing in patients with cochlear,auditory brainstem,and auditory midbrain implants: An EEG study

There is substantial variability in speech recognition ability across patients with cochlear implants (CIs), auditory brainstem implants (ABIs), and auditory midbrain implants (AMIs). To better understand how this variability is related to central processing differences, the current electroencephalography (EEG) study compared hearing abilities and auditory–cortex activation in patients with electrical stimulation at different sites of the auditory pathway. Three different groups of patients with auditory implants (Hannover Medical School; ABI: n = 6, CI: n = 6; AMI: n = 2) performed a speeded response task and a speech recognition test with auditory, visual, and audio–visual stimuli. Behavioral performance and cortical processing of auditory and audio–visual stimuli were compared between groups. ABI and AMI patients showed prolonged response times on auditory and audio–visual stimuli compared with NH listeners and CI patients. This was confirmed by prolonged N1 latencies and reduced N1 amplitudes in ABI and AMI patients. However, patients with central auditory implants showed a remarkable gain in performance when visual and auditory input was combined, in both speech and non‐speech conditions, which was reflected by a strong visual modulation of auditory–cortex activation in these individuals. In sum, the results suggest that the behavioral improvement for audio–visual conditions in central auditory implant patients is based on enhanced audio–visual interactions in the auditory cortex. Their findings may provide important implications for the optimization of electrical stimulation and rehabilitation strategies in patients with central auditory prostheses. Hum Brain Mapp 38:2206–2225, 2017. © 2017 Wiley Periodicals, Inc.     

Cerebellar contribution to auditory feedback control of speech production: Evidence from patients with spinocerebellar ataxia

The cerebellum has been implicated in the feedforward control of speech production. However, the role of the cerebellum in the feedback control of speech production remains unclear. To address this question, the present event‐related potential study examined the behavioral and neural correlates of auditory feedback control of vocal production in patients with spinocerebellar ataxia (SCA) and healthy controls. All participants were instructed to produce sustained vowels while hearing their voice unexpectedly pitch‐shifted ?200 or ?500 cents. The behavioral results revealed significantly larger vocal compensations for pitch perturbations in patients with SCA relative to healthy controls. At the cortical level, patients with SCA exhibited significantly smaller cortical P2 responses that were source localized in the right superior temporal gyrus, primary auditory cortex, and supramarginal gyrus than healthy controls. These findings indicate that reduced brain activity in the right temporal and parietal regions are significant neural contributors to abnormal auditory‐motor processing of vocal pitch regulation as a consequence of cerebellar degeneration, which may be related to disrupted reciprocal interactions between the cerebellum and cortical regions that support the top‐down modulation of auditory‐vocal integration. These differences in behavior and cortical activity between healthy controls and patients with SCA demonstrate that the cerebellum is not only essential for feedforward control but also plays a crucial role in the feedback‐based control of speech production.     

The impact of parkinson's disease on the cortical mechanisms that support auditory–motor integration for voice control

Several studies have shown sensorimotor deficits in speech processing in individuals with idiopathic Parkinson's disease (PD). The underlying neural mechanisms, however, remain poorly understood. In the present event‐related potential (ERP) study, 18 individuals with PD and 18 healthy controls were exposed to frequency‐altered feedback (FAF) while producing a sustained vowel and listening to the playback of their own voice. Behavioral results revealed that individuals with PD produced significantly larger vocal compensation for pitch feedback errors than healthy controls, and exhibited a significant positive correlation between the magnitude of their vocal responses and the variability of their unaltered vocal pitch. At the cortical level, larger P2 responses were observed for individuals with PD compared with healthy controls during active vocalization due to left‐lateralized enhanced activity in the superior and inferior frontal gyrus, premotor cortex, inferior parietal lobule, and superior temporal gyrus. These two groups did not differ, however, when they passively listened to the playback of their own voice. Individuals with PD also exhibited larger P2 responses during active vocalization when compared with passive listening due to enhanced activity in the inferior frontal gyrus, precental gyrus, postcentral gyrus, and middle temporal gyrus. This enhancement effect, however, was not observed for healthy controls. These findings provide neural evidence for the abnormal auditory–vocal integration for voice control in individuals with PD, which may be caused by their deficits in the detection and correction of errors in voice auditory feedback. Hum Brain Mapp 37:4248–4261, 2016. © 2016 Wiley Periodicals, Inc.     

Cortical auditory evoked potentials in unsuccessful cochlear implant users

In some cochlear implant users, success is not achieved in spite of optimal clinical factors (including age at implantation, duration of rehabilitation and post-implant hearing level), which may be attributed to disorders at higher levels of the auditory pathway. We used cortical auditory evoked potentials to investigate the ability to perceive and discriminate auditory stimuli in 10 unsuccessful implant users aged 8–10 years (CI) and 10 healthy age-matched controls with normal hearing (NH). Pure tones (1 and 2 kHz) and double consonant-vowel syllables were applied. The stimuli were presented in an oddball paradigm that required the subjects to react consciously. The latencies and amplitudes of the P1, N1, P2, N2 and P3 waves were analyzed, in addition to reaction times and number of responses. Significant differences in the average response times and number of responses were observed between the CI and NH groups. The latencies also indicate that the CI group took longer to perceive and discriminate between tonal and speech auditory stimuli than the NH group.     

Beta‐band activity in auditory pathways reflects speech localization and recognition in bilateral cochlear implant users

In normal‐hearing listeners, localization of auditory speech involves stimulus processing in the postero‐dorsal pathway of the auditory system. In quiet environments, bilateral cochlear implant (CI) users show high speech recognition performance, but localization of auditory speech is poor, especially when discriminating stimuli from the same hemifield. Whether this difficulty relates to the inability of the auditory system to translate binaural electrical cues into neural signals, or to a functional reorganization of auditory cortical pathways following long periods of binaural deprivation is unknown. In this electroencephalography study, we examined the processing of auditory syllables in postlingually deaf adults with bilateral CIs and in normal‐hearing adults. Participants were instructed to either recognize (“recognition” task) or localize (“localization” task) the syllables. The analysis focused on event‐related potentials and oscillatory brain responses. N1 amplitudes in CI users were larger in the localization compared with recognition task, suggesting an enhanced stimulus processing effort in the localization task. Linear beamforming of oscillatory activity in CI users revealed stronger suppression of beta‐band activity after 200 ms in the postero‐dorsal auditory pathway for the localization compared with the recognition task. In normal‐hearing adults, effects for longer latency event‐related potentials were found, but no effects were observed for N1 amplitudes or beta‐band responses. Our study suggests that difficulties in speech localization in bilateral CI users are not reflected in a functional reorganization of cortical auditory pathways. New signal processing strategies of cochlear devices preserving unambiguous binaural cues may improve auditory localization performance in bilateral CI users. Hum Brain Mapp 35:3107–3121, 2014. © 2013 Wiley Periodicals, Inc .     

Neural correlates of verbal feedback processing: an fMRI study employing overt speech

Speakers use external auditory feedback to monitor their own speech. Feedback distortion has been found to increase activity in the superior temporal areas. Using fMRI, the present study investigates the neural correlates of processing verbal feedback without distortion. In a blocked design, the following conditions were presented: (1) overt picture-naming, (2) overt picture-naming while pink noise was presented to mask external feedback, (3) covert picture-naming, (4) listening to the picture names (previously recorded from participants' own voices), and (5) listening to pink noise. The results show that auditory feedback processing involves a network of different areas related to general performance monitoring and speech-motor control. These include the cingulate cortex and the bilateral insula, supplementary motor area, bilateral motor areas, cerebellum, thalamus and basal ganglia. Our findings suggest that the anterior cingulate cortex, which is often implicated in error-processing and conflict-monitoring, is also engaged in ongoing speech monitoring. Furthermore, in the superior temporal gyrus, we found a reduced response to speaking under normal feedback conditions. This finding is interpreted in the framework of a forward model according to which, during speech production, the sensory consequence of the speech-motor act is predicted to attenuate the sensitivity of the auditory cortex.     

The role of the salience network in processing lexical and nonlexical stimuli in cochlear implant users

Previous positron emission tomography (PET) studies have shown that various cortical areas are activated to process speech signal in cochlear implant (CI) users. Nonetheless, differences in task dimension among studies and low statistical power preclude from understanding sound processing mechanism in CI users. Hence, we performed activation likelihood estimation meta‐analysis of PET studies in CI users and normal hearing (NH) controls to compare the two groups. Eight studies (58 CI subjects/92 peak coordinates; 45 NH subjects/40 peak coordinates) were included and analyzed, retrieving areas significantly activated by lexical and nonlexical stimuli. For lexical and nonlexical stimuli, both groups showed activations in the components of the dual‐stream model such as bilateral superior temporal gyrus/sulcus, middle temporal gyrus, left posterior inferior frontal gyrus, and left insula. However, CI users displayed additional unique activation patterns by lexical and nonlexical stimuli. That is, for the lexical stimuli, significant activations were observed in areas comprising salience network (SN), also known as the intrinsic alertness network, such as the left dorsal anterior cingulate cortex (dACC), left insula, and right supplementary motor area in the CI user group. Also, for the nonlexical stimuli, CI users activated areas comprising SN such as the right insula and left dACC. Previous episodic observations on lexical stimuli processing using the dual auditory stream in CI users were reconfirmed in this study. However, this study also suggests that dual‐stream auditory processing in CI users may need supports from the SN. In other words, CI users need to pay extra attention to cope with degraded auditory signal provided by the implant. Hum Brain Mapp 36:1982–1994, 2015. © 2015 Wiley Periodicals, Inc.     

Bihemispheric network dynamics coordinating vocal feedback control

Modulation of vocal pitch is a key speech feature that conveys important linguistic and affective information. Auditory feedback is used to monitor and maintain pitch. We examined induced neural high gamma power (HGP) (65–150 Hz) using magnetoencephalography during pitch feedback control. Participants phonated into a microphone while hearing their auditory feedback through headphones. During each phonation, a single real‐time 400 ms pitch shift was applied to the auditory feedback. Participants compensated by rapidly changing their pitch to oppose the pitch shifts. This behavioral change required coordination of the neural speech motor control network, including integration of auditory and somatosensory feedback to initiate change in motor plans. We found increases in HGP across both hemispheres within 200 ms of pitch shifts, covering left sensory and right premotor, parietal, temporal, and frontal regions, involved in sensory detection and processing of the pitch shift. Later responses to pitch shifts (200–300 ms) were right dominant, in parietal, frontal, and temporal regions. Timing of activity in these regions indicates their role in coordinating motor change and detecting and processing of the sensory consequences of this change. Subtracting out cortical responses during passive listening to recordings of the phonations isolated HGP increases specific to speech production, highlighting right parietal and premotor cortex, and left posterior temporal cortex involvement in the motor response. Correlation of HGP with behavioral compensation demonstrated right frontal region involvement in modulating participant's compensatory response. This study highlights the bihemispheric sensorimotor cortical network involvement in auditory feedback‐based control of vocal pitch. Hum Brain Mapp 37:1474‐1485, 2016. © 2016 Wiley Periodicals, Inc.     

Out‐of‐synchrony speech entrainment in developmental dyslexia

Developmental dyslexia is a reading disorder often characterized by reduced awareness of speech units. Whether the neural source of this phonological disorder in dyslexic readers results from the malfunctioning of the primary auditory system or damaged feedback communication between higher‐order phonological regions (i.e., left inferior frontal regions) and the auditory cortex is still under dispute. Here we recorded magnetoencephalographic (MEG) signals from 20 dyslexic readers and 20 age‐matched controls while they were listening to ～10‐s‐long spoken sentences. Compared to controls, dyslexic readers had (1) an impaired neural entrainment to speech in the delta band (0.5–1 Hz); (2) a reduced delta synchronization in both the right auditory cortex and the left inferior frontal gyrus; and (3) an impaired feedforward functional coupling between neural oscillations in the right auditory cortex and the left inferior frontal regions. This shows that during speech listening, individuals with developmental dyslexia present reduced neural synchrony to low‐frequency speech oscillations in primary auditory regions that hinders higher‐order speech processing steps. The present findings, thus, strengthen proposals assuming that improper low‐frequency acoustic entrainment affects speech sampling. This low speech‐brain synchronization has the strong potential to cause severe consequences for both phonological and reading skills. Interestingly, the reduced speech‐brain synchronization in dyslexic readers compared to normal readers (and its higher‐order consequences across the speech processing network) appears preserved through the development from childhood to adulthood. Thus, the evaluation of speech‐brain synchronization could possibly serve as a diagnostic tool for early detection of children at risk of dyslexia. Hum Brain Mapp 37:2767–2783, 2016. © 2016 Wiley Periodicals, Inc .     

Predicting auditory feedback control of speech production from subregional shape of subcortical structures

Although a growing body of research has focused on the cortical sensorimotor mechanisms that support auditory feedback control of speech production, much less is known about the subcortical contributions to this control process. This study examined whether subregional anatomy of subcortical structures assessed by statistical shape analysis is associated with vocal compensations and cortical event‐related potentials in response to pitch feedback errors. The results revealed significant negative correlations between the magnitudes of vocal compensations and subregional shape of the right thalamus, between the latencies of vocal compensations and subregional shape of the left caudate and pallidum, and between the latencies of cortical N1 responses and subregional shape of the left putamen. These associations indicate that smaller local volumes of the basal ganglia and thalamus are predictive of slower and larger neurobehavioral responses to vocal pitch errors. Furthermore, increased local volumes of the left hippocampus and right amygdala were predictive of larger vocal compensations, suggesting that there is an interplay between the memory‐related subcortical structures and auditory‐vocal integration. These results, for the first time, provide evidence for differential associations of subregional morphology of the basal ganglia, thalamus, hippocampus, and amygdala with neurobehavioral processing of vocal pitch errors, suggesting that subregional shape measures of subcortical structures can predict behavioral outcome of auditory‐vocal integration and associated neural features. Hum Brain Mapp 39:459–471, 2018. © 2017 Wiley Periodicals, Inc.     

Neurophysiological evidence of impaired musical sound perception in cochlear-implant users

Objective

Music perception with a cochlear implant (CI) can be unsatisfactory because current-day implants are primarily designed to enable speech discrimination. The present study aimed at evaluating electrophysiological correlates of musical sound perception in CI users to help achieve the long-term goal of improved restoration of hearing in those individuals.

Methods

Auditory discrimination accuracy in adult CI users (n = 12) and matched normal-hearing controls (n = 12) was measured by behavioral discrimination tasks and mismatch negativity (MMN) recordings. Discrimination profiles were obtained by using a set of clarinet sounds (original/vocoded) varying along different acoustic dimensions (frequency/intensity/duration) and deviation magnitudes (four levels).

Results

Behavioral results and MMN recordings revealed reduced auditory discrimination accuracy in CI users. An inverse relationship was found between MMN amplitudes and duration of profound deafness.

Conclusions

CI users have difficulties in discriminating small changes in the acoustic properties of musical sounds. The recently developed multi-feature MMN paradigm (Pakarinen et al., 2007) can be used to objectively evaluate discrimination abilities of CI users for musical sounds.

Significance

Measuring auditory discrimination functions by means of a multi-feature MMN paradigm could be of substantial clinical value by providing a comprehensive profile of the extent of restored hearing in CI users.     

Neurophysiological evidence of differential mechanisms involved in producing opposing and following responses to altered auditory feedback

Objective

When hearing perturbations in voice auditory feedback, people produce responses that mostly oppose the perturbation direction, whereas a few responses follow the direction of feedback perturbation. The causes of opposing and following responses, however, remain poorly understood. The present event-related potential (ERP) study sought to examine the neurophysiological processing of opposing and following responses to pitch feedback perturbations during self-monitoring of vocal production.

Method

Twelve Mandarin-native speakers participated in the experiment. Vocal and neurophysiological responses to pitch perturbations (±50 and ±200 cents) in voice auditory feedback were measured. Individual-trial responses were categorized according to the response direction and then separately averaged in groups of opposing and following responses. ERPs indexed by the P1-N1-P2 complex corresponding to two types of vocal responses were also obtained.

Results

Opposing and following vocal responses did not differ in the magnitude, but there were greater proportions of opposing to following responses to 50 cents stimuli. The amplitude and latency of the P1 and N1 components showed none of significance across conditions, whereas there was a direction × magnitude effect on the P2 response. Following responses elicited greater P2 amplitudes than opposing responses only when pitch feedback was perturbed for downward 200 cents, and upward pitch perturbation elicited greater P2 amplitudes than those with downward direction only in the production of opposing responses.

Conclusion

These findings demonstrate that cortical processing of opposing responses is different from that of following responses, which can be modulated by the physical properties of feedback perturbation.

Significance

Different neural mechanisms are involved in the production of opposing and following responses to feedback perturbations during self-monitoring of vocal production.     

Sensory preference in speech production revealed by simultaneous alteration of auditory and somatosensory feedback

The idea that humans learn and maintain accurate speech by carefully monitoring auditory feedback is widely held. But this view neglects the fact that auditory feedback is highly correlated with somatosensory feedback during speech production. Somatosensory feedback from speech movements could be a primary means by which cortical speech areas monitor the accuracy of produced speech. We tested this idea by placing the somatosensory and auditory systems in competition during speech motor learning. To do this, we combined two speech-learning paradigms to simultaneously alter somatosensory and auditory feedback in real time as subjects spoke. Somatosensory feedback was manipulated by using a robotic device that altered the motion path of the jaw. Auditory feedback was manipulated by changing the frequency of the first formant of the vowel sound and playing back the modified utterance to the subject through headphones. The amount of compensation for each perturbation was used as a measure of sensory reliance. All subjects were observed to correct for at least one of the perturbations, but auditory feedback was not dominant. Indeed, some subjects showed a stable preference for either somatosensory or auditory feedback during speech.     

Chronic Bilateral Cochlear Implant Stimulation Partially Restores Neural Binaural Sensitivity in Neonatally-Deaf Rabbits

Cochlear implant (CI) users with a prelingual onset of hearing loss show poor sensitivity to interaural time differences (ITDs), an important cue for sound localization and speech reception in noise. Similarly, neural ITD sensitivity in the inferior colliculus (IC) of neonatally-deafened animals is degraded compared with animals deafened as adults. Here, we show that chronic bilateral CI stimulation during development can partly reverse the effect of early-onset deafness on ITD sensitivity. The prevalence of ITD sensitive neurons was restored to the level of adult-deaf (AD) rabbits in the early-deaf rabbits of both sexes that received chronic stimulation and behavioral training with wearable bilateral sound processors during development. We also found a partial improvement in neural ITD sensitivity in the early-deaf and stimulated rabbits compared with unstimulated rabbits. In contrast, chronic CI stimulation did not improve temporal coding in early-deaf rabbits. The present study is the first report showing functional restoration of ITD sensitivity with CI stimulation in single neurons and highlights the importance of auditory experience during development on the maturation of binaural circuitry.SIGNIFICANCE STATEMENT Although cochlear implants (CI) are highly successful in providing speech reception in quiet for many profoundly deaf people, CI users still face difficulty in noisy everyday environment. This is partly because of their poor sensitivity to differences in the timing of sounds arriving at the two ears [interaural time differences (ITDs)], which help to identify where the sound is coming from. This problem is especially acute in those who lost hearing early in life. Here, we present the first report that sensitivity of auditory neurons to ITDs is restored by CI stimulation during development in an animal model of neonatal deafness. These findings highlight the importance of providing early binaural auditory experience with CIs in deaf children.     

Possible role of abnormal auditory feedback in conduction aphasia

A 63-yr-old man with conduction aphasia was asked to repeat 4 monosyllables, 1 polysyllable, and a sentence under normal conditions and under Delayed Auditory Feedback (DAF). In 20 normal controls, DAF produced louder and slower speech and linguistic errors similar to those found in mild expressive aphasia. In contrast, the patient with conduction aphasia showed little increase in intensity and spoke faster and with fewer linguistic errors under DAF than under normal conditions. The paradoxical improvement inducedby DAF in this patient suggests that abnormal auditory feedback may cause some of the features of conduction aphasia.     

Comparison of auditory stimulus processing in normal and autistic adolescents

This experiment investigated the possibility that autistic adolescents may avoid speech communication with the world around them by tuning out or perceptually suppressing auditory speech stimuli. The tune-out auditory supression hypothesis was investigated using the subject's own speech as the stimulus under three perceptual-motor conditions: (1) with speech in a delayed auditory feedback (DAF) mode, (2) with a white noise masking speech mode, and (3) with speech in a normal, quiet listening mode. Five autistic adolescents were compared with six normal controls on speech time duration and sound level. DAF increased the speech sound pressure level (SPL) and increased speech time duration for both groups.     

Word segmentation in phonemically identical and prosodically different sequences using cochlear implants: A case study

Cochlear implant (CI) users frequently achieve good speech understanding based on phoneme and word recognition. However, there is a significant variability between CI users in processing prosody. The aim of this study was to examine the abilities of an excellent CI user to segment continuous speech using intonational cues. A post-lingually deafened adult CI user and 22 normal hearing (NH) subjects segmented phonemically identical and prosodically different sequences in French such as ‘l’affiche’ (the poster) versus ‘la fiche’ (the sheet), both [lafi?]. All participants also completed a minimal pair discrimination task. Stimuli were presented in auditory-only and audiovisual presentation modalities. The performance of the CI user in the minimal pair discrimination task was 97% in the auditory-only and 100% in the audiovisual condition. In the segmentation task, contrary to the NH participants, the performance of the CI user did not differ from the chance level. Visual speech did not improve word segmentation. This result suggests that word segmentation based on intonational cues is challenging when using CIs even when phoneme/word recognition is very well rehabilitated. This finding points to the importance of the assessment of CI users’ skills in prosody processing and the need for specific interventions focusing on this aspect of speech communication.