Visual contrast sensitivity in deaf versus hearing populations: exploring the perceptual consequences of auditory deprivation and experience with a visual language

Early deafness in humans provides a unique opportunity to examine the perceptual consequences of altered sensory experience. In particular, visual perception in the deaf may be altered as a result of their auditory deprivation and/or because the deaf rely heavily upon a visual language (American Sign Language, or ASL, in the US). Recently, we found that deaf, but not hearing, subjects exhibit a right visual field/left hemisphere advantage on a low-level direction of motion task, a finding that has been attributed to the deaf's experience with ASL [Psychol. Sci. 10 (1999) 256; Brain Res. 405 (1987) 268]. In order to determine whether this visual field asymmetry generalizes to other low-level visual functions, in this study we measured contrast sensitivity in deaf and hearing subjects to moving stimuli over a range of speeds (0.125-64 degrees /s). We hypothesized that if ASL use drives differences between hearing and deaf subjects, such differences may occur over a restricted range of speeds most commonly found in ASL. In addition, we tested a third group, hearing native signers who learned ASL early from their deaf parents, to further assess whether potential differences between groups results from ASL use. These experiments reveal no overall differences in contrast sensitivity, nor differences in visual field asymmetries, across subject groups at any speed tested. Thus, differences previously observed between deaf and hearing subjects for discriminating the direction of moving stimuli do not generalize to contrast sensitivity for moving stimuli, a result that has implications for the neural level at which plastic changes occur in the visual system of deaf subjects.     

Altered cross-modal processing in the primary auditory cortex of congenitally deaf adults: a visual-somatosensory fMRI study with a double-flash illusion

The developing brain responds to the environment by using statistical correlations in input to guide functional and structural changes-that is, the brain displays neuroplasticity. Experience shapes brain development throughout life, but neuroplasticity is variable from one brain system to another. How does the early loss of a sensory modality affect this complex process? We examined cross-modal neuroplasticity in anatomically defined subregions of Heschl's gyrus, the site of human primary auditory cortex, in congenitally deaf humans by measuring the fMRI signal change in response to spatially coregistered visual, somatosensory, and bimodal stimuli. In the deaf Heschl's gyrus, signal change was greater for somatosensory and bimodal stimuli than that of hearing participants. Visual responses in Heschl's gyrus, larger in deaf than hearing, were smaller than those elicited by somatosensory stimulation. In contrast to Heschl's gyrus, in the superior-temporal cortex visual signal was comparable to somatosensory signal. In addition, deaf adults perceived bimodal stimuli differently; in contrast to hearing adults, they were susceptible to a double-flash visual illusion induced by two touches to the face. Somatosensory and bimodal signal change in rostrolateral Heschl's gyrus predicted the strength of the visual illusion in the deaf adults in line with the interpretation that the illusion is a functional consequence of the altered cross-modal organization observed in deaf auditory cortex. Our results demonstrate that congenital and profound deafness alters how vision and somatosensation are processed in primary auditory cortex.     

Sensory mapping in a congenitally deaf subject: MEG and fRMI studies of cross-modal non-plasticity

It has been proposed that the auditory cortex of deaf subjects may provide an example of cross-modal compensatory plasticity. We investigated whether sensory stimulation could elicit responses from auditory areas of a congenitally deaf subject. Neuromagnetic fields were recorded using a 37-channel biomagnetometer under conditions of: 1) visual stimulation; 2) somatosensory stimulation; and 3) a simple motor task. Visual items were reversing checkerboards and single light spots, presented in various portions of the visual field; somatosensory stimuli were pneumatic taps delivered to individual digit-segments and the lip; the motor task was self-paced finger tapping. In addition, functional magnetic resonance imaging was used to observe the activation elicited by full-field checkerboard and sign language stimuli. No responses to passively presented visual or somatosensory stimuli were observed in the auditory cortex. In contrast, somatosensory, motor, and visual cortices revealed evoked magnetic responses comparable to those from control subjects, indicating canonical anatomic and physiological organization in these areas. These data suggest that primary projection areas do not reveal obvious plastic effects. We suggest that in the human auditory cortex compensatory plasticity emerges primarily as a property of non-primary areas and is best observed under attentionally demanding conditions. Hum. Brain Mapping 5:437–444, 1997. © 1997 Wiley-Liss, Inc.     

Visual stimuli activate auditory cortex in deaf subjects: evidence from MEG

Studies using fMRI have demonstrated that visual stimuli activate auditory cortex in deaf subjects. Given the low temporal resolution of fMRI, it is uncertain whether this activation is associated with initial stimulus processing. Here, we used MEG in deaf and hearing subjects to evaluate whether auditory cortex, devoid of its normal input, comes to serve the visual modality early in the course of stimulus processing. In line with previous findings, visual activity was observed in the auditory cortex of deaf, but not hearing, subjects. This activity occurred within 100-400 ms of stimulus presentation and was primarily over the right hemisphere. These results add to the mounting evidence that removal of one sensory modality in humans leads to neural reorganization of the remaining modalities.     

Visual stimuli can impair auditory processing in cochlear implant users

It has been shown that visual stimulation can activate cortical regions normally devoted to auditory processing in deaf individuals. This neural activity can persist even when audition is restored through the implantation of a cochlear implant, raising the possibility that cross-modal plasticity can be detrimental to auditory performance in cochlear implant users. To determine the influence of visual information on auditory performance after restoration of hearing in deaf individuals, the ability to segregate conflicting auditory and visual information was assessed in fourteen cochlear implant users with varied degree of expertise and an equal number of participants with normal-hearing matched for gender, age and hearing performance. An auditory speech recognition task was administered in the presence of three incongruent visual stimuli (color-shift, random-dot motion and lip movement). For proficient cochlear implant users, auditory performance was equal to that of controls in the three experimental conditions where visual stimuli were presented simultaneously with auditory information. For non-proficient cochlear implant users, performance did not differ from that of matched controls when the auditory stimulus was paired with a visual stimulus that was color-shifted. However, significant differences were observed between the non-proficient cochlear implant users and their matched controls when the accompanying visual stimuli consisted of a moving random-dot pattern or incongruent lip movements. These findings raise several questions with regards to the rehabilitation of cochlear implant users.     

The Extent of Task Specificity for Visual and Tactile Sequences in the Auditory Cortex of the Deaf and Hard of Hearing

It has been proposed that the auditory cortex in the deaf humans might undergo task-specific reorganization. However, evidence remains scarce as previous experiments used only two very specific tasks (temporal processing and face perception) in visual modality. Here, congenitally deaf/hard of hearing and hearing women and men were enrolled in an fMRI experiment as we sought to fill this evidence gap in two ways. First, we compared activation evoked by a temporal processing task performed in two different modalities, visual and tactile. Second, we contrasted this task with a perceptually similar task that focuses on the spatial dimension. Additional control conditions consisted of passive stimulus observation. In line with the task specificity hypothesis, the auditory cortex in the deaf was activated by temporal processing in both visual and tactile modalities. This effect was selective for temporal processing relative to spatial discrimination. However, spatial processing also led to significant auditory cortex recruitment which, unlike temporal processing, occurred even during passive stimulus observation. We conclude that auditory cortex recruitment in the deaf and hard of hearing might involve interplay between task-selective and pluripotential mechanisms of cross-modal reorganization. Our results open several avenues for the investigation of the full complexity of the cross-modal plasticity phenomenon.SIGNIFICANCE STATEMENT Previous studies suggested that the auditory cortex in the deaf may change input modality (sound to vision) while keeping its function (e.g., rhythm processing). We investigated this hypothesis by asking deaf or hard of hearing and hearing adults to discriminate between temporally and spatially complex sequences in visual and tactile modalities. The results show that such function-specific brain reorganization, as has previously been demonstrated in the visual modality, also occurs for tactile processing. On the other hand, they also show that for some stimuli (spatial) the auditory cortex activates automatically, which is suggestive of a take-over by a different kind of cognitive function. The observed differences in processing of sequences might thus result from an interplay of task-specific and pluripotent plasticity.     

Evolution of crossmodal reorganization of the voice area in cochlear-implanted deaf patients

Psychophysical and neuroimaging studies in both animal and human subjects have clearly demonstrated that cortical plasticity following sensory deprivation leads to a brain functional reorganization that favors the spared modalities. In postlingually deaf patients, the use of a cochlear implant (CI) allows a recovery of the auditory function, which will probably counteract the cortical crossmodal reorganization induced by hearing loss. To study the dynamics of such reversed crossmodal plasticity, we designed a longitudinal neuroimaging study involving the follow-up of 10 postlingually deaf adult CI users engaged in a visual speechreading task. While speechreading activates Broca's area in normally hearing subjects (NHS), the activity level elicited in this region in CI patients is abnormally low and increases progressively with post-implantation time. Furthermore, speechreading in CI patients induces abnormal crossmodal activations in right anterior regions of the superior temporal cortex normally devoted to processing human voice stimuli (temporal voice-sensitive areas-TVA). These abnormal activity levels diminish with post-implantation time and tend towards the levels observed in NHS. First, our study revealed that the neuroplasticity after cochlear implantation involves not only auditory but also visual and audiovisual speech processing networks. Second, our results suggest that during deafness, the functional links between cortical regions specialized in face and voice processing are reallocated to support speech-related visual processing through cross-modal reorganization. Such reorganization allows a more efficient audiovisual integration of speech after cochlear implantation. These compensatory sensory strategies are later completed by the progressive restoration of the visuo-audio-motor speech processing loop, including Broca's area.     

Attention to central and peripheral visual space in a movement detection task. III. Separate effects of auditory deprivation and acquisition of a visual language

We employed event-related brain potentials (ERPs) and measures of signal detectability to compare attention to peripheral and central visual stimuli in normal hearing subjects who were born to deaf parents (HD Ss) and whose first language was American Sign Language (ASL). The results were compared with those obtained from normal hearing Ss and congenitally deaf Ss in the same paradigm. Task performance and ERPs during attention to the foveal region were similar in the 3 groups. In contrast, with attention to the peripheral stimuli the deaf Ss displayed attention effects over the occipital regions of both hemispheres that were several times larger than those in the hearing and the HD Ss. However, both HD and deaf Ss displayed lateral asymmetries in behavior and ERPs that were opposite in direction to those of the hearing Ss. Whereas hearing Ss detected the direction of target motion better when it occurred in the left visual field, deaf and HD Ss performed better for right visual field targets. Consistent with these results, the amplitude of the attention-related increases in the ERPs were larger from temporal and parietal regions of the right than the left hemisphere in hearing Ss, but were larger from the left than the right hemisphere in both the HD and the deaf Ss. These results suggest that auditory deprivation and the acquisition of a visual language have marked and different effects on the development of cortical specializations in humans.     

Vibrotactile activation of the auditory cortices in deaf versus hearing adults

Neuroplastic changes in auditory cortex as a result of lifelong perceptual experience were investigated. Adults with early-onset deafness and long-term hearing aid experience were hypothesized to have undergone auditory cortex plasticity due to somatosensory stimulation. Vibrations were presented on the hand of deaf and normal-hearing participants during functional MRI. Vibration stimuli were derived from speech or were a fixed frequency. Higher, more widespread activity was observed within auditory cortical regions of the deaf participants for both stimulus types. Life-long somatosensory stimulation due to hearing aid use could explain the greater activity observed with deaf participants.     

Ultrasound activates the auditory cortex of profoundly deaf subjects

Using three-dimensional PET, the cortical areas activated by bone-conducted ultrasound were measured from five profoundly deaf subjects and compared with the cortical areas of normal-hearing subjects activated by stimuli through bone-conducted ultrasonic, air-conducted, bone-conducted, and vibro-tactile hearing aids. All of the hearing aids, including the ultrasonic hearing aid, consistently activated the medial portion of the primary auditory cortex of the normal volunteers. The same cortical area was also significantly activated in the profoundly deaf subjects although the percentage increase in regional cerebral blood flow (rCBF) was smaller than in normal subjects. These results suggest that extra-cochlear routes convey information to the primary auditory cortex and can therefore produce detectable sound sensation even in the profoundly deaf subjects, who reported a sensation themselves.     

Morphometric comparison of the human corpus callosum in deaf and hearing subjects: an MRI study

Auditory cortices are interconnected to each other by fibers passing through the corpus callosum (CC). In totally deaf persons no auditory impulses are conveyed to the auditory cortices, hence the auditory pathways become nonfunctional. It was reported that there has been cross-modal plasticity between auditory, visual, and somatosensory cortices. In this study, our aim was to make a comparison in the CC morphometry in hearing subjects and in a selected group in which the auditory system was deprived before the age of 2. 18 deaf and 18 hearing male, handedness matched volunteers, ages varying between 28 and 56 years old were examined. Audiometrical tests were applied to both groups and then T1-weighted midsagittal MR images were obtained. Certain dimensions and areas were measured on these images. There were no statistically significant difference between deaf and hearing subjects, either when dimensions and areas were analyzed by multivariate analysis of variance or when areas were analyzed by univariate analysis of variance. Absence of any significant morphometric difference in the CC of deaf subjects could be thought as an evidence of reflection of functional cortical plasticity.     

Auditory motion direction encoding in auditory cortex and high-level visual cortex

The aim of this functional magnetic resonance imaging (fMRI) study was to identify human brain areas that are sensitive to the direction of auditory motion. Such directional sensitivity was assessed in a hypothesis-free manner by analyzing fMRI response patterns across the entire brain volume using a spherical-searchlight approach. In addition, we assessed directional sensitivity in three predefined brain areas that have been associated with auditory motion perception in previous neuroimaging studies. These were the primary auditory cortex, the planum temporale and the visual motion complex (hMT/V5+). Our whole-brain analysis revealed that the direction of sound-source movement could be decoded from fMRI response patterns in the right auditory cortex and in a high-level visual area located in the right lateral occipital cortex. Our region-of-interest-based analysis showed that the decoding of the direction of auditory motion was most reliable with activation patterns of the left and right planum temporale. Auditory motion direction could not be decoded from activation patterns in hMT/V5+. These findings provide further evidence for the planum temporale playing a central role in supporting auditory motion perception. In addition, our findings suggest a cross-modal transfer of directional information to high-level visual cortex in healthy humans.     

Cross-modal selective attention to visual and auditory stimuli modulates endogenous ERP components

In two experiments event-related potentials (ERPs) to visual and auditory stimuli were measured in 12 healthy subjects. A cross-modal and delayed response paradigm was used that allows ERPs to be obtained separately to attended and unattended stimuli under conditions in which unattended stimuli are less likely to be covertly or randomly attended. The results showed: (1) N1 enhancement with attention for standard stimuli in auditory and visual modalities and for deviant stimuli in the visual modality; (2) The onset time and scalp distribution of both the N1 for attend condition and Nd1 were similar regardless of standard or deviant stimuli in the auditory and visual modality; the onset time of Nd1 elicited by auditory and visual deviant stimuli was earlier than that of the unattended N1, and their scalp distributions were different; and (3) The Nd1 components elicited by auditory and visual deviant stimuli were distributed over the respective primary sensory areas, but Nd1 components evoked by auditory and visual standard stimuli were distributed over the frontal scalp. These results suggest that the attended N1 enhancement is primarily caused by a component with endogenous origins and that the early attention effect occurs before the exogenous components. The results support the view that the cross-modal attention to deviant stimuli modulates modality-specific processing in the brain, whereas attention to standard stimuli affects modality-nonspecific or supramodal brain systems.     

Cross‐modal reorganization of cortical afferents to dorsal auditory cortex following early‐ and late‐onset deafness

Cat auditory cortex is known to undergo cross‐modal reorganization following deafness, such that behavioral advantages in visual motion detection are abolished when a specific region of deaf auditory cortex, the dorsal zone (DZ), is deactivated. The purpose of the present investigation was to examine the connectional adaptations that might subserve this plasticity. We deposited biotinylated dextran amine (BDA; 3,000 MW), a retrograde tracer, unilaterally into the posterior portion of the suprasylvian fringe, corresponding to area DZ of hearing, early‐deafened (onset <1 month), and late‐deafened (onset >3 months) cats to reveal cortical afferent projections. Overall, the pattern of cortical projections to DZ was similar in both hearing and deafened animals. However, there was a progressive increase in projection strength among hearing and late‐ and early‐deafened cats from an extrastriate visual cortical region known to be involved in the processing of visual motion, the posterolateral lateral suprasylvian area (PLLS). Additionally, although no such change was documented for the posteromedial lateral suprasylvian area (PMLS), labeled neurons were present within a subregion of PMLS devoted to foveal vision in both late‐ and early‐deafened animals but not in hearing controls. PMLS is also an extrastriate visual motion processing area and is widely considered to be the homolog of primate middle temporal area. No changes in auditory cortical connectivity were observed among groups. These observations suggest that amplified cortical projections from extrastriate visual areas involved in visual motion processing to DZ may contribute to the cross‐modal reorganization that functionally manifests as superior visual motion detection ability in the deaf animal. J. Comp. Neurol. 522:654–675, 2014. © 2013 Wiley Periodicals, Inc.     

Neural correlates of auditory repetition priming: reduced fMRI activation in the auditory cortex

Repetition priming refers to enhanced or biased performance with repeatedly presented stimuli. Modality-specific perceptual repetition priming has been demonstrated behaviorally for both visually and auditorily presented stimuli. In functional neuroimaging studies, repetition of visual stimuli has resulted in reduced activation in the visual cortex, as well as in multimodal frontal and temporal regions. The reductions in sensory cortices are thought to reflect plasticity in modality-specific neocortex. Unexpectedly, repetition of auditory stimuli has resulted in reduced activation in multimodal and visual regions, but not in the auditory temporal lobe cortex. This finding puts the coupling of perceptual priming and modality-specific cortical plasticity into question. Here, functional magnetic resonance imaging was used with environmental sounds to reexamine whether auditory priming is associated with reduced activation in the auditory cortex. Participants heard environmental sounds (e.g., animals, machines, musical instruments, etc.) in blocks, alternating between initial and repeated presentations, and decided whether or not each sound was produced by an animal. Repeated versus initial presentations of sounds resulted in repetition priming (faster responses) and reduced activation in the right superior temporal gyrus, bilateral superior temporal sulci, and right inferior prefrontal cortex. The magnitude of behavioral priming correlated positively with reduced activation in these regions. This indicates that priming for environmental sounds is associated with modification of neural activation in modality-specific auditory cortex, as well as in multimodal areas.     

Differential modification of cortical and thalamic projections to cat primary auditory cortex following early‐ and late‐onset deafness

Following sensory deprivation, primary somatosensory and visual cortices undergo crossmodal plasticity, which subserves the remaining modalities. However, controversy remains regarding the neuroplastic potential of primary auditory cortex (A1). To examine this, we identified cortical and thalamic projections to A1 in hearing cats and those with early‐ and late‐onset deafness. Following early deafness, inputs from second auditory cortex (A2) are amplified, whereas the number originating in the dorsal zone (DZ) decreases. In addition, inputs from the dorsal medial geniculate nucleus (dMGN) increase, whereas those from the ventral division (vMGN) are reduced. In late‐deaf cats, projections from the anterior auditory field (AAF) are amplified, whereas those from the DZ decrease. Additionally, in a subset of early‐ and late‐deaf cats, area 17 and the lateral posterior nucleus (LP) of the visual thalamus project concurrently to A1. These results demonstrate that patterns of projections to A1 are modified following deafness, with statistically significant changes occurring within the auditory thalamus and some cortical areas. Moreover, we provide anatomical evidence for small‐scale crossmodal changes in projections to A1 that differ between early‐ and late‐onset deaf animals, suggesting that potential crossmodal activation of primary auditory cortex differs depending on the age of deafness onset. J. Comp. Neurol. 523:2297–2320, 2015. © 2015 Wiley Periodicals, Inc.     

Cross-modal plasticity in deaf subjects dependent on the extent of hearing loss

Cross-modal plasticity in deaf subjects is still discussed controversial. We tried to figure out whether the plasticity is dependent on the extent of hearing loss. Three groups of volunteers, comprising twelve individuals each, were investigated. They were characterized by three distinctive features, one had normal hearing, the other one lost hearing and the third had only minimal residual hearing ability. All participants, except those of group one, were capable of using German Sign Language (GSL). The groups were studied with functional MRI in a standard block design during individuals' watching sign language videos alternating with black frame. During sign language conditions, deaf subjects revealed a significant activation of the auditory cortex in both hemispheres comprising Brodmann areas (BA) 42 and 22 corresponding to the secondary associative auditory areas. Additionally, activation of the angular and supramarginal gyrus was seen. Activation of the primary auditory cortex was revealed in deaf subjects with total hearing loss during sign language tasks but not in subjects with residual hearing ability. In conclusion our results indicate a cortical reorganization of the auditory cortex comprising primary auditory fields only present in subjects with total hearing loss.     

Altered visual-evoked potentials in congenitally deaf adults

Visual-evoked potentials recorded from the scalp of congenitally deaf adults were significantly larger over both auditory and visual cortical areas than in normal hearing adults. Over temporal and frontal areas peripheral stimuli presented at long intervals elicited N150 components which were larger in deaf than in hearing subjects. Over occipital and parietal areas peripheral and foveal stimuli elicited larger P230 components in deaf than in hearing subjects. These results imply that early auditory experience influences the organization of the human brain for visual processing.     

Attention to central and peripheral visual space in a movement detection task: an event-related potential and behavioral study. II. Congenitally deaf adults

We compared the effects of focussed attention upon event-related brain potentials (ERPs) to peripherally and centrally located visual stimuli in congenitally deaf subjects (Ss). The results were compared with those obtained from a group of normal hearing Ss in the same paradigm. ERPs from deaf and hearing Ss displayed similar attention-related changes with attention to the centrally located stimuli. These included enhanced amplitudes of the N1 component (157 ms) over the occipital regions of both hemispheres. By contrast, with attention to peripheral visual stimuli, ERPs from deaf Ss displayed attention-related increases that were several times larger than those from hearing Ss and different in scalp distribution. Whereas for hearing Ss the principal effects of attention to peripheral stimuli occurred over the contralateral parietal region, in deaf Ss the effects were also observed over the occipital regions of both hemispheres. In addition, lateral asymmetries in behavior and the ERPs indicated a greater role for the right hemisphere in this task in hearing Ss, but predominance of the left hemisphere in deaf Ss. These results suggest that auditory deprivation since birth has major effects on the development of the peripheral visual system. The specific pattern of group differences is discussed in relation to other studies of the effects of unimodal deprivation on the development of remaining modalities.     

Attending to visual or auditory motion affects perception within and across modalities: an event-related potential study

The present event-related potential (ERP) study examined the role of dynamic features in multisensory binding. It was tested whether endogenous attention to the direction of motion affects processing of visual and auditory stimuli within and across modalities. Human participants perceived horizontally moving dot patterns and sounds that were presented either continuously (standards) or briefly interrupted (infrequent deviants). Their task was to detect deviants moving in a particular direction within a primary modality, but to detect all deviants irrespective of their motion direction within the secondary modality. Attending to the direction of visual motion resulted in a broad selection negativity (SN) starting at about 200 ms post-stimulus onset, and attending to the direction of auditory motion resulted in a positive difference wave at 150 ms that was followed by a broad negativity starting at about 200 ms (unimodal effects). Moreover, dot patterns moving in a direction that was attended within audition were detected faster and more accurately than oppositely moving stimuli and elicited a cross-modal SN wave. Corresponding cross-modal behavioural and ERP results were obtained for sounds moving in a direction that was attended within vision. Unimodal and cross-modal ERP attention effects partially differed in their scalp topography. The present study shows that dynamic features (direction of motion) may be used to link input across modalities and demonstrates for the first time that these multisensory interactions take place as early as about 200 ms after stimulus onset.