The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus

The spatial register of the different receptive fields of multisensory neurons in the superior colliculus (SC) plays a significant role in determining the responses of these neurons to cross-modal stimulus combinations. Spatially coincident visual-auditory stimuli fall within these overlapping receptive fields and generally produce response enhancements that exceed the individual modality-specific responses and can exceed their sum. Yet, in this context, it has not been clear how "spatial coincidence" is operationally defined. Given the large size of SC receptive fields, visual and auditory stimuli could be within their respective receptive fields even when there are substantial spatial disparities between them. Indeed, previous observations have raised the possibility that there may be a second level of determinism in how SC neurons deal with the relative spatial locations of within-field cross-modal stimuli; specifically, that multisensory response enhancements become progressively weaker as the within-field visual and auditory stimuli become increasingly disparate. While the present experiments demonstrated that SC multisensory neurons have heterogeneous receptive fields, and that the greatest number of impulses evoked were by stimuli that fell within the area of cross-modal receptive field overlap, they also indicate that there is no systematic relationship between cross-modal stimulus disparity and the magnitude of multisensory response enhancement. Thus, two within-field cross-modal stimuli produced the same proportionate change (i.e., multisensory response enhancement) when they were widely disparate as they did when they overlapped one another in space. These observations indicate that cross-modal spatial coincidence can be defined operationally by the borders of an SC neuron's receptive fields regardless of the size of those receptive fields and/or the absolute spatial disparity between within-field cross-modal stimuli. Electronic Publication     

Two cortical areas mediate multisensory integration in superior colliculus neurons

The majority of multisensory neurons in the cat superior colliculus (SC) are able to synthesize cross-modal cues (e.g., visual and auditory) and thereby produce responses greater than those elicited by the most effective single modality stimulus and, sometimes, greater than those predicted by the arithmetic sum of their modality-specific responses. The present study examined the role of corticotectal inputs from two cortical areas, the anterior ectosylvian sulcus (AES) and the rostral aspect of the lateral suprasylvian sulcus (rLS), in producing these response enhancements. This was accomplished by evaluating the multisensory properties of individual SC neurons during reversible deactivation of these cortices individually and in combination using cryogenic deactivation techniques. Cortical deactivation eliminated the characteristic multisensory response enhancement of nearly all SC neurons but generally had little or no effect on a neuron's modality-specific responses. Thus, the responses of SC neurons to combinations of cross-modal stimuli were now no different from those evoked by one or the other of these stimuli individually. Of the two cortical areas, AES had a much greater impact on SC multisensory integrative processes, with nearly half the SC neurons sampled dependent on it alone. In contrast, only a small number of SC neurons depended solely on rLS. However, most SC neurons exhibited dual dependencies, and their multisensory enhancement was mediated by either synergistic or redundant influences from AES and rLS. Corticotectal synergy was evident when deactivating either cortical area compromised the multisensory enhancement of an SC neuron, whereas corticotectal redundancy was evident when deactivation of both cortical areas was required to produce this effect. The results suggest that, although multisensory SC neurons can be created as a consequence of a variety of converging tectopetal afferents that are derived from a host of subcortical and cortical structures, the ability to synthesize cross-modal inputs, and thereby produce an enhanced multisensory response, requires functional inputs from the AES, the rLS, or both.     

The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior

Physiological and behavioral studies in cat have shown that corticotectal influences play important roles in the information-processing capabilities of superior colliculus (SC) neurons. While corticotectal inputs from the anterior ectosylvian sulcus (AES) play a comparatively small role in the unimodal responses of SC neurons, they are particularly important in rendering these neurons capable of integrating information from different sensory modalities (e.g., visual and auditory). The present experiments examined the behavioral consequences of depriving SC neurons of AES inputs, and thereby compromising their ability to integrate visual and auditory information. Selective deactivation of a variety of other cortical areas (posterolateral lateral suprasylvian cortex, PLLS; primary auditory cortex, AI; or primary visual cortex, 17/18) served as controls. Cats were trained in a perimetry device to ignore a brief, low-intensity auditory stimulus but to orient toward and approach a nearthreshold visual stimulus (a light-emitting diode, LED) to obtain food. The LED was presented at different eccentricities either alone (unimodal) or combined with the auditory stimulus (multisensory). Subsequent deactivation of the AES, with focal injections of a local anesthetic, had no effect on responses to unimodal cues regardless of their location. However, it profoundly, though reversibly, altered orientation and approach to multisensory stimuli in contralateral space. The characteristic enhancement of these responses observed when an auditory cue was presented in spatial correspondence with the visual stimulus was significantly degraded. Similarly, the inhibitory effect of a spatially disparate auditory cue was significantly ameliorated. The observed effects were specific to AES deactivation, as similar effects were not obtained with deactivation of PLLS, AI or 17/18, or saline injections into the AES. These observations are consistent with postulates that specific cortical-midbrain interactions are essential for the synthesis of multisensory information in the SC, and for the orientation and localization behaviors that depend on this synthesis.     

Alterations to multisensory and unisensory integration by stimulus competition

In environments containing sensory events at competing locations, selecting a target for orienting requires prioritization of stimulus values. Although the superior colliculus (SC) is causally linked to the stimulus selection process, the manner in which SC multisensory integration operates in a competitive stimulus environment is unknown. Here we examined how the activity of visual-auditory SC neurons is affected by placement of a competing target in the opposite hemifield, a stimulus configuration that would, in principle, promote interhemispheric competition for access to downstream motor circuitry. Competitive interactions between the targets were evident in how they altered unisensory and multisensory responses of individual neurons. Responses elicited by a cross-modal stimulus (multisensory responses) proved to be substantially more resistant to competitor-induced depression than were unisensory responses (evoked by the component modality-specific stimuli). Similarly, when a cross-modal stimulus served as the competitor, it exerted considerably more depression than did its individual component stimuli, in some cases producing more depression than predicted by their linear sum. These findings suggest that multisensory integration can help resolve competition among multiple targets by enhancing orientation to the location of cross-modal events while simultaneously suppressing orientation to events at alternate locations.     

Superior colliculus lesions preferentially disrupt multisensory orientation

The general involvement of the superior colliculus (SC) in orientation behavior and the striking parallels between the multisensory responses of SC neurons and overt orientation behaviors have led to assumptions that these neural and behavioral changes are directly linked. However, deactivation of two areas of cortex which also contain multisensory neurons, the anterior ectosylvian sulcus and rostral lateral suprasylvian sulcus have been shown to eliminate multisensory orientation behaviors, suggesting that this behavior may not involve the SC. To determine whether the SC contributes to this behavior, cats were tested in a multisensory (i.e. visual-auditory) orientation task before and after excitotoxic lesions of the SC. For unilateral SC lesions, modality-specific (i.e. visual or auditory) orientation behaviors had returned to pre-lesion levels after several weeks of recovery. In contrast, the enhancements and depressions in behavior normally seen with multisensory stimuli were severely compromised in the contralesional hemifield. No recovery of these behaviors was observed within the 6 month testing period. Immunohistochemical labeling of the SC revealed a preferential loss of parvalbumin-immunoreactive pyramidal neurons in the intermediate layers, a presumptive multisensory population that targets premotor areas of the brainstem and spinal cord. These results highlight the importance of the SC for multisensory behaviors, and suggest that the multisensory orientation deficits produced by cortical lesions are a result of the loss of cortical influences on multisensory SC neurons.     

Neural mechanisms for synthesizing sensory information and producing adaptive behaviors

The ability to integrate information from different sensory systems is a fundamental characteristic of the brain. Because different bits of information are derived from different sensory channels, their synthesis markedly enhances the detection and identification of external stimuli. The neural substrate for such “multisensory” integration is provided by neurons that receive convergent input from two or more sensory modalities. Many such multisensory neurons are found in the superior colliculus (SC), a midbrain structure that plays a significant role in overt attentive and orientation behaviors. The various principles governing the integration of visual, auditory, and somatosensory inputs in SC neurons have been explored in several species. Thus far, the evidence suggests a remarkable conservation of integrative features during vertebrate evolution. One of the most robust of these principles is based on spatial relationships: a striking enhancement in activity is induced in a multisensory neuron when two different sensory stimuli (e.g., visual and auditory) are in spatial concordance, whereas a profound response depression can be induced when these cues are spatially discordant. The most extensive physiological observations have been made in cat, and in this species the same principles that have been shown to govern multisensory integration at the level of the individual SC neuron have also been shown to govern overt attentive and orientation responses to multisensory stimuli. Most surprising, however, is the critical role played by association (i.e. anterior ectosylvian) cortex in facilitating these midbrain processes. In the absence of the modulating corticotectal influences, multisensory SC neurons in cat are unable to integrate the different sensory cues converging upon them in an adult-like fashion, and are unable to mediate overt multisensory behaviors. This situation appears quite similar to that observed during early postnatal life. When multisensory SC neurons first appear, they are able to respond to multiple sensory inputs but are unable to synthesize these inputs to significantly enhance or degrade their responses. During ontogeny, individual multisensory neurons develop this capacity abruptly, but at very different ages, until the mature population condition is reached after several postnatal months. It appears likely that the abrupt onset of this capacity in any individual SC neuron reflects the maturation of inputs from anterior ectosylvian cortex. Presumably, the functional coupling of cortex with an individual SC neuron is essential to initiate and maintain that neuron’s capability for multisensory integration throughout its life.     

A computational study of multisensory maturation in the superior colliculus (SC)

Multisensory neurons in cat SC exhibit significant postnatal maturation. The first multisensory neurons to appear have large receptive fields (RFs) and cannot integrate information across sensory modalities. During the first several months of postnatal life RFs contract, responses become more robust and neurons develop the capacity for multisensory integration. Recent data suggest that these changes depend on both sensory experience and active inputs from association cortex. Here, we extend a computational model we developed (Cuppini et al. in Front Integr Neurosci 22: 4-6, 2010) using a limited set of biologically realistic assumptions to describe how this maturational process might take place. The model assumes that during early life, cortical-SC synapses are present but not active and that responses are driven by non-cortical inputs with very large RFs. Sensory experience is modeled by a "training phase" in which the network is repeatedly exposed to modality-specific and cross-modal stimuli at different locations. Cortical-SC synaptic weights are modified during this period as a result of Hebbian rules of potentiation and depression. The result is that RFs are reduced in size and neurons become capable of responding in adult-like fashion to modality-specific and cross-modal stimuli.     

Multisensory orientation behavior is disrupted by neonatal cortical ablation

The integration of visual and auditory information can significantly amplify the sensory responses of superior colliculus (SC) neurons and the behaviors that depend on them. This response amplification depends on the development of SC inputs that are derived from two regions of cortex: the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS). Neonatal ablation of these cortico-collicular areas has been shown to disrupt the development of the multisensory enhancement capabilities of SC neurons and the present results demonstrate that it also precludes the development of the normal multisensory enhancements in orientation behavior. Animals with neonatal ablation of AES and rLS were tested at maturity and found unable to benefit from the combination of visual and auditory cues in their efforts to localize targets in contralesional space. In contrast, their ipsilesional multisensory orientation capabilities were indistinguishable from those of normal animals. However, when only one of these cortical areas was removed during early life, later behavioral consequences were negligible. Whether similar compensatory processes would occur in adult animals remains to be determined. These observations, coupled with those from previous studies, also suggest that a surprisingly high proportion of SC neurons capable of multisensory integration must be present for orientation behavior benefits to be realized. Compensatory mechanisms can achieve this if early lesions spare either AES or rLS, but even the impressive plasticity of the neonatal brain cannot compensate for the early loss of both of them.     

Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences

Many neurons in the superior colliculus (SC) are able to integrate combinations of visual, auditory, and somatosensory stimuli, thereby markedly affecting the vigor of their responses to external stimuli. However, this capacity for multisensory integration is not inborn. Rather, it appears comparatively late in postnatal development and is not expressed until the SC passes through several distinct developmental stages. As shown here, the final stage in this sequence is one in which a region of association cortex establishes functional control over the SC, thus enabling the multisensory integrative capabilities of its target SC neurons. The first example of this corticotectal input was seen at postnatal day 28. For any individual SC neuron, the onset of corticotectal influences appeared to be abrupt. Because this event occurred at very different times for different SC neurons, a period of 3-4 postnatal months was required before the adult-like condition was achieved. The protracted postnatal period required for the maturation of these corticotectal influences corresponded closely with estimates of the peak period of cortical plasticity, raising the possibility that the genesis of these corticotectal influences, and hence the onset of SC multisensory integration, occurs only after the cortex is capable of exerting experience-dependent control over SC neurons.     

Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration

Convergence of inputs from different sensory modalities onto individual neurons is a phenomenon that occurs widely throughout the brain at many phyletic levels and appears to represent a basic neural mechanism by which an organism integrates complex environmental stimuli. In the present study, neurons in the superior colliculus (SC) were used as a model to examine how single neurons deal with simultaneous cues from different sensory modalities (e.g., visual, auditory, somatosensory). The functional result of multisensory convergence on an individual cell was determined by comparing the responses evoked from it by a combined-modality (multimodal) stimulus with those elicited by each (unimodal) component of that stimulus presented alone. Superior colliculus cells exhibited profound changes in their activity when individual sensory stimuli were combined. These "multisensory interactions" were found to be widespread among deep laminae cells and fell into one of two functional categories: response enhancement, characterized by a significant increase in the number of discharges evoked; and response depression, characterized by a significant decrease in the discharges elicited. Multisensory response interactions most often reflected a multiplicative, rather than summative, change in activity. Their absolute magnitude varied from cell to cell and, when stimulus conditions were altered, within the same cell. However, the percentage change of enhanced interactions was generally inversely related to the vigor of the responses that could be evoked by presenting each unimodal stimulus alone and suggest that the potential for response amplification was greatest when responses evoked by individual stimuli were weakest. The majority of cells exhibiting multi-sensory characteristics were demonstrated to have descending efferent projections and thus had access to premotor and motor areas of the brain stem and spinal cord involved in SC-mediated attentive and orientation behaviors. These data show that multisensory convergence provides the descending efferent cells of the SC with a dynamic response character. The responses of these cells and the SC-mediated behaviors that they underlie need not be immutably tied to the presence of any single stimulus, but can vary in response to the particular complex of stimuli present in the environment at any given moment.     

Visual,auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection

Summary A select population of superior colliculus (SC) neurons receives and integrates information from the visual, auditory and somatosensory systems. Determining which SC neurons comprise this population and where they send their multisensory messages is important in understanding the functional impact of the SC on attentive and orientation behavior. One of the major routes by which the SC influences these behaviors is the tecto-reticulo-spinal tract, a descending pathway that plays an integral role in the orientation of the eyes, ears and head. Of the 182 tecto-reticulo-spinal neurons (TRSNs) encountered in the present study, almost all (94%) responded to sensory stimuli and the overwhelming majority (84%) were multisensory. The present results demonstrate that the TRSN serves as an important link among the different sensory systems and provides a substrate through which they may gain access to the circuitry mediating orientation behavior.     

Multisensory integration for orienting responses in humans requires the activation of the superior colliculus

Animal studies have shown that the superior colliculus (SC) is important for synthesising information from multiple senses into a unified map of space. Here, we tested whether the SC is a critical neural substrate for multisensory spatial integration in humans. To do so, we took advantage of neurophysiological findings revealing that the SC does not receive direct projections from short-wavelength-sensitive S cones. In a simple reaction-time task, participants responded more quickly to concurrent peripheral (extra-foveal) audiovisual (AV) stimuli than to an auditory or visual stimulus alone, a phenomenon known as the redundant target effect (RTE). We show that the nature of this RTE was dependent on the colour of the visual stimulus. When using purple short-wavelength stimuli, to which the SC is blind, RTE was simply explained by probability summation, indicating that the redundant auditory and visual channels are independent. Conversely, with red long-wavelength stimuli, visible to the SC, the RTE was related to nonlinear neural summation, which constitutes evidence of integration of different sensory information. We also demonstrate that when AV stimuli were presented at fixation, so that the spatial orienting component of the task was reduced, neural summation was possible regardless of stimulus colour. Together, these findings provide support for a pivotal role of the SC in mediating multisensory spatial integration in humans, when behaviour involves spatial orienting responses.     

Neonatal cortical ablation disrupts multisensory development in superior colliculus

The ability of cat superior colliculus (SC) neurons to synthesize information from different senses depends on influences from two areas of the cortex: the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS). Reversibly deactivating the inputs to the SC from either of these areas in normal adults severely compromises this ability and the SC-mediated behaviors that depend on it. In this study, we found that removal of these areas in neonatal animals precluded the normal development of multisensory SC processes. At maturity there was a substantial decrease in the incidence of multisensory neurons, and those multisensory neurons that did develop were highly abnormal. Their cross-modal receptive field register was severely compromised, as was their ability to integrate cross-modal stimuli. Apparently, despite the impressive plasticity of the neonatal brain, it cannot compensate for the early loss of these cortices. Surprisingly, however, neonatal removal of either AES or rLS had comparatively minor consequences on these properties. At maturity multisensory SC neurons were quite common: they developed the characteristic spatial register among their unisensory receptive fields and exhibited normal adult-like multisensory integration. These observations suggest that during early ontogeny, when the multisensory properties of SC neurons are being crafted, AES and rLS may have the ability to compensate for the loss of one another's cortico-collicular influences so that normal multisensory processes can develop in the SC.     

Multisensory processing in "unimodal" neurons: cross-modal subthreshold auditory effects in cat extrastriate visual cortex

Historically, the study of multisensory processing has examined the function of the definitive neuron type, the bimodal neuron. These neurons are excited by inputs from more than one sensory modality, and when multisensory stimuli are present, they can integrate their responses in a predictable manner. However, recent studies have revealed that multisensory processing in the cortex is not restricted to bimodal neurons. The present investigation sought to examine the potential for multisensory processing in nonbimodal (unimodal) neurons in the retinotopically organized posterolateral lateral suprasylvian (PLLS) area of the cat. Standard extracellular recordings were used to measure responses of all neurons encountered to both separate- and combined-modality stimulation. Whereas bimodal neurons behaved as predicted, the surprising result was that 16% of unimodal visual neurons encountered were significantly facilitated by auditory stimuli. Because these unimodal visual neurons did not respond to an auditory stimulus presented alone but had their visual responses modulated by concurrent auditory stimulation, they represent a new form of multisensory neuron: the subthreshold multisensory neuron. These data also demonstrate that bimodal neurons can no longer be regarded as the exclusive basis for multisensory processing.     

Level dependence of contextual modulation in auditory cortex

Responses of cortical neurons to sensory stimuli within their receptive fields can be profoundly altered by the stimulus context. In visual and somatosensory cortex, contextual interactions have been shown to change sign from facilitation to suppression depending on stimulus strength. Contextual modulation of high-contrast stimuli tends to be suppressive, but for low-contrast stimuli tends to be facilitative. This trade-off may optimize contextual integration by cortical cells and has been suggested to be a general feature of cortical processing, but it remains unknown whether a similar phenomenon occurs in auditory cortex. Here we used whole cell and single-unit recordings to investigate how contextual interactions in auditory cortical neurons depend on the relative intensity of masker and probe stimuli in a two-tone stimulus paradigm. We tested the hypothesis that relatively low-level probes should show facilitation, whereas relatively high-level probes should show suppression. We found that contextual interactions were primarily suppressive across all probe levels, and that relatively low-level probes were subject to stronger suppression than high-level probes. These results were virtually identical for spiking and subthreshold responses. This suggests that, unlike visual cortical neurons, auditory cortical neurons show maximal suppression rather than facilitation for relatively weak stimuli.     

Competition and convergence between auditory and cross-modal visual inputs to primary auditory cortical areas

Sensory neocortex is capable of considerable plasticity after sensory deprivation or damage to input pathways, especially early in development. Although plasticity can often be restorative, sometimes novel, ectopic inputs invade the affected cortical area. Invading inputs from other sensory modalities may compromise the original function or even take over, imposing a new function and preventing recovery. Using ferrets whose retinal axons were rerouted into auditory thalamus at birth, we were able to examine the effect of varying the degree of ectopic, cross-modal input on reorganization of developing auditory cortex. In particular, we assayed whether the invading visual inputs and the existing auditory inputs competed for or shared postsynaptic targets and whether the convergence of input modalities would induce multisensory processing. We demonstrate that although the cross-modal inputs create new visual neurons in auditory cortex, some auditory processing remains. The degree of damage to auditory input to the medial geniculate nucleus was directly related to the proportion of visual neurons in auditory cortex, suggesting that the visual and residual auditory inputs compete for cortical territory. Visual neurons were not segregated from auditory neurons but shared target space even on individual target cells, substantially increasing the proportion of multisensory neurons. Thus spatial convergence of visual and auditory input modalities may be sufficient to expand multisensory representations. Together these findings argue that early, patterned visual activity does not drive segregation of visual and auditory afferents and suggest that auditory function might be compromised by converging visual inputs. These results indicate possible ways in which multisensory cortical areas may form during development and evolution. They also suggest that rehabilitative strategies designed to promote recovery of function after sensory deprivation or damage need to take into account that sensory cortex may become substantially more multisensory after alteration of its input during development.     

Reference frames for representing visual and tactile locations in parietal cortex

The ventral intraparietal area (VIP) receives converging inputs from visual, somatosensory, auditory and vestibular systems that use diverse reference frames to encode sensory information. A key issue is how VIP combines those inputs together. We mapped the visual and tactile receptive fields of multimodal VIP neurons in macaque monkeys trained to gaze at three different stationary targets. Tactile receptive fields were found to be encoded into a single somatotopic, or head-centered, reference frame, whereas visual receptive fields were widely distributed between eye- to head-centered coordinates. These findings are inconsistent with a remapping of all sensory modalities in a common frame of reference. Instead, they support an alternative model of multisensory integration based on multidirectional sensory predictions (such as predicting the location of a visual stimulus given where it is felt on the skin and vice versa). This approach can also explain related findings in other multimodal areas.     

Not Just for Bimodal Neurons Anymore: The Contribution of Unimodal Neurons to Cortical Multisensory Processing

Traditionally, neuronal studies of multisensory processing proceeded by first identifying neurons that were overtly multisensory (e.g., bimodal, trimodal) and then testing them. In contrast, the present study examined, without precondition, neurons in an extrastriate visual area of the cat for their responses to separate (visual, auditory) and combined-modality (visual and auditory) stimulation. As expected, traditional bimodal forms of multisensory neurons were identified. In addition, however, many neurons that were activated only by visual stimulation (i.e., unimodal) had that response modulated by the presence of an auditory stimulus. Some unimodal neurons showed multisensory responses that were statistically different from their visual response. Other unimodal neurons had subtle multisensory effects that were detectable only at the population level. Most surprisingly, these non-bimodal neurons generated more than twice the multisensory signal in the PLLS than did the bimodal neurons. These results expand the range of multisensory convergence patterns beyond that of the bimodal neuron. However, rather than characterize a separate class of multisensory neurons, unimodal multisensory neurons may actually represent an intermediary form of multisensory convergence that exists along the functional continuum between unisensory neurons, at one end, and fully bimodal neurons at the other.     

Excitotoxic lesions of the superior colliculus preferentially impact multisensory neurons and multisensory integration

The superior colliculus (SC) plays an important role in integrating visual, auditory and somatosensory information, and in guiding the orientation of the eyes, ears and head. Previously we have shown that cats with unilateral SC lesions showed a preferential loss of multisensory orientation behaviors for stimuli contralateral to the lesion. Surprisingly, this behavioral loss was seen even under circumstances where the SC lesion was far from complete. To assess the physiological changes induced by these lesions, we employed single unit electrophysiological methods to record from individual neurons in both the intact and damaged SC following behavioral testing in two animals. In the damaged SC of these animals, multisensory neurons were preferentially reduced in incidence, comprising less than 25% of the sensory-responsive population (as compared with 49% on the control side). In those multisensory neurons that remained following the lesion, receptive fields were nearly twofold larger, and less than 25% showed normal patterns of multisensory integration, with those that did being found in areas outside of the lesion. These results strongly suggest that the multisensory behavioral deficits seen following SC lesions are the combined result of a loss of multisensory neurons and a loss of multisensory integration in those neurons that remain.