A spiking neural network architecture for nonlinear function approximation.

Multilayer perceptrons have received much attention in recent years due to their universal approximation capabilities. Normally, such models use real valued continuous signals, although they are loosely based on biological neuronal networks that encode signals using spike trains. Spiking neural networks are of interest both from a biological point of view and in terms of a method of robust signaling in particularly noisy or difficult environments. It is important to consider networks based on spike trains. A basic question that needs to be considered however, is what type of architecture can be used to provide universal function approximation capabilities in spiking networks? In this paper, we propose a spiking neural network architecture using both integrate-and-fire units as well as delays, that is capable of approximating a real valued function mapping to within a specified degree of accuracy.     

Detecting connectivity changes in neuronal networks

We develop a method from semiparametric statistics (Cox, 1972) for the purpose of tracking links and connection strengths over time in a neuronal network from spike train data. We consider application of the method as implemented in Masud and Borisyuk (2011), and evaluate its use on data generated independently of the Cox model hypothesis, in particular from a spiking model of Izhikevich in four different dynamical regimes. Then, we show how the Cox method can be used to determine statistically significant changes in network connectivity over time. Our methodology is demonstrated using spike trains from multi-electrode array measurements of networks of cultured mammalian spinal cord cells.     

Statistical technique for analysing functional connectivity of multiple spike trains

A new statistical technique, the Cox method, used for analysing functional connectivity of simultaneously recorded multiple spike trains is presented. This method is based on the theory of modulated renewal processes and it estimates a vector of influence strengths from multiple spike trains (called reference trains) to the selected (target) spike train. Selecting another target spike train and repeating the calculation of the influence strengths from the reference spike trains enables researchers to find all functional connections among multiple spike trains. In order to study functional connectivity an "influence function" is identified. This function recognises the specificity of neuronal interactions and reflects the dynamics of postsynaptic potential. In comparison to existing techniques, the Cox method has the following advantages: it does not use bins (binless method); it is applicable to cases where the sample size is small; it is sufficiently sensitive such that it estimates weak influences; it supports the simultaneous analysis of multiple influences; it is able to identify a correct connectivity scheme in difficult cases of "common source" or "indirect" connectivity. The Cox method has been thoroughly tested using multiple sets of data generated by the neural network model of the leaky integrate and fire neurons with a prescribed architecture of connections. The results suggest that this method is highly successful for analysing functional connectivity of simultaneously recorded multiple spike trains.     

Neural timing nets.

Formulations of artificial neural networks are directly related to assumptions about neural coding in the brain. Traditional connectionist networks assume channel-based rate coding, while time-delay networks convert temporally-coded inputs into rate-coded outputs. Neural timing nets that operate on time structured input spike trains to produce meaningful time-structured outputs are proposed. Basic computational properties of simple feedforward and recurrent timing nets are outlined and applied to auditory computations. Feed-forward timing nets consist of arrays of coincidence detectors connected via tapped delay lines. These temporal sieves extract common spike patterns in their inputs that can subserve extraction of common fundamental frequencies (periodicity pitch) and common spectrum (timbre). Feedforward timing nets can also be used to separate time-shifted patterns, fusing patterns with similar internal temporal structure and spatially segregating different ones. Simple recurrent timing nets consisting of arrays of delay loops amplify and separate recurring time patterns. Single- and multichannel recurrent timing nets are presented that demonstrate the separation of concurrent, double vowels. Timing nets constitute a new and general neural network strategy for performing temporal computations on neural spike trains: extraction of common periodicities, detection of recurring temporal patterns, and formation and separation of invariant spike patterns that subserve auditory objects.     

A simple indicator of nonstationarity of firing rate in spike trains

The assessment of stationarity of firing rate in neural spike trains is important but is often performed only visually. Facing the growing amount of neural data generated by multi-electrode recording, there is a need for an automatic method to identify and disqualify spike trains with highly nonstationary firing rates. In this report, we propose a simple test of nonstationarity, associated with an indicator quantifying the degree of nonstationary in a spike train. This method is compared to the Mann-Kendall test of trend detection and the Runs test on simulated and real spike trains.     

The reconstruction of individual spike trains from extracellular multineuron recordings using a neural network emulation program

A method for the reconstruction of the individual spike trains from extracellular multineuron recordings is described. A neural network emulation program is trained to recognize a sample set of digitized spikes. The digitized spikes are fed into the neural network, and the network output is used to classify spikes in terms of the training set. The system runs on any PC and its speed makes is especially well suited for the analysis of large amounts of data.     

Small-world network topology of hippocampal neuronal network is lost, in an in vitro glutamate injury model of epilepsy

Neuronal network topologies and connectivity patterns were explored in control and glutamate-injured hippocampal neuronal networks, cultured on planar multielectrode arrays. Spontaneous activity was characterized by brief episodes of synchronous firing at many sites in the array (network bursts). During such assembly activity, maximum numbers of neurons are known to interact in the network. After brief glutamate exposure followed by recovery, neuronal networks became hypersynchronous and fired network bursts at higher frequency. Connectivity maps were constructed to understand how neurons communicate during a network burst. These maps were obtained by analysing the spike trains using cross-covariance analysis and graph theory methods. Analysis of degree distribution, which is a measure of direct connections between electrodes in a neuronal network, showed exponential and Gaussian distributions in control and glutamate-injured networks, respectively. Although both the networks showed random features, small-world properties in these networks were different. These results suggest that functional two-dimensional neuronal networks in vitro are not scale-free. After brief exposure to glutamate, normal hippocampal neuronal networks became hyperexcitable and fired a larger number of network bursts with altered network topology. The small-world network property was lost and this was accompanied by a change from an exponential to a Gaussian network.     

Measuring spike train synchrony

Estimating the degree of synchrony or reliability between two or more spike trains is a frequent task in both experimental and computational neuroscience. In recent years, many different methods have been proposed that typically compare the timing of spikes on a certain time scale to be optimized by the analyst. Here, we propose the ISI-distance, a simple complementary approach that extracts information from the interspike intervals by evaluating the ratio of the instantaneous firing rates. The method is parameter free, time scale independent and easy to visualize as illustrated by an application to real neuronal spike trains obtained in vitro from rat slices. In a comparison with existing approaches on spike trains extracted from a simulated Hindemarsh-Rose network, the ISI-distance performs as well as the best time-scale-optimized measure based on spike timing.     

Effects of collateral inhibition in a model of the immature rat cerebellar cortex: multineuron correlations

A model of the immature rat cerebellar cortex is used to simulate the effect of the inhibitory recurrent collateral axons of the Purkinje cells on the spike trains in the network. Inhibition induces an important overall change in the statistical characteristics of individual spike trains. It is also instrumental in producing a strong cooperativity between the different neurons. Moreover, a functional spatial anisotropy appears. A specific entropy index is used to analyze levels of information transfer between clustered and faraway neurons in the network. The formatting effect of recurrent collateral inhibition on spike trains and on network functional dynamics is studied by means of a model of the newborn rat cerebellar cortex. This immature structure has simpler morphological characteristics and fewer physiological parameters than the adult one. It is thus a good candidate for the comparison between experimental and theoretical data. The model network is made of 256 formal neurons (FN), arranged in a square lattice. Each neuron is coupled to its eight nearest neighbors by inhibitory links. All the parameters of the different elements of the model — in particular integration of inhibitory and excitatory inputs — are given anatomical and physiological values derived from biological data. Activities of single FNs and correlations between spatially distant ones are analyzed with classical statistical techniques as well as with a specific informational entropy method we introduce. Simulation results indicate that inhibition is instrumental in: (1) the transformation of the spike train characteristics. This includes a lengthening of the mean interspike interval as well as an overall change in the statistical distribution of intervals, with an emergence of long-lasting ones; (2) the functional structuration of the network. Inhibitory connections between nearest neighbors induce a strong cooperativity between FNs. Furthermore a clear spatial anisotropy occurs in the functioning of the network, with inhibitory effects extending beyond local connectivity in preferential directions. We propose an interpretation of this functional structuration in terms of the various routes followed by the inhibition, including relay effects. The parameters of the model (levels of activities, inhibition rules and connectivities) were varied in order to test the robustness of the above results. Finally, the results are compared with those obtained in an experimental situation.     

Properties of favored patterns in spontaneous spike trains and responses of favored patterns to electroacupuncture in evoked trains.

By using 'the modified detection method', our previous study has shown that all spontaneous spike trains recorded from several areas of brain and spinal cord have favored patterns (FPs). The present study further shows that: (1) all newly detected spike trains from substantia nigra zona compacta, nucleus periventricularis hypothalami and nucleus hypothalamicus posterior also have FPs, and some spike trains from neurons in the same nucleus have a common favored pattern (CF, i.e. they share the same FP), indicating that FP and CF in spike trains are common phenomena; (2) all serial correlation coefficients of FP repetitions (in serial order) in different spike trains detected are less than 0.3 (close to 0), revealing that the repetition of FPs is a renewal process; (3) in different periods of the spike trains evoked by electroacupuncture (EA), the number of different FPs and the number of repetitions of the same representative FP either increase or decrease along with the change of firing rate. The tendencies of these changes are very similar, but after EA the repetitions of different FPs in the same spike trains change differently, showing that different (hidden) responses exist at the same time. The above results suggest that the FPs in spike trains may represent various neural codes, and 'the modified detection method of FP' can pick up more information from spike trains than the firing rate analysis, hence it is a very useful tool for the study of neural coding.     

An improved method for the estimation of firing rate dynamics using an optimal digital filter

In most neural systems, neurons communicate by means of sequences of action potentials or 'spikes'. Information encoded by spike trains is often quantified in terms of the firing rate which emphasizes the frequency of occurrence of action potentials rather than their exact timing. Common methods for estimating firing rates include the rate histogram, the reciprocal interspike interval, and the spike density function. In this study, we demonstrate the limitations of these aforementioned techniques and propose a simple yet more robust alternative. By convolving the spike train with an optimally designed Kaiser window, we show that more robust estimates of firing rate are obtained for both low and high-frequency inputs. We illustrate our approach by considering spike trains generated by simulated as well as experimental data obtained from single-unit recordings of first-order sensory neurons in the vestibular system. Improvements were seen in the prevention of aliasing, phase and amplitude distortion, as well as in the noise reduction for sinusoidal and more complex input profiles. We review the generality of the approach, and show that it can be adapted to describe neurons with sensory or motor responses that are characterized by marked nonlinearities. We conclude that our method permits more robust estimates of neural dynamics than conventional techniques across all stimulus conditions.     

Wavelet-based processing of neuronal spike trains prior to discriminant analysis

Investigations of neural coding in many brain systems have focused on the role of spike rate and timing as two means of encoding information within a spike train. Recently, statistical pattern recognition methods, such as linear discriminant analysis (LDA), have emerged as a standard approach for examining neural codes. These methods work well when data sets are over-determined (i.e., there are more observations than predictor variables). But this is not always the case in many experimental data sets. One way to reduce the number of predictor variables is to preprocess data prior to classification. Here, a wavelet-based method is described for preprocessing spike trains. The method is based on the discriminant pursuit (DP) algorithm of Buckheit and Donoho [Proc. SPIE 2569 (1995) 540-51]. DP extracts a reduced set of features that are well localized in the time and frequency domains and that can be subsequently analyzed with statistical classifiers. DP is illustrated using neuronal spike trains recorded in the motor cortex of an awake, behaving rat [Laubach et al. Nature 405 (2000) 567-71]. In addition, simulated spike trains that differed only in the timing of spikes are used to show that DP outperforms another method for preprocessing spike trains, principal component analysis (PCA) [Richmond and Optican J. Neurophysiol. 57 (1987) 147-61].     

Favored patterns in spontaneous spike trains.

By using the modified detection method, favored patterns can be detected in a total of 44 spontaneous spike trains. Among these the 'periodical burst' discharge of one sympathetic preganglionic neuron and the 'fast-slow' alternative discharge of some hypothalamic neurons have visible characteristics, hence we use them to test the reliability of our method by comparing the detected patterns with the non-sequential interval histograms and oscillograms of the spike trains. The comparisons show that our method is reliable. The spike trains of nucleus raphe magnus (NRM) and the locus coeruleus (LC) have no visible characteristics; from these the following results have been observed: (1) all spike trains have one or more favored patterns; (2) some spike trains from neurons in the same nucleus have common fragments of favored patterns; (3) the favored patterns in spike trains recorded from different nuclei are different from each other; (4) some favored patterns in spike trains of the NRM neurons remain unchanged from beginning to end in 35-min records and their repetitions are relatively stable; and (5) microinjection of normal saline or normal serum into the LC has no significant influence on the occurrence of favored patterns in 35-min records of spike trains of the LC neurons. The above results indicate that the favored patterns in spike trains are objective and regular phenomena with relative stability. It seems likely that favored pattern may be used (as an index of the neuronal activity) in combination with the microinjection technique, etc., for various studies including studies on neural coding.     

Modeling and analyzing higher-order correlations in non-Poissonian spike trains

Measuring pairwise and higher-order spike correlations is crucial for studying their potential impact on neuronal information processing. In order to avoid misinterpretation of results, the tools used for data analysis need to be carefully calibrated with respect to their sensitivity and robustness. This, in turn, requires surrogate data with statistical properties common to experimental spike trains. Here, we present a novel method to generate correlated non-Poissonian spike trains and study the impact of single-neuron spike statistics on the inference of higher-order correlations. Our method to mimic cooperative neuronal spike activity allows the realization of a large variety of renewal processes with controlled higher-order correlation structure. Based on surrogate data obtained by this procedure we investigate the robustness of the recently proposed method empirical de-Poissonization (Ehm et al., 2007). It assumes Poissonian spiking, which is common also for many other estimation techniques. We observe that some degree of deviation from this assumption can generally be tolerated, that the results are more reliable for small analysis bins, and that the degree of misestimation depends on the detailed spike statistics. As a consequence of these findings we finally propose a strategy to assess the reliability of results for experimental data.     

Nonlinear modeling of neural population dynamics for hippocampal prostheses

Developing a neural prosthesis for the damaged hippocampus requires restoring the transformation of population neural activities performed by the hippocampal circuitry. To bypass a damaged region, output spike trains need to be predicted from the input spike trains and then reinstated through stimulation. We formulate a multiple-input, multiple-output (MIMO) nonlinear dynamic model for the input–output transformation of spike trains. In this approach, a MIMO model comprises a series of physiologically-plausible multiple-input, single-output (MISO) neuron models that consist of five components each: (1) feedforward Volterra kernels transforming the input spike trains into the synaptic potential, (2) a feedback kernel transforming the output spikes into the spike-triggered after-potential, (3) a noise term capturing the system uncertainty, (4) an adder generating the pre-threshold potential, and (5) a threshold function generating output spikes. It is shown that this model is equivalent to a generalized linear model with a probit link function. To reduce model complexity and avoid overfitting, statistical model selection and cross-validation methods are employed to choose the significant inputs and interactions between inputs. The model is applied successfully to the hippocampal CA3–CA1 population dynamics. Such a model can serve as a computational basis for the development of hippocampal prostheses.     

Estimating the correlation between bursty spike trains and local field potentials

To further understand rhythmic neuronal synchronization, an increasingly useful method is to determine the relationship between the spiking activity of individual neurons and the local field potentials (LFPs) of neural ensembles. Spike field coherence (SFC) is a widely used method for measuring the synchronization between spike trains and LFPs. However, due to the strong dependency of SFC on the burst index, it is not suitable for analyzing the relationship between bursty spike trains and LFPs, particularly in high frequency bands. To address this issue, we developed a method called weighted spike field correlation (WSFC), which uses the first spike in each burst multiple times to estimate the relationship. In the calculation, the number of times that the first spike is used is equal to the spike count per burst. The performance of this method was demonstrated using simulated bursty spike trains and LFPs, which comprised sinusoids with different frequencies, amplitudes, and phases. This method was also used to estimate the correlation between pyramidal cells in the hippocampus and gamma oscillations in rats performing behaviors. Analyses using simulated and real data demonstrated that the WSFC method is a promising measure for estimating the correlation between bursty spike trains and high frequency LFPs.     

Clustering ensembles of neural network models.

We show that large ensembles of (neural network) models, obtained e.g. in bootstrapping or sampling from (Bayesian) probability distributions, can be effectively summarized by a relatively small number of representative models. In some cases this summary may even yield better function estimates. We present a method to find representative models through clustering based on the models' outputs on a data set. We apply the method on an ensemble of neural network models obtained from bootstrapping on the Boston housing data, and use the results to discuss bootstrapping in terms of bias and variance. A parallel application is the prediction of newspaper sales, where we learn a series of parallel tasks. The results indicate that it is not necessary to store all samples in the ensembles: a small number of representative models generally matches, or even surpasses, the performance of the full ensemble. The clustered representation of the ensemble obtained thus is much better suitable for qualitative analysis, and will be shown to yield new insights into the data.     

Embedded ensemble encoding hypothesis: The role of the “Prepared” cell

We here reconsider current theories of neural ensembles in the context of recent discoveries about neuronal dendritic physiology. The key physiological observation is that the dendritic plateau potential produces sustained depolarization of the cell body (amplitude 10–20 mV, duration 200–500 ms). Our central hypothesis is that synaptically‐evoked dendritic plateau potentials lead to a prepared state of a neuron that favors spike generation. The plateau both depolarizes the cell toward spike threshold, and provides faster response to inputs through a shortened membrane time constant. As a result, the speed of synaptic‐to‐action potential (AP) transfer is faster during the plateau phase. Our hypothesis relates the changes from “resting” to “depolarized” neuronal state to changes in ensemble dynamics and in network information flow. The plateau provides the Prepared state (sustained depolarization of the cell body) with a time window of 200–500 ms. During this time, a neuron can tune into ongoing network activity and synchronize spiking with other neurons to provide a coordinated Active state (robust firing of somatic APs), which would permit “binding” of signals through coordination of neural activity across a population. The transient Active ensemble of neurons is embedded in the longer‐lasting Prepared ensemble of neurons. We hypothesize that “embedded ensemble encoding” may be an important organizing principle in networks of neurons.     

Single-trial estimation of neuronal firing rates: from single-neuron spike trains to population activity

We present a method to estimate the neuronal firing rate from single-trial spike trains. The method, based on convolution of the spike train with a fixed kernel function, is calibrated by means of simulated spike trains for a representative selection of realistic dynamic rate functions. We derive rules for the optimized use and performance of the kernel method, specifically with respect to an effective choice of the shape and width of the kernel functions. An application of our technique to the on-line, single-trial reconstruction of arm movement trajectories from multiple single-unit spike trains using dynamic population vectors illustrates a possible use of the proposed method.     

Superiority of nonlinear mapping in decoding multiple single-unit neuronal spike trains: a simulation study

One of the most important building blocks of the brain–machine interface (BMI) based on neuronal spike trains is the decoding algorithm, a computational method for the reconstruction of desired information from spike trains. Previous studies have reported that a simple linear filter is effective for this purpose and that no noteworthy gain is achieved from the use of nonlinear algorithms. In order to test this premise, we designed several decoding algorithms based on the linear filter, and two nonlinear mapping algorithms using multilayer perceptron (MLP) and support vector machine regression (SVR). Their performances were assessed using multiple neuronal spike trains generated by a biophysical neuron model and by a directional tuning model of the primary motor cortex. The performances of the nonlinear algorithms, in general, were superior. The advantages of using nonlinear algorithms were more profound for cases where false-positive/negative errors occurred in spike trains. When the MLPs were trained using trial-and-error, they often showed disappointing performance comparable to that of the linear filter. The nonlinear SVR showed the highest performance, and this may be due to the superiority of SVR in training and generalization.