Comparison of a spiking neural network and an MLP for robust identification of generator dynamics in a multimachine power system

The application of a spiking neural network (SNN) and a multi-layer perceptron (MLP) for online identification of generator dynamics in a multimachine power system are compared in this paper. An integrate-and-fire model of an SNN which communicates information via the inter-spike interval is applied. The neural network identifiers are used to predict the speed and terminal voltage deviations one time-step ahead of generators in a multimachine power system. The SNN is developed in two steps: (i) neuron centers determined by offline k-means clustering and (ii) output weights obtained by online training. The sensitivity of the SNN to the neuron centers determined in the first step is evaluated on generators of different ratings and parameters. Performances of the SNN and MLP are compared to evaluate robustness on the identification of generator dynamics under small and large disturbances, and to illustrate that SNNs are capable of learning nonlinear dynamics of complex systems.     

Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition

On-line learning and recognition of spatio- and spectro-temporal data (SSTD) is a very challenging task and an important one for the future development of autonomous machine learning systems with broad applications. Models based on spiking neural networks (SNN) have already proved their potential in capturing spatial and temporal data. One class of them, the evolving SNN (eSNN), uses a one-pass rank-order learning mechanism and a strategy to evolve a new spiking neuron and new connections to learn new patterns from incoming data. So far these networks have been mainly used for fast image and speech frame-based recognition. Alternative spike-time learning methods, such as Spike-Timing Dependent Plasticity (STDP) and its variant Spike Driven Synaptic Plasticity (SDSP), can also be used to learn spatio-temporal representations, but they usually require many iterations in an unsupervised or semi-supervised mode of learning. This paper introduces a new class of eSNN, dynamic eSNN, that utilise both rank-order learning and dynamic synapses to learn SSTD in a fast, on-line mode. The paper also introduces a new model called deSNN, that utilises rank-order learning and SDSP spike-time learning in unsupervised, supervised, or semi-supervised modes. The SDSP learning is used to evolve dynamically the network changing connection weights that capture spatio-temporal spike data clusters both during training and during recall. The new deSNN model is first illustrated on simple examples and then applied on two case study applications: (1) moving object recognition using address-event representation (AER) with data collected using a silicon retina device; (2) EEG SSTD recognition for brain–computer interfaces. The deSNN models resulted in a superior performance in terms of accuracy and speed when compared with other SNN models that use either rank-order or STDP learning. The reason is that the deSNN makes use of both the information contained in the order of the first input spikes (which information is explicitly present in input data streams and would be crucial to consider in some tasks) and of the information contained in the timing of the following spikes that is learned by the dynamic synapses as a whole spatio-temporal pattern.     

Towards biological plausibility of electronic noses: A spiking neural network based approach for tea odour classification

The paper presents a novel encoding scheme for neuronal code generation for odour recognition using an electronic nose (EN). This scheme is based on channel encoding using multiple Gaussian receptive fields superimposed over the temporal EN responses. The encoded data is further applied to a spiking neural network (SNN) for pattern classification. Two forms of SNN, a back-propagation based SpikeProp and a dynamic evolving SNN are used to learn the encoded responses. The effects of information encoding on the performance of SNNs have been investigated. Statistical tests have been performed to determine the contribution of the SNN and the encoding scheme to overall odour discrimination. The approach has been implemented in odour classification of orthodox black tea (Kangra-Himachal Pradesh Region) thereby demonstrating a biomimetic approach for EN data analysis.     

Perceiving nonverbal behavior: Neural correlates of processing movement fluency and contingency in dyadic interactions

Despite the fact that nonverbal dyadic social interactions are abundant in the environment, the neural mechanisms underlying their processing are not yet fully understood. Research in the field of social neuroscience has suggested that two neural networks appear to be involved in social understanding: (1) the action observation network (AON) and (2) the social neural network (SNN). The aim of this study was to determine the differential contributions of the AON and the SNN to the processing of nonverbal behavior as observed in dyadic social interactions. To this end, we used short computer animation sequences displaying dyadic social interactions between two virtual characters and systematically manipulated two key features of movement activity, which are known to influence the perception of meaning in nonverbal stimuli: (1) movement fluency and (2) contingency of movement patterns. A group of 21 male participants rated the “naturalness” of the observed scenes on a four‐point scale while undergoing fMRI. Behavioral results showed that both fluency and contingency significantly influenced the “naturalness” experience of the presented animations. Neurally, the AON was preferentially engaged when processing contingent movement patterns, but did not discriminate between different degrees of movement fluency. In contrast, regions of the SNN were engaged more strongly when observing dyads with disturbed movement fluency. In conclusion, while the AON is involved in the general processing of contingent social actions, irrespective of their kinematic properties, the SNN is preferentially recruited when atypical kinematic properties prompt inferences about the agents' intentions. Hum Brain Mapp 35:1362–1378, 2014. © 2013 Wiley Periodicals, Inc.     

NeuCube: A spiking neural network architecture for mapping,learning and understanding of spatio-temporal brain data

The brain functions as a spatio-temporal information processing machine. Spatio- and spectro-temporal brain data (STBD) are the most commonly collected data for measuring brain response to external stimuli. An enormous amount of such data has been already collected, including brain structural and functional data under different conditions, molecular and genetic data, in an attempt to make a progress in medicine, health, cognitive science, engineering, education, neuro-economics, Brain–Computer Interfaces (BCI), and games. Yet, there is no unifying computational framework to deal with all these types of data in order to better understand this data and the processes that generated it. Standard machine learning techniques only partially succeeded and they were not designed in the first instance to deal with such complex data. Therefore, there is a need for a new paradigm to deal with STBD. This paper reviews some methods of spiking neural networks (SNN) and argues that SNN are suitable for the creation of a unifying computational framework for learning and understanding of various STBD, such as EEG, fMRI, genetic, DTI, MEG, and NIRS, in their integration and interaction. One of the reasons is that SNN use the same computational principle that generates STBD, namely spiking information processing. This paper introduces a new SNN architecture, called NeuCube, for the creation of concrete models to map, learn and understand STBD. A NeuCube model is based on a 3D evolving SNN that is an approximate map of structural and functional areas of interest of the brain related to the modeling STBD. Gene information is included optionally in the form of gene regulatory networks (GRN) if this is relevant to the problem and the data. A NeuCube model learns from STBD and creates connections between clusters of neurons that manifest chains (trajectories) of neuronal activity. Once learning is applied, a NeuCube model can reproduce these trajectories, even if only part of the input STBD or the stimuli data is presented, thus acting as an associative memory. The NeuCube framework can be used not only to discover functional pathways from data, but also as a predictive system of brain activities, to predict and possibly, prevent certain events. Analysis of the internal structure of a model after training can reveal important spatio-temporal relationships ‘hidden’ in the data. NeuCube will allow the integration in one model of various brain data, information and knowledge, related to a single subject (personalized modeling) or to a population of subjects. The use of NeuCube for classification of STBD is illustrated in a case study problem of EEG data. NeuCube models result in a better accuracy of STBD classification than standard machine learning techniques. They are robust to noise (so typical in brain data) and facilitate a better interpretation of the results and understanding of the STBD and the brain conditions under which data was collected. Future directions for the use of SNN for STBD are discussed.     

Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications

The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include ‘on the fly’ new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.     

Spontaneous pain following spinal nerve injury in mice

Autotomy behavior is frequently observed in rats and mice in which the nerves of the hindlimb are severed, denervating the paw. This is the neuroma model of neuropathic pain. A large body of evidence suggests that this behavior reflects the presence of spontaneous dysesthesia and pain. In contrast, autotomy typically does not develop in partial nerve injury pain models, leading to the belief that these animals develop hypersensibility to applied stimuli (allodynia and hyperalgesia), but not spontaneous pain. We have modified the widely used Chung (spinal nerve ligation [SNL]) model of neuropathic pain in a way that retains the fundamental neural lesion, but eliminates nociceptive sensory cover of the paw. These animals performed autotomy. Moreover, the heritable across strains predisposition to spontaneous pain behavior in this new proximal denervation model (SNN) was highly correlated with pain phenotype in the neuroma model suggesting that the pain mechanism in the two models is the same. Relative reproducibility of strain predispositions across laboratories was verified. These data indicate that the neural substrate for spontaneous pain is present in the Chung-SNL model, and perhaps in the other partial nerve injury models as well, but that spontaneous pain is not expressed as autotomy in these models because there is protective nociceptive sensory cover.     

A new model of transient focal cerebral ischemia for inducing selective neuronal necrosis

Brief cerebral ischemia leads to selective neuronal necrosis (SNN), which is characterized by neuronal death with sparing of glial and vascular elements of the central nervous system. Understanding the pathophysiology of SNN may help elucidating the mechanisms and consequences of neuronal injury in humans following brief ischemia. Contrary to the presence of reproducible models of transient global ischemia, animal models of transient focal ischemia producing SNN are scarce and have important limitations such as causing ischemia in a vast area and inducing additional insults. In this study, we developed a practical mouse model of SNN without these limitations, by compressing the distal middle cerebral artery (MCA) with a blunted micropipette for 15 min. The success of compression was evaluated by monitoring the regional cerebral blood flow, and conventional histopathology and immunolabeling of the brain sections. Seven/fourteen days after ischemia, intracisternally administered propidium iodide labeled numerous necrotic cells in the frontoparietal cortex, which were mostly NeuN-positive, but were not immunolabeled with astrocytic markers (GFAP and S100), and showed neuronal morphology with hematoxylin–eosin staining, indicating that the model successfully induced ischemic injury limited to neurons. The model could become an important tool for investigating the long-term effects of brief ischemic events like transient ischemic attacks and could offer convenient reversible distal MCA occlusion for studies using intravital microscopy.     

Artificial neural networks based early clinical prediction of mortality after spontaneous intracerebral hemorrhage

Numerous outcome prediction models have been developed for mortality and functional outcome after spontaneous intracerebral haemorrhage (ICH). However, no outcome prediction model for ICH has considered the impact of care restriction. To develop and compare results of the artificial neural networks (ANN) and logistic regression (LR) models, based on initial clinical parameters, for prediction of mortality after spontaneous ICH. Analysis has been conducted on consecutive dataset of patients with spontaneous ICH, over 5-year period in tertiary care academic hospital. Patients older than 18?years were eligible for inclusion if they had been presented within 6?h from the start of symptoms and had evidence of spontaneous supratentorial ICH on initial brain computed tomography within 24?h. Initial clinical parameters have been used to develop LR and ANN prediction models for hospital mortality as outcome measure. Models have been accessed for discrimination and calibration abilities. We have analyzed 411 patients (199 males and 212 females) with spontaneous ICH, medically treated and not withdrawn from therapy, with average age of 67.35?years. From them, 256 (62.29?%) patients died during hospital treatment and 155 (37.71?%) patients survived. In the observed dataset, ANN model overall correctly classified outcome in 93.55?% of patients, compared with 79.32?% of correct classification for the LR model. Discrimination and calibration parameters indicate that both models show an adequate fit of expected and observed values, with superiority of ANN model. Our results favour the ANN model for prediction of mortality after spontaneous ICH. Further studies of the strengths and limitations of this method are needed with larger prospective samples.     

On the inter-relations between artificial and physiological neural networks

This paper discusses the inter-relations between findings on the physiological neural network (PNN) and artificial neural networks (ANN). It discusses the interaction of progress in both PNN and ANN for the purpose of borrowing from ANN's mathematical understandings to establish pointers for further explorations to better understand the PNN, and also for the reciprocal transferring of knowledge from PNN findings to improve ANN schemes. Such improvements in ANN are essential for better handling the needs of the information technology (IT) explosion in dealing with huge data bases and where data often defy analysis and are incomplete and fuzzy. On the other hand, principles and elements of ANN designs that appear to be important and successful can serve as guides for identifying them in the PNN, to be subsequently confirmed by bioanalytical tests. Hence progress in PNN is obviously essential for progress in ANN, as is progress in ANN helpful in PNN modeling, though its laboratory confirmation is still a far lengthier process. We discuss certain specific ANN schemes with respect to the above inter-relations with PNN. We feel that the progress in both PNN and ANN research provides a major link between the thrust in information technology developments and the thrust in biological science research, which are most probably the two major focus areas of research at the dawn of the 21st century.     

Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule

Understanding how the human brain is able to efficiently perceive and understand a visual scene is still a field of ongoing research. Although many studies have focused on the design and optimization of neural networks to solve visual recognition tasks, most of them either lack neurobiologically plausible learning rules or decision-making processes. Here we present a large-scale model of a hierarchical spiking neural network (SNN) that integrates a low-level memory encoding mechanism with a higher-level decision process to perform a visual classification task in real-time. The model consists of Izhikevich neurons and conductance-based synapses for realistic approximation of neuronal dynamics, a spike-timing-dependent plasticity (STDP) synaptic learning rule with additional synaptic dynamics for memory encoding, and an accumulator model for memory retrieval and categorization. The full network, which comprised 71,026 neurons and approximately 133 million synapses, ran in real-time on a single off-the-shelf graphics processing unit (GPU). The network was constructed on a publicly available SNN simulator that supports general-purpose neuromorphic computer chips. The network achieved 92% correct classifications on MNIST in 100 rounds of random sub-sampling, which is comparable to other SNN approaches and provides a conservative and reliable performance metric. Additionally, the model correctly predicted reaction times from psychophysical experiments. Because of the scalability of the approach and its neurobiological fidelity, the current model can be extended to an efficient neuromorphic implementation that supports more generalized object recognition and decision-making architectures found in the brain.     

Generating rules with predicates,terms and variables from the pruned neural networks

Artificial neural networks (ANN) have demonstrated good predictive performance in a wide range of applications. They are, however, not considered sufficient for knowledge representation because of their inability to represent the reasoning process succinctly. This paper proposes a novel methodology Gyan that represents the knowledge of a trained network in the form of restricted first-order predicate rules. The empirical results demonstrate that an equivalent symbolic interpretation in the form of rules with predicates, terms and variables can be derived describing the overall behaviour of the trained ANN with improved comprehensibility while maintaining the accuracy and fidelity of the propositional rules.     

Testing the validity of a set of diagnostic criteria for sensory neuronopathies: a francophone collaborative study

There are no validated criteria for the diagnosis of sensory neuronopathy (SNN) yet. In a preliminary monocenter study a set of criteria relying on clinical and electrophysiological data showed good sensitivity and specificity for a diagnosis of probable SNN. The aim of this study was to test these criteria on a French multicenter study. 210 patients with sensory neuropathies from 15 francophone reference centers for neuromuscular diseases were included in the study with an expert diagnosis of non-SNN, SNN or suspected SNN according to the investigations performed in these centers. Diagnosis was obtained independently from the set of criteria to be tested. The expert diagnosis was taken as the reference against which the proposed SNN criteria were tested. The set relied on clinical and electrophysiological data easily obtainable with routine investigations. 9/61 (16.4 %) of non-SNN patients, 23/36 (63.9 %) of suspected SNN, and 102/113 (90.3 %) of SNN patients according to the expert diagnosis were classified as SNN by the criteria. The SNN criteria tested against the expert diagnosis in the SNN and non-SNN groups had 90.3 % (102/113) sensitivity, 85.2 % (52/61) specificity, 91.9 % (102/111) positive predictive value, and 82.5 % (52/63) negative predictive value. Discordance between the expert diagnosis and the SNN criteria occurred in 20 cases. After analysis of these cases, 11 could be reallocated to a correct diagnosis in accordance with the SNN criteria. The proposed criteria may be useful for the diagnosis of probable SNN in patients with sensory neuropathy. They can be reached with simple clinical and paraclinical investigations.     

Nonlinear analog predictor analysis: a coupled neural network/analog model for climate downscaling.

Synoptic downscaling models are used in climatology to predict values of weather elements at one or more stations based on values of synoptic-scale atmospheric circulation variables. This paper presents a hybrid method for climate prediction and downscaling that couples an analog, i.e., k-nearest neighbor, model to an artificial neural network (ANN) model. In the proposed method, which is based on nonlinear principal predictor analysis (NLPPA), the analog model is embedded inside an ANN, forming its output layer. Nonlinear analog predictor analysis (NLAPA) is a flexible model that maintains the ability of the analog model to preserve inter-variable relationships and model non-normal and conditional variables (such as precipitation), while taking advantage of NLPPA's ability to define an optimal set of analog predictors that maximize predictive performance. Performance on both synthetic and real-world hydroclimatological benchmark tasks indicates that the NLAPA model is capable of outperforming other forms of analog models commonly used in synoptic downscaling.     

On the inter-relations between artificial and physiological neural networks

Abstract

This paper discusses the inter-relations between findings on the physiological neural network (PNN) and artificial neural networks (ANN). It discusses the interaction of progress in both PNN and ANN for the purpose of borrowing from ANN's mathematical understandings to establish pointers for further explorations to better understand the PNN, and also for the reciprocal transferring of knowledge from PNN findings to improve ANN schemes. Such improvements in ANN are essential for better handling the needs of the information technology (IT) explosion in dealing with huge data bases and where data often defy analysis and are incomplete and fuzzy. On the other hand, principles and elements of ANN designs that appear to be important and successful can serve as guides for identifying them in the PNN, to be subsequently confirmed by bioanalytical tests. Hence progress in PNN is obviously essential for progress in ANN, as is progress in ANN helpful in PNN modeling, though its laboratory confirmation is still a far lengthier process. We discuss certain specific ANN schemes with respect to the above inter-relations with PNN. We feel that the progress in both PNN and ANN research provides a major link between the thrust in information technology developments and the thrust in biological science research, which are most probably the two major focus areas of research at the dawn of the 21st century. [Neurol Res 2001; 23: 482-488]     

Clinical prediction of antidepressant response in mood disorders: linear multivariate vs. neural network models

Predicting the outcome of antidepressant treatment by pre-treatment features would be of great usefulness for clinicians as up to 50% of major depressives may not have a satisfactory response in spite of adequate trials of antidepressant drugs. In the present article we compared a linear multivariate model of predictors with a few artificial neural network (ANN) models differing from one another by outcome definition and validation procedure. The sample consisted of a reanalysis of 116 inpatients with a major depressive episode included in a 6-week open-label trial with fluvoxamine. With the original outcome definition (responders/non-responders), ANN performed better than logistic regression (90% of correct classifications in the training sample vs. 77%). However only 62% of new patients were correctly predicted by ANN for their outcome class. Length of the index episode, psychotic features and suicidal behavior emerged as outcome predictors in both models, while demographic characteristics, personality disorders and concomitant somatic morbidity were pointed to only by ANN analysis. Increase of classes in the outcome field resulted in a more elevated error: 46.4% for three classes, 60.4% for four classes and 70.3% for five classes. Overall, our findings suggest that antidepressant outcome prediction based on clinical variables is poor. The ANN approach is as valid as traditional multivariate techniques for the analysis of psychopharmacology studies. The complex interactions modelled through ANN may eventually be applied at the clinical level for individualized therapy. However, the accuracy of prediction is still far from satisfactory from a clinical point of view.     

Artificial neural networks improve the prediction of mortality in intracerebral hemorrhage.

BACKGROUND: Artificial neural network (ANN) analysis methods have led to more sensitive diagnosis of myocardial infarction and improved prediction of mortality in breast cancer, prostate cancer, and trauma patients. Prognostic studies have identified early clinical and radiographic predictors of mortality after intracerebral hemorrhage (ICH). To date, published models have not achieved the accuracy necessary for use in making decisions to limit medical interventions. We recently reported a logistic regression model that correctly classified 79% of patients who died and 90% of patients who survived. In an attempt to improve prediction of mortality we computed an ANN model with the same data. OBJECTIVE: To determine whether an ANN analysis would provide a more accurate prediction of mortality after ICH when compared with multiple logistic regression models computed using the same data. METHODS: Analyses were conducted on data collected prospectively on 81 patients with supratentorial ICH. Multiple logistic regression was used to predict hospital mortality, then an ANN analysis was applied to the same data set. Input variables were age, gender, race, hydrocephalus, mean arterial pressure, pulse pressure, Glasgow Coma Scale score, intraventricular hemorrhage, hydrocephalus, hematoma size, hematoma location (ganglionic, thalamic, or lobar), cisternal effacement, pineal shift, history of hypertension, history of diabetes, and age. RESULTS: The ANN model correctly classified all patients (100%) as alive or dead compared with 85% correct classification for the logistic regression model. A second ANN verification model was equally accurate. The ANN was superior to the logistic regression model on all objective measures of fit. CONCLUSIONS: ANN analysis more effectively uses information for prediction of mortality in this sample of patients with ICH. A well-validated ANN may have a role in the clinical management of ICH.     

Generalized neural networks for spectral analysis: dynamics and Liapunov functions.

This paper analyzes local and global behavior of several dynamical systems which generalize some artificial neural network (ANN) semilinear models originally designed for principal component analysis (PCA) in the characterization of random vectors. These systems implicitly performed the spectral analysis of correlation (i.e. symmetric positive definite) matrices. Here, the proposed generalizations cover both nonsymmetric matrices as well as fully nonlinear models. Local stability analysis is performed via linearization and global behavior is analyzed by constructing several Liapunov functions.