A learning rule for the emergence of stable dynamics and timing in recurrent networks

Neural dynamics within recurrent cortical networks is an important component of neural processing. However, the learning rules that allow networks composed of hundreds or thousands of recurrently connected neurons to develop stable dynamical states are poorly understood. Here I use a neural network model to examine the emergence of stable dynamical states within recurrent networks. I describe a learning rule that can account both for the development of stable dynamics and guide networks to states that have been observed experimentally, specifically, states that instantiate a sparse code for time. Across trials, each neuron fires during a specific time window; by connecting the neurons to a hypothetical set of output units, it is possible to generate arbitrary spatial-temporal output patterns. Intertrial jitter of the spike time of a given neuron increases as a direct function of the delay at which it fires. These results establish a learning rule by which cortical networks can potentially process temporal information in a self-organizing manner, in the absence of specialized timing mechanisms.     

Temporal data mining for the quality assessment of hemodialysis services

OBJECTIVE: This paper describes the temporal data mining aspects of a research project that deals with the definition of methods and tools for the assessment of the clinical performance of hemodialysis (HD) services, on the basis of the time series automatically collected during hemodialysis sessions. METHODS: Intelligent data analysis and temporal data mining techniques are applied to gain insight and to discover knowledge on the causes of unsatisfactory clinical results. In particular, two new methods for association rule discovery and temporal rule discovery are applied to the time series. Such methods exploit several pre-processing techniques, comprising data reduction, multi-scale filtering and temporal abstractions. RESULTS: We have analyzed the data of more than 5800 dialysis sessions coming from 43 different patients monitored for 19 months. The qualitative rules associating the outcome parameters and the measured variables were examined by the domain experts, which were able to distinguish between rules confirming available background knowledge and unexpected but plausible rules. CONCLUSION: The new methods proposed in the paper are suitable tools for knowledge discovery in clinical time series. Their use in the context of an auditing system for dialysis management helped clinicians to improve their understanding of the patients' behavior.     

Precedence Temporal Networks to represent temporal relationships in gene expression data

The reconstruction of gene regulatory networks from gene expression time series is nowadays an interesting research challenge. A key problem in this kind of analysis is the automated extraction of precedence and synchronization between interesting patterns assumed by genes over time.The present work introduces Precedence Temporal Networks (PTN), a novel method to extract and visualize temporal relationships between genes. PTNs are a special kind of temporal network where nodes represent temporal patterns while edges identify precedence or synchronization relationships between the nodes.The method is tested on two case studies: the expression of a subset of genes in the soil amoeba Dictyostelium discoideum and of a set of well-studied genes involved in the human cell cycle regulation. The extracted networks reflect the capability of the algorithm to clearly reconstruct the timing of the considered gene sets, highlighting different stages in Dictyostelium development and in the cell cycle, respectively.     

Neural networks in the analysis of episodic growth hormone release

Pulsatile secretion of growth hormone (GH) has been observedin healthy controls as well as acromegalic patients. In healthyadults, highly regulated secretory pulses of GH occur 4-8 timeswithin 24 h. This episodic pattern of secretion seems to berelated to the optimal induction of physiological effects atthe peripheral level. In contrast to normal subjects, acromegalicpatients demonstrate an irregular pattern of excessive GH release.This pattern of secretion is responsible for many systemic effects,such as the stimulation of connective tissue growth, cardiovascularand cerebrovascular disease, diabetes mellitus and arthritis.Standard methods for the analysis of pulsatile patterns of hormonesecretion did not consistently separate the temporal dynamicsof GH release in healthy controls and acromegalic patients undervarious study conditions. Using the cutting edge technologyof artificial neural networks for time series prediction, wewere able to achieve significant separation of both groups undervarious conditions by means of the predictability of their GHsecretory dynamics. Improving the predictive results by usinga more refined system of multiple neural networks acting inparallel (adaptive mixtures of local experts), we found thatthis system performed a self-organized segmentation of hormonepulsatility. It separated phases of secretory bursts and quiescencewithout any prior knowledge of the form of a GH pulse or a modelof secretion. Comparing the predictive results for the GH dynamicswith those for computer-simulated stochastic processes, we wereable to define the irregular pattern of GH secretion in acromegalyas a random autonomous process. The introduction of neural networksto the analysis of dynamic endocrine systems might help to expandthe existing analytical approaches beyond counting frequencyand amplitude of hormone pulses.     

Use of neural networks as medical diagnosis expert systems

A major bottleneck in building expert systems is the process of acquiring the required knowledge in the form of production rules. A novel class of neural networks is proposed to articulate the knowledge it learned from a set of examples. It provides an appealing solution to the problem of knowledge acquisition. After training, the knowledge embedded in the numerical weights of trained neural networks can be easily extracted and represented in the form of production rules. The approach is demonstrated by an example of a hypothesis regarding the pathophysiology of diabetes.     

Classification of auditory brainstem responses through symbolic pattern discovery

IntroductionNumeric time series are present in a very wide range of domains, including many branches of medicine. Data mining techniques have proved to be useful for knowledge discovery in this type of data and for supporting decision-making processes.ObjectivesThe overall objective is to classify time series based on the discovery of frequent patterns. These patterns will be discovered in symbolic sequences obtained from the time series data by means of a temporal abstraction process.MethodsFirstly, we transform numeric time series into symbolic time sequences, where the symbols aim to represent the relevant domain concepts. These symbols can be defined using either public or expert domain knowledge. Then we apply a symbolic pattern discovery technique to the output symbolic sequences. This technique identifies the subsequences frequently found in a population group. These subsequences (patterns) are representative of population groups. Finally, we employ a classification technique based on the identified patterns in order to classify new individuals. Thanks to the inclusion of domain knowledge, the classification results can be explained using domain terminology. This makes the results easier to interpret for the domain specialist (physician).ResultsThis method has been applied to brainstem auditory evoked potentials (BAEPs) time series. Preliminary experiments were carried out to analyse several aspects of the method including the best configuration of the pattern discovery technique parameters. We then applied the method to the BAEPs of 83 individuals belonging to four classes (healthy, conductive hearing loss, vestibular schwannoma—brainstem involvement and vestibular schwannoma—8th-nerve involvement). According to the results of the cross-validation, overall accuracy was 99.4%, sensitivity (recall) was 97.6% and specificity was 100% (no false positives).ConclusionThe proposed method effectively reduces dimensionality. Additionally, if the symbolic transformation includes the right domain knowledge, the method arguably outputs a data representation that denotes the relevant domain concepts more clearly. The method is capable of finding patterns in BAEPs time series and is very accurate at correctly predicting whether or not new patients have an auditory-related disorder.     

Evolving connectionist systems for knowledge discovery from gene expression data of cancer tissue

Microarray techniques have made it possible to observe the expression of thousands of genes simultaneously. They have recently been applied to study gene expression patterns in tissue samples. This may lead to highly desirable improvements in the diagnosis and treatment of human diseases. Statistical and machine learning methods have recently been used to classify cancer tissue based on gene expression data. Although some of these methods have achieved a high degree of accuracy, they generally lack transparency in their classification process. This, however, is crucial for the application in the medical field. In order to overcome this obstacle, we used knowledge-based neurocomputing (KBN), since KBN seeks to gain knowledge that is comprehensible to humans. In particular, we applied evolving fuzzy neural networks (EFuNNs) to classify cancer tissue, which is illustrated on the case studies of leukaemia and colon cancer. EFuNNs belong to the evolving connectionist system paradigm (ECOS) that has been recently introduced. They are well suited for adaptive learning and knowledge discovery. Fuzzy logic rules can be extracted from the trained networks and offer knowledge about the classification process in an easily accessible form. These rules point to genes that are strongly associated with specific types of cancer and may be used for the development of new tests and treatment discoveries.     

Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects.

The cognitive and neural mechanisms underlying category-specific knowledge remain controversial. Here we report that, across multiple tasks (viewing, delayed match to sample, naming), pictures of animals and tools were associated with highly consistent, category-related patterns of activation in ventral (fusiform gyrus) and lateral (superior and middle temporal gyri) regions of the posterior temporal lobes. In addition, similar patterns of category-related activity occurred when subjects read the names of, and answered questions about, animals and tools. These findings suggest that semantic object information is represented in distributed networks that include sites for storing information about specific object attributes such as form (ventral temporal cortex) and motion (lateral temporal cortex).     

A novel time series analysis approach for prediction of dialysis in critically ill patients using echo-state networks

Background

Echo-state networks (ESN) are part of a group of reservoir computing methods and are basically a form of recurrent artificial neural networks (ANN). These methods can perform classification tasks on time series data. The recurrent ANN of an echo-state network has an 'echo-state' characteristic. This 'echo-state' functions as a fading memory: samples that have been introduced into the network in a further past, are faded away. The echo-state approach for the training of recurrent neural networks was first described by Jaeger H. et al. In clinical medicine, until this moment, no original research articles have been published to examine the use of echo-state networks.

Methods

This study examines the possibility of using an echo-state network for prediction of dialysis in the ICU. Therefore, diuresis values and creatinine levels of the first three days after ICU admission were collected from 830 patients admitted to the intensive care unit (ICU) between May 31th 2003 and November 17th 2007. The outcome parameter was the performance by the echo-state network in predicting the need for dialysis between day 5 and day 10 of ICU admission. Patients with an ICU length of stay <10 days or patients that received dialysis in the first five days of ICU admission were excluded. Performance by the echo-state network was then compared by means of the area under the receiver operating characteristic curve (AUC) with results obtained by two other time series analysis methods by means of a support vector machine (SVM) and a naive Bayes algorithm (NB).

Results

The AUC's in the three developed echo-state networks were 0.822, 0.818, and 0.817. These results were comparable to the results obtained by the SVM and the NB algorithm.

Conclusions

This proof of concept study is the first to evaluate the performance of echo-state networks in an ICU environment. This echo-state network predicted the need for dialysis in ICU patients. The AUC's of the echo-state networks were good and comparable to the performance of other classification algorithms. Moreover, the echo-state network was more easily configured than other time series modeling technologies.     

Knowledge construction from time series data using a collaborative exploration system

This paper deals with the exploration of biomedical multivariate time series to construct typical parameter evolution or scenarios. This task is known to be difficult: the temporal and multivariate nature of the data at hand and the context-sensitive aspect of data interpretation hamper the formulation of a priori knowledge about the kind of patterns that can be detected as well as their interrelations. This paper proposes a new way to tackle this problem based on a human–computer collaborative approach involving specific annotations. Three grounding principles, namely autonomy, adaptability and emergence, support the co-construction of successive abstraction levels for data interpretation. An agent-based design is proposed to support these principles. Preliminary results in a clinical context are presented to support our proposal. A comparison with two well-known time series exploration tools is furthermore performed.     

A heterogeneous population code for elapsed time in rat medial agranular cortex

The neural mechanisms underlying the temporal control of behavior are largely unknown. Here we recorded from medial agranular cortex neurons in rats while they freely behaved in a temporal production task, the peak-interval procedure. Due to variability in estimating the time of food availability, robust responding typically bracketed the expected duration, starting some time before and ending some time after the signaled delay. These response periods provided analytic "steady state" windows during which subjects actively indicated their temporal expectation of food availability. Remarkably, during these response periods, a variety of firing patterns were seen that could be broadly described as ramps, peaks, and dips, with different slopes, directions, and times at which maxima or minima occur. Regularized linear discriminant analysis indicated that these patterns provided sufficiently reliable information to discriminate the elapsed duration of responding within these response periods. Modeling this across neuron variability showed that the utilization of ramps, dips, and peaks, with different slopes and minimal/maximal rates at different times, led to a substantial improvement in temporal prediction errors, suggesting that heterogeneity in the neural representation of elapsed time may facilitate temporally controlled behavior.     

Urinary nucleosides as potential tumor markers evaluated by learning vector quantization

Modified nucleosides were recently presented as potential tumor markers for breast cancer. The patterns of the levels of urinary nucleosides are different for tumor bearing individuals and for healthy individuals. Thus, a powerful pattern recognition method is needed. Although backpropagation (BP) neural networks are becoming increasingly common in medical literature for pattern recognition, it has been shown that often-superior methods exist like learning vector quantization (LVQ) and support vector machines (SVM). The aim of this feasibility study is to get an indication of the performance of urinary nucleoside levels evaluated by LVQ in contrast to the evaluation the popular BP and SVM networks. Urine samples were collected from female breast cancer patients and from healthy females. Twelve different ribonucleosides were isolated and quantified by a high performance liquid chromatography (HPLC) procedure. LVQ, SVM and BP networks were trained and the performance was evaluated by the classification of the test sets into the categories "cancer" and "healthy". All methods showed a good classification with a sensitivity ranging from 58.8 to 70.6% at a specificity of 88.4-94.2% for the test patterns. Although the classification performance of all methods is comparable, the LVQ implementations are superior in terms of more qualitative features: the results of LVQ networks are more reproducible, as the initialization is deterministic. The LVQ networks can be trained by unbalanced sizes of the different classes. LVQ networks are fast during training, need only few parameters adjusted for training and can be retrained by patterns of "local individuals". As at least some of these features play an important role in an implementation into a medical decision support system, it is recommended to use LVQ for an extended study.     

A topographical method for the development of neural networks for artificial brain evolution

Developmental neural networks, which are constructed according to developmental rules (i.e., genes), have the potential to be differentiated into heteromorphic neural structures capable of performing various kinds of activities. The fact that the biological neural architectures are found to be highly repetitive, layered, and topographically organized has important consequences for neural development methods. The purpose of this article is to propose a neural development method that can construct topographical neural connections, that is, a topographical development method, to facilitate fast and efficient development. This is achieved by arborizing neural connections on a developmental tree that rarely produces dead connections. Modular gene expression and corresponding modular networks have an important role in a gradual evolutionary process. Gene expression for modular networks is also proposed here as a way to reduce the probability of fatal mutants created through gene alteration. The corresponding evolutionary experiment shows that various neural structures--layered, repetitive, modular, and complex ones like those in the biological brain--can be constructed and easily observed. It also demonstrates that due to the efficiency of the proposed method, large neural networks can be easily managed, thereby making it suitable for long duration evolutionary experiments.     

Temporal abstraction and inductive logic programming for arrhythmia recognition from electrocardiograms

This paper proposes a novel approach to cardiac arrhythmia recognition from electrocardiograms (ECGs). ECGs record the electrical activity of the heart and are used to diagnose many heart disorders. The numerical ECG is first temporally abstracted into series of time-stamped events. Temporal abstraction makes use of artificial neural networks to extract interesting waves and their features from the input signals. A temporal reasoner called a chronicle recogniser processes such series in order to discover temporal patterns called chronicles which can be related to cardiac arrhythmias. Generally, it is difficult to elicit an accurate set of chronicles from a doctor. Thus, we propose to learn automatically from symbolic ECG examples the chronicles discriminating the arrhythmias belonging to some specific subset. Since temporal relationships are of major importance, inductive logic programming (ILP) is the tool of choice as it enables first-order relational learning. The approach has been evaluated on real ECGs taken from the MIT-BIH database. The performance of the different modules as well as the efficiency of the whole system is presented. The results are rather good and demonstrate that integrating numerical techniques for low level perception and symbolic techniques for high level classification is very valuable.     

癫痫脑电特征波的综合检测分类方法研究

本文将小波变换、人工神经网络、专家规则判据等多种检测方法有机地结合起来 ,用于癫痫脑电特征波的检测与分类 ,以充分发挥不同方法的优势。这种综合检测分类方法是先将预处理的多导脑电时间序列经小波变换将脑电中癫痫特征波在不同尺度下分离出来 ,再对选出的癫痫嫌疑波进行特征参数提取 ,然后把特征参数送入已经训练好的人工神经网络进行分类识别 ,最后再由专家规则判断筛选并作出检测分类统计报告。研究表明 ,该方法具有很好的信号特征提取和屏蔽随机噪声能力 ,获得了较好的检出率 ;尤其适合于非平稳、非线性生物医学信号的检测分类 ,值得进一步深入研究     

A model for single and multiple knowledge based networks

The inherent black-box nature of neural networks is an important drawback with respect to the problem of explanation of neural network responses. Although several articles have tackled the problem of rule extraction from a single neural network, just a few papers have investigated rule extraction from several combined neural networks. In this article we describe how to translate symbolic rules into the Discretized Interpretable Multi-Layer Perceptron (DIMLP) and how to extract rules from one or several combined neural networks. Our approach consists of characterizing discriminant hyperplane frontiers. Unordered rules are extracted in polynomial time with respect to the size of the problem and the size of the network. Moreover, the degree of matching between extracted rules and neural network responses is 100% on training examples. We applied single DIMLP networks to 17 data sets related to medical diagnosis and medical prognosis problems. Results based on 10-fold cross-validation showed that the DIMLP model was on average as accurate as standard multi-layer perceptrons (MLP). Furthermore, DIMLP networks were significantly more accurate than CN2 on eight problems, whereas only on one problem CN2 was better than DIMLP. Finally, a non-Hodgkin lymphoma diagnosis problem based on classification of electrophoresis gels was defined. It turned out that ensembles of DIMLP networks were significantly more accurate than CN2 (96.1% +/- 1.4 versus 82.7% +/- 4.0). Finally, symbolic rules revealed the presence of five important spots for the discrimination of the class of Lymphocyte Leukemia/Chronic Lymphoid Leukemia (Lc/LLc), and the class of Centrocytic Lymphoma (Cc).     

Short-term sound temporal envelope characteristics determine multisecond time patterns of activity in human auditory cortex as shown by fMRI

Functional magnetic resonance imaging (fMRI) of human auditory cortex has demonstrated a striking range of temporal waveshapes in responses to sound. Prolonged (30 s) low-rate (2/s) noise burst trains elicit "sustained" responses, whereas high-rate (35/s) trains elicit "phasic" responses with peaks just after train onset and offset. As a step toward understanding the significance of these responses for auditory processing, the present fMRI study sought to resolve exactly which features of sound determine cortical response waveshape. The results indicate that sound temporal envelope characteristics, but not sound level or bandwidth, strongly influence response waveshapes, and thus the underlying time patterns of neural activity. The results show that sensitivity to sound temporal envelope holds in both primary and nonprimary cortical areas, but nonprimary areas show more pronounced phasic responses for some types of stimuli (higher-rate trains, continuous noise), indicating more prominent neural activity at sound onset and offset. It has been hypothesized that the neural activity underlying the onset and offset peaks reflects the beginning and end of auditory perceptual events. The present data support this idea because sound temporal envelope, the sound characteristic that most strongly influences whether fMRI responses are phasic, also strongly influences whether successive stimuli (e.g., the bursts of a train) are perceptually grouped into a single auditory event. Thus fMRI waveshape may provide a window onto neural activity patterns that reflect the segmentation of our auditory environment into distinct, meaningful events.     

Neural crest and placodal contributions in the development of the glossopharyngeal-vagal complex in the chick

By using the method of quail-to-chick transplantation of neural crest in one series (VNG) and placodal ectoderm in a second series (VPG) we were able to determine the relative contribution of cranial neural crest and placodal ectoderm to the formation of the Glossopharyngeal-vagal complex. In chimeric embryos, quail cells originating from cranial neural crest grafts of postotic levels end up in the root ganglia, while quail cells originating from placodal ectoderm of postotic levels end up in the trunk ganglia. The results clearly indicate that the caudal levels of the medulla and rostral cervical segments represent the site, and the neural crest the source, for the neurons of the root ganglia. The neurons form a homogenous population of the small-cell type. This clearly rules out any contribution to the root ganglia from placodal ectoderm. On the basis of our experiments, it is also concluded that the neurons of the trunk ganglia are purely placodal in origin and are composed of a population of cells of the large-cell type. Our experiments also provide convincing evidence for a neural crest origin for Schwann cell and ganglionic Satellite cells.     

Responses of inferior colliculus neurons to double harmonic tones

The auditory system can segregate sounds that overlap in time and frequency, if the sounds differ in acoustic properties such as fundamental frequency (f0). However, the neural mechanisms that underlie this ability are poorly understood. Responses of neurons in the inferior colliculus (IC) of the anesthetized chinchilla were measured. The stimuli were harmonic tones, presented alone (single harmonic tones) and in the presence of a second harmonic tone with a different f0 (double harmonic tones). Responses to single harmonic tones exhibited no stimulus-related temporal pattern, or in some cases, a simple envelope modulated at f0. Responses to double harmonic tones exhibited complex slowly modulated discharge patterns. The discharge pattern varied with the difference in f0 and with characteristic frequency. The discharge pattern also varied with the relative levels of the two tones; complex temporal patterns were observed when levels were equal, but as the level difference increased, the discharge pattern reverted to that associated with single harmonic tones. The results indicated that IC neurons convey information about simultaneous sounds in their temporal discharge patterns and that the patterns are produced by interactions between adjacent components in the spectrum. The representation is "low-resolution," in that it does not convey information about single resolved components from either individual sound.     

Medical data mining using evolutionary computation

In this paper, we introduce a system for discovering medical knowledge by learning Bayesian networks and rules. Evolutionary computation is used as the search algorithm. The Bayesian networks can provide an overall structure of the relationships among the attributes. The rules can capture detailed and interesting patterns in the database. The system is applied to real-life medical databases for limb fracture and scoliosis. The knowledge discovered provides insights to and allows better understanding of these two medical domains.