An intelligent sales forecasting system through integration of artificial neural networks and fuzzy neural networks with fuzzy weight elimination.

Sales forecasting plays a very prominent role in business strategy. Numerous investigations addressing this problem have generally employed statistical methods, such as regression or autoregressive and moving average (ARMA). However, sales forecasting is very complicated owing to influence by internal and external environments. Recently, artificial neural networks (ANNs) have also been applied in sales forecasting since their promising performances in the areas of control and pattern recognition. However, further improvement is still necessary since unique circumstances, e.g. promotion, cause a sudden change in the sales pattern. Thus, this study utilizes a proposed fuzzy neural network (FNN), which is able to eliminate the unimportant weights, for the sake of learning fuzzy IF-THEN rules obtained from the marketing experts with respect to promotion. The result from FNN is further integrated with the time series data through an ANN. Both the simulated and real-world problem results show that FNN with weight elimination can have lower training error compared with the regular FNN. Besides, real-world problem results also indicate that the proposed estimation system outperforms the conventional statistical method and single ANN in accuracy.     

The recalling process dynamics of associative memory neural networks in macrodynamical approach

The results of a computer study of the continuous-time version of macrodynamical system of equations governing the recalling process of associative memory neural networks are presented. The comparative analysis of two models of associative memory network—recurrent (autoassociative) and layered (feedforward)—is given. The phase portraits of macrodynamical system at a variety of representative values of parameter , the loading ratio, are obtained and the appearance of the bifurcation of equilibrium frustration (the saddle-node bifurcation) is demonstrated. The behavior of the basins of attraction of network dynamics equilibria is studied as well.     

Generalized neural networks for spectral analysis: dynamics and Liapunov functions.

This paper analyzes local and global behavior of several dynamical systems which generalize some artificial neural network (ANN) semilinear models originally designed for principal component analysis (PCA) in the characterization of random vectors. These systems implicitly performed the spectral analysis of correlation (i.e. symmetric positive definite) matrices. Here, the proposed generalizations cover both nonsymmetric matrices as well as fully nonlinear models. Local stability analysis is performed via linearization and global behavior is analyzed by constructing several Liapunov functions.     

Deterministic dynamics emerging from a cortical functional architecture.

Cerebral cortex has a range of interconnected functional architectures. Some appear random and without structure, while others are geometrical. Although the biological details certainly constrain spatial temporal patterns in neural networks, the influence that the laws of deterministic dynamics bring to bear on even isolated simple geometries are unknown. Layer II/III of primary visual cortex has long range horizontal connections with projections to and from other layers. The long range excitatory connections were modeled in isolation as an isolated laterally connected functional architecture. The Hodgkin-Huxley or Pinsky-Rinzel equations were used to simulate the neuronal elements. Waves of activity could propagate through the functional architecture; depending on the synaptic kinetics, the system could settle down into quiescence, oscillations, or seemingly random behavior. Order could be found in random-looking behavior by the application of techniques from chaos theory. Furthermore, the range and transitions of the temporal patterns in the modeled collection of neurons are similar to those found in other non-linear systems. The possibility that the temporal patterns of neurons in situ are also constrained by these mathematical laws is discussed.     

Investigating the temporal dynamics of electroencephalogram (EEG) microstates using recurrent neural networks

Electroencephalogram (EEG) microstates that represent quasi‐stable, global neuronal activity are considered as the building blocks of brain dynamics. Therefore, the analysis of microstate sequences is a promising approach to understand fast brain dynamics that underlie various mental processes. Recent studies suggest that EEG microstate sequences are non‐Markovian and nonstationary, highlighting the importance of the sequential flow of information between different brain states. These findings inspired us to model these sequences using Recurrent Neural Networks (RNNs) consisting of long‐short‐term‐memory (LSTM) units to capture the complex temporal dependencies. Using an LSTM‐based auto encoder framework and different encoding schemes, we modeled the microstate sequences at multiple time scales (200–2,000 ms) aiming to capture stably recurring microstate patterns within and across subjects. We show that RNNs can learn underlying microstate patterns with high accuracy and that the microstate trajectories are subject invariant at shorter time scales (≤400 ms) and reproducible across sessions. Significant drop in the reconstruction accuracy was observed for longer sequence lengths of 2,000 ms. These findings indirectly corroborate earlier studies which indicated that EEG microstate sequences exhibit long‐range dependencies with finite memory content. Furthermore, we find that the latent representations learned by the RNNs are sensitive to external stimulation such as stress while the conventional univariate microstate measures (e.g., occurrence, mean duration, etc.) fail to capture such changes in brain dynamics. While RNNs cannot be configured to identify the specific discriminating patterns, they have the potential for learning the underlying temporal dynamics and are sensitive to sequence aberrations characterized by changes in metal processes. Empowered with the macroscopic understanding of the temporal dynamics that extends beyond short‐term interactions, RNNs offer a reliable alternative for exploring system level brain dynamics using EEG microstate sequences.     

Global dynamics of bidirectional associative memory neural networks involving transmission delays and dead zones

In this article, a model describing the activation dynamics of bidirectional associative memory (BAM) neural networks involving transmission delays was considered. The concept of BAM networks employed in this work is improved and it includes the earlier notions known in the literature and is applied to a wider class of networks. Further, we introduced a new notion, as a measure of restoring stability and termed it as a dead zone. In this article, the influence of the presence of dead zones on the global asymptotic stability of the equilibrium pattern was investigated. Existence and uniqueness of an equilibrium pattern under fairly general and easily verifiable conditions were also established.     

Practical emotional neural networks

In this paper, we propose a limbic-based artificial emotional neural network (LiAENN) for a pattern recognition problem. LiAENN is a novel computational neural model of the emotional brain that models emotional situations such as anxiety and confidence in the learning process, the short paths, the forgetting processes, and inhibitory mechanisms of the emotional brain. In the model, the learning weights are adjusted by the proposed anxious confident decayed brain emotional learning rules (ACDBEL). In engineering applications, LiAENN is utilized in facial detection, and emotion recognition. According to the comparative results on ORL and Yale datasets, LiAENN shows a higher accuracy than other applied emotional networks such as brain emotional learning (BEL) and emotional back propagation (EmBP) based networks.     

Model selection in neural networks

In this article, we examine how model selection in neural networks can be guided by statistical procedures such as hypothesis tests, information criteria and cross validation. The application of these methods in neural network models is discussed, paying attention especially to the identification problems encountered. We then propose five specification strategies based on different statistical procedures and compare them in a simulation study. As the results of the study are promising, it is suggested that a statistical analysis should become an integral part of neural network modeling.     

Collective modes in neural networks

A theoretical model, based on response of a neural network to an external stimulus, was constructed to determine its collective modes. It is suggested that the waves observed in EEG records reflect the cooperative electrical activity of a large number of neurons. Further, an actual EEG time series was analyzed to deduce two dynamic parameters, dimension d of phase space of the neural system and the minimum number of variables nc necessary to describe the EEG pattern. We find d = 6.2 and nc = 11.     

The neural networks of music

Recent neuropsychological, transcranial Doppler sonographic, positron emission tomographic and functional nuclear magnetic resonance studies have indicated that musical perception is not dependent on the right hemisphere but on neural networks corresponding to the fundamental components of music in both hemispheres. In the brain there is no centre for music. Musicians have cerebral characteristics, anatomical as well as functional, which are correlated with the age at which they began their musical studies. This argues for cortical reorganization as a result of musical training. Whether these characteristics are to be ascribed to cortical plasticity alone, or to an innate structural property, or to both, remains an open question, however. Investigation of chromosomal defects, biochemical abnormalities and morphological features of congenital and degenerative brain diseases can provide further insight into the cerebral substrate of musicality.     

Artificial neural networks and schizophrenia

The present study is a follow-up of two different groups of young adults with a history of dyslexia problems in childhood. Group A was drawn from a larger longitudinal study where students were diagnosed with dyslexia at age 10. Group B was recruited at a child psychiatric clinic and also had dyslexic problems. Measures of educational level, life satisfaction and psychosocial factors were applied. The assessment was carried out by means of tests, questionnaires and personal interviews. The results showed lower levels of educational attainment in group B as well as lower satisfaction with health, friends and education compared to group A and a normative group. Both dyslexic groups showed more psychiatric problems than those in the normative sample.     

Facilitating neural dynamics for delay compensation: A road to predictive neural dynamics?

Goal-directed behavior is a hallmark of cognition. An important prerequisite to goal-directed behavior is that of prediction. In order to establish a goal and devise a plan, one needs to see into the future and predict possible future events. Our earlier work has suggested that compensation mechanisms for neuronal transmission delay may have led to a preliminary form of prediction. In that work, facilitating neuronal dynamics was found to be effective in overcoming delay (the Facilitating Activation Network model, or FAN). The extrapolative property of the delay compensation mechanism can be considered as prediction for incoming signals (predicting the present based on the past). The previous FAN model turns out to have a limitation especially when longer delay needs to be compensated, which requires higher facilitation rates than FAN’s normal range. We derived an improved facilitating dynamics at the neuronal level to overcome this limitation. In this paper, we tested our proposed approach in controllers for 2D pole balancing, where the new approach was shown to perform better than the previous FAN model. Next, we investigated the differential utilization of facilitating dynamics in sensory vs. motor neurons and found that motor neurons utilize the facilitating dynamics more than the sensory neurons. These findings are expected to help us better understand the role of facilitating dynamics in delay compensation, and its potential development into prediction, a necessary condition for goal-directed behavior.     

Multi-sensor integration for on-line tool wear estimation through radial basis function networks and fuzzy neural network

On-line tool wear estimation plays a very critical role in industry automation for higher productivity and product quality. In addition, appropriate and timely decision for tool change is significantly required in the machining systems. Thus, this paper is dedicated to develop an estimation system through integration of two promising technologies, artificial neural networks (ANN) and fuzzy logic. An on-line estimation system consisting of five components: (1) data collection; (2) feature extraction; (3) pattern recognition; (4) multi-sensor integration; and (5) tool/work distance compensation for tool flank wear, is proposed herein. For each sensor, a radial basis function (RBF) network is employed to recognize the extracted features. Thereafter, the decisions from multiple sensors are integrated through a proposed fuzzy neural network (FNN) model. Such a model is self-organizing and self-adjusting, and is able to learn from the experience. Physical experiments for the metal cutting process are implemented to evaluate the proposed system. The results show that the proposed system can significantly increase the accuracy of the product profile.     

Projective synchronization for fractional neural networks

In this paper, the global projective synchronization of fractional-order neural networks is investigated. First, a sufficient condition in the sense of Caputo’s fractional derivation to ensure the monotonicity of the continuous and differential functions and a new fractional-order differential inequality are derived, which play central roles in the investigation of the fractional adaptive control. Based on the preparation and some analysis techniques, some novel criteria are obtained to realize projective synchronization of fractional-order neural networks via combining open loop control and adaptive control. As some special cases, several control strategies are given to ensure the realization of complete synchronization, anti-synchronization and the stabilization of the addressed neural networks. Finally, an example with numerical simulations is given to show the effectiveness of the obtained results.     

Motor neural networks in the leech

Leeches display robust motor patterns and exhibit a relatively simple nervous system where neurons are unambiguously identified. This brief article focuses on Hirudo verbana and summarizes how research in this organism has contributed to insights in the field of motor control, where networks have been studied from population down to individual neuron perspectives.     

Delay-dependent multistability in recurrent neural networks

In this article, we focus on the delay-dependent multistability in recurrent neural networks. By constructing Lyapunov functional and using matrix inequality techniques, a novel delay-dependent multistability criterion is derived. The obtained results are more flexible and less conservative than previously known criteria. Two examples are given to show the effectiveness of the obtained criteria. Furthermore, some interesting delay-dependent dynamic behaviors have been showed in a special case, for example, we find that there is the coexistence of stable equilibria and stable limit cycles in the single neuron. Also, when the neurons are coupled, then the stable patterns are more complex.     

Adult neural progenitor cells reactivate superbursting in mature neural networks

Behavioral recovery in animal models of human CNS syndromes suggests that transplanted stem cell derivatives can augment damaged neural networks but the mechanisms behind potentiated recovery remain elusive. Here we use microelectrode array (MEA) technology to document neural activity and network integration as rat primary neurons and rat hippocampal neural progenitor cells (NPCs) differentiate and mature. The natural transition from neuroblast to functional excitatory neuron consists of intermediate phases of differentiation characterized by coupled activity. High-frequency network-wide bursting or "superbursting" is a hallmark of early plasticity that is ultimately refined into mature stable neural network activity. Microelectrode array (MEA)-plated neurons transition through this stage of coupled superbursting before establishing mature neuronal phenotypes in vitro. When plated alone, adult rat hippocampal NPC-derived neurons fail to establish the synchronized bursting activity that neurons in primary and embryonic stem cell-derived cultures readily form. However, adult rat hippocampal NPCs evoke re-emergent superbursting in electrophysiologically mature rat primary neural cultures. Developmental superbursting is thought to accompany transient states of heightened plasticity both in culture preparations and across brain regions. Future work exploring whether NPCs can re-stimulate developmental states in injury models would be an interesting test of their regenerative potential.     

Compact self-wiring in cultured neural networks

We present a novel approach for patterning cultured neural networks in which a particular geometry is achieved via anchoring of cell clusters (tens of cells/each) at specific positions. In addition, compact connections among pairs of clusters occur spontaneously through a single non-adherent straight bundle composed of axons and dendrites. The anchors that stabilize the cell clusters are either poly-D-lysine, a strong adhesive substrate, or carbon nanotubes. Square, triangular and circular structures of connectivity were successfully realized. Monitoring the dynamics of the forming networks in real time revealed that the self-assembly process is mainly driven by the ability of the neuronal cell clusters to move away from each other while continuously stretching a neurite bundle in between. Using the presented technique, we achieved networks with wiring regions which are made exclusively of neuronal processes unbound to the surface. The resulted network patterns are very stable and can be maintained for as long as 11 weeks. The approach can be used to build advanced neuro-chips for bio-sensing applications (e.g. drug and toxin detection) where the structure, stability and reproducibility of the networks are of great relevance.