Improving recall in associative memories by dynamic threshold

In this paper, a simple learning method and a dynamic threshold concept for associative memories (AMs) is presented. The learning approach is designed to store all training patterns with basins of attraction as large as possible. After the learning process stops, the dynamic threshold introduces a threshold in the recall phase. It can reduce the probability of converging to spurious states. A large number of computer simulations are implemented to show the improved recalls.     

Motor seizure patterns: a comparative analysis of tonic, psychomotor and psychogenic seizures

The motor patterns are self-regulating standardized movement models under normal and pathological conditions. Once learned and then automatized, they form the elements of the voluntary motor activity. Accordingly even the motor phenomena of epileptic seizures are defined as models or patterns. This paper is concerned with the question whether the epileptic patterns are to be associated to the neurophysiological regulatory processes of voluntary motoricity or whether these are pathological processes without relation to the physiological movement patterns. At the outset a short introduction to the neurophysiological basis of the voluntary motoricity is presented, which is built up according to the medullary organization of the cortex and taking into consideration the movement control through "feed forward" and "feed back". To the motor patterns of generalized tonic seizures connections to ontogenetic and phylogenetic old movement patterns and their activation through the epileptic excitation are denied. The tonic seizures are considered as pathological epileptic patterns, which are caused as a result of epileptic excitation, following their own dynamics. They represent a clear marking of maximum discharges of the cerebral structures, which control all other neuronal processes as well. The motor phenomena concerning the psychomotor seizures are described as specific patterns in respect of patients and types of seizures. To be differentiated are the elementary patterns of the first stage of seizures and the patterns of psychomotor twilight states. They represent denatured fragments of physiological motoricity and biological behavioural patterns. They would be brought in motion by the epileptic excitation, which in turn could lead to extensive motor phenomena.(ABSTRACT TRUNCATED AT 250 WORDS)     

Two-level hierarchy with sparsely and temporally coded patterns and its possible functional role in information processing.

We study the stability of two types of mixed states in an oscillator neural network model. Patterns to be stored are represented by complex variables whose amplitudes are either 0 (silent state) or 1 (firing state) and whose phases represent the timing of firing. The mixed states are defined in terms of a certain type of average over some specified number s of memory patterns. We define two types of such mixed states, each of which corresponds to a different type of restriction placed on this set of s memory patterns. The stability of each type is investigated both theoretically and numerically. We find that only one type of a mixed state, which consists of temporally correlated patterns, is stable. Finally, we discuss a possible functional role of such mixed states in information processing.     

Controlling chaos in a chaotic neural network.

The chaotic neural network constructed with chaotic neuron shows the associative memory function, but its memory searching process cannot be stabilized in a stored state because of the chaotic motion of the network. In this paper, a pinning control method focused on the chaotic neural network is proposed. The computer simulation proves that the chaos in the chaotic neural network can be controlled with this method and the states of the network can converge in one of its stored patterns if the control strength and the pinning density are chosen suitable. It is found that in general the threshold of the control strength of a controlled network is smaller at higher pinned density and the chaos of the chaotic neural network can be controlled more easily if the pinning control is added to the variant neurons between the initial pattern and the target pattern.     

Cognitive-behavioural theories of helplessness/hopelessness: Valid models of depression?

Helplessness and hopelessness are central aspects of cognitive-behavioural explanations for the development and persistence of depression. In this article a general overview concerning the evolution of those approaches to depression is provided. Included is a critical examination of the theories. The review of the literature suggests that those cognitive models describing helplessness/hopelessness as trait factors mediating depression do not really have a strong empirical base. The majority of those studies had been conducted in healthy or only mildly depressed subjects. Thus, there seems to be little justification for broad generalisations beyond the populations studied. It seems that some of the reported studies have not tested the underlying theories adequately (e. g. correlation had sometimes been interpreted as causation; adequate prospective longitudinal study designs had seldom been applied). Moreover, the theoretical models are not generally prepared to explain all depressive features (e. g. the possibility of a spontaneous shift in a manic episode). Despite those limitations, there is a relevant impact of the learned helplessness paradigm on preclinical research in neurobiological correlates of depressive states. Last but not least, the models are of high interest with respect to the theoretical background of important modules of cognitive-behavioural therapy and its acute and prophylactic effects. Received: 1 October 2002 / Accepted: 2 October 2002 Correspondence to Verena Henkel; MD     

Complex sound processing during human REM sleep by recovering information from long-term memory as revealed by the mismatch negativity (MMN)

Perceptual learning is thought to be the result of neural changes that take place over a period of several hours or days, allowing information to be transferred to long-term memory. Evidence suggests that contents of long-term memory may improve attentive and pre-attentive sensory processing. Therefore, it is plausible to hypothesize that learning-induced neural changes that develop during wakefulness could improve automatic information processing during human REM sleep. The MMN, an objective measure of the automatic change detection in auditory cortex, was used to evaluate long-term learning effects on pre-attentive processing during wakefulness and REM sleep. When subjects learned to discriminate two complex auditory patterns in wakefulness, an increase in the MMN was obtained in both wake and REM states. The automatic detection of the infrequent complex auditory pattern may therefore be improved in both brain states by reactivating information from long-term memory. These findings suggest that long-term learning-related neural changes are accessible during REM sleep as well.     

On redundancy in neural architecture: dynamics of a simple module-based neural network and initial-state independence

This article discusses the relationship between redundancy in neural architecture and activity (cell output or internal state) dynamics with a simple module-based neural network. In the network, a single neural cell with self-feedback is employed as a module sub-network, and all module sub-networks are connected via inter-module connections. In general, the activity dynamics of a single neural cell with positive self-feedback may have two minima in its energy surface, and the minimum the cell state converges to depends on the initial states. However, in the module-based network with all the same intra-module connections, an independence from initial states becomes conspicuous as the number of modules increases due to the architectural redundancy. Simulation and analytical studies on the network dynamics illustrate that the cell states always converge to a global minimum irrelevantly of the initial cell-states, and they never go to a local minimum when a sufficient number of modules are employed.     

Implementation of Imitation Learning using Natural Learner Central Pattern Generator Neural Networks

In this paper a new design of neural networks is introduced, which is able to generate oscillatory patterns. The fundamental building block of the neural network is O-neurons that can generate an oscillation in its transfer functions. Since the natural policy gradient learning has been used in training a central pattern generator paradigm, it is called Natural Learner CPG Neural Networks (NLCPGNN). O-neurons are connected and coupled to each other in order to shape a network and their unknown parameters are found by a natural policy gradient learning algorithm. The main contribution of this paper is design of this learning algorithm which is able to simultaneously search for the weights and topology of the network. This system is capable to obtain any complex motion and rhythmic trajectory via first layer and learn rhythmic trajectories in the second layer and converge towards all these movements. Moreover this two layers system is able to provide various features of a learner model for instance resistance against perturbations, modulation of trajectories amplitude and frequency. Simulation of the learning system in the robot simulator (WEBOTS) that is linked with MATLAB software has been done. Implementation on a real NAO robot demonstrates that the robot has learned desired motion with high accuracy. These results show proposed system produces high convergence rate and low test errors.     

Cerebral asymmetries and interhemispheric processes

Those who attempt to characterize the functions of the cerebral hemispheres tend, broadly speaking, to do so either in terms of structural specializations or of different information-processing modes. Often little attention is paid to the possible importance, in determining the outcome of experiments in this field, of interhemispheric processes. An experiment is described which concurrently studies hemispheric response differences and interhemispheric processes. Sets of dot patterns are learned, each made up of an original and three distortions of the original. The degree of distortion is systematically changed and is quantified in terms of information theory. Subjects then examine pairs of patterns and decide whether they belong to the 'same' family, i.e. an original and one of its distortions, or 'different', i.e. a pattern previously learned and a completely new one. Manual response times are recorded for 'same' and 'different' responses and functions plotted of response latencies against degree of pattern distortion. The pairs of patterns are presented under three different conditions. Either both patterns in one visual field (unilateral condition), one pattern in each visual field (bilateral condition) or the patterns one above the other straddling the vertical meridian (central condition). Response latencies are shortest for the bilateral condition, next shortest for the unilateral condition and slowest for the central condition. Models which may account for these results are examined and one crucially involving changed information transmission rates with increasing distortion of patterns to be compared is found to provide the most parsimonious explanation of the results from all three experimental conditions.     

Computational significance of transient dynamics in cortical networks

Neural responses are most often characterized in terms of the sets of environmental or internal conditions or stimuli with which their firing rate increases or decreases are correlated. Their transient (nonstationary) temporal profiles of activity have received comparatively less attention. Similarly, the computational framework of attractor neural networks puts most emphasis on the representational or computational properties of the stable states of a neural system. Here we review a couple of neurophysiological observations and computational ideas that shift the focus to the transient dynamics of neural systems. We argue that there are many situations in which the transient neural behaviour, while hopping between different attractor states or moving along 'attractor ruins', carries most of the computational and/or behavioural significance, rather than the attractor states eventually reached. Such transients may be related to the computation of temporally precise predictions or the probabilistic transitions among choice options, accounting for Weber's law in decision-making tasks. Finally, we conclude with a more general perspective on the role of transient dynamics in the brain, promoting the view that brain activity is characterized by a high-dimensional chaotic ground state from which transient spatiotemporal patterns (metastable states) briefly emerge. Neural computation has to exploit the itinerant dynamics between these states.     

Computational significance of transient dynamics in cortical networks

Neural responses are most often characterized in terms of the sets of environmental or internal conditions or stimuli with which their firing rate are correlated increases or decreases. Their transient (nonstationary) temporal profiles of activity have received comparatively less attention. Similarly, the computational framework of attractor neural networks puts most emphasis on the representational or computational properties of the stable states of a neural system. Here we review a couple of neurophysiological observations and computational ideas that shift the focus to the transient dynamics of neural systems. We argue that there are many situations in which the transient neural behaviour, while hopping between different attractor states or moving along 'attractor ruins', carries most of the computational and/or behavioural significance, rather than the attractor states eventually reached. Such transients may be related to the computation of temporally precise predictions or the probabilistic transitions among choice options, accounting for Weber's law in decision-making tasks. Finally, we conclude with a more general perspective on the role of transient dynamics in the brain, promoting the view that brain activity is characterized by a high-dimensional chaotic ground state from which transient spatiotemporal patterns (metastable states) briefly emerge. Neural computation has to exploit the itinerant dynamics between these states.     

Retraining in literal alexia: substitution of a right hemisphere perceptual strategy for impaired left hemispheric processing

An adult patient with literal alexia, agraphia, slight anomia, and dyscalculia due to a left hemisphere infarct showed lack of sequential skills while pattern recognition remained intact. Some words were recognized as patterns, but could not be read phonetically. Therapy concentrated on forming an association of the visual pattern of the complete word with the retained auditory pattern. In this way the patient learned to read several hundred words and short phrases, even as anomia worsened. The patterns learned could not be generalized to noun declension or verb conjugation, or broken into smaller words. This learning process is characteristic of right hemispheric skills which were utilized as left hemispheric functions deteriorated.     

Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system

A Fuzzy Adaptive Resonance Theory (ART) model capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns is described. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns. The generalization to learning both analog and binary input patterns is achieved by replacing appearances of the intersection operator (∩) in ART 1 by the MIN operator (Λ) of fuzzy set theory. The MIN operator reduces to the intersection operator in the binary case. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator () of fuzzy set theory play complementary roles. Complement coding uses on-cells and of-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category “boxes”. Smaller vigilance values lead to larger category boxes. Learning stops when the input space is covered by boxes. With fast learning and a finite input set of arbitrary size and composition, learning stabilizes after just one presentation of each input pattern. A fast-commit slow-recode option combines fast learning with a forgetting rule that buffers system memory against noise. Using this option, rare events can be rapidly learned, yet previously learned memories are not rapidly erased in response to statistically unreliable input fluctuations.     

A neural network model for schemas based on pattern completion

Abstract Recent developments in neuroscience have provided us with a wealth of the basic knowledge and tools which are required for neurobiological understanding of the psychological concepts. This advantage enables contemporary scientists to suggest and test brain models for psychological concepts, theories, and methods. Considering the current dominance of biological ideas in psychiatry and psychology, such models are essential in confirmation of the psychological theories of mind. In this article a brain model for schemas as essential to cognitive theory is proposed. Schemas are seen as patterns which are recognized and memorized through the training phase of an autoassociative neural network. Then, these patterns are used to complete ambiguous aspects of future experiences through thalamo and hippocampal-cortical pathways. In relation to the self or the outside world when a pattern with unknown, noisy, or vague aspects is encountered, those aspects are completed by the principal components of previously learned patterns (schema). This process is to help the observer acquire a better understanding of the environment or the self. However, the patterns which are used to complete the uncertainties about the self or the environment are sometimes not good estimates of the reality and lead the person/patient to an illusionary perception of the self/environment. In this article, the role of the mirror neuron system in pattern recognition is also explained. Psychological and biological therapeutic implications of this model are discussed and the importance of a link between dynamic and cognitive therapies is rationalized.     

Restoring partly occluded patterns: a neural network model.

This paper proposes a neural network model that has an ability to restore missing portions of partly occluded patterns. It is a multi-layered hierarchical neural network, in which visual information is processed by interaction of bottom-up and top-down signals. Memories of learned patterns are stored in the connections between cells. Occluded parts of a pattern are reconstructed mainly by top-down signals from higher stages of the network, while the unoccluded parts are reproduced mainly by signals from lower stages. The restoration progresses successfully, even if the occluded pattern is a deformed version of a learned pattern. The model tries to complete even an unlearned pattern by interpolating and extrapolating visible edges. Resemblance of local features to other learned patterns are also utilized for the restoration.     

The role of feedback from the vocal organ. I. Maintenance of stereotypical vocalizations by adult zebra finches

The stereotypical vocal patterns of adult male zebra finches (Poephila guttata) were examined before and after elimination of auditory feedback and/or feedback from the vocal organ (the syrinx). Elimination of auditory feedback was accomplished via bilateral removal of the cochleae, whereas feedback from the syrinx was eliminated by cutting hypoglossal afferent fibers while leaving hypoglossal efferents intact. Very little or no disruption of song was observed in birds which underwent deafening as well as unilateral deafferentation of the syrinx. Control experiments showed that the minor deficits observed were not attributable to lesion of pulmonary fibers in the descending branch of the vagus. There was also little deficit in song behavior of birds that were deafened and subjected to bilateral deafferentation of the syrinx. These results are consistent with the hypothesis that stable song patterns in adult passerine birds are not dependent on peripheral sources of feedback, but may be governed by a learned central control program.     

From prediction error to incentive salience: mesolimbic computation of reward motivation

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience.     

"I tried to be as defiant as possible". Case histories with autobiographical diaries: relationship with the custodial parent during childhood as recalled by young adults

The predominant structural orientation of divorce research which either ignores or under represents to a great extent the child's perspective and his/her agency is criticized. As a consequence of these methodological shortcomings not only the production of scientific knowledge but also professional practice working with children from divorced families might be negatively affected--especially the regulation of custody and visiting issues. Therefore, the aim of this article is to give children a voice in describing their own experiences of visits and relationships with the non-residential father or mother during childhood and adolescence (sometimes until adulthood). By presenting numerous detailed autobiographical narratives from children of divorced parents, the study revealed a great diversity of visiting patterns as well as different relationship qualities ranging from loving to negative. In addition, visiting patterns and relationship qualities were not always static but were dynamic and changed over time, whereby the child's personality and agency contributed to this diversity and these processes. Finally, methodological recommendations are offered and it is suggested that the child's perspective and agency should be included in research about children of divorce.     

Fornix--thalamus fibers, motivational states, and contextual retrieval.

The effect of selectively transecting fornix fibers bound for the anterior thalamic nuclei upon the means by which motivational states influence selection of learned responses was assessed. Rats were deprived of food or water on alternate days. During short sessions they had an opportunity to learn to turn one direction to obtain food and the opposite direction to obtain water in the same T-maze. The control group was a combination of animals which had received sham operations and those in which the attempted transection was off target. Experimental animals behaved much less differentially than controls. The performance levels of the experimental animals in obtaining food and water were more negatively correlated than those of controls. During early stages of the learning procedure the experimental animals were more likely to develop strong individual response preferences or biases than controls. Experimental animals also required more trials to reach criterion than controls. The results are interpreted as indicating that after transection of connections between the subiculum and the anterior thalamic nuclei, motivational states no longer act conditionally upon the selection of learned behavior for performance. Rather they are pooled together with other stimuli. Such an interpretation is derived from the theory that the hippocampus is part of a system mediating contextual retrieval of information from memory.     

Learning and Processing Abstract Words and Concepts: Insights From Typical and Atypical Development

The paper describes two plausible hypotheses concerning the learning of abstract words and concepts. According to a first hypothesis, children would learn abstract words by extracting co-occurrences among words in linguistic input, using, for example, mechanisms as described by models of Distributional Semantics. According to a second hypothesis, children would exploit the fact that abstract words tend to have more emotional associations than concrete words to infer that they refer to internal/mental states. Each hypothesis makes specific predictions with regards to when and which abstract words are more likely to be learned; also they make different predictions concerning the impact of developmental disorders. We start by providing a review of work characterizing how abstract words and concepts are learned in development, especially between the ages of 6 and 12. Second, we review some work from our group that tests the two hypotheses above. This work investigates typically developing (TD) children and children with atypical development (developmental language disorders [DLD] and autism spectrum disorder [ASD] with and without language deficits). We conclude that the use of strategies based on emotional information, or on co-occurrences in language, may play a role at different developmental stages.