Online dimensionality reduction using competitive learning and Radial Basis Function network

The general purpose dimensionality reduction method should preserve data interrelations at all scales. Additional desired features include online projection of new data, processing nonlinearly embedded manifolds and large amounts of data. The proposed method, called RBF-NDR, combines these features.RBF-NDR is comprised of two modules. The first module learns manifolds by utilizing modified topology representing networks and geodesic distance in data space and approximates sampled or streaming data with a finite set of reference patterns, thus achieving scalability. Using input from the first module, the dimensionality reduction module constructs mappings between observation and target spaces. Introduction of specific loss function and synthesis of the training algorithm for Radial Basis Function network results in global preservation of data structures and online processing of new patterns.The RBF-NDR was applied for feature extraction and visualization and compared with Principal Component Analysis (PCA), neural network for Sammon’s projection (SAMANN) and Isomap. With respect to feature extraction, the method outperformed PCA and yielded increased performance of the model describing wastewater treatment process. As for visualization, RBF-NDR produced superior results compared to PCA and SAMANN and matched Isomap. For the Topic Detection and Tracking corpus, the method successfully separated semantically different topics.     

Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search

Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which incorporates dimensionality reduction into a direct density-ratio estimation procedure. Our key idea is to find a low-dimensional subspace in which densities are significantly different and perform density-ratio estimation only in this subspace. The proposed method, D³-LHSS (Direct Density-ratio estimation with Dimensionality reduction via Least-squares Hetero-distributional Subspace Search), is shown to overcome the limitation of baseline methods.     

A theory of local learning,the learning channel,and the optimality of backpropagation

In a physical neural system, where storage and processing are intimately intertwined, the rules for adjusting the synaptic weights can only depend on variables that are available locally, such as the activity of the pre- and post-synaptic neurons, resulting in local learning rules. A systematic framework for studying the space of local learning rules is obtained by first specifying the nature of the local variables, and then the functional form that ties them together into each learning rule. Such a framework enables also the systematic discovery of new learning rules and exploration of relationships between learning rules and group symmetries. We study polynomial local learning rules stratified by their degree and analyze their behavior and capabilities in both linear and non-linear units and networks. Stacking local learning rules in deep feedforward networks leads to deep local learning. While deep local learning can learn interesting representations, it cannot learn complex input–output functions, even when targets are available for the top layer. Learning complex input–output functions requires local deep learning where target information is communicated to the deep layers through a backward learning channel. The nature of the communicated information about the targets and the structure of the learning channel partition the space of learning algorithms. For any learning algorithm, the capacity of the learning channel can be defined as the number of bits provided about the error gradient per weight, divided by the number of required operations per weight. We estimate the capacity associated with several learning algorithms and show that backpropagation outperforms them by simultaneously maximizing the information rate and minimizing the computational cost. This result is also shown to be true for recurrent networks, by unfolding them in time. The theory clarifies the concept of Hebbian learning, establishes the power and limitations of local learning rules, introduces the learning channel which enables a formal analysis of the optimality of backpropagation, and explains the sparsity of the space of learning rules discovered so far.     

A general soft label based Linear Discriminant Analysis for semi-supervised dimensionality reduction

Dealing with high-dimensional data has always been a major problem in research of pattern recognition and machine learning, and Linear Discriminant Analysis (LDA) is one of the most popular methods for dimension reduction. However, it only uses labeled samples while neglecting unlabeled samples, which are abundant and can be easily obtained in the real world. In this paper, we propose a new dimension reduction method, called “SL-LDA”, by using unlabeled samples to enhance the performance of LDA. The new method first propagates label information from the labeled set to the unlabeled set via a label propagation process, where the predicted labels of unlabeled samples, called “soft labels”, can be obtained. It then incorporates the soft labels into the construction of scatter matrixes to find a transformed matrix for dimension reduction. In this way, the proposed method can preserve more discriminative information, which is preferable when solving the classification problem. We further propose an efficient approach for solving SL-LDA under a least squares framework, and a flexible method of SL-LDA (FSL-LDA) to better cope with datasets sampled from a nonlinear manifold. Extensive simulations are carried out on several datasets, and the results show the effectiveness of the proposed method.     

From Parametric Representation to Dynamical System: Shifting Views of the Motor Cortex in Motor Control

In contrast to traditional representational perspectives in which the motor cortex is involved in motor control via neuronal preference for kinetics and kinematics, a dynamical system perspective emerging in the last decade views the motor cortex as a dynamical machine that generates motor commands by autonomous temporal evolution. In this review, we first look back at the history of the representational and dynamical perspectives and discuss their explanatory power and controversy from both empirical and computational points of view. Here, we aim to reconcile the above perspectives, and evaluate their theoretical impact, future direction, and potential applications in brain-machine interfaces.     

Convergence results for the EM approach to mixtures of experts architectures

The Expectation-Maximization (EM) algorithm is an iterative approach to maximum likelihood parameter estimation. Jordan and Jacobs recently proposed an EM algorithm for the mixture of experts architecture of Jacobs, Jordan, Nowlan and Hinton (1991) and the hierarchical mixture of experts architecture of Jordan and Jacobs (1992). They showed empirically that the EM algorithm for these architectures yields significantly faster convergence than gradient ascent. In the current paper we provide a theoretical analysis of this algorithm. We show that the algorithm can be regarded as a variable metric algorithm with its searching direction having a positive projection on the gradient of the log likelihood. We also analyze the convergence of the algorithm and provide an explicit expression for the convergence rate. In addition, we describe an acceleration technique that yields a significant speedup in simulation experiments.     

A diagnostic approach to multiple simultaneous intracerebral hemorrhages

Multiple simultaneous intracerebral hemorrhages are rare and varied in etiology. In the absence of known risk factors or obvious underlying disease, determining the cause may be problematic. We outline a diagnostic approach to multiple simultaneous intracerebral hemorrhages in the context of three case studies. We conclude that when there are no apparent risk factors, neuroimaging and identification of underlying diseases are central to determining the cause of multiple simultaneous intracerebral hemorrhage.     

Prediction of pediatric unipolar depression using multiple neuromorphometric measurements: A pattern classification approach

BackgroundDiagnosis of pediatric neuropsychiatric disorders such as unipolar depression is largely based on clinical judgment – without objective biomarkers to guide diagnostic process and subsequent therapeutic interventions. Neuroimaging studies have previously reported average group-level neuroanatomical differences between patients with pediatric unipolar depression and healthy controls. In the present study, we investigated the utility of multiple neuromorphometric indices in distinguishing pediatric unipolar depression patients from healthy controls at an individual subject level.MethodsWe acquired structural T₁-weighted scans from 25 pediatric unipolar depression patients and 26 demographically matched healthy controls. Multiple neuromorphometric indices such as cortical thickness, volume, and cortical folding patterns were obtained. A support vector machine pattern classification model was ‘trained’ to distinguish individual subjects with pediatric unipolar depression from healthy controls based on multiple neuromorphometric indices and model predictive validity (sensitivity and specificity) calculated.ResultsThe model correctly identified 40 out of 51 subjects translating to 78.4% accuracy, 76.0% sensitivity and 80.8% specificity, chi-square p-value = 0.000049. Volumetric and cortical folding abnormalities in the right thalamus and right temporal pole respectively were most central in distinguishing individual patients with pediatric unipolar depression from healthy controls.ConclusionsThese findings provide evidence that a support vector machine pattern classification model using multiple neuromorphometric indices may qualify as diagnostic marker for pediatric unipolar depression. In addition, our results identified the most relevant neuromorphometric features in distinguishing PUD patients from healthy controls.     

A group analytic approach to the education of children and teenagers

Educational settings, from classrooms to entire districts, are besieged by numerous internal and external problems. This paper presents situations that utilize group therapy concepts and methods within educational settings. Reasons for resistance to using group strategies are discussed as well as methods for resolving resistance. The need for building basic trust and cohesion is examined. Theoretical constructs are discussed. Finally, current research is described. This paper was presented as a poster at the 54th annual conference of the American Group Psychotherapy Association, February 1996, in San Francisco, CA.     

Neural networks with multiple general neuron models: A hybrid computational intelligence approach using Genetic Programming

Classical neural networks are composed of neurons whose nature is determined by a certain function (the neuron model), usually pre-specified. In this paper, a type of neural network (NN-GP) is presented in which: (i) each neuron may have its own neuron model in the form of a general function, (ii) any layout (i.e network interconnection) is possible, and (iii) no bias nodes or weights are associated to the connections, neurons or layers. The general functions associated to a neuron are learned by searching a function space. They are not provided a priori, but are rather built as part of an Evolutionary Computation process based on Genetic Programming. The resulting network solutions are evaluated based on a fitness measure, which may, for example, be based on classification or regression errors. Two real-world examples are presented to illustrate the promising behaviour on classification problems via construction of a low-dimensional representation of a high-dimensional parameter space associated to the set of all network solutions.     

A regularization approach to continuous learning with an application to financial derivatives pricing

We consider the training of neural networks in cases where the nonlinear relationship of interest gradually changes over time. One possibility to deal with this problem is by regularization where a variation penalty is added to the usual mean squared error criterion. To learn the regularized network weights we suggest the Iterative Extended Kalman Filter (IEKF) as a learning rule, which may be derived from a Bayesian perspective on the regularization problem. A primary application of our algorithm is in financial derivatives pricing, where neural networks may be used to model the dependency of the derivatives’ price on one or several underlying assets. After giving a brief introduction to the problem of derivatives pricing we present experiments with German stock index options data showing that a regularized neural network trained with the IEKF outperforms several benchmark models and alternative learning procedures. In particular, the performance may be greatly improved using a newly designed neural network architecture that accounts for no-arbitrage pricing restrictions.     

Hemispheric resource availability influences face perception: A multiple resource approach to social perception

Two experiments tested the hypothesis that social perception recruits distinct limited-capacity processing resources that are distinguished by the cerebral hemispheres. To test this hypothesis, social perception efficiency was assessed after relevant hemispheric processing resources were depleted. In Experiment 1 prime faces were unilaterally presented for 30 ms, after which centrally presented target faces were categorised by sex. In Experiment 2 prime faces were unilaterally presented for 80 ms after which centrally presented target faces were categorised by fame. Results showed that sex categorisation was slower after primes were presented in the right versus left visual field, and that fame categorisation was slower after familiar primes were presented in the left versus right visual field. The results support a multiple resource account of social perception in which the availability of resources distributed across the cerebral hemispheres influences social perception.     

A process approach to verbal memory assessment: Exploratory evidence of inefficient learning in women remitted from anorexia nervosa

Introduction: Anorexia nervosa (AN) is associated with deficits in set-shifting and cognitive flexibility, yet less is known about the persistence of these deficits after recovery and how they might contribute to reported difficulties organizing and learning new information. To address this question, the current study applied a process-focused approach, that accounts for errors and strategies by which a score is achieved, to investigate the relationship between verbal memory and executive function in women remitted from AN.

Method: Twenty-six women remitted from anorexia nervosa (RAN) and 25 control women (CW) aged 19–45 completed the California Verbal Learning Test, Second edition (CVLT-II) and the Wisconsin Card Sorting Test (WCST). Groups were compared on overall achievement scores, and on repetition, intrusion, and perseverative errors on both tests. Associations between learning and memory performance and WCST errors were also examined.

Results: RAN and CW groups did not differ on overall CVLT-II learning and memory performance or errors on the WCST, though the RAN group trended towards greater WCST non-perseverative and total errors. On the CVLT-II, the RAN group made significantly more repetition errors than CW (p = 0.010), and within-trial perseveration (WTP) errors (p = 0.044). For the CW group, CVLT-II learning and memory performance were negatively associated with errors on the WCST, whereas among RAN, primarily delayed memory was negatively correlated with WCST errors. Notably, for RAN, greater WCST perseverative responses were correlated with greater CVLT-II repetition and WTP errors, showing the convergence of perseverative responding across tasks.

Conclusions: Despite similar overall learning and memory performance, difficulties with executive control seem to persist even after symptom remission in patients with AN. Results indicate an inefficient learning process in the cognitive phenotype of AN and support the use of process approaches to refine neuropsychological assessment of AN by accounting for strategy use.     

A multi-label,semi-supervised classification approach applied to personality prediction in social media

Social media allow web users to create and share content pertaining to different subjects, exposing their activities, opinions, feelings and thoughts. In this context, online social media has attracted the interest of data scientists seeking to understand behaviours and trends, whilst collecting statistics for social sites. One potential application for these data is personality prediction, which aims to understand a user’s behaviour within social media. Traditional personality prediction relies on users’ profiles, their status updates, the messages they post, etc. Here, a personality prediction system for social media data is introduced that differs from most approaches in the literature, in that it works with groups of texts, instead of single texts, and does not take users’ profiles into account. Also, the proposed approach extracts meta-attributes from texts and does not work directly with the content of the messages. The set of possible personality traits is taken from the Big Five model and allows the problem to be characterised as a multi-label classification task. The problem is then transformed into a set of five binary classification problems and solved by means of a semi-supervised learning approach, due to the difficulty in annotating the massive amounts of data generated in social media. In our implementation, the proposed system was trained with three well-known machine-learning algorithms, namely a Naïve Bayes classifier, a Support Vector Machine, and a Multilayer Perceptron neural network. The system was applied to predict the personality of Tweets taken from three datasets available in the literature, and resulted in an approximately 83% accurate prediction, with some of the personality traits presenting better individual classification rates than others.     

A physical system approach to recognition memory for spatially transformed faces

Recognition memory for familiar and unfamiliar spatial frequency filtered faces was examined using a combination of experiments and computer simulations based on the physical-systems approach to memory. The results of the experiments showed a significant interaction in recognition performance between the frequency characteristics of the learning and test pictures of the faces. In general, recognition transfer between normal and lowpass faces was good. Transfer between normal and high-pass faces was not as good but improved with the familiarity of the faces. Transfer between high-pass and low-pass faces was poor. A simple linear associative model of the experiment which used only basic visual information about the faces as input produced a pattern of results similar to those seen in the experiments. Memory enhancers aimed at improving the encoding and storage mechanisms of the model produced results similar to those seen when observers were familiar with the faces. These results are discussed in terms of the literature on recognition of faces which have been transformed in different ways and in terms of some psychological interpretations of codes and algorithms used in connectionist models.     

A machine learning approach using EEG data to predict response to SSRI treatment for major depressive disorder

Objective

The problem of identifying, in advance, the most effective treatment agent for various psychiatric conditions remains an elusive goal. To address this challenge, we investigate the performance of the proposed machine learning (ML) methodology (based on the pre-treatment electroencephalogram (EEG)) for prediction of response to treatment with a selective serotonin reuptake inhibitor (SSRI) medication in subjects suffering from major depressive disorder (MDD).

Methods

A relatively small number of most discriminating features are selected from a large group of candidate features extracted from the subject’s pre-treatment EEG, using a machine learning procedure for feature selection. The selected features are fed into a classifier, which was realized as a mixture of factor analysis (MFA) model, whose output is the predicted response in the form of a likelihood value. This likelihood indicates the extent to which the subject belongs to the responder vs. non-responder classes. The overall method was evaluated using a “leave-n-out” randomized permutation cross-validation procedure.

Results

A list of discriminating EEG biomarkers (features) was found. The specificity of the proposed method is 80.9% while sensitivity is 94.9%, for an overall prediction accuracy of 87.9%. There is a 98.76% confidence that the estimated prediction rate is within the interval [75%, 100%].

Conclusions

These results indicate that the proposed ML method holds considerable promise in predicting the efficacy of SSRI antidepressant therapy for MDD, based on a simple and cost-effective pre-treatment EEG.

Significance

The proposed approach offers the potential to improve the treatment of major depression and to reduce health care costs.