Semi-supervised learning for ordinal Kernel Discriminant Analysis

Ordinal classification considers those classification problems where the labels of the variable to predict follow a given order. Naturally, labelled data is scarce or difficult to obtain in this type of problems because, in many cases, ordinal labels are given by a user or expert (e.g. in recommendation systems). Firstly, this paper develops a new strategy for ordinal classification where both labelled and unlabelled data are used in the model construction step (a scheme which is referred to as semi-supervised learning). More specifically, the ordinal version of kernel discriminant learning is extended for this setting considering the neighbourhood information of unlabelled data, which is proposed to be computed in the feature space induced by the kernel function. Secondly, a new method for semi-supervised kernel learning is devised in the context of ordinal classification, which is combined with our developed classification strategy to optimise the kernel parameters. The experiments conducted compare 6 different approaches for semi-supervised learning in the context of ordinal classification in a battery of 30 datasets, showing (1) the good synergy of the ordinal version of discriminant analysis and the use of unlabelled data and (2) the advantage of computing distances in the feature space induced by the kernel function.     

Scalable learning method for feedforward neural networks using minimal-enclosing-ball approximation

Training feedforward neural networks (FNNs) is one of the most critical issues in FNNs studies. However, most FNNs training methods cannot be directly applied for very large datasets because they have high computational and space complexity. In order to tackle this problem, the CCMEB (Center-Constrained Minimum Enclosing Ball) problem in hidden feature space of FNN is discussed and a novel learning algorithm called HFSR-GCVM (hidden-feature-space regression using generalized core vector machine) is developed accordingly. In HFSR-GCVM, a novel learning criterion using L2-norm penalty-based $\epsilon$-insensitive function is formulated and the parameters in the hidden nodes are generated randomly independent of the training sets. Moreover, the learning of parameters in its output layer is proved equivalent to a special CCMEB problem in FNN hidden feature space. As most CCMEB approximation based machine learning algorithms, the proposed HFSR-GCVM training algorithm has the following merits: The maximal training time of the HFSR-GCVM training is linear with the size of training datasets and the maximal space consumption is independent of the size of training datasets. The experiments on regression tasks confirm the above conclusions.     

Feature selection and multi-kernel learning for sparse representation on a manifold

Sparse representation has been widely studied as a part-based data representation method and applied in many scientific and engineering fields, such as bioinformatics and medical imaging. It seeks to represent a data sample as a sparse linear combination of some basic items in a dictionary. Gao et al. (2013) recently proposed Laplacian sparse coding by regularizing the sparse codes with an affinity graph. However, due to the noisy features and nonlinear distribution of the data samples, the affinity graph constructed directly from the original feature space is not necessarily a reliable reflection of the intrinsic manifold of the data samples. To overcome this problem, we integrate feature selection and multiple kernel learning into the sparse coding on the manifold. To this end, unified objectives are defined for feature selection, multiple kernel learning, sparse coding, and graph regularization. By optimizing the objective functions iteratively, we develop novel data representation algorithms with feature selection and multiple kernel learning respectively. Experimental results on two challenging tasks, N-linked glycosylation prediction and mammogram retrieval, demonstrate that the proposed algorithms outperform the traditional sparse coding methods.     

Constructing Bayesian formulations of sparse kernel learning methods.

We present here a simple technique that simplifies the construction of Bayesian treatments of a variety of sparse kernel learning algorithms. An incomplete Cholesky factorisation is employed to modify the dual parameter space, such that the Gaussian prior over the dual model parameters is whitened. The regularisation term then corresponds to the usual weight-decay regulariser, allowing the Bayesian analysis to proceed via the evidence framework of MacKay. There is in addition a useful by-product associated with the incomplete Cholesky factorisation algorithm, it also identifies a subset of the training data forming an approximate basis for the entire dataset in the kernel-induced feature space, resulting in a sparse model. Bayesian treatments of the kernel ridge regression (KRR) algorithm, with both constant and heteroscedastic (input dependent) variance structures, and kernel logistic regression (KLR) are provided as illustrative examples of the proposed method, which we hope will be more widely applicable.     

A novel test to determine the significance of neural selectivity to single and multiple potentially correlated stimulus features

Mutual information is a principled non-linear measure of dependence between stochastic variables, which is widely used to study the selectivity of neural responses to external stimuli. Here we define and develop a set of novel statistical independence tests based on mutual information, which quantify the significance of neural selectivity to either single features or to multiple, potentially correlated stimulus features like those often present in naturalistic stimuli. If the values of different features are correlated during stimulus presentation, it is difficult to establish if one feature is genuinely encoded by the response, or if it instead appears to be encoded only as a side effect of its correlation with another genuinely represented feature. Our tests provide a way to disambiguate between these two possibilities. We use realistic simulations of neural responses tuned to one or more correlated stimulus features to investigate how limited sampling bias correction procedures affect the statistical power of such independence tests, and we characterize the regimes in which the distribution of information values under the null hypothesis can be approximated by simple distributions (Chi-square or Gaussian). Finally, we apply these tests to experimental data to determine the significance of tuning of the band limited power (BLP) of the gamma [30-100 Hz] frequency range of the primary visual cortical local field potential to multiple correlated features during presentation of naturalistic movies. We show that gamma BLP carries significant, genuine information about orientation, space contrast and time contrast, despite the strong correlations between these features.     

A novel kernel-based maximum a posteriori classification method

Kernel methods have been widely used in pattern recognition. Many kernel classifiers such as Support Vector Machines (SVM) assume that data can be separated by a hyperplane in the kernel-induced feature space. These methods do not consider the data distribution and are difficult to output the probabilities or confidences for classification. This paper proposes a novel Kernel-based Maximum A Posteriori (KMAP) classification method, which makes a Gaussian distribution assumption instead of a linear separable assumption in the feature space. Robust methods are further proposed to estimate the probability densities, and the kernel trick is utilized to calculate our model. The model is theoretically and empirically important in the sense that: (1) it presents a more generalized classification model than other kernel-based algorithms, e.g., Kernel Fisher Discriminant Analysis (KFDA); (2) it can output probability or confidence for classification, therefore providing potential for reasoning under uncertainty; and (3) multi-way classification is as straightforward as binary classification in this model, because only probability calculation is involved and no one-against-one or one-against-others voting is needed. Moreover, we conduct an extensive experimental comparison with state-of-the-art classification methods, such as SVM and KFDA, on both eight UCI benchmark data sets and three face data sets. The results demonstrate that KMAP achieves very promising performance against other models.     

A new nonlinear similarity measure for multichannel signals

We propose a novel similarity measure, called the correntropy coefficient, sensitive to higher order moments of the signal statistics based on a similarity function called the cross-correntopy. Cross-correntropy nonlinearly maps the original time series into a high-dimensional reproducing kernel Hilbert space (RKHS). The correntropy coefficient computes the cosine of the angle between the transformed vectors. Preliminary experiments with simulated data and multichannel electroencephalogram (EEG) signals during behaviour studies elucidate the performance of the new measure versus the well-established correlation coefficient.     

Characterization of neuromagnetic brain rhythms over time scales of minutes using spatial independent component analysis

Independent component analysis (ICA) of electroencephalographic (EEG) and magnetoencephalographic (MEG) data is usually performed over the temporal dimension: each channel is one row of the data matrix, and a linear transformation maximizing the independence of component time courses is sought. In functional magnetic resonance imaging (fMRI), by contrast, most studies use spatial ICA: each time point constitutes a row of the data matrix, and independence of the spatial patterns is maximized. Here, we show the utility of spatial ICA in characterizing oscillatory neuromagnetic signals. We project the sensor data into cortical space using a standard minimum-norm estimate and apply a sparsifying transform to focus on oscillatory signals. The resulting method, spatial Fourier-ICA, provides a concise summary of the spatiotemporal and spectral content of spontaneous neuromagnetic oscillations in cortical source space over time scales of minutes. Spatial Fourier-ICA applied to resting-state and naturalistic stimulation MEG data from nine healthy subjects revealed consistent components covering the early visual, somatosensory and motor cortices with spectral peaks at ～10 and ～20 Hz. The proposed method seems valuable for inferring functional connectivity, stimulus-related modulation of rhythmic activity, and their commonalities across subjects from nonaveraged MEG data.     

Independent component analysis (ICA) of generalized spike wave discharges in fMRI: Comparison with general linear model‐based EEG‐fMRI

Most EEG‐fMRI studies in epileptic patients are analyzed using the general linear model (GLM), which assumes a known hemodynamic response function (HRF) to epileptic spikes. In contrast, independent component analysis (ICA) can extract blood‐oxygenation level dependent (BOLD) responses without imposing constraints on the HRF. ICA might therefore detect responses that vary in time and shape, and that are not detected in the GLM analysis. In this study, we compared the findings of ICA and GLM analyses in 12 patients with idiopathic generalized epilepsy. Spatial ICA was used to extract independent components from the functional magnetic resonance imaging (fMRI) data. A deconvolution method identified component time courses significantly related to the generalized EEG discharges, without constraining the shape of the HRF. The results from the ICA analysis were compared to those from the GLM analysis. GLM maps and ICA maps showed significant correlation and revealed BOLD responses in the thalamus, caudate nucleus, and default mode areas. In patients with a low rate of discharges per minute, the GLM analysis detected BOLD signal changes within the thalamus and the caudate nucleus that were not revealed by the ICA. In conclusion, ICA is a viable alternative technique to GLM analyses in EEG‐fMRI studies related to generalized discharges. This study demonstrated that the BOLD response largely resembles the standard HRF and that GLM analysis is adequate. However, ICA is more dependent on a sufficient number of events than GLM analysis. Hum Brain Mapp, 2011. © 2010 Wiley‐Liss, Inc.     

RKF-PCA: Robust kernel fuzzy PCA

Principal component analysis (PCA) is a mathematical method that reduces the dimensionality of the data while retaining most of the variation in the data. Although PCA has been applied in many areas successfully, it suffers from sensitivity to noise and is limited to linear principal components. The noise sensitivity problem comes from the least-squares measure used in PCA and the limitation to linear components originates from the fact that PCA uses an affine transform defined by eigenvectors of the covariance matrix and the mean of the data.In this paper, a robust kernel PCA method that extends the kernel PCA and uses fuzzy memberships is introduced to tackle the two problems simultaneously. We first introduce an iterative method to find robust principal components, called Robust Fuzzy PCA (RF-PCA), which has a connection with robust statistics and entropy regularization. The RF-PCA method is then extended to a non-linear one, Robust Kernel Fuzzy PCA (RKF-PCA), using kernels. The modified kernel used in the RKF-PCA satisfies the Mercer’s condition, which means that the derivation of the K-PCA is also valid for the RKF-PCA. Formal analyses and experimental results suggest that the RKF-PCA is an efficient non-linear dimension reduction method and is more noise-robust than the original kernel PCA.     

Optimal control by least squares support vector machines.

Support vector machines have been very successful in pattern recognition and function estimation problems. In this paper we introduce the use of least squares support vector machines (LS-SVM's) for the optimal control of nonlinear systems. Linear and neural full static state feedback controllers are considered. The problem is formulated in such a way that it incorporates the N-stage optimal control problem as well as a least squares support vector machine approach for mapping the state space into the action space. The solution is characterized by a set of nonlinear equations. An alternative formulation as a constrained nonlinear optimization problem in less unknowns is given, together with a method for imposing local stability in the LS-SVM control scheme. The results are discussed for support vector machines with radial basis function kernel. Advantages of LS-SVM control are that no number of hidden units has to be determined for the controller and that no centers have to be specified for the Gaussian kernels when applying Mercer's condition. The curse of dimensionality is avoided in comparison with defining a regular grid for the centers in classical radial basis function networks. This is at the expense of taking the trajectory of state variables as additional unknowns in the optimization problem, while classical neural network approaches typically lead to parametric optimization problems. In the SVM methodology the number of unknowns equals the number of training data, while in the primal space the number of unknowns can be infinite dimensional. The method is illustrated both on stabilization and tracking problems including examples on swinging up an inverted pendulum with local stabilization at the endpoint and a tracking problem for a ball and beam system.     

Optimization of rs-fMRI parameters in the Seed Correlation Analysis (SCA) in DPARSF toolbox: A preliminary study

There are a number of various methods of resting-state functional magnetic resonance imaging (rs-fMRI) analysis such as independent component analysis, multivariate autoregressive models, or seed correlation analysis however their results depend on arbitrary choice of parameters. Therefore, the aim of this work was to optimize the parameters in the seed correlation analysis using the Data Processing Assistant for Resting-State fMRI (DPARSF) toolbox for rs-fMRI data received from a Siemens Magnetom Skyra 3-Tesla scanner using a whole-brain, gradient-echo echo planar sequence with a 32-channel head coil. Different ranges of the following parameters: amplitude of low-frequency fluctuation (ALFF), Gaussian kernel at FWHM and radius of spherical ROI for 109 regions were tested for 20 healthy volunteers. The highest values of functional connectivity (FC) correlations were found for ALFF 0.01–0.08, spherical ROIs with the 8-mm radius and Gaussian kernel 8 mm at FWHM in all the studied areas that is, Auditory, Sensimotor, Visual, and Default Mode Network. The dominating influence of ALFF and smoothing on values of FC correlations was noted.     

A kernel-based framework to tensorial data analysis

Tensor-based techniques for learning allow one to exploit the structure of carefully chosen representations of data. This is a desirable feature in particular when the number of training patterns is small which is often the case in areas such as biosignal processing and chemometrics. However, the class of tensor-based models is somewhat restricted and might suffer from limited discriminative power. On a different track, kernel methods lead to flexible nonlinear models that have been proven successful in many different contexts. Nonetheless, a naïve application of kernel methods does not exploit structural properties possessed by the given tensorial representations. The goal of this work is to go beyond this limitation by introducing non-parametric tensor-based models. The proposed framework aims at improving the discriminative power of supervised tensor-based models while still exploiting the structural information embodied in the data. We begin by introducing a feature space formed by multilinear functionals. The latter can be considered as the infinite dimensional analogue of tensors. Successively we show how to implicitly map input patterns in such a feature space by means of kernels that exploit the algebraic structure of data tensors. The proposed tensorial kernel links to the MLSVD and features an interesting invariance property; the approach leads to convex optimization and fits into the same primal-dual framework underlying SVM-like algorithms.     

Gait evaluation in stroke hemiplegic patients using short physical performance battery

ObjectivesFalls can occur daily in stroke patients and appropriate independence assessments for fall prevention are required. Although previous studies evaluated the short physical performance battery (SPPB) in stroke patients, the relationship between SPPB and fall prediction and walking independence remains unclear. Therefore, we aimed to verify whether SPPB is a predictor of walking independence.Materials and methodsThe present study included 105 hemiplegic stroke patients who were admitted to the rehabilitation ward and gave consent to participate. Cross-sectional physical function and functional independence measure cognitive (FIM-C) evaluation were conducted in hemiplegic stroke patients. Logistic regression analysis using the increasing variable method (likelihood ratio) was performed to extract factors for walking independence. Cutoff values were calculated for the extracted items using the receiver operating-characteristic (ROC) curve.ResultsAmong 86 participants included in the final analysis, 36 were independent walkers and 50 were dependent walkers. In the logistic regression analysis, SPPB and FIM-C were extracted as factors for walking independence. The cutoff value was 7 [area under the curve (AUC), 0.94; sensitivity, 0.83; specificity, 0.73)] for SPPB and 32 (AUC, 0.83; sensitivity, 0.69; specificity, 0.57) for FIM-C in ROC analysisConclusionsSPPB and FIM-C were extracted as factors for walking independence. Although SPPB alone cannot determine independent walking, combined assessment of SPPB with cognitive function may enable more accurate determination of walking independence.     

A transdiagnostic evaluation of contrast avoidance across generalized anxiety disorder,major depressive disorder,and social anxiety disorder

BackgroundThe contrast avoidance model (CAM) proposes that persons with generalized anxiety disorder (GAD) are sensitive to sharp increases in negative emotion or decreases in positive emotion (i.e., negative emotional contrasts; NEC) and use worry to avoid NEC. Sensitivity to and avoidance of NEC could also be a shared feature of major depressive disorder (MDD) and social anxiety disorder (SAD).MethodsIn a large college sample (N = 1409), we used receiver operating characteristics analysis to examine the accuracy of a measure of emotional contrast avoidance in detecting probable GAD, MDD, and SAD.ResultsParticipants with probable GAD, MDD, and SAD all reported higher levels of contrast avoidance than participants without the disorder (Cohen’s d = 1.32, 1.62 and 1.53, respectively). Area under the curve, a measure of predictive accuracy, was .81, .87, and .83 for predicting probable GAD, MDD, and SAD, respectively. A cutoff score of 48.5 optimized predictive accuracy for probable GAD and SAD, and 50.5 optimized accuracy for probable MDD.ConclusionA measure of emotional contrast avoidance demonstrated excellent ability to predict probable GAD, MDD, and SAD. Sensitivity to and avoidance of NEC appears to be a transdiagnostic feature of these disorders.     

Fast Image Recognition Based on Independent Component Analysis and Extreme Learning Machine

Nowadays, image recognition has become a highly active research topic in cognitive computation community, due to its many potential applications. Generally, the image recognition task involves two subtasks: image representation and image classification. Most feature extraction approaches for image representation developed so far regard independent component analysis (ICA) as one of the essential means. However, ICA has been hampered by its extremely expensive computational cost in real-time implementation. To address this problem, a fast cognitive computational scheme for image recognition is presented in this paper, which combines ICA and the extreme learning machine (ELM) algorithm. It tries to solve the image recognition problem at a much faster speed by using ELM not only in image classification but also in feature extraction for image representation. As an example, our proposed approach is applied to the face image recognition with detailed analysis. Firstly, common feature hypothesis is introduced to extract the common visual features from universal images by the traditional ICA model in the offline recognition process, and then ELM is used to simulate ICA for the purpose of facial feature extraction in the online recognition process. Lastly, the resulting independent feature representation of the face images extracted by ELM rather than ICA will be fed into the ELM classifier, which is composed of numerous single hidden layer feed-forward networks. Experimental results on Yale face database and MNIST digit database have shown the good performance of our proposed approach, which could be comparable to the state-of-the-art techniques at a much faster speed.     

Complex independent component analysis of frequency-domain electroencephalographic data.

Independent component analysis (ICA) has proven useful for modeling brain and electroencephalographic (EEG) data. Here, we present a new, generalized method to better capture the dynamics of brain signals than previous ICA algorithms. We regard EEG sources as eliciting spatio-temporal activity patterns, corresponding to, e.g. trajectories of activation propagating across cortex. This leads to a model of convolutive signal superposition, in contrast with the commonly used instantaneous mixing model. In the frequency-domain, convolutive mixing is equivalent to multiplicative mixing of complex signal sources within distinct spectral bands. We decompose the recorded spectral-domain signals into independent components by a complex infomax ICA algorithm. First results from a visual attention EEG experiment exhibit: (1). sources of spatio-temporal dynamics in the data, (2). links to subject behavior, (3). sources with a limited spectral extent, and (4). a higher degree of independence compared to sources derived by standard ICA.     

Independent component analysis of DTI reveals multivariate microstructural correlations of white matter in the human brain

It has recently been demonstrated that specific patterns of correlation exist in diffusion tensor imaging (DTI) parameters across white matter tracts in the normal human brain. These microstructural correlations are thought to reflect phylogenetic and functional similarities between different axonal fiber pathways. However, this earlier work was limited in three major respects: (1) the analysis was restricted to only a dozen selected tracts; (2) the DTI measurements were averaged across whole tracts, whereas metrics such as fractional anisotropy (FA) are known to vary considerably within single tracts; and (3) a univariate measure of correlation was used. In this investigation, we perform an automated multivariate whole-brain voxel-based study of white matter FA correlations using independent component analysis (ICA) of tract-based spatial statistics computed from 3T DTI in 53 healthy adult volunteers. The resulting spatial maps of the independent components show voxels for which the FA values within each map co-vary across individuals. The strongest FA correlations were found in anatomically recognizable tracts and tract segments, either singly or in homologous pairs. Hence, ICA of DTI provides an automated unsupervised decomposition of the normal human brain into multiple separable microstructurally correlated white matter regions, many of which correspond to anatomically familiar classes of white matter pathways. Further research is needed to determine whether whole-brain ICA of DTI represents a novel alternative to tractography for feature extraction in studying the normal microstructure of human white matter as well as the abnormal white matter microstructure found in neurological and psychiatric disorders.     

Incremental learning of feature space and classifier for face recognition.

We have proposed a new approach to pattern recognition in which not only a classifier but also a feature space of input variables is learned incrementally. In this paper, an extended version of Incremental Principal Component Analysis (IPCA) and Resource Allocating Network with Long-Term Memory (RAN-LTM) are effectively combined to implement this idea. Since IPCA updates a feature space incrementally by rotating the eigen-axes and increasing the dimensions, the inputs of a neural classifier must also change in their values and the number of input variables. To solve this problem, we derive an approximation of the update formula for memory items, which correspond to representative training samples stored in the long-term memory of RAN-LTM. With these memory items, RAN-LTM is efficiently reconstructed and retrained to adapt to the evolution of the feature space. This function is incorporated into our face recognition system. In the experiments, the proposed incremental learning model is evaluated over a self-compiled video clip of 24 persons. The experimental results show that the incremental learning of a feature space is very effective to enhance the generalization performance of a neural classifier in a realistic face recognition task.