Neural architecture design based on extreme learning machine

Selection of the optimal neural architecture to solve a pattern classification problem entails to choose the relevant input units, the number of hidden neurons and its corresponding interconnection weights. This problem has been widely studied in many research works but their solutions usually involve excessive computational cost in most of the problems and they do not provide a unique solution. This paper proposes a new technique to efficiently design the MultiLayer Perceptron (MLP) architecture for classification using the Extreme Learning Machine (ELM) algorithm. The proposed method provides a high generalization capability and a unique solution for the architecture design. Moreover, the selected final network only retains those input connections that are relevant for the classification task. Experimental results show these advantages.     

Identification of Alzheimer's disease and mild cognitive impairment using multimodal sparse hierarchical extreme learning machine

Different modalities such as structural MRI, FDG‐PET, and CSF have complementary information, which is likely to be very useful for diagnosis of AD and MCI. Therefore, it is possible to develop a more effective and accurate AD/MCI automatic diagnosis method by integrating complementary information of different modalities. In this paper, we propose multi‐modal sparse hierarchical extreme leaning machine (MSH‐ELM). We used volume and mean intensity extracted from 93 regions of interest (ROIs) as features of MRI and FDG‐PET, respectively, and used p‐tau, t‐tau, and as CSF features. In detail, high‐level representation was individually extracted from each of MRI, FDG‐PET, and CSF using a stacked sparse extreme learning machine auto‐encoder (sELM‐AE). Then, another stacked sELM‐AE was devised to acquire a joint hierarchical feature representation by fusing the high‐level representations obtained from each modality. Finally, we classified joint hierarchical feature representation using a kernel‐based extreme learning machine (KELM). The results of MSH‐ELM were compared with those of conventional ELM, single kernel support vector machine (SK‐SVM), multiple kernel support vector machine (MK‐SVM) and stacked auto‐encoder (SAE). Performance was evaluated through 10‐fold cross‐validation. In the classification of AD vs. HC and MCI vs. HC problem, the proposed MSH‐ELM method showed mean balanced accuracies of 96.10% and 86.46%, respectively, which is much better than those of competing methods. In summary, the proposed algorithm exhibits consistently better performance than SK‐SVM, ELM, MK‐SVM and SAE in the two binary classification problems (AD vs. HC and MCI vs. HC).     

Using financial risk measures for analyzing generalization performance of machine learning models

We propose a unified machine learning model (UMLM) for two-class classification, regression and outlier (or novelty) detection via a robust optimization approach. The model embraces various machine learning models such as support vector machine-based and minimax probability machine-based classification and regression models. The unified framework makes it possible to compare and contrast existing learning models and to explain their differences and similarities.In this paper, after relating existing learning models to UMLM, we show some theoretical properties for UMLM. Concretely, we show an interpretation of UMLM as minimizing a well-known financial risk measure (worst-case value-at risk (VaR) or conditional VaR), derive generalization bounds for UMLM using such a risk measure, and prove that solving problems of UMLM leads to estimators with the minimized generalization bounds. Those theoretical properties are applicable to related existing learning models.     

Extreme learning machine for ranking: Generalization analysis and applications

The extreme learning machine (ELM) has attracted increasing attention recently with its successful applications in classification and regression. In this paper, we investigate the generalization performance of ELM-based ranking. A new regularized ranking algorithm is proposed based on the combinations of activation functions in ELM. The generalization analysis is established for the ELM-based ranking (ELMRank) in terms of the covering numbers of hypothesis space. Empirical results on the benchmark datasets show the competitive performance of the ELMRank over the state-of-the-art ranking methods.     

A new robust model of one-class classification by interval-valued training data using the triangular kernel

A robust one-class classification model as an extension of Campbell and Bennett’s (C–B) novelty detection model on the case of interval-valued training data is proposed in the paper. It is shown that the dual optimization problem to a linear program in the C–B model has a nice property allowing to represent it as a set of simple linear programs. It is proposed also to replace the Gaussian kernel in the obtained linear support vector machines by the well-known triangular kernel which can be regarded as an approximation of the Gaussian kernel. This replacement allows us to get a finite set of simple linear optimization problems for dealing with interval-valued data. Numerical experiments with synthetic and real data illustrate performance of the proposed model.     

The impact of machine learning techniques in the study of bipolar disorder: A systematic review

Machine learning techniques provide new methods to predict diagnosis and clinical outcomes at an individual level. We aim to review the existing literature on the use of machine learning techniques in the assessment of subjects with bipolar disorder. We systematically searched PubMed, Embase and Web of Science for articles published in any language up to January 2017. We found 757 abstracts and included 51 studies in our review. Most of the included studies used multiple levels of biological data to distinguish the diagnosis of bipolar disorder from other psychiatric disorders or healthy controls. We also found studies that assessed the prediction of clinical outcomes and studies using unsupervised machine learning to build more consistent clinical phenotypes of bipolar disorder. We concluded that given the clinical heterogeneity of samples of patients with BD, machine learning techniques may provide clinicians and researchers with important insights in fields such as diagnosis, personalized treatment and prognosis orientation.     

Detection of physiological noise in resting state fMRI using machine learning

We present a technique for predicting cardiac and respiratory phase on a time point by time point basis, from fMRI image data. These predictions have utility in attempts to detrend effects of the physiological cycles from fMRI image data. We demonstrate the technique both in the case where it can be trained on a subject's own data, and when it cannot. The prediction scheme uses a multiclass support vector machine algorithm. Predictions are demonstrated to have a close fit to recorded physiological phase, with median Pearson correlation scores between recorded and predicted values of 0.99 for the best case scenario (cardiac cycle trained on a subject's own data) down to 0.83 for the worst case scenario (respiratory predictions trained on group data), as compared to random chance correlation score of 0.70. When predictions were used with RETROICOR—a popular physiological noise removal tool—the effects are compared to using recorded phase values. Using Fourier transforms and seed based correlation analysis, RETROICOR is shown to produce similar effects whether recorded physiological phase values are used, or they are predicted using this technique. This was seen by similar levels of noise reduction noise in the same regions of the Fourier spectra, and changes in seed based correlation scores in similar regions of the brain. This technique has a use in situations where data from direct monitoring of the cardiac and respiratory cycles are incomplete or absent, but researchers still wish to reduce this source of noise in the image data. Hum Brain Mapp , 2013. © 2011 Wiley Periodicals, Inc.     

Brain age prediction: A comparison between machine learning models using region‐ and voxel‐based morphometric data

Brain morphology varies across the ageing trajectory and the prediction of a person''s age using brain features can aid the detection of abnormalities in the ageing process. Existing studies on such “brain age prediction” vary widely in terms of their methods and type of data, so at present the most accurate and generalisable methodological approach is unclear. Therefore, we used the UK Biobank data set (N = 10,824, age range 47–73) to compare the performance of the machine learning models support vector regression, relevance vector regression and Gaussian process regression on whole‐brain region‐based or voxel‐based structural magnetic resonance imaging data with or without dimensionality reduction through principal component analysis. Performance was assessed in the validation set through cross‐validation as well as an independent test set. The models achieved mean absolute errors between 3.7 and 4.7 years, with those trained on voxel‐level data with principal component analysis performing best. Overall, we observed little difference in performance between models trained on the same data type, indicating that the type of input data had greater impact on performance than model choice. All code is provided online in the hope that this will aid future research.     

Design of a multiple kernel learning algorithm for LS-SVM by convex programming

As a kernel based method, the performance of least squares support vector machine (LS-SVM) depends on the selection of the kernel as well as the regularization parameter (Duan, Keerthi, & Poo, 2003). Cross-validation is efficient in selecting a single kernel and the regularization parameter; however, it suffers from heavy computational cost and is not flexible to deal with multiple kernels. In this paper, we address the issue of multiple kernel learning for LS-SVM by formulating it as semidefinite programming (SDP). Furthermore, we show that the regularization parameter can be optimized in a unified framework with the kernel, which leads to an automatic process for model selection. Extensive experimental validations are performed and analyzed.     

Prediction of pediatric unipolar depression using multiple neuromorphometric measurements: A pattern classification approach

BackgroundDiagnosis of pediatric neuropsychiatric disorders such as unipolar depression is largely based on clinical judgment – without objective biomarkers to guide diagnostic process and subsequent therapeutic interventions. Neuroimaging studies have previously reported average group-level neuroanatomical differences between patients with pediatric unipolar depression and healthy controls. In the present study, we investigated the utility of multiple neuromorphometric indices in distinguishing pediatric unipolar depression patients from healthy controls at an individual subject level.MethodsWe acquired structural T₁-weighted scans from 25 pediatric unipolar depression patients and 26 demographically matched healthy controls. Multiple neuromorphometric indices such as cortical thickness, volume, and cortical folding patterns were obtained. A support vector machine pattern classification model was ‘trained’ to distinguish individual subjects with pediatric unipolar depression from healthy controls based on multiple neuromorphometric indices and model predictive validity (sensitivity and specificity) calculated.ResultsThe model correctly identified 40 out of 51 subjects translating to 78.4% accuracy, 76.0% sensitivity and 80.8% specificity, chi-square p-value = 0.000049. Volumetric and cortical folding abnormalities in the right thalamus and right temporal pole respectively were most central in distinguishing individual patients with pediatric unipolar depression from healthy controls.ConclusionsThese findings provide evidence that a support vector machine pattern classification model using multiple neuromorphometric indices may qualify as diagnostic marker for pediatric unipolar depression. In addition, our results identified the most relevant neuromorphometric features in distinguishing PUD patients from healthy controls.     

Identifying the neurophysiological effects of memory-enhancing amygdala stimulation using interpretable machine learning

BackgroundDirect electrical stimulation of the amygdala can enhance declarative memory for specific events. An unanswered question is what underlying neurophysiological changes are induced by amygdala stimulation.ObjectiveTo leverage interpretable machine learning to identify the neurophysiological processes underlying amygdala-mediated memory, and to develop more efficient neuromodulation technologies.MethodPatients with treatment-resistant epilepsy and depth electrodes placed in the hippocampus and amygdala performed a recognition memory task for neutral images of objects. During the encoding phase, 160 images were shown to patients. Half of the images were followed by brief low-amplitude amygdala stimulation. For local field potentials (LFPs) recorded from key medial temporal lobe structures, feature vectors were calculated by taking the average spectral power in canonical frequency bands, before and after stimulation, to train a logistic regression classification model with elastic net regularization to differentiate brain states.ResultsClassifying the neural states at the time of encoding based on images subsequently remembered versus not-remembered showed that theta and slow-gamma power in the hippocampus were the most important features predicting subsequent memory performance. Classifying the post-image neural states at the time of encoding based on stimulated versus unstimulated trials showed that amygdala stimulation led to increased gamma power in the hippocampus.ConclusionAmygdala stimulation induced pro-memory states in the hippocampus to enhance subsequent memory performance. Interpretable machine learning provides an effective tool for investigating the neurophysiological effects of brain stimulation.     

Training Lp norm multiple kernel learning in the primal

Some multiple kernel learning (MKL) models are usually solved by utilizing the alternating optimization method where one alternately solves SVMs in the dual and updates kernel weights. Since the dual and primal optimization can achieve the same aim, it is valuable in exploring how to perform Lp norm MKL in the primal. In this paper, we propose an Lp norm multiple kernel learning algorithm in the primal where we resort to the alternating optimization method: one cycle for solving SVMs in the primal by using the preconditioned conjugate gradient method and other cycle for learning the kernel weights. It is interesting to note that the kernel weights in our method can obtain analytical solutions. Most importantly, the proposed method is well suited for the manifold regularization framework in the primal since solving LapSVMs in the primal is much more effective than solving LapSVMs in the dual. In addition, we also carry out theoretical analysis for multiple kernel learning in the primal in terms of the empirical Rademacher complexity. It is found that optimizing the empirical Rademacher complexity may obtain a type of kernel weights. The experiments on some datasets are carried out to demonstrate the feasibility and effectiveness of the proposed method.     

Distributed semi-supervised support vector machines

The semi-supervised support vector machine (S${}^3$VM) is a well-known algorithm for performing semi-supervised inference under the large margin principle. In this paper, we are interested in the problem of training a S${}^3$VM when the labeled and unlabeled samples are distributed over a network of interconnected agents. In particular, the aim is to design a distributed training protocol over networks, where communication is restricted only to neighboring agents and no coordinating authority is present. Using a standard relaxation of the original S${}^3$VM, we formulate the training problem as the distributed minimization of a non-convex social cost function. To find a (stationary) solution in a distributed manner, we employ two different strategies: (i) a distributed gradient descent algorithm; (ii) a recently developed framework for In-Network Nonconvex Optimization (NEXT), which is based on successive convexifications of the original problem, interleaved by state diffusion steps. Our experimental results show that the proposed distributed algorithms have comparable performance with respect to a centralized implementation, while highlighting the pros and cons of the proposed solutions. To the date, this is the first work that paves the way toward the broad field of distributed semi-supervised learning over networks.     

Improving subspace learning for facial expression recognition using person dependent and geometrically enriched training sets

In this paper, the robustness of appearance-based subspace learning techniques in geometrical transformations of the images is explored. A number of such techniques are presented and tested using four facial expression databases. A strong correlation between the recognition accuracy and the image registration error has been observed. Although it is common-knowledge that appearance-based methods are sensitive to image registration errors, there is no systematic experiment reported in the literature. As a result of these experiments, the training set enrichment with translated, scaled and rotated images is proposed for confronting the low robustness of these techniques in facial expression recognition. Moreover, person dependent training is proven to be much more accurate for facial expression recognition than generic learning.     

Distributed representation of tone frequency in highly decodable spatio-temporal activity in the auditory cortex

Although the place code of tone frequency, or tonotopic map, has been widely accepted in the auditory cortex, tone-evoked activation becomes less frequency-specific at moderate or high sound pressure levels. This implies that sound frequency is not represented by a simple place code but that the information is distributed spatio-temporally irrespective of the focal activation. In this study, using a decoding-based analysis, we investigated multi-unit activities in the auditory cortices of anesthetized rats to elucidate how a tone frequency is represented in the spatio-temporal neural pattern. We attempted sequential dimensionality reduction (SDR), a specific implementation of recursive feature elimination (RFE) with support vector machine (SVM), to identify the optimal spatio-temporal window patterns for decoding test frequency. SDR selected approximately a quarter of the windows, and SDR-identified window patterns led to significantly better decoding than spatial patterns, in which temporal structures were eliminated, or high-spike-rate patterns, in which windows with high spike rates were selectively extracted. Thus, the test frequency is also encoded in temporal as well as spatial structures of neural activities and low-spike-rate windows. Yet, SDR recruited more high-spike-rate windows than low-spike-rate windows, resulting in a highly dispersive pattern that probably offers an advantage of discrimination ability. Further investigation of SVM weights suggested that low-spike-rate windows play significant roles in fine frequency differentiation. These findings support the hypothesis that the auditory cortex adopts a distributed code in tone frequency representation, in which high- and low-spike-rate activities play mutually complementary roles.