Simultaneous Feature Selection and Support Vector Machine Optimization Using the Grasshopper Optimization Algorithm

Support vector machine (SVM) is considered to be one of the most powerful learning algorithms and is used for a wide range of real-world applications. The efficiency of SVM algorithm and its performance mainly depends on the kernel type and its parameters. Furthermore, the feature subset selection that is used to train the SVM model is another important factor that has a major influence on it classification accuracy. The feature subset selection is a very important step in machine learning, specially when dealing with high-dimensional data sets. Most of the previous researches handled these important factors separately. In this paper, we propose a hybrid approach based on the Grasshopper optimisation algorithm (GOA), which is a recent algorithm inspired by the biological behavior shown in swarms of grasshoppers. The goal of the proposed approach is to optimize the parameters of the SVM model, and locate the best features subset simultaneously. Eighteen low- and high-dimensional benchmark data sets are used to evaluate the accuracy of the proposed approach. For verification, the proposed approach is compared with seven well-regarded algorithms. Furthermore, the proposed approach is compared with grid search, which is the most popular technique for tuning SVM parameters. The experimental results show that the proposed approach outperforms all of the other techniques in most of the data sets in terms of classification accuracy, while minimizing the number of selected features.     

Optimal control by least squares support vector machines.

Support vector machines have been very successful in pattern recognition and function estimation problems. In this paper we introduce the use of least squares support vector machines (LS-SVM's) for the optimal control of nonlinear systems. Linear and neural full static state feedback controllers are considered. The problem is formulated in such a way that it incorporates the N-stage optimal control problem as well as a least squares support vector machine approach for mapping the state space into the action space. The solution is characterized by a set of nonlinear equations. An alternative formulation as a constrained nonlinear optimization problem in less unknowns is given, together with a method for imposing local stability in the LS-SVM control scheme. The results are discussed for support vector machines with radial basis function kernel. Advantages of LS-SVM control are that no number of hidden units has to be determined for the controller and that no centers have to be specified for the Gaussian kernels when applying Mercer's condition. The curse of dimensionality is avoided in comparison with defining a regular grid for the centers in classical radial basis function networks. This is at the expense of taking the trajectory of state variables as additional unknowns in the optimization problem, while classical neural network approaches typically lead to parametric optimization problems. In the SVM methodology the number of unknowns equals the number of training data, while in the primal space the number of unknowns can be infinite dimensional. The method is illustrated both on stabilization and tracking problems including examples on swinging up an inverted pendulum with local stabilization at the endpoint and a tracking problem for a ball and beam system.     

Bounds on the number of hidden units of boltzmann machines

It is known that any given probability distribution of the states of the observable units of a Boltzmann machine can be realized if no limit is imposed on the number of hidden units. But very little is known about the number of hidden units necessary for such realization. We consider Boltzmann machines as associative memories and show that there exist vector sets whose memorization on a Boltzmann machine requires a number of hidden units which is exponential in the size of the vectors (i.e., the number of components in each vector). Additional results give tight bounds on the number of hidden units needed in terms of the vector set size (i.e., the number of vectors in the set). Furthermore, we show how to construct Boltzmann machines which realize negation, intersection, and composition of the vector sets memorized by given Boltzmann machines.     

Transductive SVM for reducing the training effort in BCI

A brain-computer interface (BCI) provides a communication channel that translates human intention reflected by a brain signal such as electroencephalogram (EEG) into a control signal for an output device. In this work, the main concern is to reduce the training effort for BCI, which is often tedious and time consuming. Here we introduce a transductive support vector machines (TSVM) algorithm for the classification of EEG signals associated with mental tasks. TSVM possess the property of using both labeled and unlabeled data for reducing the calibration time in BCI and achieving good performance in classification accuracy. The advantages of the proposed method over the traditional supervised support vector machines (SVM) method are confirmed by about 2%-9% higher classification accuracies on a set of EEG recordings of three subjects from three-tasks-based mental imagery experiments.     

Asynchronous decision making in a memorized paddle pressing task

This paper presents a method for asynchronous decision making using recorded neural data in a binary decision task. This is a demonstration of a technique for developing motor cortical neural prosthetics that do not rely on external cued timing information. The system presented in this paper uses support vector machines and leaky integrate-and-fire elements to predict directional paddle presses. In addition to the traditional metrics of accuracy, asynchronous systems must also optimize the time needed to make a decision. The system presented is able to predict paddle presses with a median accuracy of 88% and all decisions are made before the time of the actual paddle press. An alternative bit rate measure of performance is defined to show that the system proposed here is able to perform the task with the same efficiency as the rats.     

模糊聚类支持向量机在步态分类中的应用

背景：对于患有神经系统或骨骼肌肉系统疾病的患者,分析步态数据可以评定康复程度,制定治疗方案。如何有效地分类小样本步态数据成为重要的研究课题。 目的：用改进的支持向量机算法对小样本步态数据进行分类,准确诊断疾病。 方法：建立加入模糊C均值聚类的支持向量机算法,选用Gait Dynamics in Neuro-Degenerative Disease Data Base 40~59岁年龄段的6组数据,共720个样本数据,采用左摆间隔和左支撑间隔两维参数对步态数据建模。数据归一化后,通过模糊C均值聚类对数据进行预处理;然后用支持向量机对数据进行分类。采用不同核函数的支持向量机算法验证分类能力。 结果与结论：实验结果表明,利用改进的支持向量机算法,可以有效地对信号进行分类,有助于疾病的诊断和治疗方案的制定。     

Sparse-Representation-Based Classification with Structure-Preserving Dimension Reduction

Sparse-representation-based classification (SRC), which classifies data based on the sparse reconstruction error, has been a new technique in pattern recognition. However, the computation cost for sparse coding is heavy in real applications. In this paper, various dimension reduction methods are studied in the context of SRC to improve classification accuracy as well as reduce computational cost. A feature extraction method, i.e., principal component analysis, and feature selection methods, i.e., Laplacian score and Pearson correlation coefficient, are applied to the data preparation step to preserve the structure of data in the lower-dimensional space. Classification performance of SRC with structure-preserving dimension reduction (SRC–SPDR) is compared to classical classifiers such as k-nearest neighbors and support vector machines. Experimental tests with the UCI and face data sets demonstrate that SRC–SPDR is effective with relatively low computation cost     

A neural support vector machine

Support vector machines are state-of-the-art pattern recognition algorithms that are well founded in optimization and generalization theory but not obviously applicable to the brain. This paper presents Bio-SVM, a biologically feasible support vector machine. An unstable associative memory oscillates between support vectors and interacts with a feed-forward classification pathway. Kernel neurons blend support vectors and sensory input. Downstream temporal integration generates the classification. Instant learning of surprising events and off-line tuning of support vector weights trains the system. Emotion-based learning, forgetting trivia, sleep and brain oscillations are phenomena that agree with the Bio-SVM model. A mapping to the olfactory system is suggested.     

A Machine Learning Approach to Detect Router Advertisement Flooding Attacks in Next-Generation IPv6 Networks

Router advertisement (RA) flooding attack aims to exhaust all node resources, such as CPU and memory, attached to routers on the same link. A biologically inspired machine learning-based approach is proposed in this study to detect RA flooding attacks. The proposed technique exploits information gain ratio (IGR) and principal component analysis (PCA) for feature selection and a support vector machine (SVM)-based predictor model, which can also detect input traffic anomaly. A real benchmark dataset obtained from National Advanced IPv6 Center of Excellence laboratory is used to evaluate the proposed technique. The evaluation process is conducted with two experiments. The first experiment investigates the effect of IGR and PCA feature selection methods to identify the most contributed features for the SVM training model. The second experiment evaluates the capability of SVM to detect RA flooding attacks. The results show that the proposed technique demonstrates excellent detection accuracy and is thus an effective choice for detecting RA flooding attacks. The main contribution of this study is identification of a set of new features that are related to RA flooding attack by utilizing IGR and PCA algorithms. The proposed technique in this paper can effectively detect the presence of RA flooding attack in IPv6 network.     

Abductive Learning Ensembles for Hand Shape Identification

A novel method is presented for hand shape identification based on abductive machine learning. We developed several models and investigated their performance on raw hand shape data for 20 and 40 participants in the form of three different categories of geometric measurements: twelve finger features, two palm features, and three whole hand features. Performance was compared when using each category of features separately and when combining them together. Moreover, we describe two novel and more effective approaches using an ensemble of three abductive networks combined at either the score level or the decision level. The effect of doubling the number of participants from 20 to 40 was studied as well. The ensemble approach achieved overall identification accuracies of 100 and 98.3333 % for the 20-participant and 40-participant datasets, respectively. This compares favorably with other learning approaches tried on the same datasets, including decision trees, support vector machines, and rule-based classifiers.     

Ependymoma and pilocytic astrocytoma: Differentiation using radiomics approach based on machine learning

Mandatory accurate and specific diagnosis demands have brought about increased challenges for radiologists in pediatric posterior fossa tumor prediction and prognosis. With the development of high-performance computing and machine learning technologies, radiomics provides increasing opportunities for clinical decision-making. Several studies have applied radiomics as a decision support tool in intracranial tumors differentiation. Here we seek to achieve preoperative differentiation between ependymoma (EP) and pilocytic astrocytoma (PA) using radiomics analysis method based on machine learning. A total of 135 Magnetic Resonance Imaging (MRI) slices are divided into training sets and validation sets. Three kinds of radiomics features, including Gabor transform, texture and wavelet transform based ones are used to obtain 300 multimodal features. Kruskal–Wallis test score (KWT) and support vector machines (SVM) are applied for feature selection and tumor differentiation. The performance is investigated via accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) analysis. Results show that the accuracy, sensitivity, specificity, and AUC of the selected feature set are 0.8775, 0.9292, 0.8000, and 0.8646 respectively, having no significant differences compared with the overall feature set. For different types of features, texture features yield the best differentiation performance and the significance analysis results are consistent with this. Our study demonstrates texture features perform better than the other features. The radiomics approach based on machine learning is efficient for pediatric posterior fossa tumors differentiation and could enhance the application of radiomics methods for assisted clinical diagnosis.     

Optimizing Echo State Networks for Static Pattern Recognition

Static pattern recognition requires a machine to classify an object on the basis of a combination of attributes and is typically performed using machine learning techniques such as support vector machines and multilayer perceptrons. Unusually, in this study, we applied a successful time-series processing neural network architecture, the echo state network (ESN), to a static pattern recognition task. The networks were presented with clamped input data patterns, but in this work, they were allowed to run until their output units delivered a stable set of output activations, in a similar fashion to previous work that focused on the behaviour of ESN reservoir units. Our aim was to see if the short-term memory developed by the reservoir and the clamped inputs could deliver improved overall classification accuracy. The study utilized a challenging, high dimensional, real-world plant species spectroradiometry classification dataset with the objective of accurately detecting one of the world’s top 100 invasive plant species. Surprisingly, the ESNs performed equally well with both unsettled and settled reservoirs. Delivering a classification accuracy of 96.60%, the clamped ESNs outperformed three widely used machine learning techniques, namely support vector machines, extreme learning machines and multilayer perceptrons. Contrary to past work, where inputs were clamped until reservoir stabilization, it was found that it was possible to obtain similar classification accuracy (96.49%) by clamping the input patterns for just two repeats. The chief contribution of this work is that a recurrent architecture can get good classification accuracy, even while the reservoir is still in an unstable state.     

Support vector classifiers via gradient systems with discontinuous righthand sides

Gradient dynamical systems with discontinuous righthand sides are designed using Persidskii-type nonsmooth Lyapunov functions to work as support vector machines (SVMs) for the discrimination of nonseparable classes. The gradient systems are obtained from an exact penalty method applied to the constrained quadratic optimization problems, which are formulations of two well known SVMs. Global convergence of the trajectories of the gradient dynamical systems to the solution of the corresponding constrained problems is shown to be independent of the penalty parameters and of the parameters of the SVMs. The proposed gradient systems can be implemented as simple analog circuits as well as using standard software for integration of ODEs, and in order to use efficient integration methods with adaptive stepsize selection, the discontinuous terms are smoothed around a neighborhood of the discontinuity surface by means of the boundary layer technique. The scalability of the proposed gradient systems is also shown by means of an implementation using parallel computers, resulting in smaller processing times when compared with traditional SVM packages.     

Comparing error minimized extreme learning machines and support vector sequential feed-forward neural networks

Recently, error minimized extreme learning machines (EM-ELMs) have been proposed as a simple and efficient approach to build single-hidden-layer feed-forward networks (SLFNs) sequentially. They add random hidden nodes one by one (or group by group) and update the output weights incrementally to minimize the sum-of-squares error in the training set. Other very similar methods that also construct SLFNs sequentially had been reported earlier with the main difference that their hidden-layer weights are a subset of the data instead of being random. These approaches are referred to as support vector sequential feed-forward neural networks (SV-SFNNs), and they are a particular case of the sequential approximation with optimal coefficients and interacting frequencies (SAOCIF) method. In this paper, it is firstly shown that EM-ELMs can also be cast as a particular case of SAOCIF. In particular, EM-ELMs can easily be extended to test some number of random candidates at each step and select the best of them, as SAOCIF does. Moreover, it is demonstrated that the cost of the computation of the optimal output-layer weights in the originally proposed EM-ELMs can be improved if it is replaced by the one included in SAOCIF. Secondly, we present the results of an experimental study on 10 benchmark classification and 10 benchmark regression data sets, comparing EM-ELMs and SV-SFNNs, that was carried out under the same conditions for the two models. Although both models have the same (efficient) computational cost, a statistically significant improvement in generalization performance of SV-SFNNs vs. EM-ELMs was found in 12 out of the 20 benchmark problems.     

Phenotypic features of 1q41q42 microdeletion including WDR26 and FBXO28 are clinically recognizable: The first case from Japan

1q41q42 microdeletion syndrome has been established in 2007. Since then, more than 17 patients have been reported so far. The reported deletions showed random breakpoints and deletion regions are aligned as roof tiles. Patients with 1q41q42 microdeletion syndrome show intellectual disability, seizures, and distinctive features. Many genotype-phenotype correlation studies have been performed and some genes included in this region have been suggested as potential candidate genes. Recently, de novo variants in WDR26 and FBXO28 were identified in patients who showed consistent phenotypes with 1q41q42 microdeletion syndrome. Thus, both genes are now considered as the genes possibly responsible for 1q41q42 microdeletion syndrome. Here, the first case of a Japanese patient with a de novo 1q41q42 microdeletion is reported. Owing to the distinctive features, this syndrome would be clinically recognizable.     

An Insight into Extreme Learning Machines: Random Neurons,Random Features and Kernels

Extreme learning machines (ELMs) basically give answers to two fundamental learning problems: (1) Can fundamentals of learning (i.e., feature learning, clustering, regression and classification) be made without tuning hidden neurons (including biological neurons) even when the output shapes and function modeling of these neurons are unknown? (2) Does there exist unified framework for feedforward neural networks and feature space methods? ELMs that have built some tangible links between machine learning techniques and biological learning mechanisms have recently attracted increasing attention of researchers in widespread research areas. This paper provides an insight into ELMs in three aspects, viz: random neurons, random features and kernels. This paper also shows that in theory ELMs (with the same kernels) tend to outperform support vector machine and its variants in both regression and classification applications with much easier implementation.     

Modelling ordinal relations with SVMs: An application to objective aesthetic evaluation of breast cancer conservative treatment.

The cosmetic result is an important endpoint for breast cancer conservative treatment (BCCT), but the verification of this outcome remains without a standard. Objective assessment methods are preferred to overcome the drawbacks of subjective evaluation. In this paper a novel algorithm is proposed, based on support vector machines, for the classification of ordinal categorical data. This classifier is then applied as a new methodology for the objective assessment of the aesthetic result of BCCT. Based on the new classifier, a semi-objective score for quantification of the aesthetic results of BCCT was developed, allowing the discrimination of patients into four classes.