Ant Colony Optimization Based Feature Selection Method for QEEG Data Classification

Objective

Many applications such as biomedical signals require selecting a subset of the input features in order to represent the whole set of features. A feature selection algorithm has recently been proposed as a new approach for feature subset selection.

Methods

Feature selection process using ant colony optimization (ACO) for 6 channel pre-treatment electroencephalogram (EEG) data from theta and delta frequency bands is combined with back propagation neural network (BPNN) classification method for 147 major depressive disorder (MDD) subjects.

Results

BPNN classified R subjects with 91.83% overall accuracy and 95.55% subjects detection sensitivity. Area under ROC curve (AUC) value after feature selection increased from 0.8531 to 0.911. The features selected by the optimization algorithm were Fp1, Fp2, F7, F8, F3 for theta frequency band and eliminated 7 features from 12 to 5 feature subset.

Conclusion

ACO feature selection algorithm improves the classification accuracy of BPNN. Using other feature selection algorithms or classifiers to compare the performance for each approach is important to underline the validity and versatility of the designed combination.     

基于遗传算法与支持向量机的基因微阵列分析

背景：微阵列数据的特点是样本含量小，而变量数(基因)多达上万个。此时，传统的统计方法往往因为高维而失效了。遗传算法和支持向量机是近年来发展迅速的机器学习算法，具有很好的分类效果与降维优势。 目的：提出将遗传算法与支持向量机结合起来对样本进行分类，并与直接采用支持向量机、筛选差异表达基因后采用支持向量机的结果进行比较。 方法：采用Bioconductor提供的数据集golub,它是白血病微阵列芯片实验所得的基因表达数据集，对全部基因采用支持向量机进行分类。采用SAM软件对芯片数据的显著性分析确定不同的差异表达基因并估计错误发现率FDR，以筛选出的76个差异表达基因作为特征基因子集，再采用支持向量机进行分类。将筛选出的76个差异表达基因作为初始的特征基因集合，采用遗传算法－支持向量机再次进行特征基因选择，提高分类准确度，并与全部基因直接采用支持向量机、筛选差异表达基因后采用支持向量机的结果进行比较。同时也对特征基因在代谢通路上的分布和功能作了一定的研究。 结果与结论：通过遗传算法降维可以提高支持向量机的分类准确率，特别是剔除了数据中的大量无关基因和噪声，使得经过特征选择后分类准确率提高。结果显示遗传算法与支持向量机结合方法对分类更加有效。此外，通路分析结果显示特征基因的主要功能体现在信号传导和氨基酸代谢上。     

Rank-Adaptive Non-Negative Matrix Factorization

Dimension reduction is a challenge task in data processing, especially in high-dimensional data processing area. Non-negative matrix factorization (NMF), as a classical dimension reduction method, has a contribution to the parts-based representation for the characteristics of non-negative constraints in the NMF algorithm. In this paper, the NMF algorithm is introduced to extract local features for dimension reduction. Considering the problem of which NMF is required to define the number of the decomposition rank manually, we proposed a rank-adaptive NMF algorithm, in which the affinity propagation (AP) clustering algorithm is introduced to determine adaptively the number of the decomposition rank of NMF. Then, the rank-adaptive NMF algorithm is used to extract features for the original image. After that, a low-dimensional representation of the original image is obtained through the projection from the original images to the feature space. Finally, we used extreme learning machine (ELM) and k-nearest neighbor (KNN) as the classifier to classify those low-dimensional feature representations. The experimental results demonstrate that the decomposition rank determined by the AP clustering algorithm can reflect the characteristics of the original data. When it is combined with the classification algorithm ELM or KNN and applied to handwritten character recognition, the proposed method not only reduces the dimension of original images but also performs well in terms of classification accuracy and time consumption. A new rank-adaptive NMF algorithm is proposed based on the AP clustering algorithm and the original NMF algorithm. According to this algorithm, the low-dimensional representation of the original data can be obtained without any prior knowledge. In addition, the proposed rank-adaptive NMF algorithm combined with the ELM and KNN classification algorithms performs well.     

A Machine Learning Approach to Detect Router Advertisement Flooding Attacks in Next-Generation IPv6 Networks

Router advertisement (RA) flooding attack aims to exhaust all node resources, such as CPU and memory, attached to routers on the same link. A biologically inspired machine learning-based approach is proposed in this study to detect RA flooding attacks. The proposed technique exploits information gain ratio (IGR) and principal component analysis (PCA) for feature selection and a support vector machine (SVM)-based predictor model, which can also detect input traffic anomaly. A real benchmark dataset obtained from National Advanced IPv6 Center of Excellence laboratory is used to evaluate the proposed technique. The evaluation process is conducted with two experiments. The first experiment investigates the effect of IGR and PCA feature selection methods to identify the most contributed features for the SVM training model. The second experiment evaluates the capability of SVM to detect RA flooding attacks. The results show that the proposed technique demonstrates excellent detection accuracy and is thus an effective choice for detecting RA flooding attacks. The main contribution of this study is identification of a set of new features that are related to RA flooding attack by utilizing IGR and PCA algorithms. The proposed technique in this paper can effectively detect the presence of RA flooding attack in IPv6 network.     

Budget constrained non-monotonic feature selection

Feature selection is an important problem in machine learning and data mining. We consider the problem of selecting features under the budget constraint on the feature subset size. Traditional feature selection methods suffer from the “monotonic” property. That is, if a feature is selected when the number of specified features is set, it will always be chosen when the number of specified feature is larger than the previous setting. This sacrifices the effectiveness of the non-monotonic feature selection methods. Hence, in this paper, we develop an algorithm for non-monotonic feature selection that approximates the related combinatorial optimization problem by a Multiple Kernel Learning (MKL) problem. We justify the performance guarantee for the derived solution when compared to the global optimal solution for the related combinatorial optimization problem. Finally, we conduct a series of empirical evaluation on both synthetic and real-world benchmark datasets for the classification and regression tasks to demonstrate the promising performance of the proposed framework compared with the baseline feature selection approaches.     

Automating Ischemic Stroke Subtype Classification Using Machine Learning and Natural Language Processing

Objective: The manual adjudication of disease classification is time-consuming, error-prone, and limits scaling to large datasets. In ischemic stroke (IS), subtype classification is critical for management and outcome prediction. This study sought to use natural language processing of electronic health records (EHR) combined with machine learning methods to automate IS subtyping. Methods: Among IS patients from an observational registry with TOAST subtyping adjudicated by board-certified vascular neurologists, we analyzed unstructured text-based EHR data including neurology progress notes and neuroradiology reports using natural language processing. We performed several feature selection methods to reduce the high dimensionality of the features and 5-fold cross validation to test generalizability of our methods and minimize overfitting. We used several machine learning methods and calculated the kappa values for agreement between each machine learning approach to manual adjudication. We then performed a blinded testing of the best algorithm against a held-out subset of 50 cases. Results: Compared to manual classification, the best machine-based classification achieved a kappa of .25 using radiology reports alone, .57 using progress notes alone, and .57 using combined data. Kappa values varied by subtype being highest for cardioembolic (.64) and lowest for cryptogenic cases (.47). In the held-out test subset, machine-based classification agreed with rater classification in 40 of 50 cases (kappa .72). Conclusions: Automated machine learning approaches using textual data from the EHR shows agreement with manual TOAST classification. The automated pipeline, if externally validated, could enable large-scale stroke epidemiology research.     

Support vector machine learning‐based cerebral blood flow quantification for arterial spin labeling MRI

Purpose : To develop a multivariate machine learning classification‐based cerebral blood flow (CBF) quantification method for arterial spin labeling (ASL) perfusion MRI. Methods : The label and control images of ASL MRI were separated using a machine‐learning algorithm, the support vector machine (SVM). The perfusion‐weighted image was subsequently extracted from the multivariate (all voxels) SVM classifier. Using the same pre‐processing steps, the proposed method was compared with standard ASL CBF quantification method using synthetic data and in‐vivo ASL images. Results : As compared with the conventional univariate approach, the proposed ASL CBF quantification method significantly improved spatial signal‐to‐noise‐ratio (SNR) and image appearance of ASL CBF images. Conclusion : the multivariate machine learning‐based classification is useful for ASL CBF quantification. Hum Brain Mapp 35:2869–2875, 2014. © 2013 Wiley Periodicals, Inc.     

Clustering-Based Extraction of Near Border Data Samples for Remote Sensing Image Classification

The definition of valuable training samples and automatic classification of land cover with remote sensing data are both classical problems, which are known to be difficult and have attracted major research efforts. In this paper, a method of modified K-means-based support vector machine (SVM) classification is proposed to use a hybrid sample selection that leverages the informativeness and representativeness of training samples to classify real multi/hyperspectral images. The hybrid sample selection (close-to-cluster-border sampling and near-cluster-center sampling) is constructed on the reduced convex hulls (RCHs) of clustering structure and can reduce the risk of overtraining caused by active sample selection of active learning methods. Numerical results obtained on the classification of three challenging remote sensing images (Landsat-7 ETM+, AVIRIS Indian pines, and KSC) by comparing the proposed technique with random sampling (RS) and margin sampling (MS) demonstrate the good efficiency and high accuracy of our approach.     

Decoding and mapping task states of the human brain via deep learning

Support vector machine (SVM)‐based multivariate pattern analysis (MVPA) has delivered promising performance in decoding specific task states based on functional magnetic resonance imaging (fMRI) of the human brain. Conventionally, the SVM‐MVPA requires careful feature selection/extraction according to expert knowledge. In this study, we propose a deep neural network (DNN) for directly decoding multiple brain task states from fMRI signals of the brain without any burden for feature handcrafts. We trained and tested the DNN classifier using task fMRI data from the Human Connectome Project's S1200 dataset (N = 1,034). In tests to verify its performance, the proposed classification method identified seven tasks with an average accuracy of 93.7%. We also showed the general applicability of the DNN for transfer learning to small datasets (N = 43), a situation encountered in typical neuroscience research. The proposed method achieved an average accuracy of 89.0 and 94.7% on a working memory task and a motor classification task, respectively, higher than the accuracy of 69.2 and 68.6% obtained by the SVM‐MVPA. A network visualization analysis showed that the DNN automatically detected features from areas of the brain related to each task. Without incurring the burden of handcrafting the features, the proposed deep decoding method can classify brain task states highly accurately, and is a powerful tool for fMRI researchers.     

Scalable learning method for feedforward neural networks using minimal-enclosing-ball approximation

Training feedforward neural networks (FNNs) is one of the most critical issues in FNNs studies. However, most FNNs training methods cannot be directly applied for very large datasets because they have high computational and space complexity. In order to tackle this problem, the CCMEB (Center-Constrained Minimum Enclosing Ball) problem in hidden feature space of FNN is discussed and a novel learning algorithm called HFSR-GCVM (hidden-feature-space regression using generalized core vector machine) is developed accordingly. In HFSR-GCVM, a novel learning criterion using L2-norm penalty-based $\epsilon$-insensitive function is formulated and the parameters in the hidden nodes are generated randomly independent of the training sets. Moreover, the learning of parameters in its output layer is proved equivalent to a special CCMEB problem in FNN hidden feature space. As most CCMEB approximation based machine learning algorithms, the proposed HFSR-GCVM training algorithm has the following merits: The maximal training time of the HFSR-GCVM training is linear with the size of training datasets and the maximal space consumption is independent of the size of training datasets. The experiments on regression tasks confirm the above conclusions.     

Enhanced active segment selection for single-trial EEG classification

In this study, an electroencephalogram (EEG) analysis system is proposed for single-trial classification of both motor imagery (MI) and finger-lifting EEG data. Applying event-related brain potential (ERP) data acquired from the sensorimotor cortices, the system mainly consists of three procedures; enhanced active segment selection, feature extraction, and classification. In addition to the original use of continuous wavelet transform (CWT) and Student 2-sample t statistics, the two-dimensional (2D) anisotropic Gaussian filter further refines the selection of active segments. The multiresolution fractal features are then extracted from wavelet data by using proposed modified fractal dimension. Finally, the support vector machine (SVM) is used for classification. Compared to original active segment selection, with several popular features and classifier on both the MI and finger-lifting data from 2 data sets, the results indicate that the proposed method is promising in EEG classification.     

A novel kernel-based maximum a posteriori classification method

Kernel methods have been widely used in pattern recognition. Many kernel classifiers such as Support Vector Machines (SVM) assume that data can be separated by a hyperplane in the kernel-induced feature space. These methods do not consider the data distribution and are difficult to output the probabilities or confidences for classification. This paper proposes a novel Kernel-based Maximum A Posteriori (KMAP) classification method, which makes a Gaussian distribution assumption instead of a linear separable assumption in the feature space. Robust methods are further proposed to estimate the probability densities, and the kernel trick is utilized to calculate our model. The model is theoretically and empirically important in the sense that: (1) it presents a more generalized classification model than other kernel-based algorithms, e.g., Kernel Fisher Discriminant Analysis (KFDA); (2) it can output probability or confidence for classification, therefore providing potential for reasoning under uncertainty; and (3) multi-way classification is as straightforward as binary classification in this model, because only probability calculation is involved and no one-against-one or one-against-others voting is needed. Moreover, we conduct an extensive experimental comparison with state-of-the-art classification methods, such as SVM and KFDA, on both eight UCI benchmark data sets and three face data sets. The results demonstrate that KMAP achieves very promising performance against other models.     

A Multi-Granularity Information-Based Method for Learning High-Dimensional Bayesian Network Structures

The purpose of structure learning is to construct a qualitative relationship of Bayesian networks. Bayesian network with interpretability and logicality is widely applied in a lot of fields. With the extensive development of high-dimensional and low sample size data in some applications, structure learning of Bayesian networks for high dimension and low sample size data becomes a challenging problem. To handle this problem, we propose a method for learning high-dimensional Bayesian network structures based on multi-granularity information. First, an undirected independence graph construction method containing global structure information is designed to optimize the search space of network structure. Then, an improved agglomerative hierarchical clustering method is presented to cluster variables into sub-granules, which reduces the complexity of structure learning by considering the variable community characteristic in high-dimensional data. Finally, the corresponding sub-graphs are formed by learning the internal structure of sub-granules, and the final network structure is constructed based on the proposed construct link graph algorithm. To verify the proposed method, we conduct two types of comparison experiments: comparison experiment and embedded comparison experiment. The results of the experiments show that our approach is superior to the competitors. The results indicate that our method can not only learn structures of Bayesian network from high-dimensional data efficiently but also improve the efficiency and accuracy of network structure generated by other algorithms for high-dimensional data.

     

Optimization of wavelets for classification of movement-related cortical potentials generated by variation of force-related parameters

The paper presents a novel pattern recognition approach for the classification of single-trial movement-related cortical potentials (MRCPs) generated by variations of force-related parameters during voluntary tasks. The feature space was built from the coefficients of a discrete dyadic wavelet transformation. Mother wavelet parameterization allowed the tuning of basis functions to project the signals. The mother wavelet was optimized to minimize the classification error estimated from the training set. Classification was performed with a support vector machine (SVM) approach with optimization of the width of a Gaussian kernel and of the regularization parameter. The efficacy of the optimization procedures was representatively shown on electroencephalographic recordings from two subjects who performed unilateral isometric plantar flexions at two target torques and two rates of torque development. The proposed classification method was tested on four pairs of classes corresponding to the change in only one of the two parameters of the task. Misclassification rate (test set) in the classification of 1-s EEG activity immediately before the onset of the tasks was reduced from 50.8+/-2.9% with worst wavelet and nearest representative classifier, to 40.2+/-7.3% with optimal wavelet and nearest representative classifier, and to 15.8+/-3.4% with optimal wavelet and SVM with optimization of the kernel and regularization parameter. The proposed pattern recognition method is promising for classification of MRCPs modulated by variations of force-related parameters.     

Single-subject classification of schizophrenia using event-related potentials obtained during auditory and visual oddball paradigms

In the search for the biomarkers of schizophrenia, event-related potential (ERP) deficits obtained by applying the classic oddball paradigm are among the most consistent findings. However, the single-subject classification rate based on these parameters remains to be determined. Here, we present a data-driven approach by applying machine learning classifiers to relevant oddball ERPs. Twenty-four schizophrenic patients and 24 matched healthy controls finished auditory and visual oddball tasks while high-density electrophysiological recordings were applied. The N1 component in response to standards and target as well as the P3 component following targets were submitted to different machine learning algorithms and the resulting ERP features were submitted to further correlation analyses. We obtained a classification accuracy of 72.4 % using only two ERP components. Latencies of parietal N1 components to visual standard stimuli at electrode positions Pz and P1 were sufficient for classification. Further analysis revealed a high correlation of these features in controls and an intermediate correlation in schizophrenia patients. These data exemplarily show how automated inference may be applied to classify a pathological state in single subjects without prior knowledge of their diagnoses and illustrate the potential of machine learning algorithms for the identification of potential biomarkers. Moreover, this approach assesses the discriminative accuracy of one of the most consistent findings in schizophrenia research by means of single-subject classification.     

Robust support vector machine-trained fuzzy system

Because the SVM (support vector machine) classifies data with the widest symmetric margin to decrease the probability of the test error, modern fuzzy systems use SVM to tune the parameters of fuzzy if–then rules. But, solving the SVM model is time-consuming. To overcome this disadvantage, we propose a rapid method to solve the robust SVM model and use it to tune the parameters of fuzzy if–then rules. The robust SVM is an extension of SVM for interval-valued data classification.We compare our proposed method with SVM, robust SVM, ISVM-FC (incremental support vector machine-trained fuzzy classifier), BSVM-FC (batch support vector machine-trained fuzzy classifier), SOTFN-SV (a self-organizing TS-type fuzzy network with support vector learning) and SCLSE (a TS-type fuzzy system with subtractive clustering for antecedent parameter tuning and LSE for consequent parameter tuning) by using some real datasets. According to experimental results, the use of proposed approach leads to very low training and testing time with good misclassification rate.     

Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG

ObjectiveTo explore effective combinations of computational methods for the prediction of movement intention preceding the production of self-paced right and left hand movements from single trial scalp electroencephalogram (EEG).MethodsTwelve naïve subjects performed self-paced movements consisting of three key strokes with either hand. EEG was recorded from 128 channels. The exploration was performed offline on single trial EEG data. We proposed that a successful computational procedure for classification would consist of spatial filtering, temporal filtering, feature selection, and pattern classification. A systematic investigation was performed with combinations of spatial filtering using principal component analysis (PCA), independent component analysis (ICA), common spatial patterns analysis (CSP), and surface Laplacian derivation (SLD); temporal filtering using power spectral density estimation (PSD) and discrete wavelet transform (DWT); pattern classification using linear Mahalanobis distance classifier (LMD), quadratic Mahalanobis distance classifier (QMD), Bayesian classifier (BSC), multi-layer perceptron neural network (MLP), probabilistic neural network (PNN), and support vector machine (SVM). A robust multivariate feature selection strategy using a genetic algorithm was employed.ResultsThe combinations of spatial filtering using ICA and SLD, temporal filtering using PSD and DWT, and classification methods using LMD, QMD, BSC and SVM provided higher performance than those of other combinations. Utilizing one of the better combinations of ICA, PSD and SVM, the discrimination accuracy was as high as 75%. Further feature analysis showed that beta band EEG activity of the channels over right sensorimotor cortex was most appropriate for discrimination of right and left hand movement intention.ConclusionsEffective combinations of computational methods provide possible classification of human movement intention from single trial EEG. Such a method could be the basis for a potential brain–computer interface based on human natural movement, which might reduce the requirement of long-term training.SignificanceEffective combinations of computational methods can classify human movement intention from single trial EEG with reasonable accuracy.     

Predictive learning with structured (grouped) data

Many applications of machine learning involve sparse and heterogeneous data. For example, estimation of diagnostic models using patients’ data from clinical studies requires effective integration of genetic, clinical and demographic data. Typically all heterogeneous inputs are properly encoded and mapped onto a single feature vector, used for estimating a classifier. This approach, known as standard inductive learning, is used in most application studies. Recently, several new learning methodologies have emerged. For instance, when training data can be naturally separated into several groups (or structured), we can view model estimation for each group as a separate task, leading to a Multi-Task Learning framework. Similarly, a setting where the training data are structured, but the objective is to estimate a single predictive model (for all groups), leads to the Learning with Structured Data and SVM+ methodology recently proposed by Vapnik [(2006). Empirical inference science afterword of 2006. Springer]. This paper describes a biomedical application of these new data modeling approaches for modeling heterogeneous data using several medical data sets. The characteristics of group variables are analyzed. Our comparisons demonstrate the advantages and limitations of these new approaches, relative to standard inductive SVM classifiers.     

A Novel Multiple Feature-Based Engine Knock Detection System using Sparse Bayesian Extreme Learning Machine

Automotive engine knock is an abnormal combustion phenomenon that affects engine performance and lifetime expectancy, but it is difficult to detect. Collecting engine vibration signals from an engine cylinder block is an effective way to detect engine knock. This paper proposes an intelligent engine knock detection system based on engine vibration signals. First, filtered signals are obtained by utilizing variational mode decomposition (VMD), which decomposes the original time domain signals into a series of intrinsic mode functions (IMFs). Moreover, the values of the balancing parameter and the number of IMF modes are optimized using genetic algorithm (GA). IMFs with sample entropy higher than the mean are then selected as sensitive subcomponents for signal reconstruction and subsequently removed. A multiple feature learning approach that considers time domain statistical analysis (TDSA), multi-fractal detrended fluctuation analysis (MFDFA) and alpha stable distribution (ASD) simultaneously, is utilized to extract features from the denoised signals. Finally, the extracted features are trained by sparse Bayesian extreme learning machine (SBELM) to overcome the sensitivity of hyperparameters in conventional machine learning algorithms. A test rig is designed to collect the raw engine data. Compared with other technology combinations, the optimal scheme from signal processing to feature classification is obtained, and the classification accuracy of the proposed integrated engine knock detection method can achieve 98.27%. We successfully propose and test a universal intelligence solution for the detection task.

     

Prediction of performance level during a cognitive task from ongoing EEG oscillatory activities.

OBJECTIVE: Tracking the level of performance in cognitive tasks may be useful in environments, such as aircraft, in which the awareness of the pilots is critical for security. In this paper, the usefulness of EEG for the prediction of performance is investigated. METHODS: We present a new methodology that combines various ongoing EEG measurements to predict performance level during a cognitive task. We propose a voting approach that combines the outputs of elementary support vector machine (SVM) classifiers derived from various sets of EEG parameters in different frequency bands. The spectral power and phase synchrony of the oscillatory activities are used to classify the periods of rapid reaction time (RT) versus the slow RT responses of each subject. RESULTS: The voting algorithm significantly outperforms classical SVM and gives a good average classification accuracy across 12 subjects (71%) and an average information transfer rate (ITR) of 0.49bit/min. The main discriminating activities are laterally distributed theta power and anterio-posterior alpha synchronies, possibly reflecting the role of a visual-attentional network in performance. CONCLUSIONS: Power and synchrony measurements enable the discrimination between periods of high average reaction time versus periods of low average reaction time in a same subject. Moreover, the proposed approach is easy to interpret as it combines various types of measurements for classification, emphasizing the most informative. SIGNIFICANCE: Ongoing EEG recordings can predict the level of performance during a cognitive task. This can lead to real-time EEG monitoring devices for the anticipation of human mistakes.