Neural architecture design based on extreme learning machine

Selection of the optimal neural architecture to solve a pattern classification problem entails to choose the relevant input units, the number of hidden neurons and its corresponding interconnection weights. This problem has been widely studied in many research works but their solutions usually involve excessive computational cost in most of the problems and they do not provide a unique solution. This paper proposes a new technique to efficiently design the MultiLayer Perceptron (MLP) architecture for classification using the Extreme Learning Machine (ELM) algorithm. The proposed method provides a high generalization capability and a unique solution for the architecture design. Moreover, the selected final network only retains those input connections that are relevant for the classification task. Experimental results show these advantages.     

Anomaly-Based Intrusion Detection Using Extreme Learning Machine and Aggregation of Network Traffic Statistics in Probability Space

Recently, with the increased use of network communication, the risk of compromising the information has grown immensely. Intrusions have become more sophisticated and few methods can achieve efficient results while the network behavior constantly changes. This paper proposes an intrusion detection system based on modeling distributions of network statistics and Extreme Learning Machine (ELM) to achieve high detection rates of intrusions. The proposed model aggregates the network traffic at the IP subnetwork level and the distribution of statistics are collected for the most frequent IPv4 addresses encountered as destination. The obtained probability distributions are learned by ELM. This model is evaluated on the ISCX-IDS 2012 dataset, which is collected using a real-time testbed. The model is compared against leading approaches using the same dataset. Experimental results show that the presented method achieves an average detection rate of 91% and a misclassification rate of 9%. The experimental results show that our methods significantly improve the performance of the simple ELM despite a trade-off between performance and time complexity. Furthermore, our methods achieve good performance in comparison with the other few state-of-the-art approaches evaluated on the ISCX-IDS 2012 dataset.     

Embedding Anatomical or Functional Knowledge in Whole-Brain Multiple Kernel Learning Models

Pattern recognition models have been increasingly applied to neuroimaging data over the last two decades. These applications have ranged from cognitive neuroscience to clinical problems. A common limitation of these approaches is that they do not incorporate previous knowledge about the brain structure and function into the models. Previous knowledge can be embedded into pattern recognition models by imposing a grouping structure based on anatomically or functionally defined brain regions. In this work, we present a novel approach that uses group sparsity to model the whole brain multivariate pattern as a combination of regional patterns. More specifically, we use a sparse version of Multiple Kernel Learning (MKL) to simultaneously learn the contribution of each brain region, previously defined by an atlas, to the decision function. Our application of MKL provides two beneficial features: (1) it can lead to improved overall generalisation performance when the grouping structure imposed by the atlas is consistent with the data; (2) it can identify a subset of relevant brain regions for the predictive model. In order to investigate the effect of the grouping in the proposed MKL approach we compared the results of three different atlases using three different datasets. The method has been implemented in the new version of the open-source Pattern Recognition for Neuroimaging Toolbox (PRoNTo).     

Kernel learning at the first level of inference

Kernel learning methods, whether Bayesian or frequentist, typically involve multiple levels of inference, with the coefficients of the kernel expansion being determined at the first level and the kernel and regularisation parameters carefully tuned at the second level, a process known as model selection. Model selection for kernel machines is commonly performed via optimisation of a suitable model selection criterion, often based on cross-validation or theoretical performance bounds. However, if there are a large number of kernel parameters, as for instance in the case of automatic relevance determination (ARD), there is a substantial risk of over-fitting the model selection criterion, resulting in poor generalisation performance. In this paper we investigate the possibility of learning the kernel, for the Least-Squares Support Vector Machine (LS-SVM) classifier, at the first level of inference, i.e. parameter optimisation. The kernel parameters and the coefficients of the kernel expansion are jointly optimised at the first level of inference, minimising a training criterion with an additional regularisation term acting on the kernel parameters. The key advantage of this approach is that the values of only two regularisation parameters need be determined in model selection, substantially alleviating the problem of over-fitting the model selection criterion. The benefits of this approach are demonstrated using a suite of synthetic and real-world binary classification benchmark problems, where kernel learning at the first level of inference is shown to be statistically superior to the conventional approach, improves on our previous work (Cawley and Talbot, 2007) and is competitive with Multiple Kernel Learning approaches, but with reduced computational expense.     

Generating Word Embeddings from an Extreme Learning Machine for Sentiment Analysis and Sequence Labeling Tasks

Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, we proposed using an Extreme Learning Machine (ELM) for generating word embeddings. In this research, we apply the ELM-based Word Embeddings to the NLP task of Text Categorization, specifically Sentiment Analysis and Sequence Labeling. The ELM-based Word Embeddings utilizes a count-based approach similar to the Global Vectors (GloVe) model, where the word-context matrix is computed then matrix factorization is applied. A comparative study is done with Word2Vec and GloVe, which are the two popular state-of-the-art models. The results show that ELM-based Word Embeddings slightly outperforms the aforementioned two methods in the Sentiment Analysis and Sequence Labeling tasks.In addition, only one hyperparameter is needed using ELM whereas several are utilized for the other methods. ELM-based Word Embeddings are comparable to the state-of-the-art methods: Word2Vec and GloVe models. In addition, the count-based ELM model have word similarities to both the count-based GloVe and the predict-based Word2Vec models, with subtle differences.     

Distinct contribution of the cortico-striatal and cortico-cerebellar systems to motor skill learning

This review paper focuses on studies in healthy human subjects that examined the functional neuroanatomy and cerebral plasticity associated with the learning, consolidation and retention phases of motor skilled behaviors using modern brain imaging techniques. Evidence in support of a recent model proposed by Doyon and Ungerleider [Functional Anatomy of Motor Skill Learning. In: Squire LR, Schacter DL, editors. Neuropsychology of Memory. New York: Guilford Press, 2002.] is also discussed. The latter suggests that experience-dependent changes in the brain depend not only on the stage of learning, but also on whether subjects are required to learn a new sequence of movements (motor sequence learning) or learn to adapt to environmental perturbations (motor adaptation). This model proposes that the cortico-striatal and cortico-cerebellar systems contribute differentially to motor sequence learning and motor adaptation, respectively, and that this is most apparent during the slow learning phase (i.e. automatization) when subjects achieve asymptotic performance, as well as during reactivation of the new skilled behavior in the retention phase.     

Characterization of a temporoparietal junction subtype of Alzheimer's disease

Alzheimer's disease (AD) subtypes have been described according to genetics, neuropsychology, neuropathology, and neuroimaging. Thirty‐one patients with clinically probable AD were selected based on perisylvian metabolic decrease on FDG‐PET. They were compared to 25 patients with a typical pattern of decreased posterior metabolism. Tree‐based machine learning was used on those 56 images to create a classifier that was subsequently applied to 207 Alzheimer's Disease Neuroimaging Initiative (ADNI) patients with AD. Machine learning was also used to discriminate between the two ADNI groups based on neuropsychological scores. Compared to AD patients with a typical precuneus metabolic decrease, the new subtype showed stronger hypometabolism in the temporoparietal junction. The classifier was able to distinguish the two groups in the ADNI population. Both groups could only be distinguished cognitively by Trail Making Test‐A scores. This study further confirms that there is more than a typical metabolic pattern in probable AD with amnestic presentation.     

A Meta-Cognitive Learning Algorithm for an Extreme Learning Machine Classifier

This paper presents an efficient fast learning classifier based on the Nelson and Narens model of human meta-cognition, namely ‘Meta-cognitive Extreme Learning Machine (McELM).’ McELM has two components: a cognitive component and a meta-cognitive component. The cognitive component of McELM is a three-layered extreme learning machine (ELM) classifier. The neurons in the hidden layer of the cognitive component employ the q-Gaussian activation function, while the neurons in the input and output layers are linear. The meta-cognitive component of McELM has a self-regulatory learning mechanism that decides what-to-learn, when-to-learn, and how-to-learn in a meta-cognitive framework. As the samples in the training set are presented one-by-one, the meta-cognitive component receives the monitory signals from the cognitive component and chooses suitable learning strategies for the sample. Thus, it either deletes the sample, uses the sample to add a new neuron, or updates the output weights based on the sample, or reserves the sample for future use. Therefore, unlike the conventional ELM, the architecture of McELM is not fixed a priori, instead, the network is built during the training process. While adding a neuron, McELM chooses the centers based on the sample, and the width of the Gaussian function is chosen randomly. The output weights are estimated using the least square estimate based on the hinge-loss error function. The hinge-loss error function facilitates prediction of posterior probabilities better than the mean-square error and is hence preferred to develop the McELM classifier. While updating the network parameters, the output weights are updated using a recursive least square estimate. The performance of McELM is evaluated on a set of benchmark classification problems from the UCI machine learning repository. Performance study results highlight that meta-cognition in ELM framework enhances the decision-making ability of ELM significantly.     

Hybrid extreme rotation forest

This paper proposes the Hybrid Extreme Rotation Forest (HERF), an innovative ensemble learning algorithm for classification problems, combining classical Decision Trees with the recently proposed Extreme Learning Machines (ELM) training of Neural Networks. In the HERF algorithm, training of each individual classifier involves two steps: first computing a randomized data rotation transformation of the training data, second, training the individual classifier on the rotated data. The testing data is subjected to the same transformation as the training data, which is specific for each classifier in the ensemble. Experimental design in this paper involves (a) the comparison of factorization approaches to compute the randomized rotation matrix: the Principal Component Analysis (PCA) and the Quartimax, (b) assessing the effect of data normalization and bootstrapping training data selection, (c) all variants of single and combined ELM and decision trees, including Regularized ELM. This experimental design effectively includes other state-of-the-art ensemble approaches in the comparison, such as Voting ELM and Random Forest. We report extensive results over a collection of machine learning benchmark databases. Ranking the cross-validation results per experimental dataset and classifier tested concludes that HERF significantly improves over the other state-of-the-art ensemble classifier. Besides, we find some other results such as that the data rotation with Quartimax improves over PCA, and the relative insensitivity of the approach to regularization which may be attributable to the de facto regularization performed by the ensemble approach.     

Brain-computer interfaces for communication and control.

For many years people have speculated that electroencephalographic activity or other electrophysiological measures of brain function might provide a new non-muscular channel for sending messages and commands to the external world - a brain-computer interface (BCI). Over the past 15 years, productive BCI research programs have arisen. Encouraged by new understanding of brain function, by the advent of powerful low-cost computer equipment, and by growing recognition of the needs and potentials of people with disabilities, these programs concentrate on developing new augmentative communication and control technology for those with severe neuromuscular disorders, such as amyotrophic lateral sclerosis, brainstem stroke, and spinal cord injury. The immediate goal is to provide these users, who may be completely paralyzed, or 'locked in', with basic communication capabilities so that they can express their wishes to caregivers or even operate word processing programs or neuroprostheses. Present-day BCIs determine the intent of the user from a variety of different electrophysiological signals. These signals include slow cortical potentials, P300 potentials, and mu or beta rhythms recorded from the scalp, and cortical neuronal activity recorded by implanted electrodes. They are translated in real-time into commands that operate a computer display or other device. Successful operation requires that the user encode commands in these signals and that the BCI derive the commands from the signals. Thus, the user and the BCI system need to adapt to each other both initially and continually so as to ensure stable performance. Current BCIs have maximum information transfer rates up to 10-25bits/min. This limited capacity can be valuable for people whose severe disabilities prevent them from using conventional augmentative communication methods. At the same time, many possible applications of BCI technology, such as neuroprosthesis control, may require higher information transfer rates. Future progress will depend on: recognition that BCI research and development is an interdisciplinary problem, involving neurobiology, psychology, engineering, mathematics, and computer science; identification of those signals, whether evoked potentials, spontaneous rhythms, or neuronal firing rates, that users are best able to control independent of activity in conventional motor output pathways; development of training methods for helping users to gain and maintain that control; delineation of the best algorithms for translating these signals into device commands; attention to the identification and elimination of artifacts such as electromyographic and electro-oculographic activity; adoption of precise and objective procedures for evaluating BCI performance; recognition of the need for long-term as well as short-term assessment of BCI performance; identification of appropriate BCI applications and appropriate matching of applications and users; and attention to factors that affect user acceptance of augmentative technology, including ease of use, cosmesis, and provision of those communication and control capacities that are most important to the user. Development of BCI technology will also benefit from greater emphasis on peer-reviewed research publications and avoidance of the hyperbolic and often misleading media attention that tends to generate unrealistic expectations in the public and skepticism in other researchers. With adequate recognition and effective engagement of all these issues, BCI systems could eventually provide an important new communication and control option for those with motor disabilities and might also give those without disabilities a supplementary control channel or a control channel useful in special circumstances.     

Bayesian representation learning in the cortex regulated by acetylcholine

A brain needs to detect an environmental change and to quickly learn internal representations necessary in a new environment. This paper presents a theoretical model of cortical representation learning that can adapt to dynamic environments, incorporating the results by previous studies on the functional role of acetylcholine (ACh). We adopt the probabilistic principal component analysis (PPCA) as a functional model of cortical representation learning, and present an on-line learning method for PPCA according to Bayesian inference, including a heuristic criterion for model selection. Our approach is examined in two types of simulations with synthesized and realistic data sets, in which our model is able to re-learn new representation bases after the environment changes. Our model implies the possibility that a higher-level recognition regulates the cortical ACh release in the lower-level, and that the ACh level alters the learning dynamics of a local circuit in order to continuously acquire appropriate representations in a dynamic environment.     

High‐resolution tracking of motor disorders in Parkinson's disease during unconstrained activity

Parkinson's disease (PD) can present with a variety of motor disorders that fluctuate throughout the day, making assessment a challenging task. Paper‐based measurement tools can be burdensome to the patient and clinician and lack the temporal resolution needed to accurately and objectively track changes in motor symptom severity throughout the day. Wearable sensor‐based systems that continuously monitor PD motor disorders may help to solve this problem, although critical shortcomings persist in identifying multiple disorders at high temporal resolution during unconstrained activity. The purpose of this study was to advance the current state of the art by (1) introducing hybrid sensor technology to concurrently acquire surface electromyographic (sEMG) and accelerometer data during unconstrained activity and (2) analyzing the data using dynamic neural network algorithms to capture the evolving temporal characteristics of the sensor data and improve motor disorder recognition of tremor and dyskinesia. Algorithms were trained (n = 11 patients) and tested (n = 8 patients; n = 4 controls) to recognize tremor and dyskinesia at 1‐second resolution based on sensor data features and expert annotation of video recording during 4‐hour monitoring periods of unconstrained daily activity. The algorithms were able to make accurate distinctions between tremor, dyskinesia, and normal movement despite the presence of diverse voluntary activity. Motor disorder severity classifications averaged 94.9% sensitivity and 97.1% specificity based on 1 sensor per symptomatic limb. These initial findings indicate that new sensor technology and software algorithms can be effective in enhancing wearable sensor‐based system performance for monitoring PD motor disorders during unconstrained activities. © 2013 Movement Disorder Society     

Machine Learning for Brain Stroke: A Review

Machine Learning (ML) delivers an accurate and quick prediction outcome and it has become a powerful tool in health settings, offering personalized clinical care for stroke patients. An application of ML and Deep Learning in health care is growing however, some research areas do not catch enough attention for scientific investigation though there is real need of research. Therefore, the aim of this work is to classify state-of-arts on ML techniques for brain stroke into 4 categories based on their functionalities or similarity, and then review studies of each category systematically. A total of 39 studies were identified from the results of ScienceDirect web scientific database on ML for brain stroke from the year 2007 to 2019. Support Vector Machine (SVM) is obtained as optimal models in 10 studies for stroke problems. Besides, maximum studies are found in stroke diagnosis although number for stroke treatment is least thus, it identifies a research gap for further investigation. Similarly, CT images are a frequently used dataset in stroke. Finally SVM and Random Forests are efficient techniques used under each category. The present study showcases the contribution of various ML approaches applied to brain stroke.     

Network abnormalities among non‐manifesting Parkinson disease related LRRK2 mutation carriers

Non‐manifesting carriers (NMC) of the G2019S mutation in the LRRK2 gene represent an “at risk” group for future development of Parkinson's disease (PD) and have demonstrated task related fMRI changes. However, resting‐state networks have received less research focus, thus this study aimed to assess the integrity of the motor, default mode (DMN), salience (SAL), and dorsal attention (DAN) networks among this unique population by using two different connectivity measures: interregional functional connectivity analysis and Dependency network analysis (D_EPNA). Machine learning classification methods were used to distinguish connectivity between the two groups of participants. Forty‐four NMC and 41 non‐manifesting non‐carriers (NMNC) participated in this study; while no behavioral differences on standard questionnaires could be detected, NMC demonstrated lower connectivity measures in the DMN, SAL, and DAN compared to NMNC but not in the motor network. Significant correlations between NMC connectivity measures in the SAL and attention were identified. Machine learning classification separated NMC from NMNC with an accuracy rate above 0.8. Reduced integrity of non‐motor networks was detected among NMC of the G2019S mutation in the LRRK2 gene prior to identifiable changes in connectivity of the motor network, indicating significant non‐motor cerebral changes among populations “at risk” for future development of PD.     

Improving the Applicability of AI for Psychiatric Applications through Human-in-the-loop Methodologies

ObjectivesMachine learning (ML) and natural language processing have great potential to improve efficiency and accuracy in diagnosis, treatment recommendations, predictive interventions, and scarce resource allocation within psychiatry. Researchers often conceptualize such an approach as operating in isolation without much need for human involvement, yet it remains crucial to harness human-in-the-loop practices when developing and implementing such techniques as their absence may be catastrophic. We advocate for building ML-based technologies that collaborate with experts within psychiatry in all stages of implementation and use to increase model performance while simultaneously increasing the practicality, robustness, and reliability of the process.MethodsWe showcase pitfalls of the traditional ML framework and explain how it can be improved with human-in-the-loop techniques. Specifically, we applied active learning strategies to the automatic scoring of a story recall task and compared the results to a traditional approach.ResultsHuman-in-the-loop methodologies supplied a greater understanding of where the model was least confident or had knowledge gaps during training. As compared to the traditional framework, less than half of the training data were needed to reach a given accuracy.ConclusionsHuman-in-the-loop ML is an approach to data collection and model creation that harnesses active learning to select the most critical data needed to increase a model’s accuracy and generalizability more efficiently than classic random sampling would otherwise allow. Such techniques may additionally operate as safeguards from spurious predictions and can aid in decreasing disparities that artificial intelligence systems otherwise propagate.     

Computer aided diagnosis of schizophrenia on resting state fMRI data by ensembles of ELM

Resting state functional Magnetic Resonance Imaging (rs-fMRI) is increasingly used for the identification of image biomarkers of brain diseases or psychiatric conditions such as schizophrenia. This paper deals with the application of ensembles of Extreme Learning Machines (ELM) to build Computer Aided Diagnosis systems on the basis of features extracted from the activity measures computed over rs-fMRI data. The power of ELM to provide quick but near optimal solutions to the training of Single Layer Feedforward Networks (SLFN) allows extensive exploration of discriminative power of feature spaces in affordable time with off-the-shelf computational resources. Exploration is performed in this paper by an evolutionary search approach that has found functional activity map features allowing to achieve quite successful classification experiments, providing biologically plausible voxel-site localizations.     

Modeling longitudinal imaging biomarkers with parametric Bayesian multi‐task learning

Longitudinal imaging biomarkers are invaluable for understanding the course of neurodegeneration, promising the ability to track disease progression and to detect disease earlier than cross‐sectional biomarkers. To properly realize their potential, biomarker trajectory models must be robust to both under‐sampling and measurement errors and should be able to integrate multi‐modal information to improve trajectory inference and prediction. Here we present a parametric Bayesian multi‐task learning based approach to modeling univariate trajectories across subjects that addresses these criteria. Our approach learns multiple subjects' trajectories within a single model that allows for different types of information sharing, that is, coupling, across subjects. It optimizes a combination of uncoupled, fully coupled and kernel coupled models. Kernel‐based coupling allows linking subjects' trajectories based on one or more biomarker measures. We demonstrate this using Alzheimer's Disease Neuroimaging Initiative (ADNI) data, where we model longitudinal trajectories of MRI‐derived cortical volumes in neurodegeneration, with coupling based on APOE genotype, cerebrospinal fluid (CSF) and amyloid PET‐based biomarkers. In addition to detecting established disease effects, we detect disease related changes within the insula that have not received much attention within the literature. Due to its sensitivity in detecting disease effects, its competitive predictive performance and its ability to learn the optimal parameter covariance from data rather than choosing a specific set of random and fixed effects a priori, we propose that our model can be used in place of or in addition to linear mixed effects models when modeling biomarker trajectories. A software implementation of the method is publicly available.     

Accelerometer use during field-based physical activity research in children and adolescents with intellectual disabilities: A systematic review

Many methodological questions and issues surround the use of accelerometers as a measure of physical activity during field-based research. To ensure overall research quality and the accuracy of results, methodological decisions should be based on study research questions. This paper aims to systematically review accelerometer use during field-based research in children and adolescents with intellectual disabilities. Medline, Embase, Cochrane Library, Web of Knowledge, PsycINFO, PubMed, and a thesis database (up to May 2013) were searched to identify relevant articles. Articles which used accelerometry-based monitors, quantified activity levels, and included ambulatory children and adolescents (≤18 years) with intellectual disabilities were included. Based on best practice guidelines, a form was developed to extract data based on 17 research components of accelerometer use. The search identified 429 articles. Ten full-text articles met the criteria and were included in the review. Many shortcomings in accelerometer use were identified, with the percentage of review criteria met ranging from 12% to 47%. Various methods of accelerometer use were reported, with most use decisions not based on population-specific research. However, a lack of measurement research, e.g., calibration/validation, for children and adolescents with intellectual disabilities is limiting the ability of field-based researchers to make to the most appropriate accelerometer use decisions. The methods of accelerometer use employed can have significant effects on the quality and validity of results produced, which researchers should be more aware of. To allow informed use decisions, there should be a greater focus on measurement research related to children and adolescents with intellectual disabilities.