Fast Face Recognition Via Sparse Coding and Extreme Learning Machine

Most face recognition approaches developed so far regard the sparse coding as one of the essential means, while the sparse coding models have been hampered by the extremely expensive computational cost in the implementation. In this paper, a novel scheme for the fast face recognition is presented via extreme learning machine (ELM) and sparse coding. The common feature hypothesis is first introduced to extract the basis function from the local universal images, and then the single hidden layer feedforward network (SLFN) is established to simulate the sparse coding process for the face images by ELM algorithm. Some developments have been done to maintain the efficient inherent information embedding in the ELM learning. The resulting local sparse coding coefficient will then be grouped into the global representation and further fed into the ELM ensemble which is composed of a number of SLFNs for face recognition. The simulation results have shown the good performance in the proposed approach that could be comparable to the state-of-the-art techniques at a much higher speed.     

A Cognitive Ensemble of Extreme Learning Machines for Steganalysis Based on Risk-Sensitive Hinge Loss Function

In this paper, we propose a risk-sensitive hinge loss function-based cognitive ensemble of extreme learning machine (ELM) classifiers for JPEG steganalysis. ELM is a single hidden-layer feed-forward network that chooses the input parameters randomly and estimates the output weights analytically. For steganalysis, we have extracted 548-dimensional merge features and trained ELM to approximate the functional relationship between the merge features and class label. Further, we use a cognitive ensemble of ELM classifier with risk-sensitive hinge loss function for accurate steganalysis. As the hinge loss error function is shown to be better than mean-squared error function for classification problems, here, the individual ELM classifiers are developed based on hinge loss error function. The cognition in the ensemble of ELM obtains the weighted sum of individual classifiers by enhancing the outputs of winning classifiers for a sample, while penalizing the other classifiers for the sample. Thus, the cognitive ensemble ELM classifier positively exploits the effect of initialization in each classifier to obtain the best results. The performance of the cognitive ensemble ELM in performing the steganalysis is compared to that of a single ELM, and the existing ensemble support vector machine classifier for steganalysis. Performance results show the superior classification ability of the cognitive ensemble ELM classifier.     

Fast Image Recognition Based on Independent Component Analysis and Extreme Learning Machine

Nowadays, image recognition has become a highly active research topic in cognitive computation community, due to its many potential applications. Generally, the image recognition task involves two subtasks: image representation and image classification. Most feature extraction approaches for image representation developed so far regard independent component analysis (ICA) as one of the essential means. However, ICA has been hampered by its extremely expensive computational cost in real-time implementation. To address this problem, a fast cognitive computational scheme for image recognition is presented in this paper, which combines ICA and the extreme learning machine (ELM) algorithm. It tries to solve the image recognition problem at a much faster speed by using ELM not only in image classification but also in feature extraction for image representation. As an example, our proposed approach is applied to the face image recognition with detailed analysis. Firstly, common feature hypothesis is introduced to extract the common visual features from universal images by the traditional ICA model in the offline recognition process, and then ELM is used to simulate ICA for the purpose of facial feature extraction in the online recognition process. Lastly, the resulting independent feature representation of the face images extracted by ELM rather than ICA will be fed into the ELM classifier, which is composed of numerous single hidden layer feed-forward networks. Experimental results on Yale face database and MNIST digit database have shown the good performance of our proposed approach, which could be comparable to the state-of-the-art techniques at a much faster speed.     

An Insight into Extreme Learning Machines: Random Neurons,Random Features and Kernels

Extreme learning machines (ELMs) basically give answers to two fundamental learning problems: (1) Can fundamentals of learning (i.e., feature learning, clustering, regression and classification) be made without tuning hidden neurons (including biological neurons) even when the output shapes and function modeling of these neurons are unknown? (2) Does there exist unified framework for feedforward neural networks and feature space methods? ELMs that have built some tangible links between machine learning techniques and biological learning mechanisms have recently attracted increasing attention of researchers in widespread research areas. This paper provides an insight into ELMs in three aspects, viz: random neurons, random features and kernels. This paper also shows that in theory ELMs (with the same kernels) tend to outperform support vector machine and its variants in both regression and classification applications with much easier implementation.     

Rank-Adaptive Non-Negative Matrix Factorization

Dimension reduction is a challenge task in data processing, especially in high-dimensional data processing area. Non-negative matrix factorization (NMF), as a classical dimension reduction method, has a contribution to the parts-based representation for the characteristics of non-negative constraints in the NMF algorithm. In this paper, the NMF algorithm is introduced to extract local features for dimension reduction. Considering the problem of which NMF is required to define the number of the decomposition rank manually, we proposed a rank-adaptive NMF algorithm, in which the affinity propagation (AP) clustering algorithm is introduced to determine adaptively the number of the decomposition rank of NMF. Then, the rank-adaptive NMF algorithm is used to extract features for the original image. After that, a low-dimensional representation of the original image is obtained through the projection from the original images to the feature space. Finally, we used extreme learning machine (ELM) and k-nearest neighbor (KNN) as the classifier to classify those low-dimensional feature representations. The experimental results demonstrate that the decomposition rank determined by the AP clustering algorithm can reflect the characteristics of the original data. When it is combined with the classification algorithm ELM or KNN and applied to handwritten character recognition, the proposed method not only reduces the dimension of original images but also performs well in terms of classification accuracy and time consumption. A new rank-adaptive NMF algorithm is proposed based on the AP clustering algorithm and the original NMF algorithm. According to this algorithm, the low-dimensional representation of the original data can be obtained without any prior knowledge. In addition, the proposed rank-adaptive NMF algorithm combined with the ELM and KNN classification algorithms performs well.     

Semi-supervised Learning for Affective Common-Sense Reasoning

Background

Big social data analysis is the area of research focusing on collecting, examining, and processing large multi-modal and multi-source datasets in order to discover patterns/correlations and extract information from the Social Web. This is usually accomplished through the use of supervised and unsupervised machine learning algorithms that learn from the available data. However, these are usually highly computationally expensive, either in the training or in the prediction phase, as they are often not able to handle current data volumes. Parallel approaches have been proposed in order to boost processing speeds, but this clearly requires technologies that support distributed computations.

Methods

Extreme learning machines (ELMs) are an emerging learning paradigm, presenting an efficient unified solution to generalized feed-forward neural networks. ELM offers significant advantages such as fast learning speed, ease of implementation, and minimal human intervention. However, ELM cannot be easily parallelized, due to the presence of a pseudo-inverse calculation. Therefore, this paper aims to find a reliable method to realize a parallel implementation of ELM that can be applied to large datasets typical of Big Data problems with the employment of the most recent technology for parallel in-memory computation, i.e., Spark, designed to efficiently deal with iterative procedures that recursively perform operations over the same data. Moreover, this paper shows how to take advantage of the most recent advances in statistical learning theory (SLT) in order to address the issue of selecting ELM hyperparameters that give the best generalization performance. This involves assessing the performance of such algorithms (i.e., resampling methods and in-sample methods) by exploiting the most recent results in SLT and adapting them to the Big Data framework. The proposed approach has been tested on two affective analogical reasoning datasets. Affective analogical reasoning can be defined as the intrinsically human capacity to interpret the cognitive and affective information associated with natural language. In particular, we employed two benchmarks, each one composed by 21,743 common-sense concepts; each concept is represented according to two models of a semantic network in which common-sense concepts are linked to a hierarchy of affective domain labels.

Results

The labeled data have been split into two sets: The first 20,000 samples have been used for building the model with the ELM with the different SLT strategies, while the rest of the labeled samples, numbering 1743, have been kept apart as reference set in order to test the performance of the learned model. The splitting process has been repeated 30 times in order to obtain statistically relevant results. We ran the experiments through the use of the Google Cloud Platform, in particular, the Google Compute Engine. We employed the Google Compute Engine Platform with NM = 4 machines with two cores and 1.8 GB of RAM (machine type n1-highcpu-2) and an HDD of 30 GB equipped with Spark. Results on the affective dataset both show the effectiveness of the proposed parallel approach and underline the most suitable SLT strategies for the specific Big Data problem.

Conclusion

In this paper we showed how to build an ELM model with a novel scalable approach and to carefully assess the performance, with the use of the most recent results from SLT, for a sentiment analysis problem. Thanks to recent technologies and methods, the computational requirements of these methods have been improved to allow for the scaling to large datasets, which are typical of Big Data applications.

     

基于CT三维重建个体化股骨假体的计算机辅助设计

背景：由于人体的绝对个性化特点,标准人工假体与患者骨骼之间的误差使二者难以很好匹配。计算机辅助设计和制造个体化假体克服了其他假体的缺点,可有效地延长人工关节的使用寿命和使用质量,并可能解决人工关节的翻修问题。国内的研究尚处于起步阶段。 目的：基于CT图像的三维重建,探求个体化股骨假体计算机辅助设计在提高假体与病变骨骼匹配度中的作用。 方法：CT扫描对象为1例健康男性志愿者,排除髋关节疾患。采用GE Speed Light 16排螺旋CT对股骨中上段进行层厚3 mm扫描,得到CT数据的二维图像,利用自主开发的数据格式转换软件将CT图像转换为bmp格式。对位图编辑预处理,用Mimics8.1软件进行矢量化处理,提取股骨内外轮廓。然后输入Mimics8.1和Rapidform2004三维反求工程软件中,生成股骨内外轮廓的特征曲线,重建股骨三维模型。将股骨髓腔的特征轮廓曲线dxf文件输入计算机辅助设计建模软件Solidworks2004中,以此股骨髓腔轮廓为基础,完成个体化股骨假体的设计。 结果与结论：利用自主开发的数据格式转换软件,实现了CT图像信息的矢量转换。以CT二维图像为依据,进行三维反求,可获得精确的股骨内外轮廓三维实体模型。采用反求工程与正向计算机辅助设计相结合,可设计出匹配良好的个体化股骨假体。提示反求工程和计算机辅助设计技术为个体化假体的研制提供了一个有效可行的途径,解决假体与病变骨骼的良好匹配,可防止假体松动,提高其长期稳定性。     

Iris Segmentation Using Feature Channel Optimization for Noisy Environments

In recent years, iris recognition has been widely used in various fields. As the first step of iris recognition, segmentation accuracy is of great significance to the final recognition. However, iris images exhibit a variety of noise in the real world, which leads to lower segmentation accuracy than the ideal case. To address this problem, this paper proposes an iris segmentation method using feature channel optimization for noisy images. The method for non-ideal environments with noise is more suitable for practical applications. We add dense blocks and dilated convolutional layers to the encoder so that the information gradient flow obtained by different layers can be reused, and the receptive field can be expanded. In the decoder, based on Jensen-Shannon (JS) divergence, we first recalculate the weight of the feature channels obtained from each layer, which enhances the useful information and suppresses the interference information in the noisy environments to boost the segmentation accuracy. The proposed architecture is validated in the CASIA v4.0 interval (CASIA) and IIT Delhi v1.0 datasets (IITD). For CASIA, the mean error rate is 0.78%, and the F-measure value is 98.21%. For IITD, the mean error rate is 0.97%, and the F-measure value is 97.87%. Experimental results show that the proposed method outperforms other state-of-art methods under noisy environments, such as Gaussian blur, Gaussian noise, and salt and pepper noise.

     

Hybrid extreme rotation forest

This paper proposes the Hybrid Extreme Rotation Forest (HERF), an innovative ensemble learning algorithm for classification problems, combining classical Decision Trees with the recently proposed Extreme Learning Machines (ELM) training of Neural Networks. In the HERF algorithm, training of each individual classifier involves two steps: first computing a randomized data rotation transformation of the training data, second, training the individual classifier on the rotated data. The testing data is subjected to the same transformation as the training data, which is specific for each classifier in the ensemble. Experimental design in this paper involves (a) the comparison of factorization approaches to compute the randomized rotation matrix: the Principal Component Analysis (PCA) and the Quartimax, (b) assessing the effect of data normalization and bootstrapping training data selection, (c) all variants of single and combined ELM and decision trees, including Regularized ELM. This experimental design effectively includes other state-of-the-art ensemble approaches in the comparison, such as Voting ELM and Random Forest. We report extensive results over a collection of machine learning benchmark databases. Ranking the cross-validation results per experimental dataset and classifier tested concludes that HERF significantly improves over the other state-of-the-art ensemble classifier. Besides, we find some other results such as that the data rotation with Quartimax improves over PCA, and the relative insensitivity of the approach to regularization which may be attributable to the de facto regularization performed by the ensemble approach.     

Brief Report: Accuracy and Response Time for the Recognition of Facial Emotions in a Large Sample of Children with Autism Spectrum Disorders

The empirical literature has presented inconsistent evidence for deficits in the recognition of basic emotion expressions in children with autism spectrum disorders (ASD), which may be due to the focus on research with relatively small sample sizes. Additionally, it is proposed that although children with ASD may correctly identify emotion expression they rely on more deliberate, more time-consuming strategies in order to accurately recognize emotion expressions when compared to typically developing children. In the current study, we examine both emotion recognition accuracy and response time in a large sample of children, and explore the moderating influence of verbal ability on these findings. The sample consisted of 86 children with ASD (M _age = 10.65) and 114 typically developing children (M _age = 10.32) between 7 and 13 years of age. All children completed a pre-test (emotion word–word matching), and test phase consisting of basic emotion recognition, whereby they were required to match a target emotion expression to the correct emotion word; accuracy and response time were recorded. Verbal IQ was controlled for in the analyses. We found no evidence of a systematic deficit in emotion recognition accuracy or response time for children with ASD, controlling for verbal ability. However, when controlling for children’s accuracy in word–word matching, children with ASD had significantly lower emotion recognition accuracy when compared to typically developing children. The findings suggest that the social impairments observed in children with ASD are not the result of marked deficits in basic emotion recognition accuracy or longer response times. However, children with ASD may be relying on other perceptual skills (such as advanced word–word matching) to complete emotion recognition tasks at a similar level as typically developing children.     

Conditional Random Mapping for Effective ELM Feature Representation

Extreme learning machine (ELM) has been extensively studied, due to its fast training and good generalization. Unfortunately, the existing ELM-based feature representation methods are uncompetitive with state-of-the-art deep neural networks (DNNs) when conducting some complex visual recognition tasks. This weakness is mainly caused by two critical defects: (1) random feature mappings (RFM) by ad hoc probability distribution is unable to well project various input data into discriminative feature spaces; (2) in the ELM-based hierarchical architectures, features from previous layer are scattered via RFM in the current layer, which leads to abstracting higher level features ineffectively. To address these issues, we aim to take advantage of label information for optimizing random mapping in the ELM, utilizing an efficient label alignment metric to learn a conditional random feature mapping (CRFM) in a supervised manner. Moreover, we proposed a new CRFM-based single-layer ELM (CELM) and then extended CELM to the supervised multi-layer learning architecture (ML-CELM). Extensive experiments on various widely used datasets demonstrate our approach is more effective than original ELM-based and other existing DNN feature representation methods with rapid training/testing speed. The proposed CELM and ML-CELM are able to achieve discriminative and robust feature representation, and have shown superiority in various simulations in terms of generalization and speed.     

Decoding the individual finger movements from single‐trial functional magnetic resonance imaging recordings of human brain activity

Multivariate pattern classification analysis (MVPA) has been applied to functional magnetic resonance imaging (fMRI) data to decode brain states from spatially distributed activation patterns. Decoding upper limb movements from non‐invasively recorded human brain activation is crucial for implementing a brain–machine interface that directly harnesses an individual's thoughts to control external devices or computers. The aim of this study was to decode the individual finger movements from fMRI single‐trial data. Thirteen healthy human subjects participated in a visually cued delayed finger movement task, and only one slight button press was performed in each trial. Using MVPA, the decoding accuracy (DA) was computed separately for the different motor‐related regions of interest. For the construction of feature vectors, the feature vectors from two successive volumes in the image series for a trial were concatenated. With these spatial–temporal feature vectors, we obtained a 63.1% average DA (84.7% for the best subject) for the contralateral primary somatosensory cortex and a 46.0% average DA (71.0% for the best subject) for the contralateral primary motor cortex; both of these values were significantly above the chance level (20%). In addition, we implemented searchlight MVPA to search for informative regions in an unbiased manner across the whole brain. Furthermore, by applying searchlight MVPA to each volume of a trial, we visually demonstrated the information for decoding, both spatially and temporally. The results suggest that the non‐invasive fMRI technique may provide informative features for decoding individual finger movements and the potential of developing an fMRI‐based brain–machine interface for finger movement.     

Cholinergic inputs to laryngeal motoneurons functionally identified in vivo in rat: a combined electrophysiological and microscopic study

The intrinsic laryngeal muscles are differentially modulated during respiration as well as other states and behaviors such as hypocapnia and sleep. Previous anatomical and pharmacological studies indicate a role for acetylcholine at the level of the nucleus ambiguus in the modulation of laryngeal motoneuron (LMN) activity. The present study investigated the anatomical nature of cholinergic input to inspiratory- (ILM) and expiratory-modulated (ELM) laryngeal motoneurons in the loose formation of the nucleus ambiguus. Using combined in vivo intracellular recording, dye filling, and immunohistochemistry, we demonstrate that LMNs identified in Sprague-Dawley rat receive several close appositions from vesicular acetylcholine transporter-immunoreactive (VAChT-ir) boutons. ELMs receive a significantly greater number of close appositions (mean ± standard deviation [SD]: 47 ± 11; n = 5) than ILMs (32 ± 9; n = 8; t-test P < 0.05). For both LMN types, more close appositions were observed on the cell soma and proximal dendrites compared to distal dendrites (two-way analysis of variance [ANOVA], P < 0.0001). Using fluorescence confocal microscopy, almost 90% of VAChT-ir close appositions (n = 45 boutons on n = 4 ELMs) were colocalized with the synaptic marker synaptophysin. These results support a strong influence of cholinergic input on LMNs and may have implications in the differential modulation of laryngeal muscle activity.     

An electrophysiological response to different pitch contours in words.

A spoken word with more than one syllable contains a specific stress pattern found to be processed during spoken word recognition. The present study investigated the word's pitch contour as a single auditory parameter that marks stress. Event-related brain potentials (ERPs) were recorded while subjects made decisions to artificially pitch manipulated words. ERPs revealed that pitch contours are discriminated already within the first syllable of a word. Furthermore, behavioral responses for words with incorrect pitch contours were longer than for words with correct pitch contours. The results suggest that the pitch contour is an auditory feature of the spoken word that a listener automatically processes during spoken word recognition.     

ARTMAP neural networks for information fusion and data mining: map production and target recognition methodologies.

The Sensor Exploitation Group of MIT Lincoln Laboratory incorporated an early version of the ARTMAP neural network as the recognition engine of a hierarchical system for fusion and data mining of registered geospatial images. The Lincoln Lab system has been successfully fielded, but is limited to target/non-target identifications and does not produce whole maps. Procedures defined here extend these capabilities by means of a mapping method that learns to identify and distribute arbitrarily many target classes. This new spatial data mining system is designed particularly to cope with the highly skewed class distributions of typical mapping problems. Specification of canonical algorithms and a benchmark testbed has enabled the evaluation of candidate recognition networks as well as pre- and post-processing and feature selection options. The resulting mapping methodology sets a standard for a variety of spatial data mining tasks. In particular, training pixels are drawn from a region that is spatially distinct from the mapped region, which could feature an output class mix that is substantially different from that of the training set. The system recognition component, default ARTMAP, with its fully specified set of canonical parameter values, has become the a priori system of choice among this family of neural networks for a wide variety of applications.     

FE-ELM: A New Friend Recommendation Model with Extreme Learning Machine

Friend recommendation is one of the most popular services in location-based social network (LBSN) platforms, which recommends interested or familiar people to users. Except for the original social property and textual property in social networks, LBSN specially owns the spatial-temporal property. However, none of the existing methods fully utilized all the three properties (i.e., just one or two), which may lead to the low recommendation accuracy. Moreover, these existing methods are usually inefficient. In this paper, we propose a new friend recommendation model to solve the above shortcomings of the existing methods, called feature extraction-extreme learning machine (FE-ELM), where friend recommendation is regarded as a binary classification problem. Classification is an important task in cognitive computation community. First, we use new strategies in our FE-ELM model to extract the spatial-temporal feature, social feature, and textual feature. These features make full use of all above properties of LBSN and ensure the recommendation accuracy. Second, our FE-ELM model also takes advantage of the extreme learning machine (ELM) classifier. ELM has fast learning speed and ensures the recommendation efficiency. Extensive experiments verify the accuracy and efficiency of FE-ELM model.     

Reservoir computing and extreme learning machines for non-linear time-series data analysis

Random projection architectures such as Echo state networks (ESNs) and Extreme Learning Machines (ELMs) use a network containing a randomly connected hidden layer and train only the output weights, overcoming the problems associated with the complex and computationally demanding training algorithms traditionally used to train neural networks, particularly recurrent neural networks. In this study an ESN is shown to contain an antagonistic trade-off between the amount of non-linear mapping and short-term memory it can exhibit when applied to time-series data which are highly non-linear. To overcome this trade-off a new architecture, Reservoir with Random Static Projections (R²SP) is investigated, that is shown to offer a significant improvement in performance. A similar approach using an ELM whose input is presented through a time delay (TD-ELM) is shown to further enhance performance where it significantly outperformed the ESN and R²SP as well other architectures when applied to a novel task which allows the short-term memory and non-linearity to be varied. The hard-limiting memory of the TD-ELM appears to be best suited for the data investigated in this study, although ESN-based approaches may offer improved performance when processing data which require a longer fading memory.     

End-to-End ConvNet for Tactile Recognition Using Residual Orthogonal Tiling and Pyramid Convolution Ensemble

Tactile recognition enables robots identify target objects or environments from tactile sensory readings. The recent advancement of deep learning and biological tactile sensing inspire us proposing an end-to-end architecture ROTConvPCE-mv that performs tactile recognition using residual orthogonal tiling and pyramid convolution ensemble. Our approach uses stacks of raw frames and tactile flow as dual input, and incorporates the strength of multi-layer OTConvs (orthogonal tiling convolutions) organized in a residual learning paradigm. We empirically demonstrate that OTConvs have adjustable invariance capability to different input transformations such as translation, rotation, and scaling. To effectively capture multi-scale global context, a pyramid convolution structure is attached to the concatenated output of two residual OTConv pathways. The extensive experimental evaluations show that ROTConvPCE-mv outperforms several state-of-the-art methods with a large margin regarding recognition accuracy, robustness, and fault-tolerance. Practical suggestions and hints are summarized throughout this paper to facilitate the effective recognition using tactile sensory data.