Hyperspectral classification using an adaptive spectral-spatial kernel-based low-rank approximation

This paper presents a novel adaptive spectral-spatial kernel-based low-rank approximation method for spectral-spatial hyperspectral image (HSI) classification. In the first of three steps of the proposed method, superpixel and image patch are used together to calculate the weights in the homogeneous region. Second, an adaptive spectral-spatial kernel is defined to capture the spectral and spatial feature of HSIs. In the final step, an adaptive spectral-spatial kernel and low-rank approximation are integrated into a decision model to perform HSI classification. Extensive experimental results on Indian Pines and Pavia University demonstrate the superiority of the proposed classifier when compared with other competing classifiers.     

Discriminative spatial-spectral manifold embedding for hyperspectral image classification

In hyperspectral images (HSI) classification, it is important to combine multiple features of a certain pixel in both spatial and spectral domains to improve the classification accuracy. To achieve this goal, this article proposes a novel spatial-spectral feature dimensionality reduction algorithm based on manifold learning. For each feature, a graph Laplacian matrix is constructed based on discriminative information from training samples, and then the graph Laplacian matrices of the various features are linearly combined using a set of empirically defined weights. Finally, the feature mapping is obtained by an eigen-decomposition problem. Based on the classification results of the public Indiana Airborne Visible Infrared Imaging Spectrometer dataset and Texas Hyperspectral Digital Imagery Collection Experiment data set, the technical accuracies show that our method achieves superior performance compared to some representative HSI feature extraction and dimensionality reduction algorithms.     

A mixture generative adversarial network with category multi-classifier for hyperspectral image classification

ABSTRACT

Hyperspectral image (HSI) classification is one of the core techniques in HSI processing. In order to solve the problem of scarcity of labelled samples, a novel HSI classification framework based on mixture generative adversarial networks (MGAN) is proposed in this letter. Firstly, to overcome the drawback that MGAN cannot be directly applied for classification, a category multi-classifier is introduced into MGAN to conduct the classification task. Due to 3D convolutional neural network (3DCNN) is adopted as the category multi-classifier, the spatial information and local 3D data structure of HSI can be captured for classification, and the proposed framework is named as MGAN-3DCNN. Accordingly, a new loss function is constructed. Secondly, since the new loss function is a tripartite game which is difficult to achieve Nash equilibrium, a step-by-step training strategy is designed to solve the related minimax problem. Experiments on two HSI data sets demonstrate that the proposed MGAN-3DCNN greatly alleviates the over-fitting problem and improves the robustness of HSI classification in small-size samples.     

RA-GCN: Graph convolutional network for disease prediction problems with imbalanced data

Disease prediction is a well-known classification problem in medical applications. Graph Convolutional Networks (GCNs) provide a powerful tool for analyzing the patients’ features relative to each other. This can be achieved by modeling the problem as a graph node classification task, where each node is a patient. Due to the nature of such medical datasets, class imbalance is a prevalent issue in the field of disease prediction, where the distribution of classes is skewed. When the class imbalance is present in the data, the existing graph-based classifiers tend to be biased towards the major class(es) and neglect the samples in the minor class(es). On the other hand, the correct diagnosis of the rare positive cases (true-positives) among all the patients is vital in a healthcare system. In conventional methods, such imbalance is tackled by assigning appropriate weights to classes in the loss function which is still dependent on the relative values of weights, sensitive to outliers, and in some cases biased towards the minor class(es). In this paper, we propose a Re-weighted Adversarial Graph Convolutional Network (RA-GCN) to prevent the graph-based classifier from emphasizing the samples of any particular class. This is accomplished by associating a graph-based neural network to each class, which is responsible for weighting the class samples and changing the importance of each sample for the classifier. Therefore, the classifier adjusts itself and determines the boundary between classes with more attention to the important samples. The parameters of the classifier and weighting networks are trained by an adversarial approach. We show experiments on synthetic and three publicly available medical datasets. Our results demonstrate the superiority of RA-GCN compared to recent methods in identifying the patient’s status on all three datasets. The detailed analysis of our method is provided as quantitative and qualitative experiments on synthetic datasets.     

Hyperspectral image denoising via low-rank matrix recovery

Based on low-rank matrix recovery theory, we propose a novel method to remove the hyperspectral image noise. To robustly handle the outliers in hyperspectral images, we first build a hybrid noise model for the hyperspectral images. Then, the noise removal is achieved via two stages. In the first stage, the main fine-image features are first separated from the noise via principal component analysis (PCA) due to its good performance in signal/noise decorrelation. In the second stage, the noise removal is conducted in the low-energy PCA channels through low-rank matrix recovery because of its strong capability in dealing with badly corrupted matrices. The experimental results on both simulated and real data validated the effectiveness of the proposed method both visually and quantitatively.     

Marginal discriminant analysis using support vectors for dimensionality reduction of hyperspectral data

Feature extraction (FE) is an efficient pre-processing step in hyperspectral image (HSI) classification. This article proposes a novel supervised FE method based on graph embedding framework (GEF). This method, which is called marginal discriminant analysis using support vectors (MDSV), can be used as a linear dimensionality reduction approach. The proposed method constructs inner and support graphs to capture both global and local structures of data points. The global geometrical structure of data in each class is described by the inner graph. The support graph uses support vectors (SVs) to detect the local inter-class structure of different classes. Incorporating these graphs enables MDSV to maximize the margin between classes in the projected space. Implementation of MDSV on four benchmark hyperspectral datasets confirms its efficiency as an appropriate pre-processing method before classification of HSIs.     

A spectral-spatial SVM-based multi-layer learning algorithm for hyperspectral image classification

Conventional Markov random field (MRF) or Graph cut (GC) based support vector machine (SVM) methods for hyperspectral image (HSI) classification use MRF or GC to adjust spectral-based SVM results to increase the spatial consistency in an unsupervised way, thus, the pixels on the border and small-sized regions may be misclassified. In this letter, we propose a new framework of spectral-spatial SVM based multi-layer learning algorithm (SSMLL) for HSI classification. In the first layer of SSMLL, the spectral-based SVM is adopted to process the original HSI datasets; the nonlinear mapping is used to scale the first layer output and enhance the nonlinear structure in the second layer; in the last layer, the spatial information is incorporated into the SVM to obtain the final classification results in a supervised way. Experimental results show that the proposed SSMLL framework provides superior classification accuracy when compared to several state-of-the-art spectral-spatial SVM-based algorithms.     

Random walker with spatial constraint and hyperspectral image classification

In this article, a spatially constrained random walker approach is proposed for hyperspectral image (HSI) classification. This proposed method uses both spectral and spatial information. Image pixels are partitioned into two sets: a labelled set and an unlabelled set. The proposed method aims to label all the unlabelled pixels. The proposed technique consists of two steps. In the first step, random walker computes the posterior probability that an unlabelled pixel has the same label as a labelled pixel by using the spectral information. In order to improve the classification accuracy, Markov random fields is applied to account for spatial information in the second step. Evaluation of the developed method is done on HSIs. Experimental results are compared with those obtained using other HSI classification methods. The proposed approach performs better than other ones in terms of classification accuracy.     

Application of a homogenous patch mean kernel with within-class collaborative representation for hyperspectral imagery classification

Spectral–spatial-based classification methods demonstrate satisfying performance for hyperspectral imagery (HSI) classification. In this letter, in order to make full use of spectral and contexture information with simultaneously considering within-class information, we propose a new algorithm for HSI classification based on within-class collaborative representation and column generation (CG) strategy. The proposed accelerated homogeneous patch mean kernel (HPMK) can automatically assign a homogeneous patch for the target sample and represent the similarities between training set and assigned homogeneous patch in kernel feature space based on CG strategy. Further, for including intra-class information and improve classification efficiency, within-class collaborative representation classification (WCRC) is incorporated into new feature space to enhance the classification performance. Experiments on two real HSI data sets demonstrate that the proposed algorithm presents satisfying results in terms of classification accuracy and efficiency.     

Interferogram denoising using an iteratively refined nonlocal InSAR filter

Interferogram denoising is important for height reconstruction and deformation measurement. For high-resolution interferograms over heterogeneous areas, local biases and resolution losses may appear due to the violation of the local stationarity assumption. To address this problem, an iteratively refined nonlocal filter is proposed, whose estimation is refined iteratively by jointly using the amplitude, interferometric phase, coherence, and the pre-estimated patches. For nonlocal interferometric synthetic aperture radar (InSAR) techniques, the similarity of two pixels is computed via their surrounding patches, followed by the maximum likelihood weighted averaging of similar pixels. In this letter, the outliers in the search window is identified before the weight calculation. If the normalized probability density function (NPDF) of the central pixel and other surrounding pixels in the search window is less than the preset threshold, the pixel will be assigned with the minimum weight in the search window. Moreover, the denoising weight is calculated not only depending on the probabilistic patch-based (PPB) similarity, but also the coherence of the pixels in the search window. Both simulated and real data experiments are used to validate the effectiveness of the proposed method.     

Semi-supervised classification of radiology images with NoTeacher: A teacher that is not mean

Deep learning models achieve strong performance for radiology image classification, but their practical application is bottlenecked by the need for large labeled training datasets. Semi-supervised learning (SSL) approaches leverage small labeled datasets alongside larger unlabeled datasets and offer potential for reducing labeling cost. In this work, we introduce NoTeacher, a novel consistency-based SSL framework which incorporates probabilistic graphical models. Unlike Mean Teacher which maintains a teacher network updated via a temporal ensemble, NoTeacher employs two independent networks, thereby eliminating the need for a teacher network. We demonstrate how NoTeacher can be customized to handle a range of challenges in radiology image classification. Specifically, we describe adaptations for scenarios with 2D and 3D inputs, with uni and multi-label classification, and with class distribution mismatch between labeled and unlabeled portions of the training data. In realistic empirical evaluations on three public benchmark datasets spanning the workhorse modalities of radiology (X-Ray, CT, MRI), we show that NoTeacher achieves over 90–95% of the fully supervised AUROC with less than 5–15% labeling budget. Further, NoTeacher outperforms established SSL methods with minimal hyperparameter tuning, and has implications as a principled and practical option for semi-supervised learning in radiology applications.     

Multi-modal classification of neurodegenerative disease by progressive graph-based transductive learning

Graph-based transductive learning (GTL) is a powerful machine learning technique that is used when sufficient training data is not available. In particular, conventional GTL approaches first construct a fixed inter-subject relation graph that is based on similarities in voxel intensity values in the feature domain, which can then be used to propagate the known phenotype data (i.e., clinical scores and labels) from the training data to the testing data in the label domain. However, this type of graph is exclusively learned in the feature domain, and primarily due to outliers in the observed features, may not be optimal for label propagation in the label domain. To address this limitation, a progressive GTL (pGTL) method is proposed that gradually finds an intrinsic data representation that more accurately aligns imaging features with the phenotype data. In general, optimal feature-to-phenotype alignment is achieved using an iterative approach that: (1) refines inter-subject relationships observed in the feature domain by using the learned intrinsic data representation in the label domain, (2) updates the intrinsic data representation from the refined inter-subject relationships, and (3) verifies the intrinsic data representation on the training data to guarantee an optimal classification when applied to testing data. Additionally, the iterative approach is extended to multi-modal imaging data to further improve pGTL classification accuracy. Using Alzheimer's disease and Parkinson's disease study data, the classification accuracy of the proposed pGTL method is compared to several state-of-the-art classification methods, and the results show pGTL can more accurately identify subjects, even at different progression stages, in these two study data sets.     

Contrastive rendering with semi-supervised learning for ovary and follicle segmentation from 3D ultrasound

Segmentation of ovary and follicles from 3D ultrasound (US) is the crucial technique of measurement tools for female infertility diagnosis. Since manual segmentation is time-consuming and operator-dependent, an accurate and fast segmentation method is highly demanded. However, it is challenging for current deep-learning based methods to segment ovary and follicles precisely due to ambiguous boundaries and insufficient annotations. In this paper, we propose a contrastive rendering (C-Rend) framework to segment ovary and follicles with detail-refined boundaries. Furthermore, we incorporate the proposed C-Rend with a semi-supervised learning (SSL) framework, leveraging unlabeled data for better performance. Highlights of this paper include: (1) A rendering task is performed to estimate boundary accurately via enriched feature representation learning. (2) Point-wise contrastive learning is proposed to enhance the similarity of intra-class points and contrastively decrease the similarity of inter-class points. (3) The C-Rend plays a complementary role for the SSL framework in uncertainty-aware learning, which could provide reliable supervision information and achieve superior segmentation performance. Through extensive validation on large in-house datasets with partial annotations, our method outperforms state-of-the-art methods in various evaluation metrics for both the ovary and follicles.     

Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency

Despite that Convolutional Neural Networks (CNNs) have achieved promising performance in many medical image segmentation tasks, they rely on a large set of labeled images for training, which is expensive and time-consuming to acquire. Semi-supervised learning has shown the potential to alleviate this challenge by learning from a large set of unlabeled images and limited labeled samples. In this work, we present a simple yet efficient consistency regularization approach for semi-supervised medical image segmentation, called Uncertainty Rectified Pyramid Consistency (URPC). Inspired by the pyramid feature network, we chose a pyramid-prediction network that obtains a set of segmentation predictions at different scales. For semi-supervised learning, URPC learns from unlabeled data by minimizing the discrepancy between each of the pyramid predictions and their average. We further present multi-scale uncertainty rectification to boost the pyramid consistency regularization, where the rectification seeks to temper the consistency loss at outlier pixels that may have substantially different predictions than the average, potentially due to upsampling errors or lack of enough labeled data. Experiments on two public datasets and an in-house clinical dataset showed that: 1) URPC can achieve large performance improvement by utilizing unlabeled data and 2) Compared with five existing semi-supervised methods, URPC achieved better or comparable results with a simpler pipeline. Furthermore, we build a semi-supervised medical image segmentation codebase to boost research on this topic: https://github.com/HiLab-git/SSL4MIS.     

Salient feature extraction for hyperspectral image classification

Hyperspectral image (HSI) classification has been a vibrant area of research in recent years. In this letter, a salient feature extraction method is designed to improve the HSI classification performance. First, three consecutive bands of HSI are decomposed into several elements in order to better extract salient feature. Second, two measures of contrast that rate the uniqueness and the spatial distribution of these elements are computed. Thirdly, two measures of contrast are combined to form the saliency value of an element. The final saliency of a pixel is formed by a weighted linear combination of its surrounding elements saliency. Finally, three band sliding window is used over the spectral dimension of HSI and salient features extracted from each three adjacent bands are stacked to form the final features. The important characteristic of the proposed approach is that both local spatial information and global spatial information are taken into account in the process of feature extraction. Experimental results on two real HSI datasets demonstrate that the designed feature extraction method could improve the classification performance.     

A structural similarity-based label-smoothing algorithm for the post-processing of land-cover classification

Post-processing is able to achieve a satisfactory classification performance with a low cost and simple assumption, making it widely used in the refinement of classification maps. In this study, a novel structural similarity-based label-smoothing algorithm is developed for the post-processing of land-cover classification. Inspired by the non-local (NL) means algorithm, the proposed algorithm assigns different voting weights to the neighbouring pixels for the identification of the central pixel. Here, the voting weight of a specific neighbouring pixel depends on its structural similarity to the central pixel. In this paper, two measurements are proposed to evaluate the similarity between pixels: (1) a consistency criterion; and (2) a histogram similarity criterion. The proposed algorithm was tested on three remote-sensing images. The experimental results confirm that the proposed algorithm reduces the classification noise and preserves the detail and structural information at the same time. Compared to the traditional post-processing approaches (e.g., majority voting), the proposed algorithm exhibits a more satisfactory performance.     

Discriminative error prediction network for semi-supervised colon gland segmentation

Pixel-wise error correction of initial segmentation results provides an effective way for quality improvement. The additional error segmentation network learns to identify correct predictions and incorrect ones. The performance on error segmentation directly affects the accuracy on the test set and the subsequent self-training with the error-corrected pseudo labels. In this paper, we propose a novel label rectification method based on error correction, namely ECLR, which can be directly added after the fully-supervised segmentation framework. Moreover, it can be used to guide the semi-supervised learning (SSL) process, constituting an error correction guided SSL framework, called ECGSSL. Specifically, we analyze the types and causes of segmentation error, and divide it into intra-class error and inter-class error caused by intra-class inconsistency and inter-class similarity problems in segmentation, respectively. Further, we propose a collaborative multi-task discriminative error prediction network (DEP-Net) to highlight two error types. For better training of DEP-Net, we propose specific mask degradation methods representing typical segmentation errors. Under the fully-supervised regime, the pre-trained DEP-Net is used to directly rectify the initial segmentation results of the test set. While, under the semi-supervised regime, a dual error correction method is proposed for unlabeled data to obtain more reliable network re-training. Our method is easy to apply to different segmentation models. Extensive experiments on gland segmentation verify that ECLR yields substantial improvements based on initial segmentation predictions. ECGSSL shows consistent improvements over a supervised baseline learned only from labeled data and achieves competitive performance compared with other popular semi-supervised methods.     

Automated three-dimensional reconstruction and morphological analysis of dendritic spines based on semi-supervised learning

A dendritic spine is a small membranous protrusion from a neuron''s dendrite that typically receives input from a single synapse of an axon. Recent research shows that the morphological changes of dendritic spines have a close relationship with some specific diseases. The distribution of different dendritic spine phenotypes is a key indicator of such changes. Therefore, it is necessary to classify detected spines with different phenotypes online. Since the dendritic spines have complex three dimensional (3D) structures, current neuron morphological analysis approaches cannot classify the dendritic spines accurately with limited features. In this paper, we propose a novel semi-supervised learning approach in order to perform the online morphological classification of dendritic spines. Spines are detected by a new approach based on wavelet transform in the 3D space. A small training data set is chosen from the detected spines, which has the spines labeled by the neurobiologists. The remaining spines are then classified online by the semi-supervised learning (SSL) approach. Experimental results show that our method can quickly and accurately analyze neuron images with modest human intervention.OCIS codes: (100.0100) Image processing, (100.5010) Pattern recognition, (100.6890) Three-dimensional image processing     

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Supervised deep learning-based methods yield accurate results for medical image segmentation. However, they require large labeled datasets for this, and obtaining them is a laborious task that requires clinical expertise. Semi/self-supervised learning-based approaches address this limitation by exploiting unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images and achieve high performance in classification tasks on popular natural image datasets like ImageNet. In pixel-level prediction tasks such as segmentation, it is crucial to also learn good local level representations along with global representations to achieve better accuracy. However, the impact of the existing local contrastive loss-based methods remains limited for learning good local representations because similar and dissimilar local regions are defined based on random augmentations and spatial proximity; not based on the semantic label of local regions due to lack of large-scale expert annotations in the semi/self-supervised setting. In this paper, we propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information obtained from pseudo-labels of unlabeled images alongside limited annotated images with ground truth (GT) labels. In particular, we define the proposed contrastive loss to encourage similar representations for the pixels that have the same pseudo-label/GT label while being dissimilar to the representation of pixels with different pseudo-label/GT label in the dataset. We perform pseudo-label based self-training and train the network by jointly optimizing the proposed contrastive loss on both labeled and unlabeled sets and segmentation loss on only the limited labeled set. We evaluated the proposed approach on three public medical datasets of cardiac and prostate anatomies, and obtain high segmentation performance with a limited labeled set of one or two 3D volumes. Extensive comparisons with the state-of-the-art semi-supervised and data augmentation methods and concurrent contrastive learning methods demonstrate the substantial improvement achieved by the proposed method. The code is made publicly available at https://github.com/krishnabits001/pseudo_label_contrastive_training.