Accurate non-maximum suppression for object detection in high-resolution remote sensing images

Non-maximum suppression (NMS) is widely adopted as a post-processing step in the state-of-the-art object detection pipelines to merge the nearby detections around one object. However, its performance is affected by objects that are highly overlapped with each other, and its localization accuracy depends solely on the highest scored detection. To tackle this, an accurate NMS method is proposed in this letter, which gradually merges the highly overlapped detections in an iterative way. In each iteration, detections overlapped with the highest scored one are grouped with a harder threshold to regress for a new proposal, and then the scores within the group are softly suppressed. This process is recursively applied on the remaining detections. The proposed method can not only detect more overlapped objects, but also achieve better object localization accuracy. Experimental results demonstrate that this simple and unsupervised method can gain obvious performance improvement on the majority of classes, compared with the state-of-the-art NMS methods.     

A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images

Road segmentation from high-resolution visible remote sensing images provides an effective way for automatic road network forming. Recently, deep learning methods based on convolutional neural networks (CNNs) are widely applied in road segmentation. However, it is a challenge for most CNN-based methods to achieve high segmentation accuracy when processing high-resolution visible remote sensing images with rich details. To handle this problem, we propose a road segmentation method based on a Y-shaped convolutional network (indicated as Y-Net). Y-Net contains a two-arm feature extraction module and a fusion module. The feature extraction module includes a deep downsampling-to-upsampling sub-network for semantic features and a convolutional sub-network without downsampling for detail features. The fusion module combines all features for road segmentation. Benefiting from this scheme, the Y-Net can well segment multi-scale roads (both wide and narrow roads) from high-resolution images. The testing and comparative experiments on a public dataset and a private dataset show that Y-Net has higher segmentation accuracy than four other state-of-art methods, FCN (Fully Convolutional Network), U-Net, SegNet, and FC-DenseNet (Fully Convolutional DenseNet). Especially, Y-Net accurately segments contours of narrow roads, which are missed by the comparative methods.     

A multi-model ensemble method based on convolutional neural networks for aircraft detection in large remote sensing images

Aircraft detection is one of hot issues of remote sensing (RS) image interpretation. Different from existing methods which aim at detecting objects in regions of interest (ROI) of airports, in this work, we propose a novel and effective aircraft detection framework based on convolutional neural networks (CNN) to detect multi-scale targets in extremely large and complicated scenes. In particular, we design a constrained EdgeBoxes (CEdgeBoxes) approach to generate a modest number of target candidates quickly and precisely. Then, in order to overcome the drawbacks of using handcrafted features and traditional classifiers, a modified GoogLeNet combined with Fast Region-based CNN (R-CNN) is designed to extract useful features from RS images and then detect various kinds of multi-scale aircrafts. Most importantly, we propose a new multi-model ensemble method to decrease the false alarm rate (FAR) caused by the imbalanced distribution of complicated targets and backgrounds. Thorough experiments, which are conducted on the dataset acquired by QuickBird with a resolution of 0.6 m, demonstrate our method can effectively detect aircrafts in large scenes with low FARs.     

Learning a two-stage CNN model for multi-sized building detection in remote sensing images

Though tremendous strides have been made in building recognition, to handle multi-sized buildings is fundamental for all building detection pipelines. We explore the reason of the problem in detecting the multi-sized buildings and find that most convolutional neural network (CNN) based recognition approaches aim to be scale-invariant. The cues for recognizing a 3 pixels tall building are fundamentally different than those for recogjnizing a 300 pixels tall building. To tackle this problem, we design a novel two-stage building detection model which contains the region proposal stage and the classification stage. In the region proposal stage, we propose a novel Multi-size Fusion Region Proposal Network (MFRPN) for extracting the feature of various size building and generating wide size range of region proposals. In the classification stage, a deep CNN model is used to distinguish whether the generated region proposals are building regions or not. In order to achieve better performance, we present an improved block voting algorithm by introducing a dynamic weighting strategy which can obtain a more robust classification result increasing the classification accuracy of the region proposals. We attribute this to robust Experimental results on the challenging VHR dataset indicate that our model has a great performance.     

Object detection in optical remote sensing images by integrating object-to-object relationships

ABSTRACT

In recent years, deep-learning-based methods for remote sensing image interpretation have undergone rapid development, due to the increasing amount of image data and the advanced techniques of machine learning. The abundant spatial and contextual information within the images is helpful to improve the interpretation performance. However, the contextual information is ignored by most of the current deep-learning-based methods. In this letter, we explore the contextual information by taking advantage of the object-to-object relationship. Then, the feature representation of the individual objects can be enhanced. To be specific, we first build a knowledge database which reveals the relationship between different categories and generate a region-to-region graph that indicates the relationship between different regions of interest (RoIs). For each RoI, the features of its related regions are then combined with the original region features, and the fused features are finally used for object detection. The experiments conducted on a public ten-class object detection dataset demonstrate the validity of the proposed method.     

Vehicle detection in remote sensing images using denoizing-based convolutional neural networks

Vehicle detection in remote sensing images is a tough task and of great significance due to the fast increasing number of vehicles occurring in big cities. Recently, convolutional neural network (CNN)-based methods have achieved excellent performance in classification task due to their powerful abilities in high-level feature extraction. However, overfitting is a serious problem in CNN when applying complicated fully-connected layers, especially when the quantity of training samples is limited. In order to tackle this problem, a denoizing-based CNN called DCNN is proposed in this letter. More specially, a CNN with one fully-connected layer is pre-trained first for feature extraction. After that, features of this fully-connected layer are corrupted and used to pre-train a stacked denoizing autoencoder (SDAE) in an unsupervised way. Then, the pre-trained SDAE is added into the CNN as the fully-connected layer. After fine-tuning, DCNN can make the extracted features more robust and the detecting rate higher. With the help of our proposed locating method, vehicles can be detected effectively even when they are parked in a residential area. Comparative experiments demonstrate that our method has achieved state-of-the-art performance.     

Adaptive conditional random field classification framework based on spatial homogeneity for high-resolution remote sensing imagery

ABSTRACT

Since the conditional random field (CRF) model can integrate spectral and spatial-contextual information of high spatial resolution (HSR) remote sensing images in a unified framework, it becomes an effective approach to optimize the classification results. However, the results of traditional classification methods based on the CRF are sensitive to the parameters. In this paper, an adaptive conditional random field (ACRF) model is designed to utilize the spatial information more flexibly and improve the accuracy. In the ACRF, the spatial homogeneity is employed to achieve adaptive parameters control, which can evaluate the effect of the unary potentials and pairwise potentials of different pixels. Two datasets are used in the experiments, and the results demonstrate that the proposed method can improve the classification accuracy, alleviate salt-and-pepper noises, and retain detailed information. Compared with other methods, ACRF shows a better performance for HSR image classification, integrating the spatial-contextual and spectral information.     

A novel building change index for automatic building change detection from high-resolution remote sensing imagery

In pace with rapid urbanization, urban areas in many countries are undergoing huge changes. The large spectral variance and spatial heterogeneity within the ‘buildings’ land cover class, as well as the similar spectral properties between buildings and other urban structures, make building change detection a challenging problem. In this work, we propose a set of novel building change indices (BCIs) by combining morphological building index (MBI) and slow feature analysis (SFA) for building change detection from high-resolution imagery. MBI is a recently developed automatic building detector for high-resolution imagery, which is able to highlight building components but simultaneously suppress other urban structures. SFA is an unsupervised learning algorithm that can discriminate the changed components from the unchanged ones for multitemporal images. By effectively integrating the information from MBI and SFA, the building change components can be automatically generated. Experiments conducted on the QuickBird 2002–2005 data-set are used to validate the effectiveness of the proposed building change detection framework.     

Deep multi-feature fusion network for remote sensing images

ABSTRACT

Learning discriminative and robust features is crucial in remote sensing image processing. Many of the currently used approaches are based on Convolutional Neural Networks (CNNs). However, such approaches may not effectively capture various different semantic objects of remote sensing images. To overcome this limitation, we propose a novel end-to-end deep multi-feature fusion network (DMFN). DMFN combines two different deep architecture branches for feature representations; the global and local branch. The global branch, which consists of three losses, is used to learn discriminative features from the whole image. The local branch is then used in the partitioning of the entire image into multiple strips in order to obtain local features. The two branches are then combined, used to learn fusion feature representations for the image. The proposed method is an end-to-end framework during training. Comprehensive validation experiments on two public datasets indicate that relative to existing deep learning approaches, this strategy is superior for both retrieval and classification tasks.     

A generally applicable noise-estimating method for remote sensing images

Local mean and local standard deviations (LMLSD), which is one of the most widely used methods for estimating noise in remote sensing images, is suitable only for the images with many homogeneous regions. For those composed of heterogeneous features and textures, it may cause overestimation of noise. Edge-extracted local standard deviations (EELSD) method performs better than LMLSD in most instances, but it still cannot work out the accurate noise estimation in most heterogeneous images. Spectral and spatial de-correlation (SSDC) is an effective noise-estimation method for hyperspectral images. However, it cannot be applied to single-band or multispectral images because of the use of pixel spectral information in the calculation process. In this article, a new noise-estimating method for remote sensing images, which is based on the principle of LMLSD and has made improvements in three aspects, is proposed. The new method has been tested with several Airborne Visible Infrared Imaging Spectrometer images with different degrees of uniformity. Compared with LMLSD and EELSD, the results of the improved method are more accurate, stable, and applicable in terms of complex land cover types. Furthermore, in contrast to SSDC, this method is suitable not only for hyperspectral images but also for single-band and multispectral images.     

Change detection in multispectral images based on multiband structural information

Change vector analysis (CVA) and spectral angle mapper (SAM) are usually used to generate difference image in change detection of multispectral images. Although CVA and SAM can describe the difference between multispectral images, they are defined mathematically and lack support of human visual system (HVS) theory. Advanced structural similarity (ASSIM) complies with the pattern that human perceives the changes occurred in an objective scene. Nevertheless, ASSIM was designed for single band images and cannot be used for extracting multiband structural information from multispectral images directly. Therefore, we first propose two strategies to extract multiband structural information from multiband images. Then, we propose the approaches based on multiband structural information for change detection in multispectral images. Experimental results from one semisynthetic data set and two real data sets acquired by Sentinel-2A and QuickBird satellites validate the effectiveness of the proposed approaches.     

Aircraft detection in remote sensing images using centre-based proposal regions and invariant features

ABSTRACT

Aircraft detection in remote sensing imagery has drawn much attention in recent years, which plays an important role in various military and civil applications. While many advanced works have been developed with powerful learning algorithms in natural images, there still lacks an effective one to detect aircraft precisely in remote sensing images, especially in some complicated conditions. In this paper, a novel method is designed to detect aircraft precisely, named aircraft detection using Centre-based Proposal regions and Invariant Features (CPIF), which can handle some difficult image deformations, especially rotations. Our framework mainly contains three steps. Firstly, we propose an algorithm to extract proposal regions from remote sensing imagery. Secondly, an ensemble learning classifier with the rotation-invariant HOG is trained for aircraft classification. Lastly, we detect aircraft in remote sensing images by combining the products of the above steps. The proposed method is evaluated on a public dataset RSOD and the results are performed to demonstrate the superiority and effectiveness in comparison with the state-of-the-art methods.     

An adaptively weighted multi-feature method for object-based change detection in high spatial resolution remote sensing images

ABSTRACT

In this letter, an adaptively weighted multi-feature-based method for unsupervised object-based change detection in high-resolution remote sensing images is proposed. First, a sample selection strategy using fuzzy c-means is designed to obtain high precision pseudo-samples in an unsupervised way. Second, the multiple candidate features are categorized into spectral, geometric and textural groups and two kinds of weights are involved taking into account different contributions. The within-group weights for each feature can be calculated based on single-feature distribution curve without any prior distribution assumption, and the between-group weights for each group are decided by scatter matrices. Third, the weighted multi-feature method is used to generate a reliable difference image which is directly clustered to obtain the final change map. Compared with the other five state-of-the-art methods, the experimental results on two datasets demonstrate the effectiveness and superiority of the proposed method.     

A method based on the adaptive cuckoo search algorithm for endmember extraction from hyperspectral remote sensing images

In this letter, an endmember extraction approach based on an adaptive cuckoo search (ACS) algorithm (ACSEE) is proposed for hyperspectral remote sensing imagery. In the proposed algorithm, the problem of endmember extraction is transformed into combinatorial optimization of the candidate endmembers. The effectiveness of the cuckoo search algorithm is demonstrated by its good balance between exploitation of Lévy flights and random walk, which leads the algorithm to effectively explore the solution space and locate potential solutions to avoid falling into local optima. Furthermore, to improve the convergence characteristics of the original cuckoo search algorithm, a new strategy combined with historical information is proposed to accelerate the search process by adjusting the step size of the Lévy flights. The results of experiments conducted using simulated data and well-known hyperspectral remote sensing data indicate that the proposed ACS algorithm can be used as an alternative tool to solve the problem of endmember extraction on account of its robust stability and guarantee of optimal performance.     

A novel local variance-based filtering method for denoising remote sensing images

In this paper, a novel filtering method is designed for denoising remote sensing image. Firstly, the image domain of noisy image is partitioned into blocks for estimating the variance of Gaussian white noise. Secondly, based on the fact that the variance of the textural region is always larger than that of the homogeneous region, the noisy image is roughly divided into homogeneous and textural regions. Thirdly, a novel filter is designed and is used to reduce the noises. To this end, adaptive windows with appropriate shape and size are selected for each pixel. With the pixels in the window(s), the noise intensity of the central pixel is estimated and further qualified as a noise level. Based on noise levels, pixel values within the filter window are first updated and then filtered by using the proposed filter. Compared with other filtering methods, better performance is achieved in both noise smoothing and detail conserving.     

Image segmentation using mean shift for extracting croplands from high-resolution remote sensing imagery

A new image segmentation algorithm based on mean shift (MS) is proposed, with an objective to single out croplands in high-resolution remote sensing imagery (HRRSI). The algorithm is composed of two parts. First, in order to improve the discontinuity preserving smoothing of MS filtering for cropland HRRSI, normal and uniform kernels are used to filter inner fields and boundary areas, respectively. A new spectral bandwidth estimation is also developed for better suppressing intra-field variation. Second, a two-stage region-merging technique, with the second stage combining mutual best-fit rule and iterative thresholding, is implemented. An HRRSI scene is used for validation, the results of which indicate good performance of our method.     

A rate control algorithm for encoding multispectral images from remote sensing satellites

This letter presents a rate allocation algorithm that can be used when the Consultative Committee for Space Data Systems (CCSDS) lossy image compressor is applied to encode multispectral images. This algorithm is a numerical solution to the rate control problem when multiple information sources are compressed independently. Given the rate distortion curves and a total bit rate, the algorithm performs an efficient search to find the best rate distribution among the components that produces the lowest distortion. It is shown that the algorithm introduced here is asymptotically optimal. Furthermore, the relationship between performance and complexity can be easily controlled. The efficiency of the algorithm is tested when CCSDS compressor is used to encode the Karhunen and Loève Transform components of multispectral images. The test is performed for a group of images from different multispectral sensors. The convergence speed and the rate distortion curve are evaluated. Numerical results are compared to the performance of the same compressor when it uses the rate control algorithm based on the Gaussian rate distortion function. It is shown that the proposed algorithm can produce improvements over 2 dB in the Peak Signal-to-Noise Ratio even with a reduced computational complexity.