Scene classification via latent Dirichlet allocation using a hybrid generative/discriminative strategy for high spatial resolution remote sensing imagery

In order to capture the high-level concepts in high spatial resolution (HSR) remote sensing imagery, scene classification based on a latent Dirichlet allocation (LDA) model, a generative topic model, is a practical method to bridge the semantic gaps between the low-level features and the high-level concepts of HSR imagery. In the previous work, LDA has been considered as a scene classifier, namely C-LDA, and multiple LDA models for each scene class are built separately, where the scene class is determined by a maximum likelihood rule. The C-LDA strategy disregards the correlations between the generative topic spaces of the different scene classes. In this letter, two novel strategies of scene classification based on LDA are proposed to consider the correlations between the generative topic spaces of the different scene classes by sharing the topic spaces for all the scene classes. One of the proposed strategies utilizes LDA as part of the classifier, namely P-LDA, which generates the topic space from all the training images. A discriminative classifier (e.g., support vector machine, SVM) is also employed as the other classification part of P-LDA. The other proposed strategy employs LDA as the topic feature extractor, namely F-LDA, which generates the topic space from all the training and test images, and utilizes a discriminative classifier to classify the topic features. The experimental results using aerial orthophotographs show that the performances of the two proposed strategies for scene classification based on LDA are better than the traditional C-LDA method.     

FDTD Simulations for Ultrasound Propagation in a 2-D Breast Model

To increase the survival rates of patients with breast cancer, an ultrasound imaging system must detect tumors when they are small, with a diameter of 5 mm or less. This requires an understanding of how propagation of ultrasound energy is affected by the complex structure of the breast. In this paper, a Finite-Difference Time-Domain (FDTD) method is developed to simulate ultrasound propagation in a two-dimensional model of the human breast. The FDTD simulations make it possible to better understand the behavior of an ultrasound signal in the breast. For example, here the simulations are used to investigate the effect of fat lobes adjacent to the skin layer in a simple breast model. Experimental work performed at the University of Pennsylvania has shown that strong refraction caused by the fat lobes results in nulls in the forward transmitted field. This result was duplicated with the FDTD simulations, and it was shown that the effect of refraction is clearly evident for energy exiting the breast. The existence of strong refraction has a significant impact on ultrasound imaging since it implies that an imaging method based on a weak scattering assumption is unlikely to work well.     

Toward real-time polyp detection using fully CNNs for 2D Gaussian shapes prediction

To decrease colon polyp miss-rate during colonoscopy, a real-time detection system with high accuracy is needed. Recently, there have been many efforts to develop models for real-time polyp detection, but work is still required to develop real-time detection algorithms with reliable results. We use single-shot feed-forward fully convolutional neural networks (F-CNN) to develop an accurate real-time polyp detection system. F-CNNs are usually trained on binary masks for object segmentation. We propose the use of 2D Gaussian masks instead of binary masks to enable these models to detect different types of polyps more effectively and efficiently and reduce the number of false positives. The experimental results showed that the proposed 2D Gaussian masks are efficient for detection of flat and small polyps with unclear boundaries between background and polyp parts. The masks make a better training effect to discriminate polyps from the polyp-like false positives. The proposed method achieved state-of-the-art results on two polyp datasets. On the ETIS-LARIB dataset we achieved 86.54% recall, 86.12% precision, and 86.33% F1-score, and on the CVC-ColonDB we achieved 91% recall, 88.35% precision, and F1-score 89.65%.     

A new image mosaicking approach for the multiple camera system of the optical remote sensing satellite GaoFen1

The GaoFen1 (GF1) optical remote sensing satellite is the first in China’s series of high-resolution civilian satellites and is equipped with four wide field of view (WFV) cameras. High accuracy image mosaicking depends on the geometrical quality of the stitched satellite imagery. This letter proposes a high accuracy image mosaicking approach based on combining multiple WFV cameras into a big virtual camera (BVC). In this method, a BVC is created according to the rigorous imaging model of a single camera, from which a final stitched image and corresponding high accuracy rational function model (RFM) can be obtained simultaneously. WFV images from GF1 and reference digital elevation models from ZiYuan3 are used to validate the proposed method. The experimental results validate that seamless mosaicking can be achieved directly from the geometrical relationship of the imaging model of the BVC and WFV camera. The proposed method is able to maintain the absolute and relative internal positioning accuracy of the original images.     

Tree crown detection and delineation in satellite images using probabilistic voting

Crop yield forecasting in a region has become an important research area due to global warming and related climate changes. Although this can be performed by available statistical information, obtaining recent and up to date data to extract reliable statistical information is not easy. Very high resolution satellite images can be used for this purpose. However, manually processing these images acquired from large regions is neither feasible nor reliable. Therefore, automated methods are needed for this purpose. In this study, we propose a novel method to help forecasting the crop yield in an orchard. The number of trees in an orchard with the size and type of each tree crown gives an approximate crop that can be harvested. Therefore, we focus on both tree crown detection and delineation for this purpose. The proposed method for tree crown detection is based on probabilistic voting. For tree crown delineation, we propose a watershed segmentation based ellipse fitting method. We tested the proposed method on 17 satellite images containing 13,476 trees. We compared the method with the classical local maxima/minima filtering and a recent method in literature using three more test images. These tests indicate the strengths and weaknesses of the proposed method.     

Retinal contour modelling to reproduce two-dimensional peripheral spherical equivalent refraction

Reproduction of the peripheral spherical equivalent refraction (SER) in the eye model is critical for investigations in myopia control. Based on the derivation of a linear relationship between SER and the vergence of the wavefront at exit pupil center, a computing method is proposed to locate the retinal points to reproduce the two-dimensional (2D) distribution of SER. The method is validated by reproducing SER maps measured on both emmetropic and myopic eyes in a realistic eye model based on measurement data. By fitting the retinal points to a general ellipsoid, the limited capability of the general ellipsoid model in reproducing the 2D map of SER is calculated and compared with original data. The high accuracy in SER reproduction and low time-cost of the proposed retinal-locating method can help significantly improve the precision and accuracy of customized wide-angle eye modelling.     

Flexible Bayesian subgroup analysis in early and confirmatory trials

Subgroup analysis is one of the most important issues in clinical trials. In confirmatory trials, it is critical to investigate consistency of the treatment effect across subgroups, which could potentially result in incorrect scientific conclusion or regulatory decision. There are many challenges and methodological complications of interpreting subgroup results beyond the regulatory setting. For the early phase or proof of concept trials, particularly in basket trials, it is also important to have reliable estimation of subgroup treatment effect in order to guide the next phase go/no-go decision making when large biases can be introduced due to small sample size or random variability. In this paper, we review several recent methods that have been proposed for subgroup analysis in the Bayesian framework to correct for bias. We present simulation results from applying various novel Bayesian hierarchical models for subgroup analysis to a phase II basket trial. For different scenarios considered, we compare the average total sample size, and frequentist-like operating characteristics of power and familywise type I error rate. We compare the precision of the model estimates of the treatment effect by assessing average relative bias and the width of the 95% credible interval for the bias. We also demonstrate flexible Bayesian hierarchical models in a case study of a phase III oncology trial for subgroup treatment effect estimation to help with regulatory decision making. Finally, we conclude our findings in the discussion section and give recommendations on how these methods could be implemented in confirmatory and early phase clinical trials.     

A novel algorithm for the objective detection of tropical cyclone centres using infrared satellite images

Tropical cyclones (TCs) are one of nature’s most destructive phenomena, and a key element in forecasting TC tracks is the ability to accurately detect TC centres. In this paper, a novel algorithm has been proposed to objectively detect TC centres using infrared satellite images. Pyramid representation and optical flow technique are utilized to construct the cloud motion wind (CMW) field of each cyclone, and thereafter the centre is determined by analysing the constructed CMW field. Ten TCs formed in the Northwestern Pacific Ocean in 2014 have been tested to evaluate the performance of the present method, and TC Halong and Rammasun were analysed in detail as instances. Experimental results comparing with forecast track derived from Unisys Weather show that the proposed method provides an accurate detection of TC centres. The present algorithm has a potential to be employed to assist forecasters to detect TC centres.     

Preliminary assessment of the Advanced Himawari Imager (AHI) measurement onboard Himawari-8 geostationary satellite

The Japan Himawari-8 geostationary satellite was successfully launched into the geosynchronous orbit around 140°E on 17 October 2014. The Advanced Himawari Imager (AHI) onboard the Himawari-8 has its channels 7–16 covering from the short- to the thermal-infrared bands, of which observations can be assimilated into a data assimilation system to improve the atmospheric analysis. Before conducting any AHI data assimilation experiments, it is the first and critical step to correctly quantify AHI bias and error variance, since these two variables are required in a data assimilation system. This study investigates the bias and the error variance of the AHI infrared measurements under clear-sky conditions by using a numerical weather prediction model and a radiative transfer model. Overall, AHI observations agree favourably with the model simulations. It is noted that channels 7–14 has a cold bias of approximately 1.0 K while the cold bias reaches around 2.0 ~ 3.0 K for the longwave Channel 15 and Channel 16. The sums of the observation and model error are about 1.5 K for three water vapour channels 8, 9, 10, and approximately 1.0 K for the remaining channels. The dependence of the bias on the scene temperature and scan angle is further investigated. It shows that a warm bias is negatively correlated with the scene temperature when the scene temperature is low for those surface-sensitive channels. Besides, linear regression tests between the bias and the sensor zenith angle shows that the fitting coefficient is less than 0.015 K/degree for all ten channels, indicating no obvious scan-angle-dependent bias.     

CNN-based estimation of pre- and post-earthquake height models from single optical images for identification of collapsed buildings

In this paper, a novel approach is proposed to identify collapsed buildings after an earthquake using pre-event satellite image as well as post-event satellite image and airborne LiDAR data. In this regard, a convolutional neural network-based method is proposed for estimating a DHM from a single satellite image. A structure reconstruction strategy is designed to improve estimated height values and objects geometry by using the local features of shallow layers and employing a progressive context fusion method. The post-event images and their corresponding LiDAR data are used to train the proposed network. Subsequently, the trained network is employed to estimate a digital height model (DHM) from the pre-event satellite image. Finally, by investigating the difference image of pre- and post-event DHMs, collapsed buildings are identified. It is observed that the quality, kappa coefficient and overall accuracy of the obtained results are 84.86%, 91.15% and 98.78%, respectively, demonstrating a promising performance of the proposed approach.     

Shot boundary detection in endoscopic surgery videos using a variational Bayesian framework

Purpose

Over the last decade, the demand for content management of video recordings of surgical procedures has greatly increased. Although a few research methods have been published toward this direction, the related literature is still in its infancy. In this paper, we address the problem of shot detection in endoscopic surgery videos, a fundamental step in content-based video analysis.

Methods

The video is first decomposed into short clips that are processed sequentially. After feature extraction, we employ spatiotemporal Gaussian mixture models (GMM) for each clip and apply a variational Bayesian (VB) algorithm to approximate the posterior distribution of the model parameters. The proper number of components is handled automatically by the VBGMM algorithm. The estimated components are matched along the video sequence via their Kullback–Leibler divergence. Shot borders are defined when component tracking fails, signifying a different visual appearance of the surgical scene.

Results

Experimental evaluation was performed on laparoscopic videos containing a variable number of shots. Performance was measured via precision, recall, coverage and overflow metrics. The proposed method was compared with GMM and a shot detection method based on spatiotemporal motion differences (MotionDiff). The results demonstrate that VBGMM has higher performance than all other methods for most assessment metrics: precision and recall >80 %, coverage: 84 %. Overflow for VBGMM was worse than MotionDiff (37 vs. 27 %).

Conclusions

The proposed method generated promising results for shot border detection. Spatiotemporal modeling via VBGMMs provides a means to explore additional applications such as component tracking.

     

Comparison of approximation methods to Kullback–Leibler divergence between Gaussian mixture models for satellite image retrieval

As a probabilistic distance between two probability density functions, Kullback–Leibler divergence is widely used in many applications, such as image retrieval and change detection. Unfortunately, for some models, e.g., Gaussian Mixture Models (GMMs), Kullback–Leibler divergence is not analytically tractable. One has to resort to approximation methods. A number of methods have been proposed to address this issue. In this article, we compare seven methods, namely Monte Carlo method, matched bound approximation, product of Gaussians, variational method, unscented transformation, Gaussian approximation and min-Gaussian approximation, for approximating the Kullback–Leibler divergence between two Gaussian mixture models for satellite image retrieval. Two experiments using two public data sets have been performed. The comparison is carried out in terms of retrieval accuracy and computational time.     

Does Low Atmospheric Pressure Independently Trigger Migraine?

Although atmospheric weather changes are often listed among the common migraine triggers, studies to determine the specific weather component(s) responsible have yielded inconsistent results. Atmospheric pressure change produces air movement, and low pressure in particular is associated with warm weather, winds, clouds, dust, and precipitation, but how this effect might generate migraine is not immediately obvious. Humans are exposed to low atmospheric pressure in situations such as ascent to high altitude or traveling by airplane in a pressurized cabin. In this brief overview, we consider those conditions and experimental data delineating other elements in the atmosphere potentially related to migraine (such as Saharan dust). We conclude that the available data suggest low atmospheric pressure unaccompanied by other factors does not trigger migraine.     

Using a white matter reference to remove the dependency of global signal on experimental conditions in SPECT analyses

Proportional scaling models are often used in functional imaging studies to remove confounding of local signals by global effects. It is generally assumed that global effects are uncorrelated with experimental conditions. However, when the global effect is estimated by the global signal, defined as the intracerebral average, incorrect inference may result from the dependency of the global signal on preexisting conditions or experimental manipulations. In this paper, we propose a simple alternative method of estimating the global effect to be used in a proportional scaling model. Specifically, by defining the global signal with reference strictly to a white matter region within the centrum semiovale, the dependency is removed in experiments where white matter is unaffected by the disease effect or experimental treatments. The increase in the ability to detect changes in regional blood flow is demonstrated in a SPECT study of healthy and ill Gulf War veterans in whom it is suspected that brain abnormalities influence the traditional calculation of the global signal. Controlling for the global effect, ill veterans have significantly lower intracerebral averages than healthy controls (P = 0.0038), evidence that choice of global signal has an impact on inference. Scaling by the modified global signal proposed here results in an increase in sensitivity leading to the identification of several regions in the insula and frontal cortex where ill veterans have significantly lower SPECT emissions. Scaling by the traditional global signal results in the loss of sensitivity to detect these regional differences. Advantages of this alternative method are its computational simplicity and its ability to be easily integrated into existing analysis frameworks such as SPM.     

Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy

In this paper, we present a new Deep Convolutional Neural Networks (CNNs) dedicated to fully automatic segmentation of Glioblastoma brain tumors with high- and low-grade. The proposed CNNs model is inspired by the Occipito-Temporal pathway which has a special function called selective attention that uses different receptive field sizes in successive layers to figure out the crucial objects in a scene. Thus, using selective attention technique to develop the CNNs model, helps to maximize the extraction of relevant features from MRI images. We have also addressed two more issues: class-imbalance, and the spatial relationship among image Patches. To address the first issue, we propose two steps: an equal sampling of images Patches and an experimental analysis of the effect of weighted cross-entropy loss function on the segmentation results. In addition, to overcome the second issue, we have studied the effect of Overlapping Patches against Adjacent Patches where the Overlapping Patches show better segmentation results due to the introduction of the global context as well as the local features of the image Patches compared to the conventionnel Adjacent Patches. Our experiment results are reported on BRATS-2018 dataset where our End-to-End Deep Learning model achieved state-of-the-art performance. The median Dice score of our fully automatic segmentation model is 0.90, 0.83, 0.83 for the whole tumor, tumor core, and enhancing tumor respectively compared to the Dice score of radiologist, that is in the range 74% – 85%. Moreover, our proposed CNNs model is not only computationally efficient at inference time, but it could segment the whole brain on average 12 seconds. Finally, the proposed Deep Learning model provides an accurate and reliable segmentation result, and that makes it suitable for adopting in research and as a part of different clinical settings.     

A jitter compensation method for spaceborne line-array imagery using compressive sampling

Attitude jitter is a problem commonly experienced by remote-sensing satellites, and deteriorates the accuracy of geo-positioning and mapping. Therefore, it is important to design a method which can handle this problem. This paper presents a compensation method using compressive sampling. In this method, the influence of attitude jitter in imagery is represented by two fluctuating signals, which are called distortion signals in this paper. Using compressive sampling, the distortion signals in the remote-sensing imagery can be estimated with a small set of ground control points (GCPs). Based on the estimated distortion signals, we can conduct compensation on the remote-sensing imagery affected by attitude jitter. The significant advantage of this method is that it requires only rational function model (RFM) and few GCPs, which makes it easier to be applied than other similar methods. Experiment with image from satellite ALOS (Advanced Land Observing Satellite) and ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiaometer) showed the feasibility of the proposed method and the effect of the distribution of GCPs.     

Simultaneous modeling of pharmacokinetics and pharmacodynamics with nonparametric kinetic and dynamic models

Three models, linked in series, can be used to analyze combined pharmacokinetic (PK) and pharmacodynamic (PD) data arising from non--steady-state experiments. A PK model relates dose to plasma drug concentration (Cp); a link model relates Cp to drug concentration at the effect site (Ce); and a PD model relates Ce to drug effect (E). All three submodels can be stated parametrically. Recently the use of a nonparametric PD submodel has been proposed (CLIN PHARMACOL THER 1984;35:733-41). In this article we use an extended nonparametric approach that represents both the PK and PD models nonparametrically, but retains a parametric link model. Cp data from several PK models and E data from several PD models were simulated. After the addition of noise to both the Cp and E data, they were analyzed by both the parametric and extended nonparametric methods. The methods were compared by how well they estimated the PD model. To assess robustness, the effect of misspecification of the PK submodel on the goodness of estimation of both methods was also compared. In the absence of model misspecification, the parametric method usually estimates the PD model better than the nonparametric method. However, this difference in the performances diminishes and even reverses when the PK model is misspecified. Because one can rarely be certain that model misspecification is absent, the nonparametric approach may offer a distinct advantage for routine analysis of PK/PD data.     

A framework for the merging of pre-existing and correspondenceless 3D statistical shape models

The construction of statistical shape models (SSMs) that are rich, i.e., that represent well the natural and complex variability of anatomical structures, is an important research topic in medical imaging. To this end, existing works have addressed the limited availability of training data by decomposing the shape variability hierarchically or by combining statistical and synthetic models built using artificially created modes of variation. In this paper, we present instead a method that merges multiple statistical models of 3D shapes into a single integrated model, thus effectively encoding extra variability that is anatomically meaningful, without the need for the original or new real datasets. The proposed framework has great flexibility due to its ability to merge multiple statistical models with unknown point correspondences. The approach is beneficial in order to re-use and complement pre-existing SSMs when the original raw data cannot be exchanged due to ethical, legal, or practical reasons. To this end, this paper describes two main stages, i.e., (1) statistical model normalization and (2) statistical model integration. The normalization algorithm uses surface-based registration to bring the input models into a common shape parameterization with point correspondence established across eigenspaces. This allows the model fusion algorithm to be applied in a coherent manner across models, with the aim to obtain a single unified statistical model of shape with improved generalization ability. The framework is validated with statistical models of the left and right cardiac ventricles, the L1 vertebra, and the caudate nucleus, constructed at distinct research centers based on different imaging modalities (CT and MRI) and point correspondences. The results demonstrate that the model integration is statistically and anatomically meaningful, with potential value for merging pre-existing multi-modality statistical models of 3D shapes.     

Locally regularized spatiotemporal modeling and model comparison for functional MRI

In this work we treat fMRI data analysis as a spatiotemporal system identification problem and address issues of model formulation, estimation, and model comparison. We present a new model that includes a physiologically based hemodynamic response and an empirically derived low-frequency noise model. We introduce an estimation method employing spatial regularization that improves the precision of spatially varying noise estimates. We call the algorithm locally regularized spatiotemporal (LRST) modeling. We develop a new model selection criterion and compare our model to the SPM-GLM method. Our findings suggest that our method offers a better approach to identifying appropriate statistical models for fMRI studies.     

Block-and-octave constraint SIFT with multi-thread processing for VHR satellite image matching

Very high-resolution satellite image matching is a challenging task due to local distortion, repetitive structures, intensity changes and low efficiency. In this letter, a novel approach based on the block and octave constraint scale-invariant feature transform (SIFT) is proposed for very high-resolution satellite image matching. Details regarding multi-thread processing are provided. First, the corresponding matching block is observed using rational function model (RFM) prediction. The RFM prediction can provide a geometric constraint. In each block matching, keypoints are classified with the feature octave. The keypoints are matched in the group with the same feature octave. The block and octave constraint can narrow the keypoints search scope and be helpful for acquiring additional matches. Since blocks are independent, each corresponding block matching is parallelized to realize multi-thread processing with open multi-processing (OpenMP). Experiments are designed to test the matching performance and runtime. The results indicate the effectiveness and efficiency of the proposed approach.