Fast OCT image enhancement method based on the sigmoid-energy conservation equation

Optical coherence tomography (OCT) is an important medical diagnosis technology, but OCT images are inevitably interfered by speckle noise and other factors, which greatly reduce the quality of the OCT image. In order to improve the quality of the OCT image quickly, a fast OCT image enhancement method is proposed based on the fusion equation. The proposed method consists of three parts: edge detection, noise suppression, and image fusion. In this paper, the improved wave algorithm is used to detect the image edge and its fine features, and the averaging uncorrelated images method is used to suppress speckle noise and improve image contrast. In order to sharpen image edges while suppressing the speckle noise, a sigmoid-energy conservation equation (SE equation) is designed to fuse the edge detection image and the noise suppression image. The proposed method was tested on two publicly available datasets. Results show that the proposed method can effectively improve image contrast and sharpen image edges while suppressing the speckle noise. Compared with other state-of-the-art methods, the proposed method has better image enhancement effect and speed. Under the same or better enhancement effect, the processing speed of the proposed method is 2 ∼ 34 times faster than other methods.     

Real-time noise reduction based on ground truth free deep learning for optical coherence tomography

Optical coherence tomography (OCT) is a high-resolution non-invasive 3D imaging modality, which has been widely used for biomedical research and clinical studies. The presence of noise on OCT images is inevitable which will cause problems for post-image processing and diagnosis. The frame-averaging technique that acquires multiple OCT images at the same or adjacent locations can enhance the image quality significantly. Both conventional frame averaging methods and deep learning-based methods using averaged frames as ground truth have been reported. However, conventional averaging methods suffer from the limitation of long image acquisition time, while deep learning-based methods require complicated and tedious ground truth label preparation. In this work, we report a deep learning-based noise reduction method that does not require clean images as ground truth for model training. Three network structures, including Unet, super-resolution residual network (SRResNet), and our modified asymmetric convolution-SRResNet (AC-SRResNet), were trained and evaluated using signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), edge preservation index (EPI) and computation time (CT). The effectiveness of these three trained models on OCT images of different samples and different systems was also investigated and confirmed. The SNR improvement for different sample images for L₂-loss-trained Unet, SRResNet, and AC-SRResNet are 20.83 dB, 24.88 dB, and 22.19 dB, respectively. The SNR improvement for public images from different system for L₁-loss-trained Unet, SRResNet, and AC-SRResNet are 19.36 dB, 20.11 dB, and 22.15 dB, respectively. AC-SRResNet and SRResNet demonstrate better denoising effect than Unet with longer computation time. AC-SRResNet demonstrates better edge preservation capability than SRResNet while Unet is close to AC-SRResNet. Eventually, we incorporated Unet, SRResNet, and AC-SRResNet into our graphic processing unit accelerated OCT imaging system for online noise reduction evaluation. Real-time noise reduction for OCT images with size of 512×512 pixels for Unet, SRResNet, and AC-SRResNet at 64 fps, 19 fps, and 17 fps were achieved respectively.     

A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set

In this paper, we propose and validate a deep learning framework that incorporates both multi-atlas registration and level-set for segmenting pancreas from CT volume images. The proposed segmentation pipeline consists of three stages, namely coarse, fine, and refine stages. Firstly, a coarse segmentation is obtained through multi-atlas based 3D diffeomorphic registration and fusion. After that, to learn the connection feature, a 3D patch-based convolutional neural network (CNN) and three 2D slice-based CNNs are jointly used to predict a fine segmentation based on a bounding box determined from the coarse segmentation. Finally, a 3D level-set method is used, with the fine segmentation being one of its constraints, to integrate information of the original image and the CNN-derived probability map to achieve a refine segmentation. In other words, we jointly utilize global 3D location information (registration), contextual information (patch-based 3D CNN), shape information (slice-based 2.5D CNN) and edge information (3D level-set) in the proposed framework. These components form our cascaded coarse-fine-refine segmentation framework. We test the proposed framework on three different datasets with varying intensity ranges obtained from different resources, respectively containing 36, 82 and 281 CT volume images. In each dataset, we achieve an average Dice score over 82%, being superior or comparable to other existing state-of-the-art pancreas segmentation algorithms.     

Distortion and instability compensation with deep learning for rotational scanning endoscopic optical coherence tomography

Optical Coherence Tomography (OCT) is increasingly used in endoluminal procedures since it provides high-speed and high resolution imaging. Distortion and instability of images obtained with a proximal scanning endoscopic OCT system are significant due to the motor rotation irregularity, the friction between the rotating probe and outer sheath and synchronization issues. On-line compensation of artefacts is essential to ensure image quality suitable for real-time assistance during diagnosis or minimally invasive treatment. In this paper, we propose a new online correction method to tackle both B-scan distortion, video stream shaking and drift problem of endoscopic OCT linked to A-line level image shifting. The proposed computational approach for OCT scanning video correction integrates a Convolutional Neural Network (CNN) to improve the estimation of azimuthal shifting of each A-line. To suppress the accumulative error of integral estimation we also introduce another CNN branch to estimate a dynamic overall orientation angle. We train the network with semi-synthetic OCT videos by intentionally adding rotational distortion into real OCT scanning images. The results show that networks trained on this semi-synthetic data generalize to stabilize real OCT videos, and the algorithm efficacy is demonstrated on both ex vivo and in vivo data, where strong scanning artifacts are successfully corrected.     

Speckle reduction with edge preservation in medical ultrasonic images using a homogeneous region growing mean filter (HRGMF)

In this paper, we propose a new post formation adaptive image filtering technique, to be called the homogeneous region growing mean filter, in order to reduce speckle noise with edge preservation in medical ultrasonic images. First, to find a proper seed region, an initially assumed seed region, which is larger than the average speckle size around a given filtering point, is successively contracted according to a certain local homogeneity criterion. Once the seed region is determined, the next step is to grow the homogeneous region successively based on some measures of local homogeneity and similarity of the neighboring region. The output of the proposed filter for each filtering point is obtained from the arithmetic mean of the grown locally homogeneous region. Several simulation results are presented to illustrate the performance of the proposed filter. They show that the proposed technique effectively smoothes ultrasonic speckle and completely suppresses isolated impulsive noise over the entire texture in addition to preserving the edge information.     

3-D breast ultrasound segmentation using active contour model

In this study, we made use of the discrete active contour model to overcome the natural properties of ultrasound (US) images, speckle, noise and tissue-related textures, to segment the breast tumors precisely. Determination of the real tumor boundary with the snake-deformation process requires an initial contour estimate. However, the manual way to sketch an initial contour is very time-consuming. Thus, we propose an automatic initial contour-finding method that not only maintains the tumor shape, but also is close to the tumor boundary and inside the tumor. During the deformation process, to prevent the snake trapping into the false position caused by tissue-related texture or speckle, we added the edge information as an image feature to define the external force. In addition, because the 3-D volume of a tumor is essentially constructed by a sequence of 2-D images, our method for finding boundaries of a tumor can be extended to 3-D cases. By precisely counting the volume of the 3-D images, we can get the volume of tumor. Finally, we will show that the proposed techniques have rather good performance and lead to a satisfactory result in comparison with the estimated volume and physician's estimate.     

基于小波变换的医学超声图像去噪及增强方法

目的探求一种基于小波变换的医学超声图像去噪及增强方法。方法提出了一种基于小波分析理论的医学超声图像噪声的综合抑制方法,首先对医学超声图像进行对数变换,将乘性噪声变成加性噪声;然后进行多尺度小波变换,将图像分解成一系列不同尺度上的小波系数,对变换后不同尺度的高频子图像进行非线性小波软阈值处理,阈值处理后的高频子图像进行增强;最后,经小波逆变换和指数变换恢复去噪后图像。结果原图像中斑纹噪声被有效去除,图像边缘细节得以保留。结论该方法可有效保留细节信号,极大限度地去除斑纹噪声。     

Fast and adaptive region merging based on perceptual hashing via multi-thresholding for SAR image segmentation

Due to the serious speckle noise in synthetic aperture radar (SAR) image, segmentation of SAR images is still a challenging problem. In this paper, a novel region merging method based on perceptual hashing is proposed for SAR image segmentation. In the proposed method, perceptual hash algorithm (PHA) is utilized to calculate the degree of similarity between different regions during region merging in SAR image segmentation. After reducing the speckle noise by Lee filter which maintains the sharpness of SAR image, a set of different homogeneous regions is constructed based on multi-thresholding and treated as the input data of region merging. The new contribution of this paper is the combination of multi-thresholding for initial segmentation and perceptual hash method for the adaptive process of region merging, which preserves the texture feature of input images and reduces the time complexity of the proposed method. The experimental results on synthetic and real SAR images show that the proposed algorithm is faster and attains higher-quality segmentation results than the three recent state-of-the-art image segmentation methods.     

Tissue self-attention network for the segmentation of optical coherence tomography images on the esophagus

Automatic segmentation of layered tissue is the key to esophageal optical coherence tomography (OCT) image processing. With the advent of deep learning techniques, frameworks based on a fully convolutional network are proved to be effective in classifying pixels on images. However, due to speckle noise and unfavorable imaging conditions, the esophageal tissue relevant to the diagnosis is not always easy to identify. An effective approach to address this problem is extracting more powerful feature maps, which have similar expressions for pixels in the same tissue and show discriminability from those from different tissues. In this study, we proposed a novel framework, called the tissue self-attention network (TSA-Net), which introduces the self-attention mechanism for esophageal OCT image segmentation. The self-attention module in the network is able to capture long-range context dependencies from the image and analyzes the input image in a global view, which helps to cluster pixels in the same tissue and reveal differences of different layers, thus achieving more powerful feature maps for segmentation. Experiments have visually illustrated the effectiveness of the self-attention map, and its advantages over other deep networks were also discussed.     

Multi-penalty conditional random field approach to super-resolved reconstruction of optical coherence tomography images

Improving the spatial resolution of Optical Coherence Tomography (OCT) images is important for the visualization and analysis of small morphological features in biological tissue such as blood vessels, membranes, cellular layers, etc. In this paper, we propose a novel reconstruction approach to obtaining super-resolved OCT tomograms from multiple lower resolution images. The proposed Multi-Penalty Conditional Random Field (MPCRF) method combines four different penalty factors (spatial proximity, first and second order intensity variations, as well as a spline-based smoothness of fit) into the prior model within a Maximum A Posteriori (MAP) estimation framework. Test carried out in retinal OCT images illustrate the effectiveness of the proposed MPCRF reconstruction approach in terms of spatial resolution enhancement, as compared to previously published super resolved image reconstruction methods. Visual assessment of the MPCRF results demonstrate the potential of this method in better preservation of fine details and structures of the imaged sample, as well as retaining the sharpness of biological tissue boundaries while reducing the effects of speckle noise inherent to OCT. Quantitative evaluation using imaging metrics such as Signal-to-Noise Ratio (SNR), Contrast to Noise Ratio (CNR), Equivalent Number of Looks (ENL), and Edge Preservation Parameter show significant visual quality improvement with the MPCRF approach. Therefore, the proposed MPCRF reconstruction approach is an effective tool for enhancing the spatial resolution of OCT images without the necessity for significant imaging hardware modifications.OCIS codes: (100.0100) Image processing, (110.4500) Optical coherence tomography, (330.6130) Spatial resolution     

Improved Simultaneous Algebraic Reconstruction Technique Algorithm for Positron-Emission Tomography Image Reconstruction via Minimizing the Fast Total Variation

ContextThere has been considerable progress in the instrumentation for data measurement and computer methods for generating images of measured PET data. These computer methods have been developed to solve the inverse problem, also known as the “image reconstruction from projections” problem.AimIn this paper, we propose a modified Simultaneous Algebraic Reconstruction Technique (SART) algorithm to improve the quality of image reconstruction by incorporating total variation (TV) minimization into the iterative SART algorithm.MethodologyThe SART updates the estimated image by forward projecting the initial image onto the sinogram space. Then, the difference between the estimated sinogram and the given sinogram is back-projected onto the image domain. This difference is then subtracted from the initial image to obtain a corrected image. Fast total variation (FTV) minimization is applied to the image obtained in the SART step. The second step is the result obtained from the previous FTV update. The SART and the FTV minimization steps run iteratively in an alternating manner. Fifty iterations were applied to the SART algorithm used in each of the regularization-based methods. In addition to the conventional SART algorithm, spatial smoothing was used to enhance the quality of the image. All images were sized at 128 × 128 pixels.ResultsThe proposed algorithm successfully accomplished edge preservation. A detailed scrutiny revealed that the reconstruction algorithms differed; for example, the SART and the proposed FTV-SART algorithm effectively preserved the hot lesion edges, whereas artifacts and deviations were more likely to occur in the ART algorithm than in the other algorithms.ConclusionsCompared to the standard SART, the proposed algorithm is more robust in removing background noise while preserving edges to suppress the existent image artifacts. The quality measurements and visual inspections show a significant improvement in image quality compared to the conventional SART and Algebraic Reconstruction Technique (ART) algorithms.     

A textural approach based on Gabor functions for texture edge detection in ultrasound images

Edge detection is an important, but difficult, step in quantitative ultrasound (US) image analysis. In this paper, we present a new textural approach for detecting a class of edges in US images; namely, the texture edges with a weak regional mean gray-level difference (RMGD) between adjacent regions. The proposed approach comprises a vision model-based texture edge detector using Gabor functions and a new texture-enhancement scheme. The experimental results on the synthetic edge images have shown that the performances of the four tested textural and nontextural edge detectors are about 20%-95% worse than that of the proposed approach. Moreover, the texture enhancement may improve the performance of the proposed texture edge detector by as much as 40%. The experiments on 20 clinical US images have shown that the proposed approach can find reasonable edges for real objects of interest with the performance of 0.4 +/- 0.08 in terms of the Pratt's figure.     

基于卷积神经网络的胎儿二维超声图像丘脑横切面的自动识别

目的:胎儿二维超声图像不同切面的自动识别分类对于提高医生的工作效率具有十分重要的意义。方法:本文针对传统自动分类方法中需要先对图像进行细致分割再进行特征提取和分类识别、分类速度慢等问题,提出了一种基于深度卷积神经网络(Convolutional neural network,CNN)的胎儿二维超声图像中丘脑横切面的自动识别方法。通过对收集到的胎儿二维超声丘脑横切面进行图像增强等预处理,提出了改进的卷积神经网络算法。结果:该算法避免了对于二维超声图像复杂的前期预处理,可以直接输入原始的二维超声图像,具有很强的适应性和泛化能力。试验结果表明,该方法的识别准确率能达到94.81%。结论:此模型的提出,为医学影像自动识别技术提供了新的参考。     

Resolution enhancement and realistic speckle recovery with generative adversarial modeling of micro-optical coherence tomography

A resolution enhancement technique for optical coherence tomography (OCT), based on Generative Adversarial Networks (GANs), was developed and investigated. GANs have been previously used for resolution enhancement of photography and optical microscopy images. We have adapted and improved this technique for OCT image generation. Conditional GANs (cGANs) were trained on a novel set of ultrahigh resolution spectral domain OCT volumes, termed micro-OCT, as the high-resolution ground truth (∼1 μm isotropic resolution). The ground truth was paired with a low-resolution image obtained by synthetically degrading resolution 4x in one of (1-D) or both axial and lateral axes (2-D). Cross-sectional image (B-scan) volumes obtained from in vivo imaging of human labial (lip) tissue and mouse skin were used in separate feasibility experiments. Accuracy of resolution enhancement compared to ground truth was quantified with human perceptual accuracy tests performed by an OCT expert. The GAN loss in the optimization objective, noise injection in both the generator and discriminator models, and multi-scale discrimination were found to be important for achieving realistic speckle appearance in the generated OCT images. The utility of high-resolution speckle recovery was illustrated by an example of micro-OCT imaging of blood vessels in lip tissue. Qualitative examples applying the models to image data from outside of the training data distribution, namely human retina and mouse bladder, were also demonstrated, suggesting potential for cross-domain transferability. This preliminary study suggests that deep learning generative models trained on OCT images from high-performance prototype systems may have potential in enhancing lower resolution data from mainstream/commercial systems, thereby bringing cutting-edge technology to the masses at low cost.     

Breast Ultrasound Despeckling Using Anisotropic Diffusion Guided by Texture Descriptors

Breast ultrasound (BUS) is considered the most important adjunct method to mammography for diagnosing cancer. However, this image modality suffers from an intrinsic artifact called speckle noise, which degrades spatial and contrast resolution and obscures the screened anatomy. Hence, it is necessary to reduce speckle artifacts before performing image analysis by means of computer-aided diagnosis systems, for example. In addition, the trade-off between smoothing level and preservation of lesion contour details should be addressed by speckle reduction schemes. In this scenario, we propose a BUS despeckling method based on anisotropic diffusion guided by Log–Gabor filters (ADLG). Because we assume that different breast tissues have distinct textures, in our approach we perform a multichannel decomposition of the BUS image using Log–Gabor filters. Next, the conduction coefficient of anisotropic diffusion filtering is computed using texture responses instead of intensity values as stated originally. The proposed algorithm is validated using both synthetic and real breast data sets, with 900 and 50 images, respectively. The performance measures are compared with four existing speckle reduction schemes based on anisotropic diffusion: conventional anisotropic diffusion filtering (CADF), speckle-reducing anisotropic diffusion (SRAD), texture-oriented anisotropic diffusion (TOAD), and interference-based speckle filtering followed by anisotropic diffusion (ISFAD). The validity metrics are the Pratt’s figure of merit, for synthetic images, and the mean radial distance (in pixels), for real sonographies. Figure of merit and mean radial distance indices should tend toward ‘1’ and ‘0’, respectively, to indicate adequate edge preservation. The results suggest that ADLG outperforms the four speckle removal filters compared with respect to simulated and real BUS images. For each method—ADLG, CADF, SRAD, TOAD and ISFAD—the figure of merit median values are 0.83, 0.40, 0.39, 0.51 and 0.59, and the mean radial distance median results are 4.19, 6.29, 6.39, 6.43 and 5.88.     

Fetal abdominal contour extraction and measurement in ultrasound images

A novel method is developed for the fetal abdominal contour extraction and measurement in ultrasound images. Fetal abdominal circumference (AC) is one of the standardized measurements in the antepartum ultrasound monitoring. Among several standardized measurements, AC is best correlated with fetal growth but is also the most difficult to be accurately measured. To overcome the difficulties in the abdominal contour extraction, the proposed method is a four-step procedure that integrates several image segmentation techniques. The proposed method is able to make the best use of the strength of different segmentation algorithms, while avoiding their deficiencies. An enhanced instantaneous coefficient of variation (ICOV) edge detector is first developed to detect edges of the abdominal contour and alleviate the effects of most speckle noise. Then, the Fuzzy C-Means clustering is employed to distinguish salient edges attributable to the abdominal contour from weak edges due to the other texture. Subsequently, the iterative Hough transform is applied to determine an elliptical contour and obtain an initial estimation of the AC. Finally, the gradient vector field (GVF) snake adapts the initial ellipse to the real edges of the abdominal contour. Quantitative validation of the proposed method on synthetic images under different imaging conditions achieves satisfactory segmentation accuracy (98.78%+/-0.16%). Experiments on 150 clinical images are carried out in three aspects: comparisons between inter-observer and inter-run variation, the fitness analysis between the automatically detected ellipse and the manual delineation, and the accuracy comparisons between automatic measurements and manual measurements in estimation of fetal weight (EFW). Experimental results show that the proposed method can provide consistent and accurate measurements. The reductions of the mean absolute difference and the standard deviation of EFW based on automatic measurements are about 1.2% and 2.1%, respectively, which indicate its potential in clinical antepartum monitoring application.     

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning

Segmentation of organs or lesions from medical images plays an essential role in many clinical applications such as diagnosis and treatment planning. Though Convolutional Neural Networks (CNN) have achieved the state-of-the-art performance for automatic segmentation, they are often limited by the lack of clinically acceptable accuracy and robustness in complex cases. Therefore, interactive segmentation is a practical alternative to these methods. However, traditional interactive segmentation methods require a large number of user interactions, and recently proposed CNN-based interactive segmentation methods are limited by poor performance on previously unseen objects. To solve these problems, we propose a novel deep learning-based interactive segmentation method that not only has high efficiency due to only requiring clicks as user inputs but also generalizes well to a range of previously unseen objects. Specifically, we first encode user-provided interior margin points via our proposed exponentialized geodesic distance that enables a CNN to achieve a good initial segmentation result of both previously seen and unseen objects, then we use a novel information fusion method that combines the initial segmentation with only a few additional user clicks to efficiently obtain a refined segmentation. We validated our proposed framework through extensive experiments on 2D and 3D medical image segmentation tasks with a wide range of previously unseen objects that were not present in the training set. Experimental results showed that our proposed framework 1) achieves accurate results with fewer user interactions and less time compared with state-of-the-art interactive frameworks and 2) generalizes well to previously unseen objects.     

Wavelet denoising of multiframe optical coherence tomography data

We introduce a novel speckle noise reduction algorithm for OCT images. Contrary to present approaches, the algorithm does not rely on simple averaging of multiple image frames or denoising on the final averaged image. Instead it uses wavelet decompositions of the single frames for a local noise and structure estimation. Based on this analysis, the wavelet detail coefficients are weighted, averaged and reconstructed. At a signal-to-noise gain at about 100% we observe only a minor sharpness decrease, as measured by a full-width-half-maximum reduction of 10.5%. While a similar signal-to-noise gain would require averaging of 29 frames, we achieve this result using only 8 frames as input to the algorithm. A possible application of the proposed algorithm is preprocessing in retinal structure segmentation algorithms, to allow a better differentiation between real tissue information and unwanted speckle noise.     

Filtering of SAR images using non-local PCA

This letter presents a novel approach for despeckling synthetic aperture radar (SAR) images. The underlying principle is first to convert multiplicative speckle noise into additive by using logarithmic transformation. The patches’ similarity is exploited by using Kolmogorov–Smirnov (KS) measurements. Then, the proposed two-stage approach filters the noisy image using a nonlocal principal component analysis (PCA) shrinkage strategy by automatically estimating the level of the additive noise in the log-transformed image via Gaussian mixture model (GMM). The proposed approach has been compared with two current state-of-the-art methods, showing the very promising and competitive results on filtering the speckle noise.     

Adaptive pseudo-p-norm regularization based De-speckling of SAR images

A novel variational model is proposed to de-speckle the multiplication noise in the synthetic aperture radar (SAR) image to provide a high-quality interpretation of SAR data. The model consists of the data fidelity term and the adaptive pseudo-p-norm regularization term. The data fidelity term ensures the convexity of the model and avoids the nonlinear image transformation. With an improved ratio of exponentially weighted averages (ROEWA) operator and the edge detection image, the exponent value of the pseudo-p-norm regularization term varies with the intensity of the edge adaptively, which results in a better retention of the image geometric structure while removing the speckle noise. Compared with the total variation (TV) regularization, the staircase-like artifacts can be restrained more effectively. Experiments demonstrate that the proposed method are more competitive both visually and quantitatively than that from the existing nonlocal mean-based and variational-based methods.