超声影像下基于ResNet34-Unet的静脉分割方法研究

静脉超声图像存在噪点多、阈值分割效果不佳的问题，对此本文提出一种基于ResNet34主干网络的ResNet34-UNet分割网络模型，利用ResNet34网络残差学习的结构特点，在保证网络能够提取充足图像特征的前提下，有效避免梯度消失和网络退化问题，且34层的网络深度维持了较小的网络规模；利用U-Net结构特有的长连接（Skip Connection）模块，将静脉超声图像的深层特征与浅层特征有效融合，使静脉的识别精度得以较大幅度的提升，实现了静脉边缘的平滑分割。将300张静脉超声图像作为训练集，200张作为测试集，通过随机旋转、翻转、投影等操作进行数据集的增强，经过十轮迭代训练后得到模型的准确度（ACC）达96.3%，较全卷积神经网络（FCN）高5.9%，较DeepLab v3+高5.2%。结果表明基于ResNet34-UNet的静脉分割方法能够准确地分割静脉超声图像，为后续超声影像下静脉的自动识别与跟踪提供了技术参考。     

Self-co-attention neural network for anatomy segmentation in whole breast ultrasound

The automated whole breast ultrasound (AWBUS) is a new breast imaging technique that can depict the whole breast anatomy. To facilitate the reading of AWBUS images and support the breast density estimation, an automatic breast anatomy segmentation method for AWBUS images is proposed in this study. The problem at hand is quite challenging as it needs to address issues of low image quality, ill-defined boundary, large anatomical variation, etc. To address these issues, a new deep learning encoder-decoder segmentation method based on a self-co-attention mechanism is developed. The self-attention mechanism is comprised of spatial and channel attention module (SC) and embedded in the ResNeXt (i.e., Res-SC) block in the encoder path. A non-local context block (NCB) is further incorporated to augment the learning of high-level contextual cues. The decoder path of the proposed method is equipped with the weighted up-sampling block (WUB) to attain class-specific better up-sampling effect. Meanwhile, the co-attention mechanism is also developed to improve the segmentation coherence among two consecutive slices. Extensive experiments are conducted with comparison to several the state-of-the-art deep learning segmentation methods. The experimental results corroborate the effectiveness of the proposed method on the difficult breast anatomy segmentation problem on AWBUS images.     

Robust whole-brain segmentation: Application to traumatic brain injury

We propose a framework for the robust and fully-automatic segmentation of magnetic resonance (MR) brain images called “Multi-Atlas Label Propagation with Expectation–Maximisation based refinement” (MALP-EM). The presented approach is based on a robust registration approach (MAPER), highly performant label fusion (joint label fusion) and intensity-based label refinement using EM. We further adapt this framework to be applicable for the segmentation of brain images with gross changes in anatomy. We propose to account for consistent registration errors by relaxing anatomical priors obtained by multi-atlas propagation and a weighting scheme to locally combine anatomical atlas priors and intensity-refined posterior probabilities. The method is evaluated on a benchmark dataset used in a recent MICCAI segmentation challenge. In this context we show that MALP-EM is competitive for the segmentation of MR brain scans of healthy adults when compared to state-of-the-art automatic labelling techniques. To demonstrate the versatility of the proposed approach, we employed MALP-EM to segment 125 MR brain images into 134 regions from subjects who had sustained traumatic brain injury (TBI). We employ a protocol to assess segmentation quality if no manual reference labels are available. Based on this protocol, three independent, blinded raters confirmed on 13 MR brain scans with pathology that MALP-EM is superior to established label fusion techniques. We visually confirm the robustness of our segmentation approach on the full cohort and investigate the potential of derived symmetry-based imaging biomarkers that correlate with and predict clinically relevant variables in TBI such as the Marshall Classification (MC) or Glasgow Outcome Score (GOS). Specifically, we show that we are able to stratify TBI patients with favourable outcomes from non-favourable outcomes with 64.7% accuracy using acute-phase MR images and 66.8% accuracy using follow-up MR images. Furthermore, we are able to differentiate subjects with the presence of a mass lesion or midline shift from those with diffuse brain injury with 76.0% accuracy. The thalamus, putamen, pallidum and hippocampus are particularly affected. Their involvement predicts TBI disease progression.     

An integrated deep learning framework for joint segmentation of blood pool and myocardium

Simultaneous and automatic segmentation of the blood pool and myocardium is an important precondition for early diagnosis and pre-operative planning in patients with complex congenital heart disease. However, due to the high diversity of cardiovascular structures and changes in mechanical properties caused by cardiac defects, the segmentation task still faces great challenges. To overcome these challenges, in this study we propose an integrated multi-task deep learning framework based on the dilated residual and hybrid pyramid pooling network (DRHPPN) for joint segmentation of the blood pool and myocardium. The framework consists of three closely connected progressive sub-networks. An inception module is used to realize the initial multi-level feature representation of cardiovascular images. A dilated residual network (DRN), as the main body of feature extraction and pixel classification, preliminary predicts segmentation regions. A hybrid pyramid pooling network (HPPN) is designed for facilitating the aggregation of local information to global information, which complements DRN. Extensive experiments on three-dimensional cardiovascular magnetic resonance (CMR) images (the available dataset of the MICCAI 2016 HVSMR challenge) demonstrate that our approach can accurately segment the blood pool and myocardium and achieve competitive performance compared with state-of-the-art segmentation methods.     

Segmentation ability map: Interpret deep features for medical image segmentation

Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for short). The corresponding SA score can quantify the segmentation abilities of deep features in different layers and units to understand the deep neural networks for segmentation. In addition, our method can provide a mean SA score which can give a performance estimation of the output on the test images without ground-truth. Finally, we use the proposed ProtoSeg method to compute the segmentation map directly on input images to further understand the segmentation ability of each input image. Results are presented on segmenting tumors in brain MRI, lesions in skin images, COVID-related abnormality in CT images, prostate segmentation in abdominal MRI, and pancreatic mass segmentation in CT images. Our method can provide new insights for interpreting and explainable AI systems for medical image segmentation. Our code is available on: https://github.com/shengfly/ProtoSeg.     

Micro-Net: A unified model for segmentation of various objects in microscopy images

Object segmentation and structure localization are important steps in automated image analysis pipelines for microscopy images. We present a convolution neural network (CNN) based deep learning architecture for segmentation of objects in microscopy images. The proposed network can be used to segment cells, nuclei and glands in fluorescence microscopy and histology images after slight tuning of input parameters. The network trains at multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi-resolution deconvolution filters. The extra convolutional layers which bypass the max-pooling operation allow the network to train for variable input intensities and object size and make it robust to noisy data. We compare our results on publicly available data sets and show that the proposed network outperforms recent deep learning algorithms.     

Correlation based 3-D segmentation of the left ventricle in pediatric echocardiographic images using radio-frequency data

Clinical diagnosis of heart disease might be substantially supported by automated segmentation of the endocardial surface in three-dimensional (3-D) echographic images. Because of the poor echogenicity contrast between blood and myocardial tissue in some regions and the inherent speckle noise, automated analysis of these images is challenging. A priori knowledge on the shape of the heart cannot always be relied on, e.g., in children with congenital heart disease, segmentation should be based on the echo features solely. The objective of this study was to investigate the merit of using temporal cross-correlation of radio-frequency (RF) data for automated segmentation of 3-D echocardiographic images. Maximum temporal cross-correlation (MCC) values were determined locally from the RF-data using an iterative 3-D technique. MCC values as well as a combination of MCC values and adaptive filtered, demodulated RF-data were used as an additional, external force in a deformable model approach to segment the endocardial surface and were tested against manually segmented surfaces. Results on 3-D full volume images (Philips, iE33) of 10 healthy children demonstrate that MCC values derived from the RF signal yield a useful parameter to distinguish between blood and myocardium in regions with low echogenicity contrast and incorporation of MCC improves the segmentation results significantly. Further investigation of the MCC over the whole cardiac cycle is required to exploit the full benefit of it for automated segmentation.     

Semantic instance segmentation with discriminative deep supervision for medical images

Semantic instance segmentation is crucial for many medical image analysis applications, including computational pathology and automated radiation therapy. Existing methods for this task can be roughly classified into two categories: (1) proposal-based methods and (2) proposal-free methods. However, in medical images, the irregular shape-variations and crowding instances (e.g., nuclei and cells) make it hard for the proposal-based methods to achieve robust instance localization. On the other hand, ambiguous boundaries caused by the low-contrast nature of medical images (e.g., CT images) challenge the accuracy of the proposal-free methods. To tackle these issues, we propose a proposal-free segmentation network with discriminative deep supervision (DDS), which at the same time allows us to gain the power of the proposal-based method. The DDS module is interleaved with a carefully designed proposal-free segmentation backbone in our network. Consequently, the features learned by the backbone network become more sensitive to instance localization. Also, with the proposed DDS module, robust pixel-wise instance-level cues (especially structural information) are introduced for semantic segmentation. Extensive experiments on three datasets, i.e., a nuclei dataset, a pelvic CT image dataset, and a synthetic dataset, demonstrate the superior performance of the proposed algorithm compared to the previous works.     

Exploring uncertainty measures in deep networks for Multiple sclerosis lesion detection and segmentation

Deep learning networks have recently been shown to outperform other segmentation methods on various public, medical-image challenge datasets, particularly on metrics focused on large pathologies. For diseases such as Multiple Sclerosis (MS), however, monitoring all the focal lesions visible on MRI sequences, even very small ones, is essential for disease staging, prognosis, and evaluating treatment efficacy. Small lesion segmentation presents significant challenges to popular deep learning models. This, coupled with their deterministic predictions, hinders their clinical adoption. Uncertainty estimates for these predictions would permit subsequent revision by clinicians. We present the first exploration of multiple uncertainty estimates based on Monte Carlo (MC) dropout (Gal and Ghahramani, 2016) in the context of deep networks for lesion detection and segmentation in medical images. Specifically, we develop a 3D MS lesion segmentation CNN, augmented to provide four different voxel-based uncertainty measures based on MC dropout. We train the network on a proprietary, large-scale, multi-site, multi-scanner, clinical MS dataset, and compute lesion-wise uncertainties by accumulating evidence from voxel-wise uncertainties within detected lesions. We analyze the performance of voxel-based segmentation and lesion-level detection by choosing operating points based on the uncertainty. Uncertainty filtering improves both voxel and lesion-wise TPR and FDR on remaining, certain predictions compared to sigmoid-based TPR/FDR curves. Small lesions and lesion-boundaries are the most uncertain regions, which is consistent with human-rater variability.     

A Y-Net deep learning method for road segmentation using high-resolution visible remote sensing images

Road segmentation from high-resolution visible remote sensing images provides an effective way for automatic road network forming. Recently, deep learning methods based on convolutional neural networks (CNNs) are widely applied in road segmentation. However, it is a challenge for most CNN-based methods to achieve high segmentation accuracy when processing high-resolution visible remote sensing images with rich details. To handle this problem, we propose a road segmentation method based on a Y-shaped convolutional network (indicated as Y-Net). Y-Net contains a two-arm feature extraction module and a fusion module. The feature extraction module includes a deep downsampling-to-upsampling sub-network for semantic features and a convolutional sub-network without downsampling for detail features. The fusion module combines all features for road segmentation. Benefiting from this scheme, the Y-Net can well segment multi-scale roads (both wide and narrow roads) from high-resolution images. The testing and comparative experiments on a public dataset and a private dataset show that Y-Net has higher segmentation accuracy than four other state-of-art methods, FCN (Fully Convolutional Network), U-Net, SegNet, and FC-DenseNet (Fully Convolutional DenseNet). Especially, Y-Net accurately segments contours of narrow roads, which are missed by the comparative methods.     

Joint segmentation of anatomical and functional images: Applications in quantification of lesions from PET,PET-CT,MRI-PET,and MRI-PET-CT images

We present a novel method for the joint segmentation of anatomical and functional images. Our proposed methodology unifies the domains of anatomical and functional images, represents them in a product lattice, and performs simultaneous delineation of regions based on random walk image segmentation. Furthermore, we also propose a simple yet effective object/background seed localization method to make the proposed segmentation process fully automatic. Our study uses PET, PET-CT, MRI-PET, and fused MRI-PET-CT scans (77 studies in all) from 56 patients who had various lesions in different body regions. We validated the effectiveness of the proposed method on different PET phantoms as well as on clinical images with respect to the ground truth segmentation provided by clinicians. Experimental results indicate that the presented method is superior to threshold and Bayesian methods commonly used in PET image segmentation, is more accurate and robust compared to the other PET-CT segmentation methods recently published in the literature, and also it is general in the sense of simultaneously segmenting multiple scans in real-time with high accuracy needed in routine clinical use.     

Segmentation of thalamic nuclei using a modified k-means clustering algorithm and high-resolution quantitative magnetic resonance imaging at 1.5 T

Patient outcome in minimally invasive stereotactic neurosurgical procedures depends on the ability to accurately locate the desired functional region within the deep brain while avoiding the surrounding anatomy. Due to the lack of sufficient contrast within this region in pre-operatively acquired MR images, electrophysiological exploration and histological atlases are currently required to define the surgical target within the thalamus in the treatment of many motor-control disorders. In this paper we introduce a method for segmenting the individual thalamic nuclei based on high-resolution quantitative magnetic resonance images, providing improved target visualization. The method was tested using whole-brain T1 and T2 data acquired from four healthy individuals. Accuracy of the segmentation results was assessed by comparing the center-of-mass coordinates of the segmented nuclei, with coordinates obtained from a classic histological atlas registered to these images. Strong agreement was found, with an average Euclidean distance difference of less than 4.5 mm averaged across all nuclei and all individuals. Reproducibility of the method, determined by calculating the percent similarity of segmentation results derived from data acquired from repeated scan sessions, was greater than 85%. These results illustrate the ability to accurately and reliably segment the primary nuclei of the thalamus and suggest that the method may have utility in the study of individual nuclear regions in disease state as well as for planning deep-brain surgical procedures.     

DCAN: Deep contour-aware networks for object instance segmentation from histology images

In histopathological image analysis, the morphology of histological structures, such as glands and nuclei, has been routinely adopted by pathologists to assess the malignancy degree of adenocarcinomas. Accurate detection and segmentation of these objects of interest from histology images is an essential prerequisite to obtain reliable morphological statistics for quantitative diagnosis. While manual annotation is error-prone, time-consuming and operator-dependant, automated detection and segmentation of objects of interest from histology images can be very challenging due to the large appearance variation, existence of strong mimics, and serious degeneration of histological structures. In order to meet these challenges, we propose a novel deep contour-aware network (DCAN) under a unified multi-task learning framework for more accurate detection and segmentation. In the proposed network, multi-level contextual features are explored based on an end-to-end fully convolutional network (FCN) to deal with the large appearance variation. We further propose to employ an auxiliary supervision mechanism to overcome the problem of vanishing gradients when training such a deep network. More importantly, our network can not only output accurate probability maps of histological objects, but also depict clear contours simultaneously for separating clustered object instances, which further boosts the segmentation performance. Our method ranked the first in two histological object segmentation challenges, including 2015 MICCAI Gland Segmentation Challenge and 2015 MICCAI Nuclei Segmentation Challenge. Extensive experiments on these two challenging datasets demonstrate the superior performance of our method, surpassing all the other methods by a significant margin.     

Retinal optical coherence tomography image analysis by a restricted Boltzmann machine

Optical coherence tomography (OCT) is an emerging imaging technique for ophthalmic disease diagnosis. Two major problems in OCT image analysis are image enhancement and image segmentation. Deep learning methods have achieved excellent performance in image analysis. However, most of the deep learning-based image analysis models are supervised learning-based approaches and need a high volume of training data (e.g., reference clean images for image enhancement and accurate annotated images for segmentation). Moreover, acquiring reference clean images for OCT image enhancement and accurate annotation of the high volume of OCT images for segmentation is hard. So, it is difficult to extend these deep learning methods to the OCT image analysis. We propose an unsupervised learning-based approach for OCT image enhancement and abnormality segmentation, where the model can be trained without reference images. The image is reconstructed by Restricted Boltzmann Machine (RBM) by defining a target function and minimizing it. For OCT image enhancement, each image is independently learned by the RBM network and is eventually reconstructed. In the reconstruction phase, we use the ReLu function instead of the Sigmoid function. Reconstruction of images given by the RBM network leads to improved image contrast in comparison to other competitive methods in terms of contrast to noise ratio (CNR). For anomaly detection, hyper-reflective foci (HF) as one of the first signs in retinal OCTs of patients with diabetic macular edema (DME) are identified based on image reconstruction by RBM and post-processing by removing the HFs candidates outside the area between the first and the last retinal layers. Our anomaly detection method achieves a high ability to detect abnormalities.     

基于V-Net卷积神经网络深度学习模型自动分割腰椎CT图像中的椎旁肌

目的 观察基于V-Net卷积神经网络(CNN)的深度学习(DL)模型自动分割腰椎CT图像中的椎旁肌的价值。方法 收集471例接受腰椎CT检查患者，按7∶3比例将其分为训练集(n=330)和测试集(n=141);采用2D V-Net进行训练，建立DL模型；观察其分割腰大肌、腰方肌、椎后肌群及椎旁肌的价值。结果 基于V-Net CNN的DL模型分割椎旁肌精度良好，戴斯相似系数(DSC)均较高、肌肉横截面积误差率(CSA error)均较低；其分割训练集图像中的腰大肌、腰方肌及椎旁肌的DSC均高于测试集(P均<0.05),而分割训练集中4组肌肉的CSA error均低于测试集(P均<0.05)。测试集内两两比较结果显示，该模型分割椎后肌群的DSC最高、腰方肌的DSC最低；分割腰方肌的CSA error最高、椎旁肌的CSA error最低(P均<0.05)。结论 以基于V-Net的DL模型自动分割椎旁肌的效能较佳。     

Prior-aware autoencoders for lung pathology segmentation

Segmentation of lung pathology in Computed Tomography (CT) images is of great importance for lung disease screening. However, the presence of different types of lung pathologies with a wide range of heterogeneities in size, shape, location, and texture, on one side, and their visual similarity with respect to surrounding tissues, on the other side, make it challenging to perform reliable automatic lesion segmentation. To leverage segmentation performance, we propose a deep learning framework comprising a Normal Appearance Autoencoder (NAA) model to learn the distribution of healthy lung regions and reconstruct pathology-free images from the corresponding pathological inputs by replacing the pathological regions with the characteristics of healthy tissues. Detected regions that represent prior information regarding the shape and location of pathologies are then integrated into a segmentation network to guide the attention of the model into more meaningful delineations. The proposed pipeline was tested on three types of lung pathologies, including pulmonary nodules, Non-Small Cell Lung Cancer (NSCLC), and Covid-19 lesion on five comprehensive datasets. The results show the superiority of the proposed prior model, which outperformed the baseline segmentation models in all the cases with significant margins. On average, adding the prior model improved the Dice coefficient for the segmentation of lung nodules by 0.038, NSCLCs by 0.101, and Covid-19 lesions by 0.041. We conclude that the proposed NAA model produces reliable prior knowledge regarding the lung pathologies, and integrating such knowledge into a prior segmentation network leads to more accurate delineations.     

An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy

External beam radiotherapy (EBRT) has become the preferred options for nonsurgical treatment of prostate cancer and cervix cancer. In order to deliver higher doses to cancerous regions within these pelvic structures (i.e. prostate or cervix) while maintaining or lowering the doses to surrounding non-cancerous regions, it is critical to account for setup variation, organ motion, anatomical changes due to treatment and intra-fraction motion. In previous work, manual segmentation of the soft tissues is performed and then images are registered based on the manual segmentation. In this paper, we present an integrated automatic approach to multiple organ segmentation and nonrigid constrained registration, which can achieve these two aims simultaneously. The segmentation and registration steps are both formulated using a Bayesian framework, and they constrain each other using an iterative conditional model strategy. We also propose a new strategy to assess cumulative actual dose for this novel integrated algorithm, in order to both determine whether the intended treatment is being delivered and, potentially, whether or not a plan should be adjusted for future treatment fractions. Quantitative results show that the automatic segmentation produced results that have an accuracy comparable to manual segmentation, while the registration part significantly outperforms both rigid and nonrigid registration. Clinical application and evaluation of dose delivery show the superiority of proposed method to the procedure currently used in clinical practice, i.e. manual segmentation followed by rigid registration.     

HAL-IA: A Hybrid Active Learning framework using Interactive Annotation for medical image segmentation

High performance of deep learning models on medical image segmentation greatly relies on large amount of pixel-wise annotated data, yet annotations are costly to collect. How to obtain high accuracy segmentation labels of medical images with limited cost (e.g. time) becomes an urgent problem. Active learning can reduce the annotation cost of image segmentation, but it faces three challenges: the cold start problem, an effective sample selection strategy for segmentation task and the burden of manual annotation. In this work, we propose a Hybrid Active Learning framework using Interactive Annotation (HAL-IA) for medical image segmentation, which reduces the annotation cost both in decreasing the amount of the annotated images and simplifying the annotation process. Specifically, we propose a novel hybrid sample selection strategy to select the most valuable samples for segmentation model performance improvement. This strategy combines pixel entropy, regional consistency and image diversity to ensure that the selected samples have high uncertainty and diversity. In addition, we propose a warm-start initialization strategy to build the initial annotated dataset to avoid the cold-start problem. To simplify the manual annotation process, we propose an interactive annotation module with suggested superpixels to obtain pixel-wise label with several clicks. We validate our proposed framework with extensive segmentation experiments on four medical image datasets. Experimental results showed that the proposed framework achieves high accuracy pixel-wise annotations and models with less labeled data and fewer interactions, outperforming other state-of-the-art methods. Our method can help physicians efficiently obtain accurate medical image segmentation results for clinical analysis and diagnosis.     

Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation

Fine renal artery segmentation on abdominal CT angiography (CTA) image is one of the most important tasks for kidney disease diagnosis and pre-operative planning. It will help clinicians locate each interlobar artery’s blood-feeding region via providing the complete 3D renal artery tree masks. However, it is still a task of great challenges due to the large intra-scale changes, large inter-anatomy variation, thin structures, small volume ratio and small labeled dataset of the fine renal artery. In this paper, we propose the first semi-supervised 3D fine renal artery segmentation framework, DPA-DenseBiasNet, which combines deep prior anatomy (DPA), dense biased network (DenseBiasNet) and hard region adaptation loss (HRA): 1) Based on our proposed dense biased connection, the DenseBiasNet fuses multi-receptive field and multi-resolution feature maps for large intra-scale changes. This dense biased connection also obtains a dense information flow and dense gradient flow so that the training is accelerated and the accuracy is enhanced. 2) DPA features extracted from an autoencoder (AE) are embedded in DenseBiasNet to cope with the challenge of large inter-anatomy variation and thin structures. The AE is pre-trained (unsupervised) by numerous unlabeled data to achieve the representation ability of anatomy features and these features are embedded in DenseBiasNet. This process will not introduce incorrect labels as optimization targets and thus contributes to a stable semi-supervised training strategy that is suitable for sensitive thin structures. 3) The HRA selects the loss value calculation region dynamically according to the segmentation quality so the network will pay attention to the hard regions in the training process and keep the class balanced.Experiments demonstrated that DPA-DenseBiasNet had high predictive accuracy and generalization with the Dice coefficient of 0.884 which increased by 0.083 compared with 3D U-Net (Çiçek et al., 2016). This revealed our framework with great potential for the 3D fine renal artery segmentation in clinical practice.