Non-local statistical label fusion for multi-atlas segmentation

Multi-atlas segmentation provides a general purpose, fully-automated approach for transferring spatial information from an existing dataset (“atlases”) to a previously unseen context (“target”) through image registration. The method to resolve voxelwise label conflicts between the registered atlases (“label fusion”) has a substantial impact on segmentation quality. Ideally, statistical fusion algorithms (e.g., STAPLE) would result in accurate segmentations as they provide a framework to elegantly integrate models of rater performance. The accuracy of statistical fusion hinges upon accurately modeling the underlying process of how raters err. Despite success on human raters, current approaches inaccurately model multi-atlas behavior as they fail to seamlessly incorporate exogenous intensity information into the estimation process. As a result, locally weighted voting algorithms represent the de facto standard fusion approach in clinical applications. Moreover, regardless of the approach, fusion algorithms are generally dependent upon large atlas sets and highly accurate registration as they implicitly assume that the registered atlases form a collectively unbiased representation of the target. Herein, we propose a novel statistical fusion algorithm, Non-Local STAPLE (NLS). NLS reformulates the STAPLE framework from a non-local means perspective in order to learn what label an atlas would have observed, given perfect correspondence. Through this reformulation, NLS (1) seamlessly integrates intensity into the estimation process, (2) provides a theoretically consistent model of multi-atlas observation error, and (3) largely diminishes the need for large atlas sets and very high-quality registrations. We assess the sensitivity and optimality of the approach and demonstrate significant improvement in two empirical multi-atlas experiments.     

Automatic anatomical brain MRI segmentation combining label propagation and decision fusion

Regions in three-dimensional magnetic resonance (MR) brain images can be classified using protocols for manually segmenting and labeling structures. For large cohorts, time and expertise requirements make this approach impractical. To achieve automation, an individual segmentation can be propagated to another individual using an anatomical correspondence estimate relating the atlas image to the target image. The accuracy of the resulting target labeling has been limited but can potentially be improved by combining multiple segmentations using decision fusion. We studied segmentation propagation and decision fusion on 30 normal brain MR images, which had been manually segmented into 67 structures. Correspondence estimates were established by nonrigid registration using free-form deformations. Both direct label propagation and an indirect approach were tested. Individual propagations showed an average similarity index (SI) of 0.754+/-0.016 against manual segmentations. Decision fusion using 29 input segmentations increased SI to 0.836+/-0.009. For indirect propagation of a single source via 27 intermediate images, SI was 0.779+/-0.013. We also studied the effect of the decision fusion procedure using a numerical simulation with synthetic input data. The results helped to formulate a model that predicts the quality improvement of fused brain segmentations based on the number of individual propagated segmentations combined. We demonstrate a practicable procedure that exceeds the accuracy of previous automatic methods and can compete with manual delineations.     

A generative probability model of joint label fusion for multi-atlas based brain segmentation

Automated labeling of anatomical structures in medical images is very important in many neuroscience studies. Recently, patch-based labeling has been widely investigated to alleviate the possible mis-alignment when registering atlases to the target image. However, the weights used for label fusion from the registered atlases are generally computed independently and thus lack the capability of preventing the ambiguous atlas patches from contributing to the label fusion. More critically, these weights are often calculated based only on the simple patch similarity, thus not necessarily providing optimal solution for label fusion. To address these limitations, we propose a generative probability model to describe the procedure of label fusion in a multi-atlas scenario, for the goal of labeling each point in the target image by the best representative atlas patches that also have the largest labeling unanimity in labeling the underlying point correctly. Specifically, sparsity constraint is imposed upon label fusion weights, in order to select a small number of atlas patches that best represent the underlying target patch, thus reducing the risks of including the misleading atlas patches. The labeling unanimity among atlas patches is achieved by exploring their dependencies, where we model these dependencies as the joint probability of each pair of atlas patches in correctly predicting the labels, by analyzing the correlation of their morphological error patterns and also the labeling consensus among atlases. The patch dependencies will be further recursively updated based on the latest labeling results to correct the possible labeling errors, which falls to the Expectation Maximization (EM) framework. To demonstrate the labeling performance, we have comprehensively evaluated our patch-based labeling method on the whole brain parcellation and hippocampus segmentation. Promising labeling results have been achieved with comparison to the conventional patch-based labeling method, indicating the potential application of the proposed method in the future clinical studies.     

Fully automatic segmentation of the mitral leaflets in 3D transesophageal echocardiographic images using multi-atlas joint label fusion and deformable medial modeling

Comprehensive visual and quantitative analysis of in vivo human mitral valve morphology is central to the diagnosis and surgical treatment of mitral valve disease. Real-time 3D transesophageal echocardiography (3D TEE) is a practical, highly informative imaging modality for examining the mitral valve in a clinical setting. To facilitate visual and quantitative 3D TEE image analysis, we describe a fully automated method for segmenting the mitral leaflets in 3D TEE image data. The algorithm integrates complementary probabilistic segmentation and shape modeling techniques (multi-atlas joint label fusion and deformable modeling with continuous medial representation) to automatically generate 3D geometric models of the mitral leaflets from 3D TEE image data. These models are unique in that they establish a shape-based coordinate system on the valves of different subjects and represent the leaflets volumetrically, as structures with locally varying thickness. In this work, expert image analysis is the gold standard for evaluating automatic segmentation. Without any user interaction, we demonstrate that the automatic segmentation method accurately captures patient-specific leaflet geometry at both systole and diastole in 3D TEE data acquired from a mixed population of subjects with normal valve morphology and mitral valve disease.     

Atlas-based prostate segmentation using an hybrid registration

Purpose  This paper presents the preliminary results of a semi-automatic method for prostate segmentation of magnetic resonance images (MRI) which aims to be incorporated in a navigation system for prostate brachytherapy. Methods  The method is based on the registration of an anatomical atlas computed from a population of 18 MRI exams onto a patient image. An hybrid registration framework which couples an intensity-based registration with a robust point-matching algorithm is used for both atlas building and atlas registration. Results  The method has been validated on the same dataset that the one used to construct the atlas using the leave-one-out method. Results gives a mean error of 3.39 mm and a standard deviation of 1.95 mm with respect to expert segmentations. Conclusions  We think that this segmentation tool may be a very valuable help to the clinician for routine quantitative image exploitation.     

Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI

A whole heart segmentation (WHS) method is presented for cardiac MRI. This segmentation method employs multi-modality atlases from MRI and CT and adopts a new label fusion algorithm which is based on the proposed multi-scale patch (MSP) strategy and a new global atlas ranking scheme. MSP, developed from the scale-space theory, uses the information of multi-scale images and provides different levels of the structural information of images for multi-level local atlas ranking. Both the local and global atlas ranking steps use the information theoretic measures to compute the similarity between the target image and the atlases from multiple modalities. The proposed segmentation scheme was evaluated on a set of data involving 20 cardiac MRI and 20 CT images. Our proposed algorithm demonstrated a promising performance, yielding a mean WHS Dice score of 0.899 ± 0.0340, Jaccard index of 0.818 ± 0.0549, and surface distance error of 1.09 ± 1.11 mm for the 20 MRI data. The average runtime for the proposed label fusion was 12.58 min.     

Deep label fusion: A generalizable hybrid multi-atlas and deep convolutional neural network for medical image segmentation

Deep convolutional neural networks (DCNN) achieve very high accuracy in segmenting various anatomical structures in medical images but often suffer from relatively poor generalizability. Multi-atlas segmentation (MAS), while less accurate than DCNN in many applications, tends to generalize well to unseen datasets with different characteristics from the training dataset. Several groups have attempted to integrate the power of DCNN to learn complex data representations and the robustness of MAS to changes in image characteristics. However, these studies primarily focused on replacing individual components of MAS with DCNN models and reported marginal improvements in accuracy. In this study we describe and evaluate a 3D end-to-end hybrid MAS and DCNN segmentation pipeline, called Deep Label Fusion (DLF). The DLF pipeline consists of two main components with learnable weights, including a weighted voting subnet that mimics the MAS algorithm and a fine-tuning subnet that corrects residual segmentation errors to improve final segmentation accuracy. We evaluate DLF on five datasets that represent a diversity of anatomical structures (medial temporal lobe subregions and lumbar vertebrae) and imaging modalities (multi-modality, multi-field-strength MRI and Computational Tomography). These experiments show that DLF achieves comparable segmentation accuracy to nnU-Net (Isensee et al., 2020), the state-of-the-art DCNN pipeline, when evaluated on a dataset with similar characteristics to the training datasets, while outperforming nnU-Net on tasks that involve generalization to datasets with different characteristics (different MRI field strength or different patient population). DLF is also shown to consistently improve upon conventional MAS methods. In addition, a modality augmentation strategy tailored for multimodal imaging is proposed and demonstrated to be beneficial in improving the segmentation accuracy of learning-based methods, including DLF and DCNN, in missing data scenarios in test time as well as increasing the interpretability of the contribution of each individual modality.     

磁共振DWI对前列腺癌诊断价值的初步研究

目的：探讨磁共振扩散加权成像（DWI）对前列腺癌（Pca）的诊断价值。方法：对46例临床怀疑为Pca者行常规MRI及DWI检查,分析DWI和表观扩散系数（ADC）图并测量ADC值,根据常规MRI及DWI作出诊断并与病理结果对照。结果：常规MRI、DWI、MR＋DWI、B超诊断Pca的准确率分别为76.1%、89.1%、91.3%、71.7%,DWI诊断准确率高于B超（P＜0.05）;DWI与MR＋DWI诊断准确率无统计学差别（P＞0.05）;常规MRI与MR＋DWI诊断准确率有统计学差别（P＜0.05）。结论：在常规MRI基础上结合DWI检查能提高对Pca诊断的准确性,弥补常规MRI的不足。     

Magnetic resonance imaging of the newborn brain: manual segmentation of labelled atlases in term-born and preterm infants

Premature birth is a major and growing problem. Investigations into neuroanatomical correlates and consequences of preterm birth are hampered by complex neonatal brain anatomy and unavailability of atlases and protocols covering the whole brain. We developed delineation protocols for the manual segmentation of cerebral magnetic resonance (MR) images from newborn infants into 50 regions with comprehensive coverage of the brain. We then segmented MR scans from 15 infants born preterm at median 29, range 26-35, weeks postmenstrual age and scanned at term-corrected age, and five term-born infants born at median 41, range 39-45, weeks postmenstrual age. Total and regional brain volumes were estimated in each infant, and regional volumes expressed as a fraction of total brain volume. Total brain volumes were higher with greater age at birth and at time of scan, but once corrected for age at scan there was no difference between preterm and term infants. Fractional age-corrected regional volumes were bigger unilaterally in terms in middle and inferior temporal gyri, anterior temporal lobe, fusiform gyrus and posterior cingulate gyrus. Fractional age-corrected regional volumes were larger in preterms bilaterally in hippocampus, amygdala, thalamus and lateral ventricles, left superior temporal gyrus and right caudate nucleus. These differences were not significant after correcting for multiple hypothesis testing, but suggest subtle differences between preterms and term-borns accessible to regional analysis. Detailed illustrated protocols are made available in the Appendix.     

Multi-task learning for the segmentation of organs at risk with label dependence

Automatic segmentation of organs at risk is crucial to aid diagnoses and remains a challenging task in medical image analysis domain. To perform the segmentation, we use multi-task learning (MTL) to accurately determine the contour of organs at risk in CT images. We train an encoder-decoder network for two tasks in parallel. The main task is the segmentation of organs, entailing a pixel-level classification in the CT images, and the auxiliary task is the multi-label classification of organs, entailing an image-level multi-label classification of the CT images. To boost the performance of the multi-label classification, we propose a weighted mean cross entropy loss function for the network training, where the weights are the global conditional probability between two organs. Based on MTL, we optimize the false positive filtering (FPF) algorithm to decrease the number of falsely segmented organ pixels in the CT images. Specifically, we propose a dynamic threshold selection (DTS) strategy to prevent true positive rates from decreasing when using the FPF algorithm. We validate these methods on the public ISBI 2019 segmentation of thoracic organs at risk (SegTHOR) challenge dataset and a private medical organ dataset. The experimental results show that networks using our proposed methods outperform basic encoder-decoder networks without increasing the training time complexity.     

Robust Bayesian fusion of continuous segmentation maps

The fusion of probability maps is required when trying to analyse a collection of image labels or probability maps produced by several segmentation algorithms or human raters. The challenge is to weight the combination of maps correctly, in order to reflect the agreement among raters, the presence of outliers and the spatial uncertainty in the consensus. In this paper, we address several shortcomings of prior work in continuous label fusion. We introduce a novel approach to jointly estimate a reliable consensus map and to assess the presence of outliers and the confidence in each rater. Our robust approach is based on heavy-tailed distributions allowing local estimates of raters performances. In particular, we investigate the Laplace, the Student’s t and the generalized double Pareto distributions, and compare them with respect to the classical Gaussian likelihood used in prior works. We unify these distributions into a common tractable inference scheme based on variational calculus and scale mixture representations. Moreover, the introduction of bias and spatial priors leads to proper rater bias estimates and control over the smoothness of the consensus map. Finally, we propose an approach that clusters raters based on variational boosting, and thus may produce several alternative consensus maps. Our approach was successfully tested on MR prostate delineations and on lung nodule segmentations from the LIDC-IDRI dataset.     

The evaluation of single-view and multi-view fusion 3D echocardiography using image-driven segmentation and tracking

Real-time 3D echocardiography (RT3DE) promises a more objective and complete cardiac functional analysis by dynamic 3D image acquisition. Despite several efforts towards automation of left ventricle (LV) segmentation and tracking, these remain challenging research problems due to the poor-quality nature of acquired images usually containing missing anatomical information, speckle noise, and limited field-of-view (FOV). Recently, multi-view fusion 3D echocardiography has been introduced as acquiring multiple conventional single-view RT3DE images with small probe movements and fusing them together after alignment. This concept of multi-view fusion helps to improve image quality and anatomical information and extends the FOV. We now take this work further by comparing single-view and multi-view fused images in a systematic study. In order to better illustrate the differences, this work evaluates image quality and information content of single-view and multi-view fused images using image-driven LV endocardial segmentation and tracking. The image-driven methods were utilized to fully exploit image quality and anatomical information present in the image, thus purposely not including any high-level constraints like prior shape or motion knowledge in the analysis approaches. Experiments show that multi-view fused images are better suited for LV segmentation and tracking, while relatively more failures and errors were observed on single-view images.     

DWI在前列腺癌诊断中的应用进展

MRI在前列腺癌的早期诊断、临床分期及侵袭性评估中有重要作用,而DWI是应用最广泛的MRI功能成像序列之一。现将DWI在前列腺癌诊断中的临床应用进展进行综述。     

Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy

In this paper, we present a new Deep Convolutional Neural Networks (CNNs) dedicated to fully automatic segmentation of Glioblastoma brain tumors with high- and low-grade. The proposed CNNs model is inspired by the Occipito-Temporal pathway which has a special function called selective attention that uses different receptive field sizes in successive layers to figure out the crucial objects in a scene. Thus, using selective attention technique to develop the CNNs model, helps to maximize the extraction of relevant features from MRI images. We have also addressed two more issues: class-imbalance, and the spatial relationship among image Patches. To address the first issue, we propose two steps: an equal sampling of images Patches and an experimental analysis of the effect of weighted cross-entropy loss function on the segmentation results. In addition, to overcome the second issue, we have studied the effect of Overlapping Patches against Adjacent Patches where the Overlapping Patches show better segmentation results due to the introduction of the global context as well as the local features of the image Patches compared to the conventionnel Adjacent Patches. Our experiment results are reported on BRATS-2018 dataset where our End-to-End Deep Learning model achieved state-of-the-art performance. The median Dice score of our fully automatic segmentation model is 0.90, 0.83, 0.83 for the whole tumor, tumor core, and enhancing tumor respectively compared to the Dice score of radiologist, that is in the range 74% – 85%. Moreover, our proposed CNNs model is not only computationally efficient at inference time, but it could segment the whole brain on average 12 seconds. Finally, the proposed Deep Learning model provides an accurate and reliable segmentation result, and that makes it suitable for adopting in research and as a part of different clinical settings.     

The AGES-Reykjavik study atlases: Non-linear multi-spectral template and atlases for studies of the ageing brain

Quantitative analyses of brain structures from Magnetic Resonance (MR) image data are often performed using automatic segmentation algorithms. Many of these algorithms rely on templates and atlases in a common coordinate space. Most freely available brain atlases are generated from relatively young individuals and not always derived from well-defined cohort studies. In this paper, we introduce a publicly available multi-spectral template with corresponding tissue probability atlases and regional atlases, optimised to use in studies of ageing cohorts (mean age 75 ± 5 years). Furthermore, we provide validation data from a regional segmentation pipeline to assure the integrity of the dataset.     

Multi-perspective label based deep learning framework for cerebral vasculature segmentation in whole-brain fluorescence images

The popularity of fluorescent labelling and mesoscopic optical imaging techniques enable the acquisition of whole mammalian brain vasculature images at capillary resolution. Segmentation of the cerebrovascular network is essential for analyzing the cerebrovascular structure and revealing the pathogenesis of brain diseases. Existing deep learning methods use a single type of annotated labels with the same pixel weight to train the neural network and segment vessels. Due to the variation in the shape, density and brightness of vessels in whole-brain fluorescence images, it is difficult for a neural network trained with a single type of label to segment all vessels accurately. To address this problem, we proposed a deep learning cerebral vasculature segmentation framework based on multi-perspective labels. First, the pixels in the central region of thick vessels and the skeleton region of vessels were extracted separately using morphological operations based on the binary annotated labels to generate two different labels. Then, we designed a three-stage 3D convolutional neural network containing three sub-networks, namely thick-vessel enhancement network, vessel skeleton enhancement network and multi-channel fusion segmentation network. The first two sub-networks were trained by the two labels generated in the previous step, respectively, and pre-segmented the vessels. The third sub-network was responsible for fusing the pre-segmented results to precisely segment the vessels. We validated our method on two mouse cerebral vascular datasets generated by different fluorescence imaging modalities. The results showed that our method outperforms the state-of-the-art methods, and the proposed method can be applied to segment the vasculature on large-scale volumes.     

经DWI定位经直肠超声引导下前列腺穿刺活检术诊断前列腺癌

目的 探讨经DWI定位经直肠超声(TRUS)引导下目标穿刺方法诊断前列腺癌(PCa)的价值。方法 对50例临床疑似PCa患者行DWI,之后于TRUS引导下行经会阴前列腺穿刺术。采用系统穿刺(SB)与DWI目标穿刺(DWI-TB)相结合进行穿刺。根据穿刺病理结果,分别统计DWI-TB、SB及DWI-TB+SB的PCa检出率和穿刺阳性率。结果 DWI-TB和SB及DWI-TB+SB对PCa的检出率分别为54.00%(27/50)、46.00%(23/50)和66.00%(33/50),其中DWI-TB+SB与单纯SB相比差异有统计学意义(P<0.05);DWI-TB、SB及DWI-TB+SB穿刺阳性率分别为40.41%(59/146)、9.33%(39/418)和14.23%(72/506),三种穿刺方案穿刺阳性率差异有统计学意义(P<0.05)。结论 DWI有助于发现可疑PCa病灶,为TRUS引导下经会阴前列腺穿刺提供目标、特别是移行区目标信息,联合运用SB和DWI-TB价值较高。     

MR扩散加权成像诊断前列腺癌的价值分析

目的评价MR扩散加权成像（DWI）在前列腺良、恶性病变鉴别诊断中的价值。材料与方法共47例患者进行MRI检查,其中良性前列腺增生29例,前列腺癌18例。检查方法包括常规T1WI、T2WI、DWI和脂肪抑制成像。所有病例经穿刺活检或手术证实。观察前列腺大小、病变位置、信号特点及肿瘤侵犯情况,对DWI、ADC数据行定量分析。结果 MRI显示18例恶性肿瘤患者中,发生于外周带13例,表现为T2WI外周带内结节状、片状异常低信号区;8例位于中央带或移行带。DWI对前列腺癌的诊断敏感度为81.07%、特异度为78.01%、准确率为80.11%。结论联合应用DWI成像方法可以提高MR对前列腺癌诊断的准确率。