ACEnet: Anatomical context-encoding network for neuroanatomy segmentation

Segmentation of brain structures from magnetic resonance (MR) scans plays an important role in the quantification of brain morphology. Since 3D deep learning models suffer from high computational cost, 2D deep learning methods are favored for their computational efficiency. However, existing 2D deep learning methods are not equipped to effectively capture 3D spatial contextual information that is needed to achieve accurate brain structure segmentation. In order to overcome this limitation, we develop an Anatomical Context-Encoding Network (ACEnet) to incorporate 3D spatial and anatomical contexts in 2D convolutional neural networks (CNNs) for efficient and accurate segmentation of brain structures from MR scans, consisting of 1) an anatomical context encoding module to incorporate anatomical information in 2D CNNs and 2) a spatial context encoding module to integrate 3D image information in 2D CNNs. In addition, a skull stripping module is adopted to guide the 2D CNNs to attend to the brain. Extensive experiments on three benchmark datasets have demonstrated that our method achieves promising performance compared with state-of-the-art alternative methods for brain structure segmentation in terms of both computational efficiency and segmentation accuracy.     

Comparison of atlas-based techniques for whole-body bone segmentation

We evaluate the accuracy of whole-body bone extraction from whole-body MR images using a number of atlas-based segmentation methods. The motivation behind this work is to find the most promising approach for the purpose of MRI-guided derivation of PET attenuation maps in whole-body PET/MRI. To this end, a variety of atlas-based segmentation strategies commonly used in medical image segmentation and pseudo-CT generation were implemented and evaluated in terms of whole-body bone segmentation accuracy. Bone segmentation was performed on 23 whole-body CT/MR image pairs via leave-one-out cross validation procedure. The evaluated segmentation techniques include: (i) intensity averaging (IA), (ii) majority voting (MV), (iii) global and (iv) local (voxel-wise) weighting atlas fusion frameworks implemented utilizing normalized mutual information (NMI), normalized cross-correlation (NCC) and mean square distance (MSD) as image similarity measures for calculating the weighting factors, along with other atlas-dependent algorithms, such as (v) shape-based averaging (SBA) and (vi) Hofmann's pseudo-CT generation method. The performance evaluation of the different segmentation techniques was carried out in terms of estimating bone extraction accuracy from whole-body MRI using standard metrics, such as Dice similarity (DSC) and relative volume difference (RVD) considering bony structures obtained from intensity thresholding of the reference CT images as the ground truth. Considering the Dice criterion, global weighting atlas fusion methods provided moderate improvement of whole-body bone segmentation (DSC = 0.65 ± 0.05) compared to non-weighted IA (DSC = 0.60 ± 0.02). The local weighed atlas fusion approach using the MSD similarity measure outperformed the other strategies by achieving a DSC of 0.81 ± 0.03 while using the NCC and NMI measures resulted in a DSC of 0.78 ± 0.05 and 0.75 ± 0.04, respectively. Despite very long computation time, the extracted bone obtained from both SBA (DSC = 0.56 ± 0.05) and Hofmann's methods (DSC = 0.60 ± 0.02) exhibited no improvement compared to non-weighted IA. Finding the optimum parameters for implementation of the atlas fusion approach, such as weighting factors and image similarity patch size, have great impact on the performance of atlas-based segmentation approaches. The voxel-wise atlas fusion approach exhibited excellent performance in terms of cancelling out the non-systematic registration errors leading to accurate and reliable segmentation results. Denoising and normalization of MR images together with optimization of the involved parameters play a key role in improving bone extraction accuracy.     

Learning to segment fetal brain tissue from noisy annotations

Automatic fetal brain tissue segmentation can enhance the quantitative assessment of brain development at this critical stage. Deep learning methods represent the state of the art in medical image segmentation and have also achieved impressive results in brain segmentation. However, effective training of a deep learning model to perform this task requires a large number of training images to represent the rapid development of the transient fetal brain structures. On the other hand, manual multi-label segmentation of a large number of 3D images is prohibitive. To address this challenge, we segmented 272 training images, covering 19–39 gestational weeks, using an automatic multi-atlas segmentation strategy based on deformable registration and probabilistic atlas fusion, and manually corrected large errors in those segmentations. Since this process generated a large training dataset with noisy segmentations, we developed a novel label smoothing procedure and a loss function to train a deep learning model with smoothed noisy segmentations. Our proposed methods properly account for the uncertainty in tissue boundaries. We evaluated our method on 23 manually-segmented test images of a separate set of fetuses. Results show that our method achieves an average Dice similarity coefficient of 0.893 and 0.916 for the transient structures of younger and older fetuses, respectively. Our method generated results that were significantly more accurate than several state-of-the-art methods including nnU-Net that achieved the closest results to our method. Our trained model can serve as a valuable tool to enhance the accuracy and reproducibility of fetal brain analysis in MRI.     

Optimum template selection for atlas-based segmentation

Atlas-based segmentation of MR brain images typically uses a single atlas (e.g., MNI Colin27) for region identification. Normal individual variations in human brain structures present a significant challenge for atlas selection. Previous researches mainly focused on how to create a specific template for different requirements (e.g., for a certain population). We address atlas selection with a different approach: instead of choosing a fixed brain atlas, we use a family of brain templates for atlas-based segmentation. For each subject and each region, the template selection method automatically chooses the 'best' template with the highest local registration accuracy, based on normalized mutual information. The region classification performances of the template selection method and the single template method were quantified by the overlap ratios (ORs) and intraclass correlation coefficients (ICCs) between the manual tracings and the respective automated labeled results. Two groups of brain images and multiple regions of interest (ROIs), including the right anterior cingulate cortex (ACC) and several subcortical structures, were tested for both methods. We found that the template selection method produced significantly higher ORs than did the single template method across all of the 13 analyzed ROIs (two-tailed paired t-test, right ACC at t(8)=4.353, p=0.0024; right amygdala, matched paired t test t(8)>3.175, p<0.013; for the remaining ROIs, t(8)=4.36, p<0.002). The template selection method also provided more reliable volume estimates than the single template method with increased ICCs. Moreover, the improved accuracy of atlas-based segmentation using optimum templates approaches the accuracy of manual tracing, and thus is valid for automated brain imaging analyses.     

A combination of atlas-based and voxel-wise approaches to analyze metabolic changes in autoradiographic data from Alzheimer's mice

Murine models are commonly used in neuroscience research to improve our knowledge of disease processes and to test drug effects. To accurately study brain glucose metabolism in these animals, ex vivo autoradiography remains the gold standard. The analysis of 3D-reconstructed autoradiographic volumes using a voxel-wise approach allows clusters of voxels representing metabolic differences between groups to be revealed. However, the spatial localization of these clusters requires careful visual identification by a neuroanatomist, a time-consuming task that is often subject to misinterpretation. Moreover, the large number of voxels to be computed in autoradiographic rodent images leads to many false positives. Here, we proposed an original automated indexation of the results of a voxel-wise approach using an MRI-based 3D digital atlas, followed by the restriction of the statistical analysis using atlas-based segmentation, thus taking advantage of the specific and complementary strengths of these two approaches. In a preliminary study of transgenic Alzheimer's mice (APP/PS1), and control littermates (PS1), we were able to achieve prompt and direct anatomical indexation of metabolic changes detected between the two groups, revealing both hypo- and hypermetabolism in the brain of APP/PS1 mice. Furthermore, statistical results were refined using atlas-based segmentation: most interesting results were obtained for the hippocampus. We thus confirmed and extended our previous results by identifying the brain structures affected in this pathological model and demonstrating modified glucose uptake in structures like the olfactory bulb. Our combined approach thus paves the way for a complete and accurate examination of functional data from cerebral structures involved in models of neurodegenerative diseases.     

Automatic atlas-based three-label cartilage segmentation from MR knee images

Osteoarthritis (OA) is the most common form of joint disease and often characterized by cartilage changes. Accurate quantitative methods are needed to rapidly screen large image databases to assess changes in cartilage morphology. We therefore propose a new automatic atlas-based cartilage segmentation method for future automatic OA studies.Atlas-based segmentation methods have been demonstrated to be robust and accurate in brain imaging and therefore also hold high promise to allow for reliable and high-quality segmentations of cartilage. Nevertheless, atlas-based methods have not been well explored for cartilage segmentation. A particular challenge is the thinness of cartilage, its relatively small volume in comparison to surrounding tissue and the difficulty to locate cartilage interfaces – for example the interface between femoral and tibial cartilage.This paper focuses on the segmentation of femoral and tibial cartilage, proposing a multi-atlas segmentation strategy with non-local patch-based label fusion which can robustly identify candidate regions of cartilage. This method is combined with a novel three-label segmentation method which guarantees the spatial separation of femoral and tibial cartilage, and ensures spatial regularity while preserving the thin cartilage shape through anisotropic regularization. Our segmentation energy is convex and therefore guarantees globally optimal solutions.We perform an extensive validation of the proposed method on 706 images of the Pfizer Longitudinal Study. Our validation includes comparisons of different atlas segmentation strategies, different local classifiers, and different types of regularizers. To compare to other cartilage segmentation approaches we validate based on the 50 images of the SKI10 dataset.     

CAN: Context-assisted full Attention Network for brain tissue segmentation

Brain tissue segmentation is of great value in diagnosing brain disorders. Three-dimensional (3D) and two-dimensional (2D) segmentation methods for brain Magnetic Resonance Imaging (MRI) suffer from high time complexity and low segmentation accuracy, respectively. To address these two issues, we propose a Context-assisted full Attention Network (CAN) for brain MRI segmentation by integrating 2D and 3D data of MRI. Different from the fully symmetric structure U-Net, the CAN takes the current 2D slice, its 3D contextual skull slices and 3D contextual brain slices as the input, which are further encoded by the DenseNet and decoded by our constructed full attention network. We have validated the effectiveness of the CAN on our collected dataset PWML and two public datasets dHCP2017 and MALC2012. Our code is available at https://github.com/nwuAI/CAN.     

A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions

We describe a new fully automatic method for the segmentation of brain images that contain multiple sclerosis white matter lesions. Multichannel magnetic resonance images are used to delineate multiple sclerosis lesions while segmenting the brain into its major structures. The method is an atlas-based segmentation technique employing a topological atlas as well as a statistical atlas. An advantage of this approach is that all segmented structures are topologically constrained, thereby allowing subsequent processing such as cortical unfolding or diffeomorphic shape analysis techniques. Evaluation with both simulated and real data sets demonstrates that the method has an accuracy competitive with state-of-the-art MS lesion segmentation methods, while simultaneously segmenting the whole brain.     

Unifying framework for multimodal brain MRI segmentation based on Hidden Markov Chains

In the frame of 3D medical imaging, accurate segmentation of multimodal brain MR images is of interest for many brain disorders. However, due to several factors such as noise, imaging artifacts, intrinsic tissue variation and partial volume effects, tissue classification remains a challenging task. In this paper, we present a unifying framework for unsupervised segmentation of multimodal brain MR images including partial volume effect, bias field correction, and information given by a probabilistic atlas. Here-proposed method takes into account neighborhood information using a Hidden Markov Chain (HMC) model. Due to the limited resolution of imaging devices, voxels may be composed of a mixture of different tissue types, this partial volume effect is included to achieve an accurate segmentation of brain tissues. Instead of assigning each voxel to a single tissue class (i.e., hard classification), we compute the relative amount of each pure tissue class in each voxel (mixture estimation). Further, a bias field estimation step is added to the proposed algorithm to correct intensity inhomogeneities. Furthermore, atlas priors were incorporated using probabilistic brain atlas containing prior expectations about the spatial localization of different tissue classes. This atlas is considered as a complementary sensor and the proposed method is extended to multimodal brain MRI without any user-tunable parameter (unsupervised algorithm). To validate this new unifying framework, we present experimental results on both synthetic and real brain images, for which the ground truth is available. Comparison with other often used techniques demonstrates the accuracy and the robustness of this new Markovian segmentation scheme.     

Segmentation of 4D cardiac MR images using a probabilistic atlas and the EM algorithm

In this paper an automatic atlas-based segmentation algorithm for 4D cardiac MR images is proposed. The algorithm is based on the 4D extension of the expectation maximisation (EM) algorithm. The EM algorithm uses a 4D probabilistic cardiac atlas to estimate the initial model parameters and to integrate a priori information into the classification process. The probabilistic cardiac atlas has been constructed from the manual segmentations of 3D cardiac image sequences of 14 healthy volunteers. It provides space and time-varying probability maps for the left and right ventricles, the myocardium, and background structures such as the liver, stomach, lungs and skin. In addition to using the probabilistic cardiac atlas as a priori information, the segmentation algorithm incorporates spatial and temporal contextual information by using 4D Markov Random Fields. After the classification, the largest connected component of each structure is extracted using a global connectivity filter which improves the results significantly, especially for the myocardium. Validation against manual segmentations and computation of the correlation between manual and automatic segmentation on 249 3D volumes were calculated. We used the 'leave one out' test where the image set to be segmented was not used in the construction of its corresponding atlas. Results show that the procedure can successfully segment the left ventricle (LV) (r = 0.96), myocardium (r = 0.92) and right ventricle (r = 0.92). In addition, 4D images from 10 patients with hypertrophic cardiomyopathy were also manually and automatically segmented yielding a good correlation in the volumes of the LV (r = 0.93) and myocardium (0.94) when the atlas constructed with volunteers is blurred.     

Optimal weights for local multi-atlas fusion using supervised learning and dynamic information (SuperDyn): validation on hippocampus segmentation

We developed a novel method for spatially-local selection of atlas-weights in multi-atlas segmentation that combines supervised learning on a training set and dynamic information in the form of local registration accuracy estimates (SuperDyn). Supervised learning was applied using a jackknife learning approach and the methods were evaluated using leave-N-out cross-validation. We applied our segmentation method to hippocampal segmentation in 1.5T and 3T MRI from two datasets: 69 healthy middle-aged subjects (aged 44-49) and 37 healthy and cognitively-impaired elderly subjects (aged 72-84). Mean Dice overlap scores (left hippocampus, right hippocampus) of (83.3, 83.2) and (85.1, 85.3) from the respective datasets were found to be significantly higher than those obtained via equally-weighted fusion, STAPLE, and dynamic fusion. In addition to global surface distance and volume metrics, we also investigated accuracy at a spatially-local scale using a surface-based segmentation performance assessment method (SurfSPA), which generates cohort-specific maps of segmentation accuracy quantified by inward or outward displacement relative to the manual segmentations. These measurements indicated greater agreement with manual segmentation and lower variability for the proposed segmentation method, as compared to equally-weighted fusion.     

Deep label fusion: A generalizable hybrid multi-atlas and deep convolutional neural network for medical image segmentation

Deep convolutional neural networks (DCNN) achieve very high accuracy in segmenting various anatomical structures in medical images but often suffer from relatively poor generalizability. Multi-atlas segmentation (MAS), while less accurate than DCNN in many applications, tends to generalize well to unseen datasets with different characteristics from the training dataset. Several groups have attempted to integrate the power of DCNN to learn complex data representations and the robustness of MAS to changes in image characteristics. However, these studies primarily focused on replacing individual components of MAS with DCNN models and reported marginal improvements in accuracy. In this study we describe and evaluate a 3D end-to-end hybrid MAS and DCNN segmentation pipeline, called Deep Label Fusion (DLF). The DLF pipeline consists of two main components with learnable weights, including a weighted voting subnet that mimics the MAS algorithm and a fine-tuning subnet that corrects residual segmentation errors to improve final segmentation accuracy. We evaluate DLF on five datasets that represent a diversity of anatomical structures (medial temporal lobe subregions and lumbar vertebrae) and imaging modalities (multi-modality, multi-field-strength MRI and Computational Tomography). These experiments show that DLF achieves comparable segmentation accuracy to nnU-Net (Isensee et al., 2020), the state-of-the-art DCNN pipeline, when evaluated on a dataset with similar characteristics to the training datasets, while outperforming nnU-Net on tasks that involve generalization to datasets with different characteristics (different MRI field strength or different patient population). DLF is also shown to consistently improve upon conventional MAS methods. In addition, a modality augmentation strategy tailored for multimodal imaging is proposed and demonstrated to be beneficial in improving the segmentation accuracy of learning-based methods, including DLF and DCNN, in missing data scenarios in test time as well as increasing the interpretability of the contribution of each individual modality.     

A statistically based flow for image segmentation

In this paper we present a new algorithm for 3D medical image segmentation. The algorithm is versatile, fast, relatively simple to implement, and semi-automatic. It is based on minimizing a global energy defined from a learned non-parametric estimation of the statistics of the region to be segmented. Implementation details are discussed and source code is freely available as part of the 3D Slicer project. In addition, a new unified set of validation metrics is proposed. Results on artificial and real MRI images show that the algorithm performs well on large brain structures both in terms of accuracy and robustness to noise.     

Supervised machine learning-based classification scheme to segment the brainstem on MRI in multicenter brain tumor treatment context

Purpose

To constrain the risk of severe toxicity in radiotherapy and radiosurgery, precise volume delineation of organs at risk is required. This task is still manually performed, which is time-consuming and prone to observer variability. To address these issues, and as alternative to atlas-based segmentation methods, machine learning techniques, such as support vector machines (SVM), have been recently presented to segment subcortical structures on magnetic resonance images (MRI).

Methods

SVM is proposed to segment the brainstem on MRI in multicenter brain cancer context. A dataset composed by 14 adult brain MRI scans is used to evaluate its performance. In addition to spatial and probabilistic information, five different image intensity values (IIVs) configurations are evaluated as features to train the SVM classifier. Segmentation accuracy is evaluated by computing the Dice similarity coefficient (DSC), absolute volumes difference (AVD) and percentage volume difference between automatic and manual contours.

Results

Mean DSC for all proposed IIVs configurations ranged from 0.89 to 0.90. Mean AVD values were below \(1.5\,\hbox {cm}^{3}\), where the value for best performing IIVs configuration was \(0.85\,\hbox {cm}^{3}\), representing an absolute mean difference of \(3.99\,\%\) with respect to the manual segmented volumes.

Conclusion

Results suggest consistent volume estimation and high spatial similarity with respect to expert delineations. The proposed approach outperformed presented methods to segment the brainstem, not only in volume similarity metrics, but also in segmentation time. Preliminary results showed that the approach might be promising for adoption in clinical use.

     

3D U-Net深度学习模型基于盆腔T2WI自动分割盆腔软组织结构

目的 评估3D U-Net深度学习(DL)模型基于盆腔T2WI自动分割盆腔软组织结构的可行性.方法 回顾性分析147例经病理证实或盆腔MRI随访观察确诊的前列腺癌或良性前列腺增生患者,其中28例接受2次、121例接受1次盆腔MR扫描,共175组T2WI;手动标注T2WI所示软组织结构,包括前列腺、膀胱、直肠、双侧精囊腺...     

Advanced neonatal NeuroMRI

This article describes the potentials and challenges of quantitative analyses of human neonatal brain images using structural magnetic resonance (MR) imaging and diffusion tensor imaging. To maximize the potential of MR imaging for neonatal brain studies, the combination of both contrasts is highly beneficial. Based on the multicontrast data, a neonate brain atlas was created, which allows automated segmentation of neonate brain MR images. The accuracy, advantages, and potential pitfalls of this atlas-based segmentation approach are discussed. The accurate and reproducible MR imaging quantification achieved by this approach could be an initial step toward the successful clinical evaluation of the neonatal brain.     

A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set

In this paper, we propose and validate a deep learning framework that incorporates both multi-atlas registration and level-set for segmenting pancreas from CT volume images. The proposed segmentation pipeline consists of three stages, namely coarse, fine, and refine stages. Firstly, a coarse segmentation is obtained through multi-atlas based 3D diffeomorphic registration and fusion. After that, to learn the connection feature, a 3D patch-based convolutional neural network (CNN) and three 2D slice-based CNNs are jointly used to predict a fine segmentation based on a bounding box determined from the coarse segmentation. Finally, a 3D level-set method is used, with the fine segmentation being one of its constraints, to integrate information of the original image and the CNN-derived probability map to achieve a refine segmentation. In other words, we jointly utilize global 3D location information (registration), contextual information (patch-based 3D CNN), shape information (slice-based 2.5D CNN) and edge information (3D level-set) in the proposed framework. These components form our cascaded coarse-fine-refine segmentation framework. We test the proposed framework on three different datasets with varying intensity ranges obtained from different resources, respectively containing 36, 82 and 281 CT volume images. In each dataset, we achieve an average Dice score over 82%, being superior or comparable to other existing state-of-the-art pancreas segmentation algorithms.     

Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views

Segmentation of the placenta from fetal MRI is challenging due to sparse acquisition, inter-slice motion, and the widely varying position and shape of the placenta between pregnant women. We propose a minimally interactive framework that combines multiple volumes acquired in different views to obtain accurate segmentation of the placenta. In the first phase, a minimally interactive slice-by-slice propagation method called Slic-Seg is used to obtain an initial segmentation from a single motion-corrupted sparse volume image. It combines high-level features, online Random Forests and Conditional Random Fields, and only needs user interactions in a single slice. In the second phase, to take advantage of the complementary resolution in multiple volumes acquired in different views, we further propose a probability-based 4D Graph Cuts method to refine the initial segmentations using inter-slice and inter-image consistency. We used our minimally interactive framework to examine the placentas of 16 mid-gestation patients from MRI acquired in axial and sagittal views respectively. The results show the proposed method has 1) a good performance even in cases where sparse scribbles provided by the user lead to poor results with the competitive propagation approaches; 2) a good interactivity with low intra- and inter-operator variability; 3) higher accuracy than state-of-the-art interactive segmentation methods; and 4) an improved accuracy due to the co-segmentation based refinement, which outperforms single volume or intensity-based Graph Cuts.     

Anatomically-constrained tractography: improved diffusion MRI streamlines tractography through effective use of anatomical information

Diffusion MRI streamlines tractography suffers from a number of inherent limitations, one of which is the accurate determination of when streamlines should be terminated. Use of an accurate streamlines propagation mask from segmentation of an anatomical image confines the streamlines to the volume of the brain white matter, but does not take full advantage of all of the information available from such an image. We present a modular addition to streamlines tractography, which makes more effective use of the information available from anatomical image segmentation, and the known properties of the neuronal axons being reconstructed, to apply biologically realistic priors to the streamlines generated; we refer to this as "Anatomically-Constrained Tractography". Results indicate that some of the known false positives associated with tractography algorithms are prevented, such that the biological accuracy of the reconstructions should be improved, provided that state-of-the-art streamlines tractography methods are used.