Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains

This paper evaluates strategies for atlas selection in atlas-based segmentation of three-dimensional biomedical images. Segmentation by intensity-based nonrigid registration to atlas images is applied to confocal microscopy images acquired from the brains of 20 bees. This paper evaluates and compares four different approaches for atlas image selection: registration to an individual atlas image (IND), registration to an average-shape atlas image (AVG), registration to the most similar image from a database of individual atlas images (SIM), and registration to all images from a database of individual atlas images with subsequent multi-classifier decision fusion (MUL). The MUL strategy is a novel application of multi-classifier techniques, which are common in pattern recognition, to atlas-based segmentation. For each atlas selection strategy, the segmentation performance of the algorithm was quantified by the similarity index (SI) between the automatic segmentation result and a manually generated gold standard. The best segmentation accuracy was achieved using the MUL paradigm, which resulted in a mean similarity index value between manual and automatic segmentation of 0.86 (AVG, 0.84; SIM, 0.82; IND, 0.81). The superiority of the MUL strategy over the other three methods is statistically significant (two-sided paired t test, P < 0.001). Both the MUL and AVG strategies performed better than the best possible SIM and IND strategies with optimal a posteriori atlas selection (mean similarity index for optimal SIM, 0.83; for optimal IND, 0.81). Our findings show that atlas selection is an important issue in atlas-based segmentation and that, in particular, multi-classifier techniques can substantially increase the segmentation accuracy.     

Active deformation fields: dense deformation field estimation for atlas-based segmentation using the active contour framework

This paper presents a new and original variational framework for atlas-based segmentation. The proposed framework integrates both the active contour framework, and the dense deformation fields of optical flow framework. This framework is quite general and encompasses many of the state-of-the-art atlas-based segmentation methods. It also allows to perform the registration of atlas and target images based on only selected structures of interest. The versatility and potentiality of the proposed framework are demonstrated by presenting three diverse applications: In the first application, we show how the proposed framework can be used to simulate the growth of inconsistent structures like a tumor in an atlas. In the second application, we estimate the position of nonvisible brain structures based on the surrounding structures and validate the results by comparing with other methods. In the final application, we present the segmentation of lymph nodes in the Head and Neck CT images, and demonstrate how multiple registration forces can be used in this framework in an hierarchical manner.     

Atlas-based prostate segmentation using an hybrid registration

Purpose  This paper presents the preliminary results of a semi-automatic method for prostate segmentation of magnetic resonance images (MRI) which aims to be incorporated in a navigation system for prostate brachytherapy. Methods  The method is based on the registration of an anatomical atlas computed from a population of 18 MRI exams onto a patient image. An hybrid registration framework which couples an intensity-based registration with a robust point-matching algorithm is used for both atlas building and atlas registration. Results  The method has been validated on the same dataset that the one used to construct the atlas using the leave-one-out method. Results gives a mean error of 3.39 mm and a standard deviation of 1.95 mm with respect to expert segmentations. Conclusions  We think that this segmentation tool may be a very valuable help to the clinician for routine quantitative image exploitation.     

Comparison of atlas-based techniques for whole-body bone segmentation

We evaluate the accuracy of whole-body bone extraction from whole-body MR images using a number of atlas-based segmentation methods. The motivation behind this work is to find the most promising approach for the purpose of MRI-guided derivation of PET attenuation maps in whole-body PET/MRI. To this end, a variety of atlas-based segmentation strategies commonly used in medical image segmentation and pseudo-CT generation were implemented and evaluated in terms of whole-body bone segmentation accuracy. Bone segmentation was performed on 23 whole-body CT/MR image pairs via leave-one-out cross validation procedure. The evaluated segmentation techniques include: (i) intensity averaging (IA), (ii) majority voting (MV), (iii) global and (iv) local (voxel-wise) weighting atlas fusion frameworks implemented utilizing normalized mutual information (NMI), normalized cross-correlation (NCC) and mean square distance (MSD) as image similarity measures for calculating the weighting factors, along with other atlas-dependent algorithms, such as (v) shape-based averaging (SBA) and (vi) Hofmann's pseudo-CT generation method. The performance evaluation of the different segmentation techniques was carried out in terms of estimating bone extraction accuracy from whole-body MRI using standard metrics, such as Dice similarity (DSC) and relative volume difference (RVD) considering bony structures obtained from intensity thresholding of the reference CT images as the ground truth. Considering the Dice criterion, global weighting atlas fusion methods provided moderate improvement of whole-body bone segmentation (DSC = 0.65 ± 0.05) compared to non-weighted IA (DSC = 0.60 ± 0.02). The local weighed atlas fusion approach using the MSD similarity measure outperformed the other strategies by achieving a DSC of 0.81 ± 0.03 while using the NCC and NMI measures resulted in a DSC of 0.78 ± 0.05 and 0.75 ± 0.04, respectively. Despite very long computation time, the extracted bone obtained from both SBA (DSC = 0.56 ± 0.05) and Hofmann's methods (DSC = 0.60 ± 0.02) exhibited no improvement compared to non-weighted IA. Finding the optimum parameters for implementation of the atlas fusion approach, such as weighting factors and image similarity patch size, have great impact on the performance of atlas-based segmentation approaches. The voxel-wise atlas fusion approach exhibited excellent performance in terms of cancelling out the non-systematic registration errors leading to accurate and reliable segmentation results. Denoising and normalization of MR images together with optimization of the involved parameters play a key role in improving bone extraction accuracy.     

Ambiguity-selective consistency regularization for mean-teacher semi-supervised medical image segmentation

Semi-supervised learning has greatly advanced medical image segmentation since it effectively alleviates the need of acquiring abundant annotations from experts, wherein the mean-teacher model, known as a milestone of perturbed consistency learning, commonly serves as a standard and simple baseline. Inherently, learning from consistency can be regarded as learning from stability under perturbations. Recent improvement leans toward more complex consistency learning frameworks, yet, little attention is paid to the consistency target selection. Considering that the ambiguous regions from unlabeled data contain more informative complementary clues, in this paper, we improve the mean-teacher model to a novel ambiguity-consensus mean-teacher (AC-MT) model. Particularly, we comprehensively introduce and benchmark a family of plug-and-play strategies for ambiguous target selection from the perspectives of entropy, model uncertainty and label noise self-identification, respectively. Then, the estimated ambiguity map is incorporated into the consistency loss to encourage consensus between the two models’ predictions in these informative regions. In essence, our AC-MT aims to find out the most worthwhile voxel-wise targets from the unlabeled data, and the model especially learns from the perturbed stability of these informative regions. The proposed methods are extensively evaluated on left atrium segmentation and brain tumor segmentation. Encouragingly, our strategies bring substantial improvement over recent state-of-the-art methods. The ablation study further demonstrates our hypothesis and shows impressive results under various extreme annotation conditions.     

X-ray image segmentation for vertebral mobility analysis

Objective The goal of this work is to extract the parameters determining vertebral motion and its variation during flexion–extension movements using a computer vision tool for estimating and analyzing vertebral mobility. Materials and Methods To compute vertebral body motion parameters we propose a comparative study between two segmentation methods proposed and applied to lateral X-ray images of the cervical spine. The two vertebra contour detection methods include (1) a discrete dynamic contour model (DDCM) and (2) a template matching process associated with a polar signature system. These two methods not only enable vertebra segmentation but also extract parameters that can be used to evaluate vertebral mobility. Lateral cervical spine views including 100 views in flexion, extension and neutral orientations were available for evaluation. Vertebral body motion was evaluated by human observers and using automatic methods. Results The results provided by the automated approaches were consistent with manual measures obtained by 15 human observers. Conclusion The automated techniques provide acceptable results for the assessment of vertebral body mobility in flexion and extension on lateral views of the cervical spine. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

基于MR模板的PET/MRI衰减校正方法

目的 设计并对比基于MRI的衰减校正模板对PET图像的衰减校正效果,寻找衰减校正效果最佳模板。方法 采集30名志愿者的¹⁸F-FDG PET图像,根据性别或不同头部尺寸创建MRI模板,包括女性组（fmg）、男性组（mmg）、小尺寸头部组（smg）、大尺寸头部组（bmg）和基础组（gmg）,与PET透射模板结合后获得个体化衰减图,并最终用于¹⁸F-FDG PET图像的衰减校正;测量并对比经不同模板校正的PET图像ROI内的放射性活度值和与原始PET图像相比的相对偏差值。结果 女性受试者经衰减校正的PET图像中,其ROI内放射性活度的最小差值为4 Bq/ml,来源于fmg与gmg模板间,而男性受试者放射性活度的最小差值为1 Bq/ml,来源于mmg与fmg模板间。经5个不同模板校正后与经PET透射扫描所获得的衰减数据之间均存在差异,相对偏差值范围为-6.31%~6.32%。gmg模板与其他4个模板的相对偏差值范围为-3.44%~5.34%。结论 基于MR模板的衰减校正方法对于受试者个体不存在明显的性别差异,一般情况下,gmg模板即可满足要求。     

Automatic atlas-based three-label cartilage segmentation from MR knee images

Osteoarthritis (OA) is the most common form of joint disease and often characterized by cartilage changes. Accurate quantitative methods are needed to rapidly screen large image databases to assess changes in cartilage morphology. We therefore propose a new automatic atlas-based cartilage segmentation method for future automatic OA studies.Atlas-based segmentation methods have been demonstrated to be robust and accurate in brain imaging and therefore also hold high promise to allow for reliable and high-quality segmentations of cartilage. Nevertheless, atlas-based methods have not been well explored for cartilage segmentation. A particular challenge is the thinness of cartilage, its relatively small volume in comparison to surrounding tissue and the difficulty to locate cartilage interfaces – for example the interface between femoral and tibial cartilage.This paper focuses on the segmentation of femoral and tibial cartilage, proposing a multi-atlas segmentation strategy with non-local patch-based label fusion which can robustly identify candidate regions of cartilage. This method is combined with a novel three-label segmentation method which guarantees the spatial separation of femoral and tibial cartilage, and ensures spatial regularity while preserving the thin cartilage shape through anisotropic regularization. Our segmentation energy is convex and therefore guarantees globally optimal solutions.We perform an extensive validation of the proposed method on 706 images of the Pfizer Longitudinal Study. Our validation includes comparisons of different atlas segmentation strategies, different local classifiers, and different types of regularizers. To compare to other cartilage segmentation approaches we validate based on the 50 images of the SKI10 dataset.     

Segmentation of brain structures in presence of a space-occupying lesion

Brain deformations induced by space-occupying lesions may result in unpredictable position and shape of functionally important brain structures. The aim of this study is to propose a method for segmentation of brain structures by deformation of a segmented brain atlas in presence of a space-occupying lesion. Our approach is based on an a priori model of lesion growth (MLG) that assumes radial expansion from a seeding point and involves three steps: first, an affine registration bringing the atlas and the patient into global correspondence; then, the seeding of a synthetic tumor into the brain atlas providing a template for the lesion; finally, the deformation of the seeded atlas, combining a method derived from optical flow principles and a model of lesion growth. The method was applied on two meningiomas inducing a pure displacement of the underlying brain structures, and segmentation accuracy of ventricles and basal ganglia was assessed. Results show that the segmented structures were consistent with the patient's anatomy and that the deformation accuracy of surrounding brain structures was highly dependent on the accurate placement of the tumor seeding point. Further improvements of the method will optimize the segmentation accuracy. Visualization of brain structures provides useful information for therapeutic consideration of space-occupying lesions, including surgical, radiosurgical, and radiotherapeutic planning, in order to increase treatment efficiency and prevent neurological damage.     

Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms

Simulated deformations and images can act as the gold standard for evaluating various template-based image segmentation and registration algorithms. Traditional deformable simulation methods, such as the use of analytic deformation fields or the displacement of landmarks followed by some form of interpolation, are often unable to construct rich (complex) and/or realistic deformations of anatomical organs. This paper presents new methods aiming to automatically simulate realistic inter- and intra-individual deformations. The paper first describes a statistical approach to capturing inter-individual variability of high-deformation fields from a number of examples (training samples). In this approach, Wavelet-Packet Transform (WPT) of the training deformations and their Jacobians, in conjunction with a Markov random field (MRF) spatial regularization, are used to capture both coarse and fine characteristics of the training deformations in a statistical fashion. Simulated deformations can then be constructed by randomly sampling the resultant statistical distribution in an unconstrained or a landmark-constrained fashion. The paper also describes a model for generating tissue atrophy or growth in order to simulate intra-individual brain deformations. Several sets of simulated deformation fields and respective images are generated, which can be used in the future for systematic and extensive validation studies of automated atlas-based segmentation and deformable registration methods. The code and simulated data are available through our Web site.     

An efficient locally affine framework for the smooth registration of anatomical structures

Intra-subject and inter-subject nonlinear registration based on dense transformations requires the setting of many parameters, mainly for regularization. This task is a major issue, as the global quality of the registration will depend on it. Setting these parameters is, however, very hard, and they may have to be tuned for each patient when processing data acquired by different centers or using different protocols. Thus, we present in this article a method to introduce more coherence in the registration by using fewer degrees of freedom than with a dense registration. This is done by registering the images only on user-defined areas, using a set of affine transformations, which are optimized together in a very efficient manner. Our framework also ensures a smooth and coherent transformation thanks to a new regularization of the affine components. Finally, we ensure an invertible transformation thanks to the Log-Euclidean polyaffine framework. This allows us to get a more robust and very efficient registration method, while obtaining good results as explained below.We performed a qualitative and quantitative evaluation of the obtained results on two applications: first on atlas-based brain segmentation, comparing our results with a dense registration algorithm. Then the second application for which our framework is particularly well suited concerns bone registration in the lower-abdomen area. We obtain in this case a better positioning of the femoral heads than with a dense registration. For both applications, we show a significant improvement in computation time, which is crucial for clinical applications.     

Multi-atlas segmentation with augmented features for cardiac MR images

Multi-atlas segmentation infers the target image segmentation by combining prior anatomical knowledge encoded in multiple atlases. It has been quite successfully applied to medical image segmentation in the recent years, resulting in highly accurate and robust segmentation for many anatomical structures. However, to guide the label fusion process, most existing multi-atlas segmentation methods only utilise the intensity information within a small patch during the label fusion process and may neglect other useful information such as gradient and contextual information (the appearance of surrounding regions). This paper proposes to combine the intensity, gradient and contextual information into an augmented feature vector and incorporate it into multi-atlas segmentation. Also, it explores the alternative to the K nearest neighbour (KNN) classifier in performing multi-atlas label fusion, by using the support vector machine (SVM) for label fusion instead. Experimental results on a short-axis cardiac MR data set of 83 subjects have demonstrated that the accuracy of multi-atlas segmentation can be significantly improved by using the augmented feature vector. The mean Dice metric of the proposed segmentation framework is 0.81 for the left ventricular myocardium on this data set, compared to 0.79 given by the conventional multi-atlas patch-based segmentation (Coupé et al., 2011; Rousseau et al., 2011). A major contribution of this paper is that it demonstrates that the performance of non-local patch-based segmentation can be improved by using augmented features.     

A diffusion tensor brain template for rhesus macaques

Diffusion tensor imaging (DTI) is a powerful and noninvasive imaging method for characterizing tissue microstructure and white matter organization in the brain. While it has been applied extensively in research studies of the human brain, DTI studies of non-human primates have been performed only recently. The growing application of DTI in rhesus monkey studies would significantly benefit from a standardized framework to compare findings across different studies. A very common strategy for image analysis is to spatially normalize (co-register) the individual scans to a representative template space. This paper presents the development of a DTI brain template, UWRMAC-DTI271, for adolescent Rhesus Macaque (Macaca mulatta) monkeys. The template was generated from 271 rhesus monkeys, collected as part of a unique brain imaging genetics study. It is the largest number of animals ever used to generate a computational brain template, which enables the generation of a template that has high image quality and accounts for variability in the species. The quality of the template is further ensured with the use of DTI-TK, a well-tested and high-performance DTI spatial normalization method in human studies. We demonstrated its efficacy in monkey studies for the first time by comparing it to other commonly used scalar-methods for DTI normalization. It is anticipated that this template will play an important role in facilitating cross-site voxelwise DTI analyses in Rhesus Macaques. Such analyses are crucial in investigating the role of white matter structure in brain function, development, and other psychopathological disorders for which there are well-validated non-human primate models.     

基于定量磁化率成像人脑深层灰质核团概率性图谱的构建

目的基于定量磁化率成像(quantitative susceptibility mapping,QSM)技术制作可用于自动分割大脑深层灰质核团的概率图谱。材料与方法 15名健康受试者参与研究,所有受试者扫描均在3.0 T磁共振成像设备系统上完成。在随机选取的10名受试者得到的标准空间QSM图上,手动勾画出六个双侧脑深部灰质核团,之后采用相应的图谱评价方法选择最优概率阈值的图谱作为最终的概率图谱。在其余5名受试者得到的标准空间QSM图上,分别使用三种图谱(概率图谱、AAL图谱和Johns Hopkins图谱)自动分割和由2名研究者手动勾画出六个双侧脑深部灰质核团感兴趣区,并分别计算自动分割与手动勾画得到的区域的相似度Dice系数和磁化率值,以评价概率图谱的准确性。结果在基底节区域,概率图谱分割结果的Dice系数明显高于AAL图谱,但和Johns Hopkins图谱区别不大;在颅底和小脑区域,概率图谱分割结果的Dice系数明显高于Johns Hopkins图谱。与其他两种图谱相比,概率图谱自动分割深部核团后测量得到的磁化率值,更接近于手动勾画核团测量得到的磁化率值,其差别更小。结论基于多名受试者QSM图像构建的脑深部灰质核团概率图谱,对大脑灰质核团分割效果更加可靠,可有效提高图像分析工作的效率。     

Contour-aware network with class-wise convolutions for 3D abdominal multi-organ segmentation

Accurate delineation of multiple organs is a critical process for various medical procedures, which could be operator-dependent and time-consuming. Existing organ segmentation methods, which were mainly inspired by natural image analysis techniques, might not fully exploit the traits of the multi-organ segmentation task and could not accurately segment the organs with various shapes and sizes simultaneously. In this work, the characteristics of multi-organ segmentation are considered: the global count, position and scale of organs are generally predictable, while their local shape and appearance are volatile. Thus, we supplement the region segmentation backbone with a contour localization task to increase the certainty along delicate boundaries. Meantime, each organ has exclusive anatomical traits, which motivates us to deal with class variability with class-wise convolutions to highlight organ-specific features and suppress irrelevant responses at different field-of-views.To validate our method with adequate amounts of patients and organs, we constructed a multi-center dataset, which contains 110 3D CT scans with 24,528 axial slices, and provided voxel-level manual segmentations of 14 abdominal organs, which adds up to 1,532 3D structures in total. Extensive ablation and visualization studies on it validate the effectiveness of the proposed method. Quantitative analysis shows that we achieve state-of-the-art performance for most abdominal organs, and obtain 3.63 mm 95% Hausdorff Distance and 83.32% Dice Similarity Coefficient on an average.     

Segmentation ability map: Interpret deep features for medical image segmentation

Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for short). The corresponding SA score can quantify the segmentation abilities of deep features in different layers and units to understand the deep neural networks for segmentation. In addition, our method can provide a mean SA score which can give a performance estimation of the output on the test images without ground-truth. Finally, we use the proposed ProtoSeg method to compute the segmentation map directly on input images to further understand the segmentation ability of each input image. Results are presented on segmenting tumors in brain MRI, lesions in skin images, COVID-related abnormality in CT images, prostate segmentation in abdominal MRI, and pancreatic mass segmentation in CT images. Our method can provide new insights for interpreting and explainable AI systems for medical image segmentation. Our code is available on: https://github.com/shengfly/ProtoSeg.     

Distributed contrastive learning for medical image segmentation

Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can learn a shared model from decentralized data. But traditional FL requires fully-labeled data for training, which is very expensive to obtain. Self-supervised contrastive learning (CL) can learn from unlabeled data for pre-training, followed by fine-tuning with limited annotations. However, when adopting CL in FL, the limited data diversity on each site makes federated contrastive learning (FCL) ineffective. In this work, we propose two federated self-supervised learning frameworks for volumetric medical image segmentation with limited annotations. The first one features high accuracy and fits high-performance servers with high-speed connections. The second one features lower communication costs, suitable for mobile devices. In the first framework, features are exchanged during FCL to provide diverse contrastive data to each site for effective local CL while keeping raw data private. Global structural matching aligns local and remote features for a unified feature space among different sites. In the second framework, to reduce the communication cost for feature exchanging, we propose an optimized method FCLOpt that does not rely on negative samples. To reduce the communications of model download, we propose the predictive target network update (PTNU) that predicts the parameters of the target network. Based on PTNU, we propose the distance prediction (DP) to remove most of the uploads of the target network. Experiments on a cardiac MRI dataset show the proposed two frameworks substantially improve the segmentation and generalization performance compared with state-of-the-art techniques.     

Development of a human brain diffusion tensor template

The development of a brain template for diffusion tensor imaging (DTI) is crucial for comparisons of neuronal structural integrity and brain connectivity across populations, as well as for the development of a white matter atlas. Previous efforts to produce a DTI brain template have been compromised by factors related to image quality, the effectiveness of the image registration approach, the appropriateness of subject inclusion criteria, and the completeness and accuracy of the information summarized in the final template. The purpose of this work was to develop a DTI human brain template using techniques that address the shortcomings of previous efforts. Therefore, data containing minimal artifacts were first obtained on 67 healthy human subjects selected from an age-group with relatively similar diffusion characteristics (20–40 years of age), using an appropriate DTI acquisition protocol. Non-linear image registration based on mean diffusion-weighted and fractional anisotropy images was employed. DTI brain templates containing median and mean tensors were produced in ICBM-152 space and made publicly available. The resulting set of DTI templates is characterized by higher image sharpness, provides the ability to distinguish smaller white matter fiber structures, contains fewer image artifacts, than previously developed templates, and to our knowledge, is one of only two templates produced based on a relatively large number of subjects. Furthermore, median tensors were shown to better preserve the diffusion characteristics at the group level than mean tensors. Finally, white matter fiber tractography was applied on the template and several fiber-bundles were traced.     

Case study: an evaluation of user-assisted hierarchical watershed segmentation

This paper evaluates the effectiveness of an interactive, three-dimensional image segmentation technique that relies on watersheds. This paper presents two user-based case studies, which include two different groups of domain experts. Subjects manipulate a graphics-based front end to a hierarchy of segmented regions generated from a watershed segmentation algorithm, which is implemented in the Insight Toolkit. In the first study, medical students segment several different anatomical structures from the Visible Human Female head and neck color cryosection data. In the second study, radiologists use the interactive tool to produce models of brain tumors from MRI data. This paper presents a quantitative and qualitative comparison against hand contouring. To quantify accuracy, we estimate ground truth from the hand-contouring data using the Simultaneous Truth and Performance Estimation algorithm. We also apply metrics from the literature to estimate precision and efficiency. The watershed segmentation technique showed improved subject interaction times and increased inter-subject precision over hand contouring, with quality that is visually and statistically comparable. The analysis also identifies some failures in the watershed technique, where edges were poorly defined in the data, and note a trend in the hand-contouring results toward systematically larger segmentations, which raises questions about the wisdom of using expert segmentations to define ground truth.