A hybrid framework for 3D medical image segmentation

In this paper we propose a novel hybrid 3D segmentation framework which combines Gibbs models, marching cubes and deformable models. In the framework, first we construct a new Gibbs model whose energy function is defined on a high order clique system. The new model includes both region and boundary information during segmentation. Next we improve the original marching cubes method to construct 3D meshes from Gibbs models' output. The 3D mesh serves as the initial geometry of the deformable model. Then we deform the deformable model using external image forces so that the model converges to the object surface. We run the Gibbs model and the deformable model recursively by updating the Gibbs model's parameters using the region and boundary information in the deformable model segmentation result. In our approach, the hybrid combination of region-based methods and boundary-based methods results in improved segmentations of complex structures. The benefit of the methodology is that it produces high quality segmentations of 3D structures using little prior information and minimal user intervention. The modules in this segmentation methodology are developed within the context of the Insight ToolKit (ITK). We present experimental segmentation results of brain tumors and evaluate our method by comparing experimental results with expert manual segmentations. The evaluation results show that the methodology achieves high quality segmentation results with computational efficiency. We also present segmentation results of other clinical objects to illustrate the strength of the methodology as a generic segmentation framework.     

A quantitative evaluation of cross-participant registration techniques for MRI studies of the medial temporal lobe

Accurate cross-participant alignment within the medial temporal lobe (MTL) region is critical for fMRI studies of memory. However, traditional alignment approaches have been exceptionally poor at registering structures in this area due to significant inter-individual anatomic variability. In this study, we evaluated the performance of twelve registration approaches. Specifically, we extended several traditional approaches such as SPM's normalization and AFNI's 3dWarpDrive to improve the quality of alignment in the MTL region by using weighting masks or applying the transformations directly to ROI segmentations. In addition, we evaluated the performance of three fully deformable methods, DARTEL, Diffeomorphic Demons, and LDDMM that are effectively unconstrained by number of degrees of freedom. For each, we first assessed the method's ability to achieve optimal overlap between segmentations of subregions of the MTL across participants. Then we evaluated the smoothness of group average structural images aligned using each method to assess the blur that results when voxels of different tissue types are averaged together. In general, we found that when anatomical segmentation is possible, substantial improvement in registration accuracy can be gained in the MTL even with a small number of deformations. When segmentation is not possible, the fully deformable models provide some improvement over more traditional approaches and in a few cases even approach the performance of the ROI-based approaches. The best performance is achieved when both methods are combined. We note that these conclusions are not limited to the MTL and are easily extendable to other areas of the brain.     

Atlas-based hippocampus segmentation in Alzheimer's disease and mild cognitive impairment

This study assesses the performance of public-domain automated methodologies for MRI-based segmentation of the hippocampus in elderly subjects with Alzheimer's disease (AD) and mild cognitive impairment (MCI). Structural MR images of 54 age- and gender-matched healthy elderly individuals, subjects with probable AD, and subjects with MCI were collected at the University of Pittsburgh Alzheimer's Disease Research Center. Hippocampi in subject images were automatically segmented by using AIR, SPM, FLIRT, and the fully deformable method of Chen to align the images to the Harvard atlas, MNI atlas, and randomly selected, manually labeled subject images ("cohort atlases"). Mixed-effects statistical models analyzed the effects of side of the brain, disease state, registration method, choice of atlas, and manual tracing protocol on the spatial overlap between automated segmentations and expert manual segmentations. Registration methods that produced higher degrees of geometric deformation produced automated segmentations with higher agreement with manual segmentations. Side of the brain, presence of AD, choice of reference image, and manual tracing protocol were also significant factors contributing to automated segmentation performance. Fully automated techniques can be competitive with human raters on this difficult segmentation task, but a rigorous statistical analysis shows that a variety of methodological factors must be carefully considered to insure that automated methods perform well in practice. The use of fully deformable registration methods, cohort atlases, and user-defined manual tracings are recommended for highest performance in fully automated hippocampus segmentation.     

Learning to segment fetal brain tissue from noisy annotations

Automatic fetal brain tissue segmentation can enhance the quantitative assessment of brain development at this critical stage. Deep learning methods represent the state of the art in medical image segmentation and have also achieved impressive results in brain segmentation. However, effective training of a deep learning model to perform this task requires a large number of training images to represent the rapid development of the transient fetal brain structures. On the other hand, manual multi-label segmentation of a large number of 3D images is prohibitive. To address this challenge, we segmented 272 training images, covering 19–39 gestational weeks, using an automatic multi-atlas segmentation strategy based on deformable registration and probabilistic atlas fusion, and manually corrected large errors in those segmentations. Since this process generated a large training dataset with noisy segmentations, we developed a novel label smoothing procedure and a loss function to train a deep learning model with smoothed noisy segmentations. Our proposed methods properly account for the uncertainty in tissue boundaries. We evaluated our method on 23 manually-segmented test images of a separate set of fetuses. Results show that our method achieves an average Dice similarity coefficient of 0.893 and 0.916 for the transient structures of younger and older fetuses, respectively. Our method generated results that were significantly more accurate than several state-of-the-art methods including nnU-Net that achieved the closest results to our method. Our trained model can serve as a valuable tool to enhance the accuracy and reproducibility of fetal brain analysis in MRI.     

Prostate boundary detection and volume estimation using TRUS images for brachytherapy applications

Objective: Authors propose a semi-automatic segmentation algorithm for three-dimensional prostate boundary detection from trans-rectal ultrasound images. As a part of brachytherapy treatment with seeds for early stage prostate cancer, a patient’s prostate is scanned using a trans-rectal ultrasound probe, its boundary is manually outlined, and its volume is estimated for dosimetry purposes. Proposed algorithm requires a reduced amount of radiologist’s input, and thus speeds up the surgical procedure. Methods: The proposed segmentation algorithm utilizes texture differences between ultrasound images of the prostate and the surrounding tissues. It is carried out in the polar coordinate system and uses three-dimensional data correlation to improve the smoothness and reliability of the segmentation. The algorithm is applied to axial trans-rectal ultrasound images and the results are compared to the “ground truth” set by manual prostate boundary outlining (by experienced radiologist). Method is validated on six patients. Results: In our tests, the proposed algorithm estimated prostate volume within 95% of the original radiologist’s estimate. Conclusions: The boundary segmentation obtained from the algorithm can reduce manual input by a factor of 3, without significantly affecting the accuracy of the segmentation. The reduction in the manual input reduces the overall brachytherapy procedure time.     

Automated Registration of Freehand B-Mode Ultrasound and Magnetic Resonance Imaging of the Carotid Arteries Based on Geometric Features

An automated method for registering B-mode ultrasound (US) and magnetic resonance imaging (MRI) of the carotid arteries is proposed. The registration uses geometric features, namely, lumen centerlines and lumen segmentations, which are extracted fully automatically from the images after manual annotation of three seed points in US and MRI. The registration procedure starts with alignment of the lumen centerlines using a point-based registration algorithm. The resulting rigid transformation is used to initialize a rigid and subsequent non-rigid registration procedure that jointly aligns centerlines and segmentations by minimizing a weighted sum of the Euclidean distance between centerlines and the dissimilarity between segmentations. The method was evaluated in 28 carotid arteries from eight patients and six healthy volunteers. First, the automated US lumen segmentation method was validated and optimized in a cross-validation experiment. Next, the effect of the weighting parameter of the proposed registration dissimilarity metric and the control point spacing in the non-rigid registration was evaluated. Finally, the proposed registration method was evaluated in comparison to an existing intensity-and-point-based method, a registration using only the centerlines and a registration using manual US lumen segmentations. Registration accuracy was measured in terms of the mean surface distance between manual US segmentations and the registered MRI segmentations. The average mean surface distance was 0.78 ± 0.34 mm for all subjects, 0.65 ± 0.09 mm for healthy volunteers and 0.87 ± 0.42 mm for patients. The results for the complete set were significantly better (Wilcoxon test, p < 0.01) than the results for the intensity-and-point-based method and the centerline-based registration method. We conclude that the proposed method can robustly and accurately register US and MR images of the carotid artery, allowing multimodal analysis of the carotid plaque to improve plaque assessment.     

Hierarchical max-flow segmentation framework for multi-atlas segmentation with Kohonen self-organizing map based Gaussian mixture modeling

The incorporation of intensity, spatial, and topological information into large-scale multi-region segmentation has been a topic of ongoing research in medical image analysis. Multi-region segmentation problems, such as segmentation of brain structures, pose unique challenges in image segmentation in which regions may not have a defined intensity, spatial, or topological distinction, but rely on a combination of the three. We propose a novel framework within the Advanced segmentation tools (ASETS)², which combines large-scale Gaussian mixture models trained via Kohonen self-organizing maps, with deformable registration, and a convex max-flow optimization algorithm incorporating region topology as a hierarchy or tree. Our framework is validated on two publicly available neuroimaging datasets, the OASIS and MRBrainS13 databases, against the more conventional Potts model, achieving more accurate segmentations. Each component is accelerated using general-purpose programming on graphics processing Units to ensure computational feasibility.     

White matter lesion extension to automatic brain tissue segmentation on MRI

A fully automated brain tissue segmentation method is optimized and extended with white matter lesion segmentation. Cerebrospinal fluid (CSF), gray matter (GM) and white matter (WM) are segmented by an atlas-based k-nearest neighbor classifier on multi-modal magnetic resonance imaging data. This classifier is trained by registering brain atlases to the subject. The resulting GM segmentation is used to automatically find a white matter lesion (WML) threshold in a fluid-attenuated inversion recovery scan. False positive lesions are removed by ensuring that the lesions are within the white matter. The method was visually validated on a set of 209 subjects. No segmentation errors were found in 98% of the brain tissue segmentations and 97% of the WML segmentations. A quantitative evaluation using manual segmentations was performed on a subset of 6 subjects for CSF, GM and WM segmentation and an additional 14 for the WML segmentations. The results indicated that the automatic segmentation accuracy is close to the interobserver variability of manual segmentations.     

3D Segmentation of Medical Images Using a Fast Multistage Hybrid Algorithm

In this paper, we propose a fast multistage hybrid algorithm for 3D segmentation of medical images. We first employ a morphological recursive erosion operation to reduce the connectivity between the object to be segmented and its neighborhood; then the fast marching method is used to greatly accelerate the initial propagation of a surface front from the user defined seed structure to a surface close to the desired boundary; a morphological reconstruction method then operates on this surface to achieve an initial segmentation result; and finally morphological recursive dilation is employed to recover any structure lost in the first stage of the algorithm. This approach is tested on 60 CT or MRI images of the brain, heart and urinary system, to demonstrate the robustness of this technique across a variety of imaging modalities and organ systems. The algorithm is also validated against datasets for which “truth” is known. These measurements revealed that the algorithm achieved a mean “similarity index” of 0.966 across the three organ systems. The execution time for this algorithm, when run on a 550 MHz Dual PIII-based PC runningWindows NT, and extracting the cortex from brain MRIs, the cardiac surface from dynamic CT, and the kidneys from 3D CT, was 38, 46 and 23 s, respectively.     

Anatomically constrained and attention-guided deep feature fusion for joint segmentation and deformable medical image registration

The main objective of anatomically plausible results for deformable image registration is to improve model’s registration accuracy by minimizing the difference between a pair of fixed and moving images. Since many anatomical features are closely related to each other, leveraging supervision from auxiliary tasks (such as supervised anatomical segmentation) has the potential to enhance the realism of the warped images after registration. In this work, we employ a Multi-Task Learning framework to formulate registration and segmentation as a joint issue, in which we utilize anatomical constraint from auxiliary supervised segmentation to enhance the realism of the predicted images. First, we propose a Cross-Task Attention Block to fuse the high-level feature from both the registration and segmentation network. With the help of initial anatomical segmentation, the registration network can benefit from learning the task-shared feature correlation and rapidly focusing on the parts that need deformation. On the other hand, the anatomical segmentation discrepancy from ground-truth fixed annotations and predicted segmentation maps of initial warped images are integrated into the loss function to guide the convergence of the registration network. Ideally, a good deformation field should be able to minimize the loss function of registration and segmentation. The voxel-wise anatomical constraint inferred from segmentation helps the registration network to reach a global optimum for both deformable and segmentation learning. Both networks can be employed independently during the testing phase, enabling only the registration output to be predicted when the segmentation labels are unavailable. Qualitative and quantitative results indicate that our proposed methodology significantly outperforms the previous state-of-the-art approaches on inter-patient brain MRI registration and pre- and intra-operative uterus MRI registration tasks within our specific experimental setup, which leads to state-of-the-art registration quality scores of 0.755 and 0.731 (i.e., by 0.8% and 0.5% increases) DSC for both tasks, respectively.     

Automatic morphology-based brain segmentation (MBRASE) from MRI-T1 data

A method called morphology-based brain segmentation (MBRASE) has been developed for fully automatic segmentation of the brain from T1-weighted MR image data. The starting point is a supervised segmentation technique, which has proven highly effective and accurate for quantitation and visualization purposes. The proposed method automates the required user interaction, i.e., defining a seed point and a threshold range, and is based on the simple operations thresholding, erosion, and geodesic dilation. The thresholds are detected in a region growing process and are defined by connections of the brain to other tissues. The method is first evaluated on three computer simulated datasets by comparing the automated segmentations with the original distributions. The second evaluation is done on a total of 30 patient datasets, by comparing the automated segmentations with supervised segmentations carried out by a neuroanatomy expert. The comparison between two binary segmentations is performed both quantitatively and qualitatively. The automated segmentations are found to be accurate and robust. Consequently, the proposed method can be used as a default segmentation for quantitation and visualization of the human brain from T1-weighted MR images in routine clinical procedures.     

Combining atlas based segmentation and intensity classification with nearest neighbor transform and accuracy weighted vote

In this paper, different methods to improve atlas based segmentation are presented. The first technique is a new mapping of the labels of an atlas consistent with a given intensity classification segmentation. This new mapping combines the two segmentations using the nearest neighbor transform and is especially effective for complex and folded regions like the cortex where the registration is difficult. Then, in a multi atlas context, an original weighting is introduced to combine the segmentation of several atlases using a voting procedure. This weighting is derived from statistical classification theory and is computed offline using the atlases as a training dataset. Concretely, the accuracy map of each atlas is computed and the vote is weighted by the accuracy of the atlases. Numerical experiments have been performed on publicly available in vivo datasets and show that, when used together, the two techniques provide an important improvement of the segmentation accuracy.     

3D image segmentation by using statistical deformation models and level sets

Purpose Improved segmentation of soft objects was sought using a new method that combines level set segmentation with statistical deformation models, using prior knowledge of the shape of an object as well as information derived from the input image. Methods Statistical deformation models were created using Euclidian distance functions of binary data and a multi-hierarchical registration approach based on mutual information metric and demons deformable registration. This approach is motivated by the fact that models based on signed distance maps, traditionally combined with level set segmentation can result in irregular shapes and do not establish explicit correspondences. By using statistical deformation models as representation of shape and a maximum a posteriori (MAP) estimation model to estimate the MAP shape of the object to be segmented, a robust segmentation algorithm using accurate shape models could be developed. Results The accuracy and correctness of the synthesized models was evaluated on different 3D objects (cardiac MRI and spinal CT vertebral segment) and the segmentation algorithm was validated by performing different segmentation tasks using various image modalities. The results of this evaluation are very promising and show the potential utility of the approach. Conclusion Initial results demonstrate the approach is feasible and may be advantageous over alternative segmentation methods. Extensions of the model, which also incorporate prior knowledge about the spatial distribution of grey values, are currently under development.     

Muscles of mastication model-based MR image segmentation

Objective The muscles of mastication play a major role in the orodigestive system as the principal motive force for the mandible. An algorithm for segmenting these muscles from magnetic resonance (MR) images was developed and tested. Materials and methods Anatomical information about the muscles of mastication in MR images is used to obtain the spatial relationships relating the muscle region of interest (ROI) and head ROI. A model-based technique that involves the spatial relationships between head and muscle ROIs as well as muscle templates is developed. In the segmentation stage, the muscle ROI is derived from the model. Within the muscle ROI, anisotropic diffusion is applied to smooth the texture, followed by thresholding to exclude bone and fat. The muscle template and morphological operators are employed to obtain an initial estimate of the muscle boundary, which then serves as the input contour to the gradient vector flow snake that iterates to the final segmentation. Results The method was applied to segmentation of the masseter, lateral pterygoid and medial pterygoid in 75 images. The overlap indices (κ) achieved are 91.4, 92.1 and 91.2%, respectively. Conclusion A model-based method for segmenting the muscles of mastication from MR images was developed and tested. The results show good agreement between manual and automatic segmentations.     

Advanced image fusion to overlay coronary sinus anatomy with real-time fluoroscopy to facilitate left ventricular lead implantation in CRT

Background: Failure rate for left ventricular (LV) lead implantation in cardiac resynchronization therapy (CRT) is up to 12%. The use of segmentation tools, advanced image registration software, and high‐fidelity images from computerized tomography (CT) and cardiac magnetic resonance (CMR) of the coronary sinus (CS) can guide LV lead implantation. We evaluated the feasibility of advanced image registration onto live fluoroscopic images to allow successful LV lead placement. Methods: Twelve patients (11 male, 59 ± 16.8 years) undergoing CRT had three‐dimensional (3D) whole‐heart imaging (six CT, six CMR). Eight patients had at least one previously failed LV lead implant. Using segmentation software, anatomical models of the cardiac chambers, CS, and its branches were overlaid onto the live fluoroscopy using a prototype version of the Philips EP Navigator software to guide lead implantation. Results: We achieved high‐fidelity segmentations of cardiac chambers, coronary vein anatomy, and accurate registration between the 3D anatomical models and the live fluoroscopy in all 12 patients confirmed by balloon occlusion angiography. The CS was cannulated successfully in every patient and in 11, an LV lead was implanted successfully. (One patient had no acceptable lead values due to extensive myocardial scar.) Conclusion: Using overlaid 3D segmentations of the CS and cardiac chambers, it is feasible to guide CRT implantation in real time by fusing advanced imaging and fluoroscopy. This enabled successful CRT in a group of patients with previously failed implants. This technology has the potential to facilitate CRT and improve implant success. (PACE 2011; 34:226–234)     

Iterative multi-atlas-based multi-image segmentation with tree-based registration

In this paper, we present a multi-atlas-based framework for accurate, consistent and simultaneous segmentation of a group of target images. Multi-atlas-based segmentation algorithms consider concurrently complementary information from multiple atlases to produce optimal segmentation outcomes. However, the accuracy of these algorithms relies heavily on the precise alignment of the atlases with the target image. In particular, the commonly used pairwise registration may result in inaccurate alignment especially between images with large shape differences. Additionally, when segmenting a group of target images, most current methods consider these images independently with disregard of their correlation, thus resulting in inconsistent segmentations of the same structures across different target images. We propose two novel strategies to address these limitations: 1) a novel tree-based groupwise registration method for concurrent alignment of both the atlases and the target images, and 2) an iterative groupwise segmentation method for simultaneous consideration of segmentation information propagated from all available images, including the atlases and other newly segmented target images. Evaluation based on various datasets indicates that the proposed multi-atlas-based multi-image segmentation (MABMIS) framework yields substantial improvements in terms of consistency and accuracy over methods that do not consider the group of target images holistically.     

Segmentation of the temporalis muscle from MR data

Objective A method for segmenting the temporalis from magnetic resonance (MR) images was developed and tested. The temporalis muscle is one of the muscles of mastication which plays a major role in the mastication system. Materials and methods The temporalis region of interest (ROI) and the head ROI are defined in reference images, from which the spatial relationship between the two ROIs is derived. This relationship is used to define the temporalis ROI in a study image. Range-constrained thresholding is then employed to remove the fat, bone marrow and muscle tendon in the ROI. Adaptive morphological operations are then applied to first remove the brain tissue, followed by the removal of the other soft tissues surrounding the temporalis. Ten adult head MR data sets were processed to test this method. Results Using five data sets each for training and testing, the method was applied to the segmentation of the temporalis in 25 MR images (five from each test set). An average overlap index (κ) of 90.2% was obtained. Applying a leave-one-out evaluation method, an average κ of 90.5% was obtained from 50 test images. Conclusion A method for segmenting the temporalis from MR images was developed and tested on in vivo data sets. The results show that there is consistency between manual and automatic segmentations.     

Automatic anatomical brain MRI segmentation combining label propagation and decision fusion

Regions in three-dimensional magnetic resonance (MR) brain images can be classified using protocols for manually segmenting and labeling structures. For large cohorts, time and expertise requirements make this approach impractical. To achieve automation, an individual segmentation can be propagated to another individual using an anatomical correspondence estimate relating the atlas image to the target image. The accuracy of the resulting target labeling has been limited but can potentially be improved by combining multiple segmentations using decision fusion. We studied segmentation propagation and decision fusion on 30 normal brain MR images, which had been manually segmented into 67 structures. Correspondence estimates were established by nonrigid registration using free-form deformations. Both direct label propagation and an indirect approach were tested. Individual propagations showed an average similarity index (SI) of 0.754+/-0.016 against manual segmentations. Decision fusion using 29 input segmentations increased SI to 0.836+/-0.009. For indirect propagation of a single source via 27 intermediate images, SI was 0.779+/-0.013. We also studied the effect of the decision fusion procedure using a numerical simulation with synthetic input data. The results helped to formulate a model that predicts the quality improvement of fused brain segmentations based on the number of individual propagated segmentations combined. We demonstrate a practicable procedure that exceeds the accuracy of previous automatic methods and can compete with manual delineations.     

Automatic segmentation of calcified plaques and vessel borders in IVUS images

Objective

Intravascular ultrasound (IVUS) is a diagnostic imaging technique for tomographic visualization of coronary arteries. Automatic analysis of IVUS images is difficult due to speckle noise, artifacts of the catheter, and shadows generated by calcifications. We designed and implemented a system for automated segmentation of coronary artery IVUS images.

Methods

Two methods for automatic detection of the intima and the media-adventitia borders in IVUS coronary artery images were developed and compared. The first method uses the parametric deformable models, while the second method is based on the geometric deformable models. The initial locations of the borders are approximated using two different edge detection methods. The final borders are then defined using the two deformable models. Finally, the calcified regions between the extracted borders are identified using a Bayesian classifier. The performance of the proposed methods was evaluated using 60 different IVUS images obtained from 7 patients.

Results

Segmented images were compared with manually outlined contours. We compared the performance of calcified region characterization methods using ROC analysis and by computing the sensitivity and specificity of the Bayesian classifier, thresholding, adaptive thresholding, and textural features. The Bayesian method performed best.

Conclusion

The results shows that the geometric deformable model outperforms the parametric deformable model for automated segmentation of IVUS coronary artery images.     

Non-parametric geodesic active regions: method and evaluation for cerebral aneurysms segmentation in 3DRA and CTA

Segmentation of vascular structures is a difficult and challenging task. In this article, we present an algorithm devised for the segmentation of such structures. Our technique consists in a geometric deformable model with associated energy functional that incorporates high-order multiscale features in a non-parametric statistical framework. Although the proposed segmentation method is generic, it has been applied to the segmentation of cerebral aneurysms in 3DRA and CTA. An evaluation study over 10 clinical datasets indicate that the segmentations obtained by our method present a high overlap index with respect to the ground-truth (91.13% and 73.31%, respectively) and that the mean error distance from the surface to the ground truth is close to the in-plane resolution (0.40 and 0.38 mm, respectively). Besides, our technique favorably compares to other alternative techniques based on deformable models, namely parametric geodesic active regions and active contours without edges.