Appearance-based modeling for segmentation of hippocampus and amygdala using multi-contrast MR imaging

A new automatic model-based segmentation scheme that combines level set shape modeling and active appearance modeling (AAM) is presented. Since different MR image contrasts can yield complementary information, multi-contrast images can be incorporated into the active appearance modeling to improve segmentation performance. During active appearance modeling, the weighting of each contrast is optimized to account for the potentially varying contribution of each image while optimizing the model parameters that correspond to the shape and appearance eigen-images in order to minimize the difference between the multi-contrast test images and the ones synthesized from the shape and appearance modeling. As appearance-based modeling techniques are dependent on the initial alignment of training data, we compare (i) linear alignment of whole brain, (ii) linear alignment of a local volume of interest and (iii) non-linear alignment of a local volume of interest. The proposed segmentation scheme can be used to segment human hippocampi (HC) and amygdalae (AG), which have weak intensity contrast with their background in MRI. The experiments demonstrate that non-linear alignment of training data yields the best results and that multimodal segmentation using T1-weighted, T2-weighted and proton density-weighted images yields better segmentation results than any single contrast. In a four-fold cross validation with eighty young normal subjects, the method yields a mean Dice к of 0.87 with intraclass correlation coefficient (ICC) of 0.946 for HC and a mean Dice к of 0.81 with ICC of 0.924 for AG between manual and automatic labels.     

Fully automatic segmentation of the mitral leaflets in 3D transesophageal echocardiographic images using multi-atlas joint label fusion and deformable medial modeling

Comprehensive visual and quantitative analysis of in vivo human mitral valve morphology is central to the diagnosis and surgical treatment of mitral valve disease. Real-time 3D transesophageal echocardiography (3D TEE) is a practical, highly informative imaging modality for examining the mitral valve in a clinical setting. To facilitate visual and quantitative 3D TEE image analysis, we describe a fully automated method for segmenting the mitral leaflets in 3D TEE image data. The algorithm integrates complementary probabilistic segmentation and shape modeling techniques (multi-atlas joint label fusion and deformable modeling with continuous medial representation) to automatically generate 3D geometric models of the mitral leaflets from 3D TEE image data. These models are unique in that they establish a shape-based coordinate system on the valves of different subjects and represent the leaflets volumetrically, as structures with locally varying thickness. In this work, expert image analysis is the gold standard for evaluating automatic segmentation. Without any user interaction, we demonstrate that the automatic segmentation method accurately captures patient-specific leaflet geometry at both systole and diastole in 3D TEE data acquired from a mixed population of subjects with normal valve morphology and mitral valve disease.     

Statistical shape models for 3D medical image segmentation: A review

Statistical shape models (SSMs) have by now been firmly established as a robust tool for segmentation of medical images. While 2D models have been in use since the early 1990s, wide-spread utilization of three-dimensional models appeared only in recent years, primarily made possible by breakthroughs in automatic detection of shape correspondences. In this article, we review the techniques required to create and employ these 3D SSMs. While we concentrate on landmark-based shape representations and thoroughly examine the most popular variants of Active Shape and Active Appearance models, we also describe several alternative approaches to statistical shape modeling. Structured into the topics of shape representation, model construction, shape correspondence, local appearance models and search algorithms, we present an overview of the current state of the art in the field. We conclude with a survey of applications in the medical field and a discussion of future developments.     

Towards robust and effective shape modeling: sparse shape composition

Organ shape plays an important role in various clinical practices, e.g., diagnosis, surgical planning and treatment evaluation. It is usually derived from low level appearance cues in medical images. However, due to diseases and imaging artifacts, low level appearance cues might be weak or misleading. In this situation, shape priors become critical to infer and refine the shape derived by image appearances. Effective modeling of shape priors is challenging because: (1) shape variation is complex and cannot always be modeled by a parametric probability distribution; (2) a shape instance derived from image appearance cues (input shape) may have gross errors; and (3) local details of the input shape are difficult to preserve if they are not statistically significant in the training data. In this paper we propose a novel Sparse Shape Composition model (SSC) to deal with these three challenges in a unified framework. In our method, a sparse set of shapes in the shape repository is selected and composed together to infer/refine an input shape. The a priori information is thus implicitly incorporated on-the-fly. Our model leverages two sparsity observations of the input shape instance: (1) the input shape can be approximately represented by a sparse linear combination of shapes in the shape repository; (2) parts of the input shape may contain gross errors but such errors are sparse. Our model is formulated as a sparse learning problem. Using L1 norm relaxation, it can be solved by an efficient expectation-maximization (EM) type of framework. Our method is extensively validated on two medical applications, 2D lung localization in X-ray images and 3D liver segmentation in low-dose CT scans. Compared to state-of-the-art methods, our model exhibits better performance in both studies.     

3D image segmentation using combined shape-intensity prior models

Objective Statistical models for medical images have been developed to increase robustness in the segmentation process. In this project, a fully automatic approach to build a statistical shape-intensity model and combine this model with level set segmentation was designed, implemented and tested by applying the algorithm to clinical image data. Methods By using a hierarchical registration approach based on mutual information and demons registration, 3D statistical shape-intensity models were created by applying Principal Component Analysis. Using these models in combination with level set segmentation results in a fully automatic modeling and segmentation pipeline. Results Examples for shape-intensity models were synthesized and these models were used to automatically segment 3D MRI and CT images. Quantitative evaluation of the framework was performed by comparing automatic segmentation results to segmentation results of medical experts. Conclusion Evaluation tests in which this method was used for the automatic segmentation of femora and cardiac MRI endocardial surfaces are very promising. The implementation of an additional cost function term and the addition of information about the surroundings of an organ in the model are currently under development.     

Segmentation of embryonic and fetal 3D ultrasound images based on pixel intensity distributions and shape priors

This paper presents a novel variational segmentation framework combining shape priors and parametric intensity distribution modeling for extracting the fetal envelope on 3D obstetric ultrasound images. To overcome issues related to poor image quality and missing boundaries, we inject three types of information in the segmentation process: tissue-specific parametric modeling of pixel intensities, a shape prior for the fetal envelope and a shape model of the fetus’ back. The shape prior is encoded with Legendre moments and used to constraint the evolution of a level-set function. The back model is used to post-process the segmented fetal envelope. Results are presented on 3D ultrasound data and compared to a set of manual segmentations. The robustness of the algorithm is studied, and both visual and quantitative comparisons show satisfactory results obtained by the proposed method on the tested dataset.     

Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI

We present a framework for the analysis of short axis cardiac MRI, using statistical models of shape and appearance. The framework integrates temporal and structural constraints and avoids common optimization problems inherent in such high dimensional models. The first contribution is the introduction of an algorithm for fitting 3D active appearance models (AAMs) on short axis cardiac MRI. We observe a 44-fold increase in fitting speed and a segmentation accuracy that is on par with Gauss-Newton optimization, one of the most widely used optimization algorithms for such problems. The second contribution involves an investigation on hierarchical 2D+time active shape models (ASMs), that integrate temporal constraints and simultaneously improve the 3D AAM based segmentation. We obtain encouraging results (endocardial/epicardial error 1.43+/-0.49 mm/1.51+/-0.48 mm) on 7980 short axis cardiac MR images acquired from 33 subjects. We have placed our dataset online, for the community to use and build upon.     

Fully automatic segmentation of the Femur from 3D-CT images using primitive shape recognition and statistical shape models

Purpose

Femur segmentation is well established and widely used in computer-assisted orthopedic surgery. However, most of the robust segmentation methods such as statistical shape models (SSM) require human intervention to provide an initial position for the SSM. In this paper, we propose to overcome this problem and provide a fully automatic femur segmentation method for CT images based on primitive shape recognition and SSM.

Method

Femur segmentation in CT scans was performed using primitive shape recognition based on a robust algorithm such as the Hough transform and RANdom SAmple Consensus. The proposed method is divided into 3 steps: (1) detection of the femoral head as sphere and the femoral shaft as cylinder in the SSM and the CT images, (2) rigid registration between primitives of SSM and CT image to initialize the SSM into the CT image, and (3) fitting of the SSM to the CT image edge using an affine transformation followed by a nonlinear fitting.

Results

The automated method provided good results even with a high number of outliers. The difference of segmentation error between the proposed automatic initialization method and a manual initialization method is less than 1 mm.

Conclusion

The proposed method detects primitive shape position to initialize the SSM into the target image. Based on primitive shapes, this method overcomes the problem of inter-patient variability. Moreover, the results demonstrate that our method of primitive shape recognition can be used for 3D SSM initialization to achieve fully automatic segmentation of the femur.     

Low-rank and sparse decomposition based shape model and probabilistic atlas for automatic pathological organ segmentation

One major limiting factor that prevents the accurate delineation of human organs has been the presence of severe pathology and pathology affecting organ borders. Overcoming these limitations is exactly what we are concerned in this study. We propose an automatic method for accurate and robust pathological organ segmentation from CT images. The method is grounded in the active shape model (ASM) framework. It leverages techniques from low-rank and sparse decomposition (LRSD) theory to robustly recover a subspace from grossly corrupted data. We first present a population-specific LRSD-based shape prior model, called LRSD-SM, to handle non-Gaussian gross errors caused by weak and misleading appearance cues of large lesions, complex shape variations, and poor adaptation to the finer local details in a unified framework. For the shape model initialization, we introduce a method based on patient-specific LRSD-based probabilistic atlas (PA), called LRSD-PA, to deal with large errors in atlas-to-target registration and low likelihood of the target organ. Furthermore, to make our segmentation framework more efficient and robust against local minima, we develop a hierarchical ASM search strategy. Our method is tested on the SLIVER07 database for liver segmentation competition, and ranks 3rd in all the published state-of-the-art automatic methods. Our method is also evaluated on some pathological organs (pathological liver and right lung) from 95 clinical CT scans and its results are compared with the three closely related methods. The applicability of the proposed method to segmentation of the various pathological organs (including some highly severe cases) is demonstrated with good results on both quantitative and qualitative experimentation; our segmentation algorithm can delineate organ boundaries that reach a level of accuracy comparable with those of human raters.     

Optimizing boundary detection via Simulated Search with applications to multi-modal heart segmentation

Segmentation of medical images can be achieved with the help of model-based algorithms. Reliable boundary detection is a crucial component to obtain robust and accurate segmentation results and to enable full automation. This is especially important if the anatomy being segmented is too variable to initialize a mean shape model such that all surface regions are close to the desired contours. Several boundary detection algorithms are widely used in the literature. Most use some trained image appearance model to characterize and detect the desired boundaries. Although parameters of the boundary detection can vary over the model surface and are trained on images, their performance (i.e., accuracy and reliability of boundary detection) can only be assessed as an integral part of the entire segmentation algorithm. In particular, assessment of boundary detection cannot be done locally and independently on model parameterization and internal energies controlling geometric model properties.In this paper, we propose a new method for the local assessment of boundary detection called Simulated Search. This method takes any boundary detection function and evaluates its performance for a single model landmark in terms of an estimated geometric boundary detection error. In consequence, boundary detection can be optimized per landmark during model training. We demonstrate the success of the method for cardiac image segmentation. In particular we show that the Simulated Search improves the capture range and the accuracy of the boundary detection compared to a traditional training scheme. We also illustrate how the Simulated Search can be used to identify suitable classes of features when addressing a new segmentation task. Finally, we show that the Simulated Search enables multi-modal heart segmentation using a single algorithmic framework. On computed tomography and magnetic resonance images, average segmentation errors (surface-to-surface distances) for the four chambers and the trunks of the large vessels are in the order of 0.8 mm. For 3D rotational X-ray angiography images of the left atrium and pulmonary veins, the average error is 1.3 mm. In all modalities, the locally optimized boundary detection enables fully automatic segmentation.     

A coupled deformable model for tracking myocardial borders from real-time echocardiography using an incompressibility constraint

Real-time three-dimensional (RT3D) echocardiography is a new image acquisition technique that allows instantaneous acquisition of volumetric images for quantitative assessment of cardiac morphology and function. To quantify many important diagnostic parameters, such as ventricular volume, ejection fraction, and cardiac output, an automatic algorithm to delineate the left ventricle (LV) from RT3D echocardiographic images is essential. While a number of efforts have been made towards segmentation of the LV endocardial (ENDO) boundaries, the segmentation of epicardial (EPI) boundaries remains problematic. In this paper, we present a coupled deformable model that addresses this problem. The idea behind our method is that the volume of the myocardium is close to being constant during a cardiac cycle and our model uses this coupling as an important constraint. We employ two surfaces, each driven by the image-derived information that takes into account ultrasound physics by modeling the speckle statistics using the Nakagami distribution while maintaining the coupling. By simultaneously evolving two surfaces, the final segmentation of the myocardium is thus achieved. Results from 80 sets of synthetic data and 286 sets of real canine data were evaluated against the ground truth and against outlines from three independent observers, respectively. We show that results obtained with our incompressibility constraint were more accurate than those obtained without constraint or with a wall thickness constraint, and were comparable to those from manual segmentation.     

Carotid vasculature modeling from patient CT angiography studies for interventional procedures simulation

Objective

A practical method for patient-specific modeling of the aortic arch and the entire carotid vasculature from computed tomography angiography (CTA) scans for morphologic analysis and for interventional procedure simulation.

Materials and methods

The method starts with the automatic watershed-based segmentation of the aorta and the construction of an a-priori intensity probability distribution function for arteries. The carotid arteries are then segmented with a graph min-cut method based on a new edge weighting function that adaptively couples voxel intensity, intensity prior, and local vesselness shape prior. Finally, the same graph-cut optimization framework is used to interactively remove a few unwanted veins segments and to fill in minor vessel discontinuities caused by intensity variations.

Results

We validate our modeling method with two experimental studies on 71 multicenter clinical CTA datasets, including carotid bifurcation lumen segmentation on 56 CTAs from the MICCAI??2009 3D Segmentation Challenge. Segmentation results show that our method is comparable to the best existing methods and was successful in modeling the entire carotid vasculature with a Dice similarity measure of 84.5% (SD?=?3.3%) and MSSD 0.48?mm (SD?=?0.12?mm.) Simulation study shows that patient-specific simulations with four patient-specific models generated by our segmentation method on the ANGIO Mentor^TM simulator platform are robust, realistic, and greatly improve the simulation.

Conclusion

This constitutes a proof-of-concept that patient-specific CTA-based modeling and simulation of carotid interventional procedures are practical in a clinical environment.     

基于MeanShift方法的肝脏CT图像的自动分割

目的 探讨基于Mean Shift方法的肝脏CT图像的自动分割算法,以实现肝脏的自动分割。方法 首先对原始图像进行单次Mean Shift平滑 ,滤除噪声的影响以增强算法的鲁棒性,然后通过Mean Shift迭代自动选取初始种子点,最后采用基于区域生长的方法实现肝脏CT图像的自动分割。结果 实验证明此方法是一个准确、快速和有效的肝脏自动分割方法。结论 采用本文中提出的方法,可有效地实现肝脏的自动分割。     

Mitral annulus segmentation from four-dimensional ultrasound using a valve state predictor and constrained optical flow

Measurement of the shape and motion of the mitral valve annulus has proven useful in a number of applications, including pathology diagnosis and mitral valve modeling. Current methods to delineate the annulus from four-dimensional (4D) ultrasound, however, either require extensive overhead or user-interaction, become inaccurate as they accumulate tracking error, or they do not account for annular shape or motion. This paper presents a new 4D annulus segmentation method to account for these deficiencies. The method builds on a previously published three-dimensional (3D) annulus segmentation algorithm that accurately and robustly segments the mitral annulus in a frame with a closed valve. In the 4D method, a valve state predictor determines when the valve is closed. Subsequently, the 3D annulus segmentation algorithm finds the annulus in those frames. For frames with an open valve, a constrained optical flow algorithm is used to the track the annulus. The only inputs to the algorithm are the selection of one frame with a closed valve and one user-specified point near the valve, neither of which needs to be precise. The accuracy of the tracking method is shown by comparing the tracking results to manual segmentations made by a group of experts, where an average RMS difference of 1.67±0.63mm was found across 30 tracked frames.     

Shape regression machine and efficient segmentation of left ventricle endocardium from 2D B-mode echocardiogram

We present a machine learning approach called shape regression machine (SRM) for efficient segmentation of an anatomic structure that exhibits a deformable shape in a medical image, e.g., left ventricle endocardial wall in an echocardiogram. The SRM achieves efficient segmentation via statistical learning of the interrelations among shape, appearance, and anatomy, which are exemplified by an annotated database. The SRM is a two-stage approach. In the first stage that estimates a rigid shape to solve an automatic initialization problem, it derives a regression solution to object detection that needs just one scan in principle and a sparse set of scans in practice, avoiding the exhaustive scanning required by the state-of-the-art classification-based detection approach while yielding comparable detection accuracy. In the second stage that estimates the nonrigid shape, it again learns a nonlinear regressor to directly associate nonrigid shape with image appearance. The underpinning of both stages is a novel image-based boosting ridge regression (IBRR) method that enables multivariate, nonlinear modeling and accommodates fast evaluation. We demonstrate the efficiency and effectiveness of the SRM using experiments on segmenting the left ventricle endocardium from a B-mode echocardiogram of apical four chamber view. The proposed algorithm is able to automatically detect and accurately segment the LV endocardial border in about 120 ms.     

Deformable segmentation via sparse representation and dictionary learning

“Shape” and “appearance”, the two pillars of a deformable model, complement each other in object segmentation. In many medical imaging applications, while the low-level appearance information is weak or mis-leading, shape priors play a more important role to guide a correct segmentation, thanks to the strong shape characteristics of biological structures. Recently a novel shape prior modeling method has been proposed based on sparse learning theory. Instead of learning a generative shape model, shape priors are incorporated on-the-fly through the sparse shape composition (SSC). SSC is robust to non-Gaussian errors and still preserves individual shape characteristics even when such characteristics is not statistically significant.Although it seems straightforward to incorporate SSC into a deformable segmentation framework as shape priors, the large-scale sparse optimization of SSC has low runtime efficiency, which cannot satisfy clinical requirements. In this paper, we design two strategies to decrease the computational complexity of SSC, making a robust, accurate and efficient deformable segmentation system. (1) When the shape repository contains a large number of instances, which is often the case in 2D problems, K-SVD is used to learn a more compact but still informative shape dictionary. (2) If the derived shape instance has a large number of vertices, which often appears in 3D problems, an affinity propagation method is used to partition the surface into small sub-regions, on which the sparse shape composition is performed locally. Both strategies dramatically decrease the scale of the sparse optimization problem and hence speed up the algorithm. Our method is applied on a diverse set of biomedical image analysis problems. Compared to the original SSC, these two newly-proposed modules not only significant reduce the computational complexity, but also improve the overall accuracy.