A 3D interactive multi-object segmentation tool using local robust statistics driven active contours

Extracting anatomical and functional significant structures renders one of the important tasks for both the theoretical study of the medical image analysis, and the clinical and practical community. In the past, much work has been dedicated only to the algorithmic development. Nevertheless, for clinical end users, a well designed algorithm with an interactive software is necessary for an algorithm to be utilized in their daily work. Furthermore, the software would better be open sourced in order to be used and validated by not only the authors but also the entire community. Therefore, the contribution of the present work is twofolds: first, we propose a new robust statistics based conformal metric and the conformal area driven multiple active contour framework, to simultaneously extract multiple targets from MR and CT medical imagery in 3D. Second, an open source graphically interactive 3D segmentation tool based on the aforementioned contour evolution is implemented and is publicly available for end users on multiple platforms. In using this software for the segmentation task, the process is initiated by the user drawn strokes (seeds) in the target region in the image. Then, the local robust statistics are used to describe the object features, and such features are learned adaptively from the seeds under a non-parametric estimation scheme. Subsequently, several active contours evolve simultaneously with their interactions being motivated by the principles of action and reaction-this not only guarantees mutual exclusiveness among the contours, but also no longer relies upon the assumption that the multiple objects fill the entire image domain, which was tacitly or explicitly assumed in many previous works. In doing so, the contours interact and converge to equilibrium at the desired positions of the desired multiple objects. Furthermore, with the aim of not only validating the algorithm and the software, but also demonstrating how the tool is to be used, we provide the reader reproducible experiments that demonstrate the capability of the proposed segmentation tool on several public available data sets.     

A hybrid framework for 3D medical image segmentation

In this paper we propose a novel hybrid 3D segmentation framework which combines Gibbs models, marching cubes and deformable models. In the framework, first we construct a new Gibbs model whose energy function is defined on a high order clique system. The new model includes both region and boundary information during segmentation. Next we improve the original marching cubes method to construct 3D meshes from Gibbs models' output. The 3D mesh serves as the initial geometry of the deformable model. Then we deform the deformable model using external image forces so that the model converges to the object surface. We run the Gibbs model and the deformable model recursively by updating the Gibbs model's parameters using the region and boundary information in the deformable model segmentation result. In our approach, the hybrid combination of region-based methods and boundary-based methods results in improved segmentations of complex structures. The benefit of the methodology is that it produces high quality segmentations of 3D structures using little prior information and minimal user intervention. The modules in this segmentation methodology are developed within the context of the Insight ToolKit (ITK). We present experimental segmentation results of brain tumors and evaluate our method by comparing experimental results with expert manual segmentations. The evaluation results show that the methodology achieves high quality segmentation results with computational efficiency. We also present segmentation results of other clinical objects to illustrate the strength of the methodology as a generic segmentation framework.     

Automatic detection and segmentation of evolving processes in 3D medical images: Application to multiple sclerosis

The study of temporal series of medical images can be helpful for physicians to perform pertinent diagnoses and to help them in the follow-up of a patient: in some diseases, lesions, tumors or anatomical structures vary over time in size, position, composition, etc., either because of a natural pathological process or under the effect of a drug or a therapy. It is a laborious and subjective task to visually and manually analyze such images. Thus the objective of this work was to automatically detect regions with apparent local volume variation with a vector field operator applied to the local displacement field obtained after a non-rigid registration between two successive temporal images. On the other hand, quantitative measurements, such as the volume variation of lesions or segmentation of evolving lesions, are important. By studying the information of apparent shrinking areas in the direct and reverse displacement fields between images, we are able to segment evolving lesions. Then we propose a method to segment lesions in a whole temporal series of images. In this article we apply this approach to automatically detect and segment multiple sclerosis lesions that evolve in time series of MRI scans of the brain. At this stage, we have only applied the approach to a few experimental cases to demonstrate its potential. A clinical validation remains to be done, which will require important additional work.     

A level set method for segmentation of high-resolution polarimetric SAR images using a heterogeneous clutter model

To overcome the problem of strong speckle and texture in high-resolution polarimetric synthetic aperture radar (PolSAR) images, a novel level set segmentation method that uses a heterogeneous clutter model is proposed in this article. Because the KummerU distribution has the capability to describe the statistics of PolSAR imagery in both homogeneous and heterogeneous scenes, it is used to replace the traditional Wishart distribution as the statistical model that defines the energy function for PolSAR images in order to improve the accuracy of the segmentation. Moreover, in order to reduce the computation intensity, an enhanced distance-regularized level set evolution (DRLSE-E) term is applied to improve the computational efficiency. The experimental results obtained using synthetic and real PolSAR images show that the method described has an accuracy 10% better than level set methods based on Wishart distributions. It is also shown that adding the DRLSE-E term reduces the computation time by about a third, thus demonstrating the effectiveness of our method.     

Automatic segmentation of 3D micro-CT coronary vascular images

Although there are many algorithms available in the literature aimed at segmentation and model reconstruction of 3D angiographic images, many are focused on characterizing only a part of the vascular network. This study is motivated by the recent emerging prospects of whole-organ simulations in coronary hemodynamics, autoregulation and tissue oxygen delivery for which anatomically accurate vascular meshes of extended scale are highly desirable. The key requirements of a reconstruction technique for this purpose are automation of processing and sub-voxel accuracy. We have designed a vascular reconstruction algorithm which satisfies these two criteria. It combines automatic seeding and tracking of vessels with radius detection based on active contours. The method was first examined through a series of tests on synthetic data, for accuracy in reproduced topology and morphology of the network and was shown to exhibit errors of less than 0.5 voxel for centerline and radius detections, and 3 degrees for initial seed directions. The algorithm was then applied on real-world data of full rat coronary structure acquired using a micro-CT scanner at 20 microm voxel size. For this, a further validation of radius quantification was carried out against a partially rescanned portion of the network at 8 microm voxel size, which estimated less than 10% radius error in vessels larger than 2 voxels in radius.     

Automated 3D closed surface segmentation: application to vertebral body segmentation in CT images

Purpose

A fully automated segmentation algorithm, progressive surface resolution (PSR), is presented in this paper to determine the closed surface of approximately convex blob-like structures that are common in biomedical imaging. The PSR algorithm was applied to the cortical surface segmentation of 460 vertebral bodies on 46 low-dose chest CT images, which can be potentially used for automated bone mineral density measurement and compression fracture detection.

Methods

The target surface is realized by a closed triangular mesh, which thereby guarantees the enclosure. The surface vertices of the triangular mesh representation are constrained along radial trajectories that are uniformly distributed in 3D angle space. The segmentation is accomplished by determining for each radial trajectory the location of its intersection with the target surface. The surface is first initialized based on an input high confidence boundary image and then resolved progressively based on a dynamic attraction map in an order of decreasing degree of evidence regarding the target surface location.

Results

For the visual evaluation, the algorithm achieved acceptable segmentation for 99.35 % vertebral bodies. Quantitative evaluation was performed on 46 vertebral bodies and achieved overall mean Dice coefficient of 0.939 (with max \(=\) 0.957, min \(=\) 0.906 and standard deviation \(=\) 0.011) using manual annotations as the ground truth.

Conclusions

Both visual and quantitative evaluations demonstrate encouraging performance of the PSR algorithm. This novel surface resolution strategy provides uniform angular resolution for the segmented surface with computation complexity and runtime that are linearly constrained by the total number of vertices of the triangular mesh representation.

     

Statistical shape models for 3D medical image segmentation: A review

Statistical shape models (SSMs) have by now been firmly established as a robust tool for segmentation of medical images. While 2D models have been in use since the early 1990s, wide-spread utilization of three-dimensional models appeared only in recent years, primarily made possible by breakthroughs in automatic detection of shape correspondences. In this article, we review the techniques required to create and employ these 3D SSMs. While we concentrate on landmark-based shape representations and thoroughly examine the most popular variants of Active Shape and Active Appearance models, we also describe several alternative approaches to statistical shape modeling. Structured into the topics of shape representation, model construction, shape correspondence, local appearance models and search algorithms, we present an overview of the current state of the art in the field. We conclude with a survey of applications in the medical field and a discussion of future developments.     

Combinative multi-scale level set framework for echocardiographic image segmentation

In the automatic segmentation of echocardiographic images, a priori shape knowledge has been used to compensate for poor features in ultrasound images. This shape knowledge is often learned via an off-line training process, which requires tedious human effort and is highly expertise-dependent. More importantly, a learned shape template can only be used to segment a specific class of images with similar boundary shape. In this paper, we present a multi-scale level set framework for segmentation of endocardial boundaries at each frame in a multiframe echocardiographic image sequence. We point out that the intensity distribution of an ultrasound image at a very coarse scale can be approximately modeled by Gaussian. Then we combine region homogeneity and edge features in a level set approach to extract boundaries automatically at this coarse scale. At finer scale levels, these coarse boundaries are used to both initialize boundary detection and serve as an external constraint to guide contour evolution. This constraint functions similar to a traditional shape prior. Experimental results validate this combinative framework.     

Interaction in the segmentation of medical images: a survey

Segmentation of the object of interest is a difficult step in the analysis of digital images. Fully automatic methods sometimes fail, producing incorrect results and requiring the intervention of a human operator. This is often true in medical applications, where image segmentation is particularly difficult due to restrictions imposed by image acquisition, pathology and biological variation. In this paper we present an early review of the largely unknown territory of human-computer interaction in image segmentation. The purpose is to identify patterns in the use of interaction and to develop qualitative criteria to evaluate interactive segmentation methods. We discuss existing interactive methods with respect to the following aspects: the type of information provided by the user, how this information affects the computational part, and the purpose of interaction in the segmentation process. The discussion is based on the potential impact of each strategy on the accuracy, repeatability and interaction efficiency. Among others, these are important aspects to characterise and understand the implications of interaction to the results generated by an interactive segmentation method. This survey is focused on medical imaging, however similar patterns are expected to hold for other applications as well.     

A hybrid approach for segmentation of anatomic structures in medical images

Objective

We propose a hybrid interactive approach for the segmentation of anatomic structures in medical images with higher accuracy at lower user interaction cost.

Materials and methods

Eighteen brain MR scans from the Internet Brain Segmentation Repository are used for brain structure segmentation. A MR scan and a CT scan of an old female are used for orbital structure segmentation. The proposed approach combines shape-based interpolation, radial basis function (RBF)-based warping and model-based segmentation. With this approach, to segment a structure in a 3D image, we first delineate the structure in several slices using interactive methods, and then use shape-based interpolation to automatically generate an initial 3D model of the structure from the segmented slices. To refine the initial model, we specify a set of additional points on the structure boundary in the image, and use a RBF to warp the model so that it passes the specified points. Finally, we adopt a point-anchored active surface approach to further deform the model for a better fitting of the model with its corresponding structure in image.

Results

Two brain structures and 15 orbital structures are segmented. For each structure, it needs only to semi- automatically segment three to five 2D slices and specify two to nine additional points on the structure boundary. The time cost for each structure is about 1–3 min. The overlap ratio of the segmentation results and the ground truth is higher than 96%.

Conclusion

The proposed method for the segmentation of anatomic structure achieved higher accuracy at lower user interaction cost, and therefore promising in many applications such as surgery planning and simulation, atlas construction, and morphometric analysis of anatomic structures.     

An automatic segmentation method of a parameter-adaptive PCNN for medical images

Purpose

Since pre-processing and initial segmentation steps in medical images directly affect the final segmentation results of the regions of interesting, an automatic segmentation method of a parameter-adaptive pulse-coupled neural network is proposed to integrate the above-mentioned two segmentation steps into one. This method has a low computational complexity for different kinds of medical images and has a high segmentation precision.

Methods

The method comprises four steps. Firstly, an optimal histogram threshold is used to determine the parameter \(\alpha \) for different kinds of images. Secondly, we acquire the parameter \(\beta \) according to a simplified pulse-coupled neural network (SPCNN). Thirdly, we redefine the parameter V of the SPCNN model by sub-intensity distribution range of firing pixels. Fourthly, we add an offset \(A\times S_{\mathrm{off}}\) to improve initial segmentation precision.

Results

Compared with the state-of-the-art algorithms, the new method achieves a comparable performance by the experimental results from ultrasound images of the gallbladder and gallstones, magnetic resonance images of the left ventricle, and mammogram images of the left and the right breast, presenting the overall metric UM of 0.9845, CM of 0.8142, TM of 0.0726.

Conclusion

The algorithm has a great potential to achieve the pre-processing and initial segmentation steps in various medical images. This is a premise for assisting physicians to detect and diagnose clinical cases.

     

3D image segmentation by using statistical deformation models and level sets

Purpose Improved segmentation of soft objects was sought using a new method that combines level set segmentation with statistical deformation models, using prior knowledge of the shape of an object as well as information derived from the input image. Methods Statistical deformation models were created using Euclidian distance functions of binary data and a multi-hierarchical registration approach based on mutual information metric and demons deformable registration. This approach is motivated by the fact that models based on signed distance maps, traditionally combined with level set segmentation can result in irregular shapes and do not establish explicit correspondences. By using statistical deformation models as representation of shape and a maximum a posteriori (MAP) estimation model to estimate the MAP shape of the object to be segmented, a robust segmentation algorithm using accurate shape models could be developed. Results The accuracy and correctness of the synthesized models was evaluated on different 3D objects (cardiac MRI and spinal CT vertebral segment) and the segmentation algorithm was validated by performing different segmentation tasks using various image modalities. The results of this evaluation are very promising and show the potential utility of the approach. Conclusion Initial results demonstrate the approach is feasible and may be advantageous over alternative segmentation methods. Extensions of the model, which also incorporate prior knowledge about the spatial distribution of grey values, are currently under development.     

3D variational brain tumor segmentation using Dirichlet priors on a clustered feature set

Purpose

Brain tumor segmentation is a required step before any radiation treatment or surgery. When performed manually, segmentation is time consuming and prone to human errors. Therefore, there have been significant efforts to automate the process. But, automatic tumor segmentation from MRI data is a particularly challenging task. Tumors have a large diversity in shape and appearance with intensities overlapping the normal brain tissues. In addition, an expanding tumor can also deflect and deform nearby tissue. In our work, we propose an automatic brain tumor segmentation method that addresses these last two difficult problems.

Methods

We use the available MRI modalities (T1, T1c, T2) and their texture characteristics to construct a multidimensional feature set. Then, we extract clusters which provide a compact representation of the essential information in these features. The main idea in this work is to incorporate these clustered features into the 3D variational segmentation framework. In contrast to previous variational approaches, we propose a segmentation method that evolves the contour in a supervised fashion. The segmentation boundary is driven by the learned region statistics in the cluster space. We incorporate prior knowledge about the normal brain tissue appearance during the estimation of these region statistics. In particular, we use a Dirichlet prior that discourages the clusters from the normal brain region to be in the tumor region. This leads to a better disambiguation of the tumor from brain tissue.

Results

We evaluated the performance of our automatic segmentation method on 15 real MRI scans of brain tumor patients, with tumors that are inhomogeneous in appearance, small in size and in proximity to the major structures in the brain. Validation with the expert segmentation labels yielded encouraging results: Jaccard (58%), Precision (81%), Recall (67%), Hausdorff distance (24 mm).

Conclusions

Using priors on the brain/tumor appearance, our proposed automatic 3D variational segmentation method was able to better disambiguate the tumor from the surrounding tissue.

     

Recursive tracking of vascular tree axes in 3D medical images

Object This article describes a method for automated extraction of branching structures in three dimensional (3D) medical images. Materials and methods The algorithm recursively tracks branches and detects bifurcations by analyzing the binary connected components on the surface of a sphere that moves along the vessels. Local segmentation within the sphere is performed using a clustering algorithm based on both geometric and intensity information. It minimizes a combination of the intra-class intensity variances and of the inertia moment of the “vessel” class, which emphasizes the cylindrical structures. The algorithm was applied to 16 MRA and 12 CTA 3D images of different anatomic regions. Its capability of extracting all the branches and avoiding spurious detections was evaluated by comparing the number of extracted branches with the number of branches found by visual inspection of the datasets. Its reproducibility and sensitivity to parameter variation were also assessed. Results With a fixed parameter setting, 68 out of 286 perceptible branches were missed or partly extracted and 11 spurious branches were obtained. Increasing the weight of the geometric criterion helped in tracking the principal branches in noisy data but increased the number of missed branches. Processing time was within 5 min per dataset. Conclusion From one initial point, the algorithm extracts a vascular tree where the differences of size and of intensity between the branches are not large. Missed sub-trees can be recovered using additional starting points.     

TernaryNet: faster deep model inference without GPUs for medical 3D segmentation using sparse and binary convolutions

Purpose

Deep convolutional neural networks (DCNN) are currently ubiquitous in medical imaging. While their versatility and high-quality results for common image analysis tasks including segmentation, localisation and prediction is astonishing, the large representational power comes at the cost of highly demanding computational effort. This limits their practical applications for image-guided interventions and diagnostic (point-of-care) support using mobile devices without graphics processing units (GPU).

Methods

We propose a new scheme that approximates both trainable weights and neural activations in deep networks by ternary values and tackles the open question of backpropagation when dealing with non-differentiable functions. Our solution enables the removal of the expensive floating-point matrix multiplications throughout any convolutional neural network and replaces them by energy- and time-preserving binary operators and population counts.

Results

We evaluate our approach for the segmentation of the pancreas in CT. Here, our ternary approximation within a fully convolutional network leads to more than 90% memory reductions and high accuracy (without any post-processing) with a Dice overlap of 71.0% that comes close to the one obtained when using networks with high-precision weights and activations. We further provide a concept for sub-second inference without GPUs and demonstrate significant improvements in comparison with binary quantisation and without our proposed ternary hyperbolic tangent continuation.

Conclusions

We present a key enabling technique for highly efficient DCNN inference without GPUs that will help to bring the advances of deep learning to practical clinical applications. It has also great promise for improving accuracies in large-scale medical data retrieval.