Deformable biomechanical models: application to 4D cardiac image analysis

This article describes a methodology for creating a generic volumetric biomechanical model from different image modalities and segmenting time series of medical images using this model. The construction of such a generic model consists of three stages: geometric meshing, non-rigid deformation of the mesh in images of various modalities, and image-to-mesh information mapping through rasterization. The non-rigid deformation stage, which relies on a combination of global and local deformations, can then be used to segment time series of images, e.g. cine MRI or gated SPECT cardiac images. We believe that this type of deformable biomechanical model can play an important role in the extraction of useful quantitative local parameters of cardiac function. The biomechanical model of the heart will be coupled with an electrical model of another collaborative project in order to simulate and analyze a larger class of pathologies.     

Volume measurements in nuclear medicine gated SPECT and 4D echocardiography: validation using a dynamic cardiac phantom

Purpose: A dynamic cardiac phantom was used as a reference to compare volumes measured with gated SPECT and 4D echocardiography. Material and Methods: Gated SPECT data were acquired with a standard single-head gamma camera, and the volume reconstructions were carried out using the Mirage software by Segami. 4D echocardiography used a new prototype of rotating scan head to acquire ultrasound images during a cardiac cycle, used to reconstruct the volume deformations as a function of time. End-diastolic volume, end-systolic volume, and ejection fraction were measured using both gated SPECT and 4D echocardiography. Results: The results obtained showed a good correlation between volumes measured with the two modalities, but a slight overestimation of volumes with gated SPECT. The influence of filtering and pixel size parameters on the measured volumes was quantified for gated SPECT, in order to correct the overestimation. Conclusion: The agreement between gated SPECT (after correction) and 4D echocardiography confirmed the relevance of the comparisons. This study was an initial step before conducting clinical trials to compare exhaustively left ventricular volumes obtained with the two modalities.     

Automatic 3D segmentation of individual facial muscles using unlabeled prior information

Purpose  

Segmentation of facial soft tissues is required for surgical planning and evaluation, but this is laborious using manual methods and has been difficult to achieve with digital segmentation methods. A new automatic 3D segmentation method for facial soft tissues in magnetic resonance imaging (MRI) images was designed, implemented, and tested.     

Segmentation of the skull in MRI volumes using deformable model and taking the partial volume effect into account

Segmentation of the skull in medical imagery is an important stage in applications that require the construction of realistic models of the head. Such models are used, for example, to simulate the behavior of electro-magnetic fields in the head and to model the electrical activity of the cortex in EEG and MEG data. In this paper, we present a new approach for segmenting regions of bone in MRI volumes using deformable models. Our method takes into account the partial volume effects that occur with MRI data, thus permitting a precise segmentation of these bone regions. At each iteration of the propagation of the model, partial volume is estimated in a narrow band around the deformable model. Our segmentation method begins with a pre-segmentation stage, in which a preliminary segmentation of the skull is constructed using a region-growing method. The surface that bounds the pre-segmented skull region offers an automatic 3D initialization of the deformable model. This surface is then propagated (in 3D) in the direction of its normal. This propagation is achieved using level set method, thus permitting changes to occur in the topology of the surface as it evolves, an essential capability for our problem. The speed at which the surface evolves is a function of the estimated partial volume. This provides a sub-voxel accuracy in the resulting segmentation.     

A hybrid framework for 3D medical image segmentation

In this paper we propose a novel hybrid 3D segmentation framework which combines Gibbs models, marching cubes and deformable models. In the framework, first we construct a new Gibbs model whose energy function is defined on a high order clique system. The new model includes both region and boundary information during segmentation. Next we improve the original marching cubes method to construct 3D meshes from Gibbs models' output. The 3D mesh serves as the initial geometry of the deformable model. Then we deform the deformable model using external image forces so that the model converges to the object surface. We run the Gibbs model and the deformable model recursively by updating the Gibbs model's parameters using the region and boundary information in the deformable model segmentation result. In our approach, the hybrid combination of region-based methods and boundary-based methods results in improved segmentations of complex structures. The benefit of the methodology is that it produces high quality segmentations of 3D structures using little prior information and minimal user intervention. The modules in this segmentation methodology are developed within the context of the Insight ToolKit (ITK). We present experimental segmentation results of brain tumors and evaluate our method by comparing experimental results with expert manual segmentations. The evaluation results show that the methodology achieves high quality segmentation results with computational efficiency. We also present segmentation results of other clinical objects to illustrate the strength of the methodology as a generic segmentation framework.     

A fast deformable registration method for 4D lung CT in hybrid framework

Purpose

A pulmonary respiration model for deformable registration of lung CT for the surgery path planning and surgical navigation is an important, difficult, and time-consuming task. This paper presents a new fast deformable registration method for 4D lung CT in a hybrid framework incorporating point set registration with mutual information registration.

Method

The point sets of the lung surface and vessels are automatically extracted. Their displacement vectors are obtained by point set registration. The sum of squared Euclidean distance between the displacement vectors of these point sets and the displacement vectors based on the B-spline transformation model is minimized as a novel similarity measure to derive the rough transformation function. Finally, the rough transformation function is refined by using the mutual information-based registration method. To evaluate the effectiveness of the proposed method, the authors performed registrations on 20 4D lung volume cases from two different CT scanners. The proposed method was compared with the point set-based method, the mutual information-based method, and the ANTS method, which is a state-of-the-art deformable registration technique.

Results

The results show that the landmark distance errors and computation time of the proposed method decreased an average of 5 and 70 %, respectively, when compared to the mutual information-alone-based method. The proposed method results in an average of 28 % lower landmark distance error than registration method based on point sets in spite of increase in computation time. Moreover, compared with ANTS, the computation time of the proposed method is reduced by an average of 93 % in the case of comparable landmark distance errors.

Conclusion

The accuracy and speed of the proposed deformable registration method indicate that the method is suitable for use in a clinical image-guided intervention system.     

Simulation of spatiotemporal CT data sets using a 4D MRI-based lung motion model

Purpose

Four-dimensional CT imaging is widely used to account for motion-related effects during radiotherapy planning of lung cancer patients. However, 4D CT often contains motion artifacts, cannot be used to measure motion variability, and leads to higher dose exposure. In this article, we propose using 4D MRI to acquire motion information for the radiotherapy planning process. From the 4D MRI images, we derive a time-continuous model of the average patient-specific respiratory motion, which is then applied to simulate 4D CT data based on a static 3D CT.

Methods

The idea of the motion model is to represent the average lung motion over a respiratory cycle by cyclic B-spline curves. The model generation consists of motion field estimation in the 4D MRI data by nonlinear registration, assigning respiratory phases to the motion fields, and applying a B-spline approximation on a voxel-by-voxel basis to describe the average voxel motion over a breathing cycle. To simulate a patient-specific 4D CT based on a static CT of the patient, a multi-modal registration strategy is introduced to transfer the motion model from MRI to the static CT coordinates.

Results

Differences between model-based estimated and measured motion vectors are on average 1.39 mm for amplitude-based binning of the 4D MRI data of three patients. In addition, the MRI-to-CT registration strategy is shown to be suitable for the model transformation.

Conclusions

The application of our 4D MRI-based motion model for simulating 4D CT images provides advantages over standard 4D CT (less motion artifacts, radiation-free). This makes it interesting for radiotherapy planning.     

Fully automatic reconstruction of personalized 3D volumes of the proximal femur from 2D X-ray images

Purpose

Accurate preoperative planning is crucial for the outcome of total hip arthroplasty. Recently, 2D pelvic X-ray radiographs have been replaced by 3D CT. However, CT suffers from relatively high radiation dosage and cost. An alternative is to reconstruct a 3D patient-specific volume data from 2D X-ray images.

Methods

In this paper, based on a fully automatic image segmentation algorithm, we propose a new control point-based 2D–3D registration approach for a deformable registration of a 3D volumetric template to a limited number of 2D calibrated X-ray images and show its application to personalized reconstruction of 3D volumes of the proximal femur. The 2D–3D registration is done with a hierarchical two-stage strategy: the scaled-rigid 2D–3D registration stage followed by a regularized deformable B-spline 2D–3D registration stage. In both stages, a set of control points with uniform spacing are placed over the domain of the 3D volumetric template first. The registration is then driven by computing updated positions of these control points with intensity-based 2D–2D image registrations of the input X-ray images with the associated digitally reconstructed radiographs, which allows computing the associated registration transformation at each stage.

Results

Evaluated on datasets of 44 patients, our method achieved an overall surface reconstruction accuracy of \(0.9 \pm 0.2\,\hbox {mm}\) and an average Dice coefficient of \(94.4 \pm 1.1\,\%\). We further investigated the cortical bone region reconstruction accuracy, which is important for planning cementless total hip arthroplasty. An average cortical bone region Dice coefficient of \(85.1 \pm 2.9\,\%\) and an inner cortical bone surface reconstruction accuracy of \(0.7 \pm 0.2\,\hbox {mm}\) were found.

Conclusions

In summary, we developed a new approach for reconstruction of 3D personalized volumes of the proximal femur from 2D X-ray images. Comprehensive experiments demonstrated the efficacy of the present approach.

     

Statistical representation of high-dimensional deformation fields with application to statistically constrained 3D warping

This paper proposes a 3D statistical model aiming at effectively capturing statistics of high-dimensional deformation fields and then uses this prior knowledge to constrain 3D image warping. The conventional statistical shape model methods, such as the active shape model (ASM), have been very successful in modeling shape variability. However, their accuracy and effectiveness typically drop dramatically in high-dimensionality problems involving relatively small training datasets, which is customary in 3D and 4D medical imaging applications. The proposed statistical model of deformation (SMD) uses wavelet-based decompositions coupled with PCA in each wavelet band, in order to more accurately estimate the pdf of high-dimensional deformation fields, when a relatively small number of training samples are available. SMD is further used as statistical prior to regularize the deformation field in an SMD-constrained deformable registration framework. As a result, more robust registration results are obtained relative to using generic smoothness constraints on deformation fields, such as Laplacian-based regularization. In experiments, we first illustrate the performance of SMD in representing the variability of deformation fields and then evaluate the performance of the SMD-constrained registration, via comparing a hierarchical volumetric image registration algorithm, HAMMER, with its SMD-constrained version, referred to as SMD+HAMMER. This SMD-constrained deformable registration framework can potentially incorporate various registration algorithms to improve robustness and stability via statistical shape constraints.     

Discriminant snakes for 3D reconstruction of anatomical organs

In this work a new statistic deformable model for 3D segmentation of anatomical organs in medical images is proposed. A statistic discriminant snake performs a supervised learning of the object boundary in an image slice to segment the next slice of the image sequence. Each part of the object boundary is projected in a feature space generated by a bank of Gaussian filters. Then, clusters corresponding to different boundary pieces are constructed by means of linear discriminant analysis. Finally, a parametric classifier is generated from each contour in the image slice and embodied into the snake energy-minimization process to guide the snake deformation in the next image slice. The discriminant snake selects and classifies image features by the parametric classifier and deforms to minimize the dissimilarity between the learned and found image features. The new approach is of particular interest for segmenting 3D images with anisotropic spatial resolution, and for tracking temporal image sequences. In particular, several anatomical organs from different imaging modalities are segmented and the results compared to expert tracings.     

3D prostate model formation from non-parallel 2D ultrasound biopsy images

Biopsy of the prostate using 2D transrectal ultrasound (TRUS) guidance is the current gold standard for diagnosis of prostate cancer; however, the current procedure is limited by using 2D biopsy tools to target 3D biopsy locations. We propose a technique for patient-specific 3D prostate model reconstruction from a sparse collection of non-parallel 2D TRUS biopsy images. Our method conforms to the restrictions of current TRUS biopsy equipment and could be efficiently incorporated into current clinical biopsy procedures for needle guidance without the need for expensive hardware additions. In this paper, the model reconstruction technique is evaluated using simulated biopsy images from 3D TRUS prostate images of 10 biopsy patients. All reconstructed models are compared to their corresponding 3D manually segmented prostate models for evaluation of prostate volume accuracy and surface errors (both regional and global). The number of 2D TRUS biopsy images used for prostate modeling was varied to determine the optimal number of images necessary for accurate prostate surface estimation.     

Statistical deformable bone models for robust 3D surface extrapolation from sparse data

A majority of pre-operative planning and navigational guidance during computer assisted orthopaedic surgery routinely uses three-dimensional models of patient anatomy. These models enhance the surgeon's capability to decrease the invasiveness of surgical procedures and increase their accuracy and safety. A common approach for this is to use computed tomography (CT) or magnetic resonance imaging (MRI). These have the disadvantages that they are expensive and/or induce radiation to the patient. In this paper we propose a novel method to construct a patient-specific three-dimensional model that provides an appropriate intra-operative visualization without the need for a pre or intra-operative imaging. The 3D model is reconstructed by fitting a statistical deformable model to minimal sparse 3D data consisting of digitized landmarks and surface points that are obtained intra-operatively. The statistical model is constructed using Principal Component Analysis from training objects. Our deformation scheme efficiently and accurately computes a Mahalanobis distance weighted least square fit of the deformable model to the 3D data. Relaxing the Mahalanobis distance term as additional points are incorporated enables our method to handle small and large sets of digitized points efficiently. Formalizing the problem as a linear equation system helps us to provide real-time updates to the surgeons. Incorporation of M-estimator based weighting of the digitized points enables us to effectively reject outliers and compute stable models. We present here our evaluation results using leave-one-out experiments and extended validation of our method on nine dry cadaver bones.     

A linear programming approach to reconstructing subcellular structures from confocal images for automated generation of representative 3D cellular models

This paper presents a novel computer vision algorithm to analyze 3D stacks of confocal images of fluorescently stained single cells. The goal of the algorithm is to create representative in silico model structures that can be imported into finite element analysis software for mechanical characterization. Segmentation of cell and nucleus boundaries is accomplished via standard thresholding methods. Using novel linear programming methods, a representative actin stress fiber network is generated by computing a linear superposition of fibers having minimum discrepancy compared with an experimental 3D confocal image. Qualitative validation is performed through analysis of seven 3D confocal image stacks of adherent vascular smooth muscle cells (VSMCs) grown in 2D culture. The presented method is able to automatically generate 3D geometries of the cell’s boundary, nucleus, and representative F-actin network based on standard cell microscopy data. These geometries can be used for direct importation and implementation in structural finite element models for analysis of the mechanics of a single cell to potentially speed discoveries in the fields of regenerative medicine, mechanobiology, and drug discovery.     

Learning to map 2D ultrasound images into 3D space with minimal human annotation

In fetal neurosonography, aligning two-dimensional (2D) ultrasound scans to their corresponding plane in the three-dimensional (3D) space remains a challenging task. In this paper, we propose a convolutional neural network that predicts the position of 2D ultrasound fetal brain scans in 3D atlas space. Instead of purely supervised learning that requires heavy annotations for each 2D scan, we train the model by sampling 2D slices from 3D fetal brain volumes, and target the model to predict the inverse of the sampling process, resembling the idea of self-supervised learning.We propose a model that takes a set of images as input, and learns to compare them in pairs. The pairwise comparison is weighted by the attention module based on its contribution to the prediction, which is learnt implicitly during training. The feature representation for each image is thus computed by incorporating the relative position information to all the other images in the set, and is later used for the final prediction.We benchmark our model on 2D slices sampled from 3D fetal brain volumes at 18–22 weeks' gestational age. Using three evaluation metrics, namely, Euclidean distance, plane angles and normalized cross correlation, which account for both the geometric and appearance discrepancy between the ground-truth and prediction, in all these metrics, our model outperforms a baseline model by as much as 23%, when the number of input images increases. We further demonstrate that our model generalizes to (i) real 2D standard transthalamic plane images, achieving comparable performance as human annotations, as well as (ii) video sequences of 2D freehand fetal brain scans.     

CRUISE: cortical reconstruction using implicit surface evolution

Segmentation and representation of the human cerebral cortex from magnetic resonance (MR) images play an important role in neuroscience and medicine. A successful segmentation method must be robust to various imaging artifacts and produce anatomically meaningful and consistent cortical representations. A method for the automatic reconstruction of the inner, central, and outer surfaces of the cerebral cortex from T1-weighted MR brain images is presented. The method combines a fuzzy tissue classification method, an efficient topology correction algorithm, and a topology-preserving geometric deformable surface model (TGDM). The algorithm is fast and numerically stable, and yields accurate brain surface reconstructions that are guaranteed to be topologically correct and free from self-intersections. Validation results on real MR data are presented to demonstrate the performance of the method.     

Segmentation of 4D cardiac MR images using a probabilistic atlas and the EM algorithm

In this paper an automatic atlas-based segmentation algorithm for 4D cardiac MR images is proposed. The algorithm is based on the 4D extension of the expectation maximisation (EM) algorithm. The EM algorithm uses a 4D probabilistic cardiac atlas to estimate the initial model parameters and to integrate a priori information into the classification process. The probabilistic cardiac atlas has been constructed from the manual segmentations of 3D cardiac image sequences of 14 healthy volunteers. It provides space and time-varying probability maps for the left and right ventricles, the myocardium, and background structures such as the liver, stomach, lungs and skin. In addition to using the probabilistic cardiac atlas as a priori information, the segmentation algorithm incorporates spatial and temporal contextual information by using 4D Markov Random Fields. After the classification, the largest connected component of each structure is extracted using a global connectivity filter which improves the results significantly, especially for the myocardium. Validation against manual segmentations and computation of the correlation between manual and automatic segmentation on 249 3D volumes were calculated. We used the 'leave one out' test where the image set to be segmented was not used in the construction of its corresponding atlas. Results show that the procedure can successfully segment the left ventricle (LV) (r = 0.96), myocardium (r = 0.92) and right ventricle (r = 0.92). In addition, 4D images from 10 patients with hypertrophic cardiomyopathy were also manually and automatically segmented yielding a good correlation in the volumes of the LV (r = 0.93) and myocardium (0.94) when the atlas constructed with volunteers is blurred.     

Mutual information-based multimodality registration of cardiac ultrasound and SPECT images: a preliminary investigation

BACKGROUND: Ultrasound (US) and single photon emission computed tomography (SPECT) are the two most commonly prescribed procedures for diagnosing coronary artery disease (CAD). We have demonstrated the feasibility of multimodality registration of two-dimensional (2D) and three-dimensional (3D) cardiac US images with cardiac SPECT images with an aim to simultaneously present the complementary anatomical and perfusion information from the two modalities. We have also tested the clinicians' assessment of the clinical adequacy of the registered images. METHODS AND RESULTS: We have demonstrated temporal and spatial registration for nine sets of cardiac US and SPECT cine loops covering the entire cardiac cycle. Temporal alignment was performed by interpolation of existing SPECT images at cardiac phases corresponding to available US images. Spatial registration was performed in 3D image space using a mutual information-based approach. Experts from echocardiography and nuclear medicine determined the clinical utility of the registration by rating each registration on a scale of 1 to 5, a rating of 3 or above indicating clinical utility. 2DUS-SPECT registration (five cases) received an average rating of 4.2, whereas 3DUS-SPECT registration (four cases) received an average rating of 2.85. By one-sample t test, the overall evaluations (mean 3.58) were greater than the pre-specified clinical cut-off of 3 with p < 0.05, indicating likelihood of clinical utility. CONCLUSION: Our method demonstrates the feasibility of registering cardiac US and SPECT images in their present as well as possible future forms. Such registration has the potential to provide a more accurate and powerful tool for diagnosing CAD.     

A procedure to average 3D anatomical structures

Creating a feature-preserving average of three dimensional anatomical surfaces extracted from volume image data is a complex task. Unlike individual images, averages present right-left symmetry and smooth surfaces which give insight into typical proportions. Averaging multiple biological surface images requires careful superimposition and sampling of homologous regions. Our approach to biological surface image averaging grows out of a wireframe surface tessellation approach by Cutting et al. (1993). The surface delineating wires represent high curvature crestlines. By adding tile boundaries in flatter areas the 3D image surface is parametrized into anatomically labeled (homology mapped) grids. We extend the Cutting et al. wireframe approach by encoding the entire surface as a series of B-spline space curves. The crestline averaging algorithm developed by Cutting et al. may then be used for the entire surface. Shape preserving averaging of multiple surfaces requires careful positioning of homologous surface regions such as these B-spline space curves. We test the precision of this new procedure and its ability to appropriately position groups of surfaces in order to produce a shape-preserving average. Our result provides an average that well represents the source images and may be useful clinically as a deformable model or for animation.