Fully automatic segmentation of the Femur from 3D-CT images using primitive shape recognition and statistical shape models

Purpose

Femur segmentation is well established and widely used in computer-assisted orthopedic surgery. However, most of the robust segmentation methods such as statistical shape models (SSM) require human intervention to provide an initial position for the SSM. In this paper, we propose to overcome this problem and provide a fully automatic femur segmentation method for CT images based on primitive shape recognition and SSM.

Method

Femur segmentation in CT scans was performed using primitive shape recognition based on a robust algorithm such as the Hough transform and RANdom SAmple Consensus. The proposed method is divided into 3 steps: (1) detection of the femoral head as sphere and the femoral shaft as cylinder in the SSM and the CT images, (2) rigid registration between primitives of SSM and CT image to initialize the SSM into the CT image, and (3) fitting of the SSM to the CT image edge using an affine transformation followed by a nonlinear fitting.

Results

The automated method provided good results even with a high number of outliers. The difference of segmentation error between the proposed automatic initialization method and a manual initialization method is less than 1 mm.

Conclusion

The proposed method detects primitive shape position to initialize the SSM into the target image. Based on primitive shapes, this method overcomes the problem of inter-patient variability. Moreover, the results demonstrate that our method of primitive shape recognition can be used for 3D SSM initialization to achieve fully automatic segmentation of the femur.     

Robust statistical shape models for MRI bone segmentation in presence of small field of view

Accurate bone modeling from medical images is essential in the diagnosis and treatment of patients because it supports the detection of abnormal bone morphology, which is often responsible for many musculoskeletal diseases (MSDs) of human articulations. In a clinical setting, images of the suspected joints are acquired in a high resolution but with a small field of view (FOV) in order to maximize the image quality while reducing acquisition time. However bones are only partially visible in such small FOVs. This presents difficult challenges in automated bone segmentation and thus limits the application of sophisticated algorithms such as statistical shape models (SSM), which have been generally proven to be an efficient technique for bone segmentation. Indeed, the reduced image information affects the initialization and evolution of these deformable model-based approaches. In this paper, we present a robust multi-resolution SSM algorithm with an adapted initialization to address the segmentation of MRI bone images acquired in small FOVs for modeling and computer-aided diagnosis. Our innovation stems from the derivation of a robust SSM based on complete and corrupted shapes, as well as from a simultaneous optimization of transformation and shape parameters to yield an efficient initialization technique. We demonstrate our segmentation algorithm using 86 clinical MRI images of the femur and hip bones. These images have a varied resolution and limited FOVs. The results of our segmentation (e.g., average distance error of 1.12 ± 0.46 mm) are within the needs of image-based clinical diagnosis.     

Multi-resolution multi-object statistical shape models based on the locality assumption

Statistical shape models learned from a population of previously observed training shapes are nowadays widely used in medical image analysis to aid segmentation or classification. However, providing an appropriate and representative training population of preferably manual segmentations is typically either very labor-intensive or even impossible. Therefore, statistical shape models in practice frequently suffer from the high-dimension-low-sample-size (HDLSS) problem resulting in models with insufficient expressiveness.In this paper, a novel approach for learning representative multi-resolution multi-object statistical shape models from a small number of training samples that adequately model the variability of each individual object as well as their interrelations is presented. The method is based on the assumption of locality, which means that local shape variations have limited effects in distant areas and, therefore, can be modeled independently. This locality assumption is integrated into the standard statistical shape modeling framework by manipulating the sample covariance matrix (non-zero covariances between distant landmarks are set to zero). To allow for multi-object modeling, a method for computing distances between points located on different object shapes is proposed. Furthermore, different levels of locality are introduced by deriving a multi-resolution scheme, which is equipped with a method to combine variability information modeled at different levels into a single shape model. This combined representation of global and local variability in a single shape model allows the use of the classical active shape model strategy for model-based image segmentation.An extensive evaluation based on a public data base of 247 chest radiographs is performed to show the modeling and segmentation capabilities of the proposed approach in single- and multi-object HDLSS scenarios. The new approach is not only compared to the classical shape modeling method but also to three state-of-the-art shape modeling approaches specifically designed to cope with the HDLSS problem. The results show that the new approach significantly outperforms all other approaches in terms of generalization ability and model-based segmentation accuracy.     

Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging

Magnetic resonance (MR) imaging is often used to characterize and quantify multiple sclerosis (MS) lesions in the brain and spinal cord. The number and volume of lesions have been used to evaluate MS disease burden, to track the progression of the disease and to evaluate the effect of new pharmaceuticals in clinical trials. Accurate identification of MS lesions in MR images is extremely difficult due to variability in lesion location, size and shape in addition to anatomical variability between subjects. Since manual segmentation requires expert knowledge, is time consuming and is subject to intra- and inter-expert variability, many methods have been proposed to automatically segment lesions.The objective of this study was to carry out a systematic review of the literature to evaluate the state of the art in automated multiple sclerosis lesion segmentation. From 1240 hits found initially with PubMed and Google scholar, our selection criteria identified 80 papers that described an automatic lesion segmentation procedure applied to MS. Only 47 of these included quantitative validation with at least one realistic image. In this paper, we describe the complexity of lesion segmentation, classify the automatic MS lesion segmentation methods found, and review the validation methods applied in each of the papers reviewed. Although many segmentation solutions have been proposed, including some with promising results using MRI data obtained on small groups of patients, no single method is widely employed due to performance issues related to the high variability of MS lesion appearance and differences in image acquisition. The challenge remains to provide segmentation techniques that work in all cases regardless of the type of MS, duration of the disease, or MRI protocol, and this within a comprehensive, standardized validation framework. MS lesion segmentation remains an open problem.     

2D-3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models

Three-dimensional patient specific bone models are required in a range of medical applications, such as pre-operative surgery planning and improved guidance during surgery, modeling and simulation, and in vivo bone motion tracking. Shape reconstruction from a small number of X-ray images is desired as it lowers both the acquisition costs and the radiation dose compared to CT. We propose a method for pose estimation and shape reconstruction of 3D bone surfaces from two (or more) calibrated X-ray images using a statistical shape model (SSM). User interaction is limited to manual initialization of the mean shape. The proposed method combines a 3D distance based objective function with automatic edge selection on a Canny edge map. Landmark-edge correspondences are weighted based on the orientation difference of the projected silhouette and the corresponding image edge. The method was evaluated by rigid pose estimation of ground truth shapes as well as 3D shape estimation using a SSM of the whole femur, from stereo cadaver X-rays, in vivo biplane fluoroscopy image-pairs, and an in vivo biplane fluoroscopic sequence. Ground truth shapes for all experiments were available in the form of CT segmentations. Rigid registration of the ground truth shape to the biplane fluoroscopy achieved sub-millimeter accuracy (0.68 mm) measured as root mean squared (RMS) point-to-surface (P2S) distance. The non-rigid reconstruction from the biplane fluoroscopy using the SSM also showed promising results (1.68 mm RMS P2S). A feasibility study on one fluoroscopic time series illustrates the potential of the method for motion and shape estimation from fluoroscopic sequences with minimal user interaction.     

Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative

We present a method for the automated segmentation of knee bones and cartilage from magnetic resonance imaging (MRI) that combines a priori knowledge of anatomical shape with Convolutional Neural Networks (CNNs). The proposed approach incorporates 3D Statistical Shape Models (SSMs) as well as 2D and 3D CNNs to achieve a robust and accurate segmentation of even highly pathological knee structures. The shape models and neural networks employed are trained using data from the Osteoarthritis Initiative (OAI) and the MICCAI grand challenge “Segmentation of Knee Images 2010” (SKI10), respectively. We evaluate our method on 40 validation and 50 submission datasets from the SKI10 challenge. For the first time, an accuracy equivalent to the inter-observer variability of human readers is achieved in this challenge. Moreover, the quality of the proposed method is thoroughly assessed using various measures for data from the OAI, i.e. 507 manual segmentations of bone and cartilage, and 88 additional manual segmentations of cartilage. Our method yields sub-voxel accuracy for both OAI datasets. We make the 507 manual segmentations as well as our experimental setup publicly available to further aid research in the field of medical image segmentation. In conclusion, combining localized classification via CNNs with statistical anatomical knowledge via SSMs results in a state-of-the-art segmentation method for knee bones and cartilage from MRI data.     

A framework for the merging of pre-existing and correspondenceless 3D statistical shape models

The construction of statistical shape models (SSMs) that are rich, i.e., that represent well the natural and complex variability of anatomical structures, is an important research topic in medical imaging. To this end, existing works have addressed the limited availability of training data by decomposing the shape variability hierarchically or by combining statistical and synthetic models built using artificially created modes of variation. In this paper, we present instead a method that merges multiple statistical models of 3D shapes into a single integrated model, thus effectively encoding extra variability that is anatomically meaningful, without the need for the original or new real datasets. The proposed framework has great flexibility due to its ability to merge multiple statistical models with unknown point correspondences. The approach is beneficial in order to re-use and complement pre-existing SSMs when the original raw data cannot be exchanged due to ethical, legal, or practical reasons. To this end, this paper describes two main stages, i.e., (1) statistical model normalization and (2) statistical model integration. The normalization algorithm uses surface-based registration to bring the input models into a common shape parameterization with point correspondence established across eigenspaces. This allows the model fusion algorithm to be applied in a coherent manner across models, with the aim to obtain a single unified statistical model of shape with improved generalization ability. The framework is validated with statistical models of the left and right cardiac ventricles, the L1 vertebra, and the caudate nucleus, constructed at distinct research centers based on different imaging modalities (CT and MRI) and point correspondences. The results demonstrate that the model integration is statistically and anatomically meaningful, with potential value for merging pre-existing multi-modality statistical models of 3D shapes.     

3D quantification of mandibular asymmetry using the SPHARM-PDM tool box

Purpose  

Pretreatment diagnosis of mandibular asymmetry in orthognathic surgery patients can be improved by quantitative shape modeling and analysis. The UNC SPHARM-PDM (University of North Carolina Spherical Harmonics—Point Distribution Model) toolbox was applied to a cohort of patients and the results were evaluated.     

Joint optimization of segmentation and shape prior from level-set-based statistical shape model,and its application to the automated segmentation of abdominal organs

The goal of this study is to provide a theoretical framework for accurately optimizing the segmentation energy considering all of the possible shapes generated from the level-set-based statistical shape model (SSM). The proposed algorithm solves the well-known open problem, in which a shape prior may not be optimal in terms of an objective functional that needs to be minimized during segmentation. The algorithm allows the selection of an optimal shape prior from among all possible shapes generated from an SSM by conducting a branch-and-bound search over an eigenshape space. The proposed algorithm does not require predefined shape templates or the construction of a hierarchical clustering tree before graph-cut segmentation. It jointly optimizes an objective functional in terms of both the shape prior and segmentation labeling, and finds an optimal solution by considering all possible shapes generated from an SSM. We apply the proposed algorithm to both pancreas and spleen segmentation using multiphase computed tomography volumes, and we compare the results obtained with those produced by a conventional algorithm employing a branch-and-bound search over a search tree of predefined shapes, which were sampled discretely from an SSM. The proposed algorithm significantly improves the segmentation performance in terms of the Jaccard index and Dice similarity index. In addition, we compare the results with the state-of-the-art multiple abdominal organs segmentation algorithm, and confirmed that the performances of both algorithms are comparable to each other. We discuss the high computational efficiency of the proposed algorithm, which was determined experimentally using a normalized number of traversed nodes in a search tree, and the extensibility of the proposed algorithm to other SSMs or energy functionals.     

Statistical model based shape prediction from a combination of direct observations and various surrogates: Application to orthopaedic research

In computer-assisted orthopaedic surgery, recovering three-dimensional patient-specific anatomy from incomplete information has been focus of interest due to several factors such as less invasive surgical procedures, reduced radiation doses, and rapid intra-operative updates of the anatomy. The aim of this paper is to report results obtained combining statistical shape modeling and multivariate regression techniques for predicting bone shape from clinically and surgically relevant predictors, including sparse observations of the bone surface but also morphometric and anthropometric information. Different state of the art methods such as partial least square regression, principal component regression, canonical correlation analysis, and non-parametric kernel-based regression are compared. Clinically relevant surrogate variables and combinations are investigated on a database of 142 femur and 154 tibia shapes obtained from CT images. The results are evaluated using cross validation to quantify the prediction error. The proposed approach enables to characterize the added value of different predictors in a quantitative and localized fashion. Results indicate that complementary sources of information can be efficiently exploited to improve the accuracy of shape prediction.     

Anatomical Regurgitant Orifice Detection and Quantification from 3-D Echocardiographic Images

The vena contracta and effective regurgitant orifice area (EROA) are currently used for the clinical assessment of mitral regurgitation (MR) from 2-D color Doppler imaging. In addition to being highly user dependent and having low repeatability, these methods do not represent accurately the anatomic regurgitant orifice (ARO), which affects the adequate assessment of MR patients. We propose a novel method for semi-automatic detection and quantitative assessment of the 3-D ARO shape from 3-D transesophageal echocardiographic images. The algorithm was tested on a set of 25 patients with MR, and compared with EROA for validation. Results indicate the robustness of the proposed approach, with low variability in relation to different settings of user-defined segmentation parameters. Although EROA and ARO exhibited a good correlation (r = 0.8), relatively large biases were measured, indicating that EROA probably underestimates the real shape and size of the regurgitant orifice. Along with the higher reproducibility of the proposed approach, this highlights the limitations of current clinical approaches and underlines the importance of accurate assessment of the ARO shape for diagnosis and treatment in MR patients.     

Capturing large shape variations of liver using population-based statistical shape models

Purpose

Statistical shape models (SSMs) represent morphological variations of a specific object. When there are large shape variations, the shape parameters constitute a large space that may include incorrect parameters. The human liver is a non-rigid organ subject to large deformations due to external forces or body position changes during scanning procedures. We developed and tested a population-based model to represent the shape of liver.

Methods

Upper abdominal CT-scan input images are represented by a conventional shape model. The shape parameters of individual livers extracted from the CT scans are employed to classify them into different populations. Corresponding to each population, an SSM model is built. The liver surface parameter space is divided into several subspaces which are more compact than the original space. The proposed model was tested using 29 CT-scan liver image data sets. The method was evaluated by model compactness, reconstruction error, generality and specificity measures.

Results

The proposed model is implemented and tested using CT scans that included liver shapes with large shape variations. The method was compared with conventional and recently developed shape modeling methods. The accuracy of the proposed model was nearly twice that achieved with the conventional model. The proposed population-based model was more general compared with the conventional model. The mean reconstruction error of the proposed model was 0.029 mm while that of the conventional model was 0.052 mm.

Conclusion

A population-based model to represent the shape of liver was developed and tested with favorable results. Using this approach, the liver shapes from CT scans were modeled by a more compact, more general, and more accurate model.     

Predicting infant cortical surface development using a 4D varifold-based learning framework and local topography-based shape morphing

Longitudinal neuroimaging analysis methods have remarkably advanced our understanding of early postnatal brain development. However, learning predictive models to trace forth the evolution trajectories of both normal and abnormal cortical shapes remains broadly absent. To fill this critical gap, we pioneered the first prediction model for longitudinal developing cortical surfaces in infants using a spatiotemporal current-based learning framework solely from the baseline cortical surface. In this paper, we detail this prediction model and even further improve its performance by introducing two key variants. First, we use the varifold metric to overcome the limitations of the current metric for surface registration that was used in our preliminary study. We also extend the conventional varifold-based surface registration model for pairwise registration to a spatiotemporal surface regression model. Second, we propose a morphing process of the baseline surface using its topographic attributes such as normal direction and principal curvature sign. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we use the proposed varifold-based shape regression model to estimate geodesic cortical shape evolution trajectories for each training subject. We then build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we select the best learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints, based on similarity metrics between this baseline surface and the learnt baseline population average surface atlas. We used a leave-one-out cross validation method to predict the inner cortical surface shape at 3, 6, 9 and 12 months of age from the baseline cortical surface shape at birth. Our method attained a higher prediction accuracy and better captured the spatiotemporal dynamic change of the highly folded cortical surface than the previous proposed prediction method.     

Optimisation of orthopaedic implant design using statistical shape space analysis based on level sets

Statistical shape analysis techniques have shown to be efficient tools to build population specific models of anatomical variability. Their use is commonplace as prior models for segmentation, in which case the instance from the shape model that best fits the image data is sought. In certain cases, however, it is not just the most likely instance that must be searched, but rather the whole set of shape instances that meet certain criterion. In this paper we develop a method for the assessment of specific anatomical/morphological criteria across the shape variability found in a population. The method is based on a level set segmentation approach, and used on the parametric space of the statistical shape model of the target population, solved via a multi-level narrow-band approach for computational efficiency. Based on this technique, we develop a framework for evidence-based orthopaedic implant design. To date, implants are commonly designed and validated by evaluating implant bone fitting on a limited set of cadaver bones, which not necessarily span the whole variability in the population. Based on our framework, we can virtually fit a proposed implant design to samples drawn from the statistical model, and assess which range of the population is suitable for the implant. The method highlights which patterns of bone variability are more important for implant fitting, allowing and easing implant design improvements, as to fit a maximum of the target population. Results are presented for the optimisation of implant design of proximal human tibia, used for internal fracture fixation.     

Statistical shape model of a liver for autopsy imaging

Purpose

   Modeling the postmortem liver for autopsy imaging is a challenging problem owing to the variation in organ deformation found in cadavers and limited availability of postmortem liver CT scans. An algorithm was developed to construct a statistical shape model (SSM) for the adult postmortem liver in autopsy imaging.

Methods

   First, we investigated the relationship between SSMs obtained from in vivo liver CT scans and those from postmortem cases. Liver shapes were embedded in level set functions and statistically modeled using a spatially weighted principal components analysis. The performance of the SSMs was evaluated in terms of generalization and specificity. Several algorithms for the transformation from in vivo livers to postmortem livers were proposed to enhance the performance of an SSM for a postmortem liver, followed by a comparative study on SSMs. Specifically, five SSMs for a postmortem liver were constructed and evaluated using 32 postmortem liver labels, and postmortem liver labels synthesized from 144 in vivo liver labels were constructed using the proposed transformation algorithms. We also compared the proposed SSMs with three conventional SSMs trained from postmortem liver labels and/or in vivo liver labels.

Results

   The investigation showed that the performance of an SSM constructed using in vivo liver labels suffered when describing postmortem liver shapes. Two of the five proposed SSMs trained using synthesized postmortem livers showed the best performance with no significant differences between them, and they statistically outperformed all conventional SSMs tested.

Conclusions

   The performance of conventional SSMs can be improved by using both postmortem liver shape labels and artificial shape labels synthesized from in vivo liver shape labels.     

A 3-D finite-element model for computation of temperature profiles and regions of thermal damage during focused ultrasound surgery exposures

Although there have been numerous models implemented for modeling thermal diffusion effects during focused ultrasound surgery (FUS), most have limited themselves to representing simple situations for which analytical solutions and the use of cylindrical geometries sufficed. For modeling single lesion formation and the heating patterns from a single exposure, good results were achieved in comparison with experimental results for predicting lesion size, shape and location. However, these types of approaches are insufficient when considering the heating of multiple sites with FUS exposures when the time interval between exposures is short. In such cases, the heat dissipation patterns from initial exposures in the lesion array formation can play a significant role in the heating patterns for later exposures. Understanding the effects of adjacent lesion formation, such as this, requires a three-dimensional (3-D) representation of the bioheat equation. Thus, we have developed a 3-D finite-element representation for modeling the thermal diffusion effects during FUS exposures in clinically relevant tissue volumes. The strength of this approach over past methods is its ability to represent arbitrarily shaped 3-D situations. Initial simulations have allowed calculation of the temperature distribution as a function of time for adjacent FUS exposures in excised bovine liver, with the individually computed point temperatures comparing favorably with published measurements. In addition to modeling these temperature distributions, the model was implemented in conjunction with an algorithm for calculating the thermal dose as a way of predicting lesion shape. Although used extensively in conventional hyperthermia applications, this thermal dose criterion has only been applied in a limited number of simulations in FUS for comparison with experimental measurements. In this study, simulations were run for focal depths 2 and 3 cm below the surface of pig’s liver, using multiple intensity levels and exposure times. The results also compare favorably to published in vitro experimental measurements, which bodes well for future application to more complex problems, such as the modeling of multiple lesion arrays within complex anatomical geometries.     

Growth modeling of human mandibles using non-Euclidean metrics

From a set of 31 three-dimensional computed tomography (CT) scans we model the temporal shape and size of the human mandible for analysis, simulation, and prediction purposes. Each anatomical structure is represented using 14851 semi-landmarks, and mapped into Procrustes tangent space. Exploratory subspace analyses are performed leading to linear models of mandible shape evolution in Procrustes space. The traditional variance analysis results in a one-dimensional growth model. However, working in a non-Euclidean metric results in a multimodal model with uncorrelated modes of biological variation related to independent component analysis. The applied non-Euclidean metric is governed by the correlation structure of the estimated noise in the data. The generative models are compared, and evaluated on the basis of a cross validation study. The new non-Euclidean analysis is completely data driven. It not only gives comparable results w.r.t. previous studies of the mean modeling error, but seems to better correlate to growth, and in addition provides the data analyst with alternative hypothesis of plausible shape evolution; hence aiding in the understanding of cranio-facial growth.     

Atlas-based shape analysis and classification of retinal optical coherence tomography images using the functional shape (fshape) framework

We propose a novel approach for quantitative shape variability analysis in retinal optical coherence tomography images using the functional shape (fshape) framework. The fshape framework uses surface geometry together with functional measures, such as retinal layer thickness defined on the layer surface, for registration across anatomical shapes. This is used to generate a population mean template of the geometry-function measures from each individual. Shape variability across multiple retinas can be measured by the geometrical deformation and functional residual between the template and each of the observations. To demonstrate the clinical relevance and application of the framework, we generated atlases of the inner layer surface and layer thickness of the Retinal Nerve Fiber Layer (RNFL) of glaucomatous and normal subjects, visualizing detailed spatial pattern of RNFL loss in glaucoma. Additionally, a regularized linear discriminant analysis classifier was used to automatically classify glaucoma, glaucoma-suspect, and control cases based on RNFL fshape metrics.