2D-3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models

Three-dimensional patient specific bone models are required in a range of medical applications, such as pre-operative surgery planning and improved guidance during surgery, modeling and simulation, and in vivo bone motion tracking. Shape reconstruction from a small number of X-ray images is desired as it lowers both the acquisition costs and the radiation dose compared to CT. We propose a method for pose estimation and shape reconstruction of 3D bone surfaces from two (or more) calibrated X-ray images using a statistical shape model (SSM). User interaction is limited to manual initialization of the mean shape. The proposed method combines a 3D distance based objective function with automatic edge selection on a Canny edge map. Landmark-edge correspondences are weighted based on the orientation difference of the projected silhouette and the corresponding image edge. The method was evaluated by rigid pose estimation of ground truth shapes as well as 3D shape estimation using a SSM of the whole femur, from stereo cadaver X-rays, in vivo biplane fluoroscopy image-pairs, and an in vivo biplane fluoroscopic sequence. Ground truth shapes for all experiments were available in the form of CT segmentations. Rigid registration of the ground truth shape to the biplane fluoroscopy achieved sub-millimeter accuracy (0.68 mm) measured as root mean squared (RMS) point-to-surface (P2S) distance. The non-rigid reconstruction from the biplane fluoroscopy using the SSM also showed promising results (1.68 mm RMS P2S). A feasibility study on one fluoroscopic time series illustrates the potential of the method for motion and shape estimation from fluoroscopic sequences with minimal user interaction.     

Fully automatic segmentation of the Femur from 3D-CT images using primitive shape recognition and statistical shape models

Purpose

Femur segmentation is well established and widely used in computer-assisted orthopedic surgery. However, most of the robust segmentation methods such as statistical shape models (SSM) require human intervention to provide an initial position for the SSM. In this paper, we propose to overcome this problem and provide a fully automatic femur segmentation method for CT images based on primitive shape recognition and SSM.

Method

Femur segmentation in CT scans was performed using primitive shape recognition based on a robust algorithm such as the Hough transform and RANdom SAmple Consensus. The proposed method is divided into 3 steps: (1) detection of the femoral head as sphere and the femoral shaft as cylinder in the SSM and the CT images, (2) rigid registration between primitives of SSM and CT image to initialize the SSM into the CT image, and (3) fitting of the SSM to the CT image edge using an affine transformation followed by a nonlinear fitting.

Results

The automated method provided good results even with a high number of outliers. The difference of segmentation error between the proposed automatic initialization method and a manual initialization method is less than 1 mm.

Conclusion

The proposed method detects primitive shape position to initialize the SSM into the target image. Based on primitive shapes, this method overcomes the problem of inter-patient variability. Moreover, the results demonstrate that our method of primitive shape recognition can be used for 3D SSM initialization to achieve fully automatic segmentation of the femur.     

Planning of mandibular reconstructions based on statistical shape models

Purpose

The reconstruction of large continuity defects of the mandible is a challenging task, especially when the shape of the missing part is not known prior to operation. Today, the surgical planning is based mainly on visual judgment and the individual skills and experience of the surgeons. The objective of the current study was to develop a computer-based method that is capable of proposing a reconstruction shape from a known residual mandible part.

Methods

The volumetric data derived from 60 CT scans of mandibles were used as the basis for the novel numerical procedure. To find a standardized representation of the mandible shapes, a mesh was elaborated that follows the course of anatomical structures with a specially developed topology of quadrilaterals. These standard meshes were transformed with defined mesh modifications toward each individual mandible surface to allow for further statistical evaluations. The data were used to capture the inter-individual shape variations that were considered as random field variations and mathematically evaluated with principal component analysis. With this information of the mandibular shape variations, an algorithm was developed that proposes shapes for reconstruction planning based on given residual mandible geometry parts.

Results

The accuracy of the novel method was evaluated on six different virtually defined continuity defects that were each created on three mandibles that were not part of the initial database. Virtual reconstructions showed sufficient accuracy of the algorithm for the planning of surgical reconstructions, with average deviations toward the actual geometry of \(1.82 \pm 0.11\) mm for small missing parts and 5 mm for large hemi-lateral defects.

Conclusions

The presented algorithm may be a valuable tool for the planning of mandibular reconstructions. The proposed shapes can be used as templates for computer-aided manufacturing, e.g., with 3D printing devices that use biocompatible materials.

     

Fluoroscopy-based tracking of femoral kinematics with statistical shape models

Purpose

Precise knee kinematics assessment helps to diagnose knee pathologies and to improve the design of customized prosthetic components. The first step in identifying knee kinematics is to assess the femoral motion in the anatomical frame. However, no work has been done on pathological femurs, whose shape can be highly different from healthy ones.

Methods

We propose a new femoral tracking technique based on statistical shape models and two calibrated fluoroscopic images, taken at different flexion–extension angles. The cost function optimization is based on genetic algorithms, to avoid local minima. The proposed approach was evaluated on 3 sets of digitally reconstructed radiographic images of osteoarthritic patients.

Results

It is found that using the estimated shape, rather than that calculated from CT, significantly reduces the pose accuracy, but still has reasonably good results (angle errors around 2\(^\circ \), translation around 1.5 mm).

     

Multi-resolution multi-object statistical shape models based on the locality assumption

Statistical shape models learned from a population of previously observed training shapes are nowadays widely used in medical image analysis to aid segmentation or classification. However, providing an appropriate and representative training population of preferably manual segmentations is typically either very labor-intensive or even impossible. Therefore, statistical shape models in practice frequently suffer from the high-dimension-low-sample-size (HDLSS) problem resulting in models with insufficient expressiveness.In this paper, a novel approach for learning representative multi-resolution multi-object statistical shape models from a small number of training samples that adequately model the variability of each individual object as well as their interrelations is presented. The method is based on the assumption of locality, which means that local shape variations have limited effects in distant areas and, therefore, can be modeled independently. This locality assumption is integrated into the standard statistical shape modeling framework by manipulating the sample covariance matrix (non-zero covariances between distant landmarks are set to zero). To allow for multi-object modeling, a method for computing distances between points located on different object shapes is proposed. Furthermore, different levels of locality are introduced by deriving a multi-resolution scheme, which is equipped with a method to combine variability information modeled at different levels into a single shape model. This combined representation of global and local variability in a single shape model allows the use of the classical active shape model strategy for model-based image segmentation.An extensive evaluation based on a public data base of 247 chest radiographs is performed to show the modeling and segmentation capabilities of the proposed approach in single- and multi-object HDLSS scenarios. The new approach is not only compared to the classical shape modeling method but also to three state-of-the-art shape modeling approaches specifically designed to cope with the HDLSS problem. The results show that the new approach significantly outperforms all other approaches in terms of generalization ability and model-based segmentation accuracy.     

Instantiation and registration of statistical shape models of the femur and pelvis using 3D ultrasound imaging

Statistical shape modelling potentially provides a powerful tool for generating patient-specific, 3D representations of bony anatomy for computer-aided orthopaedic surgery (CAOS) without the need for a preoperative CT scan. Furthermore, freehand 3D ultrasound (US) provides a non-invasive method for digitising bone surfaces in the operating theatre that enables a much greater region to be sampled compared with conventional direct-contact (i.e., pointer-based) digitisation techniques. In this paper, we describe how these approaches can be combined to simultaneously generate and register a patient-specific model of the femur and pelvis to the patient during surgery. In our implementation, a statistical deformation model (SDM) was constructed for the femur and pelvis by performing a principal component analysis on the B-spline control points that parameterise the freeform deformations required to non-rigidly register a training set of CT scans to a carefully segmented template CT scan. The segmented template bone surface, represented by a triangulated surface mesh, is instantiated and registered to a cloud of US-derived surface points using an iterative scheme in which the weights corresponding to the first five principal modes of variation of the SDM are optimised in addition to the rigid-body parameters. The accuracy of the method was evaluated using clinically realistic data obtained on three intact human cadavers (three whole pelves and six femurs). For each bone, a high-resolution CT scan and rigid-body registration transformation, calculated using bone-implanted fiducial markers, served as the gold standard bone geometry and registration transformation, respectively. After aligning the final instantiated model and CT-derived surfaces using the iterative closest point (ICP) algorithm, the average root-mean-square distance between the surfaces was 3.5mm over the whole bone and 3.7mm in the region of surgical interest. The corresponding distances after aligning the surfaces using the marker-based registration transformation were 4.6 and 4.5mm, respectively. We conclude that despite limitations on the regions of bone accessible using US imaging, this technique has potential as a cost-effective and non-invasive method to enable surgical navigation during CAOS procedures, without the additional radiation dose associated with performing a preoperative CT scan or intraoperative fluoroscopic imaging. However, further development is required to investigate errors using error measures relevant to specific surgical procedures.     

Quantitative vertebral morphometry using neighbor-conditional shape models

A novel method for vertebral fracture quantification from X-ray images is presented. Using pairwise conditional shape models trained on a set of healthy spines, the most likely normal vertebra shapes are estimated conditional on the shapes of all other vertebrae in the image. The difference between the true shape and the reconstructed normal shape is subsequently used as a measure of abnormality. In contrast with the current (semi-)quantitative grading strategies this method takes the full shape into account, it develops a patient-specific reference by combining population-based information on biological variation in vertebral shape and vertebra interrelations, and it provides a continuous measure of deformity. The method is demonstrated on 282 lateral spine radiographs with in total 93 fractures. Vertebral fracture detection is shown to be in good agreement with semi-quantitative scoring by experienced radiologists and is superior to the performance of shape models alone.     

Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI

We present a framework for the analysis of short axis cardiac MRI, using statistical models of shape and appearance. The framework integrates temporal and structural constraints and avoids common optimization problems inherent in such high dimensional models. The first contribution is the introduction of an algorithm for fitting 3D active appearance models (AAMs) on short axis cardiac MRI. We observe a 44-fold increase in fitting speed and a segmentation accuracy that is on par with Gauss-Newton optimization, one of the most widely used optimization algorithms for such problems. The second contribution involves an investigation on hierarchical 2D+time active shape models (ASMs), that integrate temporal constraints and simultaneously improve the 3D AAM based segmentation. We obtain encouraging results (endocardial/epicardial error 1.43+/-0.49 mm/1.51+/-0.48 mm) on 7980 short axis cardiac MR images acquired from 33 subjects. We have placed our dataset online, for the community to use and build upon.     

A framework for the merging of pre-existing and correspondenceless 3D statistical shape models

The construction of statistical shape models (SSMs) that are rich, i.e., that represent well the natural and complex variability of anatomical structures, is an important research topic in medical imaging. To this end, existing works have addressed the limited availability of training data by decomposing the shape variability hierarchically or by combining statistical and synthetic models built using artificially created modes of variation. In this paper, we present instead a method that merges multiple statistical models of 3D shapes into a single integrated model, thus effectively encoding extra variability that is anatomically meaningful, without the need for the original or new real datasets. The proposed framework has great flexibility due to its ability to merge multiple statistical models with unknown point correspondences. The approach is beneficial in order to re-use and complement pre-existing SSMs when the original raw data cannot be exchanged due to ethical, legal, or practical reasons. To this end, this paper describes two main stages, i.e., (1) statistical model normalization and (2) statistical model integration. The normalization algorithm uses surface-based registration to bring the input models into a common shape parameterization with point correspondence established across eigenspaces. This allows the model fusion algorithm to be applied in a coherent manner across models, with the aim to obtain a single unified statistical model of shape with improved generalization ability. The framework is validated with statistical models of the left and right cardiac ventricles, the L1 vertebra, and the caudate nucleus, constructed at distinct research centers based on different imaging modalities (CT and MRI) and point correspondences. The results demonstrate that the model integration is statistically and anatomically meaningful, with potential value for merging pre-existing multi-modality statistical models of 3D shapes.     

Robust statistical shape models for MRI bone segmentation in presence of small field of view

Accurate bone modeling from medical images is essential in the diagnosis and treatment of patients because it supports the detection of abnormal bone morphology, which is often responsible for many musculoskeletal diseases (MSDs) of human articulations. In a clinical setting, images of the suspected joints are acquired in a high resolution but with a small field of view (FOV) in order to maximize the image quality while reducing acquisition time. However bones are only partially visible in such small FOVs. This presents difficult challenges in automated bone segmentation and thus limits the application of sophisticated algorithms such as statistical shape models (SSM), which have been generally proven to be an efficient technique for bone segmentation. Indeed, the reduced image information affects the initialization and evolution of these deformable model-based approaches. In this paper, we present a robust multi-resolution SSM algorithm with an adapted initialization to address the segmentation of MRI bone images acquired in small FOVs for modeling and computer-aided diagnosis. Our innovation stems from the derivation of a robust SSM based on complete and corrupted shapes, as well as from a simultaneous optimization of transformation and shape parameters to yield an efficient initialization technique. We demonstrate our segmentation algorithm using 86 clinical MRI images of the femur and hip bones. These images have a varied resolution and limited FOVs. The results of our segmentation (e.g., average distance error of 1.12 ± 0.46 mm) are within the needs of image-based clinical diagnosis.     

Assessment and stenting of tracheal stenosis using deformable shape models

This work presents a decision support system for the assessment of tracheal stenosis. In the proposed method, a statistical shape model of healthy tracheas is registered to a 3D CT image of a patient with tracheal stenosis. The registration yields an estimation of the shape of the patient's trachea as if stenosis was not present. From this point, the extent and the severity of the stenosis is assessed and stent parameters are obtained automatically. The method was extensively evaluated on simulation as well on real data and the results showed that it is accurate and fast enough to be used in the clinical setting.     

Automated liver segmentation from a postmortem CT scan based on a statistical shape model

Purpose

Automated liver segmentation from a postmortem computed tomography (PMCT) volume is a challenging problem owing to the large deformation and intensity changes caused by severe pathology and/or postmortem changes. This paper addresses this problem by a novel segmentation algorithm using a statistical shape model (SSM) for a postmortem liver.

Methods

The location and shape parameters of a liver are directly estimated from a given volume by the proposed SSM-guided expectation–maximization (EM) algorithm without any spatial standardization that might fail owing to the large deformation and intensity changes. The estimated location and shape parameters are then used as a constraint of the subsequent fine segmentation process based on graph cuts. Algorithms with eight different SSMs were trained using 144 in vivo and 32 postmortem livers, and the segmentation algorithm was tested on 32 postmortem livers in a twofold cross validation manner. The segmentation performance is measured by the Jaccard index (JI) between the segmentation result and the true liver label.

Results

The average JI of the segmentation result with the best SSM was 0.8501, which was better compared with the results obtained using conventional SSMs and the results of the previous postmortem liver segmentation with statistically significant difference.

Conclusions

We proposed an algorithm for automated liver segmentation from a PMCT volume, in which an SSM-guided EM algorithm estimated the location and shape parameters of a liver in a given volume accurately. We demonstrated the effectiveness of the proposed algorithm using actual postmortem CT volumes.

     

Optimisation of orthopaedic implant design using statistical shape space analysis based on level sets

Statistical shape analysis techniques have shown to be efficient tools to build population specific models of anatomical variability. Their use is commonplace as prior models for segmentation, in which case the instance from the shape model that best fits the image data is sought. In certain cases, however, it is not just the most likely instance that must be searched, but rather the whole set of shape instances that meet certain criterion. In this paper we develop a method for the assessment of specific anatomical/morphological criteria across the shape variability found in a population. The method is based on a level set segmentation approach, and used on the parametric space of the statistical shape model of the target population, solved via a multi-level narrow-band approach for computational efficiency. Based on this technique, we develop a framework for evidence-based orthopaedic implant design. To date, implants are commonly designed and validated by evaluating implant bone fitting on a limited set of cadaver bones, which not necessarily span the whole variability in the population. Based on our framework, we can virtually fit a proposed implant design to samples drawn from the statistical model, and assess which range of the population is suitable for the implant. The method highlights which patterns of bone variability are more important for implant fitting, allowing and easing implant design improvements, as to fit a maximum of the target population. Results are presented for the optimisation of implant design of proximal human tibia, used for internal fracture fixation.     

Robotic learning of motion using demonstrations and statistical models for surgical simulation

Purpose

   In robotic-assisted surgical training, the expertise of surgeons in maneuvering surgical instruments may be utilized to provide the motion trajectories for teaching. However, the motion primitives for trajectory planning are not known until the motion trajectory is generalized. We hypothesize that a generic model that encodes surgical skills using demonstrations and statistical models can be used by the surgical training robot to determine the motion primitive base on the motion trajectory.

Methods

   The generic model was developed from twenty-two sets of motion trajectories of soft tissue division with laparoscopic scissors collected from a robotic laparoscopic surgical training system. Adaptive mean shift method with initial bandwidth determined by the plug-in-rule method was used to identify the primitives in the motion trajectories. Gaussian Mixture Model was applied to model the underlying motion structure. Gaussian Mixture Regression was then applied to reconstruct a generic motion trajectory for the task.

Results

   The generic model and proposed method were investigated in experiments. Motion trajectory of tissue division was model and reconstructed. The motion model which was trained based on primitives determined by adaptive mean shift method produced RMS error of \(3.05^{\circ }\) and \(3.08^{\circ }\) with respect to the demonstrated trajectories of left and right instruments, respectively. The RMS error was smaller than that of k-means method and fixed bandwidth mean shift method. The dexterous features in the demonstrations were also preserved.

Conclusions

   Surgical tasks can be modeled using Gaussian Mixture Model and motion primitives identified by adaptive mean shift method with minimum user intervention. Generic motion trajectory has been successfully reconstructed based on the motion model. Investigation on the effectiveness of this method and generic model for surgical training is ongoing.     

Articular surface segmentation using active shape models for intraoperative implant assessment

Purpose

In orthopedic surgeries, it is important to avoid intra-articular implant placements, which increase revision rates and the risk of arthritis. In order to support the intraoperative assessment and correction of surgical implants, we present an automatic detection approach using cone-beam computed tomography (CBCT).

Methods

Multiple active shape models (ASM) with specified articular surface regions are used to isolate the joint spaces. Fast and easy-to-implement methods are integrated in the ASM segmentation to optimize the robustness and accuracy for intraoperative application. A cylinder detection method is applied to determine metal implants. Intersections between articular surfaces and cylinders are detected and used to find intra-articular collisions.

Results

Segmentations of two calcaneal articular surfaces were evaluated on 50 patient images and have shown average surface distance errors of 0.59 and 0.46 mm, respectively. The proposed model-independent segmentation at the specified articular surface regions allowed to significantly decrease the error by 22 and 25 % on average. The method was able to compensate suboptimal initializations for translations of up to 16 mm and rotations of up to 21\(^{\circ }\). In a human cadaver test, articular perforations could be localized with an accuracy of 0.80 mm on average.

Conclusions

A concept for automatic intraoperative detection of intra-articular implants in CBCT images was presented. The results show a reliable segmentation of articular surfaces in retrospective patient data and an accurate localization of misplaced implants in artificially created human cadaver test cases.

     

3D image segmentation by using statistical deformation models and level sets

Purpose Improved segmentation of soft objects was sought using a new method that combines level set segmentation with statistical deformation models, using prior knowledge of the shape of an object as well as information derived from the input image. Methods Statistical deformation models were created using Euclidian distance functions of binary data and a multi-hierarchical registration approach based on mutual information metric and demons deformable registration. This approach is motivated by the fact that models based on signed distance maps, traditionally combined with level set segmentation can result in irregular shapes and do not establish explicit correspondences. By using statistical deformation models as representation of shape and a maximum a posteriori (MAP) estimation model to estimate the MAP shape of the object to be segmented, a robust segmentation algorithm using accurate shape models could be developed. Results The accuracy and correctness of the synthesized models was evaluated on different 3D objects (cardiac MRI and spinal CT vertebral segment) and the segmentation algorithm was validated by performing different segmentation tasks using various image modalities. The results of this evaluation are very promising and show the potential utility of the approach. Conclusion Initial results demonstrate the approach is feasible and may be advantageous over alternative segmentation methods. Extensions of the model, which also incorporate prior knowledge about the spatial distribution of grey values, are currently under development.     

Personalized assessment of craniosynostosis via statistical shape modeling

We present a technique for the computational analysis of craniosynostosis from CT images. Our fully automatic methodology uses a statistical shape model to produce diagnostic features tailored to the anatomy of the subject. We propose a computational anatomy approach for measuring shape abnormality in terms of the closest case from a multi-atlas of normal cases. Although other authors have tackled malformation characterization for craniosynostosis in the past, our approach involves several novel contributions (automatic labeling of cranial regions via graph cuts, identification of the closest morphology to a subject using a multi-atlas of normal anatomy, detection of suture fusion, registration using masked regions and diagnosis via classification using quantitative measures of local shape and malformation). Using our automatic technique we obtained for each subject an index of cranial suture fusion, and deformation and curvature discrepancy averages across five cranial bones and six suture regions. Significant differences between normal and craniosynostotic cases were obtained using these characteristics. Machine learning achieved a 92.7% sensitivity and 98.9% specificity for diagnosing craniosynostosis automatically, values comparable to those achieved by trained radiologists. The probability of correctly classifying a new subject is 95.7%.