Geodesic estimation for large deformation anatomical shape averaging and interpolation

The goal of this research is to promote variational methods for anatomical averaging that operate within the space of the underlying image registration problem. This approach is effective when using the large deformation viscous framework, where linear averaging is not valid, or in the elastic case. The theory behind this novel atlas building algorithm is similar to the traditional pairwise registration problem, but with single image forces replaced by average forces. These group forces drive an average transport ordinary differential equation allowing one to estimate the geodesic that moves an image toward the mean shape configuration. This model gives large deformation atlases that are optimal with respect to the shape manifold as defined by the data and the image registration assumptions. We use the techniques in the large deformation context here, but they also pertain to small deformation atlas construction. Furthermore, a natural, inherently inverse consistent image registration is gained for free, as is a tool for constant arc length geodesic shape interpolation. The geodesic atlas creation algorithm is quantitatively compared to the Euclidean anatomical average to elucidate the need for optimized atlases. The procedures generate improved average representations of highly variable anatomy from distinct populations.     

Splines for diffeomorphisms

This paper develops a method for higher order parametric regression on diffeomorphisms for image regression. We present a principled way to define curves with nonzero acceleration and nonzero jerk. This work extends methods based on geodesics which have been developed during the last decade for computational anatomy in the large deformation diffeomorphic image analysis framework. In contrast to previously proposed methods to capture image changes over time, such as geodesic regression, the proposed method can capture more complex spatio-temporal deformations.We take a variational approach that is governed by an underlying energy formulation, which respects the nonflat geometry of diffeomorphisms. Such an approach of minimal energy curve estimation also provides a physical analogy to particle motion under a varying force field. This gives rise to the notion of the quadratic, the cubic and the piecewise cubic splines on the manifold of diffeomorphisms. The variational formulation of splines also allows for the use of temporal control points to control spline behavior. This necessitates the development of a shooting formulation for splines.The initial conditions of our proposed shooting polynomial paths in diffeomorphisms are analogous to the Euclidean polynomial coefficients. We experimentally demonstrate the effectiveness of using the parametric curves both for synthesizing polynomial paths and for regression of imaging data. The performance of the method is compared to geodesic regression.     

Groupwise registration with global-local graph shrinkage in atlas construction

Graph-based groupwise registration methods are widely used in atlas construction. Given a group of images, a graph is built whose nodes represent the images, and whose edges represent a geodesic path between two nodes. The distribution of images on an image manifold is explored through edge traversal in a graph. The final atlas is a mean image at the population center of the distribution on the manifold. The procedure of warping all images to the mean image turns to dynamic graph shrinkage in which nodes become closer to each other. Most conventional groupwise registration frameworks construct and shrink a graph without considering the local distribution of images on the dataset manifold and the local structure variations between image pairs. Neglecting the local information fundamentally decrease the accuracy and efficiency when population atlases are built for organs with large inter-subject anatomical variabilities. To overcome the problem, this paper proposes a global-local graph shrinkage approach that can generate accurate atlas. A connected graph is constructed automatically based on global similarities across the images to explore the global distribution. A local image distribution obtained by image clustering is used to simplify the edges of the constructed graph. Subsequently, local image similarities refine the deformation estimated through global image similarity for each image warping along the graph edges. Through the image warping, the overall simplified graph shrinks gradually to yield the atlas with respecting both global and local features. The proposed method is evaluated on 61 synthetic and 20 clinical liver datasets, and the results are compared with those of six state-of-the-art groupwise registration methods. The experimental results show that the proposed method outperforms non-global-local method approaches in terms of accuracy.     

Large deformation diffeomorphic registration of diffusion-weighted imaging data

Registration plays an important role in group analysis of diffusion-weighted imaging (DWI) data. It can be used to build a reference anatomy for investigating structural variation or tracking changes in white matter. Unlike traditional scalar image registration where spatial alignment is the only focus, registration of DWI data requires both spatial alignment of structures and reorientation of local signal profiles. As such, DWI registration is much more complex and challenging than scalar image registration. Although a variety of algorithms has been proposed to tackle the problem, most of them are restricted by the diffusion model used for registration, making it difficult to fit to the registered data a different model. In this paper we describe a method that allows any diffusion model to be fitted after registration for subsequent multifaceted analysis. This is achieved by directly aligning DWI data using a large deformation diffeomorphic registration framework. Our algorithm seeks the optimal coordinate mapping by simultaneously considering structural alignment, local signal profile reorientation, and deformation regularization. Our algorithm also incorporates a multi-kernel strategy to concurrently register anatomical structures at different scales. We demonstrate the efficacy of our approach using in vivo data and report detailed qualitative and quantitative results in comparison with several different registration strategies.     

Large deformation diffeomorphic registration of diffusion-weighted imaging data

Registration plays an important role in group analysis of diffusion-weighted imaging (DWI) data. It can be used to build a reference anatomy for investigating structural variation or tracking changes in white matter. Unlike traditional scalar image registration where spatial alignment is the only focus, registration of DWI data requires both spatial alignment of structures and reorientation of local signal profiles. As such, DWI registration is much more complex and challenging than scalar image registration. Although a variety of algorithms has been proposed to tackle the problem, most of them are restricted by the diffusion model used for registration, making it difficult to fit to the registered data a different model. In this paper we describe a method that allows any diffusion model to be fitted after registration for subsequent multifaceted analysis. This is achieved by directly aligning DWI data using a large deformation diffeomorphic registration framework. Our algorithm seeks the optimal coordinate mapping by simultaneously considering structural alignment, local signal profile reorientation, and deformation regularization. Our algorithm also incorporates a multi-kernel strategy to concurrently register anatomical structures at different scales. We demonstrate the efficacy of our approach using in vivo data and report detailed qualitative and quantitative results in comparison with several different registration strategies.     

A multi-resolution scheme for distortion-minimizing mapping between human subcortical structures based on geodesic construction on Riemannian manifolds

In this paper, we deal with a subcortical surface registration problem. Subcortical structures including hippocampi and caudates have a small number of salient features such as heads and tails unlike cortical surfaces. Therefore, it is hard, if not impossible, to perform subcortical surface registration with only such features. It is also non-trivial for neuroanatomical experts to select landmarks consistently for subcortical surfaces of different subjects. We therefore present a landmark-free approach for subcortical surface registration by measuring the amount of mesh distortion between subcortical surfaces assuming that the surfaces are represented by meshes. The input meshes can be constructed using any surface modeling tool available in the public domain since our registration method is independent of a surface modeling process. Given the source and target surfaces together with their representing meshes, the vertex positions of the source mesh are iteratively displaced while preserving the underlying surface shape in order to minimize the distortion to the target mesh. By representing each surface mesh as a point on a high-dimensional Riemannian manifold, we define a distance metric on the manifold that measures the amount of distortion from a given source mesh to the target mesh, based on the notion of isometry while penalizing triangle flipping. Under this metric, we reduce the distortion minimization problem to the problem of constructing a geodesic curve from the moving source point to the fixed target point on the manifold while satisfying the shape-preserving constraint. We adopt a multi-resolution framework to solve the problem for distortion-minimizing mapping between the source and target meshes. We validate our registration scheme through several experiments: distance metric comparison, visual validation using real data, robustness test to mesh variations, feature alignment using anatomic landmarks, consistency with previous clinical findings, and comparison with a surface-based registration method, LDDMM-surface.     

Inter-subject registration-based one-shot segmentation with alternating union network for cardiac MRI images

Medical image segmentation based on deep-learning networks makes great progress in assisting disease diagnosis. However, currently, the training of most networks still requires a large amount of data with labels. In reality, labeling a considerable number of medical images is challenging and time-consuming. In order to tackle this challenge, a new one-shot segmentation framework for cardiac MRI images based on an inter-subject registration model called Alternating Union Network (AUN) is proposed in this study. The label of the source image is warped with deformation fields discovered from AUN to segment target images directly. Initially, the volumes are pre-processed by aligning affinely and adjusting the global intensity to simplify subsequent deformation registration. AUN consists of two kinds of subnetworks trained alternately to optimize segmentation gradually. The first kind of subnetwork takes a pair of volumes as inputs and registers them using global intensity similarity. The second kind of subnetwork, which takes the predicted labels generated from the previous subnetwork and the labels refined using the information of intrinsic anatomical structures of interest as inputs, is intensity-independent and focuses attention on registering structures of interest. Specifically, the input of AUN is a pair of a labeled image with the texture in regions of interest removed and a target image. Additionally, a new similarity measurement more appropriate for registering such image pair is defined as Local Squared Error (LSE). The proposed registration-based one-shot segmentation pays attention to the problem of the lack of labeled medical images. In AUN, only one labeled volume is required and a large number of unlabeled ones can be leveraged to improve segmentation performance, which has great advantages in clinical application. In addition, the intensity-independent subnetwork and LSE proposed in this study empower the framework to segment medical images with complicated intensity distribution.     

Image registration based on virtual frame sequence analysis

Purpose This paper is to propose a new framework for medical image registration with large nonrigid deformations, which still remains one of the biggest challenges for image fusion and further analysis in many medical applications. Method Registration problem is formulated as to recover a deformation process with the known initial state and final state. To deal with large nonlinear deformations, virtual frames are proposed to be inserted to model the deformation process. A time parameter is introduced and the deformation between consecutive frames is described with a linear affine transformation. Results Experiments are conducted with simple geometric deformation as well as complex deformations presented in MRI and ultrasound images. All the deformations are characterized with nonlinearity. The positive results demonstrated the effectiveness of this algorithm. Conclusion The framework proposed in this paper is feasible to register medical images with large nonlinear deformations and is especially useful for sequential images.     

SharpMean: groupwise registration guided by sharp mean image and tree-based registration

Groupwise registration has become more and more popular due to its attractiveness for unbiased analysis of population data. One of the most popular approaches for groupwise registration is to iteratively calculate the group mean image and then register all subject images towards the latest estimated group mean image. However, its performance might be undermined by the fuzzy mean image estimated in the very beginning of groupwise registration procedure, because all subject images are far from being well-aligned at that moment. In this paper, we first point out the significance of always keeping the group mean image sharp and clear throughout the entire groupwise registration procedure, which is intuitively important but has not been explored in the literature yet. To achieve this, we resort to developing the robust mean-image estimator by the adaptive weighting strategy, where the weights are adaptive across not only the individual subject images but also all spatial locations in the image domain. On the other hand, we notice that some subjects might have large anatomical variations from the group mean image, which challenges most of the state-of-the-art registration algorithms. To ensure good registration results in each iteration, we explore the manifold of subject images and build a minimal spanning tree (MST) with the group mean image as the root of the MST. Therefore, each subject image is only registered to its parent node often with similar shapes, and its overall transformation to the group mean image space is obtained by concatenating all deformations along the paths connecting itself to the root of the MST (the group mean image). As a result, all the subjects will be well aligned to the group mean image adaptively. Our method has been evaluated in both real and simulated datasets. In all experiments, our method outperforms the conventional algorithm which generally produces a fuzzy group mean image throughout the entire groupwise registration.     

Anatomically constrained and attention-guided deep feature fusion for joint segmentation and deformable medical image registration

The main objective of anatomically plausible results for deformable image registration is to improve model’s registration accuracy by minimizing the difference between a pair of fixed and moving images. Since many anatomical features are closely related to each other, leveraging supervision from auxiliary tasks (such as supervised anatomical segmentation) has the potential to enhance the realism of the warped images after registration. In this work, we employ a Multi-Task Learning framework to formulate registration and segmentation as a joint issue, in which we utilize anatomical constraint from auxiliary supervised segmentation to enhance the realism of the predicted images. First, we propose a Cross-Task Attention Block to fuse the high-level feature from both the registration and segmentation network. With the help of initial anatomical segmentation, the registration network can benefit from learning the task-shared feature correlation and rapidly focusing on the parts that need deformation. On the other hand, the anatomical segmentation discrepancy from ground-truth fixed annotations and predicted segmentation maps of initial warped images are integrated into the loss function to guide the convergence of the registration network. Ideally, a good deformation field should be able to minimize the loss function of registration and segmentation. The voxel-wise anatomical constraint inferred from segmentation helps the registration network to reach a global optimum for both deformable and segmentation learning. Both networks can be employed independently during the testing phase, enabling only the registration output to be predicted when the segmentation labels are unavailable. Qualitative and quantitative results indicate that our proposed methodology significantly outperforms the previous state-of-the-art approaches on inter-patient brain MRI registration and pre- and intra-operative uterus MRI registration tasks within our specific experimental setup, which leads to state-of-the-art registration quality scores of 0.755 and 0.731 (i.e., by 0.8% and 0.5% increases) DSC for both tasks, respectively.     

Multi-manifold diffeomorphic metric mapping for aligning cortical hemispheric surfaces

Cortical surface-based analysis has been widely used in anatomical and functional studies because it is geometrically appropriate for the cortex. One of the main challenges in the cortical surface-based analysis is to optimize the alignment of the cortical hemispheric surfaces across individuals. In this paper, we introduce a multi-manifold large deformation diffeomorphic metric mapping (MM-LDDMM) algorithm that allows simultaneously carrying the cortical hemispheric surface and its sulcal curves from one to the other through a flow of diffeomorphisms. We present an algorithm based on recent derivation of a law of momentum conservation for the geodesics of diffeomorphic flow. Once a template is fixed, the space of initial momentum becomes an appropriate space for studying shape via geodesic flow since the flow at any point on curves and surfaces along the geodesic is completely determined by the momentum at the origin. We solve for trajectories (geodesics) of the kinetic energy by computing its variation with respect to the initial momentum and by applying a gradient descent scheme. The MM-LDDMM algorithm optimizes the initial momenta encoding the anatomical variation of each individual relative to a common coordinate system in a linear space, which provides a natural scheme for shape deformation average and template (or atlas) generation. We applied the MM-LDDMM algorithm for constructing the templates for the cortical surface and 14 sulcal curves of each hemisphere using a group of 40 subjects. The estimated template shape reflects regions which are highly variable across these subjects. Compared with existing single-manifold LDDMM algorithms, such as the LDDMM-curve mapping and the LDDMM-surface mapping, the MM-LDDMM mapping provides better results in terms of surface to surface distances in five predefined regions.     

Probabilistic inference of regularisation in non-rigid registration

A long-standing issue in non-rigid image registration is the choice of the level of regularisation. Regularisation is necessary to preserve the smoothness of the registration and penalise against unnecessary complexity. The vast majority of existing registration methods use a fixed level of regularisation, which is typically hand-tuned by a user to provide “nice" results. However, the optimal level of regularisation will depend on the data which is being processed; lower signal-to-noise ratios require higher regularisation to avoid registering image noise as well as features, and different pairs of images require registrations of varying complexity depending on their anatomical similarity. In this paper we present a probabilistic registration framework that infers the level of regularisation from the data. An additional benefit of this proposed probabilistic framework is that estimates of the registration uncertainty are obtained. This framework has been implemented using a free-form deformation transformation model, although it would be generically applicable to a range of transformation models. We demonstrate our registration framework on the application of inter-subject brain registration of healthy control subjects from the NIREP database. In our results we show that our framework appropriately adapts the level of regularisation in the presence of noise, and that inferring regularisation on an individual basis leads to a reduction in model over-fitting as measured by image folding while providing a similar level of overlap.     

Bayesian principal geodesic analysis for estimating intrinsic diffeomorphic image variability

In this paper, we present a generative Bayesian approach for estimating the low-dimensional latent space of diffeomorphic shape variability in a population of images. We develop a latent variable model for principal geodesic analysis (PGA) that provides a probabilistic framework for factor analysis in the space of diffeomorphisms. A sparsity prior in the model results in automatic selection of the number of relevant dimensions by driving unnecessary principal geodesics to zero. To infer model parameters, including the image atlas, principal geodesic deformations, and the effective dimensionality, we introduce an expectation maximization (EM) algorithm. We evaluate our proposed model on 2D synthetic data and the 3D OASIS brain database of magnetic resonance images, and show that the automatically selected latent dimensions from our model are able to reconstruct unobserved testing images with lower error than both linear principal component analysis (LPCA) in the image space and tangent space principal component analysis (TPCA) in the diffeomorphism space.     

Soliton dynamics in computational anatomy

Computational anatomy (CA) has introduced the idea of anatomical structures being transformed by geodesic deformations on groups of diffeomorphisms. Among these geometric structures, landmarks and image outlines in CA are shown to be singular solutions of a partial differential equation that is called the geodesic EPDiff equation. A recently discovered momentum map for singular solutions of EPDiff yields their canonical Hamiltonian formulation, which in turn provides a complete parameterization of the landmarks by their canonical positions and momenta. The momentum map provides an isomorphism between landmarks (and outlines) for images and singular soliton solutions of the EPDiff equation. This isomorphism suggests a new dynamical paradigm for CA, as well as new data representation.     

Homeomorphic brain image segmentation with topological and statistical atlases

Atlas-based segmentation techniques are often employed to encode anatomical information for the delineation of multiple structures in magnetic resonance images of the brain. One of the primary challenges of these approaches is to efficiently model qualitative and quantitative anatomical knowledge without introducing a strong bias toward certain anatomical preferences when segmenting new images. This paper explores the use of topological information as a prior and proposes a segmentation framework based on both topological and statistical atlases of brain anatomy. Topology can be used to describe continuity of structures, as well as the relationships between structures, and is often a critical component in cortical surface reconstruction and deformation-based morphometry. Our method guarantees strict topological equivalence between the segmented image and the atlas, and relies only weakly on a statistical atlas of shape. Tissue classification and fast marching methods are used to provide a powerful and flexible framework to handle multiple image contrasts, high levels of noise, gain field inhomogeneities, and variable anatomies. The segmentation algorithm has been validated on simulated and real brain image data and made freely available to researchers. Our experiments demonstrate the accuracy and robustness of the method and the limited influence of the statistical atlas.     

Rapid inference of personalised left-ventricular meshes by deformation-based differentiable mesh voxelization

We propose a differentiable volumetric mesh voxelization technique based on deformation of a shape-model, and demonstrate that it can be used to predict left-ventricular anatomies directly from magnetic resonance image slice data. The predicted anatomies are volumetric meshes suitable for direct inclusion in biophysical simulations. The proposed method can leverage existing (pixel-based) segmentation networks, and does not require any ground truth paired image and mesh training data. We demonstrate that this approach produces accurate predictions from few slices, and can combine information from images acquired in different views (e.g. fusing shape information from short axis and long axis slices). We demonstrate that the proposed method is several times faster than a state-of-the-art registration based method. Additionally, we show that our method can correct for slice misalignment, and is robust to incomplete and inaccurate input data. We further demonstrate that by fitting a mesh to every frame of 4D data we can determine ejection fraction, stroke volume and strain.     

Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis

Ultrasound (US) imaging is widely used for anatomical structure inspection in clinical diagnosis. The training of new sonographers and deep learning based algorithms for US image analysis usually requires a large amount of data. However, obtaining and labeling large-scale US imaging data are not easy tasks, especially for diseases with low incidence. Realistic US image synthesis can alleviate this problem to a great extent. In this paper, we propose a generative adversarial network (GAN) based image synthesis framework. Our main contributions include: (1) we present the first work that can synthesize realistic B-mode US images with high-resolution and customized texture editing features; (2) to enhance structural details of generated images, we propose to introduce auxiliary sketch guidance into a conditional GAN. We superpose the edge sketch onto the object mask and use the composite mask as the network input; (3) to generate high-resolution US images, we adopt a progressive training strategy to gradually generate high-resolution images from low-resolution images. In addition, a feature loss is proposed to minimize the difference of high-level features between the generated and real images, which further improves the quality of generated images; (4) the proposed US image synthesis method is quite universal and can also be generalized to the US images of other anatomical structures besides the three ones tested in our study (lung, hip joint, and ovary); (5) extensive experiments on three large US image datasets are conducted to validate our method. Ablation studies, customized texture editing, user studies, and segmentation tests demonstrate promising results of our method in synthesizing realistic US images.     

Whole brain diffeomorphic metric mapping via integration of sulcal and gyral curves, cortical surfaces, and images

This paper introduces a novel large deformation diffeomorphic metric mapping algorithm for whole brain registration where sulcal and gyral curves, cortical surfaces, and intensity images are simultaneously carried from one subject to another through a flow of diffeomorphisms. To the best of our knowledge, this is the first time that the diffeomorphic metric from one brain to another is derived in a shape space of intensity images and point sets (such as curves and surfaces) in a unified manner. We describe the Euler-Lagrange equation associated with this algorithm with respect to momentum, a linear transformation of the velocity vector field of the diffeomorphic flow. The numerical implementation for solving this variational problem, which involves large-scale kernel convolution in an irregular grid, is made feasible by introducing a class of computationally friendly kernels. We apply this algorithm to align magnetic resonance brain data. Our whole brain mapping results show that our algorithm outperforms the image-based LDDMM algorithm in terms of the mapping accuracy of gyral/sulcal curves, sulcal regions, and cortical and subcortical segmentation. Moreover, our algorithm provides better whole brain alignment than combined volumetric and surface registration (Postelnicu et al., 2009) and hierarchical attribute matching mechanism for elastic registration (HAMMER) (Shen and Davatzikos, 2002) in terms of cortical and subcortical volume segmentation.     

Talairach–Tournoux brain atlas registration using a metalforming principle-based finite element method

In this paper, a novel non-rigid registration method is proposed for registration of the Talairach-Tournoux brain atlas with MRI images and the Schaltenbrand-Wahren brain atlas. A metalforming principle-based finite element method with the large deformation problem is used to find the local deformation, in which finite element equations are governed by constraints in the form of displacements derived from the correspondence relationship between extracted feature points. Some detectable substructures, such as the cortical surface, ventricles and corpus callosum, are first extracted from MRI, forming feature points which are classified into different groups. The softassign method is used to establish the correspondence relationship between feature points within each group and to obtain the global transformation concurrently. The displacement constraints are then derived from the correspondence relationship. A metalforming principle-based finite element method with the large deformation problem is used in which finite element equations are reorganized and simplified by integrating the displacement constraints into the system equations. Our method not only matches the model to the data efficiently, but also decreases the degrees of freedom of the system and consequently reduces the computational cost. The method is illustrated by matching the Talairach-Tournoux brain atlas to MRI normal and pathological data and to the Schaltenbrand-Wahren brain atlas. We compare the results quantitatively between the force assignment-based method and the proposed method. The results show that the proposed method yields more accurate results in a fraction of the time taken by the previous method.     

An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration

Several biomedical applications require accurate image registration that can cope effectively with complex organ deformations. This paper addresses this problem by introducing a generic deformable registration algorithm with a new regularization scheme, which is performed through bilateral filtering of the deformation field. The proposed approach is primarily designed to handle smooth deformations both between and within body structures, and also more challenging deformation discontinuities exhibited by sliding organs. The conventional Gaussian smoothing of deformation fields is replaced by a bilateral filtering procedure, which compromises between the spatial smoothness and local intensity similarity kernels, and is further supported by a deformation field similarity kernel. Moreover, the presented framework does not require any explicit prior knowledge about the organ motion properties (e.g. segmentation) and therefore forms a fully automated registration technique. Validation was performed using synthetic phantom data and publicly available clinical 4D CT lung data sets. In both cases, the quantitative analysis shows improved accuracy when compared to conventional Gaussian smoothing. In addition, we provide experimental evidence that masking the lungs in order to avoid the problem of sliding motion during registration performs similarly in terms of the target registration error when compared to the proposed approach, however it requires accurate lung segmentation. Finally, quantification of the level and location of detected sliding motion yields visually plausible results by demonstrating noticeable sliding at the pleural cavity boundaries.