Landmark constellation models for medical image content identification and localization

Purpose

Many medical imaging tasks require the detection and localization of anatomical landmarks, for example for the initialization of model-based segmentation or to detect anatomical regions present in an image. A large number of landmark and object localization methods have been described in the literature. The detection of single landmarks may be insufficient to achieve robust localization across a variety of imaging settings and subjects. Furthermore, methods like the generalized Hough transform yield the most likely location of an object, but not an indication whether or not the landmark was actually present in the image.

Methods

For these reasons, we developed a simple and computationally efficient method combining localization results from multiple landmarks to achieve robust localization and to compute a localization confidence measure. For each anatomical region, we train a constellation model indicating the mean relative locations and location variability of a set of landmarks. This model is registered to the landmarks detected in a test image via point-based registration, using closed-form solutions. Three different outlier suppression schemes are compared, two using iterative re-weighting based on the residual landmark registration errors and the third being a variant of RANSAC. The mean weighted residual registration error serves as a confidence measure to distinguish true from false localization results. The method is optimized and evaluated on synthetic data, evaluating both the localization accuracy and the ability to classify good from bad registration results based on the residual registration error.

Results

Two application examples are presented: the identification of the imaged anatomical region in trauma CT scans and the initialization of model-based segmentation for C-arm CT scans with different target regions. The identification of the target region with the presented method was in 96 % of the cases correct.

Conclusion

The presented method is a simple solution for combining multiple landmark localization results. With appropriate parameters, outlier suppression clearly improves the localization performance over model registration without outlier suppression. The optimum choice of method and parameters depends on the expected level of noise and outliers in the application at hand, as well as on the focus on localization, classification, or both. The method allows detecting and localizing anatomical fields of view in medical images and is well suited to support a wide range of applications comprising image content identification, anatomical navigation and visualization, or initializing the pose of organ shape models.

     

Registration-based segmentation of three-dimensional ultrasound images for quantitative measurement of fetal craniofacial structure

Segmentation of a fetal head from three-dimensional (3-D) ultrasound images is a critical step in the quantitative measurement of fetal craniofacial structure. However, two main issues complicate segmentation, including fuzzy boundaries and large variations in pose and shape among different ultrasound images. In this article, we propose a new registration-based method for automatically segmenting the fetal head from 3-D ultrasound images. The proposed method first detects the eyes based on Gabor features to identify the pose of the fetus image. Then, a reference model, which is constructed from a fetal phantom and contains prior knowledge of head shape, is aligned to the image via feature-based registration. Finally, 3-D snake deformation is utilized to improve the boundary fitness between the model and image. Four clinically useful parameters including inter-orbital diameter (IOD), bilateral orbital diameter (BOD), occipital frontal diameter (OFD) and bilateral parietal diameter (BPD) are measured based on the results of the eye detection and head segmentation. Ultrasound volumes from 11 subjects were used for validation of the method accuracy. Experimental results showed that the proposed method was able to overcome the aforementioned difficulties and achieve good agreement between automatic and manual measurements.     

A hybrid method to improve target registration accuracy in surgical navigation

Background: The accuracy of surgical navigation depends greatly on that of registration between the patient and the medical image. Point-based registration has been the most common and reliable method, which typically uses skin markers. Unfortunately, high registration accuracy around the markers is not sustained at targets deeply seated within the body. To address such increase in target registration error (TRE), we proposed a hybrid point-based registration method that incorporates anatomical landmarks near the target. Material and methods: Ultrasound calibration is performed with an optical tracker for coordinate frame conversion of image coordinates into the real world. With the calibrated ultrasound probe, we could non-invasively obtain landmark positions near the target, being used together with skin markers for registration. Results: In the experiment, we examined registration accuracies achieved with and without use of an anatomical landmark. We confirmed that using an additional anatomical landmark in registration resulted in an increase in fiducial regsitration error (FRE), but a significant decrease in TRE (p < 0.001). Conclusion: We proposed and demonstrated the effectiveness of a hybrid method that uses both artificial and anatomical landmarks for patient-to-image registration. The experimental results confirmed that an improvement in TRE was evident by the proposed method, suggesting its feasibility in various spinal surgeries.     

Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations

We present an algorithm for automatic elastic registration of three-dimensional (3D) medical images. Our algorithm initially recovers the global spatial mismatch between the reference and floating images, followed by hierarchical octree-based subdivision of the reference image and independent registration of the floating image with the individual subvolumes of the reference image at each hierarchical level. Global as well as local registrations use the six-parameter full rigid-body transformation model and are based on maximization of normalized mutual information (NMI). To ensure robustness of the subvolume registration with low voxel counts, we calculate NMI using a combination of current and prior mutual histograms. To generate a smooth deformation field, we perform direct interpolation of six-parameter rigid-body subvolume transformations obtained at the last subdivision level. Our interpolation scheme involves scalar interpolation of the 3D translations and quaternion interpolation of the 3D rotational pose. We analyzed the performance of our algorithm through experiments involving registration of synthetically deformed computed tomography (CT) images. Our algorithm is general and can be applied to image pairs of any two modalities of most organs. We have demonstrated successful registration of clinical whole-body CT and positron emission tomography (PET) images using this algorithm. The registration accuracy for this application was evaluated, based on validation using expert-identified anatomical landmarks in 15 CT-PET image pairs. The algorithm's performance was comparable to the average accuracy observed for three expert-determined registrations in the same 15 image pairs.     

基于频域时域相结合的医学图像配准算法

为解决存在平移变换和旋转变换的2幅刚性医学图像的配准问题,结合图像信号傅里叶变换的平移特性和旋转特性,提出了一种基于频域与时域相结合的医学图像配准算法.该算法将旋转调整和平移调整分而治之,首先以图像信号的幅度谱巨相关确定旋转参数,再以图像的时域体素互相关确定平移参数.将该算法用于对核医学图像配准及多模态医学图像配准实验,结果表明该算法是一种速度快、精度高、鲁棒性好的医学图像配准方法.     

Calculation of the confidence intervals for transformation parameters in the registration of medical images

Images from different individuals typically cannot be registered precisely because anatomical features within the images differ across the people imaged and because the current methods for image registration have inherent technological limitations that interfere with perfect registration. Quantifying the inevitable error in image registration is therefore of crucial importance in assessing the effects that image misregistration may have on subsequent analyses in an imaging study. We have developed a mathematical framework for quantifying errors in registration by computing the confidence intervals of the estimated parameters (3 translations, 3 rotations, and 1 global scale) for the similarity transformation.The presence of noise in images and the variability in anatomy across individuals ensures that estimated registration parameters are always random variables. We assume a functional relation among intensities across voxels in the images, and we use the theory of nonlinear, least-squares estimation to show that the parameters are multivariate Gaussian distributed. We then use the covariance matrix of this distribution to compute the confidence intervals of the transformation parameters. These confidence intervals provide a quantitative assessment of the registration error across the images. Because transformation parameters are nonlinearly related to the coordinates of landmark points in the brain, we subsequently show that the coordinates of those landmark points are also multivariate Gaussian distributed. Using these distributions, we then compute the confidence intervals of the coordinates for landmark points in the image. Each of these confidence intervals in turn provides a quantitative assessment of the registration error at a particular landmark point. Because our method is computationally intensive, however, its current implementation is limited to assessing the error of the parameters in the similarity transformation across images.We assessed the performance of our method in computing the error in estimated similarity parameters by applying that method to real world dataset. Our results showed that the size of the confidence intervals computed using our method decreased – i.e. our confidence in the registration of images from different individuals increased – for increasing amounts of blur in the images. Moreover, the size of the confidence intervals increased for increasing amounts of noise, misregistration, and differing anatomy. Thus, our method precisely quantified confidence in the registration of images that contain varying amounts of misregistration and varying anatomy across individuals.     

Left ventricle wall motion quantification from echocardiographic images by non-rigid image registration

Purpose

The aim of this study is to evaluate the efficiency of applying a new non-rigid image registration method on two-dimensional echocardiographic images for computing the left ventricle (LV) myocardial motion field over a cardiac cycle.

Methods

The key feature of our method is to register all images in the sequence to a reference image (end-diastole image) using a hierarchical transformation model, which is a combination of an affine transformation for modeling the global LV motion and a free-form deformation (FFD) transformation based on B-splines for modeling the local LV deformation. Registration is done by minimizing a cost function associated with the image similarity based on a global pixel-based matching and the smoothness of transformation. The algorithm uses a fast and robust optimization strategy using a multiresolution approach for the estimation of parameters of the deformation model. The proposed algorithm is evaluated for calculating the displacement curves of two expert-identified anatomical landmarks in apical views of the LV for 10 healthy volunteers and 14 subjects with pathology. The proposed algorithm is also evaluated for classifying the regional LV wall motion abnormality using the calculation of the strain value at the end of systole in 288 segments as scored by two consensual experienced echocardiographers in a three-point scale: 1: normokinesia, 2: hypokinesia, and 3: akinesia. Moreover, we compared the results of the proposed registration algorithm to those previously obtained using the other image registration methods.

Results

Regarding to the reference two experienced echocardiographers, the results demonstrate the proposed algorithm more accurately estimates the displacement curve of the two anatomical landmarks in apical views than the other registration methods in all data set. Moreover, the p values of the t test for the strain value of each segment at the end of systole measured by the proposed algorithm show higher differences than the other registration method. These differences are between each pair of scores in all segments and in three segments of septum independently.

Conclusions

The clinical results show that the proposed algorithm can improve both the calculation of the displacement curve of every point of LV during a cardiac cycle and the classification of regional LV wall motion abnormality. Therefore, this diagnostic system can be used as a useful tool for clinical evaluation of the regional LV function.     

Integration of three-dimensional left atrial magnetic resonance images into a real-time electroanatomic mapping system: validation of a registration method

Background: The alignment of three-dimensional (3D) left atrial images acquired by magnetic resonance (MR) with the anatomical information yielded by 3D mapping systems is one of the most critical issues in image integration techniques for catheter ablation of atrial fibrillation (AF). We assessed the accuracy of a simplified method of superimposing 3D MR left atrial images on real-time left atrial electroanatomic maps (registration).
Methods: MR data on the left atrium in 40 patients with drug-refractory AF were imported into the CartoMerge™ (Biosense Webster, Inc., Diamond Bar, CA, USA) electroanatomic mapping system. Registration was obtained by combining "visual alignment" of one endocardial point and "surface registration" of a limited number of points sampled on the posterior wall of the left atrium. The accuracy of the registration process was assessed through a statistical algorithm incorporated into the CartoMerge™ system, and through the percentage of pulmonary veins (PVs) in which electrical isolation was achieved after anatomical ablation.
Results: The mean registration surface-to-point distance and ablation surface-to-point distance were 1.33 ± 0.96 mm and 1.47 ± 1.15 mm, respectively. Upon completion of the circumferential anatomical ablation around the PVs, electrical PV isolation was confirmed by a multipolar circular mapping catheter in 129 of 146 PVs (89%).
Conclusions: Our registration method, which is mainly based on the surface registration of the posterior wall of the left atrium, enables almost 90% of PVs to be isolated by means of an anatomically based catheter ablation approach.     

Geodesic estimation for large deformation anatomical shape averaging and interpolation

The goal of this research is to promote variational methods for anatomical averaging that operate within the space of the underlying image registration problem. This approach is effective when using the large deformation viscous framework, where linear averaging is not valid, or in the elastic case. The theory behind this novel atlas building algorithm is similar to the traditional pairwise registration problem, but with single image forces replaced by average forces. These group forces drive an average transport ordinary differential equation allowing one to estimate the geodesic that moves an image toward the mean shape configuration. This model gives large deformation atlases that are optimal with respect to the shape manifold as defined by the data and the image registration assumptions. We use the techniques in the large deformation context here, but they also pertain to small deformation atlas construction. Furthermore, a natural, inherently inverse consistent image registration is gained for free, as is a tool for constant arc length geodesic shape interpolation. The geodesic atlas creation algorithm is quantitatively compared to the Euclidean anatomical average to elucidate the need for optimized atlases. The procedures generate improved average representations of highly variable anatomy from distinct populations.     

Anatomically constrained and attention-guided deep feature fusion for joint segmentation and deformable medical image registration

The main objective of anatomically plausible results for deformable image registration is to improve model’s registration accuracy by minimizing the difference between a pair of fixed and moving images. Since many anatomical features are closely related to each other, leveraging supervision from auxiliary tasks (such as supervised anatomical segmentation) has the potential to enhance the realism of the warped images after registration. In this work, we employ a Multi-Task Learning framework to formulate registration and segmentation as a joint issue, in which we utilize anatomical constraint from auxiliary supervised segmentation to enhance the realism of the predicted images. First, we propose a Cross-Task Attention Block to fuse the high-level feature from both the registration and segmentation network. With the help of initial anatomical segmentation, the registration network can benefit from learning the task-shared feature correlation and rapidly focusing on the parts that need deformation. On the other hand, the anatomical segmentation discrepancy from ground-truth fixed annotations and predicted segmentation maps of initial warped images are integrated into the loss function to guide the convergence of the registration network. Ideally, a good deformation field should be able to minimize the loss function of registration and segmentation. The voxel-wise anatomical constraint inferred from segmentation helps the registration network to reach a global optimum for both deformable and segmentation learning. Both networks can be employed independently during the testing phase, enabling only the registration output to be predicted when the segmentation labels are unavailable. Qualitative and quantitative results indicate that our proposed methodology significantly outperforms the previous state-of-the-art approaches on inter-patient brain MRI registration and pre- and intra-operative uterus MRI registration tasks within our specific experimental setup, which leads to state-of-the-art registration quality scores of 0.755 and 0.731 (i.e., by 0.8% and 0.5% increases) DSC for both tasks, respectively.     

Atlas-based prostate segmentation using an hybrid registration

Purpose  This paper presents the preliminary results of a semi-automatic method for prostate segmentation of magnetic resonance images (MRI) which aims to be incorporated in a navigation system for prostate brachytherapy. Methods  The method is based on the registration of an anatomical atlas computed from a population of 18 MRI exams onto a patient image. An hybrid registration framework which couples an intensity-based registration with a robust point-matching algorithm is used for both atlas building and atlas registration. Results  The method has been validated on the same dataset that the one used to construct the atlas using the leave-one-out method. Results gives a mean error of 3.39 mm and a standard deviation of 1.95 mm with respect to expert segmentations. Conclusions  We think that this segmentation tool may be a very valuable help to the clinician for routine quantitative image exploitation.     

Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images

The aim of deformable brain image registration is to align anatomical structures, which can potentially vary with large and complex deformations. Anatomical structures vary in size and shape, requiring the registration algorithm to estimate deformation fields at various degrees of complexity. Here, we present a difficulty-aware model based on an attention mechanism to automatically identify hard-to-register regions, allowing better estimation of large complex deformations. The difficulty-aware model is incorporated into a cascaded neural network consisting of three sub-networks to fully leverage both global and local contextual information for effective registration. The first sub-network is trained at the image level to predict a coarse-scale deformation field, which is then used for initializing the subsequent sub-network. The next two sub-networks progressively optimize at the patch level with different resolutions to predict a fine-scale deformation field. Embedding difficulty-aware learning into the hierarchical neural network allows harder patches to be identified in the deeper sub-networks at higher resolutions for refining the deformation field. Experiments conducted on four public datasets validate that our method achieves promising registration accuracy with better preservation of topology, compared with state-of-the-art registration methods.     

Rethinking medical image reconstruction via shape prior,going deeper and faster: Deep joint indirect registration and reconstruction

Indirect image registration is a promising technique to improve image reconstruction quality by providing a shape prior for the reconstruction task. In this paper, we propose a novel hybrid method that seeks to reconstruct high quality images from few measurements whilst requiring low computational cost. With this purpose, our framework intertwines indirect registration and reconstruction tasks is a single functional. It is based on two major novelties. Firstly, we introduce a model based on deep nets to solve the indirect registration problem, in which the inversion and registration mappings are recurrently connected through a fixed-point interaction based sparse optimisation. Secondly, we introduce specific inversion blocks, that use the explicit physical forward operator, to map the acquired measurements to the image reconstruction. We also introduce registration blocks based deep nets to predict the registration parameters and warp transformation accurately and efficiently. We demonstrate, through extensive numerical and visual experiments, that our framework outperforms significantly classic reconstruction schemes and other bi-task method; this in terms of both image quality and computational time. Finally, we show generalisation capabilities of our approach by demonstrating their performance on fast Magnetic Resonance Imaging (MRI), sparse view computed tomography (CT) and low dose CT with measurements much below the Nyquist limit.     

3D/2D Registration with superabundant vessel reconstruction for cardiac resynchronization therapy

A key component of image guided interventions is the registration of preoperative and intraoperative images. Classical registration approaches rely on cross-modality information; however, in modalities such as MRI and X-ray there may not be sufficient cross-modality information. This paper proposes a fundamentally different registration approach which uses adjacent anatomical structures with superabundant vessel reconstruction and dynamic outlier rejection. In the targeted clinical scenario of cardiac resynchronization therapy (CRT) delivery, preoperative, non contrast-enhanced, MRI is registered to intraoperative, contrasted X-ray fluoroscopy. The adjacent anatomical structures are the left ventricle (LV) from MRI and the coronary veins reconstructed from two contrast-enhanced X-ray images. The novel concept of superabundant vessel reconstruction is introduced to bypass the standard reconstruction problem of establishing one-to-one correspondences. Furthermore, a new dynamic outlier rejection method is proposed, to enable globally optimal point set registration. The proposed approach has been qualitatively and quantitatively evaluated on phantom, clinical CT angiography with ground truth and clinical CRT data. A novel evaluation method is proposed for clinical CRT data based on previously implanted artificial aortic and mitral valves. The registration accuracy in 3D was 2.94 mm for the aortic and 3.86 mm for the mitral valve. The results are below the required accuracy identified by clinical partners to be the half-segment size (16.35 mm) of a standard American Heart Association (AHA) 16 segment model of the LV.     

GRAM: A framework for geodesic registration on anatomical manifolds

Medical image registration is a challenging problem, especially when there is large anatomical variation in the anatomies. Geodesic registration methods have been proposed to solve the large deformation registration problem. However, analytically defined geodesic paths may not coincide with biologically plausible paths of registration, since the manifold of diffeomorphisms is immensely broader than the manifold spanned by diffeomorphisms between real anatomies. In this paper, we propose a novel framework for large deformation registration using the learned manifold of anatomical variation in the data. In this framework, a large deformation between two images is decomposed into a series of small deformations along the shortest path on an empirical manifold that represents anatomical variation. Using a manifold learning technique, the major variation of the data can be visualized by a low-dimensional embedding, and the optimal group template is chosen as the geodesic mean on the manifold. We demonstrate the advantages of the proposed framework over direct registration with both simulated and real databases of brain images.     

Registration, atlas estimation and variability analysis of white matter fiber bundles modeled as currents

This paper proposes a generic framework for the registration, the template estimation and the variability analysis of white matter fiber bundles extracted from diffusion images. This framework is based on the metric on currents for the comparison of fiber bundles. This metric measures anatomical differences between fiber bundles, seen as global homologous structures across subjects. It avoids the need to establish correspondences between points or between individual fibers of different bundles. It can measure differences both in terms of the geometry of the bundles (like its boundaries) and in terms of the density of fibers within the bundle. It is robust to fiber interruptions and reconnections. In addition, a recently introduced sparse approximation algorithm allows us to give an interpretable representation of the fiber bundles and their variations in the framework of currents. First, we used this metric to drive the registration between two sets of homologous fiber bundles of two different subjects. A dense deformation of the underlying white matter is estimated, which is constrained by the bundles seen as global anatomical landmarks. By contrast, the alignment obtained from image registration is driven only by the local gradient of the image. Second, we propose a generative statistical model for the analysis of a collection of homologous bundles. This model consistently estimates prototype fiber bundles (called template), which capture the anatomical invariants in the population, a set of deformations, which align the geometry of the template to that of each subject and a set of residual perturbations. The statistical analysis of both the deformations and the residuals describe the anatomical variability in terms of geometry (stretching, torque, etc.) and "texture" (fiber density, etc.). Third, this statistical modeling allows us to simulate new synthetic bundles according to the estimated variability. This gives a way to interpret the anatomical features that the model detects consistently across the subjects. This may be used to better understand the bias introduced by the fiber extraction methods and eventually to give anatomical characterization of the normal or pathological variability of fiber bundles.     

Reducing inter-subject anatomical variation: Effect of normalization method on sensitivity of functional magnetic resonance imaging data analysis in auditory cortex and the superior temporal region

Conventional group analysis of functional MRI (fMRI) data usually involves spatial alignment of anatomy across participants by registering every brain image to an anatomical reference image. Due to the high degree of inter-subject anatomical variability, a low-resolution average anatomical model is typically used as the target template, and/or smoothing kernels are applied to the fMRI data to increase the overlap among subjects' image data. However, such smoothing can make it difficult to resolve small regions such as subregions of auditory cortex when anatomical morphology varies among subjects. Here, we use data from an auditory fMRI study to show that using a high-dimensional registration technique (HAMMER) results in an enhanced functional signal-to-noise ratio (fSNR) for functional data analysis within auditory regions, with more localized activation patterns. The technique is validated against DARTEL, a high-dimensional diffeomorphic registration, as well as against commonly used low-dimensional normalization techniques such as the techniques provided with SPM2 (cosine basis functions) and SPM5 (unified segmentation) software packages. We also systematically examine how spatial resolution of the template image and spatial smoothing of the functional data affect the results. Only the high-dimensional technique (HAMMER) appears to be able to capitalize on the excellent anatomical resolution of a single-subject reference template, and, as expected, smoothing increased fSNR, but at the cost of spatial resolution. In general, results demonstrate significant improvement in fSNR using HAMMER compared to analysis after normalization using DARTEL, or conventional normalization such as cosine basis function and unified segmentation in SPM, with more precisely localized activation foci, at least for activation in the region of auditory cortex.