Automatic registration using implicit shape representations: applications in intraoperative 3D rotational angiography to preoperative CTA registration

Purpose  A solution for automatic registration of 3D Rotational Angiography (XA) to CT/MR of the liver. Targeted for use in treatment planning of liver interventions. Methods  A shape-based approach to registration is proposed that does not require specification of landmarks nor is it prone to local minima like purely intensity-based registration methods. Through the use of vessel characteristics, accurate registration is possible even in the presence of deformations induced by catheters and respiratory motion. Results  Registration was performed on eight pairs of multiphase CT angiography and 3D rotational digital angiography datasets. Quantitative validation of the registration accuracy using vessel landmarks was performed on these datasets. The validation study showed that the method has a registration error of 9.41  ±  4.13 mm. In addition, the computation time is well below 60 s making it attractive for clinical application. Conclusion  A new method for fully automatic 3DXA to CT/MR image registration was developed and found to be efficient and accurate using clinically realistic datasets.     

Predict brain MR image registration via sparse learning of appearance and transformation

We propose a new approach to register the subject image with the template by leveraging a set of intermediate images that are pre-aligned to the template. We argue that, if points in the subject and the intermediate images share similar local appearances, they may have common correspondence in the template. In this way, we learn the sparse representation of a certain subject point to reveal several similar candidate points in the intermediate images. Each selected intermediate candidate can bridge the correspondence from the subject point to the template space, thus predicting the transformation associated with the subject point at the confidence level that relates to the learned sparse coefficient. Following this strategy, we first predict transformations at selected key points, and retain multiple predictions on each key point, instead of allowing only a single correspondence. Then, by utilizing all key points and their predictions with varying confidences, we adaptively reconstruct the dense transformation field that warps the subject to the template. We further embed the prediction–reconstruction protocol above into a multi-resolution hierarchy. In the final, we refine our estimated transformation field via existing registration method in effective manners. We apply our method to registering brain MR images, and conclude that the proposed framework is competent to improve registration performances substantially.     

Interventional digital tomosynthesis from a standard fluoroscopy system using 2D-3D registration

Interventional fluoroscopy provides guidance in a variety of minimally invasive procedures. However, three-dimensional (3D) clinically relevant information is projected onto a two-dimensional (2D) image which can make image interpretation difficult. Moreover, vasculature visualisation requires the use of iodinated contrast media which is nephrotoxic and is the primary cause of renal complications. In this article, we demonstrate how digital tomosynthesis slices can be produced on standard fluoroscopy equipment by registering the preoperative CT volume and the intraoperative fluoroscopy images using 2D-3D image registration. The proposed method automatically reconstructs patient-anatomy-specific slices and removes clutter resulting from bony anatomy. Such slices could provide additional intraoperative information which cannot be provided by the preoperative CT volume alone, such as the deformed aorta position offering improved guidance precision. Image acquisition would fit with interventional clinical work-flow and would not require a high X-ray dose. Experiments are carried out using one phantom and four clinical datasets. Phantom results showed a 3351% contrast-to-noise improvement compared to standard fluoroscopy. Patient results showed our method enabled visualization of clinically relevant features: outline of the aorta, the aortic bifurcation and some aortic calcifications.     

3D image segmentation by using statistical deformation models and level sets

Purpose Improved segmentation of soft objects was sought using a new method that combines level set segmentation with statistical deformation models, using prior knowledge of the shape of an object as well as information derived from the input image. Methods Statistical deformation models were created using Euclidian distance functions of binary data and a multi-hierarchical registration approach based on mutual information metric and demons deformable registration. This approach is motivated by the fact that models based on signed distance maps, traditionally combined with level set segmentation can result in irregular shapes and do not establish explicit correspondences. By using statistical deformation models as representation of shape and a maximum a posteriori (MAP) estimation model to estimate the MAP shape of the object to be segmented, a robust segmentation algorithm using accurate shape models could be developed. Results The accuracy and correctness of the synthesized models was evaluated on different 3D objects (cardiac MRI and spinal CT vertebral segment) and the segmentation algorithm was validated by performing different segmentation tasks using various image modalities. The results of this evaluation are very promising and show the potential utility of the approach. Conclusion Initial results demonstrate the approach is feasible and may be advantageous over alternative segmentation methods. Extensions of the model, which also incorporate prior knowledge about the spatial distribution of grey values, are currently under development.     

Automatic multimodal 2D/3D breast image registration using biomechanical FEM models and intensity-based optimization

Due to their different physical origin, X-ray mammography and Magnetic Resonance Imaging (MRI) provide complementary diagnostic information. However, the correlation of their images is challenging due to differences in dimensionality, patient positioning and compression state of the breast. Our automated registration takes over part of the correlation task. The registration method is based on a biomechanical finite element model, which is used to simulate mammographic compression. The deformed MRI volume can be compared directly with the corresponding mammogram. The registration accuracy is determined by a number of patient-specific parameters. We optimize these parameters – e.g. breast rotation – using image similarity measures. The method was evaluated on 79 datasets from clinical routine. The mean target registration error was 13.2 mm in a fully automated setting. On basis of our results, we conclude that a completely automated registration of volume images with 2D mammograms is feasible. The registration accuracy is within the clinically relevant range and thus beneficial for multimodal diagnosis.     

Segmentation and quantification of the aortic arch using joint 3D model-based segmentation and elastic image registration

Accurate quantification of the morphology of vessels is important for diagnosis and treatment of cardiovascular diseases. We introduce a new joint segmentation and registration approach for the quantification of the aortic arch morphology that combines 3D model-based segmentation with elastic image registration. With this combination, the approach benefits from the robustness of model-based segmentation and the accuracy of elastic registration. The approach can cope with a large spectrum of vessel shapes and particularly with pathological shapes that deviate significantly from the underlying model used for segmentation. The performance of the approach has been evaluated on the basis of 3D synthetic images, 3D phantom data, and clinical 3D CTA images including pathologies. We also performed a quantitative comparison with previous approaches.     

Anatomically constrained and attention-guided deep feature fusion for joint segmentation and deformable medical image registration

The main objective of anatomically plausible results for deformable image registration is to improve model’s registration accuracy by minimizing the difference between a pair of fixed and moving images. Since many anatomical features are closely related to each other, leveraging supervision from auxiliary tasks (such as supervised anatomical segmentation) has the potential to enhance the realism of the warped images after registration. In this work, we employ a Multi-Task Learning framework to formulate registration and segmentation as a joint issue, in which we utilize anatomical constraint from auxiliary supervised segmentation to enhance the realism of the predicted images. First, we propose a Cross-Task Attention Block to fuse the high-level feature from both the registration and segmentation network. With the help of initial anatomical segmentation, the registration network can benefit from learning the task-shared feature correlation and rapidly focusing on the parts that need deformation. On the other hand, the anatomical segmentation discrepancy from ground-truth fixed annotations and predicted segmentation maps of initial warped images are integrated into the loss function to guide the convergence of the registration network. Ideally, a good deformation field should be able to minimize the loss function of registration and segmentation. The voxel-wise anatomical constraint inferred from segmentation helps the registration network to reach a global optimum for both deformable and segmentation learning. Both networks can be employed independently during the testing phase, enabling only the registration output to be predicted when the segmentation labels are unavailable. Qualitative and quantitative results indicate that our proposed methodology significantly outperforms the previous state-of-the-art approaches on inter-patient brain MRI registration and pre- and intra-operative uterus MRI registration tasks within our specific experimental setup, which leads to state-of-the-art registration quality scores of 0.755 and 0.731 (i.e., by 0.8% and 0.5% increases) DSC for both tasks, respectively.     

Intensity-based registration of freehand 3D ultrasound and CT-scan images of the kidney

Objectives This paper presents a method to register a pre-operative computed-tomography (CT) volume to a sparse set of intra-operative ultra-sound (US) slices. In the context of percutaneous renal puncture, the aim is to transfer planning information to an intra-operative coordinate system. Materials and methods The spatial position of the US slices is measured by optically localizing a calibrated probe. Assuming the reproducibility of kidney motion during breathing, and no deformation of the organ, the method consists in optimizing a rigid 6 degree of freedom transform by evaluating at each step the similarity between the set of US images and the CT volume. The correlation between CT and US images being naturally rather poor, the images were preprocessed in order to increase their similarity. Among the similarity measures formerly studied in the context of medical image registration, correlation ratio turned out to be one of the most accurate and appropriate, particularly with the chosen non-derivative minimization scheme, namely Powell-Brent’s. The resulting matching transforms are compared to a standard rigid surface registration involving segmentation, regarding both accuracy and repeatability. Results The obtained results are presented and discussed.     

Efficient multi-modal dense field non-rigid registration: alignment of histological and section images

We describe a new algorithm for non-rigid registration capable of estimating a constrained dense displacement field from multi-modal image data. We applied this algorithm to capture non-rigid deformation between digital images of histological slides and digital flat-bed scanned images of cryotomed sections of the larynx, and carried out validation experiments to measure the effectiveness of the algorithm. The implementation was carried out by extending the open-source Insight ToolKit software. In diagnostic imaging of cancer of the larynx, imaging modalities sensitive to both anatomy (such as MRI and CT) and function (PET) are valuable. However, these modalities differ in their capability to discriminate the margins of tumor. Gold standard tumor margins can be obtained from histological images from cryotomed sections of the larynx. Unfortunately, the process of freezing, fixation, cryotoming and staining the tissue to create histological images introduces non-rigid deformations and significant contrast changes. We demonstrate that the non-rigid registration algorithm we present is able to capture these deformations and the algorithm allows us to align histological images with scanned images of the larynx. Our non-rigid registration algorithm constructs a deformation field to warp one image onto another. The algorithm measures image similarity using a mutual information similarity criterion, and avoids spurious deformations due to noise by constraining the estimated deformation field with a linear elastic regularization term. The finite element method is used to represent the deformation field, and our implementation enables us to assign inhomogeneous material characteristics so that hard regions resist internal deformation whereas soft regions are more pliant. A gradient descent optimization strategy is used and this has enabled rapid and accurate convergence to the desired estimate of the deformation field. A further acceleration in speed without cost of accuracy is achieved by using an adaptive mesh refinement strategy.     

可变形图像配准的CT与MRI对肝癌体积测定的前瞻性比较

目的  通过比较MRI与CT对肝癌肿瘤的诊断情况,进而进行高精度放疗计划。方法  选择2020年7月~2022年7月收治的26例不可切除的肝转移(n=8)、肝细胞癌(n=10)和胆管癌(n=8)患者作为研究对象,患者在放疗计划时进行了具有诊断质量的MRI扫描和三期CT扫描,并确定了肝内解剖参考点。在最能显示肿瘤的CT和MRI系列中,勾画了肝脏和肿瘤体积,确定了肝内解剖参考点。采用形变配准对CT和MRI的肝脏进行配准。结果  5例肝癌CT病灶数量与MRI有差异,MRI病灶多3例,CT病灶多2例。肝脏变形配准后,CT肿瘤表面与MRI肿瘤表面平均距离的人群中位数为3.7（2.2~21.3）mm。肿瘤表面积相差5 mm的中位百分比为26%(38%~86%)。转移瘤的中位符合率为81%(77%~86%),肝细胞癌的一致性为78%（44%~86%）,胆管癌的一致性为69%（25%~85%）。结论  MRI诊断的肝癌肿瘤体积与CT诊断的肝癌肿瘤体积存在显著差异,且在原发性肝癌中更为常见。     

AMNet: Adaptive multi-level network for deformable registration of 3D brain MR images

Three-dimensional (3D) deformable image registration is a fundamental technique in medical image analysis tasks. Although it has been extensively investigated, current deep-learning-based registration models may face the challenges posed by deformations with various degrees of complexity. This paper proposes an adaptive multi-level registration network (AMNet) to retain the continuity of the deformation field and to achieve high-performance registration for 3D brain MR images. First, we design a lightweight registration network with an adaptive growth strategy to learn deformation field from multi-level wavelet sub-bands, which facilitates both global and local optimization and achieves registration with high performance. Second, our AMNet is designed for image-wise registration, which adapts the local importance of a region in accordance with the complexity degrees of its deformation, and thereafter improves the registration efficiency and maintains the continuity of the deformation field. Experimental results from five publicly-available brain MR datasets and a synthetic brain MR dataset show that our method achieves superior performance against state-of-the-art medical image registration approaches.     

Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information

Maximization of mutual information of voxel intensities has been demonstrated to be a very powerful criterion for three-dimensional medical image registration, allowing robust and accurate fully automated affine registration of multimodal images in a variety of applications, without the need for segmentation or other preprocessing of the images. In this paper, we investigate the performance of various optimization methods and multiresolution strategies for maximization of mutual information, aiming at increasing registration speed when matching large high-resolution images. We show that mutual information is a continuous function of the affine registration parameters when appropriate interpolation is used and we derive analytic expressions of its derivatives that allow numerically exact evaluation of its gradient. Various multiresolution gradient- and non-gradient-based optimization strategies, such as Powell, simplex, steepest-descent, conjugate-gradient, quasi-Newton and Levenberg-Marquardt methods, are evaluated for registration of computed tomography (CT) and magnetic resonance images of the brain. Speed-ups of a factor of 3 on average compared to Powell's method at full resolution are achieved with similar precision and without a loss of robustness with the simplex, conjugate-gradient and Levenberg-Marquardt method using a two-level multiresolution scheme. Large data sets such as 256(2) x 128 MR and 512(2) x 48 CT images can be registered with subvoxel precision in <5 min CPU time on current workstations.     

Automatic 3D reconstruction of electrophysiology catheters from two-view monoplane C-arm image sequences

Purpose

Catheter guidance is a vital task for the success of electrophysiology interventions. It is usually provided through fluoroscopic images that are taken intra-operatively. The cardiologists, who are typically equipped with C-arm systems, scan the patient from multiple views rotating the fluoroscope around one of its axes. The resulting sequences allow the cardiologists to build a mental model of the 3D position of the catheters and interest points from the multiple views.

Method

We describe and compare different 3D catheter reconstruction strategies and ultimately propose a novel and robust method for the automatic reconstruction of 3D catheters in non-synchronized fluoroscopic sequences. This approach does not purely rely on triangulation but incorporates prior knowledge about the catheters. In conjunction with an automatic detection method, we demonstrate the performance of our method compared to ground truth annotations.

Results

In our experiments that include 20 biplane datasets, we achieve an average reprojection error of 0.43 mm and an average reconstruction error of 0.67 mm compared to gold standard annotation.

Conclusions

In clinical practice, catheters suffer from complex motion due to the combined effect of heartbeat and respiratory motion. As a result, any 3D reconstruction algorithm via triangulation is imprecise. We have proposed a new method that is fully automatic and highly accurate to reconstruct catheters in three dimensions.

     

3D非刚性图像运动校正在肝DCE-MRI定量成像中的应用

目的探讨3D非刚性图像运动校正在提高肝脏DCE-MRI图像质量和定量参数准确性的应用价值。方法 35例确诊肝细胞癌患者进行自由呼吸下全肝MRI灌注扫描,使用3D对比增强非刚性配准对多期动态增强扫描MRI图像进行图像运动配准,利用肝脏双血供模型和腹主动脉、门静脉时间-浓度曲线作为血管输入函数(VIF),分别获得肝癌、肝实质图像运动校正前后灌注参数(K~(trans)、K_(ep)、Ve、Vp、HPI)。比较图像运动校正前后各组定量参数差异。对运动校正前后图像质量分别进行客观和主观评价。测量病灶最大面积,比较原始图像上校正前后病灶面积的差异。结果使用3D图像运动校正后腹主动脉、门静脉时间-浓度曲线波动幅度缩小,平滑度更好。图像运动校正后肝癌K~(trans)、K_(ep)、Ve、Vp、HPI值均大于校正前,K~(trans)、Ve、Vp,值差异存在统计学差异(P0.05),K_(ep)值两者间无统计学差异(P0.05)。图像运动校正后肝实质K~(trans)、K_(ep)、Ve、Vp、HPI值均大于校正前,K~(trans)、Ve、Vp值差异存在统计学差异(P0.05);K_(ep)值两者间无统计学差异(P0.05);图像运动校正前后HPI值相近,且无统计学差异(P0.05)。肝癌在原始图像上面积平均值运动校正前大于运动校正后,但无统计学意义(P0.05)。校正后肝组织噪声低于矫正前,无统计学意义(P0.05),校正后图像主观评分明显高于矫正前,差异存在统计学意义(P0.05)。结论 3D非刚性运动校正能减少肝脏MRI多期动态增强图像运动伪影,有助于提高灌注定量参数的准确性。     

Comparison of edge-based and ridge-based registration of CT and MR brain images

In modern medicine, several different imaging techniques are frequently employed in the study of a single patient. This is useful, since different images show complementary information on the functionality and/or structure of the anatomy examined. This very difference between modalities, however, complicates the problem of proper registration of the images involved, and rules out the most basic approaches—like direct grey value correlation—to achieve registration. The observation that some common structures will always exist is supportive of the statement that registration may be feasible using edges or ridges present in the images. The existence of such structures defined in the binary sense is questionable, however, and their extraction from images requires a segmentation by definition. In this paper we propose to use fuzzy edgeness and ridgeness images, thus avoiding the need for segmentation and using more of the available information from the original images. We will show that such fuzzy images can be used to achieve accurate registration. Several ridgeness and edgeness computing operators were compared. The best registration results were obtained using a gradient magnitude operator.     

3D spatial compounding of ultrasound images using image-based nonrigid registration

Medical ultrasound images are often distorted enough to significantly limit resolution during compounding (i.e., summation of images from multiple views). A new, volumetric image registration technique has been used successfully to enable high spatial resolution in three-dimensional (3D) spatial compounding of ultrasound images. Volumetric ultrasound data were acquired by scanning a linear matrix array probe in the elevational direction in a focal lesion phantom and in a breast in vivo. To obtain partly uncorrelated views, the volume of interest was scanned at five different transducer tilt angles separated by 4° to 6°. Pairs of separate views were registered by an automatic procedure based on a mutual information metric, using global full affine and thin-plate spline warping transformations. Registration accuracy was analyzed automatically in the phantom data, and manually in vivo, yielding average registration errors of 0.31 mm and 0.65 mm, respectively. In the vicinity of the warping control points, registrations obtained with warping transformations were significantly more accurate than full affine registrations. Compounded images displayed the expected reduction in speckle noise and increase in contrast-to-noise ratio (CNR), as well as better delineation of connective tissues and reduced shadowing. Compounding also revealed some apparent low contrast lobulations that were not visible in the single-sweep images. Given expected algorithmic and hardware enhancements, nonrigid, image-based registration shows great promise for reducing tissue motion and refraction artifacts in 3D spatial compounding.     

Novel methods for 3D postoperative analysis of total knee arthroplasty using 2D-3D image registration

Background

Radiographic evaluation of patients after total knee arthroplasty is an important tool for assessing the correct position of the implants and the axis of limb alignment because this will determine long-term durability of the implants. Recently, 2D–3D medical image registration methods are developed for 3D postoperative analysis of total knee arthroplasty. However, most of these techniques have focused only on knee implants.

Methods

A 2D–3D medical image registration is implemented to compute the 3D positions of not only implants but also lower limb bones. The following 3D postoperative analysis methods for total knee arthroplasty are presented in this paper: (1) automatic calculation of relative angles of implants and bones, (2) assessment of external rotation angles of inserted implants, and (3) measurement and comparison of both flexion–extension gap balances. Finally these methods have been applied in five patients who underwent total knee replacements.

Findings

A practical method that can evaluate the patient's knee conditions has been successfully developed. The repeatability and accuracy of 2D–3D registration were around 0.2 mm as obtained from the tests using model bones. Based on the 3D information, the novel methods of postoperative analysis were proposed and successfully applied to the patients.

Interpretation

The 3D positions for both knee implants and lower limb bones can be calculated in order to perform comprehensive postoperative analyses of total knee arthroplasty. The proposed analyses of the postoperative evaluations facilitated various 3D evaluations of the status of implants, alignment of lower limb and gap balances which were not previously feasible.     

Polyrigid and polyaffine transformations: A novel geometrical tool to deal with non-rigid deformations – Application to the registration of histological slices

We describe in this paper a novel kind of geometrical transformations, named polyrigid and polyaffine. These transformations efficiently code for locally rigid or affine deformations with a small number of intuitive parameters. They can describe compactly large rigid or affine movements, unlike most free-form deformation classes. Very flexible, this tool can be readily adapted to a large variety of situations, simply by tuning the number of rigid or affine components and the number of parameters describing their regions of influence.The displacement of each spatial position is defined by a continuous trajectory that follows a differential equation which averages the influence of each rigid or affine component. We show that the resulting transformations are diffeomorphisms, smooth with respect to their parameters. We devise a new and flexible numerical scheme to allow a trade-off between computational efficiency and closeness to the ideal diffeomorphism. Our algorithms are implemented within the Insight Toolkit, whose generic programming style offers rich facilities for prototyping. In this context, we derive an effective optimization strategy of the transformations which demonstrates that this new tool is highly suitable for inference.The whole framework is exemplified successfully with the registration of histological slices. This choice is challenging, because these data often present locally rigid deformations added during their acquisition, and can also present a loss of matter, which makes their registration even more difficult.Powerful and flexible, this new tool opens up large perspectives, in non-rigid 3D rigid registration as well as in shape statistics.     

腹腔镜下肝脏手术增强现实三维影像导航平台的构建与应用

目的 实现腹腔镜下肝脏手术增强现实三维影像实时导航方案，构建平台雏形并评估其运行效果。方法 通过编写术前CT影像三维重建自动化算法、三维模型二维平面投影轮廓点集采集算法、基于人工智能的腹腔镜手术视野肝脏轮廓点集识别采集算法、“一对多”匹配算法、坐标系转换算法以及视频实时渲染增强融合算法，组成腹腔镜下肝脏手术增强现实三维影像实时导航软件，配合光学定位系统硬件和双目光学信息采集硬件，构建出导航平台雏形。在实验室仿真模型和大型动物中进行平台试运行，引入配准误差参数，评估运行效果并进行优化调整。结果 腹腔镜下肝脏手术导航软件基本实现：三维模型重建（CT图像自动分割和异色掩膜处理），手术室观测坐标系、腹腔镜视角坐标系和三维重建模型坐标系的建立及三者之间的信息转换，三维重建模型与手术视频所见结构实时配准导航，导航平台雏形构建。实验室仿真模型中的试运行配准误差为（4.1±0.4）mm，大型动物试运行2次的配准误差分别为4.6 mm和5.8 mm。结论 通过腹腔镜下肝脏手术增强现实三维影像实时导航平台雏形的设计、研发、构建以及使用，导航配准精度已经基本可以达到临床需求，未来进一步优化调整之后，有望广泛...