Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.     

Validation of 3D multimodality roadmapping in interventional neuroradiology

Three-dimensional multimodality roadmapping is entering clinical routine utilization for neuro-vascular treatment. Its purpose is to navigate intra-arterial and intra-venous endovascular devices through complex vascular anatomy by fusing pre-operative computed tomography (CT) or magnetic resonance (MR) with the live fluoroscopy image. The fused image presents the real-time position of the intra-vascular devices together with the patient's 3D vascular morphology and its soft-tissue context. This paper investigates the effectiveness, accuracy, robustness and computation times of the described methods in order to assess their suitability for the intended clinical purpose: accurate interventional navigation. The mutual information-based 3D-3D registration proved to be of sub-voxel accuracy and yielded an average registration error of 0.515 mm and the live machine-based 2D-3D registration delivered an average error of less than 0.2 mm. The capture range of the image-based 3D-3D registration was investigated to characterize its robustness, and yielded an extent of 35 mm and 25° for >80% of the datasets for registration of 3D rotational angiography (3DRA) with CT, and 15 mm and 20° for >80% of the datasets for registration of 3DRA with MR data. The image-based 3D-3D registration could be computed within 8 s, while applying the machine-based 2D-3D registration only took 1.5 μs, which makes them very suitable for interventional use.     

An integrated platform for image-guided cardiac resynchronization therapy

Cardiac resynchronization therapy (CRT) is an effective procedure for patients with heart failure but 30% of patients do not respond. This may be due to sub-optimal placement of the left ventricular (LV) lead. It is hypothesized that the use of cardiac anatomy, myocardial scar distribution and dyssynchrony information, derived from cardiac magnetic resonance imaging (MRI), may improve outcome by guiding the physician for optimal LV lead positioning. Whole heart MR data can be processed to yield detailed anatomical models including the coronary veins. Cine MR data can be used to measure the motion of the LV to determine which regions are late-activating. Finally, delayed Gadolinium enhancement imaging can be used to detect regions of scarring. This paper presents a complete platform for the guidance of CRT using pre-procedural MR data combined with live x-ray fluoroscopy. The platform was used for 21 patients undergoing CRT in a standard catheterization laboratory. The patients underwent cardiac MRI prior to their procedure. For each patient, a MRI-derived cardiac model, showing the LV lead targets, was registered to x-ray fluoroscopy using multiple views of a catheter looped in the right atrium. Registration was maintained throughout the procedure by a combination of C-arm/x-ray table tracking and respiratory motion compensation. Validation of the registration between the three-dimensional (3D) roadmap and the 2D x-ray images was performed using balloon occlusion coronary venograms. A 2D registration error of 1.2 ± 0.7?mm was achieved. In addition, a novel navigation technique was developed, called Cardiac Unfold, where an entire cardiac chamber is unfolded from 3D to 2D along with all relevant anatomical and functional information and coupled to real-time device detection. This allowed more intuitive navigation as the entire 3D scene was displayed simultaneously on a 2D plot. The accuracy of the unfold navigation was assessed off-line using 13 patient data sets by computing the registration error of the LV pacing lead electrodes which was found to be 2.2 ± 0.9?mm. Furthermore, the use of Unfold Navigation was demonstrated in real-time for four clinical cases.     

Quantification of the accuracy of MRI generated 3D models of long bones compared to CT generated 3D models

Orthopaedic fracture fixation implants are increasingly being designed using accurate 3D models of long bones based on computer tomography (CT). Unlike CT, magnetic resonance imaging (MRI) does not involve ionising radiation and is therefore a desirable alternative to CT. This study aims to quantify the accuracy of MRI-based 3D models compared to CT-based 3D models of long bones. The femora of five intact cadaver ovine limbs were scanned using a 1.5 T MRI and a CT scanner. Image segmentation of CT and MRI data was performed using a multi-threshold segmentation method. Reference models were generated by digitising the bone surfaces free of soft tissue with a mechanical contact scanner. The MRI- and CT-derived models were validated against the reference models. The results demonstrated that the CT-based models contained an average error of 0.15 mm while the MRI-based models contained an average error of 0.23 mm. Statistical validation shows that there are no significant differences between 3D models based on CT and MRI data. These results indicate that the geometric accuracy of MRI based 3D models was comparable to that of CT-based models and therefore MRI is a potential alternative to CT for generation of 3D models with high geometric accuracy.     

Effects of x-ray and CT image enhancements on the robustness and accuracy of a rigid 3D/2D image registration

A rigid body three-dimensional/two-dimensional (3D/2D) registration method has been implemented using mutual information, gradient ascent, and 3D texturemap-based digitally reconstructed radiographs. Nine combinations of commonly used x-ray and computed tomography (CT) image enhancement methods, including window leveling, histogram equalization, and adaptive histogram equalization, were examined to assess their effects on accuracy and robustness of the registration method. From a set of experiments using an anthropomorphic chest phantom, we were able to draw several conclusions. First, the CT and x-ray preprocessing combination with the widest attraction range was the one that linearly stretched the histograms onto the entire display range on both CT and x-ray images. The average attraction ranges of this combination were 71.3 mm and 61.3 deg in the translation and rotation dimensions, respectively, and the average errors were 0.12 deg and 0.47 mm. Second, the combination of the CT image with tissue and bone information and the x-ray images with adaptive histogram equalization also showed subvoxel accuracy, especially the best in the translation dimensions. However, its attraction ranges were the smallest among the examined combinations (on average 36 mm and 19 deg). Last the bone-only information on the CT image did not show convergency property to the correct registration.     

A 3D MRI sequence for computer assisted surgery of the lumbar spine

The aim of this research was to develop a magnetic resonance (MR) sequence capable of producing images suitable for use with computer assisted surgery (CAS) of the lumbar spine. These images needed good tissue contrast between bone and soft tissue to allow for image segmentation and generation of a 3D-surface model of the bone for surface registration. A 3D double echo fast gradient echo sequence was designed. Images were filtered for noise and non-uniformity and combined into a single data set. Registration experiments were carried out to directly compare segmentation of MR and computed tomography (CT) images using a physical model of a spine. These experiments showed the MR data produced adequate surface registration in 90% of the experiments compared to 100% with CT data. The MR images acquired using the sequence and processing described in this article are suitable to be used with CAS of the spine.     

Multi-feature-based plaque characterization in ex vivo MRI trained by registration to 3D histology

We present a new method for automated characterization of atherosclerotic plaque composition in ex vivo MRI. It uses MRI intensities as well as four other types of features: smoothed, gradient magnitude and Laplacian images at several scales, and the distances to the lumen and outer vessel wall. The ground truth for fibrous, necrotic and calcified tissue was provided by histology and μCT in 12 carotid plaque specimens. Semi-automatic registration of a 3D stack of histological slices and μCT images to MRI allowed for 3D rotations and in-plane deformations of histology. By basing voxelwise classification on different combinations of features, we evaluated their relative importance. To establish whether training by 3D registration yields different results than training by 2D registration, we determined plaque composition using (1) a 2D slice-based registration approach for three manually selected MRI and histology slices per specimen, and (2) an approach that uses only the three corresponding MRI slices from the 3D-registered volumes. Voxelwise classification accuracy was best when all features were used (73.3 ± 6.3%) and was significantly better than when only original intensities and distance features were used (Friedman, p < 0.05). Although 2D registration or selection of three slices from the 3D set slightly decreased accuracy, these differences were non-significant.     

Accuracy of device-specific 2D and 3D image distortion correction algorithms for magnetic resonance imaging of the head provided by a manufacturer

For the application of magnetic resonance imaging (MRI) in precision radiotherapy, image distortions must be reduced to a minimum to maintain geometrical accuracy. Recently, two-dimensional (2D) and three-dimensional (3D) algorithms for MRI-device-specific distortion corrections were developed by the manufacturers of MRI devices. A previously developed phantom (Karger C P et al 2003 Phys. Med. Biol. 48 211-21) was used to quantify and assess the size of geometrical image distortions before and after application of the 2D and 3D correction algorithm in the head region. Four different types of MRI devices with different gradient systems were measured. For comparison, measurements were also performed with two computed tomography (CT) devices. Mean distortions of up to 4.6+/-1.4 mm (maximum: 5.8 mm) were found prior to the correction. After the correction, the mean distortions were well below 2.0 mm in most cases. Distortions in the CT images were below or equal to 1.0 mm on average. Generally, the 3D algorithm produced comparable or better results than the 2D algorithm. The remaining distortions after the correction appear to be acceptable for fractionated radiotherapy.     

基于互信息的2D-3D医学图像配准

介绍一种基于互信息的2D-3D医学图像配准方法，将此方法用于X线透射图(由DRR模拟)与CT体数据的配准，对具体实现方法进行了探讨，用数据采样和灰度级压缩的方法对体数据进行预处理，通过改变平行光入射角度，计算不同方向的DRR图像，用互信息方法配准DRR图像，得到了较好的2D-3D图像配准效果。     

基于MRI和CT图像配准的颅内电极定位方法

目的应用多模医学图像配准,在颅内电极埋置术后对颅内电极进行精确定位。方法通过对颅内电极埋置前的头颅MRI图像和埋置后头颅CT图像进行配准,利用医学影像配准与分割工具包(ITK),将颅内电极位置准确地定位在MRI图像上,以建立电极位置与大脑解剖结构的联系。结果经过对10组断层图像进行配准定位,差值图像显示匹配程度较好,专家目测融合效果较为精确。在普通PC机上,以笔者所采用的数据为例,设定优化器初始步长为1,松弛因子为0.6,最小步长为0.000 2,最大迭代次数为100,整个电极定位的操作过程时间不超过1 min。结论多模医学图像配准对颅内电极定位较为准确,为医生提供了更加直观和完善的信息。     

Modality-integrated magnetic catheter tracking for x-ray vascular interventions

A novel setup for the integration of a magnetic tracking system (MTS) into a catheter laboratory environment by attaching the field generator of the MTS to the C-arm of the x-ray system was investigated. The metal interference of the x-ray system with the MTS then becomes static and can be calibrated. The registration of a tracked catheter to 2D live x-ray images with high and well-defined accuracy is thus possible for arbitrary C-arm positions. Interference with imaging by blocking the field of view is avoided. Real-time registration methods to maintain the registration of the catheter also to static 2D or 3D images (roadmaps), regardless of the C-arm rotation during catheter tracking, were investigated. Residual registration errors of the tracked catheter with respect to the static roadmaps (2D and 3D) were reduced by using the vessel geometry and shape information. The algorithms potentially allow for motion correction (e.g. due to respiration). Using the shape-based correction algorithms the average registration accuracies to static roadmaps for different C-arm angulations and catheter positions were determined to be 3.3-4.2 mm. The magnetic registration of the C-arm was furthermore allowed to compute the imaging geometry (position of the C-arm) and to produce virtual angiographic preview images before contrast injection and x-ray dose application. Thus, optimal projection geometries and collimator settings for the target region can be chosen in an 'off line' fashion. The proposed MTS-supported navigation setup on both 3D (static) and 2D (live and static) roadmaps merges the high resolution and real-time feedback of 2D x-ray images with the navigation support from 3D static images.     

Mapping of the prostate in endorectal coil-based MRI/MRSI and CT: a deformable registration and validation study

The endorectal coil is being increasingly used in magnetic resonance imaging (MRI) and MR spectroscopic imaging (MRSI) to obtain anatomic and metabolic images of the prostate with high signal-to-noise ratio (SNR). In practice, however, the use of endorectal probe inevitably distorts the prostate and other soft tissue organs, making the analysis and the use of the acquired image data in treatment planning difficult. The purpose of this work is to develop a deformable image registration algorithm to map the MRI/MRSI information obtained using an endorectal probe onto CT images and to verify the accuracy of the registration by phantom and patient studies. A mapping procedure involved using a thin plate spline (TPS) transformation was implemented to establish voxel-to-voxel correspondence between a reference image and a floating image with deformation. An elastic phantom with a number of implanted fiducial markers was designed for the validation of the quality of the registration. Radiographic images of the phantom were obtained before and after a series of intentionally introduced distortions. After mapping the distorted phantom to the original one, the displacements of the implanted markers were measured with respect to their ideal positions and the mean error was calculated. In patient studies, CT images of three prostate patients were acquired, followed by 3 Tesla (3 T) MR images with a rigid endorectal coil. Registration quality was estimated by the centroid position displacement and image coincidence index (CI). Phantom and patient studies show that TPS-based registration has achieved significantly higher accuracy than the previously reported method based on a rigid-body transformation and scaling. The technique should be useful to map the MR spectroscopic dataset acquired with ER probe onto the treatment planning CT dataset to guide radiotherapy planning.     

Segmentation accuracy of long bones

The use of three-dimensional imaging methodologies in new applications in the orthopaedic field has introduced a need for high accuracy, in addition to a correct diagnosis. The aim of this study was to quantify the absolute dimensional errors between models reconstructed from computed tomography and magnetic resonance images compared to a ground truth for various regions of the bone. Clinical CT and MRI scans were acquired from nine lower leg cadavers and the bones were subsequently cleaned from soft tissues. 3D models of the tibia were created from the segmented CT and MRI images and compared to optical scans of the cleaned bones (considered as ground truth). The 3D reconstruction using CT images resulted in an RMS error of 0.55 mm, corresponding to an overestimated CT bone model compared to the cleaned bone. MR imaging resulted in an RMS error of 0.56 mm; however, the MRI bone model was on average a small underestimation of the cleaned bone. Different regions of the bones were analysed, indicating a difference in accuracy between diaphysis and epiphysis. This study demonstrates a high accuracy for both CT and MRI imaging, supporting the feasibility of using MRI technology for the 3D reconstruction of bones in medical applications.     

Three-Dimensional Cardiac Tissue Image Registration for Analysis of In Vivo Electrical Mapping

A method is presented for registering 3D cardiac tissue images to reference data, for the purpose of analyzing recorded electrical activity. Following left-ventricular in vivo electrical mapping studies in pig hearts, MRI is used to define a reference geometry in the tissue segment around the recording electrodes. The segment is then imaged in 3D using a high-resolution serial imaging microscopy technique. The tissue processing required for this introduces segment-wide distortion. Piecewise-smooth maps are used to correct the tissue distortion and register the 3D images with the reference MRI data. The methods are validated and techniques for identifying the preferred maps are proposed. Recorded electrical activation is shown to map reliably onto cardiac tissue structure using this registration method.     

4D-CT motion estimation using deformable image registration and 5D respiratory motion modeling

Four-dimensional computed tomography (4D-CT) imaging technology has been developed for radiation therapy to provide tumor and organ images at the different breathing phases. In this work, a procedure is proposed for estimating and modeling the respiratory motion field from acquired 4D-CT imaging data and predicting tissue motion at the different breathing phases. The 4D-CT image data consist of series of multislice CT volume segments acquired in ciné mode. A modified optical flow deformable image registration algorithm is used to compute the image motion from the CT segments to a common full volume 3D-CT reference. This reference volume is reconstructed using the acquired 4D-CT data at the end-of-exhalation phase. The segments are optimally aligned to the reference volume according to a proposed a priori alignment procedure. The registration is applied using a multigrid approach and a feature-preserving image downsampling maxfilter to achieve better computational speed and higher registration accuracy. The registration accuracy is about 1.1 +/- 0.8 mm for the lung region according to our verification using manually selected landmarks and artificially deformed CT volumes. The estimated motion fields are fitted to two 5D (spatial 3D+tidal volume+airflow rate) motion models: forward model and inverse model. The forward model predicts tissue movements and the inverse model predicts CT density changes as a function of tidal volume and airflow rate. A leave-one-out procedure is used to validate these motion models. The estimated modeling prediction errors are about 0.3 mm for the forward model and 0.4 mm for the inverse model.     

A Semi-automated Approach to Improve the Efficiency of Medical Imaging Segmentation for Haptic Rendering

The Sensimmer platform represents our ongoing research on simultaneous haptics and graphics rendering of 3D models. For simulation of medical and surgical procedures using Sensimmer, 3D models must be obtained from medical imaging data, such as magnetic resonance imaging (MRI) or computed tomography (CT). Image segmentation techniques are used to determine the anatomies of interest from the images. 3D models are obtained from segmentation and their triangle reduction is required for graphics and haptics rendering. This paper focuses on creating 3D models by automating the segmentation of CT images based on the pixel contrast for integrating the interface between Sensimmer and medical imaging devices, using the volumetric approach, Hough transform method, and manual centering method. Hence, automating the process has reduced the segmentation time by 56.35% while maintaining the same accuracy of the output at ±2 voxels.     

A frequency-based approach to locate common structure for 2D-3D intensity-based registration of setup images in prostate radiotherapy

In many radiotherapy clinics, geometric uncertainties in the delivery of 3D conformal radiation therapy and intensity modulated radiation therapy of the prostate are reduced by aligning the patient's bony anatomy in the planning 3D CT to corresponding bony anatomy in 2D portal images acquired before every treatment fraction. In this paper, we seek to determine if there is a frequency band within the portal images and the digitally reconstructed radiographs (DRRs) of the planning CT in which bony anatomy predominates over non-bony anatomy such that portal images and DRRs can be suitably filtered to achieve high registration accuracy in an automated 2D-3D single portal intensity-based registration framework. Two similarity measures, mutual information and the Pearson correlation coefficient were tested on carefully collected gold-standard data consisting of a kilovoltage cone-beam CT (CBCT) and megavoltage portal images in the anterior-posterior (AP) view of an anthropomorphic phantom acquired under clinical conditions at known poses, and on patient data. It was found that filtering the portal images and DRRs during the registration considerably improved registration performance. Without filtering, the registration did not always converge while with filtering it always converged to an accurate solution. For the pose-determination experiments conducted on the anthropomorphic phantom with the correlation coefficient, the mean (and standard deviation) of the absolute errors in recovering each of the six transformation parameters were Theta(x):0.18(0.19) degrees, Theta(y):0.04(0.04) degrees, Theta(z):0.04(0.02) degrees, t(x):0.14(0.15) mm, t(y):0.09(0.05) mm, and t(z):0.49(0.40) mm. The mutual information-based registration with filtered images also resulted in similarly small errors. For the patient data, visual inspection of the superimposed registered images showed that they were correctly aligned in all instances. The results presented in this paper suggest that robust and accurate registration can be achieved with intensity-based methods by focusing on rigid bony structures in the images while diminishing the influence of artifacts with similar frequencies as soft tissue.     

基于神经网络的双X射线影像2D-3D配准算法

针对基于迭代优化的传统2D-3D医学图像配准算法运行速度慢，难以达到实时配准的要求，本研究提出一种实时2D-3D配准方法。通过将空间刚体变换参数分解到两个平面上，将2D-3D配准简化为两个步骤，包含2D-2D近似刚体配准与单参数2D-3D刚体配准。同时利用深度卷积神经网络拟合患者X射线影像残差与其对应姿态差异间的非线性映射关系，从X-DRR图像对的残差回归出空间刚体变换参数。经由头颅CT数据训练后的网络，在0.04 s内完成了高精度的双X射线配准。本研究提出的配准方法满足了放疗过程中进行实时2D-3D配准工作的要求。     

Automated 2D-3D registration of portal images and CT data using line-segment enhancement

In prostate radiotherapy, setup errors with respect to the patient's bony anatomy can be reduced by aligning 2D megavoltage (MV) portal images acquired during treatment to a reference 3D kilovoltage (kV) CT acquired for treatment planning purposes. The purpose of this study was to evaluate a fully automated 2D-3D registration algorithm to quantify setup errors in 3D through the alignment of line-enhanced portal images and digitally reconstructed radiographs computed from the CT. The line-enhanced images were obtained by correlating the images with a filter bank of short line segments, or "sticks" at different orientations. The proposed methods were validated on (1) accurately collected gold-standard data consisting of a 3D kV cone-beam CT scan of an anthropomorphic phantom of the pelvis and 2D MV portal images in the anterior-posterior (AP) view acquired at 15 different poses and (2) a conventional 3D kV CT scan and weekly 2D MV AP portal images of a patient over 8 weeks. The mean (and standard deviation) of the absolute registration error for rotations around the right-lateral (RL), inferior-superior (IS), and posterior-anterior (PA) axes were 0.212 degree (0.214 degree), 0.055 degree (0.033 degree) and 0.041 degree (0.039 degree), respectively. The corresponding registration errors for translations along the RL, IS, and PA axes were 0.161 (0.131) mm, 0.096 (0.033) mm, and 0.612 (0.485) mm. The mean (and standard deviation) of the total registration error was 0.778 (0.543) mm. Registration on the patient images was successful in all eight cases as determined visually. The results indicate that it is feasible to automatically enhance features in MV portal images of the pelvis for use within a completely automated 2D-3D registration framework for the accurate determination of patient setup errors. They also indicate that it is feasible to estimate all six transformation parameters from a 3D CT of the pelvis and a single portal image in the AP view.     

Neuronavigation accuracy dependence on CT and MR imaging parameters: a phantom-based study

Clinical benefits from neuronavigation are well established. However, the complexity of its technical environment requires a careful evaluation of different types of errors. In this work, a detailed phantom study which investigates the accuracy in a neuronavigation procedure is presented. The dependence on many different imaging parameters, such as field of view, slice thickness and different kind of sequences (sequential and spiral for CT, T1-weighted and T2-weighted for MRI), is quantified. Moreover, data based on CT images are compared to those based on MR images, taking into account MRI distortion. Finally, the contributions to global accuracy coming from image acquisition, registration and navigation itself are discussed. Results demonstrate the importance of imaging accuracy. Procedures based on CT proved to be more accurate than procedures based on MRI. In the former, values from 2 to 2.5 mm are obtained for 95% fractiles of cumulative distribution of Euclidean distances between the intended target and the reached one while, in the latter, the measured values range from 3 to 4 mm. The absence of imaging distortion proved to be crucial for registration accuracy in MR-based procedures.