Mutual information-based multimodal image registration using a novel joint histogram estimation

Mutual information (MI)-based image registration has been proved to be very effective in multimodal medical image applications. For computing the mutual information between two images, the joint histogram needs to be estimated. As we know, the joint histogram estimation through linear interpolation and partial volume (PV) interpolation methods may result in the emergency of the local extreme in mutual information registration function. The local extreme is likely to hamper the optimization process and influence the registration accuracy. In this paper, we present a novel joint histogram estimation method (HPV) by using an approximate function of Hanning windowed sinc as kernel function of partial volume interpolation. We apply it to both rigid registration and non-rigid registration. In addition, we give a new method estimating the gradient of mutual information with respect to the model parameters during non-rigid registration. By the experiments on both synthetic and real images, it is clearly shown that the new algorithm has the ability to reduce the local extreme, and the registration accuracy is improved.     

PET与MRI三维脑图像高精度配准

目的 在PC机上实现高精度的PET与MRI三维脑图像配准。方法 采用最大互信息法对6例患者PET和MRI三维脑图像进行刚体配准。使用归一化互信息作为相似性量度。在互信息计算过程中，使用Powell多参数优化法和Brent一维搜索算法。为加快配准速度，使用了多分辨金字塔方法。采用基于坐标的阈值选取方法对PET图像进行分割预处理，消除星状背景伪影。结果 配准误差平均值为2．6mm，误差中位数平均为2．7mm。结论 配准视觉效果良好，评估证明该算法可达亚体元级配准精度。     

Hierarchical segmentation-assisted multimodal registration for MR brain images

Information theory-based metric such as mutual information (MI) is widely used as similarity measurement for multimodal registration. Nevertheless, this metric may lead to matching ambiguity for non-rigid registration. Moreover, maximization of MI alone does not necessarily produce an optimal solution. In this paper, we propose a segmentation-assisted similarity metric based on point-wise mutual information (PMI). This similarity metric, termed SPMI, enhances the registration accuracy by considering tissue classification probabilities as prior information, which is generated from an expectation maximization (EM) algorithm. Diffeomorphic demons is then adopted as the registration model and is optimized in a hierarchical framework (H-SPMI) based on different levels of anatomical structure as prior knowledge. The proposed method is evaluated using Brainweb synthetic data and clinical fMRI images. Both qualitative and quantitative assessment were performed as well as a sensitivity analysis to the segmentation error. Compared to the pure intensity-based approaches which only maximize mutual information, we show that the proposed algorithm provides significantly better accuracy on both synthetic and clinical data.     

纠正弥散张量成像图像变形的一种方法

目的：采用图像变形配准的方法校正磁共振弥散张量成像（DTI）的几何畸变，以利于神经外科导航中DTI与原始图像的融合。材料和方法：采用仿射变换和B样条的变形配准方法来矫正几何畸变，配准的相似性测度采用互信息为准则。结果：弥散张量成像特定组织间由磁敏感性差异引起的几何畸变得到了一定纠正。结论：采用3D配准的方法可以对弥散张量成像的几何畸变进行纠正，改善图像质量，提高其在神经外科导航中的临床应用价值。     

全局异常信号环境下基于体素灰度多模医学图像配准研究

目的 在全局异常信号环境下,找出适合于临床应用的、满足精度和鲁棒性要求的基于体素灰度多模医学图像配准相似性测度。方法 结合对各种相似性测度的分析,对无异常信号的实际医学图像,和分别加了随机噪声及全局异常信号的多模医学图像进行配准精度的分析。结果 对各种已有成熟的相似性测度进行理论分析和实验对比研究的基础上,归一化互信息在全局异常信号环境下对多模医学图像进行配准,它们的配准精度和鲁棒性表现都令人满意,能得到准确的配准结果。而基于相关比和互信息的配准方法,不能准确地配准加了全局异常信号的多模医学图像。结论 相比于其他相似性测度,归一化互信息在全局异常信号环境下,是一个能满足配准精度和鲁棒性要求的合适相似性测度。     

Normalized gradient fields for nonlinear motion correction of DCE-MRI time series

Dynamic MR image recordings (DCE-MRI) of moving organs using bolus injections create two different types of dynamics in the images: (i) spatial motion artifacts due to patient movements, breathing and physiological pulsations that we want to counteract and (ii) signal intensity changes during contrast agent wash-in and wash-out that we want to preserve. Proper image registration is needed to counteract the motion artifacts and for a reliable assessment of physiological parameters. In this work we present a partial differential equation-based method for deformable multimodal image registration using normalized gradients and the Fourier transform to solve the Euler–Lagrange equations in a multilevel hierarchy. This approach is particularly well suited to handle the motion challenges in DCE-MRI time series, being validated on ten DCE-MRI datasets from the moving kidney. We found that both normalized gradients and mutual information work as high-performing cost functionals for motion correction of this type of data. Furthermore, we demonstrated that normalized gradients have improved performance compared to mutual information as assessed by several performance measures. We conclude that normalized gradients can be a viable alternative to mutual information regarding registration accuracy, and with promising clinical applications to DCE-MRI recordings from moving organs.     

Comparison and evaluation of rigid, affine, and nonrigid registration of breast MR images.

PURPOSE: A new nonrigid registration method, designed to reduce the effect of movement artifact in subtraction images from breast MR, is compared with existing rigid and affine registration methods. METHOD: Nonrigid registration was compared with rigid and affine registration methods and unregistered images using 54 gadolinium-enhanced 3D breast MR data sets. Twenty-seven data sets had been previously reported normal, and 27 contained a histologically proven carcinoma. The comparison was based on visual assessment and ranking by two radiologists. RESULTS: When analyzed by two radiologists independently, all three registration methods gave better-quality subtraction images than unregistered images (p < 0.01), but nonrigid registration gave significantly better results than the rigid and affine registration methods (p < 0.01). There was no significant difference between rigid and affine registration methods. CONCLUSION: Nonrigid registration significantly reduces the effects of movement artifact in subtracted contrast-enhanced breast MRI. This may enable better visualization of small tumors and those within a glandular breast.     

Quantitative and clinical analysis of SPECT image registration for epilepsy studies.

This study reports quantitative measurements of the accuracy of two popular voxel-based registration algorithms--Woods' automated image registration algorithm and mutual information correlation--and compares these with conventional surface matching (SM) registration. METHODS: The registration algorithms were compared (15 different matches each) for (a) three-dimensional brain phantom images, (b) an ictal SPECT image from a patient with partial epilepsy matched to itself after modification to simulate changes in the cerebral blood flow pattern and (c) ictal/interictal SPECT images from 15 patients with partial epilepsy. Blinded visual ranking and localization of the subtraction images derived from the patient images were also performed. RESULTS: Both voxel-based registration methods were more accurate than SM registration (P < 0.0005). Automated image registration algorithm was more accurate than mutual information correlation for the computer-simulated ictal/interictal images and the patient ictal/interictal studies (P < 0.05). The subtraction SPECTs from SM were poorer in visual ranking more often than the voxel-based methods (P < 0.05). CONCLUSION: Voxel intensity-based registration algorithms provide significant improvement in ictal/interictal SPECT registration accuracy and result in a clinically detectable improvement in the subtraction SPECT images.     

Registering histologic and MR images of prostate for image-based cancer detection

RATIONALE AND OBJECTIVES: Needle biopsy is currently the only way to confirm prostate cancer. To increase prostate cancer diagnostic rate, needles are expected to be deployed at suspicious cancer locations. High-contrast magnetic resonance (MR) imaging provides a powerful tool for detecting suspicious cancerous tissues. To do this, MR appearances of cancerous tissue should be characterized and learned from a sufficient number of prostate MR images with known cancer information. However, ground-truth cancer information is only available in histologic images. Therefore it is necessary to warp ground-truth cancerous regions in histological images to MR images by a registration procedure. The objective of this article is to develop a registration technique for aligning histological and MR images of the same prostate. MATERIAL AND METHODS: Five pairs of histological and T2-weighted MR images of radical prostatectomy specimens are collected. For each pair, registration is guided by two sets of correspondences that can be reliably established on prostate boundaries and internal salient bloblike structures of histologic and MR images. RESULTS: Our developed registration method can accurately register histologic and MR images. It yields results comparable to manual registration, in terms of landmark distance and volume overlap. It also outperforms both affine registration and boundary-guided registration methods. CONCLUSIONS: We have developed a novel method for deformable registration of histologic and MR images of the same prostate. Besides the collection of ground-truth cancer information in MR images, the method has other potential applications. An automatic, accurate registration of histologic and MR images actually builds a bridge between in vivo anatomical information and ex vivo pathologic information, which is valuable for various clinical studies.     

Automatic registration of MR and SPECT images for treatment planning in prostate cancer

RATIONALE AND OBJECTIVES: To aid in surgical and radiation therapy planning for prostate adenocarcinoma, a general-purpose automatic registration method that is based on mutual information was used to align magnetic resonance (MR) images and single photon emission computed tomographic (SPECT) images of the pelvis and prostate. MATERIALS AND METHODS: The authors assessed the effects of various factors on alignment between pairs of MR and SPECT images, including the use of particular pulse sequences in MR imaging, image voxel intensity scaling, the use of different regions on the MR-SPECT histogram, spatial masking of nonoverlapping visual data between images, and multiresolution optimization. A mutual information algorithm was used as the cost function for automatic registration. Automatic registration was deemed acceptable when it resulted in a transformation with less than 2 voxel units (6 mm) difference in translation and less than 2 degree difference in rotation from that obtained with manual registration performed independently by nuclear medicine radiologists. RESULTS: Paired sets of MR and SPECT image volumes from four of five patients were successfully registered. For successful registration, MR images must be optimal and registration must be performed at full spatial resolution and at the full intensity range. Masking, cropping, and the normalization of mutual information, used to register partially overlapping MR-SPECT volumes, were not successful. Multiresolution optimization had little effect on the accuracy and speed of the registration. CONCLUSION: Automatic registration between MR and SPECT images of the pelvis can be achieved when data acquisition and image processing are performed properly. It should prove useful for prostate cancer diagnosis, staging, and treatment planning.     

Motion tracking in MR-guided liver therapy by using navigator echoes and projection profile matching

RATIONALE AND OBJECTIVES: Image registration in magnetic resonance (MR) image-guided liver therapy enhances surgical guidance by fusing preoperative multimodality images with intraoperative images, or by fusing intramodality images to correlate serial intraoperative images to monitor the effect of therapy. The objective of this paper is to describe the application of navigator echo and projection profile matching to fast two-dimensional image registration for MR-guided liver therapy. MATERIALS AND METHODS: We obtain navigator echoes along the read-out and phase-encoding directions by using modified gradient echo imaging. This registration is made possible by masking out the liver profile from the image and performing profile matching with cross-correlation or mutual information as similarity measures. The set of experiments include a phantom study with a 2.0-T experimental MR scanner, and a volunteer and a clinical study with a 0.5-T open-configuration MR scanner, and these evaluate the accuracy and effectiveness of this method for liver therapy. RESULTS: Both the phantom and volunteer study indicate that this method can perform registration in 34 ms with root-mean-square error of 1.6 mm when the given misalignment of a liver is 30 mm. The clinical studies demonstrate that the method can track liver motion of up to approximately 40 mm. Matching profiles with cross-correlation information perform better than with mutual information in terms of robustness and speed. CONCLUSION: The proposed image registration method has potential clinical impact on and advantages for MR-guided liver therapy.     

Automatic arm removal in PET and CT images for deformable registration

Positron emission tomography (PET) imaging is rapidly expanding its role in clinical practice for cancer management. The high sensitivity of PET for functional abnormalities associated with cancer can be confounded by the minimal anatomical information it provides for cancer localization. Computed tomography (CT) provides detailed anatomical information but is less sensitive to pathologies than PET. Thus, combining (i.e., registering) PET and CT images would enable both accurate and sensitive cancer localization with respect to detailed patient anatomy. An additional application area of registration is to align CT–CT scans from serial studies on a patient on a PET/CT scanner to facilitate accurate assessment of therapeutic response from the co-aligned PET images. To facilitate image fusion, we are developing a deformable registration software system using mutual information and a B-spline model of the deformation. When applying deformable registration to whole body images, one of the obstacles is that the arms are present in PET images but not in CT images or are in different positions in serial CT images. This feature mismatch requires a preprocessing step to remove the arms where present and thus adds a manual step in an otherwise automatic algorithm. In this paper, we present a simple yet effective method for automatic arm removal. We demonstrate the efficiency and robustness of this algorithm on both clinical PET and CT images. By streamlining the entire registration process, we expect that the fusion technology will soon find its way into clinics, greatly benefiting cancer diagnosis, staging, therapy planning and treatment monitoring.     

Bidirectional elastic image registration using B-spline affine transformation

A registration scheme termed as B-spline affine transformation (BSAT) is presented in this study to elastically align two images. We define an affine transformation instead of the traditional translation at each control point. Mathematically, BSAT is a generalized form of the affine transformation and the traditional B-spline transformation (BST). In order to improve the performance of the iterative closest point (ICP) method in registering two homologous shapes but with large deformation, a bidirectional instead of the traditional unidirectional objective/cost function is proposed. In implementation, the objective function is formulated as a sparse linear equation problem, and a sub-division strategy is used to achieve a reasonable efficiency in registration. The performance of the developed scheme was assessed using both two-dimensional (2D) synthesized dataset and three-dimensional (3D) volumetric computed tomography (CT) data. Our experiments showed that the proposed B-spline affine model could obtain reasonable registration accuracy.     

Improved efficiency for MRI-SPET registration based on mutual information

Mutual information has been proposed as a criterion for image registration. The criterion is calculated from a two-dimensional grey-scale histogram of the image pair being registered. In this paper we study how sparse sampling can be used to increase speed performance using the registration algorithm of Maes et al. (IEEE Trans Med Imaging 1997; 16: 187–198) with a focus on registration of MRI-SPET brain images. In particular we investigate how sparse sampling and parameters such as the number of bins used for the grey-scale histograms and smoothing of the data prior to registration affect accuracy and robustness of the registration. The method was validated using both simulated and human data. Our results show that sparse sampling introduced local maxima into the mutual information similarity function when the number of bins used for the histograms was large. To speed up registration while retaining robustness, smoothing of the data prior to registration was used and a coarse to fine subsampling protocol, where the number of bins in the histograms were dependent on the subsampling factor, was employed. For the simulated data, the method was able to recover known transformations with an accuracy of about 1 mm. Using the human data, there were no significant differences in the recovered transformation parameters when the suggested subsampling scheme was used compared with when no subsampling was used, but there was a more than tenfold increase in speed. Our results show that, with the appropriate choice of parameters, the method can accurately register MRI-SPET brain images even when very efficient sampling protocols are used. Received 14 December 1999 and in revised form 15 March 2000     

Improved efficiency for MRI-SPET registration based on mutual information

Mutual information has been proposed as a criterion for image registration. The criterion is calculated from a two-dimensional grey-scale histogram of the image pair being registered. In this paper we study how sparse sampling can be used to increase speed performance using the registration algorithm of Maes et al. (IEEE Trans Med Imaging 1997; 16: 187-198) with a focus on registration of MRI-SPET brain images. In particular we investigate how sparse sampling and parameters such as the number of bins used for the grey-scale histograms and smoothing of the data prior to registration affect accuracy and robustness of the registration. The method was validated using both simulated and human data. Our results show that sparse sampling introduced local maxima into the mutual information similarity function when the number of bins used for the histograms was large. To speed up registration while retaining robustness, smoothing of the data prior to registration was used and a coarse to fine subsampling protocol, where the number of bins in the histograms were dependent on the subsampling factor, was employed, For the simulated data, the method was able to recover known transformations with an accuracy of about 1 mm. Using the human data, there were no significant differences in the recovered transformation parameters when the suggested subsampling scheme was used compared with when no subsampling was used, but there was a more than tenfold increase in speed. Our results show that, with the appropriate choice of parameters, the method can accurately register MRI-SPET brain images even when very efficient sampling protocols are used.     

Evaluation of Four Volume-Based Image Registration Algorithms

We evaluated 4 volume-based automatic image registration algorithms from 2 commercially available treatment planning systems (Philips Syntegra and BrainScan). The algorithms based on cross correlation (CC), local correlation (LC), normalized mutual information (NMI), and BrainScan mutual information (BSMI) were evaluated with: (1) the synthetic computed tomography (CT) images, (2) the CT and magnetic resonance (MR) phantom images, and (3) the CT and MR head image pairs from 12 patients with brain tumors. For the synthetic images, the registration results were compared with known transformation parameters, and all algorithms achieved accuracy of submillimeter in translation and subdegree in rotation. For the phantom images, the registration results were compared with those provided by frame and marker-based manual registration. For the patient images, the results were compared with anatomical landmark–based manual registration to qualitatively determine how the results were close to a clinically acceptable registration. NMI and LC outperformed CC and BSMI, with the sense of being closer to a clinically acceptable result. As for the robustness, NMI and BSMI outperformed CC and LC. A guideline of image registration in our institution was given, and final visual assessment is necessary to guarantee reasonable results.     

Evaluation of Four Volume-Based Image Registration Algorithms

We evaluated 4 volume-based automatic image registration algorithms from 2 commercially available treatment planning systems (Philips Syntegra and BrainScan). The algorithms based on cross correlation (CC), local correlation (LC), normalized mutual information (NMI), and BrainScan mutual information (BSMI) were evaluated with: (1) the synthetic computed tomography (CT) images, (2) the CT and magnetic resonance (MR) phantom images, and (3) the CT and MR head image pairs from 12 patients with brain tumors. For the synthetic images, the registration results were compared with known transformation parameters, and all algorithms achieved accuracy of submillimeter in translation and subdegree in rotation. For the phantom images, the registration results were compared with those provided by frame and marker-based manual registration. For the patient images, the results were compared with anatomical landmark–based manual registration to qualitatively determine how the results were close to a clinically acceptable registration. NMI and LC outperformed CC and BSMI, with the sense of being closer to a clinically acceptable result. As for the robustness, NMI and BSMI outperformed CC and LC. A guideline of image registration in our institution was given, and final visual assessment is necessary to guarantee reasonable results.     

First-pass myocardial perfusion image registration by maximization of normalized mutual information

PURPOSE: To evaluate a left ventricular image registration algorithm for first-pass MR myocardial perfusion. MATERIALS AND METHODS: A normalized mutual information based motion correction algorithm was proposed and tested on 27 adenosine stressed myocardial perfusion cases consisting of pretreatment and posttreatment of 15 patients undergone autologous bone marrow mononuclear cell transplant therapy. An image mask approximately covering the left and right ventricles was manually defined to include a region of interest for registration. A two-dimensional multiresolution registration approach was used to register consecutively acquired multislice images with in-plane translations. The method was validated by manual registration and singular value deconvolution based perfusion analysis. RESULTS: The proposed image registration algorithm was found to be robust in minimizing the in-plane motion of the left ventricle in first-pass myocardial perfusion. The image mask including the left and right ventricle was found to be more robust than including the left ventricle alone. A smooth estimate of normalized mutual information coefficients were achieved for images with large contrast changes. CONCLUSION: The proposed semiautomatic multiresolution registration algorithm was able to register first-pass MR myocardial perfusion images and may be useful in quantitative perfusion analysis.     

Respiratory motion in coronary magnetic resonance angiography: a comparison of different motion models

PURPOSE: To assess respiratory motion models for coronary magnetic resonance angiography (CMRA). In this study various motion models that describe the respiration-induced 3D displacements and deformations of the main coronary arteries were compared.MATERIALS AND METHODS: Multiple high-resolution 3D coronary MR images were acquired in healthy volunteers using navigator-based respiratory gating, each depicting the coronary vessels at different respiratory motion states. In the images representing the different inspiratory states the displacements and deformations of the main coronary vessels with respect to the end-expiratory state were determined, by means of elastic registration. Several correction models (superior-inferior (SI) translation, 3D translation, and 3D affine transformation) were tested and compared with respect to their ability to map a selected inspiratory to the end-expiratory motion state.RESULTS: 3D translation was found to be superior over SI translation, which is commonly used for prospective motion correction in CMRA. The 3D affine transformation was found to be the best correction model considered in this study. Furthermore, a large intersubject variability of the model parameters was observed.CONCLUSION: The results of this study indicate that a patient-adapted 3D correction model (3D translation or better 3D affine) will considerably improve prospective motion correction in CMRA.     

Automated image registration of gated cardiac single-photon emission computed tomography and magnetic resonance imaging

PURPOSE: To develop an automatic registration method for electrocardiogram-gated myocardial perfusion single-photon emission computed tomography (SPECT) and cardiac cine-magnetic resonance imaging (MRI). MATERIALS AND METHODS: Paired myocardial perfusion SPECT (MPS) and MRI from 20 patients were considered. MR images were presegmented by heart localization based on detection of cardiac motion and optimal thresholding. A registration algorithm based on mutual information was subsequently applied to all time frames or a selected subset from both modalities. RESULTS: A preprocessing step significantly improved the accuracy of the registration when compared to automatic registration performed without preprocessing. Errors in translation parameters (T(x), T(y), T(z)) averaged (1.0 +/- 1.5, 1.1 +/- 1.3, 0.9 +/- 0.9) pixels with MRI segmentation and (4.6 +/- 3.2, 3.4 +/- 2.6, 3.0 +/- 3.4) pixels without MRI segmentation. Errors in rotation parameters (R(x), R(y), R(z)) averaged (5.4 +/- 2.9, 3.4 +/- 2.7, 4.5 +/- 3.6) degrees with MRI segmentation and (9.3 +/- 6.1, 4.8 +/- 4.3, 14.6 +/- 12.6) degrees without MRI segmentation. Error was calculated as the absolute difference between the expert manual and the automatic registration transformation. CONCLUSION: Automatic registration of gated MPS and cine MRI is possible with the use of a mutual information-based technique when MR images are presegmented. Cardiac motion can be used to isolate the left ventricle (LV) on the MR images automatically, and subsequently the segmented MR images can be coregistered with gated MPS.