Improved efficiency for MRI-SPET registration based on mutual information

Mutual information has been proposed as a criterion for image registration. The criterion is calculated from a two-dimensional grey-scale histogram of the image pair being registered. In this paper we study how sparse sampling can be used to increase speed performance using the registration algorithm of Maes et al. (IEEE Trans Med Imaging 1997; 16: 187–198) with a focus on registration of MRI-SPET brain images. In particular we investigate how sparse sampling and parameters such as the number of bins used for the grey-scale histograms and smoothing of the data prior to registration affect accuracy and robustness of the registration. The method was validated using both simulated and human data. Our results show that sparse sampling introduced local maxima into the mutual information similarity function when the number of bins used for the histograms was large. To speed up registration while retaining robustness, smoothing of the data prior to registration was used and a coarse to fine subsampling protocol, where the number of bins in the histograms were dependent on the subsampling factor, was employed. For the simulated data, the method was able to recover known transformations with an accuracy of about 1 mm. Using the human data, there were no significant differences in the recovered transformation parameters when the suggested subsampling scheme was used compared with when no subsampling was used, but there was a more than tenfold increase in speed. Our results show that, with the appropriate choice of parameters, the method can accurately register MRI-SPET brain images even when very efficient sampling protocols are used. Received 14 December 1999 and in revised form 15 March 2000     

Improved efficiency for MRI-SPET registration based on mutual information

Mutual information has been proposed as a criterion for image registration. The criterion is calculated from a two-dimensional grey-scale histogram of the image pair being registered. In this paper we study how sparse sampling can be used to increase speed performance using the registration algorithm of Maes et al. (IEEE Trans Med Imaging 1997; 16: 187-198) with a focus on registration of MRI-SPET brain images. In particular we investigate how sparse sampling and parameters such as the number of bins used for the grey-scale histograms and smoothing of the data prior to registration affect accuracy and robustness of the registration. The method was validated using both simulated and human data. Our results show that sparse sampling introduced local maxima into the mutual information similarity function when the number of bins used for the histograms was large. To speed up registration while retaining robustness, smoothing of the data prior to registration was used and a coarse to fine subsampling protocol, where the number of bins in the histograms were dependent on the subsampling factor, was employed, For the simulated data, the method was able to recover known transformations with an accuracy of about 1 mm. Using the human data, there were no significant differences in the recovered transformation parameters when the suggested subsampling scheme was used compared with when no subsampling was used, but there was a more than tenfold increase in speed. Our results show that, with the appropriate choice of parameters, the method can accurately register MRI-SPET brain images even when very efficient sampling protocols are used.     

PET与MRI三维脑图像高精度配准

目的 在PC机上实现高精度的PET与MRI三维脑图像配准。方法 采用最大互信息法对6例患者PET和MRI三维脑图像进行刚体配准。使用归一化互信息作为相似性量度。在互信息计算过程中，使用Powell多参数优化法和Brent一维搜索算法。为加快配准速度，使用了多分辨金字塔方法。采用基于坐标的阈值选取方法对PET图像进行分割预处理，消除星状背景伪影。结果 配准误差平均值为2．6mm，误差中位数平均为2．7mm。结论 配准视觉效果良好，评估证明该算法可达亚体元级配准精度。     

Accuracy and reproducibility of co-registration techniques based on mutual information and normalized mutual information for MRI and SPECT brain images

We implemented a 3D co-registration technique based on mutual information (MI) including 2D image matching as a coarse pre-registration. The 2D coarse pre-registration was performed in the transverse, sagittal and coronal planes sequentially, and all six parameters were then optimized as fine registration. Normalized mutual information (NMI) was also examined as another entropy-based measure that was invariant to the overlapped area of two images. In order to compare accuracy and precision of the present method with a conventional two-level multiresolution approach, simulation was performed by 100 trials with the random initial mismatch of +/-10 degrees and +/-17.92 mm (Type-I) and +/-20 degrees and +/-40.32 mm (Type-II). For Type-I, no significant differences were found between registration errors of the multiresolution approach and the present method with the MI criterion. No biases were observed (< or =0.13 degrees and < or =0.57 mm for the multiresolution approach; < or =0.12 degrees and < or =0.57 mm for the present method) and the SDs were very small (< or =0.18 degrees and < or =0.12 mm for the multiresolution approach; < or =0.11 degrees and < or =0.11 mm for the present method). For Type-II, SDs for the multiresolution approach (< or =1.8 degrees and < or =0.88 mm) were markedly larger than those for the present method (< or =0.64 degrees and < or =0.20 mm) with MI. Success rate for the present method was 99.9%, which was higher than 97.6% for the multiresolution approach. Simulation also revealed that MI and NMI performance were almost equivalent. The choice of optimization strategy more affected accuracy and reproducibility than the choice of the registration criterion (MI or NMI) in our simulation condition. The present method is sufficiently accurate and reproducible for MRI-SPECT registration in clinical use.     

Robust rigid registration of retinal angiograms through optimization

Retinal fundus photographs are employed as standard diagnostic tools in ophthalmology. Serial photographs of the flow of fluorescein and indocyanine green (ICG) dye are used to determine the areas of the retinal lesions. For objective measurements of features, the registration of the images is a necessity. In this paper, we employ optimization techniques for registration with the help of 2-parameter translational motion model of retinal angiograms, based on non-linear pre-processing (Wiener filtering and morphological gradient) and computation of the similarity criteria for the alignment of the two gradient images for any given rigid transformation. The optimization methods are effectively employed to minimize the similarity criterion.

The presence of noise, the variations in the background and the temporal variation of the fluorescence level pose serious problems in obtaining a robust registration of the retinal images. Moreover, local search strategies are not robust in the case of ICG angiograms, even if one uses a multiresolution approach.

The present work makes a systematic comparison of different optimization techniques, namely the minimization method derived from the optical flow formulation, the Nelder-Mead local search and the HCIAC ant colony metaheuristic, each optimizing a similarity criterion for the gradient images. The impact of the resolution and median filtering of gradient image is studied and the robustness of the approaches is tested through experimental studies, performed on macular fluorescein and ICG angiographies.

Our proposed optimization techniques have shown interesting results especially for high resolution difficult registration problems. Moreover, this approach seems promising for affine (6-parameter motion model) or elastical registrations.     

Automatic registration of MR and SPECT images for treatment planning in prostate cancer

RATIONALE AND OBJECTIVES: To aid in surgical and radiation therapy planning for prostate adenocarcinoma, a general-purpose automatic registration method that is based on mutual information was used to align magnetic resonance (MR) images and single photon emission computed tomographic (SPECT) images of the pelvis and prostate. MATERIALS AND METHODS: The authors assessed the effects of various factors on alignment between pairs of MR and SPECT images, including the use of particular pulse sequences in MR imaging, image voxel intensity scaling, the use of different regions on the MR-SPECT histogram, spatial masking of nonoverlapping visual data between images, and multiresolution optimization. A mutual information algorithm was used as the cost function for automatic registration. Automatic registration was deemed acceptable when it resulted in a transformation with less than 2 voxel units (6 mm) difference in translation and less than 2 degree difference in rotation from that obtained with manual registration performed independently by nuclear medicine radiologists. RESULTS: Paired sets of MR and SPECT image volumes from four of five patients were successfully registered. For successful registration, MR images must be optimal and registration must be performed at full spatial resolution and at the full intensity range. Masking, cropping, and the normalization of mutual information, used to register partially overlapping MR-SPECT volumes, were not successful. Multiresolution optimization had little effect on the accuracy and speed of the registration. CONCLUSION: Automatic registration between MR and SPECT images of the pelvis can be achieved when data acquisition and image processing are performed properly. It should prove useful for prostate cancer diagnosis, staging, and treatment planning.     

Unsupervised estimation of myocardial displacement from tagged MR sequences using nonrigid registration.

We propose a fully automatic cardiac motion estimation technique that uses nonrigid registration between temporally adjacent images to compute the myocardial displacement field from tagged MR sequences using as inputs (sources) both horizontally and vertically tagged images. We present a new multisource nonrigid registration algorithm employing a semilocal deformation model that provides controlled smoothness. The method requires no segmentation. We apply a multiresolution optimization strategy for better speed and robustness. The accuracy of the algorithm is assessed on experimental data (animal model) and healthy volunteer data by calculating the root mean square (RMS) difference in position between the estimated tag trajectories and manual tracings outlined by an expert. For the approximately 20000 tag lines analyzed (45 slices over 20-40 time frames), the RMS difference between the automatic tag trajectories and the manually segmented tag trajectories was 0.51 pixels (0.25 mm) for the animal data and 0.49 pixels (0.49 mm) for the human volunteer data. The RMS difference in the separation between adjacent tag lines (RMS_TS) was also assessed, resulting in an RMS_TS of 0.40 pixels (0.19 mm) in the experimental data and 0.52 pixels (0.56 mm) in the volunteer data. These results confirm the subpixel accuracy achieved using the proposed methodology.     

Mutual information-based multimodal image registration using a novel joint histogram estimation

Mutual information (MI)-based image registration has been proved to be very effective in multimodal medical image applications. For computing the mutual information between two images, the joint histogram needs to be estimated. As we know, the joint histogram estimation through linear interpolation and partial volume (PV) interpolation methods may result in the emergency of the local extreme in mutual information registration function. The local extreme is likely to hamper the optimization process and influence the registration accuracy. In this paper, we present a novel joint histogram estimation method (HPV) by using an approximate function of Hanning windowed sinc as kernel function of partial volume interpolation. We apply it to both rigid registration and non-rigid registration. In addition, we give a new method estimating the gradient of mutual information with respect to the model parameters during non-rigid registration. By the experiments on both synthetic and real images, it is clearly shown that the new algorithm has the ability to reduce the local extreme, and the registration accuracy is improved.     

A comparative study of Powell's and Downhill Simplex algorithms for a fast multimodal surface matching in brain imaging.

Multimodal images registration can be very helpful for diagnostic applications. However, even if a lot of registration algorithms exist, only a few really work in clinical routines. We developed a method based on surface matching and compared two minimization algorithms: Powell's and Downhill Simplex. We studied the influence of some factors (chamfer map computation, number and order of parameters to determine, minimization criteria) on the final accuracy of the algorithm. Using this comparison, we improved some processing steps to allow a clinical use, and selected the simplex algorithm which presented the best results.     

Improving accuracy and efficiency of mutual information for multi-modal retinal image registration using adaptive probability density estimation

Mutual information (MI) is a popular similarity measure for performing image registration between different modalities. MI makes a statistical comparison between two images by computing the entropy from the probability distribution of the data. Therefore, to obtain an accurate registration it is important to have an accurate estimation of the true underlying probability distribution. Within the statistics literature, many methods have been proposed for finding the ‘optimal’ probability density, with the aim of improving the estimation by means of optimal histogram bin size selection. This provokes the common question of how many bins should actually be used when constructing a histogram. There is no definitive answer to this. This question itself has received little attention in the MI literature, and yet this issue is critical to the effectiveness of the algorithm. The purpose of this paper is to highlight this fundamental element of the MI algorithm. We present a comprehensive study that introduces methods from statistics literature and incorporates these for image registration. We demonstrate this work for registration of multi-modal retinal images: colour fundus photographs and scanning laser ophthalmoscope images. The registration of these modalities offers significant enhancement to early glaucoma detection, however traditional registration techniques fail to perform sufficiently well. We find that adaptive probability density estimation heavily impacts on registration accuracy and runtime, improving over traditional binning techniques.     

First-pass myocardial perfusion image registration by maximization of normalized mutual information

PURPOSE: To evaluate a left ventricular image registration algorithm for first-pass MR myocardial perfusion. MATERIALS AND METHODS: A normalized mutual information based motion correction algorithm was proposed and tested on 27 adenosine stressed myocardial perfusion cases consisting of pretreatment and posttreatment of 15 patients undergone autologous bone marrow mononuclear cell transplant therapy. An image mask approximately covering the left and right ventricles was manually defined to include a region of interest for registration. A two-dimensional multiresolution registration approach was used to register consecutively acquired multislice images with in-plane translations. The method was validated by manual registration and singular value deconvolution based perfusion analysis. RESULTS: The proposed image registration algorithm was found to be robust in minimizing the in-plane motion of the left ventricle in first-pass myocardial perfusion. The image mask including the left and right ventricle was found to be more robust than including the left ventricle alone. A smooth estimate of normalized mutual information coefficients were achieved for images with large contrast changes. CONCLUSION: The proposed semiautomatic multiresolution registration algorithm was able to register first-pass MR myocardial perfusion images and may be useful in quantitative perfusion analysis.     

全局异常信号环境下基于体素灰度多模医学图像配准研究

目的 在全局异常信号环境下,找出适合于临床应用的、满足精度和鲁棒性要求的基于体素灰度多模医学图像配准相似性测度。方法 结合对各种相似性测度的分析,对无异常信号的实际医学图像,和分别加了随机噪声及全局异常信号的多模医学图像进行配准精度的分析。结果 对各种已有成熟的相似性测度进行理论分析和实验对比研究的基础上,归一化互信息在全局异常信号环境下对多模医学图像进行配准,它们的配准精度和鲁棒性表现都令人满意,能得到准确的配准结果。而基于相关比和互信息的配准方法,不能准确地配准加了全局异常信号的多模医学图像。结论 相比于其他相似性测度,归一化互信息在全局异常信号环境下,是一个能满足配准精度和鲁棒性要求的合适相似性测度。     

A digital subtraction radiography scheme based on automatic multiresolution registration

OBJECTIVES: To establish a digital subtraction radiography scheme for aligning clinical in vivo radiographs based on the implementations of an automatic geometric registration method and a contrast correction technique. METHODS: Thirty-five pairs of in vivo dental radiographs from four clinical studies were used in this work. First, each image pair was automatically aligned by applying a multiresolution registration strategy using the affine transformation followed by the implementation of the projective transformation at full resolution. Then, a contrast correction technique was applied in order to produce subtraction radiographs and fused images for further clinical evaluation. The performance of the proposed registration method was assessed against a manual method based on the projective transformation. RESULTS: The qualitative assessment of the experiments based on visual inspection has shown advantageous performance of the proposed automatic registration method against the manual method. Furthermore, the quantitative analysis showed statistical difference in terms of the root mean square (RMS) error estimated over the whole images and specific regions of interest. CONCLUSIONS: The proposed automatic geometric registration method is capable of aligning radiographs acquired with or without rigorous a priori standardization. The methodology is pixel-based and does not require the application of any segmentation process prior to alignment. The employed projective transformation provides a reliable model for registering intraoral radiographs. The implemented contrast correction technique sequentially applied provides subtraction radiographs and fused images for clinical evaluation regarding the evolution of a disease or the response to a therapeutic scheme.     

Improved quality of re-sliced MR images using re-normalized sinc interpolation.

This article shows that re-normalizing the interpolation kernel for a constant integral can make a significant improvement in performance of sinc interpolation methods. A comparison was performed between standard and re-normalized sinc kernels of various sizes using data from four commonly used magnetic resonance (MR) imaging sequences. Standard rotations were performed and compared with a "gold standard" data set generated by use of a large (13 x 13 x 13) sinc kernel. Measurements of systematic pixel intensity offset error and variance of generated residuals were used to estimate resultant interpolation error. Theoretical estimates of the consequent savings in computation time were compared with the measured time required for each algorithm and with the automated image registration (AIR) program. The use of a small (5 x 5 x 5) re-normalized kernel produced relative errors comparable to those in the gold standard data set, allowing saving in computation time of up to 30 times in comparison with standard sinc interpolation. This approach brings the implementation of MR volume re-slicing much closer to the demands of a clinical environment. J. Magn. Reson. Imaging 1999;10:582-588.     

Automated 3-dimensional registration of stand-alone (18)F-FDG whole-body PET with CT.

Image registration and fusion of whole-body (18)F-FDG PET with thoracic CT would allow combination of anatomic detail from CT with functional PET information, which could lead to improved diagnosis or PET-based radiotherapy planning. METHODS: We have designed a practical and fully automated algorithm for the elastic 3-dimensional image registration of whole-body PET and CT images, which compensates for the nonlinear deformation due to breath-hold CT imaging. A set of 18 PET and CT patient datasets has been evaluated by the algorithm. Initially, a 9-parameter linear registration is performed by maximizing the mutual information (MI)-based cost function, between the CT and the combination of emission and transmission PET volumes, using progressively increased matrix sizes to increase speed and provide better convergence. Subsequently, lung contours on transmission maps and corresponding contours on CT volumes are automatically detected. A large number (few hundreds) of corresponding point pairs are automatically derived, defining a thin-plate-spline (TPS) elastic transformation of PET emission and transmission scans to match the CT scan. RESULTS: In all 18 patients the automatic linear registration with multiresolution converged close to the final alignment, but, in 10 cases, the nonlinear differences in the diaphragm position and chest wall were still clearly visible. The nonlinear adjustment, which was in the order of 40-75 mm, significantly improved the alignment between breath-hold CT and PET, especially in the areas of the diaphragm. Lung volumes measured from transmission and CT scans match closely after the warping has been applied. The average computation time is <40 s for the linear component and <30 s for the nonlinear component for a typical PET scan with 4-6 bed positions. CONCLUSION: We have developed a technique for automatic nonlinear registration of CT and PET whole-body images to common spatial coordinates. This technique may be applied for automatic fusion of PET with CT acquired on stand-alone scanners during normal breathing or breath-hold data acquisition.     

Effective incorporating spatial information in a mutual information based 3D–2D registration of a CT volume to X-ray images

This paper addresses the problem of estimating the 3D rigid poses of a CT volume of an object from its 2D X-ray projection(s). We use maximization of mutual information, an accurate similarity measure for multi-modal and mono-modal image registration tasks. However, it is known that the standard mutual information measures only take intensity values into account without considering spatial information and their robustness is questionable. In this paper, instead of directly maximizing mutual information, we propose to use a variational approximation derived from the Kullback-Leibler bound. Spatial information is then incorporated into this variational approximation using a Markov random field model. The newly derived similarity measure has a least-squares form and can be effectively minimized by a multi-resolution Levenberg-Marquardt optimizer. Experiments were conducted on datasets from two applications: (a) intra-operative patient pose estimation from a limited number (e.g. 2) of calibrated fluoroscopic images, and (b) post-operative cup orientation estimation from a single standard X-ray radiograph with/without gonadal shielding. The experiment on intra-operative patient pose estimation showed a mean target registration accuracy of 0.8 mm and a capture range of 11.5 mm, while the experiment on estimating the post-operative cup orientation from a single X-ray radiograph showed a mean accuracy below 2° for both anteversion and inclination. More importantly, results from both experiments demonstrated that the newly derived similarity measures were robust to occlusions in the X-ray image(s).     

Correction of motion artifacts from cardiac cine magnetic resonance images

RATIONALE AND OBJECTIVES: An image registration method was developed to automatically correct motion artifacts, mostly from breathing, from cardiac cine magnetic resonance (MR) images. MATERIALS AND METHODS: The location of each slice in an image stack was optimized by maximizing a similarity measure of the slice with another image slice stack. The optimization was performed iteratively and both image stacks were corrected simultaneously. Two procedures to optimize the similarity were tested: standard gradient optimization and stochastic optimization in which one slice is chosen randomly from the image stacks and its location is optimized. In this work, cine short- and long-axis images were used. In addition to visual inspection results from real data, the performance of the algorithm was evaluated quantitatively by simulating the movements in four real MR data sets. The mean error and standard deviation were defined for 50 simulated movements as each slice was randomly displaced. The error rate, defined as the percentage of non-satisfactory registration results, was evaluated. The paired t-test was used to evaluate the statistical difference between the tested optimization methods. RESULTS: The algorithm developed was successfully applied to correct motion artifacts from real and simulated data. The results, where typical motion artifacts were simulated, indicated an error rate of about 3%. Subvoxel registration accuracy was also achieved. When different optimization methods were compared, the registration accuracy of the stochastic approach proved to be superior to the standard gradient technique (P < 10(-9)). CONCLUSIONS: The novel method was capable of robustly and accurately correcting motion artifacts from cardiac cine MR images.     

Hierarchical segmentation-assisted multimodal registration for MR brain images

Information theory-based metric such as mutual information (MI) is widely used as similarity measurement for multimodal registration. Nevertheless, this metric may lead to matching ambiguity for non-rigid registration. Moreover, maximization of MI alone does not necessarily produce an optimal solution. In this paper, we propose a segmentation-assisted similarity metric based on point-wise mutual information (PMI). This similarity metric, termed SPMI, enhances the registration accuracy by considering tissue classification probabilities as prior information, which is generated from an expectation maximization (EM) algorithm. Diffeomorphic demons is then adopted as the registration model and is optimized in a hierarchical framework (H-SPMI) based on different levels of anatomical structure as prior knowledge. The proposed method is evaluated using Brainweb synthetic data and clinical fMRI images. Both qualitative and quantitative assessment were performed as well as a sensitivity analysis to the segmentation error. Compared to the pure intensity-based approaches which only maximize mutual information, we show that the proposed algorithm provides significantly better accuracy on both synthetic and clinical data.     

Software for image registration: algorithms,accuracy, efficacy

Image registration is finding increased clinical use both in aiding diagnosis and guiding therapy. There are numerous algorithms for registration, which all involve maximizing a measure of similarity between a transformed floating image and a fixed reference image. The choice of the similarity measure depends, to some extent, on the application. Methods based on the use of the joint intensity histogram have become popular because of their flexibility and robustness. A distinction is made between rigid-body and non-rigid transformations. The latter are needed for inter-subject registration or intra-subject registration in cases where the region of the body of interest is not considered rigid. Non-rigid transformation is normally achieved using a global model of the deformation but can also be defined by a set of locally rigid transformations, each constrained to a small block in the image. There is scope for further research on the incorporation of appropriate constraints, especially for the application of non-rigid transformations to nuclear medicine studies. Most of the initial practical concerns regarding image registration have been overcome and there is increasing availability of commercial software. There are several approaches to the validation of registration software, with validation of non-rigid algorithms being particularly difficult. Studies have demonstrated the accuracy on the order of half a pixel for both intra- and inter-modality registration (typically 2 to 3 mm). Although hardware-based registration has now become possible by using dual-modality instruments, software-based registration will continue to play an important role in nuclear medicine.     

Constrained reverse diffusion for thick slice interpolation of 3D volumetric MRI images

Due to physical limitations inherent in magnetic resonance imaging scanners, three dimensional volumetric scans are often acquired with anisotropic voxel resolution. We investigate several interpolation approaches to reduce the anisotropy and present a novel approach - constrained reverse diffusion for thick slice interpolation. This technique was compared to common methods: linear and cubic B-Spline interpolation and a technique based on non-rigid registration of neighboring slices. The methods were evaluated on artificial MR phantoms and real MR scans of human brain. The constrained reverse diffusion approach delivered promising results and provides an alternative for thick slice interpolation, especially for higher anisotropy factors.