Mutual information-based multimodal image registration using a novel joint histogram estimation

Mutual information (MI)-based image registration has been proved to be very effective in multimodal medical image applications. For computing the mutual information between two images, the joint histogram needs to be estimated. As we know, the joint histogram estimation through linear interpolation and partial volume (PV) interpolation methods may result in the emergency of the local extreme in mutual information registration function. The local extreme is likely to hamper the optimization process and influence the registration accuracy. In this paper, we present a novel joint histogram estimation method (HPV) by using an approximate function of Hanning windowed sinc as kernel function of partial volume interpolation. We apply it to both rigid registration and non-rigid registration. In addition, we give a new method estimating the gradient of mutual information with respect to the model parameters during non-rigid registration. By the experiments on both synthetic and real images, it is clearly shown that the new algorithm has the ability to reduce the local extreme, and the registration accuracy is improved.     

一种基于模式控制自由变形的非刚性医学图像配准算法

目的 对手术前MRI／CT图像和手术中超声图像两种模态下都可见的血管结构进行配准。方法 提出一种基于自由变形模型的多模态医学图像的非刚性配准的方法。当两个图像对准时，一种图像中的血管中心点就会对应着另一种图像下灰度脊点。对于全局变换采用刚性变换，而对于局部的形变，采用一种基于模式控制B样条的自由变形模型（FFD）来描述。配准算法采用遗传算法和共轭梯度法结合的优化策略来最小化目标函数。结果 我们设计了两个实验，分别应用于体模和临床数据来评价我们的算法。这种方法是连续而且准确的。最后的变换参数的均方差值是亚像素、亚毫米级的，在0．010弧度以内的。结论 实验结果显示本方法从配准精度和收敛速度上都得到了良好的效果。可以有效地应用于超声图像导航手术系统。     

Fourier registration of three-dimensional brain MR images: Exploiting the axis of rotation

We have developed a novel algorithm to register three-dimensional MR images that have undergone rigid body motion. The most interesting feature of the algorithm is that it reduces a general three-dimensional rotation to a simple planar rotation by finding the axis of rotation. The algorithm, which is a nontrivial three-dimensional extension of existing Fourier registration algorithms, has been tested on 30 artificially misaligned MR images of a phantom, four artificially misaligned MR images of a brain, and one case of actual patient motion. The algorithm successfully registered every image. The registration error for a voxel 10 cm from the origin for the artificially misaligned phantom images was 2.8 mm at most and had a mean of 1.2 mm and standard deviation of .7 mm. The registration parameters for the images contaminated by actual patient motion were similar to that from an established image registration algorithm. The results indicate that the algorithm is accurate, reliable, and fast. The rigid body model requires the brain to be segmented from MR images of the head before registration.     

Automatic registration of MR and SPECT images for treatment planning in prostate cancer

RATIONALE AND OBJECTIVES: To aid in surgical and radiation therapy planning for prostate adenocarcinoma, a general-purpose automatic registration method that is based on mutual information was used to align magnetic resonance (MR) images and single photon emission computed tomographic (SPECT) images of the pelvis and prostate. MATERIALS AND METHODS: The authors assessed the effects of various factors on alignment between pairs of MR and SPECT images, including the use of particular pulse sequences in MR imaging, image voxel intensity scaling, the use of different regions on the MR-SPECT histogram, spatial masking of nonoverlapping visual data between images, and multiresolution optimization. A mutual information algorithm was used as the cost function for automatic registration. Automatic registration was deemed acceptable when it resulted in a transformation with less than 2 voxel units (6 mm) difference in translation and less than 2 degree difference in rotation from that obtained with manual registration performed independently by nuclear medicine radiologists. RESULTS: Paired sets of MR and SPECT image volumes from four of five patients were successfully registered. For successful registration, MR images must be optimal and registration must be performed at full spatial resolution and at the full intensity range. Masking, cropping, and the normalization of mutual information, used to register partially overlapping MR-SPECT volumes, were not successful. Multiresolution optimization had little effect on the accuracy and speed of the registration. CONCLUSION: Automatic registration between MR and SPECT images of the pelvis can be achieved when data acquisition and image processing are performed properly. It should prove useful for prostate cancer diagnosis, staging, and treatment planning.     

A reference system for neuroanatomical localization on functional reconstructed cerebral images

The correlation of anatomical with functional brain images is of critical importance for data analysis. We have designed and tested a reference system method that provides accurate localization of neuroanatomy on functional brain images. The method differs from prior techniques using headholding devices in that it does not require head fixation and is noninvasive. The reference system can be quickly, accurately, and reproducibly applied to the patient's head with respect to the surface anatomy of the head. The reference system is easily adaptable to multiple imaging modalities. The reference system is implemented with the aid of a computer algorithm that reconstructs the anatomic images into a new image matching a selected functional image. Accuracy and reproducibility of placement of the reference system was demonstrated by the application of the system on multiple subjects by two independent observers. The use of this method provides accurate correlation between functional and anatomical brain images.     

Image registration: an essential tool for nuclear medicine

There is increasing interest in being able to automatically register medical images from either the same or different modalities. Registered images are proving useful in a range of applications, not only providing more correlative information to aid in diagnosis, but also assisting with the planning and monitoring of therapy, both surgery and radiotherapy. The practising nuclear medicine specialist is faced with a dilemma in choosing an appropriate method since the literature in the field is extensive, with conflicting evidence as to what methods are optimal. Although most barriers to implementing registration in routine practice have been removed, there remains a lack of commercial, validated software. The alternative is to install a dual-modality instrument. The objective of this review is to present a general overview of medical image registration with emphasis on the application and issues relevant to nuclear medicine.     

3种图像模式在影像引导放射治疗中的差异性研究

目的 检测并分析电子射野影像系统( EPID)、kV级平面摄像(kV planar)、kV级锥形束CT( CBCT)3种图像模式在影像引导放疗(IGRT)中摆位误差的精度.方法 通过在计划系统的模体CT图像上模拟25组摆位偏差,生成25组DRR参考图像和25组三维CT参考图像,分别与模体在原点位置获取的EPID,kV planar和CBCT 3种图像模式进行图像配准和摆位误差的测量,检测并比较3种图像模式在摆位误差测量中的精度.结果 共读取675组残留误差,3种图像及其相应的配准方法均具有较高的精确度,残留误差的平均值在x、y、z方向上均＜1 mm,CBCT的灰度值配准精度＜0.1 mm,kV planar图像在手动配准时其测量摆位误差的精度好于EPID(＜0.65 mm),在自动配准方法时与EPID具有同等的精度.结论 医院IGRT系统的3种图像模式及其相应的配准方法均可以满足临床应用,CBCT为首选,结合图像质量、成像剂量和配准精度等因素,kV planar优于EPID.     

Directed graph based image registration

In this paper, a novel image registration method is proposed to achieve accurate registration between images having large shape differences with the help of a set of appropriate intermediate templates. We first demonstrate that directionality is a key factor in both pairwise image registration and groupwise registration, which is defined in this paper to describe the influence of the registration direction and paths on the registration performance. In our solution, the intermediate template selection and intermediate template guided registration are two coherent steps with directionality being considered. To take advantage of the directionality, a directed graph is built based on the asymmetric distance defined on all ordered image pairs in the image population, which is fundamentally different from the undirected graph with symmetric distance metrics in all previous methods, and the shortest distance between template and subject on the directed graph is calculated. The allocated directed path can be thus utilized to better guide the registration by successively registering the subject through the intermediate templates one by one on the path towards the template. The proposed directed graph based solution can also be used in groupwise registration. Specifically, by building a minimum spanning arborescence (MSA) on the directed graph, the population center, i.e., a selected template, as well as the directed registration paths from all the rest of images to the population center, is determined simultaneously. The performance of directed graph based registration algorithm is demonstrated by the spatial normalization on both synthetic dataset and real brain MR images. It is shown that our method can achieve more accurate registration results than both the undirected graph based solution and the direct pairwise registration.     

Registration of MR and CT images for skull base surgery using point-like anatomical features.

We have developed a registration technique for combining magnetic resonance imaging (MRI) and computed tomography (CT) images of the skull base for use in surgical planning. The technique is based on user identification of point-like landmarks visible in both modalities. The combination of images involves a small amount of expert interaction, is relatively quick and preliminary evaluation indicates that it is accurate to within 1.5 mm. Registered or fused images can be viewed either on an image processing workstation, or fused images can be printed onto conventional film for convenience in clinical use. We present one patient in order to demonstrate the technique's indications and advantages.     

Influence of implementation parameters on registration of MR and SPECT brain images by maximization of mutual information.

Mutual-information maximization is one of the most popular algorithms for automatic image registration. However, many implementation issues have not been evaluated in a single, coherent context. METHODS: Twenty-one registrations between MR and SPECT brain images (8 patients) were achieved by mutual-information maximization with different implementation strategies. The results of a popular strategy were chosen as the standard. All other results were compared with the standard, and the statistics of misregistrations were computed. The registration speed, accuracy, precision, and success rate were assessed. RESULTS: Compared with trilinear interpolation, nearest-neighbor interpolation slightly sped the registration process, but with a lower success rate. The number of bins used to estimate the probability density function (pdf) affects the speed and robustness. Using fewer bins yielded a less robust registration. Adaptively changing the number of bins increased the registration speed and robustness. Simplex optimization increased the registration speed considerably, with a slightly degraded success rate. Simplex optimization with adaptive bin strategy improved the success rate and further decreased the registration time. Multiresolution optimization yielded a better success rate, with little effect on the accuracy and precision of registration. An increase in the number of resolution levels increased the success rate. Multisampling optimization also improved the success rate, but the results were less accurate and precise than those obtained with multiresolution optimization, with an increase in the number of levels decreasing the performance. Segmentation affected the registration speed and success rate. Because segmentation is problem specific, the effects were not conclusive. CONCLUSION: Different implementation strategies considerably affect the performance of automatic image registration by mutual-information maximization. On the basis of the experimental findings, we suggest that the best implementation strategy would include trilinear interpolation, adaptive change of the number of bins when estimating pdf, and exploitation of a simplex optimization algorithm with a multiresolution scheme.     

Hierarchical segmentation-assisted multimodal registration for MR brain images

Information theory-based metric such as mutual information (MI) is widely used as similarity measurement for multimodal registration. Nevertheless, this metric may lead to matching ambiguity for non-rigid registration. Moreover, maximization of MI alone does not necessarily produce an optimal solution. In this paper, we propose a segmentation-assisted similarity metric based on point-wise mutual information (PMI). This similarity metric, termed SPMI, enhances the registration accuracy by considering tissue classification probabilities as prior information, which is generated from an expectation maximization (EM) algorithm. Diffeomorphic demons is then adopted as the registration model and is optimized in a hierarchical framework (H-SPMI) based on different levels of anatomical structure as prior knowledge. The proposed method is evaluated using Brainweb synthetic data and clinical fMRI images. Both qualitative and quantitative assessment were performed as well as a sensitivity analysis to the segmentation error. Compared to the pure intensity-based approaches which only maximize mutual information, we show that the proposed algorithm provides significantly better accuracy on both synthetic and clinical data.     

Quantitative and clinical analysis of SPECT image registration for epilepsy studies.

This study reports quantitative measurements of the accuracy of two popular voxel-based registration algorithms--Woods' automated image registration algorithm and mutual information correlation--and compares these with conventional surface matching (SM) registration. METHODS: The registration algorithms were compared (15 different matches each) for (a) three-dimensional brain phantom images, (b) an ictal SPECT image from a patient with partial epilepsy matched to itself after modification to simulate changes in the cerebral blood flow pattern and (c) ictal/interictal SPECT images from 15 patients with partial epilepsy. Blinded visual ranking and localization of the subtraction images derived from the patient images were also performed. RESULTS: Both voxel-based registration methods were more accurate than SM registration (P < 0.0005). Automated image registration algorithm was more accurate than mutual information correlation for the computer-simulated ictal/interictal images and the patient ictal/interictal studies (P < 0.05). The subtraction SPECTs from SM were poorer in visual ranking more often than the voxel-based methods (P < 0.05). CONCLUSION: Voxel intensity-based registration algorithms provide significant improvement in ictal/interictal SPECT registration accuracy and result in a clinically detectable improvement in the subtraction SPECT images.     

Improved image registration by using fourier interpolation

This paper presents a technique for performing two-dimensional rigid-body image registration for functional magnetic resonance images (fMRI). The method provides accurate motion correction without local distortion. The approach is to perform the translation and rotation in the Fourier domain. For images sampled on a grid, such as in echo-planar imaging (EPI), one potential stumbling block to this approach is the computational burden of reconstruction, since the rotated image will no longer be on the Cartesian grid. A method of approximating rotations via local translations (shearing) is presented, which keeps the data on the Cartesian grid. This can provide quite accurate approximations with only a moderate amount of computation. A mean squared error (MSE) criterion is used for determining the registration parameters. This method is tested on several sets of simulated images and shown to have an accuracy ranging from 0.02 to 0.3 pixels for images with SNRs ranging from 100 to 10, respectively. They techniques have been tested on several sets of images. They are shown to work well on real subjects, for both echo-planar and spiral data acquisition schemes. The techniques are used in an activation study in which the subject moved his head during image collection. After use of this registration technique, the activation is easily detected.     

Respiratory motion in positron emission tomography/computed tomography: a review

The development of positron emission tomography/computed tomography (PET/CT) scanners has allowed not only straightforward but also synergistic fusion of anatomical and functional information. Combined PET/CT imaging yields an increased sensitivity and specificity beyond that which either of the 2 modalities possesses separately and therefore provides improved diagnostic accuracy. Because attenuation correction in PET is performed with the use of CT images, with CT used in the localization of disease, accurate spatial registration of PET and CT image sets is required. Correcting for the spatial mismatch caused by respiratory motion represents a particular challenge for the requisite registration accuracy as a result of differences in temporal resolution between the 2 modalities. This review provides a brief summary of the materials, methods, and results involved in multiple investigations of the correction for respiratory motion in PET/CT imaging of the thorax, with the goal of improving image quality and quantitation. Although some schemes use respiratory-phase data selection to exclude motion artifacts, others have adopted sophisticated software techniques. The various image artifacts associated with breathing motion are also described.     

PET-MR image fusion in soft tissue sarcoma: accuracy,reliability and practicality of interactive point-based and automated mutual information techniques

The fusion of functional positron emission tomography (PET) data with anatomical magnetic resonance (MR) or computed tomography images, using a variety of interactive and automated techniques, is becoming commonplace, with the technique of choice dependent on the specific application. The case of PET-MR image fusion in soft tissue is complicated by a lack of conspicuous anatomical features and deviation from the rigid-body model. Here we compare a point-based external marker technique with an automated mutual information algorithm and discuss the practicality, reliability and accuracy of each when applied to the study of soft tissue sarcoma. Ten subjects with suspected sarcoma in the knee, thigh, groin, flank or back underwent MR and PET scanning after the attachment of nine external fiducial markers. In the assessment of the point-based technique, three error measures were considered: fiducial localisation error (FLE), fiducial registration error (FRE) and target registration error (TRE). FLE, which represents the accuracy with which the fiducial points can be located, is related to the FRE minimised by the registration algorithm. The registration accuracy is best characterised by the TRE, which is the distance between corresponding points in each image space after registration. In the absence of salient features within the target volume, the TRE can be measured at fiducials excluded from the registration process. To assess the mutual information technique, PET data, acquired after physically removing the markers, were reconstructed in a variety of ways and registered with MR. Having applied the transform suggested by the algorithm to the PET scan acquired before the markers were removed, the residual distance between PET and MR marker-pairs could be measured. The manual point-based technique yielded the best results (RMS TRE =8.3 mm, max =22.4 mm, min =1.7 mm), performing better than the automated algorithm (RMS TRE =20.0 mm, max =30.5 mm, min =7.7 mm) when registering filtered back-projection PET images to MR. Image reconstruction with an iterative algorithm or registration of a composite emission-transmission image did not improve the overall accuracy of the registration process. We have demonstrated that, in this application, point-based PET-MR registration using external markers is practical, reliable and accurate to within approximately 5 mm towards the fiducial centroid. The automated algorithm did not perform as reliably or as accurately.     

Three-Dimensional MR image registration of the human brain

An automatic three-dimensional technique for registration of MR images of human brain is described. The algorithm was tested, using MR images of human brain, and was found to estimate angular offsets to within 0.5° and translational offsets to within about 1 pixel. The quality of final registration was evaluated by histogram analysis. The algorithm was found to be computationally efficient and robust.     

Improving image quality around subtle lung nodules by reducing artifacts in similar subtraction imaging

Similar subtraction imaging is useful for the detection of lung nodules; however, some artifacts on similar subtraction images reduce their utility. The authors attempted to improve the image quality of similar subtraction images by reducing artifacts caused by differences in image contrast and sharpness between two images used for similar subtraction imaging. Image contrast was adjusted using the histogram specification technique. The differences in image sharpness were compensated for using a pixel matching technique. The improvement in image quality was evaluated objectively based on the degree of artifacts and the contrast-to-noise ratio (CNR) of the lung nodules. The artifacts in similar subtraction images were reduced in 94% (17/18) of cases, and CNR was improved in 83% (15/18) of cases. The results indicate that the combination of histogram specification and pixel matching techniques is potentially useful in improving image quality in similar subtraction imaging.     

Improved least squares MR image reconstruction using estimates of k-space data consistency

This study describes a new approach to reconstruct data that has been corrupted by unfavorable magnetization evolution. In this new framework, images are reconstructed in a weighted least squares fashion using all available data and a measure of consistency determined from the data itself. The reconstruction scheme optimally balances uncertainties from noise error with those from data inconsistency, is compatible with methods that model signal corruption, and may be advantageous for more accurate and precise reconstruction with many least squares-based image estimation techniques including parallel imaging and constrained reconstruction/compressed sensing applications. Performance of the several variants of the algorithm tailored for fast spin echo and self-gated respiratory gating applications was evaluated in simulations, phantom experiments, and in vivo scans. The data consistency weighting technique substantially improved image quality and reduced noise as compared to traditional reconstruction approaches.     

Behaviors of cost functions in image registration between <Superscript>201</Superscript>Tl brain tumor single-photon emission computed tomography and magnetic resonance images

Objective  We studied the behaviors of cost functions in the registration of thallium-201 (²⁰¹Tl) brain tumor single-photon emission computed tomography (SPECT) and magnetic resonance (MR) images, as the similarity index of image positioning. Methods  A marker for image registration [technetium-99m (^99mTc) point source] was attached at three sites on the heads of 13 patients with brain tumor, from whom 42 sets of ^99mTc-²⁰¹Tl SPECT (the dual-isotope acquisition) and MR images were obtained. The ²⁰¹Tl SPECT and MR images were manually registered according to the markers. From the positions where the two images were registered, the position of the ²⁰¹Tl SPECT was moved to examine the behaviors of the three cost functions, i.e., ratio image uniformity (RIU), mutual information (MI), and normalized MI (NMI). Results  The cost functions MI and NMI reached the maximum at positions adjacent to those where the SPECT and MR images were manually registered. As for the accuracy of image registration in terms of the cost functions MI and NMI, on average, the images were accurately registered within 3° of rotation around the X-, Y-, and Z-axes, and within 1.5 mm (within 2 pixels), 3 mm (within 3 pixels), and 4 mm (within 1 slice) of translation to the X-, Y-, and Z-axes, respectively. In terms of rotation around the Z-axis, the cost function RIU reached the minimum at positions where the manual registration of the two images was substantially inadequate. Conclusions  The MI and NMI were suitable cost functions in the registration of ²⁰¹Tl SPECT and MR images. The behavior of the RIU, in contrast, was unstable, being unsuitable as an index of image registration.     

Fusion imaging of computed tomographic pulmonary angiography and SPECT ventilation/perfusion scintigraphy: initial experience and potential benefit

Purpose The objective of this study was to examine the feasibility of fusing ventilation and perfusion data from single-photon emission computed tomography (SPECT) ventilation perfusion (V/Q) scintigraphy together with computed tomographic pulmonary angiography (CTPA) data. We sought to determine the accuracy of this fusion process. In addition, we correlated the findings of this technique with the final clinical diagnosis. Methods Thirty consecutive patients (17 female, 13 male) who had undergone both CTPA and SPECT V/Q scintigraphy during their admission for investigation of potential pulmonary embolism were identified retrospectively. Image datasets from these two modalities were co-registered and fused using commercial software. Accuracy of the fusion process was determined subjectively by correlation between modalities of the anatomical boundaries and co-existent pleuro-parenchymal abnormalities. Results In all 30 cases, SPECT V/Q images were accurately fused with CTPA images. An automated registration algorithm was sufficient alone in 23 cases (77%). Additional linear z-axis scaling was applied in seven cases. There was accurate topographical co-localisation of vascular, parenchymal and pleural disease on the fused images. Nine patients who had positive CTPA performed as an initial investigation had co-localised perfusion defects on the subsequent fused CTPA/SPECT images. Three of the 11 V/Q scans initially reported as intermediate could be reinterpreted as low probability owing to co-localisation of defects with parenchymal or pleural pathology. Conclusion Accurate fusion of SPECT V/Q scintigraphy to CTPA images is possible. This technique may be clinically useful in patients who have non-diagnostic initial investigations or in whom corroborative imaging is sought.