Automatic multimodal 2D/3D breast image registration using biomechanical FEM models and intensity-based optimization

Due to their different physical origin, X-ray mammography and Magnetic Resonance Imaging (MRI) provide complementary diagnostic information. However, the correlation of their images is challenging due to differences in dimensionality, patient positioning and compression state of the breast. Our automated registration takes over part of the correlation task. The registration method is based on a biomechanical finite element model, which is used to simulate mammographic compression. The deformed MRI volume can be compared directly with the corresponding mammogram. The registration accuracy is determined by a number of patient-specific parameters. We optimize these parameters – e.g. breast rotation – using image similarity measures. The method was evaluated on 79 datasets from clinical routine. The mean target registration error was 13.2 mm in a fully automated setting. On basis of our results, we conclude that a completely automated registration of volume images with 2D mammograms is feasible. The registration accuracy is within the clinically relevant range and thus beneficial for multimodal diagnosis.     

Simulation of mammographic breast compression in 3D MR images using ICP-based B-spline deformation for multimodality breast cancer diagnosis

Purpose

   Multimodality mammography using conventional 2D mammography and dynamic contrast-enhanced 3D magnetic resonance imaging (DCE-MRI) is frequently performed for breast cancer detection and diagnosis. Combination of both imaging modalities requires superimposition of corresponding structures in mammograms and MR images. This task is challenging due to large differences in (1) dimensionality and spatial resolution, (2) variations in tissue contrast, as well as (3) differences in breast orientation and deformation during the image acquisition. A new method for multimodality breast image registration was developed and tested.

Methods

   Combined diagnosis of mammograms and MRI datasets was achieved by simulation of mammographic breast compression to overcome large differences in breast deformation. Surface information was extracted from the 3D MR image, and back-projection of the 2D breast contour in the mammogram was done. B-spline-based 3D/3D surface-based registration was then used to approximate mammographic breast compression. This breast deformation simulation was performed on 14 MRI datasets with 19 corresponding mammograms. The results were evaluated by comparison with distances between corresponding structures identified by an expert observer.

Results

   The evaluation revealed an average distance of 6.46 mm between corresponding structures, when an optimized initial alignment between both image datasets is performed. Without the optimization, the accuracy is 9.12 mm.

Conclusion

   A new surface-based method that approximates the mammographic deformation due to breast compression without using a specific complex model needed for finite-element-based methods was developed and tested with favorable results. The simulated compression can serve as foundation for a point-to-line correspondence between 2D mammograms and 3D MR image data.     

Respiratory motion compensation by model-based catheter tracking during EP procedures

In many cases, radio-frequency catheter ablation of the pulmonary veins attached to the left atrium still involves fluoroscopic image guidance. Two-dimensional X-ray navigation may also take advantage of overlay images derived from static pre-operative 3D volumetric data to add anatomical details otherwise not visible under X-ray. Unfortunately, respiratory motion may impair the utility of static overlay images for catheter navigation. We developed a novel approach for image-based 3D motion estimation and compensation as a solution to this problem. It is based on 3D catheter tracking which, in turn, relies on 2D/3D registration. To this end, a bi-plane C-arm system is used to take X-ray images of a special circumferential mapping catheter from two directions. In the first step of the method, a 3D model of the device is reconstructed. Three-dimensional respiratory motion at the site of ablation is then estimated by tracking the reconstructed catheter model in 3D based on bi-plane fluoroscopy. Phantom data and clinical data were used to assess model-based catheter tracking. Our phantom experiments yielded an average 2D tracking error of 1.4 mm and an average 3D tracking error of 1.1 mm. Our evaluation of clinical data sets comprised 469 bi-plane fluoroscopy frames (938 monoplane fluoroscopy frames). We observed an average 2D tracking error of 1.0 ± 0.4 mm and an average 3D tracking error of 0.8 ± 0.5 mm. These results demonstrate that model-based motion-compensation based on 2D/3D registration is both feasible and accurate.     

Statistical coronary motion models for 2D + t/3D registration of X-ray coronary angiography and CTA

Accurate alignment of intra-operative X-ray coronary angiography (XA) and pre-operative cardiac CT angiography (CTA) may improve procedural success rates of minimally invasive coronary interventions for patients with chronic total occlusions. It was previously shown that incorporating patient specific coronary motion extracted from 4D CTA increases the robustness of the alignment. However, pre-operative CTA is often acquired with gating at end-diastole, in which case patient specific motion is not available.For such cases, we investigate the possibility of using population based coronary motion models to provide constraints for the 2D + t/3D registration. We propose a methodology for building statistical motion models of the coronary arteries from a training population of 4D CTA datasets. We compare the 2D + t/3D registration performance of the proposed statistical models with other motion estimates, including the patient specific motion extracted from 4D CTA, the mean motion of a population, the predicted motion based on the cardiac shape.The coronary motion models, constructed on a training set of 150 patients, had a generalization accuracy of 1 mm root mean square point-to-point distance. Their 2D + t/3D registration accuracy on one cardiac cycle of 12 monoplane XA sequences was similar to, if not better than, the 4D CTA based motion, irrespective of which respiratory model and which feature based 2D/3D distance metric was used. The resulting model based coronary motion estimate showed good applicability for registration of a subsequent cardiac cycle.     

MRI to X-ray mammography registration using a volume-preserving affine transformation

X-ray mammography is routinely used in national screening programmes and as a clinical diagnostic tool. Magnetic Resonance Imaging (MRI) is commonly used as a complementary modality, providing functional information about the breast and a 3D image that can overcome ambiguities caused by the superimposition of fibro-glandular structures associated with X-ray imaging. Relating findings between these modalities is a challenging task however, due to the different imaging processes involved and the large deformation that the breast undergoes. In this work we present a registration method to determine spatial correspondence between pairs of MR and X-ray images of the breast, that is targeted for clinical use. We propose a generic registration framework which incorporates a volume-preserving affine transformation model and validate its performance using routinely acquired clinical data. Experiments on simulated mammograms from 8 volunteers produced a mean registration error of 3.8±1.6mm for a mean of 12 manually identified landmarks per volume. When validated using 57 lesions identified on routine clinical CC and MLO mammograms (n=113 registration tasks) from 49 subjects the median registration error was 13.1mm. When applied to the registration of an MR image to CC and MLO mammograms of a patient with a localisation clip, the mean error was 8.9mm. The results indicate that an intensity based registration algorithm, using a relatively simple transformation model, can provide radiologists with a clinically useful tool for breast cancer diagnosis.     

A method of 2D/3D registration of a statistical mouse atlas with a planar X-ray projection and an optical photo

The development of sophisticated and high throughput whole body small animal imaging technologies has created a need for improved image analysis and increased automation. The registration of a digital mouse atlas to individual images is a prerequisite for automated organ segmentation and uptake quantification. This paper presents a fully-automatic method for registering a statistical mouse atlas with individual subjects based on an anterior–posterior X-ray projection and a lateral optical photo of the mouse silhouette. The mouse atlas was trained as a statistical shape model based on 83 organ-segmented micro-CT images. For registration, a hierarchical approach is applied which first registers high contrast organs, and then estimates low contrast organs based on the registered high contrast organs. To register the high contrast organs, a 2D-registration-back-projection strategy is used that deforms the 3D atlas based on the 2D registrations of the atlas projections. For validation, this method was evaluated using 55 subjects of preclinical mouse studies. The results showed that this method can compensate for moderate variations of animal postures and organ anatomy. Two different metrics, the Dice coefficient and the average surface distance, were used to assess the registration accuracy of major organs. The Dice coefficients vary from 0.31 ± 0.16 for the spleen to 0.88 ± 0.03 for the whole body, and the average surface distance varies from 0.54 ± 0.06 mm for the lungs to 0.85 ± 0.10 mm for the skin. The method was compared with a direct 3D deformation optimization (without 2D-registration-back-projection) and a single-subject atlas registration (instead of using the statistical atlas). The comparison revealed that the 2D-registration-back-projection strategy significantly improved the registration accuracy, and the use of the statistical mouse atlas led to more plausible organ shapes than the single-subject atlas. This method was also tested with shoulder xenograft tumor-bearing mice, and the results showed that the registration accuracy of most organs was not significantly affected by the presence of shoulder tumors, except for the lungs and the spleen.     

Organ-focused mutual information for nonrigid multimodal registration of liver CT and Gd–EOB–DTPA-enhanced MRI

Accurate detection of liver lesions is of great importance in hepatic surgery planning. Recent studies have shown that the detection rate of liver lesions is significantly higher in gadoxetic acid-enhanced magnetic resonance imaging (Gd–EOB–DTPA-enhanced MRI) than in contrast-enhanced portal-phase computed tomography (CT); however, the latter remains essential because of its high specificity, good performance in estimating liver volumes and better vessel visibility. To characterize liver lesions using both the above image modalities, we propose a multimodal nonrigid registration framework using organ-focused mutual information (OF-MI). This proposal tries to improve mutual information (MI) based registration by adding spatial information, benefiting from the availability of expert liver segmentation in clinical protocols. The incorporation of an additional information channel containing liver segmentation information was studied. A dataset of real clinical images and simulated images was used in the validation process. A Gd–EOB–DTPA-enhanced MRI simulation framework is presented. To evaluate results, warping index errors were calculated for the simulated data, and landmark-based and surface-based errors were calculated for the real data. An improvement of the registration accuracy for OF-MI as compared with MI was found for both simulated and real datasets. Statistical significance of the difference was tested and confirmed in the simulated dataset (p < 0.01).     

Single-image super-resolution of brain MR images using overcomplete dictionaries

Resolution in Magnetic Resonance (MR) is limited by diverse physical, technological and economical considerations. In conventional medical practice, resolution enhancement is usually performed with bicubic or B-spline interpolations, strongly affecting the accuracy of subsequent processing steps such as segmentation or registration. This paper presents a sparse-based super-resolution method, adapted for easily including prior knowledge, which couples up high and low frequency information so that a high-resolution version of a low-resolution brain MR image is generated. The proposed approach includes a whole-image multi-scale edge analysis and a dimensionality reduction scheme, which results in a remarkable improvement of the computational speed and accuracy, taking nearly 26 min to generate a complete 3D high-resolution reconstruction. The method was validated by comparing interpolated and reconstructed versions of 29 MR brain volumes with the original images, acquired in a 3T scanner, obtaining a reduction of 70% in the root mean squared error, an increment of 10.3 dB in the peak signal-to-noise ratio, and an agreement of 85% in the binary gray matter segmentations. The proposed method is shown to outperform a recent state-of-the-art algorithm, suggesting a substantial impact in voxel-based morphometry studies.     

A sensitivity analysis on 3D velocity reconstruction from multiple registered echo Doppler views

We present a new method for reconstructing a 3D + t velocity field from multiple 3D + t colour Doppler images. Our technique reconstructs 3D velocity vectors from registered multiple standard 3D colour Doppler views, each of which contains a 1D projection of the blood velocity. Reconstruction is based on a scalable patch-wise Least Mean Squares approach, and a continuous velocity field is achieved by using a B-spline grid.We carry out a sensitivity analysis of clinically relevant parameters which affect the accuracy of the reconstruction, including the impact of noise, view angles and registration errors, using simulated data. A realistic simulation framework is achieved by a novel noise model to represent variations in colour Doppler images based on multiscale additive Gaussian noise. Simulations show that, if the Target Registration Error <2.5 mm, view angles are >20° and the standard deviation of noise in the input data is <10 cm/s, the reconstructed velocity field presents visually plausible flow patterns and mean error in flow rate is approximately 10% compared to 2D + t Flow MRI. These results are verified by reconstructing 3D velocity on three healthy volunteers. The technique is applied to reconstruct 3D flow on three paediatric patients showing promising results for clinical application.     

A supervised learning framework of statistical shape and probability priors for automatic prostate segmentation in ultrasound images

Prostate segmentation aids in prostate volume estimation, multi-modal image registration, and to create patient specific anatomical models for surgical planning and image guided biopsies. However, manual segmentation is time consuming and suffers from inter-and intra-observer variabilities. Low contrast images of trans rectal ultrasound and presence of imaging artifacts like speckle, micro-calcifications, and shadow regions hinder computer aided automatic or semi-automatic prostate segmentation. In this paper, we propose a prostate segmentation approach based on building multiple mean parametric models derived from principal component analysis of shape and posterior probabilities in a multi-resolution framework. The model parameters are then modified with the prior knowledge of the optimization space to achieve optimal prostate segmentation. In contrast to traditional statistical models of shape and intensity priors, we use posterior probabilities of the prostate region determined from random forest classification to build our appearance model, initialize and propagate our model. Furthermore, multiple mean models derived from spectral clustering of combined shape and appearance parameters are applied in parallel to improve segmentation accuracies. The proposed method achieves mean Dice similarity coefficient value of 0.91 ± 0.09 for 126 images containing 40 images from the apex, 40 images from the base and 46 images from central regions in a leave-one-patient-out validation framework. The mean segmentation time of the procedure is 0.67 ± 0.02 s.     

Persistent and automatic intraoperative 3D digitization of surfaces under dynamic magnifications of an operating microscope

One of the major challenges impeding advancement in image-guided surgical (IGS) systems is the soft-tissue deformation during surgical procedures. These deformations reduce the utility of the patient’s preoperative images and may produce inaccuracies in the application of preoperative surgical plans. Solutions to compensate for the tissue deformations include the acquisition of intraoperative tomographic images of the whole organ for direct displacement measurement and techniques that combines intraoperative organ surface measurements with computational biomechanical models to predict subsurface displacements. The later solution has the advantage of being less expensive and amenable to surgical workflow. Several modalities such as textured laser scanners, conoscopic holography, and stereo-pair cameras have been proposed for the intraoperative 3D estimation of organ surfaces to drive patient-specific biomechanical models for the intraoperative update of preoperative images. Though each modality has its respective advantages and disadvantages, stereo-pair camera approaches used within a standard operating microscope is the focus of this article. A new method that permits the automatic and near real-time estimation of 3D surfaces (at 1 Hz) under varying magnifications of the operating microscope is proposed. This method has been evaluated on a CAD phantom object and on full-length neurosurgery video sequences (∼1 h) acquired intraoperatively by the proposed stereovision system. To the best of our knowledge, this type of validation study on full-length brain tumor surgery videos has not been done before. The method for estimating the unknown magnification factor of the operating microscope achieves accuracy within 0.02 of the theoretical value on a CAD phantom and within 0.06 on 4 clinical videos of the entire brain tumor surgery. When compared to a laser range scanner, the proposed method for reconstructing 3D surfaces intraoperatively achieves root mean square errors (surface-to-surface distance) in the 0.28–0.81 mm range on the phantom object and in the 0.54–1.35 mm range on 4 clinical cases. The digitization accuracy of the presented stereovision methods indicate that the operating microscope can be used to deliver the persistent intraoperative input required by computational biomechanical models to update the patient’s preoperative images and facilitate active surgical guidance.     

Serum MMPs and TIMPs: May be predictors of breast carcinogenesis?

BackgroundThe involvement of matrix metalloproteinases (MMPs) and their tissue inhibitors (TIMPs) in breast cancer has been documented on palpable lesions. This study aims to assess serum MMP1, MMP-2, TIMP-1, and TIMP-2 in atypical ductal hyperplasia (ADH), lobular neoplasia (LN), ductal carcinoma in situ (DCIS) and invasive ductal carcinoma (IDC) specifically in non-palpable mammographic breast lesions.MethodsOn women with benign (n = 65), precursor [ADH (n = 18) and LN (n = 15)], preinvasive [DCIS (n = 32)] and invasive [IDC (n = 28)] lesions the serum concentrations of MMP-1, MMP-2, TIMP-1, TIMP-2, TPS, and TPA were determined with immunoenzymatic assays. All women had non-palpable mammographic breast lesions of less than 10 mm in diameter, as estimated on the mammographic views. Statistical analysis followed.ResultsTIMP-2 serum concentrations were positively associated with the severity of the lesion. On the contrary, MMP-2 levels were marginally negatively associated with severity; as evident, the MMP-2/TIMP-2 ratio significantly decreased along with severity. Regarding TIMP-1, TPS, TPA, and TIMP-1/TIMP-2, no significant associations were demonstrated. MMP-2 and the MMP-2/TIMP-2 ratio were significantly higher in the LN subgroup versus the ADH subgroup.ConclusionTIMP-2 and MMP-2/TIMP-2 ratio may exhibit meaningful changes along with progression of lesions. Extracellular cell matrix remodeling in ductal and lobular lesions may follow distinct patterns.     

Chest wall segmentation in automated 3D breast ultrasound scans

In this paper, we present an automatic method to segment the chest wall in automated 3D breast ultrasound images. Determining the location of the chest wall in automated 3D breast ultrasound images is necessary in computer-aided detection systems to remove automatically detected cancer candidates beyond the chest wall and it can be of great help for inter- and intra-modal image registration. We show that the visible part of the chest wall in an automated 3D breast ultrasound image can be accurately modeled by a cylinder. We fit the surface of our cylinder model to a set of automatically detected rib-surface points. The detection of the rib-surface points is done by a classifier using features representing local image intensity patterns and presence of rib shadows. Due to attenuation of the ultrasound signal, a clear shadow is visible behind the ribs. Evaluation of our segmentation method is done by computing the distance of manually annotated rib points to the surface of the automatically detected chest wall. We examined the performance on images obtained with the two most common 3D breast ultrasound devices in the market. In a dataset of 142 images, the average mean distance of the annotated points to the segmented chest wall was 5.59 ± 3.08 mm.     

Cortical surface shift estimation using stereovision and optical flow motion tracking via projection image registration

Stereovision is an important intraoperative imaging technique that captures the exposed parenchymal surface noninvasively during open cranial surgery. Estimating cortical surface shift efficiently and accurately is critical to compensate for brain deformation in the operating room (OR). In this study, we present an automatic and robust registration technique based on optical flow (OF) motion tracking to compensate for cortical surface displacement throughout surgery. Stereo images of the cortical surface were acquired at multiple time points after dural opening to reconstruct three-dimensional (3D) texture intensity-encoded cortical surfaces. A local coordinate system was established with its z-axis parallel to the average surface normal direction of the reconstructed cortical surface immediately after dural opening in order to produce two-dimensional (2D) projection images. A dense displacement field between the two projection images was determined directly from OF motion tracking without the need for feature identification or tracking. The starting and end points of the displacement vectors on the two cortical surfaces were then obtained following spatial mapping inversion to produce the full 3D displacement of the exposed cortical surface. We evaluated the technique with images obtained from digital phantoms and 18 surgical cases – 10 of which involved independent measurements of feature locations acquired with a tracked stylus for accuracy comparisons, and 8 others of which 4 involved stereo image acquisitions at three or more time points during surgery to illustrate utility throughout a procedure. Results from the digital phantom images were very accurate (0.05 pixels). In the 10 surgical cases with independently digitized point locations, the average agreement between feature coordinates derived from the cortical surface reconstructions was 1.7–2.1 mm relative to those determined with the tracked stylus probe. The agreement in feature displacement tracking was also comparable to tracked probe data (difference in displacement magnitude was <1 mm on average). The average magnitude of cortical surface displacement was 7.9 ± 5.7 mm (range 0.3–24.4 mm) in all patient cases with the displacement components along gravity being 5.2 ± 6.0 mm relative to the lateral movement of 2.4 ± 1.6 mm. Thus, our technique appears to be sufficiently accurate and computationally efficiency (typically ∼15 s), for applications in the OR.     

An analysis of image texture,tumor location,and MGMT promoter methylation in glioblastoma using magnetic resonance imaging

In glioblastoma (GBM), promoter methylation of the DNA repair gene O₆-methylguanine-DNA methyltransferase (MGMT) is associated with benefit from chemotherapy. Correlations between MGMT promoter methylation and visually assessed imaging features on magnetic resonance (MR) have been reported suggesting that noninvasive detection of MGMT methylation status might be possible. Our study assessed whether MGMT methylation status in GBM could be predicted using MR imaging. We conducted a retrospective analysis of MR images in patients with newly diagnosed GBM. Tumor texture was assessed by two methods. First, we analyzed texture by expert consensus describing the tumor borders, presence or absence of cysts, pattern of enhancement, and appearance of tumor signal in T2-weighted images. Then, we applied space–frequency texture analysis based on the S-transform. Tumor location within the brain was determined using automatized image registration and segmentation techniques. Their association with MGMT methylation was analyzed. We confirmed that ring enhancement assessed visually is significantly associated with unmethylated MGMT promoter status (P = 0.006). Texture features on T2-weighted images assessed by the space–frequency analysis were significantly different between methylated and unmethylated cases (P < 0.05). However, blinded classification of MGMT promoter methylation status reached an accuracy of only 71%. There were no significant differences in the locations of methylated and unmethylated GBM tumors. Our results provide further evidence that individual MR features are associated with MGMT methylation but better algorithms for predicting methylation status are needed. The relevance of this study lies on the application of novel techniques for the analysis of anatomical MR images of patients with GBM allowing the evaluation of subtleties not seen by an observer and facilitating the standardization of the methods, decreasing the potential for interobserver bias.     

Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations

We present an algorithm for automatic elastic registration of three-dimensional (3D) medical images. Our algorithm initially recovers the global spatial mismatch between the reference and floating images, followed by hierarchical octree-based subdivision of the reference image and independent registration of the floating image with the individual subvolumes of the reference image at each hierarchical level. Global as well as local registrations use the six-parameter full rigid-body transformation model and are based on maximization of normalized mutual information (NMI). To ensure robustness of the subvolume registration with low voxel counts, we calculate NMI using a combination of current and prior mutual histograms. To generate a smooth deformation field, we perform direct interpolation of six-parameter rigid-body subvolume transformations obtained at the last subdivision level. Our interpolation scheme involves scalar interpolation of the 3D translations and quaternion interpolation of the 3D rotational pose. We analyzed the performance of our algorithm through experiments involving registration of synthetically deformed computed tomography (CT) images. Our algorithm is general and can be applied to image pairs of any two modalities of most organs. We have demonstrated successful registration of clinical whole-body CT and positron emission tomography (PET) images using this algorithm. The registration accuracy for this application was evaluated, based on validation using expert-identified anatomical landmarks in 15 CT-PET image pairs. The algorithm's performance was comparable to the average accuracy observed for three expert-determined registrations in the same 15 image pairs.