几何模型匹配乳腺头足位与内外斜位X线片所示病灶

目的 观察几何模型(GM)匹配乳腺头足(CC)位与内外斜(MLO)位X线片所示病灶的价值。方法 回顾性分析493例接受乳腺CC位和MLO位X线摄影的乳腺病灶患者,共598个乳腺病灶,包括499个钙化灶和99个肿块。构建GM用于匹配CC与MLO位片所示乳腺病灶,再以环形法(AB)和直线法(SS)进行对比,分别计算匹配误差,包括GM匹配误差、AB径向误差及SS轴向误差;分析GM对CC及MLO位图像中同一病灶的匹配性能,评价其应用价值。结果 GM对乳腺钙化灶和肿块的匹配误差分别为2.85(1.45,5.08)及3.70(1.35,6.25)mm,差异无统计学意义(Z=-1.344,P=0.179)。对乳腺上部病灶,AB匹配的径向误差和SS匹配的轴向误差均大于下部病灶(P均<0.001);对乳腺外侧病灶,AB的径向误差和SS的轴向误差均大于内侧病灶(P均<0.05)。GM、AB及SS间匹配误差整体差异有统计学意义(H=93.012,P<0.001);两两比较差异均有统计学意义(P均<0.05),GM匹配性能明显优于AB和SS。GM匹配误差与摄片时乳腺压迫厚度无明显相关性...     

Breast lesion co-localisation between X-ray and MR images using finite element modelling

This paper presents a novel X-ray and MR image registration technique based on individual-specific biomechanical finite element (FE) models of the breasts. Information from 3D magnetic resonance (MR) images was registered to X-ray mammographic images using non-linear FE models subject to contact mechanics constraints to simulate the large compressive deformations between the two imaging modalities. A physics-based perspective ray-casting algorithm was used to generate 2D pseudo-X-ray projections of the FE-warped 3D MR images. Unknown input parameters to the FE models, such as the location and orientation of the compression plates, were optimised to provide the best match between the pseudo and clinical X-ray images. The methods were validated using images taken before and during compression of a breast-shaped phantom, for which 12 inclusions were tracked between imaging modalities. These methods were then applied to X-ray and MR images from six breast cancer patients. Error measures (such as centroid and surface distances) of segmented tumours in simulated and actual X-ray mammograms were used to assess the accuracy of the methods. Sensitivity analysis of the lesion co-localisation accuracy to rotation about the anterior–posterior axis was then performed. For 10 of the 12 X-ray mammograms, lesion localisation accuracies of 14 mm and less were achieved. This analysis on the rotation about the anterior–posterior axis indicated that, in cases where the lesion lies in the plane parallel to the mammographic compression plates, that cuts through the nipple, such rotations have relatively minor effects. This has important implications for clinical applicability of this multi-modality lesion registration technique, which will aid in the diagnosis and treatment of breast cancer.     

Deformable mapping using biomechanical models to relate corresponding lesions in digital breast tomosynthesis and automated breast ultrasound images

This work investigates the application of a deformable localization/mapping method to register lesions between the digital breast tomosynthesis (DBT) craniocaudal (CC) and mediolateral oblique (MLO) views and automated breast ultrasound (ABUS) images. This method was initially validated using compressible breast phantoms. This methodology was applied to 7 patient data sets containing 9 lesions. The automated deformable mapping algorithm uses finite element modeling and analysis to determine corresponding lesions based on the distance between their centers of mass (d_COM) in the deformed DBT model and the reference ABUS model. This technique shows that location information based on external fiducial markers is helpful in the improvement of registration results. However, use of external markers are not required for deformable registration results described by this methodology. For DBT (CC view) mapped to ABUS, the mean d_COM was 14.9 ± 6.8 mm based on 9 lesions using 6 markers in deformable analysis. For DBT (MLO view) mapped to ABUS, the mean d_COM was 13.7 ± 6.8 mm based on 8 lesions using 6 markers in analysis. Both DBT views registered to ABUS lesions showed statistically significant improvements (p ≤ 0.05) in registration using the deformable technique in comparison to a rigid registration. Application of this methodology could help improve a radiologist's characterization and accuracy in relating corresponding lesions between DBT and ABUS image datasets, especially for cases of high breast densities and multiple masses.     

Simulation of mammographic breast compression in 3D MR images using ICP-based B-spline deformation for multimodality breast cancer diagnosis

Purpose

   Multimodality mammography using conventional 2D mammography and dynamic contrast-enhanced 3D magnetic resonance imaging (DCE-MRI) is frequently performed for breast cancer detection and diagnosis. Combination of both imaging modalities requires superimposition of corresponding structures in mammograms and MR images. This task is challenging due to large differences in (1) dimensionality and spatial resolution, (2) variations in tissue contrast, as well as (3) differences in breast orientation and deformation during the image acquisition. A new method for multimodality breast image registration was developed and tested.

Methods

   Combined diagnosis of mammograms and MRI datasets was achieved by simulation of mammographic breast compression to overcome large differences in breast deformation. Surface information was extracted from the 3D MR image, and back-projection of the 2D breast contour in the mammogram was done. B-spline-based 3D/3D surface-based registration was then used to approximate mammographic breast compression. This breast deformation simulation was performed on 14 MRI datasets with 19 corresponding mammograms. The results were evaluated by comparison with distances between corresponding structures identified by an expert observer.

Results

   The evaluation revealed an average distance of 6.46 mm between corresponding structures, when an optimized initial alignment between both image datasets is performed. Without the optimization, the accuracy is 9.12 mm.

Conclusion

   A new surface-based method that approximates the mammographic deformation due to breast compression without using a specific complex model needed for finite-element-based methods was developed and tested with favorable results. The simulated compression can serve as foundation for a point-to-line correspondence between 2D mammograms and 3D MR image data.     

Automatic multimodal 2D/3D breast image registration using biomechanical FEM models and intensity-based optimization

Due to their different physical origin, X-ray mammography and Magnetic Resonance Imaging (MRI) provide complementary diagnostic information. However, the correlation of their images is challenging due to differences in dimensionality, patient positioning and compression state of the breast. Our automated registration takes over part of the correlation task. The registration method is based on a biomechanical finite element model, which is used to simulate mammographic compression. The deformed MRI volume can be compared directly with the corresponding mammogram. The registration accuracy is determined by a number of patient-specific parameters. We optimize these parameters – e.g. breast rotation – using image similarity measures. The method was evaluated on 79 datasets from clinical routine. The mean target registration error was 13.2 mm in a fully automated setting. On basis of our results, we conclude that a completely automated registration of volume images with 2D mammograms is feasible. The registration accuracy is within the clinically relevant range and thus beneficial for multimodal diagnosis.     

Deformable Mapping Method to Relate Lesions in Dedicated Breast CT Images to Those in Automated Breast Ultrasound and Digital Breast Tomosynthesis Images

This work demonstrates the potential for using a deformable mapping method to register lesions between dedicated breast computed tomography (bCT) and both automated breast ultrasound (ABUS) and digital breast tomosynthesis (DBT) images (craniocaudal [CC] and mediolateral oblique [MLO] views). Two multi-modality breast phantoms with external fiducial markers attached were imaged by the three modalities. The DBT MLO view was excluded for the second phantom. The automated deformable mapping algorithm uses biomechanical modeling to determine corresponding lesions based on distances between their centers of mass (d_COM) in the deformed bCT model and the reference model (DBT or ABUS). For bCT to ABUS, the mean d_COM was 5.2 ± 2.6 mm. For bCT to DBT (CC), the mean d_COM was 5.1 ± 2.4 mm. For bCT to DBT (MLO), the mean d_COM was 4.7 ± 2.5 mm. This application could help improve a radiologist's efficiency and accuracy in breast lesion characterization, using multiple imaging modalities.     

Fusion of contrast-enhanced breast MR and mammographic imaging data

Increasing use is being made of Gd-DTPA contrast-enhanced magnetic resonance imaging for breast cancer assessment since it provides 3D functional information via pharmacokinetic interaction between contrast agent and tumour vascularity, and because it is applicable to women of all ages as well as patients with post-operative scarring. Contrast-enhanced MRI (CE-MRI) is complementary to conventional X-ray mammography, since it is a relatively low-resolution functional counterpart of a comparatively high-resolution 2D structural representation. However, despite the additional information provided by MRI, mammography is still an extremely important diagnostic imaging modality, particularly for several common conditions such as ductal carcinoma in situ (DCIS) where it has been shown that there is a strong correlation between microcalcification clusters and malignancy. Pathological indicators such as calcifications and fine spiculations are not visible in CE-MRI and therefore there is clinical and diagnostic value in fusing the high-resolution structural information available from mammography with the functional data acquired from MRI imaging. This paper presents a novel data fusion technique whereby medial-lateral oblique (MLO) and cranial-caudal (CC) mammograms (2D data) are registered to 3D contrast-enhanced MRI volumes. We utilise a combination of pharmacokinetic modelling, projection geometry, wavelet-based landmark detection and thin-plate spline non-rigid 'warping' to transform the coordinates of regions of interest (ROIs) from the 2D mammograms to the spatial reference frame of the contrast-enhanced MRI volume. Of key importance is the use of a flexible wavelet-based feature extraction technique that enables feature correspondences to be robustly determined between the very different image characteristics of X-ray mammography and MRI. An evaluation of the fusion framework is demonstrated with a series of clinical cases and a total of 14 patient examples.     

Registration of 3D trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking

Two-dimensional (2D) X-ray imaging is the dominant imaging modality for cardiac interventions. However, the use of X-ray fluoroscopy alone is inadequate for the guidance of procedures that require soft-tissue information, for example, the treatment of structural heart disease. The recent availability of three-dimensional (3D) trans-esophageal echocardiography (TEE) provides cardiologists with real-time 3D imaging of cardiac anatomy. Increasingly X-ray imaging is now supported by using intra-procedure 3D TEE imaging. We hypothesize that the real-time co-registration and visualization of 3D TEE and X-ray fluoroscopy data will provide a powerful guidance tool for cardiologists. In this paper, we propose a novel, robust and efficient method for performing this registration. The major advantage of our method is that it does not rely on any additional tracking hardware and therefore can be deployed straightforwardly into any interventional laboratory. Our method consists of an image-based TEE probe localization algorithm and a calibration procedure. While the calibration needs to be done only once, the GPU-accelerated registration takes approximately from 2 to 15 s to complete depending on the number of X-ray images used in the registration and the image resolution. The accuracy of our method was assessed using a realistic heart phantom. The target registration error (TRE) for the heart phantom was less than 2 mm. In addition, we assess the accuracy and the clinical feasibility of our method using five patient datasets, two of which were acquired from cardiac electrophysiology procedures and three from trans-catheter aortic valve implantation procedures. The registration results showed our technique had mean registration errors of 1.5-4.2 mm and 95% capture range of 8.7-11.4 mm in terms of TRE.     

Learning Non-rigid Deformations for Robust,Constrained Point-based Registration in Image-Guided MR-TRUS Prostate Intervention

Accurate and robust non-rigid registration of pre-procedure magnetic resonance (MR) imaging to intra-procedure trans-rectal ultrasound (TRUS) is critical for image-guided biopsies of prostate cancer. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. TRUS-guided biopsy is the current clinical standard for prostate cancer diagnosis and assessment. State-of-the-art, clinical MR-TRUS image fusion relies upon semi-automated segmentations of the prostate in both the MR and the TRUS images to perform non-rigid surface-based registration of the gland. Segmentation of the prostate in TRUS imaging is itself a challenging task and prone to high variability. These segmentation errors can lead to poor registration and subsequently poor localization of biopsy targets, which may result in false-negative cancer detection. In this paper, we present a non-rigid surface registration approach to MR-TRUS fusion based on a statistical deformation model (SDM) of intra-procedural deformations derived from clinical training data. Synthetic validation experiments quantifying registration volume of interest overlaps of the PI-RADS parcellation standard and tests using clinical landmark data demonstrate that our use of an SDM for registration, with median target registration error of 2.98 mm, is significantly more accurate than the current clinical method. Furthermore, we show that the low-dimensional SDM registration results are robust to segmentation errors that are not uncommon in clinical TRUS data.     

Towards improved breast mass detection using dual-view mammogram matching

Breast cancer screening benefits from the visual analysis of multiple views of routine mammograms. As for clinical practice, computer-aided diagnosis (CAD) systems could be enhanced by integrating multi-view information. In this work, we propose a new multi-tasking framework that combines craniocaudal (CC) and mediolateral-oblique (MLO) mammograms for automatic breast mass detection. Rather than addressing mass recognition only, we exploit multi-tasking properties of deep networks to jointly learn mass matching and classification, towards better detection performance. Specifically, we propose a unified Siamese network that combines patch-level mass/non-mass classification and dual-view mass matching to take full advantage of multi-view information. This model is exploited in a full image detection pipeline based on You-Only-Look-Once (YOLO) region proposals. We carry out exhaustive experiments to highlight the contribution of dual-view matching for both patch-level classification and examination-level detection scenarios. Results demonstrate that mass matching highly improves the full-pipeline detection performance by outperforming conventional single-task schemes with 94.78% as Area Under the Curve (AUC) score and a classification accuracy of 0.8791. Interestingly, mass classification also improves the performance of mass matching, which proves the complementarity of both tasks. Our method further guides clinicians by providing accurate dual-view mass correspondences, which suggests that it could act as a relevant second opinion for mammogram interpretation and breast cancer diagnosis.     

PCM乳腺摄影系统与CR系统临床应用的初步比较

目的 探讨相位对比乳腺X线摄影系统(PCM)与CR系统在乳腺实体成像质量上的差异.方法 24例患者,患侧乳腺在PCM系统或CR系统(随机选择)进行轴位(CC)或侧斜位(MLO)投照,在另一系统进行另一体位投照.30名正常体检者,一侧乳腺在PCM系统或CR系统上(随机选择)进行CC及MLO投照,另一侧乳腺在另一系统进行两个体位的投照.对所得影像进行解剖及病变细节显示情况的评分,分析PCM与CR系统对于乳腺实体摄影成像质量的差异.结果 病变组:PCM系统在肿块边缘清晰度、内部结构显示及钙化边缘清晰度方面均明显优于CR系统(P=0.0003);正常体检组:正常乳腺双侧对照:PCM系统在锐利度、对比度和噪声方面均明显优于CR系统(P<0.05).结论 PCM系统与CR系统在乳腺实体成像质量上存在显著差异,PCM系统的图像基本质量及对病变细节的显示情况明显优于CR系统.     

Accuracy analysis in MRI-guided robotic prostate biopsy

Purpose

To assess retrospectively the clinical accuracy of an magnetic resonance imaging-guided robotic prostate biopsy system that has been used in the US National Cancer Institute for over 6 years.

Methods

Series of 2D transverse volumetric MR image slices of the prostate both pre (high-resolution T2-weighted)- and post (low-resolution)- needle insertions were used to evaluate biopsy accuracy. A three-stage registration algorithm consisting of an initial two-step rigid registration followed by a B-spline deformable alignment was developed to capture prostate motion during biopsy. The target displacement (distance between planned and actual biopsy target), needle placement error (distance from planned biopsy target to needle trajectory), and biopsy error (distance from actual biopsy target to needle trajectory) were calculated as accuracy assessment.

Results

A total of 90 biopsies from 24 patients were studied. The registrations were validated by checking prostate contour alignment using image overlay, and the results were accurate to within 2 mm. The mean target displacement, needle placement error, and clinical biopsy error were 5.2, 2.5, and 4.3 mm, respectively.

Conclusion

The biopsy error reported suggests that quantitative imaging techniques for prostate registration and motion compensation may improve prostate biopsy targeting accuracy.

     

乳腺癌的钼靶X线诊断

目的；探讨乳腺癌的x线征象．提高钼靶x线对乳腺癌诊断准确性。方法：87例乳腺癌患者均行x线检查，检查体位常规采用轴位（CC位）、侧斜位（MLO位）。结果；87例患者中，病变位于左侧51例，右侧35例，双乳1例,1例为多中心性病灶，其余均为单发病灶。60例肿块直径小于临床触诊。结论：钼靶x线摄影是诊断乳腺癌的首选检查方法，结合乳腺癌的各种X线征象,一般可正确诊断。     

Fusion of three-dimensional X-ray angiography and three-dimensional echocardiography

Objective Cardiovascular intervention guidance requires knowledge of heart function relative to its blood supply or venous drainage. Functional and vascular anatomic data are usually generated on different imaging systems, so fusion of the data is necessary to simultaneously visualize the results for intervention planning and guidance. The objective of this work is to establish the feasibility of fusing volumetric ultrasound (U/S) data with three-dimensional (3D) X-ray imaging data for visualization of cardiac morphology, function and coronary venous drainage. Methods Temporally resolved U/S volume data was registered with the 3D reconstruction of vascular structures derived from X-ray modeling and reconstruction. U/S image registration was obtained by optical tracking fiducial markers with simultaneous X-ray imaging. The proposed technique was applied to phantom data for accuracy assessment of the registration process and to biventricular pacemaker implantation as clinical example. Results Fusion of U/S data with 3D X-ray reconstruction data produced an RMS registration error below 2 mm. Conclusion Preliminary clinical feasibility of U/S-derived data synchronously with X-ray derived 3D coronary venography was established. This technique can be applied for fusion of functional U/S data with 3D anatomic X-ray data of the coronary veins during a biventricular pacemaker implantation procedures.     

Biomechanically constrained non-rigid MR-TRUS prostate registration using deep learning based 3D point cloud matching

A non-rigid MR-TRUS image registration framework is proposed for prostate interventions. The registration framework consists of a convolutional neural networks (CNN) for MR prostate segmentation, a CNN for TRUS prostate segmentation and a point-cloud based network for rapid 3D point cloud matching. Volumetric prostate point clouds were generated from the segmented prostate masks using tetrahedron meshing. The point cloud matching network was trained using deformation field that was generated by finite element analysis. Therefore, the network implicitly models the underlying biomechanical constraint when performing point cloud matching. A total of 50 patients’ datasets were used for the network training and testing. Alignment of prostate shapes after registration was evaluated using three metrics including Dice similarity coefficient (DSC), mean surface distance (MSD) and Hausdorff distance (HD). Internal point-to-point registration accuracy was assessed using target registration error (TRE). Jacobian determinant and strain tensors of the predicted deformation field were calculated to analyze the physical fidelity of the deformation field. On average, the mean and standard deviation were 0.94±0.02, 0.90±0.23 mm, 2.96±1.00 mm and 1.57±0.77 mm for DSC, MSD, HD and TRE, respectively. Robustness of our method to point cloud noise was evaluated by adding different levels of noise to the query point clouds. Our results demonstrated that the proposed method could rapidly perform MR-TRUS image registration with good registration accuracy and robustness.     

Deformable registration of preoperative MR,pre-resection ultrasound,and post-resection ultrasound images of neurosurgery

Purpose

Sites that use ultrasound (US) in image-guided neurosurgery (IGNS) of brain tumors generally have three sets of imaging data: preoperative magnetic resonance (MR) image, pre-resection US, and post-resection US. The MR image is usually acquired days before the surgery, the pre-resection US is obtained after the craniotomy but before the resection, and finally, the post-resection US scan is performed after the resection of the tumor. The craniotomy and tumor resection both cause brain deformation, which significantly reduces the accuracy of the MR–US alignment.

Method

Three unknown transformations exist between the three sets of imaging data: MR to pre-resection US, pre- to post-resection US, and MR to post-resection US. We use two algorithms that we have recently developed to perform the first two registrations (i.e., MR to pre-resection US and pre- to post-resection US). Regarding the third registration (MR to post-resection US), we evaluate three strategies. The first method performs a registration between the MR and pre-resection US, and another registration between the pre- and post-resection US. It then composes the two transformations to register MR and post-resection US; we call this method compositional registration. The second method ignores the pre-resection US and directly registers the MR and post-resection US; we refer to this method as direct registration. The third method is a combination of the first and second: it uses the solution of the compositional registration as an initial solution for the direct registration method. We call this method group-wise registration.

Results

We use data from 13 patients provided in the MNI BITE database for all of our analysis. Registration of MR and pre-resection US reduces the average of the mean target registration error (mTRE) from 4.1 to 2.4 mm. Registration of pre- and post-resection US reduces the average mTRE from 3.7 to 1.5 mm. Regarding the registration of MR and post-resection US, all three strategies reduce the mTRE. The initial average mTRE is 5.9 mm, which reduces to 3.3 mm with the compositional method, 2.9 mm with the direct technique, and 2.8 mm with the group-wise method.

Conclusion

Deformable registration of MR and pre- and post-resection US images significantly improves their alignment. Among the three methods proposed for registering the MR to post-resection US, the group-wise method gives the lowest TRE values. Since the running time of all registration algorithms is less than 2 min on one core of a CPU, they can be integrated into IGNS systems for interactive use during surgery.

     

A three-dimensional registration method for microUS/ microPET multimodality small-animal imaging

Small-animal models are used extensively in disease research, genomics research, drug development and developmental biology. The development of noninvasive small-animal imaging techniques with adequate spatial resolution and sensitivity is therefore of prime importance. In particular, multimodality small-animal imaging can provide complementary information. This paper presents a method for registering high-frequency ultrasonic (microUS) images with small-animal positron-emission tomography (microPET) images. Registration is performed using six external multimodality markers, each being a glass bead with a diameter of 0.43-0.60 mm, with 0.1 microl of [18F]FDG placed in each marker holder. A small-animal holder is used to transfer mice between the microPET and microUS systems. Multimodality imaging was performed on C57BL/6J black mice bearing WF-3 ovary cancer cells in the second week after tumor implantation and rigid-body image registration of the six markers was also performed. The average registration error was 0.31 mm when all six markers were used and increased as the number of markers decreased. After image registration, image segmentation and fusion are performed on the tumor. Our multimodality small-animal imaging method allows structural information from microUS to be combined with functional information from microPET, with the preliminary results showing it to be an effective tool for cancer research.     

Real-time pose estimation of devices from x-ray images: Application to x-ray/echo registration for cardiac interventions

In recent years, registration between x-ray fluoroscopy (XRF) and transesophageal echocardiography (TEE) has been rapidly developed, validated, and translated to the clinic as a tool for advanced image guidance of structural heart interventions. This technology relies on accurate pose-estimation of the TEE probe via standard 2D/3D registration methods. It has been shown that latencies caused by slow registrations can result in errors during untracked frames, and a real-time ( > 15 hz) tracking algorithm is needed to minimize these errors. This paper presents two novel similarity metrics designed for accurate, robust, and extremely fast pose-estimation of devices from XRF images: Direct Splat Correlation (DSC) and Patch Gradient Correlation (PGC). Both metrics were implemented in CUDA C, and validated on simulated and clinical datasets against prior methods presented in the literature. It was shown that by combining DSC and PGC in a hybrid method (HYB), target registration errors comparable to previously reported methods were achieved, but at much higher speeds and lower failure rates. In simulated datasets, the proposed HYB method achieved a median projected target registration error (pTRE) of 0.33 mm and a mean registration frame-rate of 12.1 hz, while previously published methods produced median pTREs greater than 1.5 mm and mean registration frame-rates less than 4 hz. In clinical datasets, the HYB method achieved a median pTRE of 1.1 mm and a mean registration frame-rate of 20.5 hz, while previously published methods produced median pTREs greater than 1.3 mm and mean registration frame-rates less than 12 hz. The proposed hybrid method also had much lower failure rates than previously published methods.