Learning Non-rigid Deformations for Robust,Constrained Point-based Registration in Image-Guided MR-TRUS Prostate Intervention

Accurate and robust non-rigid registration of pre-procedure magnetic resonance (MR) imaging to intra-procedure trans-rectal ultrasound (TRUS) is critical for image-guided biopsies of prostate cancer. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. TRUS-guided biopsy is the current clinical standard for prostate cancer diagnosis and assessment. State-of-the-art, clinical MR-TRUS image fusion relies upon semi-automated segmentations of the prostate in both the MR and the TRUS images to perform non-rigid surface-based registration of the gland. Segmentation of the prostate in TRUS imaging is itself a challenging task and prone to high variability. These segmentation errors can lead to poor registration and subsequently poor localization of biopsy targets, which may result in false-negative cancer detection. In this paper, we present a non-rigid surface registration approach to MR-TRUS fusion based on a statistical deformation model (SDM) of intra-procedural deformations derived from clinical training data. Synthetic validation experiments quantifying registration volume of interest overlaps of the PI-RADS parcellation standard and tests using clinical landmark data demonstrate that our use of an SDM for registration, with median target registration error of 2.98 mm, is significantly more accurate than the current clinical method. Furthermore, we show that the low-dimensional SDM registration results are robust to segmentation errors that are not uncommon in clinical TRUS data.     

Intra-operative 3D guidance and edema detection in prostate brachytherapy using a non-isocentric C-arm

PurposeBrachytherapy (radioactive seed insertion) has emerged as one of the most effective treatment options for patients with prostate cancer, with the added benefit of a convenient outpatient procedure. The main limitation in contemporary brachytherapy is faulty seed placement, predominantly due to the presence of intra-operative edema (tissue expansion). Though currently not available, the capability to intra-operatively monitor the seed distribution, can make a significant improvement in cancer control. We present such a system here.MethodsIntra-operative measurement of edema in prostate brachytherapy requires localization of inserted radioactive seeds relative to the prostate. Seeds were reconstructed using a typical non-isocentric C-arm, and exported to a commercial brachytherapy treatment planning system. Technical obstacles for 3D reconstruction on a non-isocentric C-arm include pose-dependent C-arm calibration; distortion correction; pose estimation of C-arm images; seed reconstruction; and C-arm to TRUS registration.ResultsIn precision-machined hard phantoms with 40–100 seeds and soft tissue phantoms with 45–87 seeds, we correctly reconstructed the seed implant shape with an average 3D precision of 0.35 mm and 0.24 mm, respectively. In a DoD Phase-1 clinical trial on six patients with 48–82 planned seeds, we achieved intra-operative monitoring of seed distribution and dosimetry, correcting for dose inhomogeneities by inserting an average of over four additional seeds in the six enrolled patients (minimum 1; maximum 9). Additionally, in each patient, the system automatically detected intra-operative seed migration induced due to edema (mean 3.84 mm, STD 2.13 mm, Max 16.19 mm).ConclusionsThe proposed system is the first of a kind that makes intra-operative detection of edema (and subsequent re-optimization) possible on any typical non-isocentric C-arm, at negligible additional cost to the existing clinical installation. It achieves a significantly more homogeneous seed distribution, and has the potential to affect a paradigm shift in clinical practice. Large scale studies and commercialization are currently underway.     

A spline-based non-linear diffeomorphism for multimodal prostate registration

This paper presents a novel method for non-rigid registration of transrectal ultrasound and magnetic resonance prostate images based on a non-linear regularized framework of point correspondences obtained from a statistical measure of shape-contexts. The segmented prostate shapes are represented by shape-contexts and the Bhattacharyya distance between the shape representations is used to find the point correspondences between the 2D fixed and moving images. The registration method involves parametric estimation of the non-linear diffeomorphism between the multimodal images and has its basis in solving a set of non-linear equations of thin-plate splines. The solution is obtained as the least-squares solution of an over-determined system of non-linear equations constructed by integrating a set of non-linear functions over the fixed and moving images. However, this may not result in clinically acceptable transformations of the anatomical targets. Therefore, the regularized bending energy of the thin-plate splines along with the localization error of established correspondences should be included in the system of equations. The registration accuracies of the proposed method are evaluated in 20 pairs of prostate mid-gland ultrasound and magnetic resonance images. The results obtained in terms of Dice similarity coefficient show an average of 0.980±0.004, average 95% Hausdorff distance of 1.63±0.48mm and mean target registration and target localization errors of 1.60±1.17mm and 0.15±0.12mm respectively.     

A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy

We propose an image guidance system for robot assisted laparoscopic radical prostatectomy (RALRP). A virtual 3D reconstruction of the surgery scene is displayed underneath the endoscope’s feed on the surgeon’s console. This scene consists of an annotated preoperative Magnetic Resonance Image (MRI) registered to intraoperative 3D Trans-rectal Ultrasound (TRUS) as well as real-time sagittal 2D TRUS images of the prostate, 3D models of the prostate, the surgical instrument and the TRUS transducer. We display these components with accurate real-time coordinates with respect to the robot system. Since the scene is rendered from the viewpoint of the endoscope, given correct parameters of the camera, an augmented scene can be overlaid on the video output. The surgeon can rotate the ultrasound transducer and determine the position of the projected axial plane in the MRI using one of the registered da Vinci instruments. This system was tested in the laboratory on custom-made agar prostate phantoms. We achieved an average total registration accuracy of 3.2  ±  1.3 mm. We also report on the successful application of this system in the operating room in 12 patients. The average registration error between the TRUS and the da Vinci system for the last 8 patients was 1.4  ±  0.3 mm and average target registration error of 2.1  ±  0.8 mm, resulting in an in vivo overall robot system to MRI mean registration error of 3.5 mm or less, which is consistent with our laboratory studies.     

MR to ultrasound registration for image-guided prostate interventions

A deformable registration method is described that enables automatic alignment of magnetic resonance (MR) and 3D transrectal ultrasound (TRUS) images of the prostate gland. The method employs a novel "model-to-image" registration approach in which a deformable model of the gland surface, derived from an MR image, is registered automatically to a TRUS volume by maximising the likelihood of a particular model shape given a voxel-intensity-based feature that represents an estimate of surface normal vectors at the boundary of the gland. The deformation of the surface model is constrained by a patient-specific statistical model of gland deformation, which is trained using data provided by biomechanical simulations. Each simulation predicts the motion of a volumetric finite element mesh due to the random placement of a TRUS probe in the rectum. The use of biomechanical modelling in this way also allows a dense displacement field to be calculated within the prostate, which is then used to non-rigidly warp the MR image to match the TRUS image. Using data acquired from eight patients, and anatomical landmarks to quantify the registration accuracy, the median final RMS target registration error after performing 100 MR-TRUS registrations for each patient was 2.40 mm.     

Image Registration Accuracy of a 3‐Dimensional Transrectal Ultrasound–Guided Prostate Biopsy System

Objective. For a follow‐up prostate biopsy procedure, it is useful to know the previous biopsy locations in anatomic relation to the current transrectal ultrasound (TRUS) scan. The goal of this study was to validate the performance of a 3‐dimensional TRUS‐guided prostate biopsy system that can accurately relocate previous biopsy sites. Methods. To correlate biopsy locations from a sequence of visits by a patient, the prostate surface data obtained from a previous visit needs to be registered to the follow‐up visits. Two interpolation methods, thin‐plate spline (TPS) and elastic warping (EW), were tested for registration of the TRUS prostate image to follow‐up scans. We validated our biopsy system using a custom‐built phantom. Beads were embedded inside the phantom and were located in each TRUS scan. We recorded the locations of the beads before and after pressures were applied to the phantom and then compared them with computer‐estimated positions to measure performance. Results. In our experiments, before system processing, the mean target registration error (TRE) ± SD was 6.4 ± 4.5 mm (range, 3–13 mm). After registration and TPS interpolation, the TRE was 5.0 ± 1.03 mm (range, 2–8 mm). After registration and EW interpolation, the TRE was 2.7 ± 0.99 mm (range, 1–4 mm). Elastic warping was significantly better than the TPS in most cases (P < .0011). For clinical applications, EW can be implemented on a graphics processing unit with an execution time of less than 2.5 seconds. Conclusions. Elastic warping interpolation yields more accurate results than the TPS for registration of TRUS prostate images. Experimental results indicate potential for clinical application of this method.     

Automatic Shape-Based Level Set Segmentation for Needle Tracking in 3-D TRUS-Guided Prostate Brachytherapy

Prostate brachytherapy is an effective treatment for early prostate cancer. The success depends critically on the correct needle implant positions. We have devised an automatic shape-based level set segmentation tool for needle tracking in 3-D transrectal ultrasound (TRUS) images, which uses the shape information and level set technique to localize the needle position and estimate the endpoint of needle in real-time. The 3-D TRUS images used in the evaluation of our tools were obtained using a 2-D TRUS transducer from Ultrasonix (Richmond, BC, Canada) and a computer-controlled stepper motor system from Thorlabs (Newton, NJ, USA). The accuracy and feedback mechanism had been validated using prostate phantoms and compared with 3-D positions of these needles derived from experts' readings. The experts' segmentation of needles from 3-D computed tomography images was the ground truth in this study. The difference between automatic and expert segmentations are within 0.1 mm for 17 of 19 implanted needles. The mean errors of automatic segmentations by comparing with the ground truth are within 0.25 mm. Our automated method allows real-time TRUS-based needle placement difference within one pixel compared with manual expert segementation.     

3D prostate model formation from non-parallel 2D ultrasound biopsy images

Biopsy of the prostate using 2D transrectal ultrasound (TRUS) guidance is the current gold standard for diagnosis of prostate cancer; however, the current procedure is limited by using 2D biopsy tools to target 3D biopsy locations. We propose a technique for patient-specific 3D prostate model reconstruction from a sparse collection of non-parallel 2D TRUS biopsy images. Our method conforms to the restrictions of current TRUS biopsy equipment and could be efficiently incorporated into current clinical biopsy procedures for needle guidance without the need for expensive hardware additions. In this paper, the model reconstruction technique is evaluated using simulated biopsy images from 3D TRUS prostate images of 10 biopsy patients. All reconstructed models are compared to their corresponding 3D manually segmented prostate models for evaluation of prostate volume accuracy and surface errors (both regional and global). The number of 2D TRUS biopsy images used for prostate modeling was varied to determine the optimal number of images necessary for accurate prostate surface estimation.     

An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy

External beam radiotherapy (EBRT) has become the preferred options for nonsurgical treatment of prostate cancer and cervix cancer. In order to deliver higher doses to cancerous regions within these pelvic structures (i.e. prostate or cervix) while maintaining or lowering the doses to surrounding non-cancerous regions, it is critical to account for setup variation, organ motion, anatomical changes due to treatment and intra-fraction motion. In previous work, manual segmentation of the soft tissues is performed and then images are registered based on the manual segmentation. In this paper, we present an integrated automatic approach to multiple organ segmentation and nonrigid constrained registration, which can achieve these two aims simultaneously. The segmentation and registration steps are both formulated using a Bayesian framework, and they constrain each other using an iterative conditional model strategy. We also propose a new strategy to assess cumulative actual dose for this novel integrated algorithm, in order to both determine whether the intended treatment is being delivered and, potentially, whether or not a plan should be adjusted for future treatment fractions. Quantitative results show that the automatic segmentation produced results that have an accuracy comparable to manual segmentation, while the registration part significantly outperforms both rigid and nonrigid registration. Clinical application and evaluation of dose delivery show the superiority of proposed method to the procedure currently used in clinical practice, i.e. manual segmentation followed by rigid registration.     

Photoacoustic imaging of prostate brachytherapy seeds

Brachytherapy seed therapy is an increasingly common way to treat prostate cancer through localized radiation. The current standard of care relies on transrectal ultrasound (TRUS) for imaging guidance during the seed placement procedure. As visualization of individual metallic seeds tends to be difficult or inaccurate under TRUS guidance, guide needles are generally tracked to infer seed placement. In an effort to improve seed visualization and placement accuracy, the use of photoacoustic (PA) imaging, which is highly sensitive to metallic objects in soft tissue, was investigated for this clinical application. The PA imaging properties of bare (i.e., embedded in pure gelatin) and tissue-embedded (at depths of up to 13 mm) seeds were investigated with a multi-wavelength (750 to 1090 nm) PA imaging technique. Results indicate that, much like ultrasonic (US) imaging, an angular dependence (i.e., seed orientation relative to imaging transducer) of the PA signal exists. Despite this shortcoming, however, PA imaging offers improved contrast, over US imaging, of a seed in prostate tissue if sufficient local fluence is achieved. Additionally, although the PA signal of a bare seed is greatest for lower laser wavelengths (e.g., 750 nm), the scattering that results from tissue tends to favor the use of higher wavelengths (e.g., 1064 nm, which is the primary wavelength of Nd:YAG lasers) when the seed is located in tissue. A combined PA and US imaging approach (i.e., PAUS imaging) shows strong potential to visualize both the seed and the surrounding anatomical environment of the prostate during brachytherapy seed placement procedures.     

Contrast-Specific Spherical Lesion Phantoms and Ancillary Analysis Software for the Objective Evaluation of Transrectal Ultrasound System Contrast Detectability

Brachytherapy is an efficacious treatment option because of its benefits for patient recovery, dose localization and conformity, but these favorable outcomes can be ensured only if the transrectal ultrasound (TRUS) system is optimized for the specific application of ultrasound-guided prostate brachytherapy. The ability to delineate the prostate from surrounding tissue during TRUS-guided prostate brachytherapy is vital for treatment planning, and consequently, so is the contrast resolution. This study describes the development of task-specific contrast-detail phantoms with clinically relevant contrast and spherical target sizes for contrast-detail performance evaluation of TRUS systems used in the brachytherapy procedure. The procedure for objective assessment of the contrast detectability of the TRUS systems is also described; a program was developed in MATLAB (R2017a, The MathWorks, Natick, MA, USA) to quantitatively analyze image quality in terms of the lesion signal-to-noise ratio (LSNR) and validated with representative control test images. The LSNR of the Hitachi EUB-7500A (2013, Hitachi, Ltd, Tokyo, Japan) TRUS system was measured on sagittal and transverse TRUS images of the contrast-detail phantoms described in this work. Results revealed the efficacy of the device as an image quality evaluation tool and the impact of the size, depth and relative contrast of the targets to the surrounding tissue on the contrast detectability of a TRUS system for both transducer arrays. The MATLAB program objectively measured the contrast detectability of the TRUS system and has the potential to determine optimized imaging parameters that could be designed as part of standardization of the imaging protocol used in TRUS-guided prostate brachytherapy for prostate cancer.     

Statistical shape and texture model of quadrature phase information for prostate segmentation

Purpose  

Prostate volume estimation from segmentation of transrectal ultrasound (TRUS) images aids in diagnosis and treatment of prostate hypertrophy and cancer. Computer-aided accurate and computationally efficient prostate segmentation in TRUS images is a challenging task, owing to low signal-to-noise ratio, speckle noise, calcifications, and heterogeneous intensity distribution in the prostate region.     

3D spatial compounding of ultrasound images using image-based nonrigid registration

Medical ultrasound images are often distorted enough to significantly limit resolution during compounding (i.e., summation of images from multiple views). A new, volumetric image registration technique has been used successfully to enable high spatial resolution in three-dimensional (3D) spatial compounding of ultrasound images. Volumetric ultrasound data were acquired by scanning a linear matrix array probe in the elevational direction in a focal lesion phantom and in a breast in vivo. To obtain partly uncorrelated views, the volume of interest was scanned at five different transducer tilt angles separated by 4° to 6°. Pairs of separate views were registered by an automatic procedure based on a mutual information metric, using global full affine and thin-plate spline warping transformations. Registration accuracy was analyzed automatically in the phantom data, and manually in vivo, yielding average registration errors of 0.31 mm and 0.65 mm, respectively. In the vicinity of the warping control points, registrations obtained with warping transformations were significantly more accurate than full affine registrations. Compounded images displayed the expected reduction in speckle noise and increase in contrast-to-noise ratio (CNR), as well as better delineation of connective tissues and reduced shadowing. Compounding also revealed some apparent low contrast lobulations that were not visible in the single-sweep images. Given expected algorithmic and hardware enhancements, nonrigid, image-based registration shows great promise for reducing tissue motion and refraction artifacts in 3D spatial compounding.     

Robotic assistance for ultrasound-guided prostate brachytherapy

We present a robotically assisted prostate brachytherapy system and test results in training phantoms and Phase-I clinical trials. The system consists of a transrectal ultrasound (TRUS) and a spatially co-registered robot, fully integrated with an FDA-approved commercial treatment planning system. The salient feature of the system is a small parallel robot affixed to the mounting posts of the template. The robot replaces the template interchangeably, using the same coordinate system. Established clinical hardware, workflow and calibration remain intact. In all phantom experiments, we recorded the first insertion attempt without adjustment. All clinically relevant locations in the prostate were reached. Non-parallel needle trajectories were achieved. The pre-insertion transverse and rotational errors (measured with a Polaris optical tracker relative to the template's coordinate frame) were 0.25 mm (STD=0.17 mm) and 0.75 degrees (STD=0.37 degrees). In phantoms, needle tip placement errors measured in TRUS were 1.04 mm (STD=0.50mm). A Phase-I clinical feasibility and safety trial has been successfully completed with the system. We encountered needle tip positioning errors of a magnitude greater than 4mm in only 2 of 179 robotically guided needles, in contrast to manual template guidance where errors of this magnitude are much more common. Further clinical trials are necessary to determine whether the apparent benefits of the robotic assistant will lead to improvements in clinical efficacy and outcomes.     

Validation of vessel-based registration for correction of brain shift

The displacement and deformation of brain tissue is a major source of error in image-guided neurosurgery systems. We have designed and implemented a method to detect and correct brain shift using pre-operative MR images and intraoperative Doppler ultrasound data and present its validation with both real and simulated data. The algorithm uses segmented vessels from both modalities, and estimates the deformation using a modified version of the iterative closest point (ICP) algorithm. We use the least trimmed squares (LTS) to reduce the number of outliers in the point matching procedure. These points are used to drive a thin-plate spline transform to achieve non-linear registration. Validation was completed in two parts. First, the technique was tested and validated using realistic simulations where the results were compared to the known deformation. The registration technique recovered 75% of the deformation in the region of interest accounting for deformations as large as 20 mm. Second, we performed a PVA-cryogel phantom study where both MR and ultrasound images of the phantom were obtained for three different deformations. The registration results based on MR data were used as a gold standard to evaluate the performance of the ultrasound based registration. On average, deformations of 7.5 mm magnitude were corrected to within 1.6 mm for the ultrasound based registration and 1.07 mm for the MR based registration.     

Registration of freehand 3D ultrasound and magnetic resonance liver images

We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a "bronze standard" registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the "bronze standard" and this is improved to 3.6 mm after running the intensity-based algorithm.     

Prostate segmentation in transrectal ultrasound using magnetic resonance imaging priors

Purpose

In the current standard of care, real-time transrectal ultrasound (TRUS) is commonly used for prostate brachytherapy guidance. As TRUS provides limited soft tissue contrast, segmenting the prostate gland in TRUS images is often challenging and subject to inter-observer and intra-observer variability, especially at the base and apex where the gland boundary is hard to define. Magnetic resonance imaging (MRI) has higher soft tissue contrast allowing the prostate to be contoured easily. In this paper, we aim to show that prostate segmentation in TRUS images informed by MRI priors can improve on prostate segmentation that relies only on TRUS images.

Methods

First, we compare the TRUS-based prostate segmentation used in the treatment of 598 patients with a high-quality MRI prostate atlas and observe inconsistencies at the apex and base. Second, motivated by this finding, we propose an alternative TRUS segmentation technique that is fully automatic and uses MRI priors. The algorithm uses a convolutional neural network to segment the prostate in TRUS images at mid-gland, where the gland boundary can be clearly seen. It then reconstructs the gland boundary at the apex and base with the aid of a statistical shape model built from an MRI atlas of 78 patients.

Results

Compared to the clinical TRUS segmentation, our method achieves similar mid-gland segmentation results in the 598-patient database. For the seven patients who had both TRUS and MRI, our method achieved more accurate segmentation of the base and apex with the MRI segmentation used as ground truth.

Conclusion

Our results suggest that utilizing MRI priors in TRUS prostate segmentation could potentially improve the performance at base and apex.

     

Extended-Field-of-View Three-Dimensional Transesophageal Echocardiography Using Image-Based X-Ray Probe Tracking

The use of ultrasound imaging for guidance of cardiac interventional procedures is limited by the small field of view of the ultrasound volume. A larger view can be created by image-based registration of several partially overlapping volumes, but automatic registration is likely to fail unless the registration is initialized close to the volumes' correct alignment. In this article, we use X-ray images to track a transesophageal ultrasound probe and thereby provide initial position information for the registration of the ultrasound volumes. The tracking is possible using multiple X-rays or just a single X-ray for each probe position. We test the method in a phantom experiment and find that with at least 50% overlap, 88% of volume pairs are correctly registered when tracked using three X-rays and 86% when using single X-rays. Excluding failed registrations with errors greater than 10 mm, the average registration accuracy is 2.92 mm between ultrasound volumes and 4.75 mm for locating an ultrasound volume in X-ray space. We conclude that the accuracy and robustness of the registrations are sufficient to provide useful images for interventional guidance.     

125I微粒子植入术联合内分泌治疗局部晚期前列腺癌(附14例报告)

目的探讨放射性微粒子植入术联合内分泌治疗局部晚期前列腺癌。方法回顾14例局部晚期前列腺癌患者,术前经直肠超声(TRUS)确定前列腺体积,描绘前列腺的轮廓和横断面,运用三维治疗计划系统制定放射治疗方案,在TRUS引导下,植入放射性粒子125I。结果14例患者随访18～24月,治疗后PSA值、前列腺体积、IPSS评分均较术前有明显改善(P<0.05),一例出现放射性直肠炎并发症,经中药保留灌肠治疗后好转。结论TRUS引导下125I粒子植入联合内分泌治疗是治疗晚期局部前列腺癌的一种可行的方法,有较好的疗效。