3D ultrasound-CT registration of the liver using combined landmark-intensity information

Purpose  An important issue in computer-assisted surgery of the liver is a fast and reliable transfer of preoperative resection plans to the intraoperative situation. One problem is to match the planning data, derived from preoperative CT or MR images, with 3D ultrasound images of the liver, acquired during surgery. As the liver deforms significantly in the intraoperative situation non-rigid registration is necessary. This is a particularly challenging task because pre- and intraoperative image data stem from different modalities and ultrasound images are generally very noisy. Methods  One way to overcome these problems is to incorporate prior knowledge into the registration process. We propose a method of combining anatomical landmark information with a fast non-parametric intensity registration approach. Mathematically, this leads to a constrained optimization problem. As distance measure we use the normalized gradient field which allows for multimodal image registration. Results  A qualitative and quantitative validation on clinical liver data sets of three different patients has been performed. We used the distance of dense corresponding points on vessel center lines for quantitative validation. The combined landmark and intensity approach improves the mean and percentage of point distances above 3 mm compared to rigid and thin-plate spline registration based only on landmarks. Conclusion  The proposed algorithm offers the possibility to incorporate additional a priori knowledge—in terms of few landmarks—provided by a human expert into a non-rigid registration process.     

Validation of vessel-based registration for correction of brain shift

The displacement and deformation of brain tissue is a major source of error in image-guided neurosurgery systems. We have designed and implemented a method to detect and correct brain shift using pre-operative MR images and intraoperative Doppler ultrasound data and present its validation with both real and simulated data. The algorithm uses segmented vessels from both modalities, and estimates the deformation using a modified version of the iterative closest point (ICP) algorithm. We use the least trimmed squares (LTS) to reduce the number of outliers in the point matching procedure. These points are used to drive a thin-plate spline transform to achieve non-linear registration. Validation was completed in two parts. First, the technique was tested and validated using realistic simulations where the results were compared to the known deformation. The registration technique recovered 75% of the deformation in the region of interest accounting for deformations as large as 20 mm. Second, we performed a PVA-cryogel phantom study where both MR and ultrasound images of the phantom were obtained for three different deformations. The registration results based on MR data were used as a gold standard to evaluate the performance of the ultrasound based registration. On average, deformations of 7.5 mm magnitude were corrected to within 1.6 mm for the ultrasound based registration and 1.07 mm for the MR based registration.     

Validation of a hybrid Doppler ultrasound vessel-based registration algorithm for neurosurgery

Purpose

We describe and validate a novel hybrid nonlinear vessel registration algorithm for intra-operative updating of preoperative magnetic resonance (MR) images using Doppler ultrasound (US) images acquired on the dura for the correction of brain-shift and registration inaccuracies. We also introduce an US vessel appearance simulator that generates vessel images similar in appearance to that acquired with US from MR angiography data.

Methods

Our registration uses the minimum amount of preprocessing to extract vessels from the raw volumetric images. This prevents the removal of important registration information and minimizes the introduction of artifacts that may affect robustness, while reducing the amount of extraneous information in the image to be processed, thus improving the convergence speed of the algorithm. We then completed 3 rounds of validation for our vessel registration method for robustness and accuracy using (i) a large number of synthetic trials generated with our US vessel simulator, (ii) US images acquired from a real physical phantom made from polyvinyl alcohol cryogel, and (iii) real clinical data gathered intra-operatively from 3 patients.

Results

Resulting target registration errors (TRE) of less than 2.5?mm are achieved in more than 90?% of the synthetic trials when the initial TREs are less than 20?mm. TREs of less than 2?mm were achieved when the technique was applied to the physical phantom, and TREs of less than 3?mm were achieved on clinical data.

Conclusions

These test trials show that the proposed algorithm is not only accurate but also highly robust to noise and missing vessel segments when working with US images acquired in a wide range of real-world conditions.     

Estimation of intraoperative brain shift by combination of stereovision and doppler ultrasound: phantom and animal model study

Purpose

Combination of various intraoperative imaging modalities potentially can reduce error of brain shift estimation during neurosurgical operations. In the present work, a new combination of surface imaging and Doppler US images is proposed to calculate the displacements of cortical surface and deformation of internal vessels in order to estimate the targeted brain shift using a Finite Element Model (FEM). Registration error in each step and the overall performance of the method are evaluated.

Methods

The preoperative steps include constructing a FEM from MR images and extracting vascular tree from MR Angiography (MRA). As the first intraoperative step, after the craniotomy and with the dura opened, a designed checkerboard pattern is projected on the cortex surface and projected landmarks are scanned and captured by a stereo camera (Int J Imaging Syst Technol 23(4):294–303, 2013. doi:  10.1002/ima.22064). This 3D point cloud should be registered to boundary nodes of FEM in the region of interest. For this purpose, we developed a new non-rigid registration method, called finite element drift that is more compatible with the underlying nature of deformed object. The presented algorithm outperforms other methods such as coherent point drift when the deformation is local or non-coherent. After registration, the acquired displacement vectors are used as boundary conditions for FE model. As the second step, by tracking a 2D Doppler ultrasound probe swept on the parenchyma, a 3D image of deformed vascular tree is constructed. Elastic registration of this vascular point cloud to the corresponding preoperative data results the second series of displacement vector applicable to closest internal nodes of FEM. After running FE analysis, the displacement of all nodes is calculated. The brain shift is then estimated as displacement of nodes in boundary of a deep target, e.g., a tumor. We used intraoperative MR (iMR) images as the references for measuring the performance of the brain shift estimator. In the present study, two set of tests were performed using: (a) a deformable brain phantom with surface data and (b) an alive brain of an approximately big dog with surface data and US Doppler images. In our designed phantom, small tubes connected to an inflatable balloon were considered as displaceable targets and in the animal model, the target was modeled by a cyst which was created by an injection.

Results

In the phantom study, the registration error for the surface points before FE analysis and for the target points after running FE model were \({<}0.76\) and 1.4 mm, respectively. In a real condition of operating room for animal model, the registration error was about 1 mm for the surface, 1.9 mm for the vascular tree and 1.55 mm for the target points.

Conclusions

The proposed projected surface imaging in conjunction with the Doppler US data combined in a powerful biomechanical model can result an acceptable performance in calculation of deformation during surgical navigation. However, the projected landmark method is sensitive to ambient light and surface conditions and the Doppler ultrasound suffers from noise and 3D image construction problems, the combination of these two methods applied on a FEM has an eligible performance.

     

3D/2D Registration with superabundant vessel reconstruction for cardiac resynchronization therapy

A key component of image guided interventions is the registration of preoperative and intraoperative images. Classical registration approaches rely on cross-modality information; however, in modalities such as MRI and X-ray there may not be sufficient cross-modality information. This paper proposes a fundamentally different registration approach which uses adjacent anatomical structures with superabundant vessel reconstruction and dynamic outlier rejection. In the targeted clinical scenario of cardiac resynchronization therapy (CRT) delivery, preoperative, non contrast-enhanced, MRI is registered to intraoperative, contrasted X-ray fluoroscopy. The adjacent anatomical structures are the left ventricle (LV) from MRI and the coronary veins reconstructed from two contrast-enhanced X-ray images. The novel concept of superabundant vessel reconstruction is introduced to bypass the standard reconstruction problem of establishing one-to-one correspondences. Furthermore, a new dynamic outlier rejection method is proposed, to enable globally optimal point set registration. The proposed approach has been qualitatively and quantitatively evaluated on phantom, clinical CT angiography with ground truth and clinical CRT data. A novel evaluation method is proposed for clinical CRT data based on previously implanted artificial aortic and mitral valves. The registration accuracy in 3D was 2.94 mm for the aortic and 3.86 mm for the mitral valve. The results are below the required accuracy identified by clinical partners to be the half-segment size (16.35 mm) of a standard American Heart Association (AHA) 16 segment model of the LV.     

Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine

We present a groupwise US to CT registration algorithm for guiding percutaneous spinal interventions. In addition, we introduce a comprehensive validation scheme that accounts for changes in the curvature of the spine between preoperative and intraoperative imaging. In our registration methodology, each vertebra in CT is treated as a sub-volume and transformed individually. A biomechanical model is used to constrain the displacement of the vertebrae relative to one another. The sub-volumes are then reconstructed into a single volume. During each iteration of registration, an US image is simulated from the reconstructed CT volume and an intensity-based similarity metric is calculated with the real US image. Validation studies are performed on CT and US images from a sheep cadaver, five patient-based phantoms designed to preserve realistic curvatures of the spine and a sixth patient-based phantom where the curvature of the spine is changed between preoperative and intraoperative imaging. For datasets where the spine curve between two imaging modalities was artificially perturbed, the proposed methodology was able to register initial misalignments of up to 20mm with a success rate of 95%. For the phantom with a physical change in the curvature of the spine introduced between the US and CT datasets, the registration success rate was 98.5%. Finally, the registration success rate for the sheep cadaver with soft-tissue information was 87%. The results demonstrate that our algorithm allows for robust registration of US and CT datasets, regardless of a change in the patients pose between preoperative and intraoperative image acquisitions.     

Serial registration of intraoperative MR images of the brain

The increased use of image-guided surgery systems during neurosurgery has brought to prominence the inaccuracies of conventional intraoperative navigation systems caused by shape changes such as those due to brain shift. We propose a method to track the deformation of the brain and update preoperative images using intraoperative MR images acquired at different crucial time points during surgery. We use a deformable surface matching algorithm to capture the deformation of boundaries of key structures (cortical surface, ventricles and tumor) throughout the neurosurgical procedure, and a linear finite element elastic model to infer a volumetric deformation. The boundary data are extracted from intraoperative MR images using a real-time intraoperative segmentation algorithm. The algorithm has been applied to a sequence of intraoperative MR images of the brain exhibiting brain shift and tumor resection. Our results characterize the brain shift after opening of the dura and at the different stages of tumor resection, and brain swelling afterwards. Analysis of the average deformation capture was assessed by comparing landmarks identified manually and the results indicate an accuracy of 0.7+/-0.6 mm (mean+/-S.D.) for boundary surface landmarks, of 0.9+/-0.6 mm for landmarks inside the boundary surfaces, and 1.6+/-0.9 mm for landmarks in the vicinity of the tumor.     

Evaluation of voxel-based registration of 3-D power Doppler ultrasound and 3-D magnetic resonance angiographic images of carotid arteries

Spatial registration and fusion of ultrasound (US) images with other modalities may aid clinical interpretation. We implemented and evaluated on patient data an automated retrospective registration of magnetic resonance angiography (MRA) carotid bifurcation images with 3-D power Doppler ultrasound (PD US) and indirectly with 3-D B-mode US. Volumes were initially thresholded to reduce the uncorrelated noise signals. The registration algorithm subsequently maximized the mutual information measure between the PD US and 3-D MRA via iterative simplex search to find best "rigid body" transformation. We rated the performance of the algorithm visually on (n = 5) clinical MRA and 3-D PD US datasets. We also evaluated quantitatively the effect of thresholding, initial misalignment of the paired volumes and the reproducibility registration. We investigated the effect of image artefacts by simulation experiments. Preregistration misalignments of up to 5 mm in the transaxial plane, up to 10 mm along the axis of the carotids and up to 40 degrees resulted in 107 of 110 successful registrations, with translational and rotational errors of 0.32 mm +/- 0.3 mm and 1.6 +/- 2.1 degrees. The algorithm was not affected by missing arterial segments of up to 8 mm in length. The average registration time was 4 min. We conclude that the algorithm could be applied to 3-D US PD and MRA data for automated multimodality registration of carotid vessels without the use of fiducials.     

Cadaver validation of intensity-based ultrasound to CT registration

A method is presented for the rigid registration of tracked B-mode ultrasound images to a CT volume of a femur and pelvis. This registration can allow tracked surgical instruments to be aligned with the CT image or an associated preoperative plan. Our method is fully automatic and requires no manual segmentation of either the ultrasound images or the CT volume. The parameter which is directly related to the speed of sound through tissue has also been included in the registration optimisation process. Experiments have been carried out on six cadaveric femurs and three cadaveric pelves. Registration results were compared with a "gold standard" registration acquired using bone implanted fiducial markers. Results show the registration method to be accurate, on average, to 1.6 mm root-mean-square target registration error.     

Validation of automated ultrasound-CT registration of vertebrae

Purpose

Image-guided spine surgery requires registration of the patient anatomy and preoperative computed tomography (CT) images. A technique for intraoperative ultrasound image registration to preoperative CT scans was developed and tested. Validation of the ultrasound-CT registration technique was performed using porcine cadavers.

Methods

An ultrasound-CT registration technique was evaluated using 18 thoracic and lumbar vertebrae of 3 porcine cadavers with 10 different sweep patterns for ultrasound acquisition. For each sweep pattern at each vertebra, 100 randomly simulated initial misalignments were introduced. Each misalignment was registered. The resulting registration transformations were compared to gold standard registrations based on implanted fiducials to assess accuracy and robustness of the technique.

Results

The orthogonal-sweep acquisition was found to perform best and yielded a registration accuracy of 1.65 mm across all vertebrae on all porcine cadavers, where 82.5% of the registrations resulted in target registration errors below the 2 mm threshold recommended by a joint report from the experts in the field. In addition, we found that registration accuracy varies by the sweep pattern and vertebral level, but neighboring vertebrae tend to result in statistically similar accuracy. Ultrasound-CT registration took an average of 2.5 min to run, and the total registration time per vertebra (also including time for ultrasound acquisition and reconstruction) is approximately 8 min.

Conclusions

A previously described ultrasound-CT registration technique yields clinically acceptable accuracy and robustness on multiple vertebrae across multiple porcine cadavers. The total registration time is shorter than that of surface point-based manual registration.

     

Multi-modal registration of speckle-tracked freehand 3D ultrasound to CT in the lumbar spine

A method for registration of speckle-tracked freehand 3D ultrasound (US) to preoperative CT volumes of the spine is proposed. We register the US volume to the CT volume by creating individual US "sub-volumes", each consisting of a small section of the entire US volume. The registration proceeds incrementally from the beginning of the US volume to the end, registering every sub-volume, where each sub-volume contains overlapping images with the previous sub-volume. Each registration is performed by generating simulated US images from the CT volume. As a by-product of our registration, the significant drift error common in speckle-tracked US volumes is corrected for. Results are validated through a phantom study of plastic spine phantoms created from clinical patient CT data as well as an animal study using a lamb cadaver. Results demonstrate that we were able to successfully register a speckle-tracked US volume to CT volume in four out of five phantoms with a success rate of greater than 98%. The final error of the registered US volumes decreases by over 50 percent from the speckle tracking error to consistently below 3 mm. Studies on the lamb cadaver showed a mean registration error consistently below 2 mm.     

Extended-Field-of-View Three-Dimensional Transesophageal Echocardiography Using Image-Based X-Ray Probe Tracking

The use of ultrasound imaging for guidance of cardiac interventional procedures is limited by the small field of view of the ultrasound volume. A larger view can be created by image-based registration of several partially overlapping volumes, but automatic registration is likely to fail unless the registration is initialized close to the volumes' correct alignment. In this article, we use X-ray images to track a transesophageal ultrasound probe and thereby provide initial position information for the registration of the ultrasound volumes. The tracking is possible using multiple X-rays or just a single X-ray for each probe position. We test the method in a phantom experiment and find that with at least 50% overlap, 88% of volume pairs are correctly registered when tracked using three X-rays and 86% when using single X-rays. Excluding failed registrations with errors greater than 10 mm, the average registration accuracy is 2.92 mm between ultrasound volumes and 4.75 mm for locating an ultrasound volume in X-ray space. We conclude that the accuracy and robustness of the registrations are sufficient to provide useful images for interventional guidance.     

Biomechanics-based graph matching for augmented CT-CBCT

Purpose

Augmenting intraoperative cone beam computed tomography (CBCT) images with preoperative computed tomography data in the context of image-guided liver therapy is proposed. The expected benefit is an improved visualization of tumor(s), vascular system and other internal structures of interest.

Method

An automatic elastic registration based on matching of vascular trees extracted from both the preoperative and intraoperative images is presented. Although methods dedicated to nonrigid graph matching exist, they are not efficient when large intraoperative deformations of tissues occur, as is the case during the liver surgery. The contribution is an extension of the graph matching algorithm using Gaussian process regression (GPR) (Serradell et al. in IEEE Trans Pattern Anal Mach Intell 37(3):625–638, 2015): First, an improved GPR matching is introduced by imposing additional constraints during the matching when the number of hypothesis is large; like the original algorithm, this extended version does not require a manual initialization of matching. Second, a fast biomechanical model is employed to make the method capable of handling large deformations.

Results

The proposed automatic intraoperative augmentation is evaluated on both synthetic and real data. It is demonstrated that the algorithm is capable of handling large deformations, thus being more robust and reliable than previous approaches. Moreover, the time required to perform the elastic registration is compatible with the intraoperative navigation scenario.

Conclusion

A biomechanics-based graph matching method, which can handle large deformations and augment intraoperative CBCT, is presented and evaluated.

     

Clinical validation of vessel-based registration for correction of brain-shift

In this paper, we have tested and validated a vessel-based registration technique for correction of brain-shift using retrospective clinical data from five patients: three patients with brain tumors, one patient with an aneurysm and one patient with an arteriovenous malformation. The algorithm uses vessel centerlines extracted from segmented pre-operative MRA data and intra-operative power Doppler ultrasound images to compute first a linear fit and then a thin-plate spline transform in order to achieve non-linear registration. The method was validated using (i) homologous landmarks identified in the original data, (ii) selected vessels, excluded from the fitting procedure and (iii) manually segmented, non-vascular structures. The tracking of homologous landmarks show that we are able to correct the deformation to within 1.25 mm, and the validation using excluded vessels and anatomical structures show an accuracy of 1mm. Pre-processing of the data can be completed in 30 s per dataset, and registrations can be performed in less than 30s. This makes the technique well suited for intra-operative use.     

Registration of 3D freehand ultrasound to a bone model for orthopedic procedures of the forearm

Purpose

For guidance of orthopedic surgery, the registration of preoperative images and corresponding surgical plans with the surgical setting can be of great value. Ultrasound (US) is an ideal modality for surgical guidance, as it is non-ionizing, real time, easy to use, and requires minimal (magnetic/radiation) safety limitations. By extracting bone surfaces from 3D freehand US and registering these to preoperative bone models, complementary information from these modalities can be fused and presented in the surgical realm.

Methods

A partial bone surface is extracted from US using phase symmetry and a factor graph-based approach. This is registered to the detailed 3D bone model, conventionally generated for preoperative planning, based on a proposed multi-initialization and surface-based scheme robust to partial surfaces.

Results

36 forearm US volumes acquired using a tracked US probe were independently registered to a 3D model of the radius, manually extracted from MRI. Given intraoperative time restrictions, a computationally efficient algorithm was determined based on a comparison of different approaches. For all 36 registrations, a mean (± SD) point-to-point surface distance of \(0.57\,(\pm \,0.08)\,\hbox {mm}\) was obtained from manual gold standard US bone annotations (not used during the registration) to the 3D bone model.

Conclusions

A registration framework based on the bone surface extraction from 3D freehand US and a subsequent fast, automatic surface alignment robust to single-sided view and large false-positive rates from US was shown to achieve registration accuracy feasible for practical orthopedic scenarios and a qualitative outcome indicating good visual image alignment.

     

Fiducial-based registration with a touchable region model

Purpose

Image-guided surgery requires registration between an image coordinate system and an intraoperative coordinate system that is typically referenced to a tracking device. In fiducial-based registration methods, this is achieved by localizing points (fiducials) in each coordinate system. Often, both localizations are performed manually, first by picking a fiducial point in the image and then by using a hand-held tracked pointer to physically touch the corresponding fiducial on the patient. These manual procedures introduce localization error that is user-dependent and can significantly decrease registration accuracy. Thus, there is a need for a registration method that is tolerant of imprecise fiducial localization in the preoperative and intraoperative phases.

Methods

We propose the iterative closest touchable point (ICTP) registration framework, which uses model-based localization and a touchable region model. This method consists of three stages: (1) fiducial marker localization in image space, using a fiducial marker model, (2) initial registration with paired-point registration, and (3) fine registration based on the iterative closest point method.

Results

We perform phantom experiments with a fiducial marker design that is commonly used in neurosurgery. The results demonstrate that ICTP can provide accuracy improvements compared to the standard paired-point registration method that is widely used for surgical navigation and surgical robot systems, especially in cases where the surgeon introduces large localization errors.

Conclusions

The results demonstrate that the proposed method can reduce the effect of the surgeon’s localization performance on the accuracy of registration, thereby producing more consistent and less user-dependent registration outcomes.

     

Segmentation of the liver in ultrasound: a dynamic texture approach

Objective

The segmentation of ultrasound (US) images is useful for several applications in computer aided interventions including the registration of pre-operative CT or MRI to intra-operative US. Shadowing, intensity inhomogeneity and speckle are the common effects on US images. They render the segmentation algorithms developed for other modalities inappropriate due to poor robustness. We present a novel method for classification of hepatic structures including vasculature and liver parenchyma on US images.

Methods

The method considers B-mode US images as a dynamic texture. The dynamics of each pixel are modelled as an auto regressive (AR) process perturbed with Gaussian noise. The linear coefficients and noise variance are estimated pixel-wise using Neumaier and Schneider’s algorithm. Together with mean intensity they comprise a parametric space in which classification (maximum a posteriori or K-nearest neighbour) of each pixel is performed. We emphasize the novelty of studying dynamics rather than static features such as intensity in the segmentation of various structures.

Results

We assessed the automatic segmentations of ten US sequences using Dice Similarity Coefficients. The algorithm’s capability of vessel extraction was tested on three sequences where Doppler US failed to capture vasculature.

Conclusion

The modelling of image dynamics with AR process combined with MAP classifier produced robust segmentation results indicating that the method has a good potential for intra-operative use.