Serial registration of intraoperative MR images of the brain

The increased use of image-guided surgery systems during neurosurgery has brought to prominence the inaccuracies of conventional intraoperative navigation systems caused by shape changes such as those due to brain shift. We propose a method to track the deformation of the brain and update preoperative images using intraoperative MR images acquired at different crucial time points during surgery. We use a deformable surface matching algorithm to capture the deformation of boundaries of key structures (cortical surface, ventricles and tumor) throughout the neurosurgical procedure, and a linear finite element elastic model to infer a volumetric deformation. The boundary data are extracted from intraoperative MR images using a real-time intraoperative segmentation algorithm. The algorithm has been applied to a sequence of intraoperative MR images of the brain exhibiting brain shift and tumor resection. Our results characterize the brain shift after opening of the dura and at the different stages of tumor resection, and brain swelling afterwards. Analysis of the average deformation capture was assessed by comparing landmarks identified manually and the results indicate an accuracy of 0.7+/-0.6 mm (mean+/-S.D.) for boundary surface landmarks, of 0.9+/-0.6 mm for landmarks inside the boundary surfaces, and 1.6+/-0.9 mm for landmarks in the vicinity of the tumor.     

Fast and robust multi-atlas segmentation of brain magnetic resonance images

We introduce an optimised pipeline for multi-atlas brain MRI segmentation. Both accuracy and speed of segmentation are considered. We study different similarity measures used in non-rigid registration. We show that intensity differences for intensity normalised images can be used instead of standard normalised mutual information in registration without compromising the accuracy but leading to threefold decrease in the computation time. We study and validate also different methods for atlas selection. Finally, we propose two new approaches for combining multi-atlas segmentation and intensity modelling based on segmentation using expectation maximisation (EM) and optimisation via graph cuts. The segmentation pipeline is evaluated with two data cohorts: IBSR data (N = 18, six subcortial structures: thalamus, caudate, putamen, pallidum, hippocampus, amygdala) and ADNI data (N =   60, hippocampus). The average similarity index between automatically and manually generated volumes was 0.849 (IBSR, six subcortical structures) and 0.880 (ADNI, hippocampus). The correlation coefficient for hippocampal volumes was 0.95 with the ADNI data. The computation time using a standard multicore PC computer was about 3–4 min. Our results compare favourably with other recently published results.     

A global optimisation method for robust affine registration of brain images

Registration is an important component of medical image analysis and for analysing large amounts of data it is desirable to have fully automatic registration methods. Many different automatic registration methods have been proposed to date, and almost all share a common mathematical framework - one of optimising a cost function. To date little attention has been focused on the optimisation method itself, even though the success of most registration methods hinges on the quality of this optimisation. This paper examines the assumptions underlying the problem of registration for brain images using inter-modal voxel similarity measures. It is demonstrated that the use of local optimisation methods together with the standard multi-resolution approach is not sufficient to reliably find the global minimum. To address this problem, a global optimisation method is proposed that is specifically tailored to this form of registration. A full discussion of all the necessary implementation details is included as this is an important part of any practical method. Furthermore, results are presented for inter-modal, inter-subject registration experiments that show that the proposed method is more reliable at finding the global minimum than several of the currently available registration packages in common usage.     

Comparison of edge-based and ridge-based registration of CT and MR brain images

In modern medicine, several different imaging techniques are frequently employed in the study of a single patient. This is useful, since different images show complementary information on the functionality and/or structure of the anatomy examined. This very difference between modalities, however, complicates the problem of proper registration of the images involved, and rules out the most basic approaches—like direct grey value correlation—to achieve registration. The observation that some common structures will always exist is supportive of the statement that registration may be feasible using edges or ridges present in the images. The existence of such structures defined in the binary sense is questionable, however, and their extraction from images requires a segmentation by definition. In this paper we propose to use fuzzy edgeness and ridgeness images, thus avoiding the need for segmentation and using more of the available information from the original images. We will show that such fuzzy images can be used to achieve accurate registration. Several ridgeness and edgeness computing operators were compared. The best registration results were obtained using a gradient magnitude operator.     

Fast and robust extraction of hippocampus from MR images for diagnostics of Alzheimer's disease

Assessment of temporal lobe atrophy from magnetic resonance images is a part of clinical guidelines for the diagnosis of prodromal Alzheimer's disease. As hippocampus is known to be among the first areas affected by the disease, fast and robust definition of hippocampus volume would be of great importance in the clinical decision making. We propose a method for computing automatically the volume of hippocampus using a modified multi-atlas segmentation framework, including an improved initialization of the framework and the correction of partial volume effect. The method produced a high similarity index, 0.87, and correlation coefficient, 0.94, with semi-automatically generated segmentations. When comparing hippocampus volumes extracted from 1.5T and 3T images, the absolute value of the difference was low: 3.2% of the volume. The correct classification rate for Alzheimer's disease and cognitively normal cases was about 80% while the accuracy 65% was obtained for classifying stable and progressive mild cognitive impairment cases. The method was evaluated in three cohorts consisting altogether about 1000 cases, the main emphasis being in the analysis of the ADNI cohort. The computation time of the method is about 2 minutes on a standard laptop computer. The results show a clear potential for applying the method in clinical practice.     

Segmentation of brain 3D MR images using level sets and dense registration

This paper presents a strategy for the segmentation of brain from volumetric MR images which integrates 3D segmentation and 3D registration processes. The segmentation process is based on the level set formalism. A closed 3D surface propagates towards the desired boundaries through the iterative evolution of a 4D implicit function. In this work, the propagation relies on a robust evolution model including adaptive parameters. These depend on the input data and on statistical distribution models. The main contribution of this paper is the use of an automatic registration method to initialize the surface, as an alternative solution to manual initialization. The registration is achieved through a robust multiresolution and multigrid minimization scheme. This coupling significantly improves the quality of the method, since the segmentation is faster, more reliable and fully automatic. Quantitative and qualitative results on both synthetic and real volumetric brain MR images are presented and discussed.     

Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images

The aim of deformable brain image registration is to align anatomical structures, which can potentially vary with large and complex deformations. Anatomical structures vary in size and shape, requiring the registration algorithm to estimate deformation fields at various degrees of complexity. Here, we present a difficulty-aware model based on an attention mechanism to automatically identify hard-to-register regions, allowing better estimation of large complex deformations. The difficulty-aware model is incorporated into a cascaded neural network consisting of three sub-networks to fully leverage both global and local contextual information for effective registration. The first sub-network is trained at the image level to predict a coarse-scale deformation field, which is then used for initializing the subsequent sub-network. The next two sub-networks progressively optimize at the patch level with different resolutions to predict a fine-scale deformation field. Embedding difficulty-aware learning into the hierarchical neural network allows harder patches to be identified in the deeper sub-networks at higher resolutions for refining the deformation field. Experiments conducted on four public datasets validate that our method achieves promising registration accuracy with better preservation of topology, compared with state-of-the-art registration methods.     

Improved optimization for the robust and accurate linear registration and motion correction of brain images

Linear registration and motion correction are important components of structural and functional brain image analysis. Most modern methods optimize some intensity-based cost function to determine the best registration. To date, little attention has been focused on the optimization method itself, even though the success of most registration methods hinges on the quality of this optimization. This paper examines the optimization process in detail and demonstrates that the commonly used multiresolution local optimization methods can, and do, get trapped in local minima. To address this problem, two approaches are taken: (1) to apodize the cost function and (2) to employ a novel hybrid global-local optimization method. This new optimization method is specifically designed for registering whole brain images. It substantially reduces the likelihood of producing misregistrations due to being trapped by local minima. The increased robustness of the method, compared to other commonly used methods, is demonstrated by a consistency test. In addition, the accuracy of the registration is demonstrated by a series of experiments with motion correction. These motion correction experiments also investigate how the results are affected by different cost functions and interpolation methods.     

Cross contrast multi-channel image registration using image synthesis for MR brain images

Multi-modal deformable registration is important for many medical image analysis tasks such as atlas alignment, image fusion, and distortion correction. Whereas a conventional method would register images with different modalities using modality independent features or information theoretic metrics such as mutual information, this paper presents a new framework that addresses the problem using a two-channel registration algorithm capable of using mono-modal similarity measures such as sum of squared differences or cross-correlation. To make it possible to use these same-modality measures, image synthesis is used to create proxy images for the opposite modality as well as intensity-normalized images from each of the two available images. The new deformable registration framework was evaluated by performing intra-subject deformation recovery, intra-subject boundary alignment, and inter-subject label transfer experiments using multi-contrast magnetic resonance brain imaging data. Three different multi-channel registration algorithms were evaluated, revealing that the framework is robust to the multi-channel deformable registration algorithm that is used. With a single exception, all results demonstrated improvements when compared against single channel registrations using the same algorithm with mutual information.     

Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms

Simulated deformations and images can act as the gold standard for evaluating various template-based image segmentation and registration algorithms. Traditional deformable simulation methods, such as the use of analytic deformation fields or the displacement of landmarks followed by some form of interpolation, are often unable to construct rich (complex) and/or realistic deformations of anatomical organs. This paper presents new methods aiming to automatically simulate realistic inter- and intra-individual deformations. The paper first describes a statistical approach to capturing inter-individual variability of high-deformation fields from a number of examples (training samples). In this approach, Wavelet-Packet Transform (WPT) of the training deformations and their Jacobians, in conjunction with a Markov random field (MRF) spatial regularization, are used to capture both coarse and fine characteristics of the training deformations in a statistical fashion. Simulated deformations can then be constructed by randomly sampling the resultant statistical distribution in an unconstrained or a landmark-constrained fashion. The paper also describes a model for generating tissue atrophy or growth in order to simulate intra-individual brain deformations. Several sets of simulated deformation fields and respective images are generated, which can be used in the future for systematic and extensive validation studies of automated atlas-based segmentation and deformable registration methods. The code and simulated data are available through our Web site.     

AMNet: Adaptive multi-level network for deformable registration of 3D brain MR images

Three-dimensional (3D) deformable image registration is a fundamental technique in medical image analysis tasks. Although it has been extensively investigated, current deep-learning-based registration models may face the challenges posed by deformations with various degrees of complexity. This paper proposes an adaptive multi-level registration network (AMNet) to retain the continuity of the deformation field and to achieve high-performance registration for 3D brain MR images. First, we design a lightweight registration network with an adaptive growth strategy to learn deformation field from multi-level wavelet sub-bands, which facilitates both global and local optimization and achieves registration with high performance. Second, our AMNet is designed for image-wise registration, which adapts the local importance of a region in accordance with the complexity degrees of its deformation, and thereafter improves the registration efficiency and maintains the continuity of the deformation field. Experimental results from five publicly-available brain MR datasets and a synthetic brain MR dataset show that our method achieves superior performance against state-of-the-art medical image registration approaches.     

Fast and accurate registration of cranial CT images with A-mode ultrasound

Purpose  Within the CRANIO project, a navigation module based on preoperative computed tomography (CT) data was developed for Computer and Robot Assisted Neurosurgery. The approach followed for non-invasive user-interactive registration of cranial CT images with the physical operating space consists of surface-based registration following pre-registration based on anatomical landmarks. Surface-based registration relies on bone surface points digitized transcutaneously by means of an optically tracked A-mode ultrasound (US) probe. As probe alignment and thus bone surface point digitization may be time-consuming, we investigated how to obtain high registration accuracy despite inaccurate pre-registration and a limited number of digitized bone surface points. Furthermore, we aimed at efficient man-machine-interaction during the probe alignment process. Finally, we addressed the problem of registration plausibility estimation in our approach. Method  We modified the Iterative Closest Point (ICP) algorithm, presented by Besl and McKay and frequently used for surface-based registration, such that it can escape from local minima of the cost function to be iteratively minimized. The random-based ICP (R-ICP) we developed is less influenced by the quality of the pre-registration as it can escape from local minima close to the starting point for iterative optimization in the 6D domain of rigid transformations. The R-ICP is also better suited to approximate the global minimum as it can escape from local minima in the vicinity of the global minimum, too. Furthermore, we developed both CT-less and CT-based probe alignment tools along with appropriate man-machine strategies for a more time-efficient palpation process. To improve registration reliability, we developed a simple plausibility test based on data readily available after registration. Results  In a cadaver study, where we evaluated the R-ICP algorithm, the probe alignment tools, and the plausibility test, the R-ICP algorithm consistently outperformed the ICP algorithm. Almost no influence of the pre-registration on the final R-ICP registration accuracy could be observed. The probe alignment tools were judged to be useful and allowed for the digitization of 18 bone surface points within 2 min on average. The plausibility test was helpful to detect poor registration accuracy. Conclusion  The R-ICP algorithm can provide high registration accuracy despite inaccurate pre-registration and a very limited number of data points. R-ICP registration was shown to be practical and robust versus the quality of the pre-registration. Time-efficiency of the cranial palpation process may be greatly increased and should encourage clinical acceptance.     

Automated 3-D registration of MR and CT images of the head

This paper discusses the application of voxel similarity measures in the automated registration of clinically acquired MR and CT data of the head. We describe a novel single-start multi-resolution approach to the optimization of these measures, and the issues involved in applying this to data having a range of different fields of view and sampling resolution. We compare four proposed measures of voxel similarity using the same optimization scheme when presented with 10 pairs of images with a range of initial misregistrations. The registration estimates are compared with those provided by manual point-based registration and evaluated by visual inspection to give an assessment of the robustness and accuracy of the different measures. One full-volume CT image set is used to investigate the performance of each measure when used to align truncated images from different regions in the head. The soft tissue correlation and mutual information measures were found to provide the most robust measures of misregistration, providing results comparable to or better than those from manual point-based registration for all but the most truncated image volumes.     

Fast and robust 3D ultrasound registration – Block and game theoretic matching

Real-time 3D US has potential for image guidance in minimally invasive liver interventions. However, motion caused by patient breathing makes it hard to visualize a localized area, and to maintain alignment with pre-operative information. In this work we develop a fast affine registration framework to compensate in real-time for liver motion/displacement due to breathing. The affine registration of two consecutive ultrasound volumes in time is performed using block-matching. For a set of evenly distributed points in one volume and their correspondences in the other volume, we propose a robust outlier rejection method to reject false matches. The inliers are then used to determine the affine transformation. The approach is evaluated on 13 4D ultrasound sequences acquired from 8 subjects. For 91 pairs of 3D ultrasound volumes selected from these sequences, a mean registration error of 1.8 mm is achieved. A graphics processing unit (GPU) implementation runs the 3D US registration at 8 Hz.     

Automated separation of gray and white matter from MR images of the human brain

A simple automatic procedure for segmentation of gray and white matter in high resolution 1.5T T1-weighted MR human brain images was developed and validated. The algorithm is based on histogram shape analysis of MR images that were corrected for scanner nonuniformity. Calibration and validation was done on a set of 80 MR images of human brains. The automatic method's values for the gray and white matter volumes were compared with the values from thresholds set twice by the best three of six raters. The automatic procedure was shown to perform as good as the best rater, where the average result of the best three raters was taken as reference. The method was also compared with two other histogram-based threshold methods, which yielded comparable results. The conclusion of the study thus is that automated threshold based methods can separate gray and white matter from MR brain images as reliably as human raters using a thresholding procedure.     

Deformable registration of diffusion tensor MR images with explicit orientation optimization

In this paper, we present a novel deformable registration algorithm for diffusion tensor MR images that enables explicit optimization of tensor reorientation. The optimization seeks a piecewise affine transformation that divides the image domain into uniform regions and transform each region affinely. The objective function captures both the image similarity and the smoothness of the transformation across region boundaries. The image similarity enables explicit orientation optimization by incorporating tensor reorientation, which is necessary for warping diffusion tensor images. The objective function is formulated in a way that allows explicit implementation of analytic derivatives to drive fast and accurate optimization using the conjugate gradient method. By explicitly optimizing tensor reorientation, the algorithm is designed to take advantage of similarity measures comparing tensors as a whole. The optimal transformation is hierarchically refined in a subdivision framework. A comparison with affine registration for inter-subject normalization of 8 subjects shows that the proposed algorithm improves the alignment of several major white matter structures examined: the anterior thalamic radiations, the inferior fronto-occipital fasciculi, the corticospinal/corticobulbar tracts and the genu and the splenium of the corpus callosum. The alignment of white matter structures is assessed using a novel scheme of computing distances between the corresponding fiber bundles derived from tractography.     

A robust method for extraction and automatic segmentation of brain images

A new protocol is introduced for brain extraction and automatic tissue segmentation of MR images. For the brain extraction algorithm, proton density and T2-weighted images are used to generate a brain mask encompassing the full intracranial cavity. Segmentation of brain tissues into gray matter (GM), white matter (WM), and cerebral spinal fluid (CSF) is accomplished on a T1-weighted image after applying the brain mask. The fully automatic segmentation algorithm is histogram-based and uses the Expectation Maximization algorithm to model a four-Gaussian mixture for both global and local histograms. The means of the local Gaussians for GM, WM, and CSF are used to set local thresholds for tissue classification. Reproducibility of the extraction procedure was excellent, with average variation in intracranial capacity (TIC) of 0.13 and 0.66% TIC in 12 healthy normal and 33 Alzheimer brains, respectively. Repeatability of the segmentation algorithm, tested on healthy normal images, indicated scan-rescan differences in global tissue volumes of less than 0.30% TIC. Reproducibility at the regional level was established by comparing segmentation results within the 12 major Talairach subdivisions. Accuracy of the algorithm was tested on a digital brain phantom, and errors were less than 1% of the phantom volume. Maximal Type I and Type II classification errors were low, ranging between 2.2 and 4.3% of phantom volume. The algorithm was also insensitive to variation in parameter initialization values. The protocol is robust, fast, and its success in segmenting normal as well as diseased brains makes it an attractive clinical application.     

Fast and robust extraction of surrogate respiratory signal from intra-operative liver ultrasound images

Purpose

   In model-based respiratory motion estimation for the liver or other abdominal organs, the surrogate respiratory signal is usually obtained by using special tracking devices from skin or diaphragm, and subsequently applied to parameterize a 4D motion model for prediction or compensation. However, due to the intrinsic limits and economical costs of these tracking devices, the identification of the respiratory signal directly from intra-operative ultrasound images is a more attractive alternative.

Methods

   We propose a fast and robust method to extract the respiratory motion of the liver from an intra-operative 2D ultrasound image sequence. Our method employs a preprocess to remove speckle-like noises in the ultrasound images and utilizes the normalized cross-correlation to measure the image similarity fast. More importantly, we present a novel adaptive search strategy, which makes full use of the inter-frame dependency of the image sequence. This search strategy narrows the search range of the optimal matching, thus greatly reduces the search time, and makes the matching process more robust and accurate.

Results

   The experimental results on four volunteers demonstrate that our method is able to extract the respiratory signal from an image sequence of 256 image frames in 5 s. The quantitative evaluation using the correlation coefficient reveals that the respiratory motion, extracted near the liver boundaries and vessels, is highly consistent with the reference motion tracked by an EM device.

Conclusions

   Our method can use 2D ultrasound to track natural landmarks from the liver as surrogate respiratory signal and hence provide a feasible solution to replace special tracking devices.     

Co-registration of intra-operative brain surface photographs and pre-operative MR images

Purpose

   Brain shift, the change in configuration of the brain after opening the dura mater, is a significant problem for neuronavigation. Brain structures at intra-operative deformed positions must be matched with corresponding structures in the pre-operative 3D planning data. A method to co-register the cortical surface from intra-operative microscope images with pre-operative MRI-segmented data was developed and tested.

Methods

   Automated classification of sulci on MRI-extracted cortical surfaces was tested by comparison with user guided marking of prominent sulci on an intra-operative photography. A variational registration method with a fidelity energy for 3D deformations of the cortical surface in conjunction with a higher-order, linear elastic prior energy was used for the actual registration. The minimization of this energy was performed with a regularized gradient descent scheme using finite elements for spatial discretization. The sulcal classification method was tested on eight different clinical MRI data sets by comparison of the deformed MRI scans with intra-operative photographs of the brain surface.

Results

   User intervention was required for marking sulci on the photographs demonstrating the potential for incorporating an automatic classifier. The actual registration was validated first on an artificial testbed. The complete algorithm for the co-registration of actual clinical MRI data was successful for eight different patients.

Conclusions

   Pre-operative MRI scans can be registered to intra-operative brain surface photographs using a surface-to-surface registration method. This co-registration method has potential applications in neurosurgery, particularly during functional procedures.     

MR-based automatic delineation of volumes of interest in human brain PET images using probability maps

The purpose of this study was to develop and validate an observer-independent approach for automatic generation of volume-of-interest (VOI) brain templates to be used in emission tomography studies of the brain. The method utilizes a VOI probability map created on the basis of a database of several subjects' MR-images, where VOI sets have been defined manually. High-resolution structural MR-images and 5-HT(2A) receptor binding PET-images (in terms of (18)F-altanserin binding) from 10 healthy volunteers and 10 patients with mild cognitive impairment were included for the analysis. A template including 35 VOIs was manually delineated on the subjects' MR images. Through a warping algorithm template VOI sets defined from each individual were transferred to the other subjects MR-images and the voxel overlap was compared to the VOI set specifically drawn for that particular individual. Comparisons were also made for the VOI templates 5-HT(2A) receptor binding values. It was shown that when the generated VOI set is based on more than one template VOI set, delineation of VOIs is better reproduced and shows less variation as compared both to transfer of a single set of template VOIs as well as manual delineation of the VOI set. The approach was also shown to work equally well in individuals with pronounced cerebral atrophy. Probability-map-based automatic delineation of VOIs is a fast, objective, reproducible, and safe way to assess regional brain values from PET or SPECT scans. In addition, the method applies well in elderly subjects, even in the presence of pronounced cerebral atrophy.