Segmentation of brain 3D MR images using level sets and dense registration

This paper presents a strategy for the segmentation of brain from volumetric MR images which integrates 3D segmentation and 3D registration processes. The segmentation process is based on the level set formalism. A closed 3D surface propagates towards the desired boundaries through the iterative evolution of a 4D implicit function. In this work, the propagation relies on a robust evolution model including adaptive parameters. These depend on the input data and on statistical distribution models. The main contribution of this paper is the use of an automatic registration method to initialize the surface, as an alternative solution to manual initialization. The registration is achieved through a robust multiresolution and multigrid minimization scheme. This coupling significantly improves the quality of the method, since the segmentation is faster, more reliable and fully automatic. Quantitative and qualitative results on both synthetic and real volumetric brain MR images are presented and discussed.     

Serial registration of intraoperative MR images of the brain

The increased use of image-guided surgery systems during neurosurgery has brought to prominence the inaccuracies of conventional intraoperative navigation systems caused by shape changes such as those due to brain shift. We propose a method to track the deformation of the brain and update preoperative images using intraoperative MR images acquired at different crucial time points during surgery. We use a deformable surface matching algorithm to capture the deformation of boundaries of key structures (cortical surface, ventricles and tumor) throughout the neurosurgical procedure, and a linear finite element elastic model to infer a volumetric deformation. The boundary data are extracted from intraoperative MR images using a real-time intraoperative segmentation algorithm. The algorithm has been applied to a sequence of intraoperative MR images of the brain exhibiting brain shift and tumor resection. Our results characterize the brain shift after opening of the dura and at the different stages of tumor resection, and brain swelling afterwards. Analysis of the average deformation capture was assessed by comparing landmarks identified manually and the results indicate an accuracy of 0.7+/-0.6 mm (mean+/-S.D.) for boundary surface landmarks, of 0.9+/-0.6 mm for landmarks inside the boundary surfaces, and 1.6+/-0.9 mm for landmarks in the vicinity of the tumor.     

Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis

In prostate cancer radiotherapy, computed tomography (CT) is widely used for dose planning purposes. However, because CT has low soft tissue contrast, it makes manual contouring difficult for major pelvic organs. In contrast, magnetic resonance imaging (MRI) provides high soft tissue contrast, which makes it ideal for accurate manual contouring. Therefore, the contouring accuracy on CT can be significantly improved if the contours in MRI can be mapped to CT domain by registering MRI with CT of the same subject, which would eventually lead to high treatment efficacy. In this paper, we propose a bi-directional image synthesis based approach for MRI-to-CT pelvic image registration. First, we use patch-wise random forest with auto-context model to learn the appearance mapping from CT to MRI domain, and then vice versa. Consequently, we can synthesize a pseudo-MRI whose anatomical structures are exactly same with CT but with MRI-like appearance, and a pseudo-CT as well. Then, our MRI-to-CT registration can be steered in a dual manner, by simultaneously estimating two deformation pathways: 1) one from the pseudo-CT to the actual CT and 2) another from actual MRI to the pseudo-MRI. Next, a dual-core deformation fusion framework is developed to iteratively and effectively combine these two registration pathways by using complementary information from both modalities. Experiments on a dataset with real pelvic CT and MRI have shown improved registration performance of the proposed method by comparing it to the conventional registration methods, thus indicating its high potential of translation to the routine radiation therapy.     

Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images

The aim of deformable brain image registration is to align anatomical structures, which can potentially vary with large and complex deformations. Anatomical structures vary in size and shape, requiring the registration algorithm to estimate deformation fields at various degrees of complexity. Here, we present a difficulty-aware model based on an attention mechanism to automatically identify hard-to-register regions, allowing better estimation of large complex deformations. The difficulty-aware model is incorporated into a cascaded neural network consisting of three sub-networks to fully leverage both global and local contextual information for effective registration. The first sub-network is trained at the image level to predict a coarse-scale deformation field, which is then used for initializing the subsequent sub-network. The next two sub-networks progressively optimize at the patch level with different resolutions to predict a fine-scale deformation field. Embedding difficulty-aware learning into the hierarchical neural network allows harder patches to be identified in the deeper sub-networks at higher resolutions for refining the deformation field. Experiments conducted on four public datasets validate that our method achieves promising registration accuracy with better preservation of topology, compared with state-of-the-art registration methods.     

MR image synthesis by contrast learning on neighborhood ensembles

Automatic processing of magnetic resonance images is a vital part of neuroscience research. Yet even the best and most widely used medical image processing methods will not produce consistent results when their input images are acquired with different pulse sequences. Although intensity standardization and image synthesis methods have been introduced to address this problem, their performance remains dependent on knowledge and consistency of the pulse sequences used to acquire the images. In this paper, an image synthesis approach that first estimates the pulse sequence parameters of the subject image is presented. The estimated parameters are then used with a collection of atlas or training images to generate a new atlas image having the same contrast as the subject image. This additional image provides an ideal source from which to synthesize any other target pulse sequence image contained in the atlas. In particular, a nonlinear regression intensity mapping is trained from the new atlas image to the target atlas image and then applied to the subject image to yield the particular target pulse sequence within the atlas. Both intensity standardization and synthesis of missing tissue contrasts can be achieved using this framework. The approach was evaluated on both simulated and real data, and shown to be superior in both intensity standardization and synthesis to other established methods.     

AMNet: Adaptive multi-level network for deformable registration of 3D brain MR images

Three-dimensional (3D) deformable image registration is a fundamental technique in medical image analysis tasks. Although it has been extensively investigated, current deep-learning-based registration models may face the challenges posed by deformations with various degrees of complexity. This paper proposes an adaptive multi-level registration network (AMNet) to retain the continuity of the deformation field and to achieve high-performance registration for 3D brain MR images. First, we design a lightweight registration network with an adaptive growth strategy to learn deformation field from multi-level wavelet sub-bands, which facilitates both global and local optimization and achieves registration with high performance. Second, our AMNet is designed for image-wise registration, which adapts the local importance of a region in accordance with the complexity degrees of its deformation, and thereafter improves the registration efficiency and maintains the continuity of the deformation field. Experimental results from five publicly-available brain MR datasets and a synthetic brain MR dataset show that our method achieves superior performance against state-of-the-art medical image registration approaches.     

Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms

Simulated deformations and images can act as the gold standard for evaluating various template-based image segmentation and registration algorithms. Traditional deformable simulation methods, such as the use of analytic deformation fields or the displacement of landmarks followed by some form of interpolation, are often unable to construct rich (complex) and/or realistic deformations of anatomical organs. This paper presents new methods aiming to automatically simulate realistic inter- and intra-individual deformations. The paper first describes a statistical approach to capturing inter-individual variability of high-deformation fields from a number of examples (training samples). In this approach, Wavelet-Packet Transform (WPT) of the training deformations and their Jacobians, in conjunction with a Markov random field (MRF) spatial regularization, are used to capture both coarse and fine characteristics of the training deformations in a statistical fashion. Simulated deformations can then be constructed by randomly sampling the resultant statistical distribution in an unconstrained or a landmark-constrained fashion. The paper also describes a model for generating tissue atrophy or growth in order to simulate intra-individual brain deformations. Several sets of simulated deformation fields and respective images are generated, which can be used in the future for systematic and extensive validation studies of automated atlas-based segmentation and deformable registration methods. The code and simulated data are available through our Web site.     

Fast and robust registration of PET and MR images of human brain

In recent years, mutual information has proved to be an excellent criterion for registration of intra-individual images from different modalities. Multi-resolution coarse-to-fine optimization was proposed for speeding-up of the registration process. The aim of our work was to further improve registration speed without compromising robustness or accuracy. We present and evaluate two procedures for co-registration of positron emission tomography (PET) and magnetic resonance (MR) images of human brain that combine a multi-resolution approach with an automatic segmentation of input image volumes into areas of interest and background. We show that an acceleration factor of 10 can be achieved for clinical data and that a suitable preprocessing can improve robustness of registration. Emphasis was laid on creation of an automatic registration system that could be used routinely in a clinical environment. For this purpose, an easy-to-use graphical user interface has been developed. It allows physicians with no special knowledge of the registration algorithm to perform a fast and reliable alignment of images. Registration progress is presented on the fly on a fusion of images and enables visual checking during a registration.     

Comparison of edge-based and ridge-based registration of CT and MR brain images

In modern medicine, several different imaging techniques are frequently employed in the study of a single patient. This is useful, since different images show complementary information on the functionality and/or structure of the anatomy examined. This very difference between modalities, however, complicates the problem of proper registration of the images involved, and rules out the most basic approaches—like direct grey value correlation—to achieve registration. The observation that some common structures will always exist is supportive of the statement that registration may be feasible using edges or ridges present in the images. The existence of such structures defined in the binary sense is questionable, however, and their extraction from images requires a segmentation by definition. In this paper we propose to use fuzzy edgeness and ridgeness images, thus avoiding the need for segmentation and using more of the available information from the original images. We will show that such fuzzy images can be used to achieve accurate registration. Several ridgeness and edgeness computing operators were compared. The best registration results were obtained using a gradient magnitude operator.     

Predict brain MR image registration via sparse learning of appearance and transformation

We propose a new approach to register the subject image with the template by leveraging a set of intermediate images that are pre-aligned to the template. We argue that, if points in the subject and the intermediate images share similar local appearances, they may have common correspondence in the template. In this way, we learn the sparse representation of a certain subject point to reveal several similar candidate points in the intermediate images. Each selected intermediate candidate can bridge the correspondence from the subject point to the template space, thus predicting the transformation associated with the subject point at the confidence level that relates to the learned sparse coefficient. Following this strategy, we first predict transformations at selected key points, and retain multiple predictions on each key point, instead of allowing only a single correspondence. Then, by utilizing all key points and their predictions with varying confidences, we adaptively reconstruct the dense transformation field that warps the subject to the template. We further embed the prediction–reconstruction protocol above into a multi-resolution hierarchy. In the final, we refine our estimated transformation field via existing registration method in effective manners. We apply our method to registering brain MR images, and conclude that the proposed framework is competent to improve registration performances substantially.     

Single-image super-resolution of brain MR images using overcomplete dictionaries

Resolution in Magnetic Resonance (MR) is limited by diverse physical, technological and economical considerations. In conventional medical practice, resolution enhancement is usually performed with bicubic or B-spline interpolations, strongly affecting the accuracy of subsequent processing steps such as segmentation or registration. This paper presents a sparse-based super-resolution method, adapted for easily including prior knowledge, which couples up high and low frequency information so that a high-resolution version of a low-resolution brain MR image is generated. The proposed approach includes a whole-image multi-scale edge analysis and a dimensionality reduction scheme, which results in a remarkable improvement of the computational speed and accuracy, taking nearly 26 min to generate a complete 3D high-resolution reconstruction. The method was validated by comparing interpolated and reconstructed versions of 29 MR brain volumes with the original images, acquired in a 3T scanner, obtaining a reduction of 70% in the root mean squared error, an increment of 10.3 dB in the peak signal-to-noise ratio, and an agreement of 85% in the binary gray matter segmentations. The proposed method is shown to outperform a recent state-of-the-art algorithm, suggesting a substantial impact in voxel-based morphometry studies.     

Analysis of serial magnetic resonance images of mouse brains using image registration

The aim of this paper is to investigate techniques that can identify and quantify cross-sectional differences and longitudinal changes in vivo from magnetic resonance images of murine models of brain disease. Two different approaches have been compared. The first approach is a segmentation-based approach: Each subject at each time point is automatically segmented into a number of anatomical structures using atlas-based segmentation. This allows cross-sectional and longitudinal analyses of group differences on a structure-by-structure basis. The second approach is a deformation-based approach: Longitudinal changes are quantified by the registration of each subject's follow-up images to that subject's baseline image. In addition the baseline images can be registered to an atlas allowing voxel-wise analysis of cross-sectional differences between groups. Both approaches have been tested on two groups of mice: A transgenic model of Alzheimer's disease and a wild-type background strain, using serial imaging performed over the age range from 6–14 months. We show that both approaches are able to identify longitudinal and cross-sectional differences. However, atlas-based segmentation suffers from the inability to detect differences across populations and across time in regions which are much smaller than the anatomical regions. In contrast to this, the deformation-based approach can detect statistically significant differences in highly localized areas.     

Spatio-temporal free-form registration of cardiac MR image sequences

In this paper we present two registration algorithms for the spatio-temporal alignment of cardiac MR image sequences. Both algorithms have the ability to correct spatial misalignment between the images sequences caused by global and local shape differences. In addition, they have the ability to correct temporal misalignment caused by differences in the length of the cardiac cycles and by differences in the dynamic properties of the hearts. The algorithms use a 4D deformable transformation model which is separated into spatial and temporal components. The first registration algorithm optimizes the spatial and temporal transformation models simultaneously, while the second registration algorithm optimizes the temporal transformation component before optimizing the spatial component. For the evaluation of the spatio-temporal registration methods we have acquired 15 MR image sequences from healthy volunteers. The registration methods were quantitatively evaluated by measuring the overlap and surface distance of anatomical regions and qualitatively by visual inspection. The results demonstrate that a significant improvement in the alignment of the image sequences is achieved by the use of the deformable spatio-temporal transformation model. We demonstrate the use of the method for the construction of a probabilistic MR cardiac atlas representing the anatomy and function of a healthy heart.     

Hippocampus segmentation in MR images using atlas registration, voxel classification, and graph cuts

Since hippocampal volume has been found to be an early biomarker for Alzheimer's disease, there is large interest in automated methods to accurately, robustly, and reproducibly extract the hippocampus from MRI data. In this work we present a segmentation method based on the minimization of an energy functional with intensity and prior terms, which are derived from manually labelled training images. The intensity energy is based on a statistical intensity model that is learned from the training images. The prior energy consists of a spatial and regularity term. The spatial prior is obtained from a probabilistic atlas created by registering the training images to the unlabelled target image, and deforming and averaging the training labels. The regularity prior energy encourages smooth segmentations. The resulting energy functional is globally minimized using graph cuts. The method was evaluated using image data from a population-based study on diseases among the elderly. Two set of images were used: a small set of 20 manually labelled MR images and a larger set of 498 images, for which manual volume measurements were available, but no segmentations. This data was previously used in a volumetry study that found significant associations between hippocampal volume and cognitive decline and incidence of dementia. Cross-validation experiments with the labelled set showed similarity indices of 0.852 and 0.864 and mean surface distances of 0.40 and 0.36 mm for the left and right hippocampus. 83% of the automated segmentations of the large set were rated as ‘good’ by a trained observer. Also, the proposed method was used to repeat the manual hippocampal volumetry study. The automatically obtained hippocampal volumes showed significant associations with cognitive decline and dementia, similar to the manually measured volumes. Finally, direct quantitative and qualitative comparisons showed that the proposed method outperforms a multi-atlas based segmentation method.     

Multi-modal registration for correlative microscopy using image analogies

Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies for the same biological specimen. In this paper, we propose an image registration method for correlative microscopy, which is challenging due to the distinct appearance of biological structures when imaged with different modalities. Our method is based on image analogies and allows to transform images of a given modality into the appearance-space of another modality. Hence, the registration between two different types of microscopy images can be transformed to a mono-modality image registration. We use a sparse representation model to obtain image analogies. The method makes use of corresponding image training patches of two different imaging modalities to learn a dictionary capturing appearance relations. We test our approach on backscattered electron (BSE) scanning electron microscopy (SEM)/confocal and transmission electron microscopy (TEM)/confocal images. We perform rigid, affine, and deformable registration via B-splines and show improvements over direct registration using both mutual information and sum of squared differences similarity measures to account for differences in image appearance.     

循环生成对抗网络基于颅脑MR图生成伪CT图模型

目的 采用改进循环生成对抗网络（UCycleGAN）基于颅脑MR图映射模型生成伪CT图。方法 对50例鼻咽癌颅脑MR图与CT图进行配准及预处理；以U-net网络并添加L1距离函数替换原始循环GAN （CycleGAN）模型生成器的深度残差网络。随机选取40例图像作为训练数据对UCycleGAN模型进行训练，将剩余10例用于测试；比较生成伪CT图与原始图像质量的差异，并与以ResNet、U-net的CycleGAN以及Pix2Pix生成的图像进行对比。结果 相比其他模型，以UCycleGAN模型生成的伪CT图与原始CT图更为接近，体素平均绝对误差（MAE）为（81.45±3.87） HU，峰值信噪比（PSNR）为（34.13±3.28） dB，结构相似性（SSIM）为0.87±0.03。采用UCycleGAN模型生成的伪CT图的MAE小于、而SSIM明显大于其他3种模型（P均<0.05）；UCycleGAN伪CT图的PSNR大于CycleGAN_ResNet图像（P<0.05）。结论 利用UCycleGAN可基于颅脑MR图生成伪CT图；改良后CycleGAN模型的准确性更高。     

A global optimisation method for robust affine registration of brain images

Registration is an important component of medical image analysis and for analysing large amounts of data it is desirable to have fully automatic registration methods. Many different automatic registration methods have been proposed to date, and almost all share a common mathematical framework - one of optimising a cost function. To date little attention has been focused on the optimisation method itself, even though the success of most registration methods hinges on the quality of this optimisation. This paper examines the assumptions underlying the problem of registration for brain images using inter-modal voxel similarity measures. It is demonstrated that the use of local optimisation methods together with the standard multi-resolution approach is not sufficient to reliably find the global minimum. To address this problem, a global optimisation method is proposed that is specifically tailored to this form of registration. A full discussion of all the necessary implementation details is included as this is an important part of any practical method. Furthermore, results are presented for inter-modal, inter-subject registration experiments that show that the proposed method is more reliable at finding the global minimum than several of the currently available registration packages in common usage.     

混合角点检测算法用于脑磁共振图像配准

目的针对现有角点检测算法的不足,提出结合Harris、Susan的混合角点检测算法,并应用于脑MR图像配准中。方法首先通过Harris算子、Susan算子分别提取图像中Harris角点和Susan角点;然后对Harris角点和Susan角点取并集;通过引入两个加权因子ω1和ω2,分别对Harris角点响应值与Susan角点响应值进行加权计算,获得其角点强度,从而筛选出新的角点集合;通过归一化相关法和投票策略筛选出精确匹配的角点对;最后采用Powell算法进一步优化,获得图像最终配准参数值。结果混合角点检测算法应用于脑MR图像配准能获得较高的配准精度和较好的稳定性。结论相比于目前的角点检测算法,本文算法更适用于脑MR图像配准。     

Deriving high contrast fluorescence microscopy images through low contrast noisy image stacks

Contrast in fluorescence microscopy images allows for the differentiation between different structures by their difference in intensities. However, factors such as point-spread function and noise may reduce it, affecting its interpretability. We identified that fluctuation of emitters in a stack of images can be exploited to achieve increased contrast when compared to the average and Richardson-Lucy deconvolution. We tested our methods on four increasingly challenging samples including tissue, in which case results were comparable to the ones obtained by structured illumination microscopy in terms of contrast.     

Acoustic radiation contrast in MR images for breast cancer diagnostics--initial phantom study

Acoustic radiation contrast in magnetic resonance images is an approach to visualize the changes in ultrasonic loss and viscoelastic changes of the sample with the resolution of a magnetic resonance imaging (MRI) system. By irradiating ultrasound (US) into a tissue-mimicking sample, a displacement along the US beam path caused by the acoustic radiation force is obtained. This displacement varies with the US intensity, the duration of irradiation, the US attenuation and the viscoelastic properties of the sample. US pulses of 2.5 MHz with a duration of 20 ms and an intensity of <17 W/cm² are used. An MRI sequence was programmed to produce images in which the magnitude of the displacement is visualized by gray value changes. In addition, a finite element simulation of the measurements was performed to demonstrate the feasibility of the method. Through examination of the measurements and the simulations, information about viscoelastic changes was achieved. In this work, measurements on different breast phantoms are presented. (E-mail: marcus.radicke@gmx.de)