Enhancement of contrast regions in suboptimal ultrasound images with application to echocardiography

In this paper we propose a novel feature-based contrast enhancement approach to enhance the quality of noisy ultrasound (US) images. Our approach uses a phase-based feature detection algorithm, followed by sparse surface interpolation and subsequent nonlinear postprocessing. We first exploited the intensity-invariant property of phase-based acoustic feature detection to select a set of relevant image features in the data. Then, an approximation to the low-frequency components of the sparse set of selected features was obtained using a fast surface interpolation algorithm. Finally, a nonlinear postprocessing step was applied. Results of applying the method to echocardiographic sequences (2-D + T) are presented. The results demonstrate that the method can successfully enhance the intensity of the interesting features in the image. Better balanced contrasted images are obtained, which is important and useful both for manual processing and assessment by a clinician, and for computer analysis of the sequence. An evaluation protocol is proposed in the case of echocardiographic data and quantitative results are presented. We show that the correction is consistent over time and does not introduce any temporal artefacts. (E-mail: djamal@robots.ox.ac.uk)     

Assimilating intraoperative data with brain shift modeling using the adjoint equations

Biomechanical models of brain deformation are increasingly being used to nonrigidly register preoperative MR (pMR) images of the brain to the surgical scene. These model estimates can potentially be improved by incorporating sparse displacement data available in the operating room (OR), but integrating the intraoperative information with model calculations is a nontrivial problem. We present an inverse method to estimate the unknown boundary and volumetric forces necessary to achieve a least-squares fit between the model and the data that is formulated in terms of the adjoint equations, which are solved directly by the method of representers. The scheme is illustrated in a 2D simulation and in a 2D approximation based on a patient case using actual OR data.     

A novel data calibration scheme for electrical impedance tomography

In electrical impedance tomography surface measurements of voltages and currents are recorded and the image reconstruction algorithm uses this set of boundary data to estimate internal electrical properties of the region under investigation. Therefore correct and accurate modelling of the current and voltage distributions (forward model) is an essential part of any reconstruction method. In this paper, we explored the root cause of a boundary layer effect in the reconstructed conductivity map and found it to be an artefact arising from 2D to 3D data-model mismatch within the imaging algorithm. We propose a data calibration scheme that improves the reconstruction results by removing these boundary or edge effects. We present both two-dimensional and three-dimensional images for agar phantoms using this data calibration scheme which are markedly better than their counterparts recovered when the measurement data are not calibrated with the procedure outlined herein.     

Ultrasound image reconstruction from plane wave radio-frequency data by self-supervised deep neural network

Image reconstruction from radio-frequency (RF) data is crucial for ultrafast plane wave ultrasound (PWUS) imaging. Compared with the traditional delay-and-sum (DAS) method based on relatively imprecise assumptions, sparse regularization (SR) method directly solves the inverse problem of image reconstruction and has presented significant improvement in the image quality when the frame rate remains high. However, the computational complexity of SR is too high for practical implementation, which is inherently associated with its iterative process. In this work, a deep neural network (DNN), which is trained with an incorporated loss function including sparse regularization terms, is proposed to reconstruct PWUS images from RF data with significantly reduced computational time. It is remarkable that, a self-supervised learning scheme, in which the RF data are utilized as both the inputs and the labels during the training process, is employed to overcome the lack of the “ideal” ultrasound images as the labels for DNN. In addition, it has been also verified that the trained network can be used on the RF data obtained with steered plane waves (PWs), and thus the image quality can be further improved with coherent compounding. Using simulation data, the proposed method has significantly shorter reconstruction time (∼10 ms) than the conventional SR method (∼1-5 mins), with comparable spatial resolution and 1.5-dB higher contrast-to-noise ratio (CNR). Besides, the proposed method with single PW can achieve higher CNR than DAS with 75 PWs in reconstruction of in-vivo images of human carotid arteries.     

Optimizing boundary detection via Simulated Search with applications to multi-modal heart segmentation

Segmentation of medical images can be achieved with the help of model-based algorithms. Reliable boundary detection is a crucial component to obtain robust and accurate segmentation results and to enable full automation. This is especially important if the anatomy being segmented is too variable to initialize a mean shape model such that all surface regions are close to the desired contours. Several boundary detection algorithms are widely used in the literature. Most use some trained image appearance model to characterize and detect the desired boundaries. Although parameters of the boundary detection can vary over the model surface and are trained on images, their performance (i.e., accuracy and reliability of boundary detection) can only be assessed as an integral part of the entire segmentation algorithm. In particular, assessment of boundary detection cannot be done locally and independently on model parameterization and internal energies controlling geometric model properties.In this paper, we propose a new method for the local assessment of boundary detection called Simulated Search. This method takes any boundary detection function and evaluates its performance for a single model landmark in terms of an estimated geometric boundary detection error. In consequence, boundary detection can be optimized per landmark during model training. We demonstrate the success of the method for cardiac image segmentation. In particular we show that the Simulated Search improves the capture range and the accuracy of the boundary detection compared to a traditional training scheme. We also illustrate how the Simulated Search can be used to identify suitable classes of features when addressing a new segmentation task. Finally, we show that the Simulated Search enables multi-modal heart segmentation using a single algorithmic framework. On computed tomography and magnetic resonance images, average segmentation errors (surface-to-surface distances) for the four chambers and the trunks of the large vessels are in the order of 0.8 mm. For 3D rotational X-ray angiography images of the left atrium and pulmonary veins, the average error is 1.3 mm. In all modalities, the locally optimized boundary detection enables fully automatic segmentation.     

A magnetic resonance spectroscopy driven initialization scheme for active shape model based prostate segmentation

Segmentation of the prostate boundary on clinical images is useful in a large number of applications including calculation of prostate volume pre- and post-treatment, to detect extra-capsular spread, and for creating patient-specific anatomical models. Manual segmentation of the prostate boundary is, however, time consuming and subject to inter- and intra-reader variability. T2-weighted (T2-w) magnetic resonance (MR) structural imaging (MRI) and MR spectroscopy (MRS) have recently emerged as promising modalities for detection of prostate cancer in vivo. MRS data consists of spectral signals measuring relative metabolic concentrations, and the metavoxels near the prostate have distinct spectral signals from metavoxels outside the prostate. Active Shape Models (ASM's) have become very popular segmentation methods for biomedical imagery. However, ASMs require careful initialization and are extremely sensitive to model initialization. The primary contribution of this paper is a scheme to automatically initialize an ASM for prostate segmentation on endorectal in vivo multi-protocol MRI via automated identification of MR spectra that lie within the prostate. A replicated clustering scheme is employed to distinguish prostatic from extra-prostatic MR spectra in the midgland. The spatial locations of the prostate spectra so identified are used as the initial ROI for a 2D ASM. The midgland initializations are used to define a ROI that is then scaled in 3D to cover the base and apex of the prostate. A multi-feature ASM employing statistical texture features is then used to drive the edge detection instead of just image intensity information alone. Quantitative comparison with another recent ASM initialization method by Cosio showed that our scheme resulted in a superior average segmentation performance on a total of 388 2D MRI sections obtained from 32 3D endorectal in vivo patient studies. Initialization of a 2D ASM via our MRS-based clustering scheme resulted in an average overlap accuracy (true positive ratio) of 0.60, while the scheme of Cosio yielded a corresponding average accuracy of 0.56 over 388 2D MR image sections. During an ASM segmentation, using no initialization resulted in an overlap of 0.53, using the Cosio based methodology resulted in an overlap of 0.60, and using the MRS-based methodology resulted in an overlap of 0.67, with a paired Student's t-test indicating statistical significance to a high degree for all results. We also show that the final ASM segmentation result is highly correlated (as high as 0.90) to the initialization scheme.     

A non-parametric vessel detection method for complex vascular structures

Modern medical imaging techniques enable the acquisition of in vivo high resolution images of the vascular system. Most common methods for the detection of vessels in these images, such as multiscale Hessian-based operators and matched filters, rely on the assumption that at each voxel there is a single cylinder. Such an assumption is clearly violated at the multitude of branching points that are easily observed in all, but the most focused vascular image studies. In this paper, we propose a novel method for detecting vessels in medical images that relaxes this single cylinder assumption. We directly exploit local neighborhood intensities and extract characteristics of the local intensity profile (in a spherical polar coordinate system) which we term as the polar neighborhood intensity profile. We present a new method to capture the common properties shared by polar neighborhood intensity profiles for all the types of vascular points belonging to the vascular system. The new method enables us to detect vessels even near complex extreme points, including branching points. Our method demonstrates improved performance over standard methods on both 2D synthetic images and 3D animal and clinical vascular images, particularly close to vessel branching regions.     

Physical model-based non-rigid registration incorporating statistical shape information

This paper describes two new atlas-based methods of 2D single modality non-rigid registration using the combined power of physical and statistical shape models. The transformations are constrained to be consistent with the physical properties of deformable elastic solids in the first method and those of viscous fluids in the second, to maintain smoothness and continuity. A Bayesian formulation, based on each physical model, an intensity similarity measure, and statistical shape information embedded in corresponding boundary points, is employed to derive more accurate and robust approaches to non-rigid registration. A dense set of forces arises from the intensity similarity measure to accommodate complex anatomical details. A sparse set of forces constrains consistency with statistical shape models derived from a training set. A number of experiments were performed on both synthetic and real medical images of the brain and heart to evaluate the approaches. It is shown that statistical boundary shape information significantly augments and improves physical model-based non-rigid registration and the two methods we present each have advantages under different conditions.     

Deep Consensus Network: Aggregating predictions to improve object detection in microscopy images

Detection of cells and particles in microscopy images is a common and challenging task. In recent years, detection approaches in computer vision achieved remarkable improvements by leveraging deep learning. Microscopy images pose challenges like small and clustered objects, low signal to noise, and complex shape and appearance, for which current approaches still struggle. We introduce Deep Consensus Network, a new deep neural network for object detection in microscopy images based on object centroids. Our network is trainable end-to-end and comprises a Feature Pyramid Network-based feature extractor, a Centroid Proposal Network, and a layer for ensembling detection hypotheses over all image scales and anchors. We suggest an anchor regularization scheme that favours prior anchors over regressed locations. We also propose a novel loss function based on Normalized Mutual Information to cope with strong class imbalance, which we derive within a Bayesian framework. In addition, we introduce an improved algorithm for Non-Maximum Suppression which significantly reduces the algorithmic complexity. Experiments on synthetic data are performed to provide insights into the properties of the proposed loss function and its robustness. We also applied our method to challenging data from the TUPAC16 mitosis detection challenge and the Particle Tracking Challenge, and achieved results competitive or better than state-of-the-art.     

Cardiac material markers from tagged MR images

Tagged magnetic resonance imaging (MRI) has shown great promise in non-invasive analysis of heart motion. To replace implanted markers as a gold standard, however, tagged MRI must be able to track a sparse set of material points, so-called material markers, with high accuracy. This paper presents a new method for generating accurate motion estimates over a sparse set of material points using standard, parallel-tagged MR images. Each tracked point is located at the intersection of three tag surfaces, each of which is estimated using a thin-plate spline. The intersections are determined by an iterative alternating projections algorithm for which a proof of convergence is provided. The resulting data sets are compatible with applications developed to exploit implanted marker data. One set of these material markers from a normal human volunteer is examined in detail using several methods to visualize the markers. Numerical results that include additional studies are also discussed. Finally, an error analysis is presented using a computer-simulated left ventricle for which material markers are tracked with an RMS error of approximately 0.2 mm for typical imaging parameters and noise levels.     

Temporal sparse free-form deformations

FFD represent a widely used model for the non-rigid registration of medical images. The balance between robustness to noise and accuracy in modelling localised motion is typically controlled by the control point grid spacing and the amount of regularisation. More recently, TFFD have been proposed which extend the FFD approach in order to recover smooth motion from temporal image sequences. In this paper, we revisit the classic FFD approach and propose a sparse representation using the principles of compressed sensing. The sparse representation can model both global and local motion accurately and robustly. We view the registration as a deformation reconstruction problem. The deformation is reconstructed from a pair of images (or image sequences) with a sparsity constraint applied to the parametric space. Specifically, we introduce sparsity into the deformation via L₁ regularisation, and apply a bending energy regularisation between neighbouring control points within each level to encourage a grouped sparse solution. We further extend the sparsity constraint to the temporal domain and propose a TSFFD which can capture fine local details such as motion discontinuities in both space and time without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate deformations in dynamic 2D and 3D image sequences. Compared to the classic FFD and TFFD approach, a significant increase in registration accuracy can be observed in natural images as well as in cardiac images.     

Efficient seamline determination for UAV image mosaicking using edge detection

Image mosaicking of data from individual high-resolution unmanned aerial vehicle (UAV) images is required to obtain sufficient coverage over extensive roads. During the mosaicking process, seamlines may be generated due to differences in illumination or projection between individual images, or the presence of moving objects. This study presents an efficient seamline determination technique based on edge detection for UAV road surface images. The algorithm can be divided into three main steps. First, we detect the edges in the overlapping intensity image within the road boundary. Next, we obtain an automatic seamline passing through regions of non-attraction in areas of overlap. Finally, we adjust the values of the overlapping region using the values of the corresponding individual images by following the coordinates of the seamline detected in the second step, ultimately creating an image mosaic. The experiment using UAV images of a road surface demonstrates that the proposed method produces a satisfactory result. The proposed method can be applied for quick mosaicking of UAV images intended for maintaining road safety.     

Data assimilation using a gradient descent method for estimation of intraoperative brain deformation

Biomechanical models that simulate brain deformation are gaining attention as alternatives for brain shift compensation. One approach, known as the “forced-displacement method”, constrains the model to exactly match the measured data through boundary condition (BC) assignment. Although it improves model estimates and is computationally attractive, the method generates fictitious forces and may be ill-advised due to measurement uncertainty. Previously, we have shown that by assimilating intraoperatively acquired brain displacements in an inversion scheme, the Representer algorithm (REP) is able to maintain stress-free BCs and improve model estimates by 33% over those without data guidance in a controlled environment. However, REP is computationally efficient only when a few data points are used for model guidance because its costs scale linearly in the number of data points assimilated, thereby limiting its utility (and accuracy) in clinical settings. In this paper, we present a steepest gradient descent algorithm (SGD) whose computational complexity scales nearly invariantly with the number of measurements assimilated by iteratively adjusting the forcing conditions to minimize the difference between measured and model-estimated displacements (model-data misfit). Solutions of full linear systems of equations are achieved with a parallelized direct solver on a shared-memory, eight-processor Linux cluster. We summarize the error contributions from the entire process of model-updated image registration compensation and we show that SGD is able to attain model estimates comparable to or better than those obtained with REP, capturing about 74–82% of tumor displacement, but with a computational effort that is significantly less (a factor of 4-fold or more reduction relative to REP) and nearly invariant to the amount of sparse data involved when the number of points assimilated is large. Based on five patient cases, an average computational cost of approximately 2 min for estimating whole-brain deformation has been achieved with SGD using 100 sparse data points, suggesting the new algorithm is sufficiently fast with adequate accuracy for routine use in the operating room (OR).     

Comparison of "silent" clustered and sparse temporal fMRI acquisitions in tonal and speech perception tasks

In the functional imaging of auditory cortical functions, long silent periods between the data acquisitions prevent interferences between scanner noise and the auditory stimulus processing. Recent fMRI studies have shown that sparse temporal acquisition designs are advantageous over continuous scanning protocols on physiological, perceptual, and cognitive levels. Sparse temporal acquisition schemes (STA) which use a single volume acquisition after each trial imply the advantage of auditory stimulation devoid of ambient scanner noise but have the drawback of a reduced statistical power. To alleviate this effect, STA schemes have been extended to clustered-sparse temporal acquisition (CTA) designs which record several subsequent BOLD contrast images in rapid succession. In the present study, we collected data from 13 healthy volunteers performing a speech and a tonal discrimination task using both a CTA and STA scheme to carry out a systematic evaluation of these acquisition protocols. By statistical modeling of the fMRI data sets, we revealed stronger effect sizes for the STA protocol regardless of the task, reflecting the better signal-to-noise-ratio of MR images acquired with this scheme. In contrast, we demonstrate higher statistical power for the use of a CTA protocol. Accordingly, in the context of standard fMRI analysis, the CTA protocol clearly outperformed the STA scheme at the level of single-subject analysis and fixed-effects group analysis. Our results clearly suggest that it is advantageous to acquire several sample points per trial if one wants to use the benefit of "silent" fMRI. Furthermore, our data demonstrate the feasibility of the clustered acquisition of subsequent imaging volumes along the T1-decay.     

Detection and segmentation of drusen deposits on human retina: potential in the diagnosis of age-related macular degeneration

Assessment of the risk for the development of age-related macular degeneration requires reliable detection and quantitative mapping of retinal abnormalities that are considered as precursors of the disease. Typical signs for the latter are the so-called drusen that appear as abnormal white-yellow deposits on the retina. Segmentation of these features using conventional image analysis methods is quite complicated mainly due to the non-uniform illumination and the variability of the pigmentation of the background tissue. This paper presents a novel segmentation algorithm for the automatic detection and mapping of drusen in retina images acquired with the aid of a digital Fundus camera. We employ a modified adaptive histogram equalization, namely the multilevel histogram equalization (MLE) scheme, for enhancing local intensity structures. For the detection of drusen in retina images, we develop a novel segmentation technique, the histogram-based adaptive local thresholding (HALT), which extracts the useful information from an image without being affected by the presence of other structures. We provide experimental results from the application of our technique to real images, where certain abnormalities (drusen) have slightly different characteristics from the background. The performance of the algorithm is established through statistical analysis of the results. This analysis indicates that the proposed drusen detector gives reliable detection accuracy in both position and mass size.     

A new variational shape-from-orientation approach to correcting intensity inhomogeneities in magnetic resonance images

A new intensity inhomogeneity correction algorithm based on a variational shape-from-orientation formulation is presented. Unlike most previous methods, the proposed algorithm is fully automatic, widely applicable and very efficient. Since no prior classification knowledge about the image is assumed in the proposed algorithm, it can be applied to correct intensity inhomogeneities for a wide variety of medical images. In this paper, a finite-element method is used to model the smooth bias-field function. Orientation constraints for the bias-field function are computed at the nodal locations of the regular discretization grid away from the boundary between different class regions. The selection of reliable orientation constraints is facilitated by the goodness of fit of a first-order polynomial model to the neighborhood of each nodal location. The automatically selected orientation constraints are integrated in a regularization framework, which leads to minimization of a convex and quadratic energy function. This energy minimization is accomplished by solving a linear system with a large, sparse, symmetric and positive semi-definite stiffness matrix. We employ an adaptive preconditioned conjugate-gradient algorithm to solve the linear system very efficiently. Experimental results on a variety of magnetic resonance images are given to demonstrate the effectiveness and efficiency of the proposed algorithm.     

星形算法对心尖四腔切面左心室内膜全自动勾画的初步研究

目的初步测试一种叫做星形算法的全自动超声边界探测方法的实用价值.方法星形算法采用了从左心室腔内向四周作放射形探测的心内膜探测方法,并且运用了以平均相邻距离和局部平均边界灰度为构件的代价函数来勾画边界.结果本文用八组带有各种质量问题的舒张末期和收缩末期心尖四腔切面图像对星形算法进行测试,与手工勾画的心内膜相比,星形算法勾画边界的平均半径误差率为12.90%,所围平均面积误差率为10.93%.结论星形算法能够在临床条件下实现心内膜的全自动勾画.     

Registration of freehand 3D ultrasound and magnetic resonance liver images

We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a "bronze standard" registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the "bronze standard" and this is improved to 3.6 mm after running the intensity-based algorithm.     

Automatic X-ray landmark detection and shape segmentation via data-driven joint estimation of image displacements

In this paper, we propose a new method for fully-automatic landmark detection and shape segmentation in X-ray images. To detect landmarks, we estimate the displacements from some randomly sampled image patches to the (unknown) landmark positions, and then we integrate these predictions via a voting scheme. Our key contribution is a new algorithm for estimating these displacements. Different from other methods where each image patch independently predicts its displacement, we jointly estimate the displacements from all patches together in a data driven way, by considering not only the training data but also geometric constraints on the test image. The displacements estimation is formulated as a convex optimization problem that can be solved efficiently. Finally, we use the sparse shape composition model as the a priori information to regularize the landmark positions and thus generate the segmented shape contour. We validate our method on X-ray image datasets of three different anatomical structures: complete femur, proximal femur and pelvis. Experiments show that our method is accurate and robust in landmark detection, and, combined with the shape model, gives a better or comparable performance in shape segmentation compared to state-of-the art methods. Finally, a preliminary study using CT data shows the extensibility of our method to 3D data.     

Evaluation of image features and search strategies for segmentation of bone structures in radiographs using Active Shape Models

In this paper, we evaluate various image features and different search strategies for fitting Active Shape Models (ASM) to bone object boundaries in digitized radiographs. The original ASM method iteratively refines the pose and shape parameters of the point distribution model driving the ASM by a least squares fit of the shape to update the target points at the estimated object boundary position, as determined by a suitable object boundary criterion. We propose an improved search procedure that is more robust against outlier configurations in the boundary target points by requiring subsequent shape changes to be smooth, which is imposed by a smoothness constraint on the displacement of neighbouring target points at each iteration and implemented by a minimal cost path approach. We compare the original ASM search method and our improved search algorithm with a third method that does not rely on iteratively refined target point positions, but instead optimizes a global Bayesian objective function derived from statistical a priori contour shape and image models. Extensive validation of these methods on a database containing more than 400 images of the femur, humerus and calcaneus using the manual expert segmentation as ground truth shows that our minimal cost path method is the most robust. We also evaluate various measures for capturing local image appearance around each boundary point and conclude that the Mahalanobis distance applied to normalized image intensity profiles extracted normal to the shape is the most suitable criterion among the tested ones for guiding the ASM optimization.