Automated segmentation of retinal blood vessels in spectral domain optical coherence tomography scans

The correct segmentation of blood vessels in optical coherence tomography (OCT) images may be an important requirement for the analysis of intra-retinal layer thickness in human retinal diseases. We developed a shape model based procedure for the automatic segmentation of retinal blood vessels in spectral domain (SD)-OCT scans acquired with the Spectralis OCT system. The segmentation procedure is based on a statistical shape model that has been created through manual segmentation of vessels in a training phase. The actual segmentation procedure is performed after the approximate vessel position has been defined by a shadowgraph that assigns the lateral vessel positions. The active shape model method is subsequently used to segment blood vessel contours in axial direction. The automated segmentation results were validated against the manual segmentation of the same vessels by three expert readers. Manual and automated segmentations of 168 blood vessels from 34 B-scans were analyzed with respect to the deviations in the mean Euclidean distance and surface area. The mean Euclidean distance between the automatically and manually segmented contours (on average 4.0 pixels respectively 20 μm against all three experts) was within the range of the manually marked contours among the three readers (approximately 3.8 pixels respectively 18 μm for all experts). The area deviations between the automated and manual segmentation also lie within the range of the area deviations among the 3 clinical experts. Intra reader variability for the experts was between 0.9 and 0.94. We conclude that the automated segmentation approach is able to segment blood vessels with comparable accuracy as expert readers and will provide a useful tool in vessel analysis of whole C-scans, and in particular in multicenter trials.     

Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier

Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. In this work, we propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN).In the proposed method, a 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN. Tracking is terminated when no direction can be identified with high certainty. The CNN is trained using manually annotated centerlines in training images. No image preprocessing is required, so that the process is guided solely by the local image values around the tracker’s location.The CNN was trained using a training set consisting of 8 CCTA images with a total of 32 manually annotated centerlines provided in the MICCAI 2008 Coronary Artery Tracking Challenge (CAT08). Evaluation was performed within the CAT08 challenge using a test set consisting of 24 CCTA test images in which 96 centerlines were extracted. The extracted centerlines had an average overlap of 93.7% with manually annotated reference centerlines. Extracted centerline points were highly accurate, with an average distance of 0.21 mm to reference centerline points. Based on these results the method ranks third among 25 publicly evaluated methods in CAT08. In a second test set consisting of 50 CCTA scans acquired at our institution (UMCU), an expert placed 5448 markers in the coronary arteries, along with radius measurements. Each marker was used as a seed point to extract a single centerline, which was compared to the other markers placed by the expert. This showed strong correspondence between extracted centerlines and manually placed markers. In a third test set containing 36 CCTA scans from the MICCAI 2014 Challenge on Automatic Coronary Calcium Scoring (orCaScore), fully automatic seeding and centerline extraction was evaluated using a segment-wise analysis. This showed that the algorithm is able to fully-automatically extract on average 92% of clinically relevant coronary artery segments. Finally, the limits of agreement between reference and automatic artery radius measurements were found to be below the size of one voxel in both the CAT08 dataset and the UMCU dataset. Extraction of a centerline based on a single seed point required on average 0.4 ± 0.1 s and fully automatic coronary tree extraction required around 20 s.The proposed method is able to accurately and efficiently determine the direction and radius of coronary arteries based on information derived directly from the image data. The method can be trained with limited training data, and once trained allows fast automatic or interactive extraction of coronary artery trees from CCTA images.     

Characterization of changes in blood vessel width and tortuosity in retinopathy of prematurity using image analysis

Many retinal diseases are characterised by changes to retinal vessels. For example, a common condition associated with retinopathy of prematurity (ROP) is so-called plus disease, characterised by increased vascular dilation and tortuosity. This paper presents a general technique for segmenting out vascular structures in retinal images, and characterising the segmented blood vessels. The segmentation technique consists of several steps. Morphological preprocessing is used to emphasise linear structures such as vessels. A second derivative operator is used to further emphasise thin vascular structures, and is followed by a final morphological filtering stage. Thresholding of this image is used to provide a segmented vascular mask. Skeletonisation of this mask allows identification of points in the image where vessels cross (bifurcations and crossing points) and allows the width and tortuosity of vessel segments to be calculated. The accuracy of the segmentation stage is quite dependent on the parameters used, particularly at the thresholding stage. However, reliable measurements of vessel width and tortuosity were shown using test images. Using these tools, a set of images drawn from 23 subjects being screened for the presence of threshold ROP disease is considered. Of these subjects, 11 subsequently required treatment for ROP, 9 had no evidence of ROP, and 3 had spontaneously regressed ROP. The average vessel width and tortuosity for the treated subjects was 96.8 microm and 1.125. The corresponding figures for the non-treated cohort were 86.4 microm and 1.097. These differences were statistically significant at the 99% and 95% significance level, respectively. Subjects who progressed to threshold disease during the course of screening showed an average increase in vessel width of 9.6 microm and in tortuosity of +0.008. Only the change in width was statistically significant. Applying a simple retrospective screening paradigm based solely on vessel width and tortuosity yields a screening test with a sensitivity and specificity of 82% and 75%. Factors confounding a more accurate test include poor image quality, inaccuracies in vessel segmentation, inaccuracies in measurement of vessel width and tortuosity, and limitations inherent in screening based solely on examination of the posterior pole.     

Probabilistic vessel axis tracing and its application to vessel segmentation with stream surfaces and minimum cost paths

We propose a novel framework to segment vessels on their cross-sections. It starts with a probabilistic vessel axis tracing in a gray-scale three-dimensional angiogram, followed by vessel boundary delineation on cross-sections derived from the extracted axis. It promotes a more intuitive delineation of vessel boundaries which are mostly round on the cross-sections. The prior probability density function of the axis tracer's formulation permits seamless integration of user guidance to produce continuous traces through regions that contain furcations, diseased portions, kissing vessels (vessels in close proximity to each other) and thin vessels. The contour that outlines the vessel boundary in a 3-D space is determined as the minimum cost path on a weighted directed acyclic graph derived from each cross-section. The user can place anchor points to force the contour to pass through. The contours obtained are tiled to approximate the vessel boundary surface. Since we use stream surfaces generated w.r.t. the traced axis as cross-sections, non-intersecting adjacent cross-sections are guaranteed. Therefore, the tiling can be achieved by joining vertices of adjacent contours. The vessel boundary surface is then deformed under constrained movements on the cross-sections and is voxelized to produce the final vascular segmentation. Experimental results on synthetic and clinical data have shown that the vessel axes extracted by our tracer are continuous and less jittered as compared with the other two trace-based algorithms. Furthermore, the segmentation algorithm with cross-sections are robust to noise and can delineate vessel boundaries that have level of variability similar to those obtained manually.     

Dual-consistency semi-supervision combined with self-supervision for vessel segmentation in retinal OCTA images

Optical coherence tomography angiography(OCTA) is an advanced noninvasive vascular imaging technique that has important implications in many vision-related diseases. The automatic segmentation of retinal vessels in OCTA is understudied, and the existing segmentation methods require large-scale pixel-level annotated images. However, manually annotating labels is time-consuming and labor-intensive. Therefore, we propose a dual-consistency semi-supervised segmentation network incorporating multi-scale self-supervised puzzle subtasks(DCSS-Net) to tackle the challenge of limited annotations. First, we adopt a novel self-supervised task in assisting semi-supervised networks in training to learn better feature representations. Second, we propose a dual-consistency regularization strategy that imposed data-based and feature-based perturbation to effectively utilize a large number of unlabeled data, alleviate the overfitting of the model, and generate more accurate segmentation predictions. Experimental results on two OCTA retina datasets validate the effectiveness of our DCSS-Net. With very little labeled data, the performance of our method is comparable with fully supervised methods trained on the entire labeled dataset.     

Segmentation of blood vessels from red-free and fluorescein retinal images

The morphology of the retinal blood vessels can be an important indicator for diseases like diabetes, hypertension and retinopathy of prematurity (ROP). Thus, the measurement of changes in morphology of arterioles and venules can be of diagnostic value. Here we present a method to automatically segment retinal blood vessels based upon multiscale feature extraction. This method overcomes the problem of variations in contrast inherent in these images by using the first and second spatial derivatives of the intensity image that gives information about vessel topology. This approach also enables the detection of blood vessels of different widths, lengths and orientations. The local maxima over scales of the magnitude of the gradient and the maximum principal curvature of the Hessian tensor are used in a multiple pass region growing procedure. The growth progressively segments the blood vessels using feature information together with spatial information. The algorithm is tested on red-free and fluorescein retinal images, taken from two local and two public databases. Comparison with first public database yields values of 75.05% true positive rate (TPR) and 4.38% false positive rate (FPR). Second database values are of 72.46% TPR and 3.45% FPR. Our results on both public databases were comparable in performance with other authors. However, we conclude that these values are not sensitive enough so as to evaluate the performance of vessel geometry detection. Therefore we propose a new approach that uses measurements of vessel diameters and branching angles as a validation criterion to compare our segmented images with those hand segmented from public databases. Comparisons made between both hand segmented images from public databases showed a large inter-subject variability on geometric values. A last evaluation was made comparing vessel geometric values obtained from our segmented images between red-free and fluorescein paired images with the latter as the "ground truth". Our results demonstrated that borders found by our method are less biased and follow more consistently the border of the vessel and therefore they yield more confident geometric values.     

Multi-perspective label based deep learning framework for cerebral vasculature segmentation in whole-brain fluorescence images

The popularity of fluorescent labelling and mesoscopic optical imaging techniques enable the acquisition of whole mammalian brain vasculature images at capillary resolution. Segmentation of the cerebrovascular network is essential for analyzing the cerebrovascular structure and revealing the pathogenesis of brain diseases. Existing deep learning methods use a single type of annotated labels with the same pixel weight to train the neural network and segment vessels. Due to the variation in the shape, density and brightness of vessels in whole-brain fluorescence images, it is difficult for a neural network trained with a single type of label to segment all vessels accurately. To address this problem, we proposed a deep learning cerebral vasculature segmentation framework based on multi-perspective labels. First, the pixels in the central region of thick vessels and the skeleton region of vessels were extracted separately using morphological operations based on the binary annotated labels to generate two different labels. Then, we designed a three-stage 3D convolutional neural network containing three sub-networks, namely thick-vessel enhancement network, vessel skeleton enhancement network and multi-channel fusion segmentation network. The first two sub-networks were trained by the two labels generated in the previous step, respectively, and pre-segmented the vessels. The third sub-network was responsible for fusing the pre-segmented results to precisely segment the vessels. We validated our method on two mouse cerebral vascular datasets generated by different fluorescence imaging modalities. The results showed that our method outperforms the state-of-the-art methods, and the proposed method can be applied to segment the vasculature on large-scale volumes.     

Delineation of blood vessels in pediatric retinal images using decision trees-based ensemble classification

Purpose

Automatic segmentation of the retinal vasculature is a first step in computer-assisted diagnosis and treatment planning. The extraction of retinal vessels in pediatric retinal images is challenging because of comparatively wide arterioles with a light streak running longitudinally along the vessel’s center, the central vessel reflex. A new method for automatic segmentation was developed and tested.

Method

   A supervised method for retinal vessel segmentation in the images of multi-ethnic school children was developed based on ensemble classifier of bootstrapped decision trees. A collection of dual Gaussian, second derivative of Gaussian and Gabor filters, along with the generalized multiscale line strength measure and morphological transformation is used to generate the feature vector. The feature vector encodes information to handle the normal vessels as well as the vessels with the central reflex. The methodology is evaluated on CHASE_DB1, a relatively new public retinal image database of multi-ethnic school children, which is a subset of retinal images from the Child Heart and Health Study in England (CHASE) dataset.

Results

   The segmented retinal images from the CHASE_DB1 database produced best case accuracy, sensitivity and specificity of 0.96, 0.74 and 0.98, respectively, and worst case measures of 0.94, 0.67 and 0.98, respectively.

Conclusion

   A new retinal blood vessel segmentation algorithm was developed and tested with a shared database. The observed accuracy, speed, robustness and simplicity suggest that the algorithm may be a suitable tool for automated retinal image analysis in large population-based studies.     

MUTE: A multilevel-stimulated denoising strategy for single cataractous retinal image dehazing

In this research, we studied the duality between cataractous retinal image dehazing and image denoising and proposed that the dehazing task for cataractous retinal images can be achieved with the combination of image denoising and sigmoid function. To do so, we introduce the double-pass fundus reflection model in the YPbPr color space and developed a multilevel stimulated denoising strategy termed MUTE. The transmission matrix of the cataract layer is expressed as the superposition of denoised raw images of different levels weighted by pixel-wise sigmoid functions. We further designed an intensity-based cost function that can guide the updating of the model parameters. They are updated by gradient descent with adaptive momentum estimation, which gives us the final refined transmission matrix of the cataract layer. We tested our methods on cataract retinal images from both public and proprietary databases, and we compared the performance of our method with other state-of-the-art enhancement methods. Both visual assessments and objective assessments show the superiority of the proposed method. We further demonstrated three potential applications including blood vessel segmentation, retinal image registrations, and diagnosing with enhanced images that may largely benefit from our proposed methods.     

Blood vessel segmentation and width estimation in ultra-wide field scanning laser ophthalmoscopy

Features of the retinal vasculature, such as vessel widths, are considered biomarkers for systemic disease. The aim of this work is to present a supervised approach to vessel segmentation in ultra-wide field of view scanning laser ophthalmoscope (UWFoV SLO) images and to evaluate its performance in terms of segmentation and vessel width estimation accuracy. The results of the proposed method are compared with ground truth measurements from human observers and with existing state-of-the-art techniques developed for fundus camera images that we optimized for UWFoV SLO images. Our algorithm is based on multi-scale matched filters, a neural network classifier and hysteresis thresholding. After spline-based refinement of the detected vessel contours, the vessel widths are estimated from the binary maps. Such analysis is performed on SLO images for the first time. The proposed method achieves the best results, both in vessel segmentation and in width estimation, in comparison to other automatic techniques.OCIS codes: (100.2960) Image analysis, (100.3008) Image recognition, algorithms and filters, (100.4996) Pattern recognition, neural networks, (170.4470) Ophthalmology     

Exploratory Dijkstra forest based automatic vessel segmentation: applications in video indirect ophthalmoscopy (VIO)

We present a methodology for extracting the vascular network in the human retina using Dijkstra's shortest-path algorithm. Our method preserves vessel thickness, requires no manual intervention, and follows vessel branching naturally and efficiently. To test our method, we constructed a retinal video indirect ophthalmoscopy (VIO) image database from pediatric patients and compared the segmentations achieved by our method and state-of-the-art approaches to a human-drawn gold standard. Our experimental results show that our algorithm outperforms prior state-of-the-art methods, for both single VIO frames and automatically generated, large field-of-view enhanced mosaics. We have made the corresponding dataset and source code freely available online.     

Chest wall segmentation in automated 3D breast ultrasound scans

In this paper, we present an automatic method to segment the chest wall in automated 3D breast ultrasound images. Determining the location of the chest wall in automated 3D breast ultrasound images is necessary in computer-aided detection systems to remove automatically detected cancer candidates beyond the chest wall and it can be of great help for inter- and intra-modal image registration. We show that the visible part of the chest wall in an automated 3D breast ultrasound image can be accurately modeled by a cylinder. We fit the surface of our cylinder model to a set of automatically detected rib-surface points. The detection of the rib-surface points is done by a classifier using features representing local image intensity patterns and presence of rib shadows. Due to attenuation of the ultrasound signal, a clear shadow is visible behind the ribs. Evaluation of our segmentation method is done by computing the distance of manually annotated rib points to the surface of the automatically detected chest wall. We examined the performance on images obtained with the two most common 3D breast ultrasound devices in the market. In a dataset of 142 images, the average mean distance of the annotated points to the segmented chest wall was 5.59 ± 3.08 mm.     

Learning layer-specific edges for segmenting retinal layers with large deformations

We present an algorithm for layer-specific edge detection in retinal optical coherence tomography images through a structured learning algorithm to reinforce traditional graph-based retinal layer segmentation. The proposed algorithm simultaneously identifies individual layers and their corresponding edges, resulting in the computation of layer-specific edges in 1 second. These edges augment classical dynamic programming based segmentation under layer deformation, shadow artifacts noise, and without heuristics or prior knowledge. We considered Duke’s online data set containing 110 B-scans of 10 diabetic macular edema subjects with 8 retinal layers annotated by two experts for experimentation, and achieved a mean distance error of 1.38 pixels whereas that of the state-of-the-art was 1.68 pixels.OCIS codes: (170.4500) Optical coherence tomography, (100.6950) Tomographic image processing, (170.6935) Tissue characterization, (170.1610) Clinical applications     

Vessel-CAPTCHA: An efficient learning framework for vessel annotation and segmentation

Deep learning techniques for 3D brain vessel image segmentation have not been as successful as in the segmentation of other organs and tissues. This can be explained by two factors. First, deep learning techniques tend to show poor performances at the segmentation of relatively small objects compared to the size of the full image. Second, due to the complexity of vascular trees and the small size of vessels, it is challenging to obtain the amount of annotated training data typically needed by deep learning methods. To address these problems, we propose a novel annotation-efficient deep learning vessel segmentation framework. The framework avoids pixel-wise annotations, only requiring weak patch-level labels to discriminate between vessel and non-vessel 2D patches in the training set, in a setup similar to the CAPTCHAs used to differentiate humans from bots in web applications. The user-provided weak annotations are used for two tasks: (1) to synthesize pixel-wise pseudo-labels for vessels and background in each patch, which are used to train a segmentation network, and (2) to train a classifier network. The classifier network allows to generate additional weak patch labels, further reducing the annotation burden, and it acts as a second opinion for poor quality images. We use this framework for the segmentation of the cerebrovascular tree in Time-of-Flight angiography (TOF) and Susceptibility-Weighted Images (SWI). The results show that the framework achieves state-of-the-art accuracy, while reducing the annotation time by $\sim$77% w.r.t. learning-based segmentation methods using pixel-wise labels for training.     

Mass preserving image registration for lung CT

This paper presents a mass preserving image registration algorithm for lung CT images. To account for the local change in lung tissue intensity during the breathing cycle, a tissue appearance model based on the principle of preservation of total lung mass is proposed. This model is incorporated into a standard image registration framework with a composition of a global affine and several free-form B-Spline transformations with increasing grid resolution. The proposed mass preserving registration method is compared to registration using the sum of squared intensity differences as a similarity function on four groups of data: 44 pairs of longitudinal inspiratory chest CT scans with small difference in lung volume; 44 pairs of longitudinal inspiratory chest CT scans with large difference in lung volume; 16 pairs of expiratory and inspiratory CT scans; and 5 pairs of images extracted at end exhale and end inhale phases of 4D-CT images. Registration errors, measured as the average distance between vessel tree centerlines in the matched images, are significantly lower for the proposed mass preserving image registration method in the second, third and fourth group, while there is no statistically significant difference between the two methods in the first group. Target registration error, assessed via a set of manually annotated landmarks in the last group, was significantly smaller for the proposed registration method.     

基于通道注意力双路径架构网络分割视网膜血管

目的 评价基于通道注意力的双路径架构网络（DPCA-Net）算法分割视网膜血管的效果。方法 基于DRIVE及CHASE_DB1公开数据集,通过在网络中引入通道注意力机制,融合于主路径与次路径网络中提取的特征,构建端-端DPCA-Net视网膜血管分割深度学习体系结构,并构建未引入通道注意力机制的双路径架构网络（DP-Net）算法,评价其分割视网膜血管的效果。结果 DPCA-Net算法可正确识别中央血管反射区中的血管;亮斑区大部分血管被正确识别,背景干扰区中仅小部分背景被认为是血管,黑斑区部分形状类似血管的黑斑被认为是血管。DPCA-Net算法分割DRIVE/CHASE_DB1数据集中视网膜血管的准确率为95.58%/96.34%,敏感度为80.37%/77.70%,特异度为97.80%/98.22%,F1值为82.24%/79.55%;除基于DRIVE数据集的敏感度之外均高于DP-Net算法。结论 相比DP-Net算法,DPCA-Net算法能学习更多血管分割特征,且对病变区域不敏感,分割视网膜血管效果较好。     

Retinal Nerve Fiber Layer Segmentation on FD-OCT Scans of Normal Subjects and Glaucoma Patients

Automated measurements of the retinal nerve fiber layer thickness on circular OCT B-Scans provide physicians additional parameters for glaucoma diagnosis. We propose a novel retinal nerve fiber layer segmentation algorithm for frequency domain data that can be applied on scans from both normal healthy subjects, as well as glaucoma patients, using the same set of parameters. In addition, the algorithm remains almost unaffected by image quality. The main part of the segmentation process is based on the minimization of an energy function consisting of gradient and local smoothing terms. A quantitative evaluation comparing the automated segmentation results to manually corrected segmentations from three reviewers is performed. A total of 72 scans from glaucoma patients and 132 scans from normal subjects, all from different persons, composed the database for the evaluation of the segmentation algorithm. A mean absolute error per A-Scan of 2.9 μm was achieved on glaucomatous eyes, and 3.6 μm on healthy eyes. The mean absolute segmentation error over all A-Scans lies below 10 μm on 95.1% of the images. Thus our approach provides a reliable tool for extracting diagnostic relevant parameters from OCT B-Scans for glaucoma diagnosis.     

Tensor-cut: A tensor-based graph-cut blood vessel segmentation method and its application to renal artery segmentation

Blood vessel segmentation plays a fundamental role in many computer-aided diagnosis (CAD) systems, such as coronary artery stenosis quantification, cerebral aneurysm quantification, and retinal vascular tree analysis. Fine blood vessel segmentation can help build a more accurate computer-aided diagnosis system and help physicians gain a better understanding of vascular structures. The purpose of this article is to develop a blood vessel segmentation method that can improve segmentation accuracy in tiny blood vessels. In this work, we propose a tensor-based graph-cut method for blood vessel segmentation. With our method, each voxel can be modeled by a second-order tensor, allowing the capture of the intensity information and the geometric information for building a more accurate model for blood vessel segmentation. We compared our proposed method’s accuracy to several state-of-the-art blood vessel segmentation algorithms and performed experiments on both simulated and clinical CT datasets. Both experiments showed that our method achieved better state-of-the-art results than the competing techniques. The mean centerline overlap ratio of our proposed method is 84% on clinical CT data. Our proposed blood vessel segmentation method outperformed other state-of-the-art methods by 10% on clinical CT data. Tiny blood vessels in clinical CT data with a 1-mm radius can be extracted using the proposed technique. The experiments on a clinical dataset showed that the proposed method significantly improved the segmentation accuracy in tiny blood vessels.     

基于MRI的动物肝脏脉管分割与三维重建

目的探索基于磁共振成像的肝脏脉管(血管、胆管)三维重建的方法,以显示肝脏中血管、胆管走行立体图像,从而指导劈离式肝脏移植手术.方法综合利用二维图像的多种平滑和分割方法,得到肝实质和肝脉管二维图像序列,并在此基础上三维重建肝实质和肝脉管.结果由MR图像序列可三维重建清晰的肝实质和肝血管立体图像,并且可得到肝叶、段间血管及胆管的走行及相互关系.结论利用MRI和图像分割以及图像三维重建技术指导劈离式肝移植手术,具有重要的临床意义.     

Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema

We present a fully automatic algorithm to identify fluid-filled regions and seven retinal layers on spectral domain optical coherence tomography images of eyes with diabetic macular edema (DME). To achieve this, we developed a kernel regression (KR)-based classification method to estimate fluid and retinal layer positions. We then used these classification estimates as a guide to more accurately segment the retinal layer boundaries using our previously described graph theory and dynamic programming (GTDP) framework. We validated our algorithm on 110 B-scans from ten patients with severe DME pathology, showing an overall mean Dice coefficient of 0.78 when comparing our KR + GTDP algorithm to an expert grader. This is comparable to the inter-observer Dice coefficient of 0.79. The entire data set is available online, including our automatic and manual segmentation results. To the best of our knowledge, this is the first validated, fully-automated, seven-layer and fluid segmentation method which has been applied to real-world images containing severe DME.OCIS codes: (100.0100) Image processing, (170.4500) Optical coherence tomography, (170.4470) Ophthalmology