Flux driven automatic centerline extraction

We present a fast, robust and automatic method for computing centerline paths through tubular structures for application to virtual endoscopy. The key idea is to utilize a skeletonization algorithm which exploits properties of the average outward flux of the gradient vector field of a Euclidean distance function from the boundary of the structure. The algorithm is modified to yield a collection of 3D curves, each of which is locally centered. The approach requires no user interaction, is virtually parameter free and has low computational complexity. We validate the method quantitatively on a number of synthetic data sets with known centerlines and qualitatively on colon and vessel data segmented from CT and CRA images.     

A system for determination of 3D vessel tree centerlines from biplane images

With the increasing number and complexity of therapeutic coronary interventions, there is an increasing need for accurate quantitative measurements. These interventions and measurements may be facilitated by accurate and reproducible magnifications and orientations of the vessel structures, specifically by accurate 3D vascular tree centerlines. A number of methods have been proposed to calculate 3D vascular tree centerlines from biplane images. In general, the calculated magnifications and orientations are accurate to within approximately 1–3% and 2–5°, respectively. Here, we present a complete system for determination of the 3D vessel centerlines from biplane angiograms without the use of a calibration object. Subsequent to indication of the vessel centerlines, the imaging geometry and 3D centerlines are calculated automatically and within approximately 2 min. The system was evaluated in terms of the intra- and inter-user variations of the various calculated quantities. The reproducibilities obtained with this system are comparable to or better than the accuracies and reproducibilities quoted for other proposed methods. Based on these results and those reported in earlier studies, we believe that this system will provide accurate and reproducible vascular tree centerlines from biplane images while the patient is still on the table, and thereby will facilitate interventions and associated quantitative analyses of the vasculature.     

Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier

Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. In this work, we propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN).In the proposed method, a 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN. Tracking is terminated when no direction can be identified with high certainty. The CNN is trained using manually annotated centerlines in training images. No image preprocessing is required, so that the process is guided solely by the local image values around the tracker’s location.The CNN was trained using a training set consisting of 8 CCTA images with a total of 32 manually annotated centerlines provided in the MICCAI 2008 Coronary Artery Tracking Challenge (CAT08). Evaluation was performed within the CAT08 challenge using a test set consisting of 24 CCTA test images in which 96 centerlines were extracted. The extracted centerlines had an average overlap of 93.7% with manually annotated reference centerlines. Extracted centerline points were highly accurate, with an average distance of 0.21 mm to reference centerline points. Based on these results the method ranks third among 25 publicly evaluated methods in CAT08. In a second test set consisting of 50 CCTA scans acquired at our institution (UMCU), an expert placed 5448 markers in the coronary arteries, along with radius measurements. Each marker was used as a seed point to extract a single centerline, which was compared to the other markers placed by the expert. This showed strong correspondence between extracted centerlines and manually placed markers. In a third test set containing 36 CCTA scans from the MICCAI 2014 Challenge on Automatic Coronary Calcium Scoring (orCaScore), fully automatic seeding and centerline extraction was evaluated using a segment-wise analysis. This showed that the algorithm is able to fully-automatically extract on average 92% of clinically relevant coronary artery segments. Finally, the limits of agreement between reference and automatic artery radius measurements were found to be below the size of one voxel in both the CAT08 dataset and the UMCU dataset. Extraction of a centerline based on a single seed point required on average 0.4 ± 0.1 s and fully automatic coronary tree extraction required around 20 s.The proposed method is able to accurately and efficiently determine the direction and radius of coronary arteries based on information derived directly from the image data. The method can be trained with limited training data, and once trained allows fast automatic or interactive extraction of coronary artery trees from CCTA images.     

A linear programming approach to reconstructing subcellular structures from confocal images for automated generation of representative 3D cellular models

This paper presents a novel computer vision algorithm to analyze 3D stacks of confocal images of fluorescently stained single cells. The goal of the algorithm is to create representative in silico model structures that can be imported into finite element analysis software for mechanical characterization. Segmentation of cell and nucleus boundaries is accomplished via standard thresholding methods. Using novel linear programming methods, a representative actin stress fiber network is generated by computing a linear superposition of fibers having minimum discrepancy compared with an experimental 3D confocal image. Qualitative validation is performed through analysis of seven 3D confocal image stacks of adherent vascular smooth muscle cells (VSMCs) grown in 2D culture. The presented method is able to automatically generate 3D geometries of the cell’s boundary, nucleus, and representative F-actin network based on standard cell microscopy data. These geometries can be used for direct importation and implementation in structural finite element models for analysis of the mechanics of a single cell to potentially speed discoveries in the fields of regenerative medicine, mechanobiology, and drug discovery.     

Deep probabilistic tracking of particles in fluorescence microscopy images

Tracking of particles in temporal fluorescence microscopy image sequences is of fundamental importance to quantify dynamic processes of intracellular structures as well as virus structures. We introduce a probabilistic deep learning approach for fluorescent particle tracking, which is based on a recurrent neural network that mimics classical Bayesian filtering. Compared to previous deep learning methods for particle tracking, our approach takes into account uncertainty, both aleatoric and epistemic uncertainty. Thus, information about the reliability of the computed trajectories is determined. Manual tuning of tracking parameters is not necessary and prior knowledge about the noise statistics is not required. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. For correspondence finding, we introduce a neural network which computes assignment probabilities jointly across multiple detections as well as determines the probabilities of missing detections. Training requires only simulated data and therefore tedious manual annotation of ground truth is not needed. We performed a quantitative performance evaluation based on synthetic and real 2D as well as 3D fluorescence microscopy images. We used image data of the Particle Tracking Challenge as well as real time-lapse fluorescence microscopy images displaying virus structures and chromatin structures. It turned out that our approach yields state-of-the-art results or improves the tracking results compared to previous methods.     

A CANDLE for a deeper in vivo insight

A new Collaborative Approach for eNhanced Denoising under Low-light Excitation (CANDLE) is introduced for the processing of 3D laser scanning multiphoton microscopy images. CANDLE is designed to be robust for low signal-to-noise ratio (SNR) conditions typically encountered when imaging deep in scattering biological specimens. Based on an optimized non-local means filter involving the comparison of filtered patches, CANDLE locally adapts the amount of smoothing in order to deal with the noise inhomogeneity inherent to laser scanning fluorescence microscopy images. An extensive validation on synthetic data, images acquired on microspheres and in vivo images is presented. These experiments show that the CANDLE filter obtained competitive results compared to a state-of-the-art method and a locally adaptive optimized non-local means filter, especially under low SNR conditions (PSNR<8dB). Finally, the deeper imaging capabilities enabled by the proposed filter are demonstrated on deep tissue in vivo images of neurons and fine axonal processes in the Xenopus tadpole brain.     

Label-free monitoring of spatiotemporal changes in the stem cell cytoskeletons in time-lapse phase-contrast microscopy

Investigation of the dynamic structural changes in the actin cytoskeleton during cell migration provides crucial information about the physiological conditions of a stem cell during in-vitro culture. Here we proposed a quantitative analytical model associated with texture extraction with cell tracking techniques for in situ monitoring of the cytoskeletal density change of stem cells in phase-contrast microscopy without fluorescence staining. The reliability of the model in quantifying the texture density with different orientation was first validated using a series of simulated textural images. The capability of the method to reflect the spatiotemporal regulation of the cytoskeletal structure of a living stem cell was further proved by applying it to a set of 72 h phase-contrast microscopic video of the growth dynamics of mesenchymal stem cells in vitro culture.     

Platelet cytoskeleton: immunofluorescence studies on ADP and collagen-activated platelets

Immunofluorescence studies reveal that platelet changes induced by adenosine diphosphate and collagen do not include the reorganization of the cytoskeleton in such a way as to expose actin, alpha-actinin, or vinculin. However, when such platelets were made permeable by saponin, these cytoskeletal proteins were present. In studies with collagen, fluorescence was observed along the fibers at areas of platelet adhesion and where no platelets were seen by phase microscopy. No fluorescence was observed with collagen treated with platelet-poor plasma. Scanning electron microscopy of collagen samples treated with platelet-rich plasma revealed a fibrillar meshwork with single platelets, platelet aggregates, and nodular structures that were smaller in size than individual platelets. These nodular structures may represent remnants of platelets still attached to the collagen after platelet detachment has occurred. These tenacious collagen-platelet membrane-binding sites have associated with them cytoplasmic alpha-actinin and vinculin, proteins that have been proposed by others to anchor actin filaments to the plasma membrane.     

Semi-automated discrimination of retinal pigmented epithelial cells in two-photon fluorescence images of mouse retinas

Automated image segmentation is a critical step toward achieving a quantitative evaluation of disease states with imaging techniques. Two-photon fluorescence microscopy (TPM) has been employed to visualize the retinal pigmented epithelium (RPE) and provide images indicating the health of the retina. However, segmentation of RPE cells within TPM images is difficult due to small differences in fluorescence intensity between cell borders and cell bodies. Here we present a semi-automated method for segmenting RPE cells that relies upon multiple weak features that differentiate cell borders from the remaining image. These features were scored by a search optimization procedure that built up the cell border in segments around a nucleus of interest. With six images used as a test, our method correctly identified cell borders for 69% of nuclei on average. Performance was strongly dependent upon increasing retinosome content in the RPE. TPM image analysis has the potential of providing improved early quantitative assessments of diseases affecting the RPE.OCIS codes: (170.3880) Medical and biological imaging, (170.2520) Fluorescence microscopy, (100.2960) Image analysis     

Automatic rib segmentation and labeling in computed tomography scans using a general framework for detection, recognition and segmentation of objects in volumetric data

A system for automatic segmentation and labeling of the complete rib cage in chest CT scans is presented. The method uses a general framework for automatic detection, recognition and segmentation of objects in three-dimensional medical images. The framework consists of five stages: (1) detection of relevant image structures, (2) construction of image primitives, (3) classification of the primitives, (4) grouping and recognition of classified primitives and (5) full segmentation based on the obtained groups. For this application, first 1D ridges are extracted in 3D data. Then, primitives in the form of line elements are constructed from the ridge voxels. Next a classifier is trained to classify the primitives in foreground (ribs) and background. In the grouping stage centerlines are formed from the foreground primitives and rib numbers are assigned to the centerlines. In the final segmentation stage, the centerlines act as initialization for a seeded region growing algorithm. The method is tested on 20 CT-scans. Of the primitives, 97.5% is classified correctly (sensitivity is 96.8%, specificity is 97.8%). After grouping, 98.4% of the ribs are recognized. The final segmentation is qualitatively evaluated and is very accurate for over 80% of all ribs, with slight errors otherwise.     

Varicella-zoster virus infection influences expression and organization of actin and alpha-tubulin but does not affect lamin A and vimentin

OBJECTIVE: The aim of this study was to examine the effects of varicella-zoster virus (VZV) infection on the cytoskeletal components actin, lamin A, alpha-tubulin and vimentin. METHODS: The expression patterns of these four proteins during VZV infection were studied by Northern and Western blotting. The filaments were also studied in their cellular environment by immunofluorescence using confocal microscopy. Treatment with nocodazole and cytochalasin B was performed to examine the effects of the destruction of actin or tubulin networks on the VZV replicative cycle. RESULTS: The amounts of the mRNAs of actin, lamin A, alpha-tubulin and vimentin decreased slightly at 48 h post infection (p.i.) with VZV. The cellular content of the lamin A protein appeared to remain stable during the time period analyzed, whereas the amounts of actin, alpha-tubulin and vimentin decreased slightly at 24 h p.i. until the end of the viral cycle. Rearrangement of microfilaments and microtubules was observed at 24 h p.i. The addition of nocodazole or cytochalasin B decreased viral replication. CONCLUSIONS: During the VZV replicative cycle, tubulin and actin networks undergo significant changes including fiber elongation. If destroyed intentionally, viral replication is diminished, suggesting that these systems are vital for an efficient infection and viral replication.     

On the usability of synthetic data for improving the robustness of deep learning-based segmentation of cardiac magnetic resonance images

Deep learning-based segmentation methods provide an effective and automated way for assessing the structure and function of the heart in cardiac magnetic resonance (CMR) images. However, despite their state-of-the-art performance on images acquired from the same source (same scanner or scanner vendor) as images used during training, their performance degrades significantly on images coming from different domains. A straightforward approach to tackle this issue consists of acquiring large quantities of multi-site and multi-vendor data, which is practically infeasible. Generative adversarial networks (GANs) for image synthesis present a promising solution for tackling data limitations in medical imaging and addressing the generalization capability of segmentation models. In this work, we explore the usability of synthesized short-axis CMR images generated using a segmentation-informed conditional GAN, to improve the robustness of heart cavity segmentation models in a variety of different settings. The GAN is trained on paired real images and corresponding segmentation maps belonging to both the heart and the surrounding tissue, reinforcing the synthesis of semantically-consistent and realistic images. First, we evaluate the segmentation performance of a model trained solely with synthetic data and show that it only slightly underperforms compared to the baseline trained with real data. By further combining real with synthetic data during training, we observe a substantial improvement in segmentation performance (up to 4% and 40% in terms of Dice score and Hausdorff distance) across multiple data-sets collected from various sites and scanner. This is additionally demonstrated across state-of-the-art 2D and 3D segmentation networks, whereby the obtained results demonstrate the potential of the proposed method in tackling the presence of the domain shift in medical data. Finally, we thoroughly analyze the quality of synthetic data and its ability to replace real MR images during training, as well as provide an insight into important aspects of utilizing synthetic images for segmentation.     

Automatic optimal filament segmentation with sub-pixel accuracy using generalized linear models and B-spline level-sets

Biological filaments, such as actin filaments, microtubules, and cilia, are often imaged using different light-microscopy techniques. Reconstructing the filament curve from the acquired images constitutes the filament segmentation problem. Since filaments have lower dimensionality than the image itself, there is an inherent trade-off between tracing the filament with sub-pixel accuracy and avoiding noise artifacts. Here, we present a globally optimal filament segmentation method based on B-spline vector level-sets and a generalized linear model for the pixel intensity statistics. We show that the resulting optimization problem is convex and can hence be solved with global optimality. We introduce a simple and efficient algorithm to compute such optimal filament segmentations, and provide an open-source implementation as an ImageJ/Fiji plugin. We further derive an information-theoretic lower bound on the filament segmentation error, quantifying how well an algorithm could possibly do given the information in the image. We show that our algorithm asymptotically reaches this bound in the spline coefficients. We validate our method in comprehensive benchmarks, compare with other methods, and show applications from fluorescence, phase-contrast, and dark-field microscopy.     

Wavelet-based background and noise subtraction for fluorescence microscopy images

Fluorescence microscopy images are inevitably contaminated by background intensity contributions. Fluorescence from out-of-focus planes and scattered light are important sources of slowly varying, low spatial frequency background, whereas background varying from pixel to pixel (high frequency noise) is introduced by the detection system. Here we present a powerful, easy-to-use software, wavelet-based background and noise subtraction (WBNS), which effectively removes both of these components. To assess its performance, we apply WBNS to synthetic images and compare the results quantitatively with the ground truth and with images processed by other background removal algorithms. We further evaluate WBNS on real images taken with a light-sheet microscope and a super-resolution stimulated emission depletion microscope. For both cases, we compare the WBNS algorithm with hardware-based background removal techniques and present a quantitative assessment of the results. WBNS shows an excellent performance in all these applications and significantly enhances the visual appearance of fluorescence images. Moreover, it may serve as a pre-processing step for further quantitative analysis.     

The endothelial cytoskeleton as a target of electroporation-based therapies

Electroporation-based therapies, such as electrochemotherapy and electrogene therapy, result in the disruption of blood vessel networks in vivo and cause changes in blood flow and vascular permeability. The effects of electroporation on the cytoskeleton of cultured primary endothelial cells and on endothelial monolayer permeability were investigated to elucidate possible mechanisms involved. Human umbilical vein endothelial cells (HUVECs) were electroporated in situ and then immunofluorescence staining for filamentous actin, beta-tubulin, vimentin, and VE-cadherin as well as Western blotting analysis of levels of phosphorylated myosin light chain and cytoskeletal proteins were performed. Endothelial permeability was determined by monitoring the passage of FITC-coupled dextran through endothelial monolayers. Exposure of endothelial cells to electric pulses resulted in a profound disruption of microfilament and microtubule cytoskeletal networks, loss of contractility, and loss of vascular endothelial cadherin from cell-to-cell junctions immediately after electroporation. These effects were voltage dependent and reversible because cytoskeletal structures recovered within 60 min of electroporation with up to 40 V, without any significant loss of cell viability. The cytoskeletal effects of electroporation were paralleled by a rapid increase in endothelial monolayer permeability. These results suggest that the remodeling of the endothelial cytoskeleton and changes in endothelial barrier function could contribute to the vascular disrupting actions of electroporation-based therapies and provide an insight into putative mechanisms responsible for the observed increase in permeability and cessation of blood flow in vivo.     

Automated Registration of Freehand B-Mode Ultrasound and Magnetic Resonance Imaging of the Carotid Arteries Based on Geometric Features

An automated method for registering B-mode ultrasound (US) and magnetic resonance imaging (MRI) of the carotid arteries is proposed. The registration uses geometric features, namely, lumen centerlines and lumen segmentations, which are extracted fully automatically from the images after manual annotation of three seed points in US and MRI. The registration procedure starts with alignment of the lumen centerlines using a point-based registration algorithm. The resulting rigid transformation is used to initialize a rigid and subsequent non-rigid registration procedure that jointly aligns centerlines and segmentations by minimizing a weighted sum of the Euclidean distance between centerlines and the dissimilarity between segmentations. The method was evaluated in 28 carotid arteries from eight patients and six healthy volunteers. First, the automated US lumen segmentation method was validated and optimized in a cross-validation experiment. Next, the effect of the weighting parameter of the proposed registration dissimilarity metric and the control point spacing in the non-rigid registration was evaluated. Finally, the proposed registration method was evaluated in comparison to an existing intensity-and-point-based method, a registration using only the centerlines and a registration using manual US lumen segmentations. Registration accuracy was measured in terms of the mean surface distance between manual US segmentations and the registered MRI segmentations. The average mean surface distance was 0.78 ± 0.34 mm for all subjects, 0.65 ± 0.09 mm for healthy volunteers and 0.87 ± 0.42 mm for patients. The results for the complete set were significantly better (Wilcoxon test, p < 0.01) than the results for the intensity-and-point-based method and the centerline-based registration method. We conclude that the proposed method can robustly and accurately register US and MR images of the carotid artery, allowing multimodal analysis of the carotid plaque to improve plaque assessment.     

Robust Rician noise estimation for MR images

In this paper, a new object-based method to estimate noise in magnitude MR images is proposed. The main advantage of this object-based method is its robustness to background artefacts such as ghosting. The proposed method is based on the adaptation of the Median Absolute Deviation (MAD) estimator in the wavelet domain for Rician noise. The MAD is a robust and efficient estimator initially proposed to estimate Gaussian noise. In this work, the adaptation of MAD operator for Rician noise is performed by using only the wavelet coefficients corresponding to the object and by correcting the estimation with an iterative scheme based on the SNR of the image. During the evaluation, a comparison of the proposed method with several state-of-the-art methods is performed. A quantitative validation on synthetic phantom with and without artefacts is presented. A new validation framework is proposed to perform quantitative validation on real data. The impact of the accuracy of noise estimation on the performance of a denoising filter is also studied. The results obtained on synthetic images show the accuracy and the robustness of the proposed method. Within the validation on real data, the proposed method obtained very competitive results compared to the methods under study.     

Optimized approaches for optical sectioning and resolution enhancement in 2D structured illumination microscopy

The use of structured illumination in fluorescence microscopy allows the suppression of out of focus light and an increase in effective spatial resolution. In this paper we consider different approaches for reconstructing 2D structured illumination images in order to combine these two attributes, to allow fast, optically sectioned, superresolution imaging. We present a linear reconstruction method that maximizes the axial frequency extent of the combined 2D structured illumination passband along with an empirically optimized approximation to this scheme. These reconstruction methods are compared to other schemes using structured illumination images of fluorescent samples. For sinusoidal excitation at half the incoherent cutoff frequency we find that removing information in the zero order passband except for a small region close to the excitation frequency, where it replaces the complementary information from the displaced first order passband, enables optimal reconstruction of optically sectioned images with enhanced spatial resolution.OCIS codes: (180.2520) Fluorescence microscopy, (180.6900) Three-dimensional microscopy, (100.6640) Superresolution     

High-resolution 3D optical microscopy inside the beating zebrafish heart using prospective optical gating

3D fluorescence imaging is a fundamental tool in the study of functional and developmental biology, but effective imaging is particularly difficult in moving structures such as the beating heart. We have developed a non-invasive real-time optical gating system that is able to exploit the periodic nature of the motion to acquire high resolution 3D images of the normally-beating zebrafish heart without any unnecessary exposure of the sample to harmful excitation light. In order for the image stack to be artefact-free, it is essential to use a synchronization source that is invariant as the sample is scanned in 3D. We therefore describe a scheme whereby fluorescence image slices are scanned through the sample while a brightfield camera sharing the same objective lens is maintained at a fixed focus, with correction of sample drift also included. This enables us to maintain, throughout an extended 3D volume, the same standard of synchronization we have previously demonstrated in and near a single 2D plane. Thus we are able image the complete beating zebrafish heart exactly as if the heart had been artificially stopped, but sidestepping this undesirable interference with the heart and instead allowing the heart to beat as normal.OCIS codes: (170.2520) Fluorescence microscopy, (110.2960) Image analysis, (170.3880) Medical and biological imaging, (180.6900) Three-dimensional microscopy, (110.6880) Three-dimensional image acquisition     

SPIM-fluid: open source light-sheet based platform for high-throughput imaging

Light sheet fluorescence microscopy has recently emerged as the technique of choice for obtaining high quality 3D images of whole organisms/embryos with low photodamage and fast acquisition rates. Here we present an open source unified implementation based on Arduino and Micromanager, which is capable of operating Light Sheet Microscopes for automatized 3D high-throughput imaging on three-dimensional cell cultures and model organisms like zebrafish, oriented to massive drug screening.OCIS codes: (180.2520) Fluorescence microscopy, (110.6880) Three-dimensional image acquisition