Efficient workflow for automatic segmentation of the right heart based on 2D echocardiography

The present study aimed to present a workflow algorithm for automatic processing of 2D echocardiography images. The workflow was based on several sequential steps. For each step, we compared different approaches. Epicardial 2D echocardiography datasets were acquired during various open-chest beating-heart surgical procedures in three porcine hearts. We proposed a metric called the global index that is a weighted average of several accuracy coefficients, indices and the mean processing time. This metric allows the estimation of the speed and accuracy for processing each image. The global index ranges from 0 to 1, which facilitates comparison between different approaches. The second step involved comparison among filtering, sharpening and segmentation techniques. During the noise reduction step, we compared the median filter, total variation filter, bilateral filter, curvature flow filter, non-local means filter and mean shift filter. To clarify the endocardium borders of the right heart, we used the linear sharpen. Lastly, we applied watershed segmentation, clusterisation, region-growing, morphological segmentation, image foresting segmentation and isoline delineation. We assessed all the techniques and identified the most appropriate workflow for echocardiography image segmentation of the right heart. For successful processing and segmentation of echocardiography images with minimal error, we found that the workflow should include the total variation filter/bilateral filter, linear sharpen technique, isoline delineation/region-growing segmentation and morphological post-processing. We presented an efficient and accurate workflow for the precise diagnosis of cardiovascular diseases. We introduced the global index metric for image pre-processing and segmentation estimation.     

CEREBRUM: a fast and fully-volumetric Convolutional Encoder-decodeR for weakly-supervised sEgmentation of BRain strUctures from out-of-the-scanner MRI

Many functional and structural neuroimaging studies call for accurate morphometric segmentation of different brain structures starting from image intensity values of MRI scans. Current automatic (multi-) atlas-based segmentation strategies often lack accuracy on difficult-to-segment brain structures and, since these methods rely on atlas-to-scan alignment, they may take long processing times. Alternatively, recent methods deploying solutions based on Convolutional Neural Networks (CNNs) are enabling the direct analysis of out-of-the-scanner data. However, current CNN-based solutions partition the test volume into 2D or 3D patches, which are processed independently. This process entails a loss of global contextual information, thereby negatively impacting the segmentation accuracy. In this work, we design and test an optimised end-to-end CNN architecture that makes the exploitation of global spatial information computationally tractable, allowing to process a whole MRI volume at once. We adopt a weakly supervised learning strategy by exploiting a large dataset composed of 947 out-of-the-scanner (3 Tesla T1-weighted 1mm isotropic MP-RAGE 3D sequences) MR Images. The resulting model is able to produce accurate multi-structure segmentation results in only a few seconds. Different quantitative measures demonstrate an improved accuracy of our solution when compared to state-of-the-art techniques. Moreover, through a randomised survey involving expert neuroscientists, we show that subjective judgements favour our solution with respect to widely adopted atlas-based software.     

Quantitative characterization of human breast tissue based on deep learning segmentation of 3D optical coherence tomography images

In this study, we performed dual-modality optical coherence tomography (OCT) characterization (volumetric OCT imaging and quantitative optical coherence elastography) on human breast tissue specimens. We trained and validated a U-Net for automatic image segmentation. Our results demonstrated that U-Net segmentation can be used to assist clinical diagnosis for breast cancer, and is a powerful enabling tool to advance our understanding of the characteristics for breast tissue. Based on the results obtained from U-Net segmentation of 3D OCT images, we demonstrated significant morphological heterogeneity in small breast specimens acquired through diagnostic biopsy. We also found that breast specimens affected by different pathologies had different structural characteristics. By correlating U-Net analysis of structural OCT images with mechanical measurement provided by quantitative optical coherence elastography, we showed that the change of mechanical properties in breast tissue is not directly due to the change in the amount of dense or porous tissue.     

The segmentation of colorectal MRI images

One of the key criteria that informs patient management decisions for colorectal cancer is the extent of the shortest distance from the edge of the primary tumour to the edge of the mesorectum, also referred to as circumferential resection margin (CRM). This region is resected during surgery. The CRM is difficult for clinicians to measure accurately, particularly from 2D slice data. We present a method for automatically calculating and visualising the CRM distances in colorectal cancer MR images. We use local phase of the monogenic signal calculated from the MR image intensities to find edge and ridge features within the data. A non-parametric mixture model is then used to describe image intensity values within level set framework in order to segment the mesorectal fascia and the corresponding tumour and lymph nodes, as distinct regions. This segmentation is used to provide an automatic analysis of the shortest distance resection margin, and we show that this is consistent with that of the clinically accepted MERCURY method. We use the segmentation to provide a 3D visualisation of where the resection margin is smallest. Finally, we reconstruct a 3D map of the segmented anatomy. Both the visualisation methods provide a useful tool to aid surgeons in their treatment planning.     

Fully automatic segmentation of the mitral leaflets in 3D transesophageal echocardiographic images using multi-atlas joint label fusion and deformable medial modeling

Comprehensive visual and quantitative analysis of in vivo human mitral valve morphology is central to the diagnosis and surgical treatment of mitral valve disease. Real-time 3D transesophageal echocardiography (3D TEE) is a practical, highly informative imaging modality for examining the mitral valve in a clinical setting. To facilitate visual and quantitative 3D TEE image analysis, we describe a fully automated method for segmenting the mitral leaflets in 3D TEE image data. The algorithm integrates complementary probabilistic segmentation and shape modeling techniques (multi-atlas joint label fusion and deformable modeling with continuous medial representation) to automatically generate 3D geometric models of the mitral leaflets from 3D TEE image data. These models are unique in that they establish a shape-based coordinate system on the valves of different subjects and represent the leaflets volumetrically, as structures with locally varying thickness. In this work, expert image analysis is the gold standard for evaluating automatic segmentation. Without any user interaction, we demonstrate that the automatic segmentation method accurately captures patient-specific leaflet geometry at both systole and diastole in 3D TEE data acquired from a mixed population of subjects with normal valve morphology and mitral valve disease.     

Semi-automatic segmentation and detection of aorta dissection wall in MDCT angiography

Aorta dissection is a serious vascular disease produced by a rupture of the tunica intima of the vessel wall that can be lethal to the patient. The related diagnosis is strongly based on images, where the multi-detector CT is the most generally used modality. We aim at developing a semi-automatic segmentation tool for aorta dissections, which will isolate the dissection (or flap) from the rest of the vascular structure. The proposed method is based on different stages, the first one being the semi-automatic extraction of the aorta centerline and its main branches, allowing an subsequent automatic segmentation of the outer wall of the aorta, based on a geodesic level set framework. This segmentation is then followed by an extraction the center of the dissected wall as a 3D mesh using an original algorithm based on the zero crossing of two vector fields. Our method has been applied to five datasets from three patients with chronic aortic dissection. The comparison with manually segmented dissections shows an average absolute distance value of about half a voxel. We believe that the proposed method, which tries to solve a problem that has attracted little attention to the medical image processing community, provides a new and interesting tool to isolate the intimal flap that can provide very useful information to the clinician.     

ACME: Automatic feature extraction for cell migration examination through intravital microscopy imaging

Cell detection and tracking applied to in vivo fluorescence microscopy has become an essential tool in biomedicine to characterize 4D (3D space plus time) biological processes at the cellular level. Traditional approaches to cell motion analysis by microscopy imaging, although based on automatic frameworks, still require manual supervision at some points of the system. Hence, when dealing with a large amount of data, the analysis becomes incredibly time-consuming and typically yields poor biological information. In this paper, we propose a fully-automated system for segmentation, tracking and feature extraction of migrating cells within blood vessels in 4D microscopy imaging. Our system consists of a robust 3D convolutional neural network (CNN) for joint blood vessel and cell segmentation, a 3D tracking module with collision handling, and a novel method for feature extraction, which takes into account the particular geometry in the cell-vessel arrangement. Experiments on a large 4D intravital microscopy dataset show that the proposed system achieves a significantly better performance than the state-of-the-art tools for cell segmentation and tracking. Furthermore, we have designed an analytical method of cell behaviors based on the automatically extracted features, which supports the hypotheses related to leukocyte migration posed by expert biologists. This is the first time that such a comprehensive automatic analysis of immune cell migration has been performed, where the total population under study reaches hundreds of neutrophils and thousands of time instances.     

Three-dimensional printed models in congenital heart disease

The purpose of this article is to discuss technical considerations and current applications of three-dimensional (3D) printing in congenital heart disease (CHD). CHD represent an attractive field for the application of 3D printed models, with consistent progress made in the past decade. Current 3D models are able to reproduce complex cardiac and extra-cardiac anatomy including small details with very limited range of errors (<1 mm), so this tool could be of value in the planning of surgical or percutaneous treatments for selected cases of CHD. However, the steps involved in the building of 3D models, consisting of image acquisition and selection, segmentation, and printing are highly operator dependent. Current 3D models may be rigid or flexible, but unable to reproduce the physiologic variations during the cardiac cycle. Furthermore, high costs and long average segmentation and printing times (18–24 h) limit a more extensive use. There is a need for better standardization of the procedure employed for collection of the images, the segmentation methods and processes, the phase of cardiac cycle used, and in the materials employed for printing. More studies are necessary to evaluate the diagnostic accuracy and cost-effectiveness of 3D printed models in congenital cardiac care.     

Interaction in the segmentation of medical images: a survey

Segmentation of the object of interest is a difficult step in the analysis of digital images. Fully automatic methods sometimes fail, producing incorrect results and requiring the intervention of a human operator. This is often true in medical applications, where image segmentation is particularly difficult due to restrictions imposed by image acquisition, pathology and biological variation. In this paper we present an early review of the largely unknown territory of human-computer interaction in image segmentation. The purpose is to identify patterns in the use of interaction and to develop qualitative criteria to evaluate interactive segmentation methods. We discuss existing interactive methods with respect to the following aspects: the type of information provided by the user, how this information affects the computational part, and the purpose of interaction in the segmentation process. The discussion is based on the potential impact of each strategy on the accuracy, repeatability and interaction efficiency. Among others, these are important aspects to characterise and understand the implications of interaction to the results generated by an interactive segmentation method. This survey is focused on medical imaging, however similar patterns are expected to hold for other applications as well.     

BrainSuite: an automated cortical surface identification tool

We describe a new magnetic resonance (MR) image analysis tool that produces cortical surface representations with spherical topology from MR images of the human brain. The tool provides a sequence of low-level operations in a single package that can produce accurate brain segmentations in clinical time. The tools include skull and scalp removal, image nonuniformity compensation, voxel-based tissue classification, topological correction, rendering, and editing functions. The collection of tools is designed to require minimal user interaction to produce cortical representations. In this paper we describe the theory of each stage of the cortical surface identification process. We then present classification validation results using real and phantom data. We also present a study of interoperator variability.     

GIST: an interactive, GPU-based level set segmentation tool for 3D medical images

While level sets have demonstrated a great potential for 3D medical image segmentation, their usefulness has been limited by two problems. First, 3D level sets are relatively slow to compute. Second, their formulation usually entails several free parameters which can be very difficult to correctly tune for specific applications. The second problem is compounded by the first. This paper describes a new tool for 3D segmentation that addresses these problems by computing level-set surface models at interactive rates. This tool employs two important, novel technologies. First is the mapping of a 3D level-set solver onto a commodity graphics card (GPU). This mapping relies on a novel mechanism for GPU memory management. The interactive rates level-set PDE solver give the user immediate feedback on the parameter settings, and thus users can tune free parameters and control the shape of the model in real time. The second technology is the use of intensity-based speed functions, which allow a user to quickly and intuitively specify the behavior of the deformable model. We have found that the combination of these interactive tools enables users to produce good, reliable segmentations. To support this observation, this paper presents qualitative results from several different datasets as well as a quantitative evaluation from a study of brain tumor segmentations.     

Joint level-set shape modeling and appearance modeling for brain structure segmentation

This paper presents a new fully automatic model-based segmentation algorithm, which combines level-set methods to model the shape of brain structures and their variation with active appearance modeling to generate images that are used to drive the segmentation. The new algorithm incorporates multi-modality images to improve the segmentation performance and the recursive least square (RLS) algorithm is adopted to minimize the difference between test image and the one synthesized from the shape and appearance modeling. When compared with manual segmentation, the 2D and 3D experiments demonstrate that the new algorithm is computationally efficient and robust and is promising for automatic segmentation of the lateral ventricles.     

A review of 3D vessel lumen segmentation techniques: Models,features and extraction schemes

Vascular diseases are among the most important public health problems in developed countries. Given the size and complexity of modern angiographic acquisitions, segmentation is a key step toward the accurate visualization, diagnosis and quantification of vascular pathologies.Despite the tremendous amount of past and on-going dedicated research, vascular segmentation remains a challenging task. In this paper, we review state-of-the-art literature on vascular segmentation, with a particular focus on 3D contrast-enhanced imaging modalities (MRA and CTA). We structure our analysis along three axes: models, features and extraction schemes. We first detail model-based assumptions on the vessel appearance and geometry which can embedded in a segmentation approach. We then review the image features that can be extracted to evaluate these models. Finally, we discuss how existing extraction schemes combine model and feature information to perform the segmentation task.Each component (model, feature and extraction scheme) plays a crucial role toward the efficient, robust and accurate segmentation of vessels of interest. Along each axis of study, we discuss the theoretical and practical properties of recent approaches and highlight the most advanced and promising ones.     

Qualitative Evaluation of a Novel 3D Volumetric Radiotherapy Segmentation Tool

IntroductionA novel 3D volumetric segmentation tool allows the user to outline using a small number of points on a range of planes. Unique 3D volumetric “sculpting” tools enable editing of the resulting structures across multiple slices concurrently. This article reports the results of radiation oncologists' preclinical evaluation of the tool.MethodsThree clinicians outlined prostate and seminal vesicles on 14 data sets using the traditional slice-by-slice method and the new 3D tool. The project gathered focus-group feedback to gather rich data relating to clinician perceptions of the new 3D outlining paradigm. Emergent themes were identified and categorised for discussion.ResultsRadiation oncologists reported high levels of satisfaction with the outlines arising from both paradigms. The volumetric sculpting was a challenge, but participants enjoyed using points in orthogonal planes and felt that the paradigm had potential value in terms of speed and smooth volume creation.ConclusionThis study has demonstrated that a 3D volumetric outlining system is felt to have potential value by radiation oncologists for accelerating clinician-directed prostate and seminal vesicle segmentation. The new tool was well-received and reported to be capable of producing very rapid and smooth volumes. The novelty of the approach required significant training input and a radically different approach of minimal point placement. Further testing of this software with a less time-poor cohort may be indicated to gain reliable quantitative data relating to the impact on segmentation time.     

Fast automatic myocardial segmentation in 4D cine CMR datasets

A novel automatic 3D+time left ventricle (LV) segmentation framework is proposed for cardiac magnetic resonance (CMR) datasets. The proposed framework consists of three conceptual blocks to delineate both endo and epicardial contours throughout the cardiac cycle: (1) an automatic 2D mid-ventricular initialization and segmentation; (2) an automatic stack initialization followed by a 3D segmentation at the end-diastolic phase; and (3) a tracking procedure. Hereto, we propose to adapt the recent B-spline Explicit Active Surfaces (BEAS) framework to the properties of CMR images by integrating dedicated energy terms. Moreover, we extend the coupled BEAS formalism towards its application in 3D MR data by adapting it to a cylindrical space suited to deal with the topology of the image data. Furthermore, a fast stack initialization method is presented for efficient initialization and to enforce consistent cylindrical topology. Finally, we make use of an anatomically constrained optical flow method for temporal tracking of the LV surface.The proposed framework has been validated on 45 CMR datasets taken from the 2009 MICCAI LV segmentation challenge. Results show the robustness, efficiency and competitiveness of the proposed method both in terms of accuracy and computational load.     

Generalizable multi-task,multi-domain deep segmentation of sparse pediatric imaging datasets via multi-scale contrastive regularization and multi-joint anatomical priors

Clinical diagnosis of the pediatric musculoskeletal system relies on the analysis of medical imaging examinations. In the medical image processing pipeline, semantic segmentation using deep learning algorithms enables an automatic generation of patient-specific three-dimensional anatomical models which are crucial for morphological evaluation. However, the scarcity of pediatric imaging resources may result in reduced accuracy and generalization performance of individual deep segmentation models. In this study, we propose to design a novel multi-task, multi-domain learning framework in which a single segmentation network is optimized over the union of multiple datasets arising from distinct parts of the anatomy. Unlike previous approaches, we simultaneously consider multiple intensity domains and segmentation tasks to overcome the inherent scarcity of pediatric data while leveraging shared features between imaging datasets. To further improve generalization capabilities, we employ a transfer learning scheme from natural image classification, along with a multi-scale contrastive regularization aimed at promoting domain-specific clusters in the shared representations, and multi-joint anatomical priors to enforce anatomically consistent predictions. We evaluate our contributions for performing bone segmentation using three scarce and pediatric imaging datasets of the ankle, knee, and shoulder joints. Our results demonstrate that the proposed approach outperforms individual, transfer, and shared segmentation schemes in Dice metric with statistically sufficient margins. The proposed model brings new perspectives towards intelligent use of imaging resources and better management of pediatric musculoskeletal disorders.     

Fully automatic three-dimensional visualization of intravascular optical coherence tomography images: methods and feasibility in vivo

Intravascular optical coherence tomography (IV-OCT) is an imaging modality that can be used for the assessment of intracoronary stents. Recent publications pointed to the fact that 3D visualizations have potential advantages compared to conventional 2D representations. However, 3D imaging still requires a time consuming manual procedure not suitable for on-line application during coronary interventions. We propose an algorithm for a rapid and fully automatic 3D visualization of IV-OCT pullbacks. IV-OCT images are first processed for the segmentation of the different structures. This also allows for automatic pullback calibration. Then, according to the segmentation results, different structures are depicted with different colors to visualize the vessel wall, the stent and the guide-wire in details. Final 3D rendering results are obtained through the use of a commercial 3D DICOM viewer. Manual analysis was used as ground-truth for the validation of the segmentation algorithms. A correlation value of 0.99 and good limits of agreement (Bland Altman statistics) were found over 250 images randomly extracted from 25 in vivo pullbacks. Moreover, 3D rendering was compared to angiography, pictures of deployed stents made available by the manufacturers and to conventional 2D imaging corroborating visualization results. Computational time for the visualization of an entire data sets resulted to be ~74 sec. The proposed method allows for the on-line use of 3D IV-OCT during percutaneous coronary interventions, potentially allowing treatments optimization.OCIS codes: (170.4500) Optical coherence tomography, (100.6890) Three-dimensional image processing, (330.5000) Vision - patterns and recognition     

Automatic change detection in multimodal serial MRI: application to multiple sclerosis lesion evolution

The automatic analysis of subtle changes between MRI scans is an important tool for assessing disease evolution over time. Manual labeling of evolutions in 3D data sets is tedious and error prone. Automatic change detection, however, remains a challenging image processing problem. A variety of MRI artifacts introduce a wide range of unrepresentative changes between images, making standard change detection methods unreliable. In this study we describe an automatic image processing system that addresses these issues. Registration errors and undesired anatomical deformations are compensated using a versatile multiresolution deformable image matching method that preserves significant changes at a given scale. A nonlinear intensity normalization method is associated with statistical hypothesis test methods to provide reliable change detection. Multimodal data is optionally exploited to reduce the false detection rate. The performance of the system was evaluated on a large database of 3D multimodal, MR images of patients suffering from relapsing remitting multiple sclerosis (MS). The method was assessed using receiver operating characteristics (ROC) analysis, and validated in a protocol involving two neurologists. The automatic system outperforms the human expert, detecting many lesion evolutions that are missed by the expert, including small, subtle changes.