Learning to segment fetal brain tissue from noisy annotations

Automatic fetal brain tissue segmentation can enhance the quantitative assessment of brain development at this critical stage. Deep learning methods represent the state of the art in medical image segmentation and have also achieved impressive results in brain segmentation. However, effective training of a deep learning model to perform this task requires a large number of training images to represent the rapid development of the transient fetal brain structures. On the other hand, manual multi-label segmentation of a large number of 3D images is prohibitive. To address this challenge, we segmented 272 training images, covering 19–39 gestational weeks, using an automatic multi-atlas segmentation strategy based on deformable registration and probabilistic atlas fusion, and manually corrected large errors in those segmentations. Since this process generated a large training dataset with noisy segmentations, we developed a novel label smoothing procedure and a loss function to train a deep learning model with smoothed noisy segmentations. Our proposed methods properly account for the uncertainty in tissue boundaries. We evaluated our method on 23 manually-segmented test images of a separate set of fetuses. Results show that our method achieves an average Dice similarity coefficient of 0.893 and 0.916 for the transient structures of younger and older fetuses, respectively. Our method generated results that were significantly more accurate than several state-of-the-art methods including nnU-Net that achieved the closest results to our method. Our trained model can serve as a valuable tool to enhance the accuracy and reproducibility of fetal brain analysis in MRI.     

MB-FSGAN: Joint segmentation and quantification of kidney tumor on CT by the multi-branch feature sharing generative adversarial network

The segmentation of the kidney tumor and the quantification of its tumor indices (i.e., the center point coordinates, diameter, circumference, and cross-sectional area of the tumor) are important steps in tumor therapy. These quantifies the tumor morphometrical details to monitor disease progression and accurately compare decisions regarding the kidney tumor treatment. However, manual segmentation and quantification is a challenging and time-consuming process in practice and exhibit a high degree of variability within and between operators. In this paper, MB-FSGAN (multi-branch feature sharing generative adversarial network) is proposed for simultaneous segmentation and quantification of kidney tumor on CT. MB-FSGAN consists of multi-scale feature extractor (MSFE), locator of the area of interest (LROI), and feature sharing generative adversarial network (FSGAN). MSFE makes strong semantic information on different scale feature maps, which is particularly effective in detecting small tumor targets. The LROI extracts the region of interest of the tumor, greatly reducing the time complexity of the network. FSGAN correctly segments and quantifies kidney tumors through joint learning and adversarial learning, which effectively exploited the commonalities and differences between the two related tasks. Experiments are performed on CT of 113 kidney tumor patients. For segmentation, MB-FSGAN achieves a pixel accuracy of 95.7%. For the quantification of five tumor indices, the R² coefficient of tumor circumference is 0.9465. The results show that the network has reliable performance and shows its effectiveness and potential as a clinical tool.     

High resolution histopathology image generation and segmentation through adversarial training

Semantic segmentation of histopathology images can be a vital aspect of computer-aided diagnosis, and deep learning models have been effectively applied to this task with varying levels of success. However, their impact has been limited due to the small size of fully annotated datasets. Data augmentation is one avenue to address this limitation. Generative Adversarial Networks (GANs) have shown promise in this respect, but previous work has focused mostly on classification tasks applied to MR and CT images, both of which have lower resolution and scale than histopathology images. There is limited research that applies GANs as a data augmentation approach for large-scale image semantic segmentation, which requires high-quality image-mask pairs. In this work, we propose a multi-scale conditional GAN for high-resolution, large-scale histopathology image generation and segmentation. Our model consists of a pyramid of GAN structures, each responsible for generating and segmenting images at a different scale. Using semantic masks, the generative component of our model is able to synthesize histopathology images that are visually realistic. We demonstrate that these synthesized images along with their masks can be used to boost segmentation performance, especially in the semi-supervised scenario.     

Shape constrained CNN for segmentation guided prediction of myocardial shape and pose parameters in cardiac MRI

Semantic segmentation using convolutional neural networks (CNNs) is the state-of-the-art for many medical image segmentation tasks including myocardial segmentation in cardiac MR images. However, the predicted segmentation maps obtained from such standard CNN do not allow direct quantification of regional shape properties such as regional wall thickness. Furthermore, the CNNs lack explicit shape constraints, occasionally resulting in unrealistic segmentations. In this paper, we use a CNN to predict shape parameters of an underlying statistical shape model of the myocardium learned from a training set of images. Additionally, the cardiac pose is predicted, which allows to reconstruct the myocardial contours. The integrated shape model regularizes the predicted contours and guarantees realistic shapes. We enforce robustness of shape and pose prediction by simultaneously performing pixel-wise semantic segmentation during training and define two loss functions to impose consistency between the two predicted representations: one distance-based loss and one overlap-based loss. We evaluated the proposed method in a 5-fold cross validation on an in-house clinical dataset with 75 subjects and on the ACDC and LVQuan19 public datasets. We show that the two newly defined loss functions successfully increase the consistency between shape and pose parameters and semantic segmentation, which leads to a significant improvement of the reconstructed myocardial contours. Additionally, these loss functions drastically reduce the occurrence of unrealistic shapes in the semantic segmentation output.     

Multi-task learning for the segmentation of organs at risk with label dependence

Automatic segmentation of organs at risk is crucial to aid diagnoses and remains a challenging task in medical image analysis domain. To perform the segmentation, we use multi-task learning (MTL) to accurately determine the contour of organs at risk in CT images. We train an encoder-decoder network for two tasks in parallel. The main task is the segmentation of organs, entailing a pixel-level classification in the CT images, and the auxiliary task is the multi-label classification of organs, entailing an image-level multi-label classification of the CT images. To boost the performance of the multi-label classification, we propose a weighted mean cross entropy loss function for the network training, where the weights are the global conditional probability between two organs. Based on MTL, we optimize the false positive filtering (FPF) algorithm to decrease the number of falsely segmented organ pixels in the CT images. Specifically, we propose a dynamic threshold selection (DTS) strategy to prevent true positive rates from decreasing when using the FPF algorithm. We validate these methods on the public ISBI 2019 segmentation of thoracic organs at risk (SegTHOR) challenge dataset and a private medical organ dataset. The experimental results show that networks using our proposed methods outperform basic encoder-decoder networks without increasing the training time complexity.     

Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images

Tumor classification and segmentation are two important tasks for computer-aided diagnosis (CAD) using 3D automated breast ultrasound (ABUS) images. However, they are challenging due to the significant shape variation of breast tumors and the fuzzy nature of ultrasound images (e.g., low contrast and signal to noise ratio). Considering the correlation between tumor classification and segmentation, we argue that learning these two tasks jointly is able to improve the outcomes of both tasks. In this paper, we propose a novel multi-task learning framework for joint segmentation and classification of tumors in ABUS images. The proposed framework consists of two sub-networks: an encoder-decoder network for segmentation and a light-weight multi-scale network for classification. To account for the fuzzy boundaries of tumors in ABUS images, our framework uses an iterative training strategy to refine feature maps with the help of probability maps obtained from previous iterations. Experimental results based on a clinical dataset of 170 3D ABUS volumes collected from 107 patients indicate that the proposed multi-task framework improves tumor segmentation and classification over the single-task learning counterparts.     

LMISA: A lightweight multi-modality image segmentation network via domain adaptation using gradient magnitude and shape constraint

In medical image segmentation, supervised machine learning models trained using one image modality (e.g. computed tomography (CT)) are often prone to failure when applied to another image modality (e.g. magnetic resonance imaging (MRI)) even for the same organ. This is due to the significant intensity variations of different image modalities. In this paper, we propose a novel end-to-end deep neural network to achieve multi-modality image segmentation, where image labels of only one modality (source domain) are available for model training and the image labels for the other modality (target domain) are not available. In our method, a multi-resolution locally normalized gradient magnitude approach is firstly applied to images of both domains for minimizing the intensity discrepancy. Subsequently, a dual task encoder-decoder network including image segmentation and reconstruction is utilized to effectively adapt a segmentation network to the unlabeled target domain. Additionally, a shape constraint is imposed by leveraging adversarial learning. Finally, images from the target domain are segmented, as the network learns a consistent latent feature representation with shape awareness from both domains. We implement both 2D and 3D versions of our method, in which we evaluate CT and MRI images for kidney and cardiac tissue segmentation. For kidney, a public CT dataset (KiTS19, MICCAI 2019) and a local MRI dataset were utilized. The cardiac dataset was from the Multi-Modality Whole Heart Segmentation (MMWHS) challenge 2017. Experimental results reveal that our proposed method achieves significantly higher performance with a much lower model complexity in comparison with other state-of-the-art methods. More importantly, our method is also capable of producing superior segmentation results than other methods for images of an unseen target domain without model retraining. The code is available at GitHub (https://github.com/MinaJf/LMISA) to encourage method comparison and further research.     

A learning-based wrapper method to correct systematic errors in automatic image segmentation: consistently improved performance in hippocampus, cortex and brain segmentation

We propose a simple but generally applicable approach to improving the accuracy of automatic image segmentation algorithms relative to manual segmentations. The approach is based on the hypothesis that a large fraction of the errors produced by automatic segmentation are systematic, i.e., occur consistently from subject to subject, and serves as a wrapper method around a given host segmentation method. The wrapper method attempts to learn the intensity, spatial and contextual patterns associated with systematic segmentation errors produced by the host method on training data for which manual segmentations are available. The method then attempts to correct such errors in segmentations produced by the host method on new images. One practical use of the proposed wrapper method is to adapt existing segmentation tools, without explicit modification, to imaging data and segmentation protocols that are different from those on which the tools were trained and tuned. An open-source implementation of the proposed wrapper method is provided, and can be applied to a wide range of image segmentation problems. The wrapper method is evaluated with four host brain MRI segmentation methods: hippocampus segmentation using FreeSurfer (Fischl et al., 2002); hippocampus segmentation using multi-atlas label fusion (Artaechevarria et al., 2009); brain extraction using BET (Smith, 2002); and brain tissue segmentation using FAST (Zhang et al., 2001). The wrapper method generates 72%, 14%, 29% and 21% fewer erroneously segmented voxels than the respective host segmentation methods. In the hippocampus segmentation experiment with multi-atlas label fusion as the host method, the average Dice overlap between reference segmentations and segmentations produced by the wrapper method is 0.908 for normal controls and 0.893 for patients with mild cognitive impairment. Average Dice overlaps of 0.964, 0.905 and 0.951 are obtained for brain extraction, white matter segmentation and gray matter segmentation, respectively.     

Head and neck tumor segmentation in PET/CT: The HECKTOR challenge

This paper relates the post-analysis of the first edition of the HEad and neCK TumOR (HECKTOR) challenge. This challenge was held as a satellite event of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020, and was the first of its kind focusing on lesion segmentation in combined FDG-PET and CT image modalities. The challenge’s task is the automatic segmentation of the Gross Tumor Volume (GTV) of Head and Neck (H&N) oropharyngeal primary tumors in FDG-PET/CT images. To this end, the participants were given a training set of 201 cases from four different centers and their methods were tested on a held-out set of 53 cases from a fifth center. The methods were ranked according to the Dice Score Coefficient (DSC) averaged across all test cases. An additional inter-observer agreement study was organized to assess the difficulty of the task from a human perspective. 64 teams registered to the challenge, among which 10 provided a paper detailing their approach. The best method obtained an average DSC of 0.7591, showing a large improvement over our proposed baseline method and the inter-observer agreement, associated with DSCs of 0.6610 and 0.61, respectively. The automatic methods proved to successfully leverage the wealth of metabolic and structural properties of combined PET and CT modalities, significantly outperforming human inter-observer agreement level, semi-automatic thresholding based on PET images as well as other single modality-based methods. This promising performance is one step forward towards large-scale radiomics studies in H&N cancer, obviating the need for error-prone and time-consuming manual delineation of GTVs.     

SSA-Net: Spatial self-attention network for COVID-19 pneumonia infection segmentation with semi-supervised few-shot learning

Coronavirus disease (COVID-19) broke out at the end of 2019, and has resulted in an ongoing global pandemic. Segmentation of pneumonia infections from chest computed tomography (CT) scans of COVID-19 patients is significant for accurate diagnosis and quantitative analysis. Deep learning-based methods can be developed for automatic segmentation and offer a great potential to strengthen timely quarantine and medical treatment. Unfortunately, due to the urgent nature of the COVID-19 pandemic, a systematic collection of CT data sets for deep neural network training is quite difficult, especially high-quality annotations of multi-category infections are limited. In addition, it is still a challenge to segment the infected areas from CT slices because of the irregular shapes and fuzzy boundaries. To solve these issues, we propose a novel COVID-19 pneumonia lesion segmentation network, called Spatial Self-Attention network (SSA-Net), to identify infected regions from chest CT images automatically. In our SSA-Net, a self-attention mechanism is utilized to expand the receptive field and enhance the representation learning by distilling useful contextual information from deeper layers without extra training time, and spatial convolution is introduced to strengthen the network and accelerate the training convergence. Furthermore, to alleviate the insufficiency of labeled multi-class data and the long-tailed distribution of training data, we present a semi-supervised few-shot iterative segmentation framework based on re-weighting the loss and selecting prediction values with high confidence, which can accurately classify different kinds of infections with a small number of labeled image data. Experimental results show that SSA-Net outperforms state-of-the-art medical image segmentation networks and provides clinically interpretable saliency maps, which are useful for COVID-19 diagnosis and patient triage. Meanwhile, our semi-supervised iterative segmentation model can improve the learning ability in small and unbalanced training set and can achieve higher performance.     

CMC-Net: 3D calf muscle compartment segmentation with sparse annotation

Accurate 3D segmentation of calf muscle compartments in volumetric MR images is essential to diagnose as well as assess progression of muscular diseases. Recently, good segmentation performance was achieved using state-of-the-art deep learning approaches, which, however, require large amounts of annotated data for training. Considering that obtaining sufficiently large medical image annotation datasets is often difficult, time-consuming, and requires expert knowledge, minimizing the necessary sizes of expert-annotated training datasets is of great importance. This paper reports CMC-Net, a new deep learning framework for calf muscle compartment segmentation in 3D MR images that selects an effective small subset of 2D slices from the 3D images to be labelled, while also utilizing unannotated slices to facilitate proper generalization of the subsequent training steps. Our model consists of three parts: (1) an unsupervised method to select the most representative 2D slices on which expert annotation is performed; (2) ensemble model training employing these annotated as well as additional unannotated 2D slices; (3) a model-tuning method using pseudo-labels generated by the ensemble model that results in a trained deep network capable of accurate 3D segmentations. Experiments on segmentation of calf muscle compartments in 3D MR images show that our new approach achieves good performance with very small annotation ratios, and when utilizing full annotation, it outperforms state-of-the-art full annotation segmentation methods. Additional experiments on a 3D MR thigh dataset further verify the ability of our method in segmenting leg muscle groups with sparse annotation.     

Cross-dimensional transfer learning in medical image segmentation with deep learning

Over the last decade, convolutional neural networks have emerged and advanced the state-of-the-art in various image analysis and computer vision applications. The performance of 2D image classification networks is constantly improving and being trained on databases made of millions of natural images. Conversely, in the field of medical image analysis, the progress is also remarkable but has mainly slowed down due to the relative lack of annotated data and besides, the inherent constraints related to the acquisition process. These limitations are even more pronounced given the volumetry of medical imaging data. In this paper, we introduce an efficient way to transfer the efficiency of a 2D classification network trained on natural images to 2D, 3D uni- and multi-modal medical image segmentation applications. In this direction, we designed novel architectures based on two key principles: weight transfer by embedding a 2D pre-trained encoder into a higher dimensional U-Net, and dimensional transfer by expanding a 2D segmentation network into a higher dimension one. The proposed networks were tested on benchmarks comprising different modalities: MR, CT, and ultrasound images. Our 2D network ranked first on the CAMUS challenge dedicated to echo-cardiographic data segmentation and surpassed the state-of-the-art. Regarding 2D/3D MR and CT abdominal images from the CHAOS challenge, our approach largely outperformed the other 2D-based methods described in the challenge paper on Dice, RAVD, ASSD, and MSSD scores and ranked third on the online evaluation platform. Our 3D network applied to the BraTS 2022 competition also achieved promising results, reaching an average Dice score of 91.69% (91.22%) for the whole tumor, 83.23% (84.77%) for the tumor core and 81.75% (83.88%) for enhanced tumor using the approach based on weight (dimensional) transfer. Experimental and qualitative results illustrate the effectiveness of our methods for multi-dimensional medical image segmentation.     

Hippocampus segmentation in MR images using atlas registration, voxel classification, and graph cuts

Since hippocampal volume has been found to be an early biomarker for Alzheimer's disease, there is large interest in automated methods to accurately, robustly, and reproducibly extract the hippocampus from MRI data. In this work we present a segmentation method based on the minimization of an energy functional with intensity and prior terms, which are derived from manually labelled training images. The intensity energy is based on a statistical intensity model that is learned from the training images. The prior energy consists of a spatial and regularity term. The spatial prior is obtained from a probabilistic atlas created by registering the training images to the unlabelled target image, and deforming and averaging the training labels. The regularity prior energy encourages smooth segmentations. The resulting energy functional is globally minimized using graph cuts. The method was evaluated using image data from a population-based study on diseases among the elderly. Two set of images were used: a small set of 20 manually labelled MR images and a larger set of 498 images, for which manual volume measurements were available, but no segmentations. This data was previously used in a volumetry study that found significant associations between hippocampal volume and cognitive decline and incidence of dementia. Cross-validation experiments with the labelled set showed similarity indices of 0.852 and 0.864 and mean surface distances of 0.40 and 0.36 mm for the left and right hippocampus. 83% of the automated segmentations of the large set were rated as ‘good’ by a trained observer. Also, the proposed method was used to repeat the manual hippocampal volumetry study. The automatically obtained hippocampal volumes showed significant associations with cognitive decline and dementia, similar to the manually measured volumes. Finally, direct quantitative and qualitative comparisons showed that the proposed method outperforms a multi-atlas based segmentation method.     

Segmentation of image ensembles via latent atlases

Spatial priors, such as probabilistic atlases, play an important role in MRI segmentation. However, the availability of comprehensive, reliable and suitable manual segmentations for atlas construction is limited. We therefore propose a method for joint segmentation of corresponding regions of interest in a collection of aligned images that does not require labeled training data. Instead, a latent atlas, initialized by at most a single manual segmentation, is inferred from the evolving segmentations of the ensemble. The algorithm is based on probabilistic principles but is solved using partial differential equations (PDEs) and energy minimization criteria. We evaluate the method on two datasets, segmenting subcortical and cortical structures in a multi-subject study and extracting brain tumors in a single-subject multi-modal longitudinal experiment. We compare the segmentation results to manual segmentations, when those exist, and to the results of a state-of-the-art atlas-based segmentation method. The quality of the results supports the latent atlas as a promising alternative when existing atlases are not compatible with the images to be segmented.     

Segmentation of 4D cardiac MR images using a probabilistic atlas and the EM algorithm

In this paper an automatic atlas-based segmentation algorithm for 4D cardiac MR images is proposed. The algorithm is based on the 4D extension of the expectation maximisation (EM) algorithm. The EM algorithm uses a 4D probabilistic cardiac atlas to estimate the initial model parameters and to integrate a priori information into the classification process. The probabilistic cardiac atlas has been constructed from the manual segmentations of 3D cardiac image sequences of 14 healthy volunteers. It provides space and time-varying probability maps for the left and right ventricles, the myocardium, and background structures such as the liver, stomach, lungs and skin. In addition to using the probabilistic cardiac atlas as a priori information, the segmentation algorithm incorporates spatial and temporal contextual information by using 4D Markov Random Fields. After the classification, the largest connected component of each structure is extracted using a global connectivity filter which improves the results significantly, especially for the myocardium. Validation against manual segmentations and computation of the correlation between manual and automatic segmentation on 249 3D volumes were calculated. We used the 'leave one out' test where the image set to be segmented was not used in the construction of its corresponding atlas. Results show that the procedure can successfully segment the left ventricle (LV) (r = 0.96), myocardium (r = 0.92) and right ventricle (r = 0.92). In addition, 4D images from 10 patients with hypertrophic cardiomyopathy were also manually and automatically segmented yielding a good correlation in the volumes of the LV (r = 0.93) and myocardium (0.94) when the atlas constructed with volunteers is blurred.     

Deep Learning-Based Segmentation of Nodules in Thyroid Ultrasound: Improving Performance by Utilizing Markers Present in the Images

Computer-aided segmentation of thyroid nodules in ultrasound imaging could assist in their accurate characterization. In this study, using data for 1278 nodules, we proposed and evaluated two methods for deep learning-based segmentation of thyroid nodules that utilize calipers present in the images. The first method used approximate nodule masks generated based on the calipers. The second method combined manual annotations with automatic guidance by the calipers. When only approximate nodule masks were used for training, the achieved Dice similarity coefficient (DSC) was 85.1%. The performance of a network trained using manual annotations was DSC = 90.4%. When the guidance by the calipers was added, the performance increased to DSC = 93.1%. An increase in the number of cases used for training resulted in increased performance for all methods. The proposed method utilizing the guidance by calipers matched the performance of the network that did not use it with a reduced number of manually annotated training cases.     

Validation of a fully automated 3D hippocampal segmentation method using subjects with Alzheimer's disease mild cognitive impairment, and elderly controls

We introduce a new method for brain MRI segmentation, called the auto context model (ACM), to segment the hippocampus automatically in 3D T1-weighted structural brain MRI scans of subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI). In a training phase, our algorithm used 21 hand-labeled segmentations to learn a classification rule for hippocampal versus non-hippocampal regions using a modified AdaBoost method, based on approximately 18,000 features (image intensity, position, image curvatures, image gradients, tissue classification maps of gray/white matter and CSF, and mean, standard deviation, and Haar filters of size 1x1x1 to 7x7x7). We linearly registered all brains to a standard template to devise a basic shape prior to capture the global shape of the hippocampus, defined as the pointwise summation of all the training masks. We also included curvature, gradient, mean, standard deviation, and Haar filters of the shape prior and the tissue classified images as features. During each iteration of ACM - our extension of AdaBoost - the Bayesian posterior distribution of the labeling was fed back in as an input, along with its neighborhood features as new features for AdaBoost to use. In validation studies, we compared our results with hand-labeled segmentations by two experts. Using a leave-one-out approach and standard overlap and distance error metrics, our automated segmentations agreed well with human raters; any differences were comparable to differences between trained human raters. Our error metrics compare favorably with those previously reported for other automated hippocampal segmentations, suggesting the utility of the approach for large-scale studies.     

Segmentation evaluation with sparse ground truth data: Simulating true segmentations as perfect/imperfect as those generated by humans

Fully annotated data sets play important roles in medical image segmentation and evaluation. Expense and imprecision are the two main issues in generating ground truth (GT) segmentations. In this paper, in an attempt to overcome these two issues jointly, we propose a method, named SparseGT, which exploit variability among human segmenters to maximally save manual workload in GT generation for evaluating actual segmentations by algorithms. Pseudo ground truth (p-GT) segmentations are created by only a small fraction of workload and with human-level perfection/imperfection, and they can be used in practice as a substitute for fully manual GT in evaluating segmentation algorithms at the same precision.p-GT segmentations are generated by first selecting slices sparsely, where manual contouring is conducted only on these sparse slices, and subsequently filling segmentations on other slices automatically. By creating p-GT with different levels of sparseness, we determine the largest workload reduction achievable for each considered object, where the variability of the generated p-GT is statistically indistinguishable from inter-segmenter differences in full manual GT segmentations for that object. Furthermore, we investigate the segmentation evaluation errors introduced by variability in manual GT by applying p-GT in evaluation of actual segmentations by an algorithm.Experiments are conducted on ∼500 computed tomography (CT) studies involving six objects in two body regions, Head & Neck and Thorax, where optimal sparseness and corresponding evaluation errors are determined for each object and each strategy. Our results indicate that creating p-GT by the concatenated strategy of uniformly selecting sparse slices and filling segmentations via deep-learning (DL) network show highest manual workload reduction by ∼80-96% without sacrificing evaluation accuracy compared to fully manual GT. Nevertheless, other strategies also have obvious contributions in different situations. A non-uniform strategy for slice selection shows its advantage for objects with irregular shape change from slice to slice. An interpolation strategy for filling segmentations can achieve ∼60-90% of workload reduction in simulating human-level GT without the need of an actual training stage and shows potential in enlarging data sets for training p-GT generation networks. We conclude that not only over 90% reduction in workload is feasible without sacrificing evaluation accuracy but also the suitable strategy and the optimal sparseness level achievable for creating p-GT are object- and application-specific.     

Automatic Shape-Based Level Set Segmentation for Needle Tracking in 3-D TRUS-Guided Prostate Brachytherapy

Prostate brachytherapy is an effective treatment for early prostate cancer. The success depends critically on the correct needle implant positions. We have devised an automatic shape-based level set segmentation tool for needle tracking in 3-D transrectal ultrasound (TRUS) images, which uses the shape information and level set technique to localize the needle position and estimate the endpoint of needle in real-time. The 3-D TRUS images used in the evaluation of our tools were obtained using a 2-D TRUS transducer from Ultrasonix (Richmond, BC, Canada) and a computer-controlled stepper motor system from Thorlabs (Newton, NJ, USA). The accuracy and feedback mechanism had been validated using prostate phantoms and compared with 3-D positions of these needles derived from experts' readings. The experts' segmentation of needles from 3-D computed tomography images was the ground truth in this study. The difference between automatic and expert segmentations are within 0.1 mm for 17 of 19 implanted needles. The mean errors of automatic segmentations by comparing with the ground truth are within 0.25 mm. Our automated method allows real-time TRUS-based needle placement difference within one pixel compared with manual expert segementation.     

Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation

Fine renal artery segmentation on abdominal CT angiography (CTA) image is one of the most important tasks for kidney disease diagnosis and pre-operative planning. It will help clinicians locate each interlobar artery’s blood-feeding region via providing the complete 3D renal artery tree masks. However, it is still a task of great challenges due to the large intra-scale changes, large inter-anatomy variation, thin structures, small volume ratio and small labeled dataset of the fine renal artery. In this paper, we propose the first semi-supervised 3D fine renal artery segmentation framework, DPA-DenseBiasNet, which combines deep prior anatomy (DPA), dense biased network (DenseBiasNet) and hard region adaptation loss (HRA): 1) Based on our proposed dense biased connection, the DenseBiasNet fuses multi-receptive field and multi-resolution feature maps for large intra-scale changes. This dense biased connection also obtains a dense information flow and dense gradient flow so that the training is accelerated and the accuracy is enhanced. 2) DPA features extracted from an autoencoder (AE) are embedded in DenseBiasNet to cope with the challenge of large inter-anatomy variation and thin structures. The AE is pre-trained (unsupervised) by numerous unlabeled data to achieve the representation ability of anatomy features and these features are embedded in DenseBiasNet. This process will not introduce incorrect labels as optimization targets and thus contributes to a stable semi-supervised training strategy that is suitable for sensitive thin structures. 3) The HRA selects the loss value calculation region dynamically according to the segmentation quality so the network will pay attention to the hard regions in the training process and keep the class balanced.Experiments demonstrated that DPA-DenseBiasNet had high predictive accuracy and generalization with the Dice coefficient of 0.884 which increased by 0.083 compared with 3D U-Net (Çiçek et al., 2016). This revealed our framework with great potential for the 3D fine renal artery segmentation in clinical practice.