Automatic Segmentation and Probe Guidance for Real-Time Assistance of Ultrasound-Guided Femoral Nerve Blocks

Ultrasound-guided regional anesthesia can be challenging, especially for inexperienced physicians. The goal of the proposed methods is to create a system that can assist a user in performing ultrasound-guided femoral nerve blocks. The system indicates in which direction the user should move the ultrasound probe to investigate the region of interest and to reach the target site for needle insertion. Additionally, the system provides automatic real-time segmentation of the femoral artery, the femoral nerve and the two layers fascia lata and fascia iliaca. This aids in interpretation of the 2-D ultrasound images and the surrounding anatomy in 3-D. The system was evaluated on 24 ultrasound acquisitions of both legs from six subjects. The estimated target site for needle insertion and the segmentations were compared with those of an expert anesthesiologist. Average target distance was 8.5 mm with a standard deviation of 2.5 mm. The mean absolute differences of the femoral nerve and the fascia segmentations were about 1–3 mm.     

Phase Grouping-Based Needle Segmentation in 3-D Trans-rectal Ultrasound-Guided Prostate Trans-perineal Therapy

A robust and efficient needle segmentation method used to localize and track the needle in 3-D trans-rectal ultrasound (TRUS)-guided prostate therapy is proposed. The algorithmic procedure begins by cropping the 3-D US image containing a needle; then all voxels in the cropped 3-D image are grouped into different line support regions (LSRs) based on the outer product of the adjacent voxels' gradient vector. Two different needle axis extraction methods in the candidate LSR are presented: least-squares fitting and 3-D randomized Hough transform. Subsequent local optimization refines the position of the needle axis. Finally, the needle endpoint is localized by finding an intensity drop along the needle axis. The proposed methods were validated with 3-D TRUS tissue-mimicking agar phantom images, chicken breast phantom images and patient images obtained during prostate cryotherapy. The results of the in vivo test indicate that our method can localize the needle accurately and robustly with a needle endpoint localization accuracy <1.43 mm and detection accuracy >84%, which are favorable for 3-D TRUS-guided prostate trans-perineal therapy.     

Prostate segmentation in transrectal ultrasound using magnetic resonance imaging priors

Purpose

In the current standard of care, real-time transrectal ultrasound (TRUS) is commonly used for prostate brachytherapy guidance. As TRUS provides limited soft tissue contrast, segmenting the prostate gland in TRUS images is often challenging and subject to inter-observer and intra-observer variability, especially at the base and apex where the gland boundary is hard to define. Magnetic resonance imaging (MRI) has higher soft tissue contrast allowing the prostate to be contoured easily. In this paper, we aim to show that prostate segmentation in TRUS images informed by MRI priors can improve on prostate segmentation that relies only on TRUS images.

Methods

First, we compare the TRUS-based prostate segmentation used in the treatment of 598 patients with a high-quality MRI prostate atlas and observe inconsistencies at the apex and base. Second, motivated by this finding, we propose an alternative TRUS segmentation technique that is fully automatic and uses MRI priors. The algorithm uses a convolutional neural network to segment the prostate in TRUS images at mid-gland, where the gland boundary can be clearly seen. It then reconstructs the gland boundary at the apex and base with the aid of a statistical shape model built from an MRI atlas of 78 patients.

Results

Compared to the clinical TRUS segmentation, our method achieves similar mid-gland segmentation results in the 598-patient database. For the seven patients who had both TRUS and MRI, our method achieved more accurate segmentation of the base and apex with the MRI segmentation used as ground truth.

Conclusion

Our results suggest that utilizing MRI priors in TRUS prostate segmentation could potentially improve the performance at base and apex.

     

Segmentation of iodine brachytherapy implants in fluoroscopy

Purpose

In prostate brachytherapy, intraoperative dosimetry would allow for evaluation of the implant quality while the patient is still in treatment position. Such a mechanism, however, requires 3-D visualization of the deposited seeds relative to the prostate. It follows that accurate and robust seed segmentation is of critical importance in achieving intraoperative dosimetry.

Methods

Implanted iodine brachytherapy seeds are segmented via a region-based implicit active contour model. Overlapping seed groups are then resolved using a template-based declustering technique.

Results

Ground truth seed coordinates were obtained through manual segmentation. A total of 57 clinical C-arm images from 10 patients were used to validate the proposed algorithm. This resulted in two failed images and a 96.0% automatic detection rate with a corresponding 2.2% false-positive rate in the remaining 55 images. The mean centroid error between the manual and automatic segmentations was 1.2 pixels.

Conclusions

Robust and accurate iodine seed segmentation can be achieved through the proposed segmentation workflow.     

On evaluating brain tissue classifiers without a ground truth

In this paper, we present a set of techniques for the evaluation of brain tissue classifiers on a large data set of MR images of the head. Due to the difficulty of establishing a gold standard for this type of data, we focus our attention on methods which do not require a ground truth, but instead rely on a common agreement principle. Three different techniques are presented: the Williams' index, a measure of common agreement; STAPLE, an Expectation Maximization algorithm which simultaneously estimates performance parameters and constructs an estimated reference standard; and Multidimensional Scaling, a visualization technique to explore similarity data. We apply these different evaluation methodologies to a set of eleven different segmentation algorithms on forty MR images. We then validate our evaluation pipeline by building a ground truth based on human expert tracings. The evaluations with and without a ground truth are compared. Our findings show that comparing classifiers without a gold standard can provide a lot of interesting information. In particular, outliers can be easily detected, strongly consistent or highly variable techniques can be readily discriminated, and the overall similarity between different techniques can be assessed. On the other hand, we also find that some information present in the expert segmentations is not captured by the automatic classifiers, suggesting that common agreement alone may not be sufficient for a precise performance evaluation of brain tissue classifiers.     

The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge

There is a large body of literature linking anatomic and geometric characteristics of kidney tumors to perioperative and oncologic outcomes. Semantic segmentation of these tumors and their host kidneys is a promising tool for quantitatively characterizing these lesions, but its adoption is limited due to the manual effort required to produce high-quality 3D segmentations of these structures. Recently, methods based on deep learning have shown excellent results in automatic 3D segmentation, but they require large datasets for training, and there remains little consensus on which methods perform best. The 2019 Kidney and Kidney Tumor Segmentation challenge (KiTS19) was a competition held in conjunction with the 2019 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) which sought to address these issues and stimulate progress on this automatic segmentation problem. A training set of 210 cross sectional CT images with kidney tumors was publicly released with corresponding semantic segmentation masks. 106 teams from five continents used this data to develop automated systems to predict the true segmentation masks on a test set of 90 CT images for which the corresponding ground truth segmentations were kept private. These predictions were scored and ranked according to their average Sørensen-Dice coefficient between the kidney and tumor across all 90 cases. The winning team achieved a Dice of 0.974 for kidney and 0.851 for tumor, approaching the inter-annotator performance on kidney (0.983) but falling short on tumor (0.923). This challenge has now entered an “open leaderboard” phase where it serves as a challenging benchmark in 3D semantic segmentation.     

A 5-MHz cylindrical dual-layer transducer array for 3-D transrectal ultrasound imaging

Two-dimensional transrectal ultrasound (TRUS) is being used in guiding prostate biopsies and treatments. In many cases, the TRUS probes are moved manually or mechanically to acquire volumetric information, making the imaging slow, user dependent, and unreliable. A real-time three-dimensional (3-D) TRUS system could improve reliability and volume rates of imaging during these procedures. In this article, the authors present a 5-MHz cylindrical dual-layer transducer array capable of real-time 3-D transrectal ultrasound without any mechanically moving parts. Compared with fully sampled 2-D arrays, this design substantially reduces the channel count and fabrication complexity. This dual-layer transducer uses PZT elements for transmit and P[VDF-TrFE] copolymer elements for receive, respectively. The mechanical flexibility of both diced PZT and copolymer makes it practical for transrectal applications. Full synthetic aperture 3-D data sets were acquired by interfacing the transducer with a Verasonics Data Acquisition System. Offline 3-D beamforming was then performed to obtain volumes of two wire phantoms and a cyst phantom. Generalized coherence factor was applied to improve the contrast of images. The measured -6-dB fractional bandwidth of the transducer was 62% with a center frequency of 5.66 MHz. The measured lateral beamwidths were 1.28 mm and 0.91 mm in transverse and longitudinal directions, respectively, compared with a simulated beamwidth of 0.92 mm and 0.74 mm.     

Needle deflection estimation: prostate brachytherapy phantom experiments

Purpose

The performance of a fusion-based needle deflection estimation method was experimentally evaluated using prostate brachytherapy phantoms. The accuracy of the needle deflection estimation was determined. The robustness of the approach with variations in needle insertion speed and soft tissue biomechanical properties was investigated.

Methods

A needle deflection estimation method was developed to determine the amount of needle bending during insertion into deformable tissue by combining a kinematic deflection model with measurements taken from two electromagnetic trackers placed at the tip and the base of the needle. Experimental verification of this method for use in prostate brachytherapy needle insertion procedures was performed. A total of 21 beveled tip, 18 ga, 200 mm needles were manually inserted at various speeds through a template and toward different targets distributed within 3 soft tissue mimicking polyvinyl chloride prostate phantoms of varying stiffness. The tracked positions of both the needle tip and base were recorded, and Kalman filters were applied to fuse the sensory information. The estimation results were validated using ground truth obtained from fluoroscopy images.

Results

The manual insertion speed ranged from 8 to 34 mm/s, needle deflection ranged from 5 to 8 mm at an insertion depth of 76 mm, and the elastic modulus of the soft tissue ranged from 50 to 150 kPa. The accuracy and robustness of the estimation method were verified within these ranges. When compared to purely model-based estimation, we observed a reduction in needle tip position estimation error by \(52\pm 17\)  % (mean  \(\pm \)  SD) and the cumulative deflection error by \(57\pm 19\)  %.

Conclusions

Fusion of electromagnetic sensors demonstrated significant improvement in estimating needle deflection compared to model-based methods. The method has potential clinical applicability in the guidance of needle placement medical interventions, particularly prostate brachytherapy.     

Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver

2D ultrasound (US) image guidance is used in minimally invasive procedures in the liver to visualize the target and the needle. Needle insertion using 2D ultrasound keeping the transducer position to view needle and reach target is challenging. Dedicated needle holders attached to the US transducer help to target in plane and at a specific angle. A drawback of this is that, the probe is fixed to the needle and cannot be rotated to assess the position of the needle in a perpendicular plane. In this study, we propose an automatic needle detection and tracking method using 3D US imaging to improve image guidance and visualization of the target in the liver with respect to the needle during these interventional procedures. The method utilizes a convolutional neural network for detection of the needle in 3D US images. In a subsequent step, the output of the convolutional neural network is used to detect needle candidates, which are fed into a final tracking step to determine the real needle position. The needle position is used to present two perpendicular cross-sectional planes of the 3D US image containing the needle in both directions. Performance of the method was evaluated in phantoms and in-vivo data by calculating the needle position distance and needle orientation angle between segmented needles and reference ground truth needles, which were manually annotated by an observer. The method successfully detects the needle position and orientation with mean errors of 1 mm and 2°, respectively. The proposed method yields a robust automatic needle detection and visualization at a frame rate of 3 Hz in 3D ultrasound imaging of the liver.     

Statistical shape and texture model of quadrature phase information for prostate segmentation

Purpose  

Prostate volume estimation from segmentation of transrectal ultrasound (TRUS) images aids in diagnosis and treatment of prostate hypertrophy and cancer. Computer-aided accurate and computationally efficient prostate segmentation in TRUS images is a challenging task, owing to low signal-to-noise ratio, speckle noise, calcifications, and heterogeneous intensity distribution in the prostate region.     

Learning Non-rigid Deformations for Robust,Constrained Point-based Registration in Image-Guided MR-TRUS Prostate Intervention

Accurate and robust non-rigid registration of pre-procedure magnetic resonance (MR) imaging to intra-procedure trans-rectal ultrasound (TRUS) is critical for image-guided biopsies of prostate cancer. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. TRUS-guided biopsy is the current clinical standard for prostate cancer diagnosis and assessment. State-of-the-art, clinical MR-TRUS image fusion relies upon semi-automated segmentations of the prostate in both the MR and the TRUS images to perform non-rigid surface-based registration of the gland. Segmentation of the prostate in TRUS imaging is itself a challenging task and prone to high variability. These segmentation errors can lead to poor registration and subsequently poor localization of biopsy targets, which may result in false-negative cancer detection. In this paper, we present a non-rigid surface registration approach to MR-TRUS fusion based on a statistical deformation model (SDM) of intra-procedural deformations derived from clinical training data. Synthetic validation experiments quantifying registration volume of interest overlaps of the PI-RADS parcellation standard and tests using clinical landmark data demonstrate that our use of an SDM for registration, with median target registration error of 2.98 mm, is significantly more accurate than the current clinical method. Furthermore, we show that the low-dimensional SDM registration results are robust to segmentation errors that are not uncommon in clinical TRUS data.     

3D prostate model formation from non-parallel 2D ultrasound biopsy images

Biopsy of the prostate using 2D transrectal ultrasound (TRUS) guidance is the current gold standard for diagnosis of prostate cancer; however, the current procedure is limited by using 2D biopsy tools to target 3D biopsy locations. We propose a technique for patient-specific 3D prostate model reconstruction from a sparse collection of non-parallel 2D TRUS biopsy images. Our method conforms to the restrictions of current TRUS biopsy equipment and could be efficiently incorporated into current clinical biopsy procedures for needle guidance without the need for expensive hardware additions. In this paper, the model reconstruction technique is evaluated using simulated biopsy images from 3D TRUS prostate images of 10 biopsy patients. All reconstructed models are compared to their corresponding 3D manually segmented prostate models for evaluation of prostate volume accuracy and surface errors (both regional and global). The number of 2D TRUS biopsy images used for prostate modeling was varied to determine the optimal number of images necessary for accurate prostate surface estimation.     

Automated aorta segmentation in low-dose chest CT images

Purpose

   Abnormalities of aortic surface and aortic diameter can be related to cardiovascular disease and aortic aneurysm. Computer-based aortic segmentation and measurement may aid physicians in related disease diagnosis. This paper presents a fully automated algorithm for aorta segmentation in low-dose non-contrast CT images.

Methods

   The original non-contrast CT scan images as well as their pre-computed anatomy label maps are used to locate the aorta and identify its surface. First a seed point is located inside the aortic lumen. Then, a cylindrical model is progressively fitted to the 3D image space to track the aorta centerline. Finally, the aortic surface is located based on image intensity information. This algorithm has been trained and tested on 359 low-dose non-contrast CT images from VIA-ELCAP and LIDC public image databases. Twenty images were used for training to obtain the optimal set of parameters, while the remaining images were used for testing. The segmentation result has been evaluated both qualitatively and quantitatively. Sixty representative testing images were used to establish a partial ground truth by manual marking on several axial image slices.

Results

   Compared to ground truth marking, the segmentation result had a mean Dice Similarity Coefficient of 0.933 (maximum 0.963 and minimum 0.907). The average boundary distance between manual segmentation and automatic segmentation was 1.39 mm with a maximum of 1.79 mm and a minimum of 0.83 mm.

Conclusion

   Both qualitative and quantitative evaluations have shown that the presented algorithm is able to accurately segment the aorta in low-dose non-contrast CT images.     

Case study: an evaluation of user-assisted hierarchical watershed segmentation

This paper evaluates the effectiveness of an interactive, three-dimensional image segmentation technique that relies on watersheds. This paper presents two user-based case studies, which include two different groups of domain experts. Subjects manipulate a graphics-based front end to a hierarchy of segmented regions generated from a watershed segmentation algorithm, which is implemented in the Insight Toolkit. In the first study, medical students segment several different anatomical structures from the Visible Human Female head and neck color cryosection data. In the second study, radiologists use the interactive tool to produce models of brain tumors from MRI data. This paper presents a quantitative and qualitative comparison against hand contouring. To quantify accuracy, we estimate ground truth from the hand-contouring data using the Simultaneous Truth and Performance Estimation algorithm. We also apply metrics from the literature to estimate precision and efficiency. The watershed segmentation technique showed improved subject interaction times and increased inter-subject precision over hand contouring, with quality that is visually and statistically comparable. The analysis also identifies some failures in the watershed technique, where edges were poorly defined in the data, and note a trend in the hand-contouring results toward systematically larger segmentations, which raises questions about the wisdom of using expert segmentations to define ground truth.     

A geometric flow for segmenting vasculature in proton-density weighted MRI

Modern neurosurgery takes advantage of magnetic resonance images (MRI) of a patient’s cerebral anatomy and vasculature for planning before surgery and guidance during the procedure. Dual echo acquisitions are often performed that yield proton-density (PD) and T2-weighted images to evaluate edema near a tumor or lesion. In this paper we develop a novel geometric flow for segmenting vasculature in PD images, which can also be applied to the easier cases of MR angiography data or Gadolinium enhanced MRI. Obtaining vasculature from PD data is of clinical interest since the acquisition of such images is widespread, the scanning process is non-invasive, and the availability of vessel segmentation methods could obviate the need for an additional angiographic or contrast-based sequence during preoperative imaging. The key idea is to first apply Frangi’s vesselness measure [Frangi, A., Niessen, W., Vincken, K.L., Viergever, M.A., 1998. Multiscale vessel enhancement filtering. In: International Conference on Medical Image Computing and Computer Assisted Intervention, vol. 1496 of Lecture Notes in Computer Science, pp. 130–137] to find putative centerlines of tubular structures along with their estimated radii. This measure is then distributed to create a vector field which allows the flux maximizing flow algorithm of Vasilevskiy and Siddiqi [Vasilevskiy, A., Siddiqi, K., 2002. Flux maximizing geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (12), 1565–1578] to be applied to recover vessel boundaries. We carry out a qualitative validation of the approach on PD, MR angiography and Gadolinium enhanced MRI volumes and suggest a new way to visualize the segmentations in 2D with masked projections. We validate the approach quantitatively on a single-subject data set consisting of PD, phase contrast (PC) angiography and time of flight (TOF) angiography volumes, with an expert segmented version of the TOF volume viewed as the ground truth. We then validate the approach quantitatively on 19 PD data sets from a new digital brain phantom, with semi-automatically obtained labels from the corresponding angiography volumes viewed as ground truth. A significant finding is that both for the single-subject and multi-subject studies, 90% or more of the vasculature in the ground truth segmentation is recovered from the automatic segmentation of the other volumes.     

Automated segmentation of retinal blood vessels in spectral domain optical coherence tomography scans

The correct segmentation of blood vessels in optical coherence tomography (OCT) images may be an important requirement for the analysis of intra-retinal layer thickness in human retinal diseases. We developed a shape model based procedure for the automatic segmentation of retinal blood vessels in spectral domain (SD)-OCT scans acquired with the Spectralis OCT system. The segmentation procedure is based on a statistical shape model that has been created through manual segmentation of vessels in a training phase. The actual segmentation procedure is performed after the approximate vessel position has been defined by a shadowgraph that assigns the lateral vessel positions. The active shape model method is subsequently used to segment blood vessel contours in axial direction. The automated segmentation results were validated against the manual segmentation of the same vessels by three expert readers. Manual and automated segmentations of 168 blood vessels from 34 B-scans were analyzed with respect to the deviations in the mean Euclidean distance and surface area. The mean Euclidean distance between the automatically and manually segmented contours (on average 4.0 pixels respectively 20 μm against all three experts) was within the range of the manually marked contours among the three readers (approximately 3.8 pixels respectively 18 μm for all experts). The area deviations between the automated and manual segmentation also lie within the range of the area deviations among the 3 clinical experts. Intra reader variability for the experts was between 0.9 and 0.94. We conclude that the automated segmentation approach is able to segment blood vessels with comparable accuracy as expert readers and will provide a useful tool in vessel analysis of whole C-scans, and in particular in multicenter trials.     

Acoustic Radiation Force Impulse (ARFI) imaging-based needle visualization

Ultrasound-guided needle placement is widely used in the clinical setting, particularly for central venous catheter placement, tissue biopsy and regional anesthesia. Difficulties with ultrasound guidance in these areas often result from steep needle insertion angles and spatial offsets between the imaging plane and the needle. Acoustic Radiation Force Impulse (ARFI) imaging leads to improved needle visualization because it uses a standard diagnostic scanner to perform radiation force based elasticity imaging, creating a displacement map that displays tissue stiffness variations. The needle visualization in ARFI images is independent of needle-insertion angle and also extends needle visibility out of plane. Although ARFI images portray needles well, they often do not contain the usual B-mode landmarks. Therefore, a three-step segmentation algorithm has been developed to identify a needle in an ARFI image and overlay the needle prediction on a coregistered B-mode image. The steps are: (1) contrast enhancement by median filtration and Laplacian operator filtration, (2) noise suppression through displacement estimate correlation coefficient thresholding and (3) smoothing by removal of outliers and best-fit line prediction. The algorithm was applied to data sets from horizontal 18, 21 and 25 gauge needles between 0-4 mm offset in elevation from the transducer imaging plane and to 18G needles on the transducer axis (in plane) between 10 degrees and 35 degrees from the horizontal. Needle tips were visualized within 2 mm of their actual position for both horizontal needle orientations up to 1.5 mm offset in elevation from the transducer imaging plane and on-axis angled needles between 10 degrees-35 degrees above the horizontal orientation. We conclude that segmented ARFI images overlaid on matched B-mode images hold promise for improved needle visibility in many clinical applications.     

Expert-driven label fusion in multi-atlas-based segmentation of the prostate using weighted atlases

Purpose

Automated segmentation is required for radiotherapy treatment planning, and multi-atlas methods are frequently used for this purpose. The combination of multiple intermediate results from multi-atlas segmentation into a single segmentation map can be achieved by label fusion. A method that includes expert knowledge in the label fusion phase of multi-atlas-based segmentation was developed. The method was tested by application to prostate segmentation, and the accuracy was compared to standard techniques.

Methods

The selective and iterative method for performance level estimation (SIMPLE) algorithm for label fusion was modified with a weight map given by an expert that indicates the importance of each region in the evaluation of segmentation results. Voxel-based weights specified by an expert when performing the label fusion step in atlas-based segmentation were introduced into the modified SIMPLE algorithm. These weights incorporate expert knowledge on accuracy requirements in different regions of a segmentation. Using this knowledge, segmentation accuracy in regions known to be important can be improved by sacrificing segmentation accuracy in less important regions. Contextual information such as the presence of vulnerable tissue is then used in the segmentation process. This method using weight maps to fine-tune the result of multi-atlas-based segmentation was tested using a set of 146 atlas images consisting of an MR image of the lower abdomen and a prostate segmentation. Each image served as a target in a set of leave-one-out experiments. These experiments were repeated for a weight map derived from the clinical practice in our hospital.

Results

The segmentation accuracy increased 6 % in regions that border vulnerable tissue using expert-specified voxel-based weight maps. This was achieved at the cost of a 4 % decrease in accuracy in less clinically relevant regions.

Conclusion

The inclusion of expert knowledge in a multi-atlas-based segmentation procedure was shown to be feasible for prostate segmentation. This method allows an expert to ensure that automatic segmentation is most accurate in critical regions. This improved local accuracy can increase the practical value of automatic segmentation.     

Three-dimensional ultrasound-guided core needle breast biopsy

A new core needle breast biopsy system guided by 3-D ultrasound (US) is proposed. Our device provides rapid imaging and real-time guidance, as well as breast stabilization and a needle guidance apparatus using 3-D imaging. The targeting accuracy of our device was tested by inserting a 14-gauge biopsy needle into agar phantoms under 3-D US guidance. A total of 18 0.8-mm stainless-steel beads embedded in the phantoms defined each of the four target positions tested. Positioning accuracy was calculated by comparing needle tip position to the preinsertion bead position, as measured by three observers three times each on 3-D US. The interobserver standard error of measurement was no more than 0.14 mm for the beads and 0.27 mm for the needle tips. A 3-D principal component analysis was performed to obtain the population distribution of needle tip position relative to the target beads for the four target positions. The 3-D 95% confidence intervals were found to have total widths ranging from 0.43 to 1.71 mm, depending on direction and bead position.