Delineating anatomical boundaries using the boundary fragment model

In this paper we present a method to automatically isolate relevant anatomical boundary positions in an image using only the structure of edges. The purpose of this method is to facilitate model-based segmentation algorithms which rely on accurate initialisation and assume that the correct anatomical boundary positions are close to the current model surface.The method is built around a weak parts-based shape model – the Boundary Fragment Model (BFM) – which represents an object by sections of its boundary. Following previous literature, we use the BFM in a boosted classifier framework to first automatically detect the object of interest. Extending previous work, we use the BFM to drive a classifier which isolates boundary candidates from spurious and irrelevant edge responses. The application of our algorithm leads to a labelled edge map which encodes the positions of (multiple) object boundaries.By way of illustrating what is a general solution, the task of identifying the endocardium and epicardium in three-dimensional ultrasound images is completely examined, including a detailed analysis of the parameters which impact on the model construction, the structure of the learned edge response classifier, and implementation concerns. For completeness, we also demonstrate how the output boundary positions can be used in a full model-based segmentation framework.     

Automatic initialization of an active shape model of the prostate

In this work is reported a new method for automatic segmentation of the boundary of the prostate, in transurethral ultrasound images. The scheme is based on a robust automatic initialization of an active shape model (ASM) of the prostate, which is subsequently fitted to the boundary of the gland. The initialization of the ASM is based on pixel classification to estimate the prostate region in an ultrasound image, followed by automatic adjustment – using a multipopulation genetic algorithm (MPGA) – of the initial pose of the ASM to the binary image produced by the classifier. The initial pose is next adjusted to the gray level ultrasound image, using the MPGA. After automatic initialization, the ASM is adjusted to the gray level ultrasound image to produce the final prostate contour. The method provides fast and robust segmentation of the prostate boundary. Validation results on 22 ultrasound images are reported with 1.74 mm of mean boundary error and an estimated processing time of 66 s per image. Our automatic initialization method can be applied with the ASMs of different organs in various imaging modalities.     

An adaptive snake model for ultrasound image segmentation: modified trimmed mean filter, ramp integration and adaptive weighting parameters

The snake model is a widely-used approach to finding the boundary of the object of interest in an ultrasound image. However, due to the speckles, the weak edges and the tissue-related textures in an ultrasound image, conventional snake models usually cannot obtain the desired boundary satisfactorily. In this paper, we propose a new adaptive snake model for ultrasound image segmentation. The proposed snake model is composed of three major techniques, namely, the modified trimmed mean (MTM) filtering, ramp integration and adaptive weighting parameters. With the advantages of the mean and median filters, the MTM filter is employed to alleviate the speckle interference in the segmentation process. The weak edge enhancement by ramp integration attempts to capture the slowly varying edges, which are hard to capture by conventional snake models. The adaptive weighting parameter allows weighting of each energy term to change adaptively during the deformation process. The proposed snake model has been verified on the phantom and clinical ultrasound images. The experimental results showed that the proposed snake model achieves a reasonable performance with an initial contour placed 10 to 20 pixels away from the desired boundary. The mean minimal distances from the derived boundary to the desired boundary have been shown to be less than 3.5 (for CNR > or = 0.5) and 2.5 pixels, respectively, for the phantom and ultrasound images.     

Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis

Ultrasound (US) imaging is widely used for anatomical structure inspection in clinical diagnosis. The training of new sonographers and deep learning based algorithms for US image analysis usually requires a large amount of data. However, obtaining and labeling large-scale US imaging data are not easy tasks, especially for diseases with low incidence. Realistic US image synthesis can alleviate this problem to a great extent. In this paper, we propose a generative adversarial network (GAN) based image synthesis framework. Our main contributions include: (1) we present the first work that can synthesize realistic B-mode US images with high-resolution and customized texture editing features; (2) to enhance structural details of generated images, we propose to introduce auxiliary sketch guidance into a conditional GAN. We superpose the edge sketch onto the object mask and use the composite mask as the network input; (3) to generate high-resolution US images, we adopt a progressive training strategy to gradually generate high-resolution images from low-resolution images. In addition, a feature loss is proposed to minimize the difference of high-level features between the generated and real images, which further improves the quality of generated images; (4) the proposed US image synthesis method is quite universal and can also be generalized to the US images of other anatomical structures besides the three ones tested in our study (lung, hip joint, and ovary); (5) extensive experiments on three large US image datasets are conducted to validate our method. Ablation studies, customized texture editing, user studies, and segmentation tests demonstrate promising results of our method in synthesizing realistic US images.     

A dual-snake model of high penetrability for ultrasound image boundary extraction

Most deformable models require the initial contour to be placed close to the boundary of the object of interest for boundary extraction of ultrasound (US) images, which is impractical in many clinical applications. To allow a distant initial contour, a new dual-snake model promising high penetrability through the interference of the noises is proposed in this paper. The proposed dual-snake model features a new far-reaching external force, called the discrete gradient flow, a connected component-weighted image force, and an effective stability evaluation of two underlying snakes. The experimental results show that, with a distant initial contour, the mean distance from the derived boundary to the desired boundary is less than 1.4 pixels, and most snake elements are within 2.7 pixels of the desired boundaries for the synthetic images with CNR ≥ 1. For the clinical US images, the mean distance is less than 1.9 pixels, and most snake elements are within 3 pixels of the desired boundaries. (E-mail: chung@lotus.mc.ntu.edu.tw)     

A 3D generalization of user-steered live-wire segmentation

We have been developing user-steered image segmentation methods for situations which require considerable human assistance in object definition. In the past, we have presented two paradigms, referred to as live-wire and live-lane, for segmenting 2D/3D/4D object boundaries in a slice-by-slice fashion, and demonstrated that live-wire and live-lane are more repeatable, with a statistical significance level of P < 0.03, and are 1.5-2.5 times faster, with a statistical significance level of P < 0.02, than manual tracing. In this paper, we introduce a 3D generalization of the live-wire approach for segmenting 3D/4D object boundaries which further reduces the time spent by the user in segmentation. In a 2D live-wire, given a slice, for two specified points (pixel vertices) on the boundary of the object, the best boundary segment is the minimum-cost path between the two points, described as a set of oriented pixel edges. This segment is found via Dijkstra's algorithm as the user anchors the first point and moves the cursor to indicate the second point. A complete 2D boundary is identified as a set of consecutive boundary segments forming a "closed", "connected", "oriented" contour. The strategy of the 3D extension is that, first, users specify contours via live-wiring on a few slices that are orthogonal to the natural slices of the original scene. If these slices are selected strategically, then we have a sufficient number of points on the 3D boundary of the object to subsequently trace optimum boundary segments automatically in all natural slices of the 3D scene. A 3D object boundary may define multiple 2D boundaries per slice. The points on each 2D boundary form an ordered set such that when the best boundary segment is computed between each pair of consecutive points, a closed, connected, oriented boundary results. The ordered set of points on each 2D boundary is found from the way the users select the orthogonal slices. Based on several validation studies involving segmentation of the bones of the foot in MR images, we found that the 3D extension of live-wire is more repeatable, with a statistical significance level of P < 0.0001, and 2-6 times faster, with a statistical significance level of P < 0.01, than the 2D live-wire method, and 3-15 times faster than manual tracing.     

Texture feature analysis for breast ultrasound image enhancement

Texture analysis of breast ultrasound B-scans has been widely applied to the segmentation and classification of breast tumors. We present a parametric imaging method based on the texture features to preserve tumor edges and retain the texture information simultaneously. Four texture-feature parameters--homogeneity, contrast, energy and variance--were evaluated using the gray-level co-occurrence matrix. The local texture-feature parameter was assigned as the new pixel located at the center of the sliding window at each position. This process yielded the texture-feature parametric image as the map of texture-feature values. The signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) were estimated to show the quality improvement of the images. The contours outlined from 11 experienced physicians and the gradient vector flow (GVF) snake algorithm segmentations were adopted to verify the edge enhancement of texture-feature parametric images. In addition, the Fisher's linear discriminant analysis (FLDA) and receiver-operating-characteristic (ROC) curve were used to test the performance of breast tumor classifications between texture-feature parametric images and B-scan images. The results show that the variance images have higher CNR and SNR estimates than those in the B-scan images. There was a high agreement between the physician's manual contours and the GVF snake automatic segmentations in the variance images, and the mean area overlap was over 93%. The area under the ROC curve from the B-scan images had 0.81 and 95% confidence interval of 0.72-0.88, and the texture-feature parametric images had 0.90 and 95% confidence interval of 0.84-0.96. These findings indicate that the texture-feature parametric imaging method can be not only useful for determining the location of the lesion boundary but also as a tool to improve the accuracy of breast tumor classifications.     

WorkStream4D直接冠状位图像重组技术在鼻窦CT检查中的应用

目的探讨螺旋CT横轴位扫描，4D时空成像概念（WorkStream4D）直接冠状住重组图像在鼻窦检查中的价值。方法对30例患者进行鼻窦区横轴位扫描，使用德国西门子SOMATOM Definition双源CT（Dual Source CT，DSCT），行鼻窦横轴位螺旋扫描，扫描采集层厚0．6mm×64，Pitch值0．9。按照两种方法重组冠状位图像：方法A，采用Work Stream 4D技术，由原始数据（raw data）直接重组得到冠状住图像：方法B将原始数据重建成层厚lmm，层间距0．5mm薄层横轴位图像，利用随机3D软件MPR（multiple planar reformat）功能得到冠状位重组图像。将每住患者的两组不同方法的冠状住图像进行比较，主要分析图像质量的差别及工作流程的差异。结果WorkStream4D直接冠状住重组图像与常规冠状住重组图像在鼻窦、软组织及骨组织的显示方面无差别，但WorkStream4D直接冠状住重组技术不需要横轴位薄层图像，减少了每次检查产生的图像数量。结论WorkStream4D是一种新的高效自动的数据工作流程，优化了CT成像过程的效率和灵活性，大大降低了人动操作的无效时间消耗，减少每次检查中所产生的数据，为多层螺旋CT（MSCT）出现后带来的庞大的数据流造成的瓶颈效应提供了较好的解决方案。     

Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs

A new computer aided detection (CAD) system is presented for the detection of pulmonary nodules on chest radiographs. Here, we present the details of the proposed algorithm and provide a performance analysis using a publicly available database to serve as a benchmark for future research efforts. All aspects of algorithm training were done using an independent dataset containing 167 chest radiographs with a total of 181 lung nodules. The publicly available test set was created by the Standard Digital Image Database Project Team of the Scientific Committee of the Japanese Society of Radiological Technology (JRST). The JRST dataset used here is comprised of 154 chest radiographs containing one radiologist confirmed nodule each (100 malignant cases, 54 benign cases). The CAD system uses an active shape model for anatomical segmentation. This is followed by a new weighted-multiscale convergence-index nodule candidate detector. A novel candidate segmentation algorithm is proposed that uses an adaptive distance-based threshold. A set of 114 features is computed for each candidate. A Fisher linear discriminant (FLD) classifier is used on a subset of 46 features to produce the final detections. Our results indicate that the system is able to detect 78.1% of the nodules in the JRST test set with and average of 4.0 false positives per image (excluding 14 cases containing lung nodules in retrocardiac and subdiaphragmatic regions of the lung).     

Standardization of three-dimensional images in obstetrics and gynecology: consensus statement.

Standardization of the display of ultrasound images has so far only been achieved in transabdominal two-dimensional (2D) sonography. In contrast, there is a lack of uniformity in the demonstration of transvaginal 2D ultrasound images. The described non-uniformity frequently leads to confusion in the assessment of an image, in particular with regard to the accurate anatomical assignment of left/right and dorsal/ventral. Three-dimensional (3D) sonography offers a unique opportunity to avoid this confusion in the interpretation of ultrasound images, because, independent of primary volume acquisition, the volume can always be rotated so that the stored object can at all times be visualized in a known anatomical position, rendering it of no importance whether the image acquired transvaginally is demonstrated from above or from below. This will also be important in allowing fusion of ultrasound image data with computed tomographic, magnetic resonance and/or positron emission tomography images. In this article we suggest that standardization of transabdominal and transvaginal 3D images does not only provide the inexperienced physician/sonographer with a guide to spatial orientation, but also serves to avoid erroneous topographical interpretations.     

Slice thickness measurements

A new slice thickness test object is discussed, with results of feasibility tests using 13 transducers from six different commercial scanners presented. Both inplane beamwidth and slice thickness measurements could be easily performed and interpreted. For proper equipment gain settings the slice thickness measurement results were found to be independent of receiver gain with a tissue-mimicking (TM) fill in the test object. This is explained by a simple model in which test object low-amplitude internal fill echoes effectively act as an invariant measurement reference level with respect to the slice thickness profile echoes. This model also predicts that in clinical images the inplane lateral resolution (and slice thickness) will be governed by the local image contrast between adjacent tissues as well as the beam profile. Slice thickness measurement results from various types of real-time transducers demonstrate their inherent slice thickness focal characteristics.     

An early vision-based snake model for ultrasound image segmentation

Due to the speckles and the ill-defined edges of the object of interest, the classic image-segmentation techniques are usually ineffective in segmenting ultrasound (US) images. In this paper, we present a new algorithm for segmenting general US images that is composed of two major techniques; namely, the early-vision model and the discrete-snake model. By simulating human early vision, the early-vision model can capture both grey-scale and textural edges while the speckle noise is suppressed. By performing deformation only on the peaks of the distance map, the discrete-snake model promises better noise immunity and more accurate convergence. Moreover, the constraint for most conventional snake models that the initial contour needs to be located very close to the actual boundary has been relaxed substantially. The performance of the proposed snake model has been shown to be comparable to manual delineation and superior to that of the gradient vector flow (GVF) snake model.     

A fully automatic and robust brain MRI tissue classification method

A novel, fully automatic, adaptive, robust procedure for brain tissue classification from 3D magnetic resonance head images (MRI) is described in this paper. The procedure is adaptive in that it customizes a training set, by using a 'pruning' strategy, such that the classification is robust against anatomical variability and pathology. Starting from a set of samples generated from prior tissue probability maps (a 'model') in a standard, brain-based coordinate system ('stereotaxic space'), the method first reduces the fraction of incorrectly labeled samples in this set by using a minimum spanning tree graph-theoretic approach. Then, the corrected set of samples is used by a supervised kNN classifier for classifying the entire 3D image. The classification procedure is robust against variability in the image quality through a non-parametric implementation: no assumptions are made about the tissue intensity distributions. The performance of this brain tissue classification procedure is demonstrated through quantitative and qualitative validation experiments on both simulated MRI data (10 subjects) and real MRI data (43 subjects). A significant improvement in output quality was observed on subjects who exhibit morphological deviations from the model due to aging and pathology.     

A tissue-specific adaptive texture filter for medical ultrasound images

The interpretation of ultrasound images remains a difficult task and the opinion of different doctors is generally not unequivocal. Therefore, there is a growing interest in the field of computer-aided diagnosis. In the field of medical image processing, computer-aided diagnosis includes image enhancement to facilitate visual interpretation, automatic indication of affected areas, organs and other regions of medical interest, the performance of automatic measurements and image registration. In this article, we introduce a new algorithm for ultrasound image enhancement that employs a multivariate texture classifier based on the co-occurrence matrix, which, in combination with an adaptive texture smoothing filter, is used to enhance the visual difference between and improve boundary detection between healthy neonatal brain tissue and tissue affected by periventricular leukomalacia. For a quantitative comparison, we delineate the periventricular leukomalacia-affected regions with two different active contours before and after processing 10 images with the proposed technique and several speckle filters from the literature. The semi-automatic delineations thus obtained are compared with the manual delineations of a neonatologist. In all cases, the average delineation achieved with the proposed technique is closer to that of the manual expert delineation than when the images are processed with the other techniques.     

Repulsive force based snake model to segment and track neuronal axons in 3D microscopy image stacks

The branching patterns of axons and dendrites are fundamental structural properties that affect the synaptic connectivity of axons. Although today three-dimensional images of fluorescently labeled processes can be obtained to study axonal branching, there are no robust methods of tracing individual axons. This paper describes a repulsive force based snake model to segment and track axonal profiles in 3D images. This new method segments all the axonal profiles in a 2D image and then uses the results obtained from that image as prior information to help segment the adjacent 2D image. In this way, the segmentation successfully connects axonal profiles over hundreds of images in a 3D image stack. Individual axons can then be extracted based on the segmentation results. The utility and performance of the method are demonstrated using 3D axonal images obtained from transgenic mice that express fluorescent protein.     

Cell-based dual snake model: a new approach to extracting highly winding boundaries in the ultrasound images

Two common deficiencies of most conventional deformable models are the need to place the initial contour very close to the desired boundary and the incapability of capturing a highly winding boundary for sonographic boundary extraction. To remedy these two deficiencies, a new deformable model (namely, the cell-based dual snake model) is proposed in this paper. The basic idea is to apply the dual snake model in the cell-based deformation manner. While the dual snake model provides an effective mechanism allowing a distant initial contour, the cell-based deformation makes it possible to catch the winding characteristics of the desired boundary. The performance of the proposed cell-based dual snake model has been evaluated on synthetic images with simulated speckles and on the clinical ultrasound (US) images. The experimental results show that the mean distances from the derived to the desired boundary points are 0.9 +/- 0.42 pixels and 1.29 +/- 0.39 pixels for the synthetic and the clinical US images, respectively.     

Interactive segmentation of abdominal aortic aneurysms in CTA images

A model-based approach to interactive segmentation of abdominal aortic aneurysms from CTA data is presented. After manual delineation of the aneurysm sac in the first slice, the method automatically detects the contour in subsequent slices, using the result from the previous slice as a reference. If an obtained contour is not sufficiently accurate, the user can intervene and provide an additional manual reference contour. The method is inspired by the active shape model (ASM) segmentation scheme (), in which a statistical shape model, derived from corresponding landmark points in manually labeled training images, is fitted to the image in an iterative manner. In our method, a shape model of the contours in two adjacent image slices is progressively fitted to the entire volume. The contour obtained in one slice thus constrains the possible shapes in the next slice. The optimal fit is determined on the basis of multi-resolution gray level models constructed from gray value patches sampled around each landmark. We propose to use the similarity of adjacent image slices for this gray level model, and compare these to single-slice features that are more generally used with ASM. The performance of various image features is evaluated in leave-one-out experiments on 23 data sets. Features that use the similarity of adjacent image slices outperform measures based on single-slice features in all cases. The average number of slices in our datasets is 51, while on average eight manual initializations are required, which decreases operator segmentation time by a factor of 6.     

A deep learning framework for hyperspectral image classification using spatial pyramid pooling

In this letter, a new deep learning framework for spectral–spatial classification of hyperspectral images is presented. The proposed framework serves as an engine for merging the spatial and spectral features via suitable deep learning architecture: stacked autoencoders (SAEs) and deep convolutional neural networks (DCNNs) followed by a logistic regression (LR) classifier. In this framework, SAEs is aimed to get useful high-level features for the one-dimensional features which is suitable for the dimension reduction of spectral features, while DCNNs can learn rich features from the training data automatically and has achieved state-of-the-art performance in many image classification databases. Though the DCNNs has shown robustness to distortion, it only extracts features of the same scale, and hence is insufficient to tolerate large-scale variance of object. As a result, spatial pyramid pooling (SPP) is introduced into hyperspectral image classification for the first time by pooling the spatial feature maps of the top convolutional layers into a fixed-length feature. Experimental results with widely used hyperspectral data indicate that classifiers built in this deep learning-based framework provide competitive performance.     

超声乳腺肿瘤图像的边缘提取

目的探求乳腺肿瘤超声图像的边缘提取。方法广义梯度矢量流Snake模型已经成功地用于噪声相对比较小的CT、MRI等医学图像,然而乳腺肿瘤超声图像对比度低,斑点噪声大,很难将该模型直接应用于乳腺肿瘤超声图像。本文针对乳腺肿瘤超声图像的特点如图像对比度低,斑点噪声大,部分边缘缺失,肿瘤内部微细结构分布复杂(如血管,钙化灶等),特别恶性肿瘤还具有复杂形状等,采用相应的图像处理技术如非线性各向异性扩散滤除斑点噪声,形态学滤波器平滑图像,直方图均衡化提高图像的对比度,最后将该模型引入到乳腺肿瘤超声图像边缘提取。结果实验对158例乳腺肿瘤超声图像进行边缘提取,定量和定性分析均获得满意的结果。结论本文方法可以有效地用于超声乳腺肿瘤图像的边缘提取。