基于卷积神经网络的胎儿二维超声图像丘脑横切面的自动识别

目的:胎儿二维超声图像不同切面的自动识别分类对于提高医生的工作效率具有十分重要的意义。方法:本文针对传统自动分类方法中需要先对图像进行细致分割再进行特征提取和分类识别、分类速度慢等问题,提出了一种基于深度卷积神经网络(Convolutional neural network,CNN)的胎儿二维超声图像中丘脑横切面的自动识别方法。通过对收集到的胎儿二维超声丘脑横切面进行图像增强等预处理,提出了改进的卷积神经网络算法。结果:该算法避免了对于二维超声图像复杂的前期预处理,可以直接输入原始的二维超声图像,具有很强的适应性和泛化能力。试验结果表明,该方法的识别准确率能达到94.81%。结论:此模型的提出,为医学影像自动识别技术提供了新的参考。     

Phase Grouping-Based Needle Segmentation in 3-D Trans-rectal Ultrasound-Guided Prostate Trans-perineal Therapy

A robust and efficient needle segmentation method used to localize and track the needle in 3-D trans-rectal ultrasound (TRUS)-guided prostate therapy is proposed. The algorithmic procedure begins by cropping the 3-D US image containing a needle; then all voxels in the cropped 3-D image are grouped into different line support regions (LSRs) based on the outer product of the adjacent voxels' gradient vector. Two different needle axis extraction methods in the candidate LSR are presented: least-squares fitting and 3-D randomized Hough transform. Subsequent local optimization refines the position of the needle axis. Finally, the needle endpoint is localized by finding an intensity drop along the needle axis. The proposed methods were validated with 3-D TRUS tissue-mimicking agar phantom images, chicken breast phantom images and patient images obtained during prostate cryotherapy. The results of the in vivo test indicate that our method can localize the needle accurately and robustly with a needle endpoint localization accuracy <1.43 mm and detection accuracy >84%, which are favorable for 3-D TRUS-guided prostate trans-perineal therapy.     

基于深度神经网络的颈动脉斑块超声图像诊断技术研究

目的 构建颈动脉斑块超声图像数据集并探讨深度学习技术对颈动脉斑块自动分类诊断的应用价值。方法 首先采集254例患者和354例正常人的颈部动脉超声图像,每例采集2幅图像,构建共包含1216幅图像的颈动脉超声图像数据集；然后,基于已构建的颈动脉超声图像数据集对传统的HOG+SVM方法和14种不同结构的深度神经网络模型进行训练；最后,通过三个量化指标(分类精确率、召回率和F1值)确定现有的颈动脉斑块超声图像分类性能最好的深度神经网络模型。 结果 通过综合比较15种不同的颈动脉斑块超声图像分类方法,得出性能最好的模型是深度残差网络模型ResNet50,其精确率、召回率和F1值分别为97.36%、97.32%和97.34%。 结论 本文通过数据集构建、模型选择、模型训练和测试验证了深度学习技术在颈动脉斑块超声图像自动诊断应用中的有效性,其中深度残差网络模型ResNet50对颈动脉超声图像能进行高准确度自动分类。     

Registration-based segmentation of three-dimensional ultrasound images for quantitative measurement of fetal craniofacial structure

Segmentation of a fetal head from three-dimensional (3-D) ultrasound images is a critical step in the quantitative measurement of fetal craniofacial structure. However, two main issues complicate segmentation, including fuzzy boundaries and large variations in pose and shape among different ultrasound images. In this article, we propose a new registration-based method for automatically segmenting the fetal head from 3-D ultrasound images. The proposed method first detects the eyes based on Gabor features to identify the pose of the fetus image. Then, a reference model, which is constructed from a fetal phantom and contains prior knowledge of head shape, is aligned to the image via feature-based registration. Finally, 3-D snake deformation is utilized to improve the boundary fitness between the model and image. Four clinically useful parameters including inter-orbital diameter (IOD), bilateral orbital diameter (BOD), occipital frontal diameter (OFD) and bilateral parietal diameter (BPD) are measured based on the results of the eye detection and head segmentation. Ultrasound volumes from 11 subjects were used for validation of the method accuracy. Experimental results showed that the proposed method was able to overcome the aforementioned difficulties and achieve good agreement between automatic and manual measurements.     

Standard Plane Localization in Ultrasound by Radial Component Model and Selective Search

Acquisition of the standard plane is crucial for medical ultrasound diagnosis. However, this process requires substantial experience and a thorough knowledge of human anatomy. Therefore it is very challenging for novices and even time consuming for experienced examiners. We proposed a hierarchical, supervised learning framework for automatically detecting the standard plane from consecutive 2-D ultrasound images. We tested this technique by developing a system that localizes the fetal abdominal standard plane from ultrasound video by detecting three key anatomical structures: the stomach bubble, umbilical vein and spine. We first proposed a novel radial component-based model to describe the geometric constraints of these key anatomical structures. We then introduced a novel selective search method which exploits the vessel probability algorithm to produce probable locations for the spine and umbilical vein. Next, using component classifiers trained by random forests, we detected the key anatomical structures at their probable locations within the regions constrained by the radial component-based model. Finally, a second-level classifier combined the results from the component detection to identify an ultrasound image as either a “fetal abdominal standard plane” or a “non- fetal abdominal standard plane.” Experimental results on 223 fetal abdomen videos showed that the detection accuracy of our method was as high as 85.6% and significantly outperformed both the full abdomen and the separate anatomy detection methods without geometric constraints. The experimental results demonstrated that our system shows great promise for application to clinical practice.     

三维超声对胎儿骨骼畸形的研究

目的 探讨三维超声在胎儿骨骼系统发育畸形成像中的临床价值。方法 采用表面及最大透明成像方法，对22例产前常规二维超声检查怀疑胎儿骨骼异常的病例，进行三维超声成像检查，并在诊断中运用电子刀技术，动态三维技术。通过和产后病理解剖对照，比较二维、三维图像提供畸形的部位及细节的准确性。结果 21例能进行三维扫查，并得出表面或最大透明图像，产前二维超声对病变部位及细节判断准确分别为16例（72.7%）,和9例(40.9%)，而三维超声判断准确分别为21例（95.5%）和20例(91%)。结论 表面三维超声立体成像能够得出骨骼畸形的外貌特征，采用最大透明成像方式，能够了解畸形的骨骼数量和形态改变。三维超声为胎儿骨骼系统发育异常的诊断，提供了重要手段，是二维超声的重要补充。诊断中，电子解剖刀，动态三维技术进一步提高了三维图像的准确性。     

Deep Convolutional Neural Network-Based Automatic Classification of Neonatal Hip Ultrasound Images: A Novel Data Augmentation Approach with Speckle Noise Reduction

Neonatal hip ultrasound imaging has been widely used for a few decades in the diagnosis of developmental dysplasia of the hip (DDH). Graf's method of hip ultrasonography is still the most reproducible because of its classification system; yet, the reproducibility is questionable as a result of the high dependency on the skills of the sonographer and the evaluator. A computer-aided diagnosis system may help evaluators increase their precision in the diagnosis of DDH using Graf's method. This study describes a fully automatic computer-aided diagnosis system for the classification of hip ultrasound images captured in Graf's standard plane based on convolutional neural networks (CNNs). Automatically segmented image patches containing all of the necessary anatomical structures were given to the proposed CNN system to extract discriminative features and classify the recorded hips. For ease of evaluation, the data set was divided into three groups: normal, mild dysplasia and severe dysplasia. This study proposes a different approach to data augmentation using speckle noise reduction with an optimized Bayesian non-local mean filter. Data augmentation with this filter increased the accuracy of the proposed CNN system from 92.29% to 97.70%. This new approach for automatic classification of DDH, classifies dysplastic neonatal hips with a high accuracy rate and might help evaluators to increase their evaluation success.     

Method for Carotid Artery 3-D Ultrasound Image Segmentation Based on CSWin Transformer

Precise segmentation of carotid artery (CA) structure is an important prerequisite for the medical assessment and detection of carotid plaques. For automatic segmentation of the media–adventitia boundary (MAB) and lumen–intima boundary (LIB) in 3-D ultrasound images of the CA, a U-shaped CSWin transformer (U-CSWT) is proposed. Both the encoder and decoder of the U-CSWT are composed of hierarchical CSWT modules, which can capture rich global context information in the 3-D image. Experiments were performed on a 3-D ultrasound image data set of the CA, and the results indicate that the U-CSWT performs better than other convolutional neural network (CNN)-based and CNN–transformer hybrid methods. The model yields Dice coefficients of 94.6 ± 3.0% and 90.8 ± 5.1% for the MAB and LIB in the common carotid artery (CCA) and 92.9 ± 4.9% and 89.6 ± 6.2% for MAB and LIB in the bifurcation, respectively. Our U-CSWT is expected to become an effective method for automatic segmentation of 3-D ultrasound images of CA.     

Computer-aided diagnosis for the classification of breast masses in automated whole breast ultrasound images

New automated whole breast ultrasound (ABUS) machines have recently been developed and the ultrasound (US) volume dataset of the whole breast can be acquired in a standard manner. The purpose of this study was to develop a novel computer-aided diagnosis system for classification of breast masses in ABUS images. One hundred forty-seven cases (76 benign and 71 malignant breast masses) were obtained by a commercially available ABUS system. Because the distance of neighboring slices in ABUS images is fixed and small, these continuous slices were used for reconstruction as three-dimensional (3-D) US images. The 3-D tumor contour was segmented using the level-set segmentation method. Then, the 3-D features, including the texture, shape and ellipsoid fitting were extracted based on the segmented 3-D tumor contour to classify benign and malignant tumors based on the logistic regression model. The Student’s t test, Mann-Whitney U test and receiver operating characteristic (ROC) curve analysis were used for statistical analysis. From the Az values of ROC curves, the shape features (0.9138) are better than the texture features (0.8603) and the ellipsoid fitting features (0.8496) for classification. The difference was significant between shape and ellipsoid fitting features (p = 0.0382). However, combination of ellipsoid fitting features and shape features can achieve a best performance with accuracy of 85.0% (125/147), sensitivity of 84.5% (60/71), specificity of 85.5% (65/76) and the area under the ROC curve Az of 0.9466. The results showed that ABUS images could be used for computer-aided feature extraction and classification of breast tumors. (E-mail: rfchang@csie.ntu.edu.tw)     

Automatic Segmentation of Calcifications in Intravascular Ultrasound Images Using Snakes and the Contourlet Transform

It is valuable to detect calcifications in intravascular ultrasound images for studies of coronary artery diseases. An image segmentation method based on snakes and the Contourlet transform is proposed to automatically and accurately detect calcifications. With the Contourlet transform, an original image is decomposed into low-pass bands and band-pass directional sub-bands. The 2-D Renyi's entropy is used to adaptively threshold the low-pass bands in a multiresolution hierarchy to determine regions-of-interest (ROIs). Then a mean intensity ratio, reflecting acoustic shadowing, is presented to classify calcifications from noncalcifications and obtain initial contours of calcifications. The anisotropic diffusion is used in bandpass directional sub-bands to suppress noise and preserve calcific edges. Finally, the contour deformation in the boundary vector field is used to obtain final contours of calcifications. The method was evaluated via 60 simulated images and 86 in vivo images. It outperformed a recently proposed method, the Santos Filho method, by 2.76% and 14.53%, in terms of the sensitivity and specificity of calcification detection, respectively. The area under the receiver operating characteristic curve increased by 0.041. The relative mean distance error, relative difference degree, relative arc difference, relative thickness difference and relative length difference were reduced by 5.73%, 19.79%, 11.62%, 12.06% and 20.51%, respectively. These results reveal that the proposed method can automatically and accurately detect calcifications and delineate their boundaries. (E-mail: yywang@fudan.edu.cn)     

Automatic bone localization and fracture detection from volumetric ultrasound images using 3-D local phase features

This article presents a novel method for bone segmentation from three-dimensional (3-D) ultrasound images that derives intensity-invariant 3-D local image phase measures that are then employed for extracting ridge-like features similar to those that occur at soft tissue/bone interfaces. The main contributions in this article include: (1) the extension of our previously proposed phase-symmetry-based bone surface extraction from two-dimensional (2-D) to 3-D images using 3-D Log-Gabor filters; (2) the design of a new framework for accuracy evaluation based on using computed tomography as a gold standard that allows the assessment of surface localization accuracy across the entire 3-D surface; (3) the quantitative validation of accuracy of our 3-D phase-processing approach on both intact and fractured bone surfaces using phantoms and ex vivo 3-D ultrasound scans; and (4) the qualitative validation obtained by scanning emergency room patients with distal radius and pelvis fractures. We show a 41% improvement in surface localization error over the previous 2-D phase symmetry method. The results demonstrate clearly visible segmentations of bone surfaces with a localization accuracy of <0.6 mm and mean errors in estimating fracture displacements below 0.6 mm. The results show that the proposed method is successful even for situations when the bone surface response is weak due to shadowing from muscle and fascia interfaces above the bone, which is a situation where the 2-D method fails.     

Fetal Hands: A Comprehensive Review of Prenatal Assessment and Diagnosis Over the Past 40 Years

Fetal skeletal dysplasias involving limbs and hands are rare congenital malformations. Prenatal two-dimensional ultrasound diagnosis of fetal limb defects has a sensitivity of about 30%; however, an increased detection rate may be obtained using three-dimensional (3-D) ultrasound in the rendering mode. 3-D ultrasound may be used as a complementary method providing additional information. Currently, magnetic resonance imaging (MRI), with the emergence of ultrafast imaging techniques and new sequences, allows for better diagnosis of several fetal skeletal dysplasias such as limb reduction defects and neuromuscular disorders. 3-D volumetric images from ultrasound or MRI scan data allow 3-D ultrasound reconstructions of virtual/physical models, and virtual reality can help researchers to improve our understanding of both normal and abnormal fetal limb/hand anatomy. In this article, we review the embryological development of fetal hands and their main anomalies including prenatal diagnostic methods, genetic counseling, the role of orthopedic and plastic surgery reconstruction, and new perspectives in fetal surgery.     

Segmentation of breast ultrasound image with semantic classification of superpixels

Breast cancer is a great threat to females. Ultrasound imaging has been applied extensively in diagnosis of breast cancer. Due to the poor image quality, segmentation of breast ultrasound (BUS) image remains a very challenging task. Besides, BUS image segmentation is a crucial step for further analysis. In this paper, we proposed a novel method to segment the breast tumor via semantic classification and merging patches. The proposed method firstly selects two diagonal points to crop a region of interest (ROI) on the original image. Then, histogram equalization, bilateral filter and pyramid mean shift filter are adopted to enhance the image. The cropped image is divided into many superpixels using simple linear iterative clustering (SLIC). Furthermore, some features are extracted from the superpixels and a bag-of-words model can be created. The initial classification can be obtained by a back propagation neural network (BPNN). To refine preliminary result, k-nearest neighbor (KNN) is used for reclassification and the final result is achieved. To verify the proposed method, we collected a BUS dataset containing 320 cases. The segmentation results of our method have been compared with the corresponding results obtained by five existing approaches. The experimental results show that our method achieved competitive results compared to conventional methods in terms of TP and FP, and produced good approximations to the hand-labelled tumor contours with comprehensive consideration of all metrics (the F1-score = 89.87% ± 4.05%, and the average radial error = 9.95% ± 4.42%).     

Registration of 3D fetal neurosonography and MRI

We propose a method for registration of 3D fetal brain ultrasound with a reconstructed magnetic resonance fetal brain volume. This method, for the first time, allows the alignment of models of the fetal brain built from magnetic resonance images with 3D fetal brain ultrasound, opening possibilities to develop new, prior information based image analysis methods for 3D fetal neurosonography. The reconstructed magnetic resonance volume is first segmented using a probabilistic atlas and a pseudo ultrasound image volume is simulated from the segmentation. This pseudo ultrasound image is then affinely aligned with clinical ultrasound fetal brain volumes using a robust block-matching approach that can deal with intensity artefacts and missing features in the ultrasound images. A qualitative and quantitative evaluation demonstrates good performance of the method for our application, in comparison with other tested approaches. The intensity average of 27 ultrasound images co-aligned with the pseudo ultrasound template shows good correlation with anatomy of the fetal brain as seen in the reconstructed magnetic resonance image.     

Image-based cardiac gating for three-dimensional intravascular ultrasound imaging

Three-dimensional (3-D) intravascular ultrasound (US), or IVUS, provides valuable insight into the tissue characteristics of the coronary wall and plaque composition. However, artefacts due to cardiac motion and vessel wall pulsation limit the accuracy and variability of coronary lumen and plaque volume measurement in 3-D IVUS images. ECG-gated image acquisition can reduce these artefacts but it requires recording the ECG signal and may increase image acquisition time. The goal of our study was to reconstruct a 3-D IVUS image with negligible cardiac motion and vessel pulsation artefacts, by developing an image-based gating method to track 2-D IVUS images over the cardiac cycle. Our approach involved selecting 2-D IVUS images belonging to the same cardiac phase from an asynchronously-acquired series, by tracking the changing lumen contour over the cardiac cycle. The algorithm was tested with IVUS images of a custom-built coronary vessel phantom and with patient images. The artefact reduction achieved using the image-gating approach was > 86% in the in vitro images and > 80% in the in vivo images in our study. Our study shows that image-based gating of IVUS images provides a useful method for accurate reconstruction of 3-D IVUS images with reduced cardiac motion artefact.     

Strain-Initialized Robust Bone Surface Detection in 3-D Ultrasound

Three-dimensional ultrasound has been increasingly considered as a safe radiation-free alternative to radiation-based fluoroscopic imaging for surgical guidance during computer-assisted orthopedic interventions, but because ultrasound images contain significant artifacts, it is challenging to automatically extract bone surfaces from these images. We propose an effective way to extract 3-D bone surfaces using a surface growing approach that is seeded from 2-D bone contours. The initial 2-D bone contours are estimated from a combination of ultrasound strain images and envelope power images. Novel features of the proposed method include: (i) improvement of a previously reported 2-D strain imaging-based bone segmentation method by incorporation of a depth-dependent cumulative power of the envelope into the elastographic data; (ii) incorporation of an echo decorrelation measure-based weight to fuse the strain and envelope maps; (iii) use of local statistics of the bone surface candidate points to detect the presence of any bone discontinuity; and (iv) an extension of our 2-D bone contour into a 3-D bone surface by use of an effective surface growing approach. Our new method produced average improvements in the mean absolute error of 18% and 23%, respectively, on 2-D and 3-D experimental phantom data, compared with those of two state-of-the-art bone segmentation methods. Validation on 2-D and 3-D clinical in vivo data also reveals, respectively, an average improvement in the mean absolute fitting error of 55% and an 18-fold improvement in the computation time.     

Tunable image quality control of 3-D ultrasound using switchable CycleGAN

In contrast to 2-D ultrasound (US) for uniaxial plane imaging, a 3-D US imaging system can visualize a volume along three axial planes. This allows for a full view of the anatomy, which is useful for gynecological (GYN) and obstetrical (OB) applications. Unfortunately, the 3-D US has an inherent limitation in resolution compared to the 2-D US. In the case of 3-D US with a 3-D mechanical probe, for example, the image quality is comparable along the beam direction, but significant deterioration in image quality is often observed in the other two axial image planes. To address this, here we propose a novel unsupervised deep learning approach to improve 3-D US image quality. In particular, using unmatched high-quality 2-D US images as a reference, we trained a recently proposed switchable CycleGAN architecture so that every mapping plane in 3-D US can learn the image quality of 2-D US images. Thanks to the switchable architecture, our network can also provide real-time control of image enhancement level based on user preference, which is ideal for a user-centric scanner setup. Extensive experiments with clinical evaluation confirm that our method offers significantly improved image quality as well user-friendly flexibility.     

Lumbar Ultrasound Image Feature Extraction and Classification with Support Vector Machine

Needle entry site localization remains a challenge for procedures that involve lumbar puncture, for example, epidural anesthesia. To solve the problem, we have developed an image classification algorithm that can automatically identify the bone/interspinous region for ultrasound images obtained from lumbar spine of pregnant patients in the transverse plane. The proposed algorithm consists of feature extraction, feature selection and machine learning procedures. A set of features, including matching values, positions and the appearance of black pixels within pre-defined windows along the midline, were extracted from the ultrasound images using template matching and midline detection methods. A support vector machine was then used to classify the bone images and interspinous images. The support vector machine model was trained with 1,040 images from 26 pregnant subjects and tested on 800 images from a separate set of 20 pregnant patients. A success rate of 95.0% on training set and 93.2% on test set was achieved with the proposed method. The trained support vector machine model was further tested on 46 off-line collected videos, and successfully identified the proper needle insertion site (interspinous region) in 45 of the cases. Therefore, the proposed method is able to process the ultrasound images of lumbar spine in an automatic manner, so as to facilitate the anesthetists' work of identifying the needle entry site.     

Multifeature Fusion Classification Method for Adaptive Endoscopic Ultrasonography Tumor Image

Endoscopic ultrasonography (EUS) has been found to be of great advantage in the diagnosis of digestive tract submucosal tumors. However, EUS-based diagnosis is limited by variability in subjective interpretation on the part of doctors. Tumor classification of ultrasound images with the computer-aided diagnosis system can significantly improve the diagnostic efficiency and accuracy of doctors. In this study, we proposed a multifeature fusion classification method for adaptive EUS tumor images. First, for different ultrasound tumor images, we selected the region of interest based on prior information to facilitate the estimation in the subsequent works. Second, we proposed a method based on image gray histogram feature extraction with principal component analysis dimensionality reduction, which learns the gray distribution of different tumor images effectively. Third, we fused the reduced grayscale features with the improved local binary pattern features and gray-level co-occurrence matrix features, and then used the multiclassification support vector machine. Finally, in the experiment, we selected the 431 ultrasound images of 109 patients in the hospital and compared the experimental effects of different features and different classifiers. The results revealed that the proposed method performed best, with the highest accuracy of 96.18% and an area under the curve of 99%. It is evident that the method proposed in this study can efficiently contribute to the classification of EUS tumor images.     

A Closed-Form Differential Formulation for Ultrasound Spatial Calibration: Multi-wedge Phantom

Calibration is essential in freehand 3-D ultrasound to find the spatial transformation from the image coordinates to the sensor coordinate system. Calibration accuracy has significant impact on image-guided interventions. We introduce a new mathematical framework that uses differential measurements to achieve high calibration accuracy. Accurate measurements of axial differences in ultrasound images of a multi-wedge phantom are used to calculate the calibration matrix with a closed-form solution. The multi-wedge phantom has been designed based on the proposed differential framework and can be mass-produced inexpensively using a 3-D printer. The proposed method enables easy, fast and highly accurate ultrasound calibration, which is essential for most current ultrasound-guided applications and also widens the range of new applications. The precision of the method using only a single image of the phantom is comparable to that of the standard N-wire method using 50 images. The method can also directly take advantage of the fine sampling rate of radiofrequency ultrasound data to achieve very high calibration accuracy. With 100 radiofrequency ultrasound images, the method achieves a point reconstruction error of 0.09 ± 0.39 mm.