Real-time ultrasound transducer localization in fluoroscopy images by transfer learning from synthetic training data

The fusion of image data from trans-esophageal echography (TEE) and X-ray fluoroscopy is attracting increasing interest in minimally-invasive treatment of structural heart disease. In order to calculate the needed transformation between both imaging systems, we employ a discriminative learning (DL) based approach to localize the TEE transducer in X-ray images. The successful application of DL methods is strongly dependent on the available training data, which entails three challenges: (1) the transducer can move with six degrees of freedom meaning it requires a large number of images to represent its appearance, (2) manual labeling is time consuming, and (3) manual labeling has inherent errors.This paper proposes to generate the required training data automatically from a single volumetric image of the transducer. In order to adapt this system to real X-ray data, we use unlabeled fluoroscopy images to estimate differences in feature space density and correct covariate shift by instance weighting. Two approaches for instance weighting, probabilistic classification and Kullback–Leibler importance estimation (KLIEP), are evaluated for different stages of the proposed DL pipeline. An analysis on more than 1900 images reveals that our approach reduces detection failures from 7.3% in cross validation on the test set to zero and improves the localization error from 1.5 to 0.8 mm. Due to the automatic generation of training data, the proposed system is highly flexible and can be adapted to any medical device with minimal efforts.     

Automatic bone localization and fracture detection from volumetric ultrasound images using 3-D local phase features

This article presents a novel method for bone segmentation from three-dimensional (3-D) ultrasound images that derives intensity-invariant 3-D local image phase measures that are then employed for extracting ridge-like features similar to those that occur at soft tissue/bone interfaces. The main contributions in this article include: (1) the extension of our previously proposed phase-symmetry-based bone surface extraction from two-dimensional (2-D) to 3-D images using 3-D Log-Gabor filters; (2) the design of a new framework for accuracy evaluation based on using computed tomography as a gold standard that allows the assessment of surface localization accuracy across the entire 3-D surface; (3) the quantitative validation of accuracy of our 3-D phase-processing approach on both intact and fractured bone surfaces using phantoms and ex vivo 3-D ultrasound scans; and (4) the qualitative validation obtained by scanning emergency room patients with distal radius and pelvis fractures. We show a 41% improvement in surface localization error over the previous 2-D phase symmetry method. The results demonstrate clearly visible segmentations of bone surfaces with a localization accuracy of <0.6 mm and mean errors in estimating fracture displacements below 0.6 mm. The results show that the proposed method is successful even for situations when the bone surface response is weak due to shadowing from muscle and fascia interfaces above the bone, which is a situation where the 2-D method fails.     

A framework for guiding learning in ultrasound scanning

Ultrasound scanning is provided by a range of health professionals who need to be trained to a proficient level. In respect of education and training in ultrasound scanning, little attention has been given to how scanning skills are acquired and what assists and hinders the learning process. This study aims to develop a framework for guiding learning in ultrasound scanning. Overt participant observation and semi-structured interviews generated data on four learners undertaking a 12-month postgraduate ultrasound programme. Narrative analysis of the interview data was used to reveal dominant themes related to stages in learning to scan. Dominant themes associated with learning to scan were communication with the patient, navigation skills, image interpretation skills, observation of expert practice, feedback on performance and random practise. Detailed interpretation of the themes through narrative analysis provided characteristics of learning for each stage of a four staged process. This study provides an insight into the key features of scan performance and how scanning skills are acquired over a four-staged approach. These themes and characteristics are presented in a framework for guiding learning in ultrasound scanning.     

On the usability of synthetic data for improving the robustness of deep learning-based segmentation of cardiac magnetic resonance images

Deep learning-based segmentation methods provide an effective and automated way for assessing the structure and function of the heart in cardiac magnetic resonance (CMR) images. However, despite their state-of-the-art performance on images acquired from the same source (same scanner or scanner vendor) as images used during training, their performance degrades significantly on images coming from different domains. A straightforward approach to tackle this issue consists of acquiring large quantities of multi-site and multi-vendor data, which is practically infeasible. Generative adversarial networks (GANs) for image synthesis present a promising solution for tackling data limitations in medical imaging and addressing the generalization capability of segmentation models. In this work, we explore the usability of synthesized short-axis CMR images generated using a segmentation-informed conditional GAN, to improve the robustness of heart cavity segmentation models in a variety of different settings. The GAN is trained on paired real images and corresponding segmentation maps belonging to both the heart and the surrounding tissue, reinforcing the synthesis of semantically-consistent and realistic images. First, we evaluate the segmentation performance of a model trained solely with synthetic data and show that it only slightly underperforms compared to the baseline trained with real data. By further combining real with synthetic data during training, we observe a substantial improvement in segmentation performance (up to 4% and 40% in terms of Dice score and Hausdorff distance) across multiple data-sets collected from various sites and scanner. This is additionally demonstrated across state-of-the-art 2D and 3D segmentation networks, whereby the obtained results demonstrate the potential of the proposed method in tackling the presence of the domain shift in medical data. Finally, we thoroughly analyze the quality of synthetic data and its ability to replace real MR images during training, as well as provide an insight into important aspects of utilizing synthetic images for segmentation.     

基于迁移学习的乳腺肿瘤超声图像智能分类诊断

目的 探讨迁移学习方法对乳腺良恶性肿瘤超声图像分类的价值。方法 回顾性分析经病理证实的447例乳腺肿瘤的超声声像图,采用主成分分析法对原始图像进行分析提取;在Matlab 7.0软件中编程实现迁移学习,将量化的图像特征作为输入数据,利用迁移学习对乳腺良恶性肿瘤进行智能分类。结果 乳腺恶性肿瘤的边缘粗糙度、坚固度、邻域灰度差矩阵粗糙度、肿瘤后方与周围区域回声差异及水平方向高频分量和垂直方向低频分量的直方图能量均明显高于良性肿瘤（P均<0.05）。超声和迁移学习方法诊断乳腺恶性肿瘤的敏感度分别为96.21%（127/132）和96.04%（97/101）,特异度为66.35%（209/315）和98.49%（196/199）,准确率为75.17%（336/447）和97.67%（293/300）。结论 超声图像特征定量化可为识别良恶性乳腺肿瘤提供客观的量化参数;迁移学习可有效对乳腺良恶性肿瘤的声像图进行分类。     

督导教学联合PAL教学法在超声医学专业技能培训中的应用

目的探讨督导教学联合同伴互助学习（PAL）教学法在超声医学专业技能培训教学中的应用价值。 方法选取2020年6月至2022年6月浙江大学医学院附属第二医院超声医学科2018级、2019级住院医师共34人。采用督导教学联合PAL教学法,对2018级、2019级住院医师进行各系统操作切面带教培训与讲解、互助组学习以及带教老师督导教学。每周进行一次系统切面考核（过程考核）,按专业技能考试评分表,80分为合格。培训结束后进行结业技能考试（结业考试）。统计结业技能考试通过率,以及采用Likert 5级评分法调查住院医师对该课程的满意程度,以评估该教学方法的有效性。 结果应用督导教学联合PAL教学法后,超声医学科2018级、2019级住院医师每周进行的专业技能操作考核（过程考核）成绩均合格（≥80分）,考试平均分分别为（88.29±2.78）、（87.49±4.51）分;住院医师结业技能考试（结业考试）通过率为100%。收回满意度调查问卷34份,结果表明,住院医师均对该教学方法和内容安排满意或非常满意,均认为督导教学联合PAL教学法有助于掌握专业操作技能,提高学习效率,促进与老师的交流。 结论督导教学联合同伴PAL教学法在住院医师规范化培训超声专业技能培训中应用效果较好,可为超声专业技能培训教学提供新的思路。     

Cross-dimensional transfer learning in medical image segmentation with deep learning

Over the last decade, convolutional neural networks have emerged and advanced the state-of-the-art in various image analysis and computer vision applications. The performance of 2D image classification networks is constantly improving and being trained on databases made of millions of natural images. Conversely, in the field of medical image analysis, the progress is also remarkable but has mainly slowed down due to the relative lack of annotated data and besides, the inherent constraints related to the acquisition process. These limitations are even more pronounced given the volumetry of medical imaging data. In this paper, we introduce an efficient way to transfer the efficiency of a 2D classification network trained on natural images to 2D, 3D uni- and multi-modal medical image segmentation applications. In this direction, we designed novel architectures based on two key principles: weight transfer by embedding a 2D pre-trained encoder into a higher dimensional U-Net, and dimensional transfer by expanding a 2D segmentation network into a higher dimension one. The proposed networks were tested on benchmarks comprising different modalities: MR, CT, and ultrasound images. Our 2D network ranked first on the CAMUS challenge dedicated to echo-cardiographic data segmentation and surpassed the state-of-the-art. Regarding 2D/3D MR and CT abdominal images from the CHAOS challenge, our approach largely outperformed the other 2D-based methods described in the challenge paper on Dice, RAVD, ASSD, and MSSD scores and ranked third on the online evaluation platform. Our 3D network applied to the BraTS 2022 competition also achieved promising results, reaching an average Dice score of 91.69% (91.22%) for the whole tumor, 83.23% (84.77%) for the tumor core and 81.75% (83.88%) for enhanced tumor using the approach based on weight (dimensional) transfer. Experimental and qualitative results illustrate the effectiveness of our methods for multi-dimensional medical image segmentation.     

Multilevel structure-preserved GAN for domain adaptation in intravascular ultrasound analysis

The poor generalizability of intravascular ultrasound (IVUS) analysis methods caused by the great diversity of IVUS datasets is hopefully addressed by the domain adaptation strategy. However, existing domain adaptation models underperform in intravascular structural preservation, because of the complex pathology and low contrast in IVUS images. Losing structural information during the domain adaptation would lead to inaccurate analyses of vascular states. In this paper, we propose a Multilevel Structure-Preserved Generative Adversarial Network (MSP-GAN) for transferring IVUS domains while maintaining intravascular structures. On the generator-discriminator baseline, the MSP-GAN integrates the transformer, contrastive restraint, and self-ensembling strategy, for effectively preserving structures in multi-levels, including global, local, and fine levels. For the global-level pathology maintenance, the generator explores long-range dependencies in IVUS images via an incorporated vision transformer. For the local-level anatomy consistency, a region-to-region correspondence is forced between the translated and source images via a superpixel-wise multiscale contrastive (SMC) constraint. For reducing distortions of fine-level structures, a self-ensembling mean teacher generates the pixel-wise pseudo-label and restricts the translated image via an uncertainty-aware teacher-student consistency (TSC) constraint. Experiments were conducted on 20 MHz and 40 MHz IVUS datasets from different medical centers. Ablation studies illustrate that each innovation contributes to intravascular structural preservation. Comparisons with representative domain adaptation models illustrate the superiority of the MSP-GAN in the structural preservation. Further comparisons with the state-of-the-art IVUS analysis accuracy demonstrate that the MSP-GAN is effective in enlarging the generalizability of diverse IVUS analysis methods and promoting accurate vessel and lumen segmentation and stenosis-related parameter quantification.     

A generalizable and robust deep learning algorithm for mitosis detection in multicenter breast histopathological images

Mitosis counting of biopsies is an important biomarker for breast cancer patients, which supports disease prognostication and treatment planning. Developing a robust mitotic cell detection model is highly challenging due to its complex growth pattern and high similarities with non-mitotic cells. Most mitosis detection algorithms have poor generalizability across image domains and lack reproducibility and validation in multicenter settings. To overcome these issues, we propose a generalizable and robust mitosis detection algorithm (called FMDet), which is independently tested on multicenter breast histopathological images. To capture more refined morphological features of cells, we convert the object detection task as a semantic segmentation problem. The pixel-level annotations for mitotic nuclei are obtained by taking the intersection of the masks generated from a well-trained nuclear segmentation model and the bounding boxes provided by the MIDOG 2021 challenge. In our segmentation framework, a robust feature extractor is developed to capture the appearance variations of mitotic cells, which is constructed by integrating a channel-wise multi-scale attention mechanism into a fully convolutional network structure. Benefiting from the fact that the changes in the low-level spectrum do not affect the high-level semantic perception, we employ a Fourier-based data augmentation method to reduce domain discrepancies by exchanging the low-frequency spectrum between two domains. Our FMDet algorithm has been tested in the MIDOG 2021 challenge and ranked first place. Further, our algorithm is also externally validated on four independent datasets for mitosis detection, which exhibits state-of-the-art performance in comparison with previously published results. These results demonstrate that our algorithm has the potential to be deployed as an assistant decision support tool in clinical practice. Our code has been released at https://github.com/Xiyue-Wang/1st-in-MICCAI-MIDOG-2021-challenge.     

Mapping spatial and temporal changes in carotid atherosclerosis from three-dimensional ultrasound images

This study was designed to evaluate changes in carotid atherosclerosis using plaque and wall thickness maps derived from three-dimensional ultrasound (3DUS) images. Five subjects with carotid stenosis were scanned at baseline and 3 mo as part of a placebo-controlled intensive statin treatment study and three subjects with moderate atherosclerosis were scanned at baseline and again within 14 +/- 2 d. 3DUS-derived vessel wall volume (VWV) was measured using manual segmentation to provide segmentation contours that were used to generate scan and rescan carotid atherosclerosis thickness maps and thickness difference maps. There was no significant difference in VWV between scan and rescan for the three subjects scanned twice in 2 wk or the single subject treated with placebo. There was a significant difference between scan and rescan VWV for carotid stenosis subjects treated with atorvastatin (p < 0.001). Carotid atherosclerosis thickness difference maps showed visual qualitative evidence of thickness changes in vessel wall and plaque thickness in the common carotid artery for all statin-treated subjects and no change in a placebo-treated subject and subjects scanned twice in 2 wk. Carotid atherosclerosis thickness difference maps generated from 3DUS images provide evidence of vessel wall and plaque thickness changes for all subjects assessed.     

射频超声评价亚临床甲状腺功能减退症颈动脉弹性的临床价值

目的应用射频超声技术检测亚临床甲状腺功能减退症(SCH)患者颈动脉弹性功能。方法 SCH患者93例,据促甲状腺激素(TSH)水平分为SCH1组(TSH 4.2~10.0 m U/L)46例,SCH2组(TSH10.0 m U/L)47例,体检健康者50例为对照组。检测其血清总胆固醇、甘油三酯、低密度脂蛋白。射频超声检测其颈动脉弹性参数:内—中膜厚度(IMT)、顺应性系度(CC)、膨胀系数(DC)、弹性系数(α、β)、脉搏波速度(PWV)及动脉反射波增强指数(AIx)。结果 SCH2组总胆固醇、低密度脂蛋白高于对照组(P0.05)。与对照组相比,SCH1组DC、β、PWV、AIX均增高(P0.05),SCH2组IMT、CC、DC、α、β、PWV、AIx均增高(P0.05)。SCH2组IMT、α、β、PWV、AIX高于SCH1组(P0.05)。DC与IMT、α、β、PWV呈负相关(P0.01)。CC与IMT、α、β、PWV呈负相关,与DC呈正相关(均P0.01)。AIX与IMT呈正相关,与DC呈负相关(均P0.01)。结论射频超声能尽早发现颈动脉弹性改变,为评估SCH患者大动脉损害提供重要的检测方法。     

Standard Plane Localization in Ultrasound by Radial Component Model and Selective Search

Acquisition of the standard plane is crucial for medical ultrasound diagnosis. However, this process requires substantial experience and a thorough knowledge of human anatomy. Therefore it is very challenging for novices and even time consuming for experienced examiners. We proposed a hierarchical, supervised learning framework for automatically detecting the standard plane from consecutive 2-D ultrasound images. We tested this technique by developing a system that localizes the fetal abdominal standard plane from ultrasound video by detecting three key anatomical structures: the stomach bubble, umbilical vein and spine. We first proposed a novel radial component-based model to describe the geometric constraints of these key anatomical structures. We then introduced a novel selective search method which exploits the vessel probability algorithm to produce probable locations for the spine and umbilical vein. Next, using component classifiers trained by random forests, we detected the key anatomical structures at their probable locations within the regions constrained by the radial component-based model. Finally, a second-level classifier combined the results from the component detection to identify an ultrasound image as either a “fetal abdominal standard plane” or a “non- fetal abdominal standard plane.” Experimental results on 223 fetal abdomen videos showed that the detection accuracy of our method was as high as 85.6% and significantly outperformed both the full abdomen and the separate anatomy detection methods without geometric constraints. The experimental results demonstrated that our system shows great promise for application to clinical practice.     

超声射频信号技术评价健康成人颈总动脉结构及功能

目的 应用超声射频信号的血管内-中膜分析技术（QIMT）、血管硬度定量分析技术(QAS) 评价健康成人颈总动脉结构及功能。方法 360例健康成人根据年龄分成3组：青年组（20~39岁组）、中年组（40~59岁组）和老年组（60~79岁组）,每组均120人,分别应用QIMT测量双侧颈总动脉内-中膜厚度（IMT）,QAS技术测量双侧颈总动脉的弹性参数,包括扩张系数（DD）、顺应性系数（CC）、硬度指数(α、β)、脉搏波传导速度(PWV),分析各项参数在双侧颈总动脉、不同性别、不同年龄组之间的差异。结果 ①颈总动脉QIMT和QAS指标在颈总动脉左右两侧组间差异均无统计学意义;②颈总动脉DC在两性别组之间的差异有统计学意义,男性高于女性(P<0.05);③从青年组经中年组到老年组,IMT依次增厚、DC依次减小,差异均有统计学意义(P<0.05),与青年组比较,中年组和老年组的CC减低,α、β及PWV增加,差异有统计学意义(P<0.05),而在中年组与老年组之间差异没有统计学意义。结论 健康成人颈总动脉结构及功能随年龄、性别变化存在差异。     

Using a social networking tool for blended learning in staff training: Sharing experience from practice

Advances in neonatal care has improved survival rates for critically ill and premature babies. This means that neonatal nurses are required to be proficient in a range of skills necessary to care for these babies. Adequate knowledge including clinical and critical decision making skills is key to provide this care to a high standard. Lack of neonatal nurses in England means that many units are forced to recruit nurses with little or no experience in neonatal field stretching their limits to its maximum. This article talks about the author's experience of dealing with such a situation when a massive recruitment took place in her hospital. It addresses the use of blended learning with social networking as the online learning tool in planning and implementing an induction programme for newly recruited nurses. It also looks at the benefits, challenges and safety issues of using social networking as an educational strategy.     

不同形式的运动训练对血管性痴呆大鼠学习记忆及海马区突触可塑性的影响

目的:观察自主运动、强迫运动和功能性电刺激诱导的运动对血管性痴呆(vascular dementia,VD)大鼠学习记忆、海马区突触可塑性的影响。方法:成年Wistar雄性大鼠,体重250—300g;用10%水合氯醛(300mg/kg)行腹腔注射麻醉,采用双侧颈总动脉永久性结扎法制作血管性痴呆模型。造模成功后大鼠在跑轮中适应3天(剔除运动量不能达到每天270m的大鼠),采用随机数字法分为假手术组、模型组、自主运动组、强迫运动组和功能性电刺激组,每组各8只。假手术组:仅暴露双侧颈总动脉,但不接扎,术后大鼠置于笼中自由活动;模型组:采用双侧颈总动脉永久性结扎法制作VD模型,术后大鼠置于笼中自由活动;自主运动组:造模1周后大鼠在跑轮(直径31.8cm,宽度10cm,旋转阻力约相当于100g物体的重力)上自由运动,用传感器记录跑过的圈数,每天270圈;强迫运动组:造模1周后大鼠在电动跑轮(直径31.8cm,长度40cm,转速9r/min)上运动,每天治疗30min;功能性电刺激组:造模1周后开始治疗,诱导大鼠前肢产生以9m/min行走时的动作,每天治疗30min。以上五组于治疗14d后,采用新奇事物识别实验测试大鼠学习记忆能力。取大鼠海马组织采用Western blot技术检测上述各组SYN、SYP、PSD-95及MAP-2、TAU蛋白表达。采用免疫组织化学染色法检测海马CA1区微管结合蛋白的变化。结果:(1)新奇事物识别实验:训练阶段各组大鼠对两个相同物体的探寻指数无显著性差异,24h后进行测试,自主运动组、强迫运动组和功能性电刺激组新奇事物认知指数与模型组比较,差异均具有显著性(P0.05),自主运动组新奇事物认知指数与强迫运动组、功能性电刺激组比较,差异均有显著性(P0.05),而功能性电刺激组与强迫运动组比较无显著性差异。(2)海马区SYN、SYP、PSD-95、MAP-2、TAU蛋白表达水平:自主运动组、强迫运动组和功能性电刺激组SYN、PSD-95、MAP-2、TAU蛋白表达水平均明显高于模型组(P0.05),功能性电刺激组、自主运动组和强迫运动组两两比较无显著性差异;上述5组SYP蛋白表达组间无显著性差异。结论:自主运动、强迫运动和功能性电刺激诱导的运动均可促进VD大鼠学习记忆能力的恢复,其可能机制为运动训练促进海马区SYN、PSD-95、MAP-2、TAU蛋白表达,改善海马区突触可塑性。