Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks

Purpose

During needle interventions, successful automated detection of the needle immediately after insertion is necessary to allow the physician identify and correct any misalignment of the needle and the target at early stages, which reduces needle passes and improves health outcomes.

Methods

We present a novel approach to localize partially inserted needles in 3D ultrasound volume with high precision using convolutional neural networks. We propose two methods based on patch classification and semantic segmentation of the needle from orthogonal 2D cross-sections extracted from the volume. For patch classification, each voxel is classified from locally extracted raw data of three orthogonal planes centered on it. We propose a bootstrap resampling approach to enhance the training in our highly imbalanced data. For semantic segmentation, parts of a needle are detected in cross-sections perpendicular to the lateral and elevational axes. We propose to exploit the structural information in the data with a novel thick-slice processing approach for efficient modeling of the context.

Results

Our introduced methods successfully detect 17 and 22 G needles with a single trained network, showing a robust generalized approach. Extensive ex-vivo evaluations on datasets of chicken breast and porcine leg show 80 and 84% F1-scores, respectively. Furthermore, very short needles are detected with tip localization errors of less than 0.7 mm for lengths of only 5 and 10 mm at 0.2 and 0.36 mm voxel sizes, respectively.

Conclusion

Our method is able to accurately detect even very short needles, ensuring that the needle and its tip are maximally visible in the visualized plane during the entire intervention, thereby eliminating the need for advanced bi-manual coordination of the needle and transducer.

     

Usefulness of real-time contrast-enhanced ultrasound guided coaxial needle biopsy for focal liver lesions

Purpose

To evaluate the utility of real-time contrast-enhanced ultrasound (CEUS)-guided coaxial needle biopsies for focal liver lesions (FLL) that were inconspicuous or could not be accurately identified the active site on B-mode ultrasound (US).

Materials and methods

This prospective study included 76 patients who had CEUS-guided coaxial needle biopsies for FLL between December 2015 and June 2017. We recorded characteristics of target lesions. We evaluated conspicuity of target lesions and accuracy of identifying the active site of target lesions on B-mode US and CEUS using a 5-point scale. Patients were divided into three groups, and analyzed according to body mass index (BMI). Based on the final diagnosis, the diagnostic performance was evaluated.

Results

The mean size and depth of target lesions were 41.5 ± 28.5 and 47.9 ± 18.9 mm on CEUS, respectively. In arterial phase, the enhanced pattern of target lesions varied. The conspicuity of target lesions and accuracy of identifying the active site of target lesions was significantly improved on CEUS compared to B-mode US (p < 0.05). The three BMI groups had significant differences in conspicuity of target lesions after using CEUS (p < 0.05). The high BMI group had a greater change in conspicuity of lesions compared to the normal BMI group or the low BMI group (p < 0.05). The sensitivity, specificity, and accuracy of this technique for the diagnosis of FLL were 92.8%, 100%, and 93.4%, respectively.

Conclusion

Real-time CEUS-guided coaxial needle biopsy can be very useful for FLL that are inconspicuous or cannot be accurately identified the active site on B-mode US.

     

Stradx: real-time acquisition and visualization of freehand three-dimensional ultrasound

Conventional freehand three-dimensional (3-D) ultrasound is a multi-stage process. First, the clinician scans the area of interest. Next, the ultrasound data is used to construct a 3-D voxel array, which can then be visualized by, for example, any-plane slicing. The strict separation of data acquisition and visualization disturbs the interactive nature of the ultrasound examination. Furthermore, some systems require the clinician to wait for an unacceptable amount of time while the voxel array is constructed. In this paper, we describe a novel freehand 3-D ultrasound system which allows accurate acquisition of the raw data and immediate visualization of arbitrary slices through the data. Minimal processing separates the acquisition and visualization processes: in particular, at no stage is a voxel array constructed. Instead, the standard graphics hardware found inside most desktop computers is exploited to synthesize arbitrary slices directly from the raw B-scans.     

实时超声弹性成像评价离体牛肝组织内射频消融灶的实验研究

目的探讨实时超声弹性成像评价离体牛肝组织内射频消融灶的可行性。方法取新鲜牛肝1只,切为6块作为形成射频消融灶的材料。采用RITA1500型射频治疗仪分别对6块牛肝组织行射频消融24次,按消融灶深度将实验分成3组：深1cm、3cm、5cm组,每组8个消融灶。消融结束后分别采用常规二维超声及实时超声弹性成像测量消融灶最大上下径、前后径及左右径,按椭球体公式计算消融灶体积,并与大体标本测量的体积进行比较。结果消融后实时超声弹性图像能显示16个消融灶呈均匀蓝色,与周边正常组织分界清楚。深1cm组二维超声,实时超声弹性成像和大体标本测量的消融灶平均体积分别为（514．8±63．4）mm^3,（748．1±98．1）mm^3,（766．3±43．2）mm^3。深3cm组3种方法测量的消融灶平均体积分别为（498．1±154．9）mm^3,（694．4±48．1）mm^3,（707．5±25．0）mm^3。深1cm与深3cm组二维超声、实时超声弹性成像与大体标本测量结果比较差异有统计学意义（P〈0．05）,二维超声无法准确估计消融灶的真实大小;而实时超声弹性成像能很好地反应深1cm和深3cm组消融灶的真实大小,其与大体标本测量体积之间具有较好的相关性（r=0．6064,P〈0．05）。随着消融灶深度的增加,实时超声弹性成像获取消融灶图像的难度增大,当消融灶深度达到5cm时,该技术难以获取消融灶的弹性图像。结论实时超声弹性成像有望成为评价肝组织内射频消融灶形态及大小的有效方法,但应用时对消融灶深度有一定限制。     

Biplane X-ray angiograms, intravascular ultrasound, and 3D visualization of coronary vessels

The technology for determination of the 3D vascular tree and quantitative characterization of the vessel lumen and vessel wall has become available. With this technology, cardiologists will no longer rely primarily on visual inspection of coronary angiograms but use sophisticated modeling techniques combining images from various modalities for the evaluation of coronary artery disease and the effects of treatment. Techniques have been developed which allow the calculation of the imaging geometry and the 3D position of the vessel centerlines of the vascular tree from biplane views without a calibration object, i.e., from the images themselves, removing the awkwardness of moving the patient to obtain 3D information. With the geometry and positional information, techniques for reconstructing the vessel lumen can now be applied that provide more accurate estimates of the area and shape of the vessel lumen. In conjunction with these developments, techniques have been developed for combining information from intravascular ultrasound images with the information obtained from angiography. The combination of these technologies will yield a more comprehensive characterization and understanding of coronary artery disease and should lead to improved and perhaps less invasive patient care.     

静脉留置针在超声造影中的应用及护理

超声造影堪称超声发展的第三次革命，超声造影通过外周静脉超声造影剂显示观察区微血管的动态显像，鉴别不同病理组织的造影专用成像技术的开发大大提高了超声在临床上的价值，由于造影剂为磷脂六氟化硫微泡制剂，平均微泡直径约2．5μm，为避免过细注射针对微泡的破坏，达到高标准的超声造影成像，现探讨在静脉留置针以及给药过程的护理技术和技巧。     

静脉留置针在超声造影中的应用及护理

超声造影堪称超声发展的第三次革命,超声造影通过外周静脉超声造影剂显示观察区微血管的动态显像,鉴别不同病理组织的造影专用成像技术的开发大大提高了超声在临床上的价值,由于造影剂为磷脂六氟化硫微泡制剂,平均微泡直径约2.5μm,为避免过细注射针对微泡的破坏,达到高标准的超声造影成像,现探讨在静脉留置针以及给药过程的护理技术和技巧。1资料与方法1.1资料本组病人77例,其中男45例,女32例;年龄21岁～78岁;肝占位52例,胆囊癌4例,其中伴有肝转移者3例,胆囊平滑肌瘤1例,肾占位6例、淋巴瘤12例、乳腺占位2例。1.2穿刺材料20G静脉留置针一…     

Improved detection and biopsy of solid liver lesions using pulse-inversion ultrasound scanning and contrast agent infusion

The purpose of this study was to assess the ability of pulse-inversion ultrasound (US) scanning (PIUS), combined with an IV contrast agent, to detect malignant liver lesions and its impact on patient management (resectability). Additionally, to determine the feasibility of US-guided biopsy of new PIUS-findings at the same session. A total of 30 patients with known or clinically suspected cancer underwent conventional B-mode scanning and PIUS with IV-administered contrast agent. The number of liver metastases in the right and the left liver lobe, respectively, was recorded. All patients with additional findings by PIUS underwent US-guided biopsy. PIUS provided additional information in 18 patients (60%); of these, 13 (43%) had additional metastases. Of 19 patients found resectable by conventional US, 9 (47%) were considered inoperable using PIUS supported by biopsies. Biopsies of additional findings were performed in 17 of 18 patients. All biopsies of additional findings confirmed malignancy. PIUS with an IV contrast agent increased the ability to detect liver metastases compared to conventional US scanning. The technique had a high impact on patient management. The results showed that PIUS-guided biopsy was possible. PIUS with IV contrast will undoubtedly become an important diagnostic tool in the evaluation of patients with metastatic liver disease.     

Integrated endoscope for real-time 3D ultrasound imaging and hyperthermia: feasibility study

The goal of this research is to determine the feasibility of using a single endoscopic probe for the combined purpose of real-time 3D (RT3D) ultrasound imaging of a target organ and the delivery of ultrasound therapy to facilitate the absorption of compounds for cancer treatment. Recent research in ultrasound therapy has shown that ultrasound-mediated drug delivery improves absorption of treatments for prostate, cervical and esophageal cancer. The ability to combine ultrasound hyperthermia and 3D imaging could improve visualization and targeting of cancerous tissues. In this study, numerical modeling and experimental measurements were developed to determine the feasibility of combined therapy and imaging with a 1 cm diameter endoscopic RT3D probe with 504 transmitters and 252 receive channels. This device operates at 5 MHz and has a 6.3 mm x 6.3 mm aperture to produce real time 3D pyramidal scans of 60-120 degrees incorporating 64 x 64 = 4096 image lines at 30 volumes/sec interleaved with a 3D steerable therapy beam. A finite-element mesh was constructed with over 128,000 elements in LS-DYNA to simulate the induced temperature rise from our transducer with a 3 cm deep focus in tissue. Quarter-symmetry of the transducer was used to reduce mesh size and computation time. Based on intensity values calculated in Field II using the transducer's array geometry, a minimum I(SPTA) of 3.6 W/cm2 is required from our endoscope probe in order to induce a temperature rise of 4 degrees C within five minutes. Experimental measurements of the array's power output capabilities were conducted using a PVDF hydrophone placed 3 cm away from the face of the transducer in a watertank. Using a PDA14 Signatec data acquisition board to capture full volumes of transmitted ultrasound data, it was determined that the probe can presently maintain intensity values up to 2.4 W/cm2 over indefinite times for therapeutic applications combined with intermittent 3D scanning to maintain targeting. These values were acquired using 8 cycle bursts at a prf of 6 kHz. Ex vivo heating experiments of excised pork tissue yielded a maximum temperature rises of 2.3 degrees C over 5 minutes of ultrasound exposure with an average rise of 1.8 +/- 0.2 degrees C over 5 trials. Modifications to the power supply and transducer array may enable us to reach the higher intensities required to facilitate drug delivery therapy.     

Robust real-time 3D respiratory motion detection using time-of-flight cameras

Purpose  Respiratory motion of the patient during data acquisition causes artifacts in the field of emission or computed tomography. Respiratory gating allows to track and correct these artifacts. Materials and methods  In this paper, we present a system that uses the fairly new and off-the-shelf time-of-flight (ToF) technology to compute a dense estimate of the three-dimensional respiratory motion of a patient. The work is characterized by three key contributions. The first is the employment of ToF sensors. Using ToF sensors, it is feasible to acquire a dense 3D surface model of the chest and abdomen of the patient with more than 15 frames per second. The second contribution is an algorithm to derive a surface representation which enables the estimation of the 3D respiratory motion of the patient, which is sufficient to compute 1D breathing signals for scalable specific regions of interest like chest and abdomen. The proposed data-driven algorithm models the chest and abdomen three-dimensionally by fitting distinct planes to different regions of the torso of the patient. The third contribution is the possibility to derive a sub-millimeter accurate 1D respiratory motion signal by observing the displacement of each plane. Results  Our ToF modeling approach enables marker less, real-time, 3D tracking of patient respiratory motion with an accuracy of 0.1 mm. Conclusion  Thus, our approach provides 1D breathing signals for scalable anatomical regions of interest with sufficient accuracy for artifact reduction in SPECT or X-ray angiography.     

Registration of freehand 3D ultrasound and magnetic resonance liver images

We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a "bronze standard" registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the "bronze standard" and this is improved to 3.6 mm after running the intensity-based algorithm.     

Real-time ultrasound visualization of tongue movement during swallowing

Using noninvasive real-time ultrasound, tongue movement was visualized during single swallowing in eight normal subjects and one neurologically impaired patient with dysphagia and chronic aspiration. In normals, a clearly defined muscular wave of the tongue, traveling at approximately 15 cm/sec, carried a 5-cc test water bolus posteriorly. In the patient who had 12th cranial nerve weakness, there was complete absence of normal tongue activity and no midtongue bolus formation or transmission.     

3D and 4D ultrasound in fetal cardiac scanning: a new look at the fetal heart.

Over the last decade we have been witness to a burgeoning literature on three-dimensional (3D) and four-dimensional (4D) ultrasound-based studies of the fetal cardiovascular system. Recent advances in the technology of 3D/4D ultrasound systems allow almost real-time 3D/4D fetal heart scans. It appears that 3D/4D ultrasound in fetal echocardiography may make a significant contribution to interdisciplinary management team consultation, health delivery systems, parental counseling, and professional training.Our aim is to review the state of the art in 3D/4D fetal echocardiography through the literature and index cases of normal and anomalous fetal hearts.     

Illustrative visualization of 3D planning models for augmented reality in liver surgery

Purpose  

Augmented reality (AR) obtains increasing acceptance in the operating room. However, a meaningful augmentation of the surgical view with a 3D visualization of planning data which allows reliable comparisons of distances and spatial relations is still an open request.     

复合三维超声成像技术在肝脏肿瘤的诊断价值

目的研究复合三维超声成像技术在肝肿瘤诊断中的临床价值。方法应用GE Voluson730三维超声成像系统对50个肝占位性病灶进行三维组织成像、三维血管成像及三维肿瘤体积精测。结果①彩色多普勒显示血管的断面图像，表现为点状、短棒状血管形态。彩色多普勒与三维超声成像技术的结合更易于显示肿瘤血管“抱球征”、“提篮征”、“周围血管绕行征”的立体构型。②三维超声对肿瘤体积的测量较二维超声精确。结论复合三维超声成像技术与传统的二维超声成像相比，有许多独特的优点，能提供更加丰富的诊断信息。     

2D与4D超声检测婴儿髋关节临床研究

目的通过运用2D与4D超声对婴儿髋关节进行检查,重点观察婴儿髋臼骨顶形态在4D超声图像中的变化情况,评价4D超声对婴儿髋关节的诊断价值。方法随机对58例婴儿的双侧髋关节进行2D超声检查定位,再采用4D超声检查。其中,男34例,女24例,年龄最小1d,最大3个月。所有受检者均检查双侧髋关节,在2D超声检查的基础上,进行4D超声检查。根据髋臼骨顶的4D超声图像形态,将髋关节分为优、良、差三个级别。结果共获得116张2D髋关节超声图像,获得114张4D髋关节超声图像,其中,1个Ⅰb型和1个Ⅱa型髋关节,因小儿哭闹躁动而无法获得4D图像。在114张4D髋关节超声图像中,74个髋关节为优,其中Ⅰa型5个,Ⅰb型40个,Ⅱa型29个;38个髋关节为良,其中Ⅰb型15个,Ⅱa型22个,D型1个;2个髋关节为差,其中Ⅰb型1个,Ⅲ型1个。所有优级的髋关节中不含有D型、Ⅲ型髋关节。结论应用4D超声检查,可以弥补2D超声扫描中无法获得的髋臼骨顶部立体形态图像,通过对髋关节立体空间的显示,更加直观地评价髋臼的发育状况,从而对发育性髋关节脱位(DDH)作出更为可靠的诊断。婴儿髋关节4D超声检查是一种很有发展前途的技术,如果结合2D超声检查,将对DDH的诊断以及治疗中的随访观察发挥其独特的价值。     

Progress in ring array transducers for real-time 3D ultrasound guidance of cardiac interventional devices

As a treatment for aortic stenosis, several companies have recently introduced prosthetic heart valves designed to be deployed through a catheter using an intravenous or transapical approach. This procedure can either take the place of open heart surgery with some ofthe devices or delay it with others. Real-time 3D ultrasound could enable continuous monitoring of these structures before, during and after deployment. We have developed a 2D ring array integrated with a 30 French catheter that is used for transapical prosthetic heart valve implantation. The transducer array was built using three 46 cm long flex circuits from MicroConnex (Snoqualmie, WA) which terminate in an interconnect that plugs directly into our system cable; thus, no cable soldering is required. This transducer consists of 210 elements at 0.157 mm interelement spacing and operates at 5 MHz. Average measured element bandwidth was 26% and average round-trip 50 ohm insertion loss was -58.1 dB after correcting for diffractive losses. The transducer was wrapped around the 1 cm diameter lumen of a heart-valve deployment catheter. Prosthetic heart valve images were obtained in water-tank studies.