Stereoscopic augmented reality for laparoscopic surgery

Background

Conventional laparoscopes provide a flat representation of the three-dimensional (3D) operating field and are incapable of visualizing internal structures located beneath visible organ surfaces. Computed tomography (CT) and magnetic resonance (MR) images are difficult to fuse in real time with laparoscopic views due to the deformable nature of soft-tissue organs. Utilizing emerging camera technology, we have developed a real-time stereoscopic augmented-reality (AR) system for laparoscopic surgery by merging live laparoscopic ultrasound (LUS) with stereoscopic video. The system creates two new visual cues: (1) perception of true depth with improved understanding of 3D spatial relationships among anatomical structures, and (2) visualization of critical internal structures along with a more comprehensive visualization of the operating field.

Methods

The stereoscopic AR system has been designed for near-term clinical translation with seamless integration into the existing surgical workflow. It is composed of a stereoscopic vision system, a LUS system, and an optical tracker. Specialized software processes streams of imaging data from the tracked devices and registers those in real time. The resulting two ultrasound-augmented video streams (one for the left and one for the right eye) give a live stereoscopic AR view of the operating field. The team conducted a series of stereoscopic AR interrogations of the liver, gallbladder, biliary tree, and kidneys in two swine.

Results

The preclinical studies demonstrated the feasibility of the stereoscopic AR system during in vivo procedures. Major internal structures could be easily identified. The system exhibited unobservable latency with acceptable image-to-video registration accuracy.

Conclusions

We presented the first in vivo use of a complete system with stereoscopic AR visualization capability. This new capability introduces new visual cues and enhances visualization of the surgical anatomy. The system shows promise to improve the precision and expand the capacity of minimally invasive laparoscopic surgeries.     

Breast cancer surgery with augmented reality

Introduction: Innovations in 3D spatial technology and augmented reality imaging driven by digital high-tech industrial science have accelerated experimental advances in breast cancer imaging and the development of medical procedures aimed to reduce invasiveness. Presentation of case: A 57-year-old post-menopausal woman presented with screen-detected left-sided breast cancer. After undergoing all staging and pre-operative studies the patient was proposed for conservative breast surgery with tumor localization. During surgery, an experimental digital and non-invasive intra-operative localization method with augmented reality was compared with the standard pre-operative localization with carbon tattooing (institutional protocol). The breast surgeon wearing an augmented reality headset (Hololens) was able to visualize the tumor location projection inside the patient’s left breast in the usual supine position. Discussion: This work describes, to our knowledge, the first experimental test with a digital non-invasive method for intra-operative breast cancer localization using augmented reality to guide breast conservative surgery. In this case, a successful overlap of the previous standard pre-operative marks with carbon tattooing and tumor visualization inside the patient’s breast with augmented reality was obtained. Conclusion: Breast cancer conservative guided surgery with augmented reality can pave the way for a digital non-invasive method for intra-operative tumor localization.     

增强现实、混合现实在泌尿外科导航手术中的应用

正目前的医学成像设备可以创建单个患者的疾病图像,将图像数据重建后的三维图像可以指导诊断和制定手术预案,在术中提供详细的导航指引。这种计算机辅助手术(computer assisted surgery,CAS)可以提高疾病诊断的准确性,同时提高手术的精准性(图1)。增强现实(augmented reality,AR)和混合现实(mixed reality,MR)是CAS的一种医学影像的应用,AR是虚拟信息叠加在现实事物上,相当于真实世界与数字化信息的结合,例     

Real world usability analysis of two augmented reality headsets in visceral surgery

Recent developments in the field of augmented reality (AR) have enabled new use cases in surgery. Initial set‐up of an appropriate infrastructure for maintaining an AR surgical workflow requires investment in appropriate hardware. We compared the usability of the Microsoft HoloLens and Meta 2 head mounted displays (HMDs). Fifteen medicine students tested each device and were questioned with a variant of the System Usability Scale (SUS). Two surgeons independently tested the devices in an intraoperative setting. In our adapted SUS, ergonomics, ease of use, and visual clarity of the display did not differ significantly between HMD groups. The field of view (FOV) was smaller in the Microsoft HoloLens than the Meta 2 and significantly more study subjects (80% vs. 13.3%; P < 0.001) felt limited through the FOV. Intraoperatively, decreased mobility due to the necessity of an AC adapter and additional computing device for the Meta 2 proved to be limiting. Object stability was rated superior in the Microsoft HoloLens than the Meta 2 by our surgeons and lead to increased use. In summary, after examination of the Meta 2 and the Microsoft HoloLens, we found key advantages in the Microsoft HoloLens which provided palpable benefits in a surgical setting.     

Current status of augmented reality in cerebrovascular surgery: a systematic review

Neurosurgical Review - Augmented reality (AR) is an adjuvant tool in neuronavigation to improve spatial and anatomic understanding. The present review aims to describe the current status of...     

Deformable three-dimensional model architecture for interactive augmented reality in minimally invasive surgery

Background

Surgical procedures have undergone considerable advancement during the last few decades. More recently, the availability of some imaging methods intraoperatively has added a new dimension to minimally invasive techniques. Augmented reality in surgery has been a topic of intense interest and research.

Methods

Augmented reality involves usage of computer vision algorithms on video from endoscopic cameras or cameras mounted in the operating room to provide the surgeon additional information that he or she otherwise would have to recognize intuitively. One of the techniques combines a virtual preoperative model of the patient with the endoscope camera using natural or artificial landmarks to provide an augmented reality view in the operating room. The authors?? approach is to provide this with the least number of changes to the operating room. Software architecture is presented to provide interactive adjustment in the registration of a three-dimensional (3D) model and endoscope video.

Results

Augmented reality including adrenalectomy, ureteropelvic junction obstruction, and retrocaval ureter and pancreas was used to perform 12 surgeries. The general feedback from the surgeons has been very positive not only in terms of deciding the positions for inserting points but also in knowing the least change in anatomy.

Conclusions

The approach involves providing a deformable 3D model architecture and its application to the operating room. A 3D model with a deformable structure is needed to show the shape change of soft tissue during the surgery. The software architecture to provide interactive adjustment in registration of the 3D model and endoscope video with adjustability of every 3D model is presented.     

Extended,virtual and augmented reality in thoracic surgery: a systematic review

 Open in a separate windowOBJECTIVESExtended reality (XR), encompassing both virtual reality (VR) and augmented reality, allows the user to interact with a computer-generated environment based on reality. In essence, the immersive nature of VR and augmented reality technology has been warmly welcomed in all aspects of medicine, gradually becoming increasingly feasible to incorporate into everyday practice. In recent years, XR has become increasingly adopted in thoracic surgery, although the extent of its applications is unclear. Here, we aim to review the current applications of XR in thoracic surgery.METHODSA systematic database search was conducted of original articles that explored the use of VR and/or augmented reality in thoracic surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to December 2020.RESULTSOur search yielded 1494 citations, of which 21 studies published from 2007 to 2019 were included in this review. Three main areas were identified: (i) the application of XR in thoracic surgery training; (ii) preoperative planning of thoracic procedures; and (iii) intraoperative assistance. Overall, XR could produce progression along the learning curve, enabling trainees to reach acceptable standards before performing in the operating theatre. Preoperatively, through the generation of 3D-renderings of the thoracic cavity and lung anatomy, VR increases procedural accuracy and surgical confidence through familiarization of the patient’s anatomy. XR-assisted surgery may have therapeutic use particularly for complex cases, where conventional methods would yield inadequate outcomes due to inferior accuracy.CONCLUSIONXR represents a salient step towards improving thoracic surgical training, as well as enhancing preoperative planning and intraoperative guidance.     

腔镜技术在甲状腺手术中的应用选择

随着腔镜技术的发展及甲状腺手术技巧的不断完善,腔镜技术在甲状腺手术中的应用越来越广泛。然而腔镜甲状腺手术在适应证选择及如何合理选择腔镜技术在实际应用中仍存在争议。腔镜技术为甲状腺疾病患者带来的价值是不可否认的,但严格把握适应证、规范的治疗策略、成熟的操作技巧以及根据患者病情提供个体化治疗仍为关键。     

Intra-operative heparin release during lung surgery.

OBJECTIVES: Heparin has long been thought to be biosynthesized and stored in the granules of mast cells that are most prevalent in the lungs and gastro-intestinal tract. In response to stimuli such inflammation and trauma, mast cells degranulate and consequently release heparin. This prospective study was designed to investigate if operative trauma during lung mobilization could enhance heparin release into both the pulmonary and systemic circulations. METHODS: Prospective investigations and data collection were carried out on 34 patients undergoing elective thoracotomies for 19 patients with chronic inflammatory disease and 15 with lung carcinoma. Heparin assay using the high performance liquid chromatography method was carried out on four blood samples from each patient. Sample 1 was taken pre-operatively from the radial artery. Intra-operatively following lung mobilization and prior to excision, sample 2 was taken from the draining pulmonary vein and at the same time, sample 3 from the radial artery. Postoperatively, the next morning, sample 4 was taken from the radial artery. RESULTS: The mean values for serum heparin levels in pg/ml of samples 1-4 were found to be 205.1 (SD+/-282.1), 366.0 (SD+/-371.7), 337.2 (SD+/-225.3) and 250.8 (SD+/-282.2), respectively. These results show that intraoperative serum heparin levels (samples 2 and 3) are significantly higher (P = 0.0016, P = 0.0014, respectively) than pre-operative values (sample 1). The difference between sample 2 (pulmonary) and sample 3 (systemic circulation) was not significant (P = 0.6508). Although postoperative heparin levels (sample 4) were found to be higher than pre-operative values, yet it was not statistically significant (P = 0.1340). The mean of pre-operative heparin levels in patients with lung carcinoma and inflammatory diseases were 136.2 (SD+/-62.6) and 259.4 (SD+/-368.3), respectively. Intra-operatively, heparin levels increased to 260.9 (SD+/-139.7) and 449 (SD+/-470.7), respectively. These results suggest that the mean heparin level for patients with inflammatory lung diseases was higher than that for carcinoma patients. CONCLUSIONS: Within the context of lung surgery for carcinoma or inflammatory diseases, it appears that operative trauma enhances heparin release into both the pulmonary and systemic circulations, possibly through pulmonary mast cell degranulation. Thus, an episodic auto anti-coagulant effect is established during the course of surgery. Such findings may partly provide an understanding of the excessive bleeding encountered during some thoracotomies and the recognized reduced incidence of thrombo-embolic complications among thoracic surgical patients. Should an unexplained bleeding occur during the course of surgery, an excess of heparin release is recommended to be kept in mind as a possible cause.