Extended,virtual and augmented reality in thoracic surgery: a systematic review

 Open in a separate windowOBJECTIVESExtended reality (XR), encompassing both virtual reality (VR) and augmented reality, allows the user to interact with a computer-generated environment based on reality. In essence, the immersive nature of VR and augmented reality technology has been warmly welcomed in all aspects of medicine, gradually becoming increasingly feasible to incorporate into everyday practice. In recent years, XR has become increasingly adopted in thoracic surgery, although the extent of its applications is unclear. Here, we aim to review the current applications of XR in thoracic surgery.METHODSA systematic database search was conducted of original articles that explored the use of VR and/or augmented reality in thoracic surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to December 2020.RESULTSOur search yielded 1494 citations, of which 21 studies published from 2007 to 2019 were included in this review. Three main areas were identified: (i) the application of XR in thoracic surgery training; (ii) preoperative planning of thoracic procedures; and (iii) intraoperative assistance. Overall, XR could produce progression along the learning curve, enabling trainees to reach acceptable standards before performing in the operating theatre. Preoperatively, through the generation of 3D-renderings of the thoracic cavity and lung anatomy, VR increases procedural accuracy and surgical confidence through familiarization of the patient’s anatomy. XR-assisted surgery may have therapeutic use particularly for complex cases, where conventional methods would yield inadequate outcomes due to inferior accuracy.CONCLUSIONXR represents a salient step towards improving thoracic surgical training, as well as enhancing preoperative planning and intraoperative guidance.     

虚拟现实技术在脊柱外科的应用

人体脊柱解剖结构复杂,对手术操作技术要求高,年轻医师熟练掌握手术技巧需要长期反复的训练。传统的“见一、做一、教一”的师徒带教模式虽然仍是主流,但已无法适应医学的快速发展,术者在手术操作过程中兼顾帮带,无形中亦增加了手术风险[1-2]。尸体训练是过去几十年来实际手术的最佳替代[3],其触觉反馈真实,保真度高,对解剖结构、位置认识直观,但尸体资源紧缺,且容易传播疾病;此外,新鲜尸体的使用时间有限,无法反复练习[4-5],甚至在--些国家还存在伦理等问题[6]。     

Current status of augmented reality in cerebrovascular surgery: a systematic review

Neurosurgical Review - Augmented reality (AR) is an adjuvant tool in neuronavigation to improve spatial and anatomic understanding. The present review aims to describe the current status of...     

Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review

Background & aimUtilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice.Methods & resultsPRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits.ConclusionWhilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons’ specific needs should increase uptake.     

虚拟现实技术在脊柱外科中的应用现状

正虚拟现实(virtual reality,VR)技术在20世纪60年代由Sutherland提出[1],是一种以计算机生成的3D图像数据文件为基础,通过感官聚焦或者剥夺装置,使操作者能够沉浸在设备传送的预编程数字化世界中的数字技术。VR技术有着沉浸性、交互性、构想性三个重要基本特征[2]。VR技术常与增强现实(augmented reality,AR)技术和混合     

Augmented and virtual reality in spine surgery,current applications and future potentials

BACKGROUND CONTEXTThe field of artificial intelligence (AI) is rapidly advancing, especially with recent improvements in deep learning (DL) techniques. Augmented (AR) and virtual reality (VR) are finding their place in healthcare, and spine surgery is no exception. The unique capabilities and advantages of AR and VR devices include their low cost, flexible integration with other technologies, user-friendly features and their application in navigation systems, which makes them beneficial across different aspects of spine surgery. Despite the use of AR for pedicle screw placement, targeted cervical foraminotomy, bone biopsy, osteotomy planning, and percutaneous intervention, the current applications of AR and VR in spine surgery remain limited.PURPOSEThe primary goal of this study was to provide the spine surgeons and clinical researchers with the general information about the current applications, future potentials, and accessibility of AR and VR systems in spine surgery.STUDY DESIGN/SETTINGWe reviewed titles of more than 250 journal papers from google scholar and PubMed with search words: augmented reality, virtual reality, spine surgery, and orthopaedic, out of which 89 related papers were selected for abstract review. Finally, full text of 67 papers were analyzed and reviewed.METHODSThe papers were divided into four groups: technological papers, applications in surgery, applications in spine education and training, and general application in orthopaedic. A team of two reviewers performed paper reviews and a thorough web search to ensure the most updated state of the art in each of four group is captured in the review.RESULTSIn this review we discuss the current state of the art in AR and VR hardware, their preoperative applications and surgical applications in spine surgery. Finally, we discuss the future potentials of AR and VR and their integration with AI, robotic surgery, gaming, and wearables.CONCLUSIONSAR and VR are promising technologies that will soon become part of standard of care in spine surgery.     

Use of augmented reality for image-guided spine procedures

European Spine Journal - Because of its ability to superimpose imaging data on a patient, while anchoring the user’s view to the immediate surroundings, augmented reality (AR) has the...     

增强现实、混合现实在泌尿外科导航手术中的应用

正目前的医学成像设备可以创建单个患者的疾病图像,将图像数据重建后的三维图像可以指导诊断和制定手术预案,在术中提供详细的导航指引。这种计算机辅助手术(computer assisted surgery,CAS)可以提高疾病诊断的准确性,同时提高手术的精准性(图1)。增强现实(augmented reality,AR)和混合现实(mixed reality,MR)是CAS的一种医学影像的应用,AR是虚拟信息叠加在现实事物上,相当于真实世界与数字化信息的结合,例     

Breast cancer surgery with augmented reality

Introduction: Innovations in 3D spatial technology and augmented reality imaging driven by digital high-tech industrial science have accelerated experimental advances in breast cancer imaging and the development of medical procedures aimed to reduce invasiveness. Presentation of case: A 57-year-old post-menopausal woman presented with screen-detected left-sided breast cancer. After undergoing all staging and pre-operative studies the patient was proposed for conservative breast surgery with tumor localization. During surgery, an experimental digital and non-invasive intra-operative localization method with augmented reality was compared with the standard pre-operative localization with carbon tattooing (institutional protocol). The breast surgeon wearing an augmented reality headset (Hololens) was able to visualize the tumor location projection inside the patient’s left breast in the usual supine position. Discussion: This work describes, to our knowledge, the first experimental test with a digital non-invasive method for intra-operative breast cancer localization using augmented reality to guide breast conservative surgery. In this case, a successful overlap of the previous standard pre-operative marks with carbon tattooing and tumor visualization inside the patient’s breast with augmented reality was obtained. Conclusion: Breast cancer conservative guided surgery with augmented reality can pave the way for a digital non-invasive method for intra-operative tumor localization.     

Stereoscopic augmented reality for laparoscopic surgery

Background

Conventional laparoscopes provide a flat representation of the three-dimensional (3D) operating field and are incapable of visualizing internal structures located beneath visible organ surfaces. Computed tomography (CT) and magnetic resonance (MR) images are difficult to fuse in real time with laparoscopic views due to the deformable nature of soft-tissue organs. Utilizing emerging camera technology, we have developed a real-time stereoscopic augmented-reality (AR) system for laparoscopic surgery by merging live laparoscopic ultrasound (LUS) with stereoscopic video. The system creates two new visual cues: (1) perception of true depth with improved understanding of 3D spatial relationships among anatomical structures, and (2) visualization of critical internal structures along with a more comprehensive visualization of the operating field.

Methods

The stereoscopic AR system has been designed for near-term clinical translation with seamless integration into the existing surgical workflow. It is composed of a stereoscopic vision system, a LUS system, and an optical tracker. Specialized software processes streams of imaging data from the tracked devices and registers those in real time. The resulting two ultrasound-augmented video streams (one for the left and one for the right eye) give a live stereoscopic AR view of the operating field. The team conducted a series of stereoscopic AR interrogations of the liver, gallbladder, biliary tree, and kidneys in two swine.

Results

The preclinical studies demonstrated the feasibility of the stereoscopic AR system during in vivo procedures. Major internal structures could be easily identified. The system exhibited unobservable latency with acceptable image-to-video registration accuracy.

Conclusions

We presented the first in vivo use of a complete system with stereoscopic AR visualization capability. This new capability introduces new visual cues and enhances visualization of the surgical anatomy. The system shows promise to improve the precision and expand the capacity of minimally invasive laparoscopic surgeries.     

Microsurgery simulators in virtual reality: review

Surgical training is undergoing a rapid transformation, which has been influenced by advances in computer modeling. Increased pressure to reduce the use of animals in technical training has led to a new approach in teaching microsurgery. This new technology may prove to be a cost-effective, portable, and nonhazardous way forward in microsurgical training. This paper reviews the current state of available technological models used in microsurgical training. In doing so, we review articles from the latest journals and authenticated Internet websites to compare and contrast these various methods. Finally, we look at the specific technique that has potential impact on the future modeling of microsurgical techniques.     

虚拟现实、增强现实和混合现实技术在胸壁肿瘤切除及重建手术治疗中的应用

目的探讨虚拟现实(VR)、增强现实(AR)及混合现实(MR)技术在胸壁肿瘤切除及重建精确手术治疗中的意义。 方法对2例胸壁肿瘤患者行胸部CT薄层扫描、胸部MR平扫＋增强检查,获得CT和MRI的医学数字成像和通信(DICOM)数据,输入计算机进行电脑数据3D模型建立,而后实体打印。同时建立虚拟模型及手术预演,术中在裸眼和VR＋AR＋MR下决定手术切除范围,并结合模型对比,扩大切除胸壁肿瘤后行胸壁重建手术。 结果2例患者手术顺利,术后恢复良好出院。VR＋AR＋MR技术更能精确手术切缘、切除肿瘤,并对胸壁后组织器官有更精准的了解。 结论VR＋AR＋MR技术对手术切缘及胸壁后组织器官的了解更精准,为3D打印材料置入提供了技术保障。     

Intra-operative support via augmented reality techniques in endoscopic surgery.

Computer assisted operation planning systems are gaining increasing recognition in the field of surgery. These systems offer new possibilities for preparing an intervention, with the goal of reducing the amount of expensive operating-room time required for the intervention. The safest and most effective surgical approach should always be selected, but it is often difficult to transfer the output of the planning system to the intra-operative situation so that the planning results can be considered during the actual intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore methods are being developed to bridge the gap between the external planning session and the intra-operative case: Augmented Reality (AR) techniques are used to overlay preoperative scanned image data, as well as results of the planning session, on the operation field.     

Procedural virtual reality simulation in minimally invasive surgery

Background

Simulation of procedural tasks has the potential to bridge the gap between basic skills training outside the operating room (OR) and performance of complex surgical tasks in the OR. This paper provides an overview of procedural virtual reality (VR) simulation currently available on the market and presented in scientific literature for laparoscopy (LS), flexible gastrointestinal endoscopy (FGE), and endovascular surgery (EVS).

Methods

An online survey was sent to companies and research groups selling or developing procedural VR simulators, and a systematic search was done for scientific publications presenting or applying VR simulators to train or assess procedural skills in the PUBMED and SCOPUS databases.

Results

The results of five simulator companies were included in the survey. In the literature review, 116 articles were analyzed (45 on LS, 43 on FGE, 28 on EVS), presenting a total of 23 simulator systems. The companies stated to altogether offer 78 procedural tasks (33 for LS, 12 for FGE, 33 for EVS), of which 17 also were found in the literature review. Although study type and used outcomes vary between the three different fields, approximately 90 % of the studies presented in the retrieved publications for LS found convincing evidence to confirm the validity or added value of procedural VR simulation. This was the case in approximately 75 % for FGE and EVS.

Conclusions

Procedural training using VR simulators has been found to improve clinical performance. There is nevertheless a large amount of simulated procedural tasks that have not been validated. Future research should focus on the optimal use of procedural simulators in the most effective training setups and further investigate the benefits of procedural VR simulation to improve clinical outcome.