Applications of augmented and virtual reality in spine surgery and education: A review

As the complexity and minimally invasive nature of spine surgery continues to grow, so must the surgeon's ability to “view” and interact with the surgical field. Augmented reality (AR) provides a digital overlay of a real-world environment, helping the surgeon to visualize deep anatomic landmarks and surgical trajectory, such as for an osteotomy cut or pedicle screw. In contrast, virtual reality (VR) is an entirely digital environment that can be used for simulated surgeries or technical trainings without the need for a physical patient. Here we review the current clinical applications of AR and VR in spine surgery and education.     

Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review

Background & aimUtilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice.Methods & resultsPRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits.ConclusionWhilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons’ specific needs should increase uptake.     

Machine learning driven tools in orthopaedics and spine surgery: Hype or reality? Applications and perception of 31 physician opinions

This narrative review focusses on the applications and implementation of artificial intelligence (AI) in spine surgery. We present examples of machine learning (ML) and deep learning (DL) applications, illustrating the potential of AI-derived technologies in supporting orthopaedic surgery, clinical decision making and educational purposes. We also describe the clinical hurdles that AI applications face. Finally, we present results from a survey evaluating the opinions of orthopaedic surgeons concerning the future of implementing AI-driven tools into orthopaedic and spine surgery.     

Extended,virtual and augmented reality in thoracic surgery: a systematic review

 Open in a separate windowOBJECTIVESExtended reality (XR), encompassing both virtual reality (VR) and augmented reality, allows the user to interact with a computer-generated environment based on reality. In essence, the immersive nature of VR and augmented reality technology has been warmly welcomed in all aspects of medicine, gradually becoming increasingly feasible to incorporate into everyday practice. In recent years, XR has become increasingly adopted in thoracic surgery, although the extent of its applications is unclear. Here, we aim to review the current applications of XR in thoracic surgery.METHODSA systematic database search was conducted of original articles that explored the use of VR and/or augmented reality in thoracic surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to December 2020.RESULTSOur search yielded 1494 citations, of which 21 studies published from 2007 to 2019 were included in this review. Three main areas were identified: (i) the application of XR in thoracic surgery training; (ii) preoperative planning of thoracic procedures; and (iii) intraoperative assistance. Overall, XR could produce progression along the learning curve, enabling trainees to reach acceptable standards before performing in the operating theatre. Preoperatively, through the generation of 3D-renderings of the thoracic cavity and lung anatomy, VR increases procedural accuracy and surgical confidence through familiarization of the patient’s anatomy. XR-assisted surgery may have therapeutic use particularly for complex cases, where conventional methods would yield inadequate outcomes due to inferior accuracy.CONCLUSIONXR represents a salient step towards improving thoracic surgical training, as well as enhancing preoperative planning and intraoperative guidance.     

The role of 3D digital applications in manual therapy education – A scoping review

BackgroundCurrently, teaching methods for developing physical assessment and palpation skills in manual therapy is challenging for both learners and educators. 3D digital technologies such as virtual reality (VR) may facilitate and/or address these challenges. However, their current usage in improving learning outcomes in manual therapy education is unknown.MethodsThe following electronic databases were searched from Jan 2005 to April 2021: PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Physiotherapy Evidence Database (PEDro), Science Direct and Google Scholar. Two independent reviewers reviewed the articles to assess for eligibility. Studies of any methodology that investigated the use or application of the 3D digital applications were included in the review. Primary outcomes included any outcome related to learning based on the updated Blooms taxonomy. Narrative synthesis was used to synthesize data from the included studies.ResultsA total of 4 articles were included in the final review. The main findings were classified into the following key concepts: (1) perception of tissue stiffness, (2) improved student self-efficacy in performing MT techniques, and (3) barriers and facilitators for utilizing 3D technologies. All included outcomes met understanding, applying, analysing and evaluating levels of Blooms taxonomy; however, no outcomes met the ‘creating’ level of Bloom's taxonomy.DiscussionOur review found that there are no AR, VR or mixed applications that specifically serve the needs of MT education in relation to joint motion assessment, but applications are available that can be readily used or potentially adapted to train skills of tissue palpation.     

Deep learning in spine surgery

Deep learning is increasingly impactful for healthcare research and delivery. In orthopaedics, there has been a significant increase in the number of publications using deep learning for interpreting radiographs; however, there are only a few applications of deep learning specific to spine surgery. In this review, we discuss the potential of deep learning within spine surgery in four contexts: diagnosis (1), prognosis (2), patient management (3) and integration with virtual/augmented reality, robotic surgery and biomedical wearables (4). Additionally, we discuss current literature, future potentials and provide takeaways for clinicians who wish to apply deep learning in their research and practice.     

What is going on in augmented reality simulation in laparoscopic surgery?

Background  To prevent unnecessary errors and adverse results of laparoscopic surgery, proper training is of paramount importance. A safe way to train surgeons for laparoscopic skills is simulation. For this purpose traditional box trainers are often used, however they lack objective assessment of performance. Virtual reality laparoscopic simulators assess performance, but lack realistic haptic feedback. Augmented reality (AR) combines a virtual reality (VR) setting with real physical materials, instruments, and feedback. This article presents the current developments in augmented reality laparoscopic simulation. Methods  Pubmed searches were performed to identify articles regarding surgical simulation and augmented reality. Identified companies manufacturing an AR laparoscopic simulator received the same questionnaire referring to the features of the simulator. Results  Seven simulators that fitted the definition of augmented reality were identified during the literature search. Five of the approached manufacturers returned a completed questionnaire, of which one simulator appeared to be VR and was therefore not applicable for this review. Conclusion  Several augmented reality simulators have been developed over the past few years and they are improving rapidly. We recommend the development of AR laparoscopic simulators for component tasks of procedural training. AR simulators should be implemented in current laparoscopic training curricula, in particular for laparoscopic suturing training.     

Establishing a curriculum for the acquisition of laparoscopic psychomotor skills in the virtual reality environment

BackgroundThe unique psychomotor skills required in laparoscopy result in reduced patient safety during the early part of the learning curve. Evidence suggests that these may be safely acquired in the virtual reality (VR) environment. Several VR simulators are available, each preloaded with several psychomotor skills tasks that provide users with computer-generated performance metrics. This review aimed to evaluate the usefulness of specific psychomotor skills tasks and metrics, and how trainers might build an effective training curriculum.MethodsWe performed a comprehensive literature search.ResultsThe vast majority of VR psychomotor skills tasks show construct validity for one or more metrics. These are commonly for time and motion parameters. Regarding training schedules, distributed practice is preferred over massed practice. However, a degree of supervision may be needed to counter the limitations of VR training.ConclusionsIn the future, standardized proficiency scores should facilitate local institutions in establishing VR laparoscopic psychomotor skills curricula.     

Digital Education in General Thoracic Surgery: A Narrative Review

BackgroundSince advanced technologies were introduced into surgical education, a variety of new programs have been developed. However, a comprehensive review of digital education in general thoracic surgery has not been performed. This narrative review was conducted was to identify the current applications of digital education in general thoracic surgery.MethodsA literature search was performed using keywords related to general thoracic surgery and digital education, including e-learning and virtual simulation, up to September 2021. Studies published in English investigating the effect of digital education in general thoracic surgery were included.ResultsThirteen studies met the criteria. The settings were in undergraduate (n = 6) and postgraduate education (residency) (n = 5) and mixed audience with other disciplines (n = 2). Theoretical knowledge (n = 5), technical skills (n = 4), and both knowledge and technical skills (n = 4) were the stated educational objectives for the studies. The didactic materials were transferred to hardware, software, or online platforms and delivered with multimedia materials. Technical skills training for bronchoscopy and chest tube insertion (n = 5) were offered using virtual reality and computer-based simulations. Subjective evaluation was done in 10 studies. Although after the digital education training there was observed improvement in knowledge or skills in 8 studies, studies were not designed to test for superiority compared with controls through randomized controlled studies.ConclusionsThis review summarizes the current applications of digital education in general thoracic surgery and helps establish the needs for future studies in this field.     

Virtual reality hand therapy: A new tool for nonopioid analgesia for acute procedural pain,hand rehabilitation,and VR embodiment therapy for phantom limb pain

IntroductionAffordable virtual reality (VR) technology is now widely available. Billions of dollars are currently being invested into improving and mass producing VR and augmented reality products.Purpose of the StudyThe purpose of the present study is to explore the potential of immersive VR to make physical therapy/occupational therapy less painful, more fun, and to help motivate patients to cooperate with their hand therapist.DiscussionThe following topics are covered: a) psychological influences on pain perception, b) the logic of how VR analgesia works, c) evidence for reduction of acute procedural pain during hand therapy, d) recent major advances in VR technology, and e) future directions—immersive VR embodiment therapy for phantom limb (chronic) pain.ConclusionVR hand therapy has potential for a wide range of patient populations needing hand therapy, including acute pain and potentially chronic pain patients. Being in VR helps reduce the patients’ pain, making it less painful for patients to move their hand/fingers during hand therapy, and gamified VR can help motivate the patient to perform therapeutic hand exercises, and make hand therapy more fun. In addition, VR camera–based hand tracking technology may be used to help therapists monitor how well patients are doing their hand therapy exercises, and to quantify whether adherence to treatment increases long-term functionality. Additional research and development into using VR as a tool for hand therapist is recommended for both acute pain and persistent pain patient populations.     

The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review

Background  Virtual reality (VR) as surgical training tool has become a state-of-the-art technique in training and teaching skills for minimally invasive surgery (MIS). Although intuitively appealing, the true benefits of haptic (VR training) platforms are unknown. Many questions about haptic feedback in the different areas of surgical skills (training) need to be answered before adding costly haptic feedback in VR simulation for MIS training. This study was designed to review the current status and value of haptic feedback in conventional and robot-assisted MIS and training by using virtual reality simulation. Methods  A systematic review of the literature was undertaken using PubMed and MEDLINE. The following search terms were used: Haptic feedback OR Haptics OR Force feedback AND/OR Minimal Invasive Surgery AND/OR Minimal Access Surgery AND/OR Robotics AND/OR Robotic Surgery AND/OR Endoscopic Surgery AND/OR Virtual Reality AND/OR Simulation OR Surgical Training/Education. Results  The results were assessed according to level of evidence as reflected by the Oxford Centre of Evidence-based Medicine Levels of Evidence. Conclusions  In the current literature, no firm consensus exists on the importance of haptic feedback in performing minimally invasive surgery. Although the majority of the results show positive assessment of the benefits of force feedback, results are ambivalent and not unanimous on the subject. Benefits are least disputed when related to surgery using robotics, because there is no haptic feedback in currently used robotics. The addition of haptics is believed to reduce surgical errors resulting from a lack of it, especially in knot tying. Little research has been performed in the area of robot-assisted endoscopic surgical training, but results seem promising. Concerning VR training, results indicate that haptic feedback is important during the early phase of psychomotor skill acquisition.     

The specialty mentor effect in enhancing surgical experience of medical students: A randomised control trial

IntroductionThe positive learning experiences of students on surgical rotations which subsequently influence career choice may be delineated into practical and interpersonal themed factors. It remains unclear the relative impact each component has on the student experience and subsequent specialisation.AimsWe evaluate the influence of having senior resident mentor during practical simulation in orthopaedic surgery has on medical student interest in surgery; their comfort in theatre; and its role in enhancing knowledge acquisition within the rotation.MethodsMedical students undergoing clinical rotations in a Regional University Hospital were randomised to undertaking a virtual reality simulated operation independently or performing under the guidance of an experienced resident. Baseline levels of interest in surgery, comfort in theatre, perceived barriers to surgical learning and entry to surgery were established and compared to answers following completion of the tutorial. Qualitative feedback was collected regarding the benefits and limitations to the experience. Presented according to CONSORT guidelines.ResultsParticipants in the trainee guided group achieved significantly higher simulated performance scores compared to the control (p < 0001), with an increase in interest in orthopaedic surgery from baseline expressed to a statistically significant degree (p = 0.01). Participants in the unsupervised group demonstrated no significant increase in interest in Orthopaedic surgery (p = 0.3). 100% participants strongly agreed it was a beneficial learning experience which would be useful in surgical curricula.ConclusionSurgical trainee guided simulation led to improved performance and interest in orthopaedic surgery, while virtual practical experience was felt to be a useful learning tool independent of supervision. Further research is needed to establish the role of interpersonal interactions in student surgical experience.     

Is blockchain ready for orthopaedics? A systematic review

IntroductionThe unique attributes of distributed ledger blockchain systems including robust security, immutability, transparency, and decentralisation, make them highly suitable solutions for many healthcare-related problems. This review examines the potential applications for blockchain technology in the field of orthopaedics, by taking a systematic approach to the evolving blockchain literature and mapping potential use cases against the current needs of orthopaedic practice.MethodA literature search was performed using Pubmed, EMBASE, OVID and the Cochrane library with the primary aim of identifying detailed accounts of blockchain solutions and use cases in healthcare. These articles were then reviewed and mapped against current orthopaedic practice to illustrate applications specific to that specialty.ResultsOne hundred and forty-one papers were identified which described case studies, simulations, or detailed proposals of blockchain solutions in healthcare. Most studies described blockchain solutions at the simulated or prototype testing phase, with only 10 case studies describing blockchains in “real-world” use. The most frequently cited use cases for blockchain technology involved the storage, security and sharing of electronic medical records. Other blockchain solutions focused on the “Internet of Things”, research, COVID 19, supply chains and radiology. There were no solutions focusing specifically on orthopaedics. Many of the described blockchain solutions had considerable scope for application in orthopaedic practice however, providing the potential for greater inter-institutional collaboration, cross border data exchange, enhanced patient participation, and more robust and transparent research practices.ConclusionBlockchain solutions for healthcare are increasing in number and scope and have multiple applications relevant to orthopaedic practice. The orthopaedic community needs to be aware of this innovative and growing field of computer science so that surgeons can leverage the power of blockchain safely for the future of orthopaedics.     

Virtual reality-assisted robotic surgery simulation

For more than a decade, advancing computer technologies have allowed incorporation of virtual reality (VR) into surgical training. This has become especially important in training for laparoscopic procedures, which often are complex and leave little room for error. With the advent of robotic surgery and the development and prevalence of a commercial surgical system (da Vinci robot; Intuitive Surgical, Sunnyvale, CA), a valid VR-assisted robotic surgery simulator could minimize the steep learning curve associated with many of these complex procedures and thus enable better outcomes. To date, such simulation does not exist; however, several agencies and corporations are involved in making this dream a reality. We review the history and progress of VR simulation in surgical training, its promising applications in robotic-assisted surgery, and the remaining challenges to implementation.     

Augmented reality in minimally invasive spine surgery: early efficiency and complications of percutaneous pedicle screw instrumentation

BACKGROUND CONTEXTAugmented reality (AR) employs an optical projection directly onto the user's retina, allowing complex image overlay on the natural visual field. In general, pedicle screw accuracy rates have improved with increasingly use of technology, with navigation-based instrumentation described as accurate in 89%–100% of cases. Emerging AR technology in spine surgery builds upon current spinal navigation to provide 3-dimensional imaging of the spine and powerfully reduce the impact of inherent ergonomic and efficiency difficulties.PURPOSEThis publication describes the first known series of in vivo pedicle screws placed percutaneously using AR technology for MIS applications.STUDY DESIGN / SETTINGAfter IRB approval, 3 senior surgeons at 2 institutions contributed cases from June, 2020 – March, 2022. 164 total MIS cases in which AR used for placement of percutaneous pedicle screw instrumentation with spinal navigation were identified prospectively.PATIENT SAMPLE155 (94.5%) were performed for degenerative pathology, 6 (3.6%) for tumor and 3 (1.8%) for spinal deformity.  These cases amounted to a total of 606 pedicle screws; 590 (97.3%) were placed in the lumbar spine, with 16 (2.7%) thoracic screws placed.OUTCOME MEASURESPatient demographics and surgical metrics including total posterior construct time (defined as time elapsed from preincision instrument registration to final screw placement), clinical complications and instrumentation revision rates were recorded in a secure and de-identified database.METHODSThe AR system used features a wireless headset with transparent near-eye display which projects intra-operative 3D imaging directly onto the surgeon's retina. After patient positioning, 1 percuntaneous and 1 superficial reference marker are placed. Intra-operative CT data is processed to the headset and integrates into the surgeon's visual field creating a “see-through” 3D effect in addition to 2D standard navigation images. MIS pedicle screw placement is then carried out percutaneously through single line of sight using navigated instruments.RESULTSTime elapsed from registration and percutaneous approach to final screw placement averaged 3 minutes and 54 seconds per screw.  Analysis of the learning curve revealed similar surgical times in the early cases compared to the cases performed with more experience with the system.  No instrumentation was revised for clinical or radiographic complication at final available follow-up ranging from 6–24 months. A total of 3 screws (0.49%) were replaced intra-operatively. No clinical effects via radiculopathy or neurologic deficit postoperatively were noted.CONCLUSIONSThis is the first report of the use of AR for placement of spinal pedicle screws using minimally invasive techniques.   This series of 164 cases confirmed efficiency and safety of screw placement with the inherent advantages of AR technologies over legacy enabling technologies.     

Development of a structured virtual reality curriculum for laparoscopic appendicectomy

BackgroundLaparoscopic appendicectomy (LA) is a common surgical emergency procedure mainly performed by trainees. The aim was to develop a step-wise structured virtual reality (VR) curriculum for LA to allow junior surgeons to hone their skills in a safe and controlled environment.MethodsA prospective randomized study was designed using a high-fidelity VR simulator. Thirty-five novices and 25 experts participated in the assessment and their performances were compared to assess construct validity. Learning curve analysis was performed.ResultsFive of the psychomotor tasks and all appendicectomy tasks showed construct validity. Learning was demonstrated in the majority of construct-valid tasks. A novel goal-directed VR curriculum for LA was constructed.ConclusionsA step-wise structured VR curriculum for LA is proposed with a framework which includes computer generated metrics and supports deliberate practice, spacing intervals, human instruction/feedback and assessment. Future study should test the feasibility of its implementation and transferability of acquired skill.     

Applications of artificial intelligence for patients with peripheral artery disease

ObjectiveApplications of artificial intelligence (AI) have been reported in several cardiovascular diseases but its interest in patients with peripheral artery disease (PAD) has been so far less reported. The aim of this review was to summarize current knowledge on applications of AI in patients with PAD, to discuss current limits, and highlight perspectives in the field.MethodsWe performed a narrative review based on studies reporting applications of AI in patients with PAD. The MEDLINE database was independently searched by two authors using a combination of keywords to identify studies published between January 1995 and December 2021. Three main fields of AI were investigated including natural language processing (NLP), computer vision and machine learning (ML).ResultsNLP and ML brought new tools to improve the screening, the diagnosis and classification of the severity of PAD. ML was also used to develop predictive models to better assess the prognosis of patients and develop real-time prediction models to support clinical decision-making. Studies related to computer vision mainly aimed at creating automatic detection and characterization of arterial lesions based on Doppler ultrasound examination or computed tomography angiography. Such tools could help to improve screening programs, enhance diagnosis, facilitate presurgical planning, and improve clinical workflow.ConclusionsAI offers various applications to support and likely improve the management of patients with PAD. Further research efforts are needed to validate such applications and investigate their accuracy and safety in large multinational cohorts before their implementation in daily clinical practice.