Novices Outperform Experienced Laparoscopists on Virtual Reality Laparoscopy Simulator

Background and Objectives:

Virtual reality has been poorly studied among gynecologic surgeons. The aim of this study was to evaluate whether performance on the Minimally Invasive Surgery Trainer-Virtual Reality (MISTVR) laparoscopic trainer reflects laparoscopic experience among gynecologic surgeons and trainees.

Methods:

Twenty-six medical students, residents, and attending gynecologic surgeons completed a MIST-VR training program. A new simulated task was then presented to each participant, who repeated the task until proficiency was reached.

Results:

Attending physicians performed poorly when compared with medical students, requiring more than twice the number of attempts to reach proficiency (Mann-Whitney P<0.01). Among medical students and residents, there was an association between years of live laparoscopy experience and poor simulator performance (Spearman r P=0.01).

Conclusion:

Increased operating room experience and age were associated with worsening simulator performance. Several potential explanations for this trend are discussed, including lack of tactile and contextual feedback. Caution should be exercised when considering current virtual reality simulator technology as a measure of experience or ability among gynecologic surgeons.     

Effects of Virtual Reality Simulator Training Method and Observational Learning on Surgical Performance

Background  

Virtual reality (VR) simulators and Web-based instructional videos are valuable supplemental training resources in surgical programs, but it is unclear how to optimally integrate them into minimally invasive surgical training.     

Performance on a Virtual Reality Angled Laparoscope Task Correlates with Spatial Ability of Trainees

The aim of the present study was to investigate whether trainees’ performance on a virtual reality angled laparoscope navigation task correlates with scores obtained on a validated conventional test of spatial ability. 56 participants of a surgery workshop performed an angled laparoscope navigation task on the Xitact LS 500 virtual reality Simulator. Performance parameters were correlated with the score of a validated paper-and-pencil test of spatial ability. Performance at the conventional spatial ability test significantly correlated with performance at the virtual reality task for overall task score (p < 0.001), task completion time (p < 0.001) and economy of movement (p = 0.035), not for endoscope travel speed (p = 0.947). In conclusion, trainees’ performance in a standardized virtual reality camera navigation task correlates with their innate spatial ability. This VR session holds potential to serve as an assessment tool for trainees.     

Virtual Reality Simulation Facilitates Resident Training in Total Hip Arthroplasty: A Randomized Controlled Trial

BackgroundNo study has yet assessed the efficacy of virtual reality (VR) simulation for teaching orthopedic surgery residents. In this blinded, randomized, and controlled trial, we asked if the use of VR simulation improved postgraduate year (PGY)-1 orthopedic residents’ performance in cadaver total hip arthroplasty and if the use of VR simulation had a preferentially beneficial effect on specific aspects of surgical skills or knowledge.MethodsFourteen PGY-1 orthopedic residents completed a written pretest and a single cadaver total hip arthroplasty (THA) to establish baseline levels of knowledge and surgical ability before 7 were randomized to VR-THA simulation. All participants then completed a second cadaver THA and retook the test to assess for score improvements. The primary outcomes were improvement in test and cadaver THA scores.ResultsThere was no significant difference in the improvement in test scores between the VR and control groups (P = .078). In multivariate regression analysis, the VR cohort demonstrated a significant improvement in overall cadaver THA scores (P = .048). The VR cohort demonstrated greater improvement in each specific score category compared with the control group, but this trend was only statistically significant for technical performance (P = .009).ConclusionsVR-simulation improves PGY-1 resident surgical skills but has no significant effect on medical knowledge. The most significant improvement was seen in technical skills. We anticipate that VR simulation will become an indispensable part of orthopedic surgical education, but further study is needed to determine how best to use VR simulation within a comprehensive curriculum.Level of EvidenceLevel 1.     

A Pilot Study of Surgical Training Using a Virtual Robotic Surgery Simulator

Background and Objectives:

Our objectives were to compare the utility of learning a suturing task on the virtual reality da Vinci Skills Simulator versus the da Vinci Surgical System dry laboratory platform and to assess user satisfaction among novice robotic surgeons.

Methods:

Medical trainees were enrolled prospectively; one group trained on the virtual reality simulator, and the other group trained on the da Vinci dry laboratory platform. Trainees received pretesting and post-testing on the dry laboratory platform. Participants then completed an anonymous online user experience and satisfaction survey.

Results:

We enrolled 20 participants. Mean pretest completion times did not significantly differ between the 2 groups. Training with either platform was associated with a similar decrease in mean time to completion (simulator platform group, 64.9 seconds [P = .04]; dry laboratory platform group, 63.9 seconds [P < .01]). Most participants (58%) preferred the virtual reality platform. The majority found the training “definitely useful” in improving robotic surgical skills (mean, 4.6) and would attend future training sessions (mean, 4.5).

Conclusion:

Training on the virtual reality robotic simulator or the dry laboratory robotic surgery platform resulted in significant improvements in time to completion and economy of motion for novice robotic surgeons. Although there was a perception that both simulators improved performance, there was a preference for the virtual reality simulator. Benefits unique to the simulator platform include autonomy of use, computerized performance feedback, and ease of setup. These features may facilitate more efficient and sophisticated simulation training above that of the conventional dry laboratory platform, without loss of efficacy.     

Psychomotor performance measured in a virtual environment correlates with technical skills in the operating room

Background  This study was conducted to validate the role of virtual reality computer simulation as an objective method for assessing laparoscopic technical skills. The authors aimed to investigate whether performance in the operating room, assessed using a modified Objective Structured Assessment of Technical Skill (OSATS), correlated with the performance parameters registered by a virtual reality laparoscopic trainer (LapSim). Methods  The study enrolled 10 surgical residents (3 females) with a median of 5.5 years (range, 2–6 years) since graduation who had similar limited experience in laparoscopic surgery (median, 5; range, 1–16 laparoscopic cholecystectomies). All the participants performed three repetitions of seven basic skills tasks on the LapSim laparoscopic trainer and one laparoscopic cholecystectomy in the operating room. The operating room procedure was video recorded and blindly assessed by two independent observers using a modified OSATS rating scale. Assessment in the operating room was based on three parameters: time used, error score, and economy of motion score. During the tasks on the LapSim, time, error (tissue damage and millimeters of tissue damage [tasks 2–6], error score [incomplete target areas, badly placed clips, and dropped clips [task 7]), and economy of movement parameters (path length and angular path) were registered. The correlation between time, economy, and error parameters during the simulated tasks and the operating room procedure was statistically assessed using Spearman’s test. Results  Significant correlations were demonstrated between the time used to complete the operating room procedure and time used for task 7 (r _s = 0.74; p = 0.015). The error score demonstrated during the laparoscopic cholecystectomy correlated well with the tissue damage in three of the seven tasks (p < 0.05), the millimeters of tissue damage during two of the tasks, and the error score in task 7 (r _s = 0.67; p = 0.034). Furthermore, statistically significant correlations were observed between the economy of motion score from the operative procedure and LapSim’s economy parameters (path length and angular path in six of the tasks) (p < 0.05). Conclusions  The current study demonstrated significant correlations between operative performance in the operating room (assessed using a well-validated rating scale) and psychomotor performance in virtual environment assessed by a computer simulator. This provides strong evidence for the validity of the simulator system as an objective tool for assessing laparoscopic skills. Virtual reality simulation can be used in practice to assess technical skills relevant for minimally invasive surgery.     

Discriminative validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST-VR) using criteria levels based on expert performance

Background: Increasing constraints on the time and resources needed to train surgeons have led to a new emphasis on finding innovative ways to teach surgical skills outside the operating room. Virtual reality training has been proposed as a method to both instruct surgical students and evaluate the psychomotor components of minimally invasive surgery ex vivo. Methods: The performance of 100 laparoscopic novices was compared to that of 12 experienced (>50 minimally invasive procedures) and 12 inexperienced (<10 minimally invasive procedures) laparoscopic surgeons. The values of the experienced surgeons performance were used as benchmark comparators (or criterion measures). Each subject completed six tasks on the Minimally Invasive Surgical Trainer—Virtual Reality (MIST-VR) three times. The outcome measures were time to complete the task, number of errors, economy of instrument movement, and economy of diathermy. Results: After three trials, the mean performance of the medical students approached that of the experienced surgeons. However, 7–27% of the scores of the students fell more than two SD below the mean scores of the experienced surgeons (the criterion level). Conclusions: The MIST-VR system is capable of evaluating the psychomotor skills necessary in laparoscopic surgery and discriminating between experts and novices. Furthermore, although some novices improved their skills quickly, a subset had difficulty acquiring the psychomotor skills. The MIST-VR may be useful in identifying that subset of novices.     

Learning curves and impact of psychomotor training on performance in simulated colonoscopy: a randomized trial using a virtual reality endoscopy trainer

Background The aim of this study was to analyze the learning curve for the GI Mentor II endoscope trainer and to determine whether psychomotor training can contribute to an improvement in the performance of virtual colonoscopy.Methods To analyze the learning curve, 28 subjects were divided into three groups on the basis of their experience with gastrointestinal (GI) endoscopy: experienced surgeons (group 1, performed > 200 endoscopic procedures, (n = 8)) residents (group 2, performed < 50 endoscopic procedures, (n = 10)); and medical students (group 3, never performed GI endoscopy, (n = 10)). The participants were tested on the GI Mentor II virtual reality simulator 10 consecutive times. Assessment of the learning curve was based on the following three parameters: time used, number of punctured balloons, and number of wall collisions. In the second part of the study, 20 subjects who had never performed GI endoscopy were included. After performing a virtual colonoscopy, they were randomized to a group that received psychomotor training and a control group. Finally, all subjects performed a virtual colonoscopy. Assessment of endoscopic skills during the colonoscopy was based on nine parameters dealing with psychomotor skills.Results The learning curve for time expended reached a plateau after the second repetition for group 1 (Friedmans test, p < 0.05), after the fifth repetition for group 2 (p < 0.05), and after the seventh repetition for group 3 (p < 0.05). Experienced surgeons did not improve their scores for regarding number of balloons punctured or number of wall collisions (p > 0.05), indicating the absence of a learning curve for these parameters. Group 2 improved their scores up to the fourth and fifth repetitions, respectively (p < 0.05), and group 3 up to the fifth and seventh repetitions, respectively (p < 0.05). Experienced surgeons achieved the best performance, followed by group 2 and then group 3. The surgeons who had received psychomotor training performed the second virtual colonoscopy significantly faster than the control group (Mann-Whitney test, p < 0.001) and made significantly greater improvement in all other parameters as well.Conclusions There were different learning curves for surgeons depending on their endoscopic background. The familiarization rate on the simulator was proportional to the endoscopic experience of the surgeons. Psychomotor training had a significant effect on the performance of a simulated colonoscopy.     

Acquisition of flexible cystoscopy skills on a virtual reality simulator by experts and novices

Study Type – Therapy (case control)
Level of Evidence 3b

OBJECTIVE

To assess the construct validity of the URO Mentor^TM (Simbionix Corp., Cleveland, OH, USA) virtual reality training model for several variables of skills training in cysto‐urethroscopy, addressing two research questions: (i) Does training on the URO Mentor significantly improve novices’ performance in terms of time, trauma, areas inspected and Global Rating Scale (GRS) score?; (ii) is discrimination between different levels of expertise possible using the URO Mentor?

METHODS

Thirty experts and 50 novices performed seven tasks on the URO Mentor during one training session. The first, fourth and seventh tasks were ‘test tasks’ to evaluate participants’ performance. The simulator recorded procedure time and trauma; a supervisor scored which areas were inspected and gave scores on the GRS. A two‐way analysis of variance with repeated‐measures test was used to analyse experts’ and novices’ performances, with P < 0.05 considered to indicate statistical significance. Effect sizes (ES) were calculated to quantify the practical significance of the results; ES of 0.10, 0.30, and 0.50 were considered small, medium and large, respectively.

RESULTS

Novices’ performances showed a significant improvement with large ES in time (linear trend of learning curve P < 0.001, ES 0.66) and mean GRS score (linear trend P < 0.001, ES 0.84, quadratic trend P = 0.018, ES 0.24). There was a medium improvement for trauma (linear trend P < 0.001, ES 0.40) and a small improvement in areas inspected (linear trend P = 0.032, ES 0.21). That the 95% confidence intervals of the measures on the first task of experts and novices did not coincide indicates that differentiation between experts and novices on the four variables measured can be achieved using the URO Mentor.

CONCLUSIONS

Training on the URO Mentor appears to result in a medium to large improvement of novices’ performances for time, trauma, areas inspected and GRS scores. Moreover, discrimination between different levels of expertise is possible using this simulator.     

Predictive validity of driving-simulator assessments following traumatic brain injury: a preliminary study

Objective: To evaluate whether driving simulator and road test evaluations can predict long-term driving performance, we conducted a prospective study on 11 patients with moderate to severe traumatic brain injury. Sixteen healthy subjects were also tested to provide normative values on the simulator at baseline.

Method: At their initial evaluation (time-1), subjects' driving skills were measured during a 30-minute simulator trial using an automated 12-measure Simulator Performance Index (SPI), while a trained observer also rated their performance using a Driving Performance Inventory (DPI). In addition, patients were evaluated on the road by a certified driving evaluator. Ten months later (time-2), family members observed patients driving for at least 3 hours over 4 weeks and rated their driving performance using the DPI.

Results: At time-1, patients were significantly impaired on automated SPI measures of driving skill, including: speed and steering control, accidents, and vigilance to a divided-attention task. These simulator indices significantly predicted the following aspects of observed driving performance at time-2: handling of automobile controls, regulation of vehicle speed and direction, higher-order judgment and self-control, as well as a trend-level association with car accidents. Automated measures of simulator skill (SPI) were more sensitive and accurate than observational measures of simulator skill (DPI) in predicting actual driving performance. To our surprise, the road test results at time-1 showed no significant relation to driving performance at time-2.

Conclusion: Simulator-based assessment of patients with brain injuries can provide ecologically valid measures that, in some cases, may be more sensitive than a traditional road test as predictors of long-term driving performance in the community.