Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator

Background The aim of this study was to develop summary metrics and assess the construct validity for a virtual reality laparoscopic simulator (LapSim) by comparing the learning curves of three groups with different levels of laparoscopic expertise.Methods Three groups of subjects (‘expert’, ‘junior’, and ‘naïve’) underwent repeated trials on three LapSim tasks. Formulas were developed to calculate scores for efficiency (‘time-error’) and economy of ‘motion’ (‘motion’) using metrics generated by the software after each drill. Data (mean ± SD) were evaluated by analysis of variance (ANOVA). Significance was set at p < 0.05.Results All three groups improved significantly from baseline to final for both ‘time-error’ and ‘motion’ scores. There were significant differences between groups in time error performances at baseline and final, due to higher scores in the ‘expert’ group. A significant difference in ‘motion’ scores was seen only at baseline.Conclusion We have developed summary metrics for the LapSim that differentiate among levels of laparoscopic experience. This study also provides evidence of construct validity for the LapSim.Presented in part at the annual meeting of The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), Denver, CO, USA April 2004     

A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill

BACKGROUND: Studies have demonstrated the beneficial effect of training novice laparoscopic surgeons using virtual reality (VR) simulators, although there is still no consensus regarding an optimal VR training curriculum. This study aims to establish and validate a structured VR curriculum to provide an evidence-based approach for laparoscopic training programmes. METHODS: The minimally invasive VR simulator (MIST-VR) has 12 abstract laparoscopic tasks, each at 3 graduated levels of difficulty (easy, medium, and hard). Twenty medical students completed 2 sessions of all tasks at the easy level, 10 sessions at the medium level, and finally 5 sessions of the 2 most complex tasks at the hard level. At the medium level, subjects were randomized into 2 equal groups performing either all 12 tasks (group A) or the 2 most complex tasks (group B). Performance was measured by time taken, path length, and errors for each hand. The results were compared between groups, and to those of 10 experienced laparoscopic surgeons. RESULTS: Baseline performance of both groups was similar at the easy level. At the medium level, learning curves for all 3 parameters reached plateau at the second (group A, P < .05) and sixth (group B, P < .05) repetitions. Performance at the hard level was similar between the 2 groups, and all achieved the pre-set expert criteria. CONCLUSION: A graduated laparoscopic training curriculum enables trainees to familiarise, train and be assessed on laparoscopic VR simulators. This study can aid the incorporation of VR simulation into established surgical training programmes.     

Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality

BACKGROUND: Very few studies have addressed the transferability of skills from virtual reality (VR) to real life. The aim of this study was to assess the feasibility and effectiveness of teaching intracorporeal knot tying (ICKT) by VR simulation only. METHODS: Twenty novices underwent structured training of basic skills training on the Minimally Invasive Surgical Trainer simulator (Mentice AB, Gothenburg, Sweden) followed by knot tying training on the LapSim simulator (Surgical Science, Gothenburg, Sweden). They were assessed pre- and post-training on a video trainer. Assessment of performance included motion tracking and video-based checklist. Nonparametric statistical analysis was used, and P < .05 was deemed significant. RESULTS: All participants completed a correct knot as compared with only 25% before VR training. Time to completion was 66% faster and knot quality 45% better after VR training. Significant reduction in number of movements (P = .006) and distance traveled (P < .000) by both hands after VR training. CONCLUSIONS: Teaching ICKT by VR simulators only is feasible and effective. Furthermore, this study highlights the complementary use of different VR simulators within a structured curriculum.     

Virtual reality and computer-enhanced training devices equally improve laparoscopic surgical skill in novices

BACKGROUND: The study aim was to compare the effectiveness of virtual reality and computer-enhanced videoscopic training devices for training novice surgeons in complex laparoscopic skills. METHODS: Third-year medical students received instruction on laparoscopic intracorporeal suturing and knot tying and then underwent a pretraining assessment of the task using a live porcine model. Students were then randomized to objectives-based training on either the virtual reality (n=8) or computer-enhanced (n=8) training devices for 4 weeks, after which the assessment was repeated. RESULTS: Posttraining performance had improved compared with pretraining performance in both task completion rate (94% versus 18%; P<0.001*) and time [181+/-58 (SD) versus 292+/-24*]. Performance of the 2 groups was comparable before and after training. Of the subjects, 88% thought that haptic cues were important in simulators. Both groups agreed that their respective training systems were effective teaching tools, but computer-enhanced device trainees were more likely to rate their training as representative of reality (P<0.01). CONCLUSIONS: Training on virtual reality and computer-enhanced devices had equivalent effects on skills improvement in novices. Despite the perception that haptic feedback is important in laparoscopic simulation training, its absence in the virtual reality device did not impede acquisition of skill.     

Challenges to the development of complex virtual reality surgical simulations

Virtual reality simulation in surgical training has become more widely used and intensely investigated in an effort to develop safer, more efficient, measurable training processes. The development of virtual reality simulation of surgical procedures has begun, but well-described technical obstacles must be overcome to permit varied training in a clinically realistic computer-generated environment. These challenges include development of realistic surgical interfaces and physical objects within the computer-generated environment, modeling of realistic interactions between objects, rendering of the surgical field, and development of signal processing for complex events associated with surgery. Of these, the realistic modeling of tissue objects that are fully responsive to surgical manipulations is the most challenging. Threats to early success include relatively limited resources for development and procurement, as well as smaller potential for return on investment than in other simulation industries that face similar problems. Despite these difficulties, steady progress continues to be made in these areas. If executed properly, virtual reality offers inherent advantages over other training systems in creating a realistic surgical environment and facilitating measurement of surgeon performance. Once developed, complex new virtual reality training devices must be validated for their usefulness in formative training and assessment of skill to be established.     

Laparoscopic appendicectomy: a training model for laparoscopic right hemicolectomy?

OBJECTIVE: To analyse the outcome of laparoscopic appendicectomy and right hemicolectomy and see if the surgical approach to the former can be applied to the latter. METHOD: A prospective electronic laparoscopic database identified 330 appendicectomies and 78 right hemicolectomies (using this approach) between 1996 and 2005. RESULTS: Three hundred and thirty patients (188 males: median age 38 years, range 17-74 years) underwent laparoscopic appendicectomy; 270 (82%) were performed by trainees (higher surgical trainee 71%, basic surgical trainee 12%). The median operative time for trainees was 35 min (14-75 min) with a conversion rate 2%. There were no intra-operative complications. The postoperative complication rate excluding minor wound infection (5.5%) was 1.5%. There were no deaths. The median hospital stay was 2 days (1-15 days). The 30-day readmission rate was 1%. Seventy-eight patients (23-93 years) underwent laparoscopic right hemicolectomy during 2004/5; trainees performed parts thereof in the majority or all of the surgery in 25 cases. The median operation time was 55 min: trainees 115 (65-145 min). There was one conversion. The median hospital stay was 4 days (2-23 days) falling to 3 for the last 20 operations (1-8 days). There were two readmissions for wound sepsis and small bowel obstruction and three deaths (3.8%): anastomotic leak (one), C difficile infection leading to renal failure (one) and duodenal perforation (one). CONCLUSION: Laparoscopic appendicectomy is a safe, predictable, easily learnt operation and an ideal model for learning the skills and principles required for more advanced laparoscopic colorectal interventions and in particular, right hemicolectomy.     

Consensus guidelines for validation of virtual reality surgical simulators

The Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes is a newly formed sub-group of the European Association of Endoscopic Surgeons (EAES). This work group undertook a review of validation evidence for surgical simulators and the resulting consensus is presented in this article. Using clinical guidelines criteria, the evidence for validation for six different simulators was rated and subsequently translated to a level of recommendation for each system. The simulators could be divided into two basic types; systems for laparoscopic general surgery and flexible gastrointestinal endoscopy. Selection of simulators for inclusion in this consensus was based on their availability and relatively widespread usage as of July 2004. Whilst level 2 recommendations were achieved for a few systems, it was clear that there was an overall lack of published validation studies with rigorous experimental methodology. Since the consensus meeting, there have been a number of new articles, system upgrades and new devices available. The work group intends to update these consensus guidelines on a regular basis, with the resulting article available on the EAES website ( ). Sponsored by the EAES Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes, residing under the European Association for Endoscopic Surgery     

A tale of two trainers: virtual reality versus a video trainer for acquisition of basic laparoscopic skills

Background

This study aimed to assess the transferability of basic laparoscopic skills between a virtual reality simulator (MIST-VR) and a video trainer box (D-Box).

Methods

Forty-six medical students were randomized into 2 groups, training on MIST-VR or D-Box. After training with one modality, a crossover assessment on the other was performed.

Results

When tested on MIST-VR, the MIST-VR group showed significantly shorter time (90.3 seconds vs 188.6 seconds, P <.001), better economy of movements (4.40 vs 7.50, P <.001), and lower score (224.7 vs 527.0, P <.001). However, when assessed on the D-Box, there was no difference between the groups for time (402.0 seconds vs 325.6 seconds, P = .152), total hand movements (THC) (289 vs 262, P = .792), or total path length (TPL) (34.9 m vs 34.6 m, P = .388).

Conclusion

Both simulators provide significant improvement in performance. Our results indicate that skills learned on the MIST-VR are transferable to the D-Box, but the opposite cannot be demonstrated.     

Development and evaluation of a trauma decision-making simulator in Oculus virtual reality

Background

Consumer-available virtual-reality technology was launched in 2016 with strong foundations in the entertainment-industry. We developed an innovative medical-training simulator on the Oculus? Gear-VR platform. This novel application was developed utilising internationally recognised Advanced Trauma Life Support (ATLS) principles, requiring decision-making skills for critically-injured virtual-patients.

Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies

BACKGROUND: Virtual reality (VR) training has been shown previously to improve intraoperative performance during part of a laparoscopic cholecystectomy. The aim of this study was to assess the effect of proficiency-based VR training on the outcome of the first 10 entire cholecystectomies performed by novices. METHODS: Thirteen laparoscopically inexperienced residents were randomized to either (1) VR training until a predefined expert level of performance was reached, or (2) the control group. Videotapes of each resident's first 10 procedures were reviewed independently in a blinded fashion and scored for predefined errors. RESULTS: The VR-trained group consistently made significantly fewer errors (P = .0037). On the other hand, residents in the control group made, on average, 3 times as many errors and used 58% longer surgical time. CONCLUSIONS: The results of this study show that training on the VR simulator to a level of proficiency significantly improves intraoperative performance during a resident's first 10 laparoscopic cholecystectomies.     

Objective assessment of laparoscopic skills using a virtual reality stimulator

Background Virtual reality simulation has a great potential as a training and assessment tool of laparoscopic skills. The study was carried out to investigate whether the LapSim system (Surgical Science Ltd., Gothenburg, Sweden) was able to differentiate between subjects with different laparoscopic experience and thus to demonstrate its construct validity. Methods Subjects 24 were divided into two groups: experienced (performed > 100 laparoscopic procedures, n = 10) and beginners (performed <10 laparoscopic procedures, n = 14). Assessment of laparoscopic skills was based on parameters measured by the computer system. Results Experienced surgeons performed consistently better than the residents. Significant differences in the parameters time and economy of motion existed between the two groups in seven of seven tasks. Regarding error parameters, differences existed in most but not all tasks. Conclusion LapSim was able to differentiate between subjects with different laparoscopic experience. This indicates that the system measures skills relevant for laparoscopic surgery and can be used in training programs as a valid assessment tool.     

The importance of haptic feedback in laparoscopic suturing training and the additive value of virtual reality simulation

Introduction Previous studies on the difference between physical, augmented and virtual reality (VR) simulation state that haptic feedback is an important feature in laparoscopic suturing simulation. Objective assessment is important to improve skills during training. This study focuses on the additive value of VR simulation for laparoscopic suturing training. Methods All participants of several European Association for Endoscopic Surgery (EAES)-approved laparoscopic skills courses (N = 45) filled out a questionnaire on their opinion on laparoscopic suturing training. Additionally, participants with little or no laparoscopic suturing experience were allotted to two groups: group A (N = 10), who started training on the box trainer and subsequently the VR simulator (SimSurgery), and group B (N = 10), who began on the VR simulator followed by the box. Finally, suturing and knot-tying skills were assessed by an expert observer, using a standard evaluation form (eight items on five-point-Likert scale). The same was done after the initial training on the box in group A, as a control. Significant differences were calculated with the independent-sample t-test and the paired t-test. Results The total score of group A was higher than both group B and control (means of 30.80, 27.60, 28.20, respectively), but not significantly. The only tendency to a significant difference between group A and B was found in ‘taking proper bites’ (mean 4.10 versus 3.60, p = 0.054). All the participants scored the features of the box trainer significantly higher than those of the VR simulator (p < 0.001), 46.7% was of the opinion that the box alone would be sufficient for laparoscopic suturing training. Conclusion From this study we can conclude that VR simulation does not have a significant additional value in laparoscopic suturing training, over traditional box trainers. One should consider that the future development in VR simulation should focus on basic skills and component tasks of procedural training in laparoscopic surgery, rather than laparoscopic suturing.     

Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills

BACKGROUND: The study was carried out to analyze the learning rate for laparoscopic skills on a virtual reality training system and to establish whether the simulator was able to differentiate between surgeons with different laparoscopic experience. METHODS: Forty-one surgeons were divided into three groups according to their experience in laparoscopic surgery: masters (group 1, performed more than 100 cholecystectomies), intermediates (group 2, between 15 and 80 cholecystectomies), and beginners (group 3, fewer than 10 cholecystectomies) were included in the study. The participants were tested on the Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) 10 consecutive times within a 1-month period. Assessment of laparoscopic skills included time, errors, and economy of hand movement, measured by the simulator. RESULTS: The learning curves regarding time reached plateau after the second repetition for group 1, the fifth repetition for group 2, and the seventh repetition for group 3 (Friedman's tests P <0.05). Experienced surgeons did not improve their error or economy of movement scores (Friedman's tests, P >0.2) indicating the absence of a learning curve for these parameters. Group 2 error scores reached plateau after the first repetition, and group 3 after the fifth repetition. Group 2 improved their economy of movement score up to the third repetition and group 3 up to the sixth repetition (Friedman's tests, P <0.05). Experienced surgeons (group 1) demonstrated best performance parameters, followed by group 2 and group 3 (Mann-Whitney test P <0.05). CONCLUSIONS: Different learning curves existed for surgeons with different laparoscopic background. The familiarization rate on the simulator was proportional to the operative experience of the surgeons. Experienced surgeons demonstrated best laparoscopic performance on the simulator, followed by those with intermediate experience and the beginners. These differences indicate that the scoring system of MIST-VR is sensitive and specific to measuring skills relevant for laparoscopic surgery.     

Surgical planning for microsurgical excision of cerebral arterio-venous malformations using virtual reality technology

Background  To evaluate the feasibility of surgical planning using a virtual reality platform workstation in the treatment of cerebral arterio-venous malformations (AVMs) Methods  Patient- specific data of multiple imaging modalities were co-registered, fused and displayed as a 3D stereoscopic object on the Dextroscope, a virtual reality surgical planning platform. This system allows for manipulation of 3D data and for the user to evaluate and appreciate the angio-architecture of the nidus with regards to position and spatial relationships of critical feeders and draining veins. We evaluated the ability of the Dextroscope to influence surgical planning by providing a better understanding of the angio-architecture as well as its impact on the surgeon’s pre- and intra-operative confidence and ability to tackle these lesions. Findings  Twenty four patients were studied. The mean age was 29.65 years. Following pre-surgical planning on the Dextroscope, 23 patients underwent microsurgical resection after pre-surgical virtual reality planning, during which all had documented complete resection of the AVM. Planning on the virtual reality platform allowed for identification of critical feeders and draining vessels in all patients. The appreciation of the complex patient specific angio-architecture to establish a surgical plan was found to be invaluable in the conduct of the procedure and was found to enhance the surgeon’s confidence significantly. Conclusion  Surgical planning of resection of an AVM with a virtual reality system allowed detailed and comprehensive analysis of 3D multi-modality imaging data and, in our experience, proved very helpful in establishing a good surgical strategy, enhancing intra-operative spatial orientation and increasing surgeon’s confidence.     

Construct validity for the LAPSIM laparoscopic surgical simulator

Background The skills required for laparoscopic surgery are amenable to simulator-based training. Several computerized devices are now available. We hypothesized that the LAPSIM simulator can be shown to distinguish novice from experienced laparoscopic surgeons, thus establishing construct validity.Methods We tested residents of all levels and attending laparoscopic surgeons. The subjects were tested on eight software modules. Pass/fail (P/F), time (T), maximum level achieved (MLA), tissue damage (TD), motion, and error scores were compared using the t-test and analysis of variance.Results A total of 54 subjects were tested. The most significant difference was found when we compared the most (seven attending surgeons) and least experienced (10 interns) subjects. Grasping showed significance at P/F and MLA (p < 0.03). Clip applying was significant for P/F, MLA, motion, and errors (p < 0.02). Laparoscopic suturing was significant for P/F, MLA, T, TD, as was knot error (p < 0.05). This finding held for novice, intermediate, and expert subjects (p < 0.05) and for suturing time between attending surgeons and residents (postgraduate year [PGY] 1-4) (p < 0.05).Conclusions LAPSIM has construct validity to distinguish between expert and novice laparoscopists. Suture simulation can be used to discriminate between individuals at different levels of residency and expert surgeons.Presented at the annual meeting of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES), Denver, CO, USA, April 1–3, 2004     

Computer-enhanced laparoscopic training system (CELTS): bridging the gap

Background There is a large and growing gap between the need for better surgical training methodologies and the systems currently available for such training. In an effort to bridge this gap and overcome the disadvantages of the training simulators now in use, we developed the Computer-Enhanced Laparoscopic Training System (CELTS).Methods CELTS is a computer-based system capable of tracking the motion of laparoscopic instruments and providing feedback about performance in real time. CELTS consists of a mechanical interface, a customizable set of tasks, and an Internet-based software interface. The special cognitive and psychomotor skills a laparoscopic surgeon should master were explicitly defined and transformed into quantitative metrics based on kinematics analysis theory. A single global standardized and task-independent scoring system utilizing a z-score statistic was developed. Validation exercises were performed.Results The scoring system clearly revealed a gap between experts and trainees, irrespective of the task performed; none of the trainees obtained a score above the threshold that distinguishes the two groups. Moreover, CELTS provided educational feedback by identifying the key factors that contributed to the overall score. Among the defined metrics, depth perception, smoothness of motion, instrument orientation, and the outcome of the task are major indicators of performance and key parameters that distinguish experts from trainees. Time and path length alone, which are the most commonly used metrics in currently available systems, are not considered good indicators of performance.Conclusion CELTS is a novel and standardized skills trainer that combines the advantages of computer simulation with the features of the traditional and popular training boxes. CELTS can easily be used with a wide array of tasks and ensures comparability across different training conditions. This report further shows that a set of appropriate and clinically relevant performance metrics can be defined and a standardized scoring system can be designed.     

Usefulness of a virtual reality simulator or training box for endoscopic surgery training

This study investigated whether the use of a simulator for endoscopic surgery training improves the performance of actual operations. For the study, 16 medical students were divided into three groups: a virtual reality (VR) simulator group (n = 6), a training box (TB) group (n = 6), and a control group (n = 4). The VR and TB groups received 2 h of training per day for 5 days, after which they were requested to perform intestinal dissection, to close a gastric perforation, and to perform cholecystectomy in pigs. Performance was assessed on the basis of the operating time and the error score. Although there were no differences between the three groups in terms of the total operating time or error score, the VR and TB groups required less time for closure of the gastric perforation than the control group. In addition, the operating time decreased as actual operating experience increased in the VR group. In the TB group, the time for closure of a gastric perforation was shorter when the supervisor had seen the operation before training. These findings demonstrated that a simulator is useful for acquiring psychomotor skills, but does not immediately enable clinical performance of an operation. More actual experience and good supervision are essential for increasing the effectiveness of training with the VR and TB simulators, respectively.     

Construct and face validity and task workload for laparoscopic camera navigation: virtual reality versus videotrainer systems at the SAGES Learning Center

Background Laparoscopic camera navigation (LCN) training on simulators has demonstrated transferability to actual operations, but no comparative data exist. The objective of this study was to compare the construct and face validity, as well as workload, of two previously validated virtual reality (VR) and videotrainer (VT) systems. Methods Attendees (n = 90) of the SAGES 2005 Learning Center performed two repetitions on both VR (EndoTower™) and VT (Tulane Trainer) LCN systems using 30° laparoscopes and completed a questionnaire regarding demographics, simulator characteristics, and task workload. Construct validity was determined by comparing the performance scores of subjects with various levels of experience according to five parameters and face validity according to eight. The validated NASA-TLX questionnaire that rates the mental, physical, and temporal demand of a task as well as the performance, effort, and frustration of the subject was used for workload measurement. Results Construct validity was demonstrated for both simulators according to the number of basic laparoscopic cases (p = 0.005), number of advanced cases (p < 0.001), and frequency of angled scope use (p < 0.001), and only for VT according to training level (p < 0.001) and fellowship training (p = 0.008). Face validity ratings on a 1-20 scale averaged 15.4 ± 3 for VR vs. 16 ± 2.6 for VT (p = 0.04). Ninety-six percent of participants rated both simulators as valid educational tools. The NASA-TLX overall workload score was 69.5 ± 24 for VR vs. 68.8 ± 20.5 for VT (p = 0.31). Conclusions This is the largest study to date that compares two validated LCN simulators. While subtle differences exist, both VR and VT simulators demonstrated excellent construct validity, good face validity, and acceptable workload parameters. These systems thus represent useful training devices and should be widely used to improve surgical performance. Presented at the SAGES 2006 Annual Meeting, Dallas, TX, April 26–29