Face and construct validation of a virtual peg transfer simulator

Background

The Fundamentals of Laparoscopic Surgery (FLS) trainer box is now established as a standard for evaluating minimally invasive surgical skills. A particularly simple task in this trainer box is the peg transfer task which is aimed at testing the surgeon’s bimanual dexterity, hand–eye coordination, speed, and precision. The Virtual Basic Laparoscopic Skill Trainer (VBLaST©) is a virtual version of the FLS tasks which allows automatic scoring and real-time, subjective quantification of performance without the need of a human proctor. In this article we report validation studies of the VBLaST© peg transfer (VBLaST-PT©) simulator.

Methods

Thirty-five subjects with medical background were divided into two groups: experts (PGY 4–5, fellows, and practicing surgeons) and novices (PGY 1–3). The subjects were asked to perform the peg transfer task on both the FLS trainer box and the VBLaST-PT© simulator; their performance was evaluated based on established metrics of error and time. A new length of trajectory (LOT) metric has also been introduced for offline analysis. A questionnaire was used to rate the realism of the virtual system on a 5-point Likert scale.

Results

Preliminary face validation of the VBLaST-PT© with 34 subjects rated on a 5-point Likert scale questionnaire revealed high scores for all aspects of simulation, with 3.53 being the lowest mean score across all questions. A two-tailed Mann–Whitney test performed on the total scores showed significant (p = 0.001) difference between the groups. A similar test performed on the task time (p = 0.002) and the LOT (p = 0.004) separately showed statistically significant differences between the experts and the novices (p < 0.05). The experts appear to be traversing shorter overall trajectories in less time than the novices.

Conclusion

VBLaST-PT© showed both face and construct validity and has promise as a substitute for the FLS for training peg transfer skills.     

Initial validation of the ProMIS surgical simulator as an objective measure of robotic task performance

Virtual reality robotic simulation has gained widespread momentum. In order to determine the value of virtual reality robotic simulation and its objective metrics, a reality-based robotic surgical training platform with similar analytic capabilities must be developed and validated. The ProMIS laparoscopic surgical simulator is a widely available reality-based simulation platform that has been previously validated as an objective measure of laparoscopic task performance. In this study, we evaluated the validity of the ProMIS laparoscopic surgical simulator as an objective measure of robotic task performance. Volunteers were recruited from two experience groups (novice and expert). All subjects completed three tasks (peg transfer, precision cutting, intracorporeal suture/knot) in the ProMIS laparoscopic simulator using the da Vinci robotic surgical system. Motion analysis data was obtained by the ProMIS computerized optical tracking system and objective metrics recorded included time, path length, economy of motion, and observer-recorded penalty scores. The novice group consisted of 10 subjects with no previous robotic surgical experience. The expert group consisted of 10 subjects with robotic experience. The expert group outperformed the novice group in all three tasks. Subjects rated this training platform as easy to use, as an accurate measure of their robotic surgical proficiency, and as relevant to robotic surgery. The experts described the simulator platform as useful for training and agreed with incorporating it into a residency curriculum. This study demonstrates that the ProMIS laparoscopic simulator is a face, content, and construct valid reality-based simulation platform that can be used for objectively measuring robotic task performance.     

Oncologic outcomes of robotic thyroidectomy: 5-year experience with propensity score matching

Background

The oncologic outcome of robotic thyroidectomy is not yet well established. The aim of this study was to evaluate the recurrence rate after robotic thyroidectomy in comparison with conventional thyroidectomy for papillary thyroid carcinoma (PTC).

Methods

We analyzed 896 patients with PTC who either underwent robotic (212 patients using a gasless unilateral axillary or an axillo-breast approach) or conventional cervical thyroidectomy (684 patients) with/without central neck dissection between October 2008 and February 2014. We excluded patients who underwent concomitant lateral neck dissection or completion thyroidectomy, and cases with T4 tumor, tumor lager than 4 cm, other types of thyroid cancer, recurrent cancer, and distant metastasis. A propensity score matching analysis was done with ten covariates including age, gender, body mass index, tumor size, multifocality, bilaterality, extrathyroidal extension, type of thyroidectomy, extent of central neck dissection, and RAI ablation to reduce selection bias.

Results

In baseline data, the male ratio and the mean age were lower in the robotic group. Stage, multifocality, and bilaterality were higher in the conventional group. The rate of total thyroidectomy was higher in the conventional group. The two matched groups of each 185 patients were produced and well balanced by propensity score matching. In the comparison of propensity score matched groups, operative time was longer in the robotic group (P < 0.001), and postoperative complications did not differ between the two groups, except for transient hypoparathyroidism and formation of seroma. The recurrence rate did not differ between the two groups after a mean follow-up of 43.6 months (0.5 and 1.1 % in the robotic and conventional groups, respectively, P = 0.375).

Conclusion

The oncologic outcome of robotic thyroidectomy in 5-year experience is comparable to that of conventional thyroidectomy in selected patients with PTC.

     

Salient haptic skills trainer: initial validation of a novel simulator for training force-based laparoscopic surgical skills

Background

There is an increasing need for efficient training simulators to teach advanced laparoscopic skills beyond those imparted by a box trainer. In particular, force-based or haptic skills must be addressed in simulators, especially because a large percentage of surgical errors are caused by the over-application of force. In this work, the efficacy of a novel, salient haptic skills simulator is tested as a training tool for force-based laparoscopic skills.

Methods

Thirty novices with no previous laparoscopic experience trained on the simulator using a pre-test–feedback–post-test experiment model. Ten participants were randomly assigned to each of the three salient haptic skills—grasping, probing, and sweeping—on the simulator. Performance was assessed by comparing force performance metrics before and after training on the simulator.

Results

Data analysis indicated that absolute error decreased significantly for all three salient skills after training. Participants also generally decreased applied forces after training, especially at lower force levels. Overall, standard deviations also decreased after training, suggesting that participants improved their variability of applied forces.

Conclusions

The novel, salient haptic skills simulator improved the precision and accuracy of participants when applying forces with the simulator. These results suggest that the simulator may be a viable tool for laparoscopic force skill training. However, further work must be undertaken to establish full validity. Nevertheless, this work presents important results toward addressing simulator-based force-skills training specifically and surgical skills training in general.     

机器人手术系统辅助妇科手术60例报告

目的:探讨达芬奇机器人手术系统辅助妇科手术的安全性及可行性。方法:回顾分析2009年2月至2012年5月为60例患者行达芬奇机器人手术系统辅助妇科手术的临床资料。结果:9例行子宫内膜癌分期手术,8例行全子宫切除术,1例行子宫次全切除术,26例行子宫肌瘤切除术,10例行子宫内膜异位症手术,2例行卵巢肿瘤手术,4例行宫颈癌根治术;其中2例中转开腹。手术时间平均(132.22±40.42)min,术中出血量平均(136.58±60.14)ml,术后肠蠕动恢复时间平均(23.6±13.4)h,术后平均住院(8.2±5.1)d。术后1例发生不全肠梗阻,怀疑脏器穿孔,再次经腹腔镜探查未发现异常,余者均无并发症发生。结论:达芬奇机器人手术系统应用于妇科领域安全、可行,但要求术者不仅要熟悉盆腔解剖结构,具有丰富的开腹手术经验,而且应具备娴熟的腹腔镜技术,术中遇意外情况可妥善处理。     

Fibreoptic airway training: correlation of simulator performance and clinical skill

PURPOSE: Simulation centres, where trainees can practise technical procedures on models of varying fidelity, provide a training option that allows them to acquire skills in a controlled environment prior to clinical performance. It has been proposed that the time to complete a simulator task may translate to proficiency in the clinical setting. The objective of this study was to determine whether time to complete a simulator task translates to clinical fibreoptic manipulation (FOM) performance. METHODS: Thirty registered respiratory therapists at a teaching hospital were recruited as subjects for a single-blinded randomized trial. Subjects were randomized to training on either a low fidelity (n = 15) or high fidelity (n = 15) model. After training, each subject was tested for the time required to complete a specific task on his/her respective model. Subjects then performed a fibreoptic orotracheal intubation (FOI) on healthy, consenting, and anesthetised patients requiring intubation for elective surgery. Performance was measured independently by blinded examiners using a checklist and global rating scale (GRS); and time was measured from insertion of the fibreoptic scope to visualization of the carina. Data were analyzed using Spearman rank order correlation coefficients. RESULTS: There was no correlation between the time to complete a task on either the high or low fidelity simulators, and the clinical FOI performance as assessed by a checklist, GRS, and time to complete the FOM (all P = NS). CONCLUSION: These results suggest that simulator-based, task-orientated time measurement may not be a good indicator of FOI performance in the clinical setting.     

Face,content and construct validation of the first virtual reality laparoscopic nephrectomy simulator

Study Type – Therapy (case series)
Level of Evidence 4

OBJECTIVE

To evaluate the face, content and construct validity, and to identify whether participants improved with practice, for the Procedicus MIST Nephrectomy? simulator (Mentice, Gothenburg, Sweden), which incorporates force feedback and can record numerous performance measures (metrics) during the simulation, and which is the first virtual reality simulator for laparoscopic nephrectomy.

SUBJECTS AND METHODS

Eight expert urological laparoscopic surgeons, 10 novices and 10 trainee urologists completed at least one simulated retroperitoneal radical nephrectomy. They completed a structured questionnaire to assess face and content validity; the performance of experts and novices were then compared to evaluate construct validity.

RESULTS

Face validity was established by the experts who all felt the simulator was a good training tool. Content validity was established by the experts who rated all aspects of the simulator as above average for realism. When performance metrics were analysed, experts completed the simulated nephrectomy significantly faster than novices, with fewer errors, less simulated haemorrhage and less tool travel, which established construct validity. After practice, both novices and trainees were able to perform the simulation faster, with fewer errors, less blood loss and less tool travel.

CONCLUSIONS

This study has established the face, content and construct validity for the Procedicus MIST Nephrectomy. The simulator can distinguish experts from novices and it has potential as a laparoscopic training tool for urology.     

Virtual reality uterine resectoscopic simulator: face and construct validation and comparative evaluation in an educational environment

Background and Objectives:

Recognizing that resectoscopic simulation may have an educational role, this pilot study was designed to evaluate the face validity and educational utility of a virtual reality uterine resectoscope training system.

Methods:

A pilot prospective comparative study of novice and expert hysteroscopists'' performance on a targeting exercise and myomectomy with the virtual loop electrode. At baseline, expert and novice resectoscopists each performed both exercises. Following instruction, novices practiced each exercise a total of 9 times with the 10th recorded as the training outcome. Results were compared both to baseline and to those of the experts. Data were analyzed with the paired t and Wilcoxon rank sum tests as appropriate.

Results:

At baseline, all experts touched 4 targets in a mean of 33 seconds with no perforations, compared to a mean of 2 for the 11 novices in a mean of 57 seconds (P=0.0034) with one perforation. In 3 minutes, the experts removed a mean of 97.3% of the virtual myoma, compared to 66.1% for the novices (P=0.0153). On the 10th “run,” novices touched a mean of 4 targets in a mean of 23 seconds, an improvement from baseline (P=0.0004) and improved to 89% on the myoma resection exercise (P=0.0515) 36.3% over baseline.

Conclusion:

Although this pilot study has a relatively small sample size and represents the results at one institution, it demonstrates that virtual reality resectoscopic systems have the potential to measure and improve the technical skills of novices before they operate on human patients.     

Correlation of the RENAL nephrometry score with warm ischemia time after robotic partial nephrectomy

Purpose

The RENAL nephrometry score (RNS) was developed to quantify complexity of renal tumors in a reproducible manner. We aim to determine whether individual categories of the RNS have different impact on the warm ischemia time (WIT) for patients undergoing robotic partial nephrectomy (RPN).

Methods

In a retrospective analysis of a prospectively maintained database, we identified 251 consecutive patients who underwent RPN between January 2007 and June 2010. RNS was determined in 187 with available imaging. Univariable analysis and multivariable linear regression analysis were performed to identify which factors were more significantly associated with WIT.

Results

Overall RNS was of low (4–6), moderate (7–9), and high complexity (10–12) in 84 (45 %), 80 (43 %), and 23 (12 %) patients, respectively. There was no association between gender (p = 0.6), BMI (p = 0.3), or anterior/posterior location (A) (p = 0.8), and WIT. On univariable analysis, longer WIT was associated with size (R) >4 cm (p < 0.0001), entirely endophytic properties (E) (p = 0.005), tumor <4 mm from the collecting system/sinus (N) (p < 0.0001), and location between the polar lines (L) (p = 0.004). Total RNS and WIT were highly correlated (Spearman correlation coefficient = 0.54, p < 0.0001). There was a significant trend of higher WIT with increased tumor complexity (p for trend <0.0001). After multivariable analysis, only R (p = 0.0003), E (p = 0.003), and N (p = 0.00002) components of the RNS were significantly associated with WIT.

Conclusions

The A and L subcategories of the RNS have no significant impact on the WIT of patients undergoing RPN. WIT is significantly dependent upon the other subcategories, as well as the overall RNS. These findings can be used to preoperatively predict which tumor characteristics will likely affect WIT and may be useful in preoperative counseling as well as planning of approach.     

Face, content and construct validity of a virtual reality simulator for robotic surgery (SEP Robot)

INTRODUCTION

This study aims to establish face, content and construct validation of the SEP Robot (SimSurgery, Oslo, Norway) in order to determine its value as a training tool.

SUBJECTS AND METHODS

The tasks used in the validation of this simulator were arrow manipulation and performing a surgeon''s knot. Thirty participants (18 novices, 12 experts) completed the procedures.

RESULTS

The simulator was able to differentiate between experts and novices in several respects. The novice group required more time to complete the tasks than the expert group, especially suturing. During the surgeon''s knot exercise, experts significantly outperformed novices in maximum tightening stretch, instruments dropped, maximum winding stretch and tool collisions in addition to total task time. A trend was found towards the use of less force by the more experienced participants.

CONCLUSIONS

The SEP robotic simulator has demonstrated face, content and construct validity as a virtual reality simulator for robotic surgery. With steady increase in adoption of robotic surgery world-wide, this simulator may prove to be a valuable adjunct to clinical mentorship.     

Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills

BACKGROUND: The study was carried out to analyze the learning rate for laparoscopic skills on a virtual reality training system and to establish whether the simulator was able to differentiate between surgeons with different laparoscopic experience. METHODS: Forty-one surgeons were divided into three groups according to their experience in laparoscopic surgery: masters (group 1, performed more than 100 cholecystectomies), intermediates (group 2, between 15 and 80 cholecystectomies), and beginners (group 3, fewer than 10 cholecystectomies) were included in the study. The participants were tested on the Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) 10 consecutive times within a 1-month period. Assessment of laparoscopic skills included time, errors, and economy of hand movement, measured by the simulator. RESULTS: The learning curves regarding time reached plateau after the second repetition for group 1, the fifth repetition for group 2, and the seventh repetition for group 3 (Friedman's tests P <0.05). Experienced surgeons did not improve their error or economy of movement scores (Friedman's tests, P >0.2) indicating the absence of a learning curve for these parameters. Group 2 error scores reached plateau after the first repetition, and group 3 after the fifth repetition. Group 2 improved their economy of movement score up to the third repetition and group 3 up to the sixth repetition (Friedman's tests, P <0.05). Experienced surgeons (group 1) demonstrated best performance parameters, followed by group 2 and group 3 (Mann-Whitney test P <0.05). CONCLUSIONS: Different learning curves existed for surgeons with different laparoscopic background. The familiarization rate on the simulator was proportional to the operative experience of the surgeons. Experienced surgeons demonstrated best laparoscopic performance on the simulator, followed by those with intermediate experience and the beginners. These differences indicate that the scoring system of MIST-VR is sensitive and specific to measuring skills relevant for laparoscopic surgery.