Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training

BACKGROUND: The fundamentals of laparoscopic surgery (FLS) program has been extensively validated for use as a high-stakes examination for certification purposes, but optimal methods for its use in skills training have not been described. This study aimed to investigate the feasibility of implementing a proficiency-based FLS skills training curriculum and to evaluate its effectiveness in preparing trainees for certification. METHODS: For this study, 21 novice medical students at two institutions viewed video tutorials, then performed one repetition of the five FLS tasks as a pretest. The pretests were scored using standard testing metrics. The trainees next practiced the tasks over a 2-month period until they achieved proficiency for all the tasks. A modified on-the-fly scoring system based on expert-derived performance was used. The trainees were posttested using the high-stakes examination format. RESULTS: No trainee passed the certification examination at pretesting. The trainees achieved proficiency for 96% of the five tasks during training, which required 9.7 +/- 2.4 h (range, 6-14 h) and 119 +/- 31 repetitions (range, 66-161 repetitions). The trainees rated the proficiency levels as "moderately difficult" (3.0 +/- 0.7 on a 5-point scale) and "highly appropriate" (4.7 +/- 0.1 on a 5-point scale). At posttesting, 100% of the trainees passed the certification examination and demonstrated significant improvement compared with pretesting for normalized score (468 +/- 24 vs 126 +/- 75; p < 0.001), self-rated laparoscopic comfort (89.4% vs 4.8%; p < 0.001), and skill level (3.6 +/- 0.9 vs 1.2 +/- 0.5; p < 0.001, 5-point scale). CONCLUSIONS: This proficiency-based curriculum is feasible for training novices and uniformly allows sufficient skill acquisition for FLS certification. Endorsed by the Society of American Gastrointestinal Endoscopic Surgeons (SAGES), this curriculum is available for use as an optimal method for FLS skills training. More widespread adoption of this curriculum is encouraged.     

Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises

Background

We previously developed nine inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity, workload, and expert levels for the nine exercises.

Methods

Expert robotic surgeons (n?=?8, fellows and faculty) and novice trainees (n?=?4, medical students) each performed three to five consecutive repetitions of nine previously reported exercises (five FLS models with or without modifications and four custom-made models). Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established (preliminary) proficiency level and a composite score equaled the sum of the nine normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort, and frustration levels of each task.

Results

Experts had performed 119 (range?=?15–600) robotic operations; novices had observed ≤1 robotic operation. For all nine tasks and the composite score, experts achieved significantly better performance than novices (932?±?67 vs. 618?±?111, respectively; P?Conclusion Using objective performance metrics, all nine exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the nine structured exercises are suitable for proficiency-based robotic training.     

Preparing medical students to enter surgery residencies

Background

This study was designed to develop and evaluate an integrated cognitive and proficiency-based skills curriculum based on American College of Surgeons Graduate Medical Education Committee (ACGME) competencies to prepare students for surgery internships.

Methods

Course content included cadaver dissections, didactic sessions, team training, and training in clinical and technical skills. Using previously validated skills curricula (12 open and 5 Fundamental Laparoscopic Surgery [FLS] tasks), trainees underwent pretest, self-practice to proficiency, and post-test. Surveys regarding confidence levels and skills were administered.

Results

Mean course evaluation score was 4.5 ± .6 on a 5-point Likert scale. Trainees (n = 9) achieved proficiency on open tasks and FLS tasks 2-5. The mean confidence self-rating on 51 skills increased on a 5-point Likert scale from 2.4 ± .6 to 4.0 ± .6 (P < .001).

Conclusions

This integrated curriculum did improve confidence levels, and skills proficiency can be achieved in an abbreviated time. The anticipated result would be enhanced baseline abilities for internship.     

Value of fundamentals of laparoscopic surgery training in a fourth-year medical school advanced surgical skills elective

BackgroundFew data are available describing the benefits of initiating fundamentals of laparoscopic surgery (FLS) training during medical school. We hypothesized that an intense 1-month surgical skills elective that included FLS task training for fourth-year medical students (MS4s) would result in performance levels indistinguishable from graduating chief residents (PGY5) who had received clinical skill training and access to self-guided FLS curriculum.MethodsFrom July 2007 through June 2011, 114 MS4s participated in a 1-month advanced surgical skills elective. The curriculum for the elective included cadaver dissections, patient management presentations, and surgical skill training (open surgical skills and basic laparoscopic skills modules performed on FLS trainers and virtual reality laparoscopic simulators). From June 2009 through June 2011, 21 PGY5s graduated who had never received formalized FLS skills training. These residents were tested on FLS by a certified proctor and the results recorded. The performance outcome measure was task completion time. Unpaired Student's t-test was used to compare the performance measures for each group.ResultsAll PGY5s achieved FLS certification on their first attempt and completed enough cases for graduation. The MS4 group showed significantly better performance than the PGY5 group in the peg transfer and circle cut (P < 0.05). No difference was seen in the knot tying tasks between the two groups (P > 0.05)ConclusionsIncorporating FLS training into a 1 month-long medical school surgery elective enabled MS4s to achieve FLS performance similar to, or better than, the performance achieved by PGY5 surgery residents. We support the integration of FLS skills task training as a standard part of the skills training curriculum for medical students.     

Proficiency training on a virtual reality robotic surgical skills curriculum

Introduction

The clinical application of robotic surgery is increasing. The skills necessary to perform robotic surgery are unique from those required in open and laparoscopic surgery. A validated laparoscopic surgical skills curriculum (Fundamentals of Laparoscopic Surgery or FLS?) has transformed the way surgeons acquire laparoscopic skills. There is a need for a similar skills training and assessment tool for robotic surgery. Our research group previously developed and validated a robotic training curriculum in a virtual reality (VR) simulator. We hypothesized that novice robotic surgeons could achieve proficiency levels defined by more experienced robotic surgeons on the VR robotic curriculum, and that this would result in improved performance on the actual daVinci Surgical System?.

Methods

25 medical students with no prior robotic surgery experience were recruited. Prior to VR training, subjects performed 2 FLS tasks 3 times each (Peg Transfer, Intracorporeal Knot Tying) using the daVinci Surgical System? docked to a video trainer box. Task performance for the FLS tasks was scored objectively. Subjects then practiced on the VR simulator (daVinci Skills Simulator) until proficiency levels on all 5 tasks were achieved before completing a post-training assessment of the 2 FLS tasks on the daVinci Surgical System? in the video trainer box.

Results

All subjects to complete the study (1 dropped out) reached proficiency levels on all VR tasks in an average of 71 (± 21.7) attempts, accumulating 164.3 (± 55.7) minutes of console training time. There was a significant improvement in performance on the robotic FLS tasks following completion of the VR training curriculum.

Conclusions

Novice robotic surgeons are able to attain proficiency levels on a VR simulator. This leads to improved performance in the daVinci surgical platform on simulated tasks. Training to proficiency on a VR robotic surgery simulator is an efficient and viable method for acquiring robotic surgical skills.     

Beyond task time: automated measurement augments fundamentals of laparoscopic skills methodology

Background

Laparoscopic psychomotor skills are challenging to learn and objectively evaluate. The Fundamentals of Laparoscopic Skills (FLS) program provides a popular, inexpensive, widely-studied, and reported method for evaluating basic laparoscopic skills. With an emphasis on training safety before efficiency, we present data that explore the metrics in the FLS curriculum.

Materials and methods

A multi-institutional (n = 3) cross-sectional study enrolled subjects (n = 98) of all laparoscopic skill levels to perform FLS tasks in an instrumented box trainer. Recorded task videos were postevaluated by faculty reviewers (n = 2) blinded to subject identity using a modified Objective Structured Assessment of Technical Skills (OSATS) protocol. FLS scores were computed for each completed task and compared with demographically established skill levels (training level and number of procedures), video review scoring, and objective performance metrics including path length, economy of motion, and peak grasping force.

Results

Three criteria used to determine expert skill, training and experience level, blinded review of performance by faculty via OSATS, and FLS scores, disagree in establishing concurrent validity for determining “true experts” in FLS tasks. FLS-scoring exhibited near-perfect correlation with task time for all three tasks (Pearson r = 0.99, 1.00, 1.00 with P <0.00000001). FLS error penalties had negligible effect on FLS scores. Peak grasping force did not correlate with task time or FLS scores.

Conclusions

FLS technical skills scores presented negligible benefit beyond the measurement of task time. FLS scoring is weighted more toward speed than precision and may not significantly address poor tissue handling skills, especially regarding excessive grasping force. Categories of experience or training level may not form a suitable basis for establishing proficiency thresholds or for construct validity studies for technical skills.     

Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program

The purpose of this article is to describe our experience with the incorporation of a proficiency-based laparoscopic skills curriculum in a busy surgical training program that aims to improve the technical proficiency of residents. The curriculum has a cognitive component and a manual skills component and is adjusted to resident training level. It is based on the Fundamentals of Laparoscopic Surgery program and includes basic laparoscopic virtual-reality tasks of the Lap Mentor simulator (Simbionix USA Corp., Cleveland, Ohio). Training occurs in weekly 1-hour sessions until expert-derived performance goals are achieved. Maintenance training ensures skill retention. Performance is assessed with objective metrics and is supported with feedback and an award system. Resident workload is assessed at regular intervals. Knowledge tests and manual skills tests are administered at the beginning and end of the academic year to assess resident performance improvement and curriculum effectiveness. Resident attendance rates and training progress are monitored continuously, and training sessions are adjusted to individual needs. Our curriculum has been implemented for several months. Our experience so far suggests that it is imperative to have dedicated supervising personnel and dedicated training time in the busy week of the surgical resident to ensure attendance. Our next step is to incorporate the 20 modules of the new Association of Program Directors in Surgery (ADPS)/American College of Surgeons (ACS) national skills curriculum into our skills training program, to expand its cognitive component by incorporating additional procedural videos, and to adapt scenario-based training on trauma and critical care on human patient simulators.     

Skill retention following proficiency-based laparoscopic simulator training

BACKGROUND: Proficiency-based curricula using both virtual reality (VR) and videotrainer (VT) simulators have proven to be efficient and maximally effective, but little is known about the retention of acquired skills. The purpose of this study was to assess skill retention after completion of a validated laparoscopic skills curriculum. METHODS: Surgery residents (n=14) with no previous VR or VT experience were enrolled in an Institutional Review Board-approved protocol and sequentially practiced 12 Minimally Invasive Surgical Trainer-VR and 5 VT tasks until proficiency levels were achieved. One VR (manipulate diathermy) and 1 VT (bean drop) tasks were selected for assessment at baseline, after training completion (posttest), and at retention. RESULTS: All residents completed the curriculum. Posttest assessment occurred at 13.2 +/- 11.8 days and retention assessment at 7.0 +/- 4.0 months. After an early performance decrement at posttest (17%-45%), the acquired skill was maintained up to the end of the follow-up period. For VR, scores were 81.5 +/- 23.5 at baseline, 33.3 +/- 1.8 at proficiency, 48.4 +/- 9.2 at posttest, and 48.4 +/- 11.8 at retention. For VT, scores were 49.4 +/- 12.5 at baseline, 22.0 +/- 1.4 at proficiency, 25.6 +/- 3.6 at posttest, and 26.4 +/- 4.2 at retention. Skill retention was better for VT, compared with VR (P < .02). The extent of skill deterioration did not correlate with training duration or resident level. CONCLUSIONS: Although residents do not retain all acquired skills (more so for VR than for VT) according to simulator assessment, proficiency-based training on simulators results in durable skills. Additional studies are warranted to further optimize curriculum design, investigate simulator differences, and establish training methods that improve skill retention.     

Proficiency-based Fundamentals of Laparoscopic Surgery skills training results in durable performance improvement and a uniform certification pass rate

Background  

The authors have previously documented a 100% certification pass rate immediately after a proficiency-based skills training curriculum for the Fundamentals of Laparoscopic Surgery (FLS) program. This study aimed to determine the durability of skills acquired after initial training.     

Ensuring competency: Are fundamentals of laparoscopic surgery training and certification necessary for practicing surgeons and operating room personnel?

Background

Certification in fundamentals of laparoscopic surgery (FLS) is required by the American board of surgery for graduating residents. This study aimed to evaluate the feasibility and need for certifying practicing surgeons and to assess proficiency of operating room (OR) personnel.

Methods

Through a patient safety and health care delivery effectiveness grant, investigators at four state medical schools received funding for FLS certification of all attending surgeons and OR personnel credentialed in laparoscopy. Data were voluntarily collected under an institutional review board-approved protocol. Surgeons performed a single repetition of the FLS tasks oriented to the FLS proficiency-based curriculum and online cognitive materials and were encouraged to self-practice. The FLS certification examination was administered 2 months later under standard conditions. Operating room nurses and scrub technicians were enrolled in a curriculum with cognitive materials and a multistation skills practicum. Baseline and completion questionnaires were administered. Performance was assessed using signed-rank and χ² analysis.

Results

The study aimed to enroll 99 surgeons. Subsequently, 87 surgeons completed at least one portion of the curriculum, 72 completed the entire curriculum (73 % compliance), 83 completed the baseline skills assessment, and 27 (33 %) failed. The self-reported practice time was 3.7 ± 2.5 h. At certification (n = 76), skills performance had improved from 317 ± 102.9 to 402 ± 54.2 (p < 0.0001). One surgeon (1.3 %) failed the skills certification, and nine (11.8 %) failed the cognitive exam. Remediation was completed by six surgeons. Of the 64 enrolled OR personnel, 22 completed the curriculum (34 % compliance). All achieved proficiency at skills, and 60 % passed the cognitive exam.

Conclusions

This study demonstrated that FLS certification for practicing surgeons and proficiency verification for OR personnel are feasible. A baseline skills failure rate of 33 % and a certification failure rate of 13 % suggest that FLS certification may be necessary to ensure surgeon competency. Fortunately, with only moderate practice, significant improvement can be achieved.     

PGY-specific benchmarks improve resident performance on Fundamentals of Laparoscopic Surgery tasks

Background

Although expert proficiency times for Fundamentals of Laparoscopic Surgery (FLS) tasks exist, these times are not always attainable for junior residents. We hypothesize that post-graduate year (PGY)-specific benchmarks will improve resident performance of FLS tasks.

Implementation, construct validity, and benefit of a proficiency-based knot-tying and suturing curriculum

OBJECTIVES: The aim of this proficiency-based, open knot-tying and suturing study was to evaluate the feasibility of implementing this curriculum within a residency program, and to assess construct validity and educational benefit. METHODS: PGY1 residents (n = 37) were enrolled in an Institutional Review Board (IRB)-approved prospective study that was conducted over a 12-week period. Trainees viewed a video tutorial during orientation and as needed; they self-practiced to proficiency for 12 standardized knot-tying, practiced suturing tasks; performed 1 repetition of each task at baseline and posttesting; and completed questionnaires. RESULTS: Curriculum implementation required 376 person-hours, and material costs were $776. All trainees achieved proficiency within allotted 12 weeks. Overall, trainees completed 141 +/- 80 repetitions over 12.7 +/- 5.3 hours in addition to performing 13.4 +/- 12.4 operations. Baseline trainee and expert performance were significantly different for all 12 tasks and composite score (732 +/- 294 vs 1488 +/- 26, p < 0.001), which supported construct validity. Baseline trainees demonstrated significant improvement at posttesting according to composite scores (732 +/- 294 vs 1503 +/- 131, p < 0.001), which validates skill acquisition. CONCLUSIONS: Implementation of this proficiency-based curriculum within the constraints of a residency program is feasible. This curriculum is educationally beneficial and cost effective; our data support construct validity. Evaluation of transferability to the operating room and more widespread adoption of this curriculum are warranted.     

A cost-effective proficiency-based knot-tying and suturing curriculum for residency programs

BACKGROUND: The purpose of this study was to develop a structured open skills curriculum for knot-tying and suturing using expert-derived performance goals and to examine its feasibility, cost-effectiveness, and construct validity. METHODS: Using commercially available bench models, 11 standardized tasks (ranging from 2-handed knot-tying to running subcuticular closure) were developed and scored using previously validated metrics based on time and errors. Expert performance was used to establish training endpoints and to create a video tutorial. PGY 1 residents (n = 4) were enrolled in a prospective Institutional Review Board-approved pilot study that included proctored orientation and baseline testing, self-training to proficiency, and proctored post-testing (conducted over a 4-wk period). Baseline trainee scores were compared with expert scores to evaluate construct validity. RESULTS: The 11 tasks proved relatively robust, and excellent feedback was obtained from the trainees regarding educational benefit. Overall, trainees performed 144 +/- 33 repetitions over 11 +/- 2 h. Trainees achieved proficiency for 4.6% of the 11 tasks at baseline, 91% during training, and 84% at post-testing. Trainees demonstrated significant improvement from baseline to post-testing, validating skill acquisition; baseline trainee and expert performance were significantly different, confirming construct validity. Curriculum development cost $1200 and required 72 man-hours. Incremental training cost less than $12 per participant and required 8 man-hours per rotation using the video-based self-practice curriculum. In response to participant feedback, two of the 11 tasks were modified and a twelfth task was added. CONCLUSIONS: This curriculum is cost-effective, feasible within the context of residency training, educationally beneficial, and demonstrates construct validity. More widespread adoption of standardized, validated skills curricula such as this by residency programs is warranted.     

A Randomized Prospective Study Comparing Acquisition of Laparoscopic Skills in Three-Dimensional (3D) vs. Two-Dimensional (2D) Laparoscopy

Background

We aimed to compare the performance of novices with three-dimensional (3D) versus two-dimensional (2D) laparoscopy using Fundamentals of Laparoscopic Surgery (FLS) tasks.

Methods

Fifty-six novices with no uncorrected visual problems were randomly allocated to 2D and 3D groups. All candidates practiced FLS tasks on a box trainer until they achieved proficiency. Their performance was assessed by considering completion time, number of repetitions, and number of errors following the validated FLS proficiency criteria.

Results

Twenty-five participants in each group completed the training curriculum. The median performance time (in minutes) for the 3D group was 216, which was less than that of the 2D group of 247 min (P = 0.266). The median numbers of repetitions and errors were lower for the 3D group versus the 2D group: 108 versus 121 (P = 0.008) and 27 versus 105 (P < 0.001), respectively.

Conclusion

Stereoscopic vision improved accuracy in laparoscopic skills for novices, which was manifested in reduced numbers of repetitions and errors. However, it does not affect the global performance time across all tasks.     

Proficiency-based laparoscopic simulator training leads to improved operating room skill that is resistant to decay

The aim of this study was to assess skill retention in the operating room following completion of a proficiency-based laparoscopic skills curriculum. Novices (n = 15) were randomized to a control and a training group that practiced to proficiency on the Fundamentals of Laparoscopic Surgery suturing model. The performance of both groups was assessed on the simulator and on a live porcine laparoscopic Nissen fundoplication model at training completion (posttest) and 5 months later (retention test). Training to proficiency required 4.7 +/-1.2 hours and 41 +/- 10 repetitions. Trained participants outperformed controls, and their performance deteriorated slightly between posttests and retention tests on the simulator (505 +/- 22 vs 462 +/- 50, respectively; P < .05) but not in operating room (263 +/- 138 vs 279 +/- 88, respectively; P = .38). Proficiency-based simulator training results in durable improvement in operative skill of trainees even in the absence of practice for up to 5 months. Minute simulator performance changes do not translate to the operating room.     

The surgery clerkship: an opportunity for preclinical credentialing in urinary catheterization

BackgroundThe authors previously developed a comprehensive, proficiency-based robotic training curriculum that aimed to address 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to determine the content and face validity of this curriculum.MethodsExpert robotic surgeons (n = 12) rated each deconstructed skill regarding relevance to robotic operations, were oriented to the curricular components, performed 3 to 5 repetitions on the 9 exercises, and rated each exercise.ResultsIn terms of content validity, experts rated all 23 deconstructed skills as highly relevant (4.5 on a 5-point scale). Ratings for the 9 inanimate exercises indicated moderate to thorough measurement of designated skills. For face validity, experts indicated that each exercise effectively measured relevant skills (100% agreement) and was highly effective for training and assessment (4.5 on a 5-point scale).ConclusionsThese data indicate that the 23 deconstructed skills accurately represent the appropriate content for robotic skills training and strongly support content and face validity for this curriculum.     

Validation of a virtual reality-based robotic surgical skills curriculum

Background

The clinical application of robotic-assisted surgery (RAS) is rapidly increasing. The da Vinci Surgical System? is currently the only commercially available RAS system. The skills necessary to perform robotic surgery are unique from those required for open and laparoscopic surgery. A validated laparoscopic surgical skills curriculum (fundamentals of laparoscopic surgery or FLS?) has transformed the way surgeons acquire laparoscopic skills. There is a need for a similar skills training and assessment tool specific for robotic surgery. Based on previously published data and expert opinion, we developed a robotic skills curriculum. We sought to evaluate this curriculum for evidence of construct validity (ability to discriminate between users of different skill levels).

Methods

Four experienced surgeons (>20 RAS) and 20 novice surgeons (first-year medical students with no surgical or RAS experience) were evaluated. The curriculum comprised five tasks utilizing the da Vinci? Skills Simulator (Pick and Place, Camera Targeting 2, Peg Board 2, Matchboard 2, and Suture Sponge 3). After an orientation to the robot and a period of acclimation in the simulator, all subjects completed three consecutive repetitions of each task. Computer-derived performance metrics included time, economy of motion, master work space, instrument collisions, excessive force, distance of instruments out of view, drops, missed targets, and overall scores (a composite of all metrics).

Results

Experienced surgeons significantly outperformed novice surgeons in most metrics. Statistically significant differences were detected for each task in regards to mean overall scores and mean time (seconds) to completion.

Conclusions

The curriculum we propose is a valid method of assessing and distinguishing robotic surgical skill levels on the da Vinci Si? Surgical System. Further study is needed to establish proficiency levels and to demonstrate that training on the simulator with the proposed curriculum leads to improved robotic surgical performance in the operating room.     

Does gender predict performance of novices undergoing Fundamentals of Laparoscopic Surgery (FLS) training?

BackgroundThis study was performed to assess the hypothesis that performance levels for Fundamentals of Laparoscopic Surgery (FLS) tasks were not dependent on trainee gender.MethodsInitial and final task completion times for 4 FLS tasks performed by 132 novices (4th-year medical students and 1st-year residents) were collated by task type and compared by gender.ResultsAll novices improved significantly with training (P > .001) on all tasks. Initial performance by men was better than by women but only reached significance for peg transfer and intracorporeal knot tying (P > .05). With training, women's performance became equivalent to that of men but showed a comparable or greater response to training.ConclusionsThe gender-related differences noted in initial FLS task performance disappeared after training. Gender displayed no effect on FLS training outcomes. The use of initial FLS task performance levels, seemingly objective selection criteria, would introduce gender bias into the ranking process.     

Developing criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year

BACKGROUND: Proficiency-based residency training programs can be more efficient than the current duration-based formats. For their successful implementation, appropriate proficiency criteria must be developed. The objective of this study was to investigate the relationship between technical skill performances assessed using computer- and expert-based methods and training year. An assumption was that asymptotes in performance as a function of training year can be used to set the proficiency level for a technical skill, so the value at which the asymptote occurs can be labeled as the proficiency criteria. STUDY DESIGN: Thirty-eight general surgery residents performed one-handed knot tying on bench-top simulators at two levels of difficulty: superficial and deep. Motion-efficiency measures and expert-based measures were used to evaluate performance. Total number of operations (ie, surgical volume) that each trainee participated in during residency was also acquired. RESULTS: On the superficial model, asymptotes were observed at year 1 for motion-efficiency and year 3 for expert-based measures. On the deep model, asymptotes were observed at year 2 for motion-efficiency and year 4 for expert-based measures. CONCLUSIONS: The data demonstrate the challenges associated with defining technical skills proficiency criteria. Different asymptotes were observed for the two assessment methods and neither covaried substantially with surgical volume. These data suggest that this asymptote approach in defining proficiency criteria can be suitable for development of proficiency-based residency training programs. The sensitivity of this approach to the type of assessment method and to the functional difficulty of the simulators used for assessment must be considered.     

Comprehensive proficiency-based inanimate training for robotic surgery: reliability, feasibility, and educational benefit

Background

We previously developed a comprehensive proficiency-based robotic training curriculum demonstrating construct, content, and face validity. This study aimed to assess reliability, feasibility, and educational benefit associated with curricular implementation.

Methods

Over an 11-month period, 55 residents, fellows, and faculty (robotic novices) from general surgery, urology, and gynecology were enrolled in a 2-month curriculum: online didactics, half-day hands-on tutorial, and self-practice using nine inanimate exercises. Each trainee completed a questionnaire and performed a single proctored repetition of each task before (pretest) and after (post-test) training. Tasks were scored for time and errors using modified FLS metrics. For inter-rater reliability (IRR), three trainees were scored by two raters and analyzed using intraclass correlation coefficients (ICC). Data from eight experts were analyzed using ICC and Cronbach’s α to determine test-retest reliability and internal consistency, respectively. Educational benefit was assessed by comparing baseline (pretest) and final (post-test) trainee performance; comparisons used Wilcoxon signed-rank test.

Results

Of the 55 trainees that pretested, 53 (96?%) completed all curricular components in 9–17?h and reached proficiency after completing an average of 72?±?28 repetitions over 5?±?1?h. Trainees indicated minimal prior robotic experience and “poor comfort” with robotic skills at baseline (1.8?±?0.9) compared to final testing (3.1?±?0.8, p?p?p?p?Conclusion This curriculum is associated with high reliability measures, demonstrated feasibility for a large cohort of trainees, and yielded significant educational benefit. Further studies and adoption of this curriculum are encouraged.