Technical skills assessment as part of the selection process for a fellowship in minimally invasive surgery

Introduction

Selection of candidates for surgical fellowships has traditionally been based on subjective evaluations by the program directors and references from previous positions. The introduction of well-validated objective methods of assessment has allowed us to evaluate candidates’ technical skills and base the selection process on objective, reliable, and transparent criteria. The aim of the study was to assess the applicability of such methods in current practice.

Materials and methods

Prospective study. Eight surgeons, applying for a fellowship position in minimally invasive surgery (MIS), performed a previously validated assessment curriculum using a Virtual-Reality Laparoscopic Trainer (LapSim^® 3.0, Surgical Science, Gothenburgh, Sweden). Technical performance was evaluated using criteria registered by the simulator, i.e., time, error score, and efficiency of movements score. Candidates performed all the tasks in easy end medium level until reaching predefined criteria. If proficiency criteria were not achieved on easy or medium level after nine repetitions the test was considered as failed. Additionally, all applicants underwent an interview by two independent attending surgeons. Each applicant received a grade on a ten-point scale.

Results

Five out of the eight candidates failed the technical skills assessment test. One candidate failed to achieve proficiency criteria on easy level, one on medium, and three on difficult level. Evaluation scores, based on the interview of the candidates showed a good interrater reliability (Cronbach’s α = 0.8). There was no significant correlation between the interviewers rating, and the applicants technical skills demonstrated during the test on the VR trainer (Spearman’s ρ = 0.182, p = 0.696).

Conclusions

Evaluations by senior surgeons are reproducible and reliable. The introduction of technical skills assessment has the potential to improve the current method of candidate selection, making it more valid, objective, and transparent.