Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: Validity and reliability

OBJECTIVE: Resident surgical skills are acquired mainly through observing and later performing procedures in the operating room. Evaluation of surgical skills has traditionally been done through subjective faculty evaluation, a technique that has poor reliability and unknown validity. Our goal was to develop specific surgical tasks, both laparoscopic and open abdominal, that could be objectively and reliably evaluated in a bench laboratory setting. STUDY DESIGN: The prospective development of a reliable and valid resident surgical skills test in a bench laboratory setting was our goal. A written test of surgical knowledge and 12 skills tests were administered to 36 residents. Laparoscopic bench tasks were simulated with the use of a box and camera with a video display. Six laparoscopic tasks were assessed, including placing pegs on a board, running the bowel simulation, and other tasks that involve hand-eye coordination and manual dexterity. Open abdominal skills simulated incision closure, suturing a vaginal cuff, knot tying, and using a tie on a passer. Residents were timed at each given station and were given a rating score by 2 examiners. RESULTS: Knowledge scores showed a significant improvement by residency level. Assessment of construct validity (the ability to discriminate among residency levels) demonstrated significant differences on the rating of overall performance and individual tasks by level (determined by 1-way analysis of variance). Interrater reliability (agreement between 2 raters) with the use of intraclass correlation was 0.79 for the total score. The cost to administer the bench laboratory test was less than $50 and required 30 hours of faculty time. CONCLUSION: The results of this study suggest that surgical bench laboratory tasks can assess residents' surgical skills with good reliability and validity on most tasks. Our previous study, which used an animal laboratory, was expensive, and the bench laboratory model may provide an alternative means to assess surgical skills.