Evaluation of laparoscopic skills: a 2-year follow-up during residency training.

OBJECTIVE: To evaluate laparoscopic technical skill in surgical residents over a 2-year period. DESIGN: The laparoscopic technical skills of general surgical residents were evaluated using the MISTELS program. This provides an objective evaluation of laparoscopic skill, taking into account precision and speed. SETTING: Inanimate laparoscopic skills centre. PARTICIPANTS: Ten general surgical residents (5 PGY1, 3 PGY2 and 2 PGY3 residents) who were required to complete 3 structured laparoscopic tasks. OUTCOME MEASURES: A composite score incorporating precision and timing was assigned to each task. The paired t-test was used to compare performance of each resident at the 2 levels of their residency training for each task. Linear regression analysis was used to correlated level of training and total score (sum of all tasks). RESULTS: Linear regression analysis demonstrated a highly significant correlation between level of training and total score (r = 0.82, p < 0.01). There was a significant increase in scores in the cutting and suturing task over the 2-year period (p < 0.01). Transferring skills did not improve significantly (p = 0.11). CONCLUSIONS: Performance in the simulator improved over residency training and was correlated highly with postgraduate year. This simulator model is a valuable teaching tool for training and evaluation of basic laparoscopic tasks in laparoscopic surgery.     

Proving the value of simulation in laparoscopic surgery

OBJECTIVE: To assess the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) physical laparoscopic simulator for construct and predictive validity and for its educational utility. SUMMARY BACKGROUND DATA: MISTELS is the physical simulator incorporated by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in their Fundamentals of Laparoscopic Surgery (FLS) program. MISTELS' metrics have been shown to have high interrater and test-retest reliability and to correlate with skill in animal surgery. METHODS: Over 200 surgeons and trainees from 5 countries were assessed using MISTELS in a series of experiments to assess the validity of the system and to evaluate whether practicing MISTELS basic skills (transferring) would result in skill acquisition transferable to complex laparoscopic tasks (suturing). RESULTS: Face validity was confirmed through questioning 44 experienced laparoscopic surgeons using global rating scales. MISTELS scores increased progressively with increasing laparoscopic experience (n = 215, P < 0.0001), and residents followed over time improved their scores (n = 24, P < 0.0001), evidence of construct validity. Results in the host institution did not differ from 5 beta sites (n = 215, external validity). MISTELS scores correlated with a highly reliable validated intraoperative rating of technical skill during laparoscopic cholecystectomy (n = 19, r = 0.81, P < 0.0004; concurrent validity). Novice laparoscopists were randomized to practice/no practice of the transfer drill for 4 weeks. Improvement in intracorporeal suturing skill was significantly related to practice but not to baseline ability, career goals, or gender (P < 0.001). CONCLUSION: MISTELS is a practical and inexpensive inanimate system developed to teach and measure technical skills in laparoscopy. This system is reliable, valid, and a useful educational tool.     

Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents

BACKGROUND: Technical skills of residents have traditionally been evaluated using subjective In-Training Evaluation Reports (ITERs). We have developed the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS), an objective measure of laparoscopic technical ability. The purpose of the study was to assess the concurrent validity of the MISTELS by exploring the relationship between MISTELS score and ITER assessment. STUDY DESIGN: Fifty surgery residents were assessed on the MISTELS system. Concurrent ITER assessments of technical skill were collected, and the proportion of superior ratings for the year was calculated. Statistical comparisons were performed by ANOVA and chi-square analysis. The Pearson correlation coefficient was used to compare the scores in the MISTELS with the ITER ratings. RESULTS: The 50 residents received 277 ITERs for the year, of which 103 (37%) were "superior," 170 (61%) "satisfactory," 4 (1%) "borderline," and 0 "unsatisfactory." The MISTELS score correlated moderately well with the proportion of superior ITER scores (r = 0.51, p < 0.01). Residents who passed the MISTELS had a higher proportion of superior ITER assessments than those who failed the MISTELS (p = 0.02), but residents who performed below their expected level on the MISTELS still received mainly satisfactory ITERs (82 +/- 18%). CONCLUSIONS: The ITER assessment is poor at identifying residents with below-average technical skills. Residents who perform well in the MISTELS laparoscopic simulator also have better ITER evaluations, providing evidence for the concurrent validity of the MISTELS. Multiple assessment instruments are recommended for assessment of technical competency.     

Evaluating laparoscopic skills

BACKGROUND: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) was developed to assess laparoscopic skills and to score them objectively. This system has been described previously. The purpose of the current study was to determine a pass/fail threshold. METHODS: In this study, 165 individuals were tested and grouped according to their clinical competency in laparoscopic surgery. The noncompetent group consisted of medical students and surgical residents in their first 2 years of training (n = 83). The competent group consisted of chief general surgical residents in their last year of training, laparoscopy fellows, and practicing laparoscopic surgeons (n = 82). The Mann-Whitney U test was used to evaluate differences in task performance between the two groups. RESULTS: There was a significant difference in total scores and individual MISTELS task scores between the noncompetent and competent laparoscopic surgeons (189 vs 372.5; p <0.0001). By setting specific pass/fail total score thresholds (cutoff scores), competent surgeons can be discriminated from noncompetent surgeons. CONCLUSION: An objective pass/fail evaluation can be given to individuals tested with the MISTELS system.     

Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program

The purpose of this article is to describe our experience with the incorporation of a proficiency-based laparoscopic skills curriculum in a busy surgical training program that aims to improve the technical proficiency of residents. The curriculum has a cognitive component and a manual skills component and is adjusted to resident training level. It is based on the Fundamentals of Laparoscopic Surgery program and includes basic laparoscopic virtual-reality tasks of the Lap Mentor simulator (Simbionix USA Corp., Cleveland, Ohio). Training occurs in weekly 1-hour sessions until expert-derived performance goals are achieved. Maintenance training ensures skill retention. Performance is assessed with objective metrics and is supported with feedback and an award system. Resident workload is assessed at regular intervals. Knowledge tests and manual skills tests are administered at the beginning and end of the academic year to assess resident performance improvement and curriculum effectiveness. Resident attendance rates and training progress are monitored continuously, and training sessions are adjusted to individual needs. Our curriculum has been implemented for several months. Our experience so far suggests that it is imperative to have dedicated supervising personnel and dedicated training time in the busy week of the surgical resident to ensure attendance. Our next step is to incorporate the 20 modules of the new Association of Program Directors in Surgery (ADPS)/American College of Surgeons (ACS) national skills curriculum into our skills training program, to expand its cognitive component by incorporating additional procedural videos, and to adapt scenario-based training on trauma and critical care on human patient simulators.     

Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training

BACKGROUND: The fundamentals of laparoscopic surgery (FLS) program has been extensively validated for use as a high-stakes examination for certification purposes, but optimal methods for its use in skills training have not been described. This study aimed to investigate the feasibility of implementing a proficiency-based FLS skills training curriculum and to evaluate its effectiveness in preparing trainees for certification. METHODS: For this study, 21 novice medical students at two institutions viewed video tutorials, then performed one repetition of the five FLS tasks as a pretest. The pretests were scored using standard testing metrics. The trainees next practiced the tasks over a 2-month period until they achieved proficiency for all the tasks. A modified on-the-fly scoring system based on expert-derived performance was used. The trainees were posttested using the high-stakes examination format. RESULTS: No trainee passed the certification examination at pretesting. The trainees achieved proficiency for 96% of the five tasks during training, which required 9.7 +/- 2.4 h (range, 6-14 h) and 119 +/- 31 repetitions (range, 66-161 repetitions). The trainees rated the proficiency levels as "moderately difficult" (3.0 +/- 0.7 on a 5-point scale) and "highly appropriate" (4.7 +/- 0.1 on a 5-point scale). At posttesting, 100% of the trainees passed the certification examination and demonstrated significant improvement compared with pretesting for normalized score (468 +/- 24 vs 126 +/- 75; p < 0.001), self-rated laparoscopic comfort (89.4% vs 4.8%; p < 0.001), and skill level (3.6 +/- 0.9 vs 1.2 +/- 0.5; p < 0.001, 5-point scale). CONCLUSIONS: This proficiency-based curriculum is feasible for training novices and uniformly allows sufficient skill acquisition for FLS certification. Endorsed by the Society of American Gastrointestinal Endoscopic Surgeons (SAGES), this curriculum is available for use as an optimal method for FLS skills training. More widespread adoption of this curriculum is encouraged.     

Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying

BACKGROUND: Advanced surgical skills such as laparoscopic suturing are difficult to learn in an operating room environment. The use of simulation within a defined skills-training curriculum is attractive for instructor, trainee, and patient. This study examined the impact of a curriculum-based approach to laparoscopic suturing and knot tying. STUDY DESIGN: Senior surgery residents in a university-based general surgery residency program were prospectively enrolled and randomized to receive either a simulation-based laparoscopic suturing curriculum (TR group, n=11) or standard clinical training (NR group, n=11). During a laparoscopic Nissen fundoplication, placement of two consecutive intracorporeally knotted sutures was video recorded for analysis. Operative performance was assessed by two reviewers blinded to subject training status using a validated, error-based system to an interrater agreement of >or=80%. Performance measures assessed were time, errors, and needle manipulations, and comparisons between groups were made using an unpaired t-test. RESULTS: Compared with NR subjects, TR subjects performed significantly faster (total time, 526+/-189 seconds versus 790+/-171 seconds; p < 0.004), made significantly fewer errors (total errors, 25.6+/-9.3 versus 37.1+/-10.2; p < 0.01), and had 35% fewer excess needle manipulations (18.5+/-10.5 versus 27.3+/-8.6; p < 0.05). CONCLUSIONS: Subjects who receive simulation-based training demonstrate superior intraoperative performance of a highly complex surgical skill. Integration of such skills training should become standard in a surgical residency's skills curriculum.     

An objective scoring system for laparoscopic cholecystectomy

Background: Direct observation with structured criteria for performance is the most reliable and valid method of assessing technical skill during operative procedures. We developed such a system to evaluate technical performance during a laparoscopic cholecystectomy. The reliability and validity of the system were tested by analyzing the correlation among three observers in a multicenter study and comparing performance with years of surgical experience.

Study Design: Thirty consecutive cases of laparoscopic cholecystectomy were recorded on videotape, 10 from each of 3 institutions. Independent scores were generated by three observers examining each of the videotapes, providing a total of 90 scores. Points were awarded for successful completion of each of 23 different steps required to perform a laparoscopic cholecystectomy. Error points were tabulated based on the frequency and relative severity of each of 21 potential technical mistakes during the operation. The final score was assumed to be a relative measure of technical skill and was derived by subtracting error points from points awarded for completion of each step of the procedure. Pearson correlation coefficients were used to assess agreement among examiners and correlation with year of surgical experience.

Results: Agreement in final scores among the three observers was excellent (r = 0.74–0.96) despite the fact that one observer assigned significantly fewer error points. Correlation between year of experience and two-handed technique scoring was good (r = 0.5, p = 0.057), but the correlation between experience and one-handed technique scores was poor (r = 0.02).

Conclusions: The technical skills required to perform laparoscopic cholecystectomy can reliably be measured using this tool. This method can be used to track the learning curve of surgeons in training, evaluate the efficacy of alternative training tools, and provide a means of self-assessment for the trainee.     

The Steinberg-Bernstein Centre for Minimally Invasive Surgery at McGill University

Surgical skills and simulation centers have been developed in recent years to meet the educational needs of practicing surgeons, residents, and students. The rapid pace of innovation in surgical procedures and technology, as well as the overarching desire to enhance patient safety, have driven the development of simulation technology and new paradigms for surgical education. McGill University has implemented an innovative approach to surgical education in the field of minimally invasive surgery. The goal is to measure surgical performance in the operating room using practical, reliable, and valid metrics, which allow the educational needs of the learner to be established and enable feedback and performance to be tracked over time. The GOALS system and the MISTELS program have been developed to measure operative performance and minimally invasive surgical technical skills in the inanimate skills lab, respectively. The MISTELS laparoscopic simulation-training program has been incorporated as the manual skills education and evaluation component of the Fundamentals of Laparoscopic Surgery program distributed by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the American College of Surgeons.     

Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery

OBJECTIVE: The aim of this study was to compare learning curves for laparoscopic cholecystectomy (LC) after training on a proficiency based virtual reality (VR) curriculum with that of a traditionally trained group. SUMMARY BACKGROUND DATA: Simulator-based training has been shown to improve technical performance during real laparoscopic procedures, although research to date has not proven the persistence of this effect over subsequent cases. MATERIAL AND METHODS: Twenty novice surgeons underwent baseline laparoscopic skills testing followed by a 1-day didactic training session. Control subjects (n = 10) performed 5 cadaveric porcine LCs each; VR-trained subjects (n = 10) completed a VR training curriculum followed by 3 porcine LCs each. A further 10 experienced laparoscopic surgeons (>100 LCs) performed 2 porcine LCs each to define benchmark levels. Technical skill assessment was by motion analysis and video-based global rating scores (out of 35). RESULTS: There were no intergroup differences in baseline skill. The first LC revealed significant differences between control and VR groups for time (median 4590 seconds vs. 2165 seconds, P = 0.038), path length (169.2 meters vs. 86.8 meters, P = 0.009), number of movements (2446 vs. 1029, P = 0.009), and video scores (17 vs. 25, P = 0.001). The VR group, although not a control, achieved video and dexterity scores equivalent to expert levels of performance. CONCLUSIONS: A proficiency based VR training curriculum shortens the learning curve on real laparoscopic procedures when compared with traditional training methods. This may be a more cost- and time-effective approach, and supports the need for simulator-based practice to be integrated into surgical training programs.     

Laparoscopic training on bench models: better and more cost effective than operating room experience?

BACKGROUND: Developing technical skill is essential to surgical training, but using the operating room for basic skill acquisition may be inefficient and expensive, especially for laparoscopic operations. This study determines if laparoscopic skills training using simulated tasks on a video-trainer improves the operative performance of surgery residents. STUDY DESIGN: Second- and third-year residents (n= 27) were prospectively randomized to receive formal laparoscopic skills training or to a control group. At baseline, residents had a validated global assessment of their ability to perform a laparoscopic cholecystectomy based on direct observation by three evaluators who were blinded to the residents' randomization status. Residents were also tested on five standardized video-trainer tasks. The training group practiced the video-trainer tasks as a group for 30 minutes daily for 10 days. The control group received no formal training. All residents repeated the video-trainer test and underwent a second global assessment by the same three blinded evaluators at the end of the 1-month rotation. Within-person improvement was determined; improvement was adjusted for differences in baseline performance. RESULTS: Five residents were unable to participate because of scheduling problems; 9 residents in the training group and 13 residents in the control group completed the study. Baseline laparoscopic experience, video-trainer scores, and global assessments were not significantly different between the two groups. The training group on average practiced the video-trainer tasks 138 times (range 94 to 171 times); the control group did not practice any task. The trained group achieved significantly greater adjusted improvement in video-trainer scores (five of five tasks) and global assessments (four of eight criteria) over the course of the four-week curriculum, compared with controls. CONCLUSIONS: Intense training improves video-eye-hand skills and translates into improved operative performance for junior surgery residents. Surgical curricula should contain laparoscopic skills training.     

Beyond task time: automated measurement augments fundamentals of laparoscopic skills methodology

Background

Laparoscopic psychomotor skills are challenging to learn and objectively evaluate. The Fundamentals of Laparoscopic Skills (FLS) program provides a popular, inexpensive, widely-studied, and reported method for evaluating basic laparoscopic skills. With an emphasis on training safety before efficiency, we present data that explore the metrics in the FLS curriculum.

Materials and methods

A multi-institutional (n = 3) cross-sectional study enrolled subjects (n = 98) of all laparoscopic skill levels to perform FLS tasks in an instrumented box trainer. Recorded task videos were postevaluated by faculty reviewers (n = 2) blinded to subject identity using a modified Objective Structured Assessment of Technical Skills (OSATS) protocol. FLS scores were computed for each completed task and compared with demographically established skill levels (training level and number of procedures), video review scoring, and objective performance metrics including path length, economy of motion, and peak grasping force.

Results

Three criteria used to determine expert skill, training and experience level, blinded review of performance by faculty via OSATS, and FLS scores, disagree in establishing concurrent validity for determining “true experts” in FLS tasks. FLS-scoring exhibited near-perfect correlation with task time for all three tasks (Pearson r = 0.99, 1.00, 1.00 with P <0.00000001). FLS error penalties had negligible effect on FLS scores. Peak grasping force did not correlate with task time or FLS scores.

Conclusions

FLS technical skills scores presented negligible benefit beyond the measurement of task time. FLS scoring is weighted more toward speed than precision and may not significantly address poor tissue handling skills, especially regarding excessive grasping force. Categories of experience or training level may not form a suitable basis for establishing proficiency thresholds or for construct validity studies for technical skills.     

Construct validity of the ProMIS laparoscopic simulator

Background  Laparoscopic surgery challenges both the surgical novice and experienced open surgeon with unique psychomotor adaptations. Surgical skills assessment has historically relied on subjective opinion and case experience. Objective performance metrics have stimulated much interest in surgical education over the last decade and proficiency-based simulation has been proposed as a paradigm shift in surgical skills training. New assessment tools must be subjected to scientific validation. This study examined the construct validity of a hybrid laparoscopic simulator with in-built motion tracking technology. Methods  Volunteers were recruited from four experience groups (consultant surgeon, senior trainee, junior trainee, medical student). All subjects completed questionnaires and three tasks on the ProMIS laparoscopic simulator (laparoscope orientation, object positioning, sharp dissection). Motion analysis data was obtained via optical tracking of instrument movements. Objective metrics included time, path length (economy of movement), smoothness (controlled handling) and observer-recorded penalty scores. Results  One hundred and sixty subjects completed at least one of the three tasks. Significant group differences were confirmed for number of years qualified, age and case experience. Significant differences were found between experts and novices in all three tasks. Sharp dissection was the strongest discriminator of four recognised laparoscopic skill groups: consultants outperformed students and juniors in all three performance metrics and objective penalty score (p < 0.05), and only accuracy of dissection did not distinguish them from senior trainees (p = 0.261). Seniors dissected faster, more efficiently and more accurately than juniors and students (p < 0.05). Conclusions  ProMIS provides a construct valid laparoscopic simulator and is a feasible tool to assess skills in a cross-section of surgical experience groups. ProMIS has the potential to objectively measure pre-theatre dexterity practice until an agreed proficiency level of dexterity is achieved. Future work should now examine whether training to expert criterion levels on ProMIS correlates with actual operative performance.     

Reverse-alignment surgical skills assessment

Background During the course of certain laparoscopic procedures, a surgeon may encounter circumstances where the camera and working instruments are not aligned along the same axis. When the alignment is off by 180°, the laparoscope is pointed directly at the surgeon and a mirror image of the operative field is created. Under reverse alignment conditions, surgeons often become disoriented and task performance may be degraded. The aim of this study is to measure the degree to which reverse alignment conditions impair the performance of operators of varying laparoscopic surgical skill and experience. Methods Nineteen general surgery residents and 3 attending surgeons were tested in a video trainer. A task designed to simulate tack placement during laparoscopic ventral hernia repair was created. Time required to successfully complete this task in both forward and reverse alignment conditions was measured for all participants. For this study, the impact of visual-spatial discordances of angles other than 180° on task performance was not assessed. Data regarding prior laparoscopic case volume and MISTELS scores were collected for each resident. Results Significant and strong correlations were observed between resident experience and individual MISTELS scores. No significant correlation was found between reverse image times and any other surrogate measure of experience or skill for the residents. All study participants required a significantly longer period of time to complete the simulated tack task in reverse alignment compared to forward alignment conditions. Attendings required 65% more time and resident times increased by 470%. Attendings completed the task significantly faster than residents in reverse alignment conditions. Conclusions Reverse-alignment surgical skills can be improved with experience, but may require a volume of cases exceeding that encountered in residency training. Reverse image skills are not derivative from surgical skills developed in a video trainer with forward orientation.     

Skill retention following proficiency-based laparoscopic simulator training

BACKGROUND: Proficiency-based curricula using both virtual reality (VR) and videotrainer (VT) simulators have proven to be efficient and maximally effective, but little is known about the retention of acquired skills. The purpose of this study was to assess skill retention after completion of a validated laparoscopic skills curriculum. METHODS: Surgery residents (n=14) with no previous VR or VT experience were enrolled in an Institutional Review Board-approved protocol and sequentially practiced 12 Minimally Invasive Surgical Trainer-VR and 5 VT tasks until proficiency levels were achieved. One VR (manipulate diathermy) and 1 VT (bean drop) tasks were selected for assessment at baseline, after training completion (posttest), and at retention. RESULTS: All residents completed the curriculum. Posttest assessment occurred at 13.2 +/- 11.8 days and retention assessment at 7.0 +/- 4.0 months. After an early performance decrement at posttest (17%-45%), the acquired skill was maintained up to the end of the follow-up period. For VR, scores were 81.5 +/- 23.5 at baseline, 33.3 +/- 1.8 at proficiency, 48.4 +/- 9.2 at posttest, and 48.4 +/- 11.8 at retention. For VT, scores were 49.4 +/- 12.5 at baseline, 22.0 +/- 1.4 at proficiency, 25.6 +/- 3.6 at posttest, and 26.4 +/- 4.2 at retention. Skill retention was better for VT, compared with VR (P < .02). The extent of skill deterioration did not correlate with training duration or resident level. CONCLUSIONS: Although residents do not retain all acquired skills (more so for VR than for VT) according to simulator assessment, proficiency-based training on simulators results in durable skills. Additional studies are warranted to further optimize curriculum design, investigate simulator differences, and establish training methods that improve skill retention.     

Simulation Training in Laparoscopy Using a Computerized Physical Reality Simulator

Objective:

To describe a new simulator, SurgicalSIM LTS, and summarize our preliminary experience with system.

Methods:

LTS was evaluated in 3 studies: (1) 124 participants from 3 Canadian universities: 13 students; 30 residents, fellows, attendings from surgery; 59 gynecologists; 22 urologists were classified based on laparoscopic experience as novice, intermediate, competent, or expert. All were tested on the LTS. Seventy-four were tested on the LTS and MISTELS (McGill Inanimate System for Training and Evaluation of Laparoscopic Skills). Participants completed a satisfaction questionnaire. (2) Twenty-five international gynecologists in-training at Kiel Gynaecologic Endoscopy Center, and 15 students from the center pretested on LTS underwent voluntary additional trials and posttesting. (3) Seventeen experienced laparoscopic surgeons from 3 specialties were recruited to perform on randomly assigned simulators involving 5 commercial, computer-based systems. The surgeons practiced repetitively for 1.5 days. Efficient, error-free performance was measured and proficiency score formulas were developed.

Results:

Study A: LTS showed a good correlation with level of experience (P=0.000) and MISTELS (0.79). Satisfaction: LTS vs MISTELS 79.9 vs 70.4 (P=0.012). Study B: Posttest scores were significantly better in all tasks for both groups, P<0.0001. Group mean scores with ≤5 trials were significantly better than with 2 or 3 trials (P<0.012, P<0.018). Study C: LTS had the highest effectiveness rating of the 5 simulators.

Conclusions:

A new computerized physical reality simulator can be used to assess/train laparoscopic technical skills.     

Measuring laparoscopic operative skill in a video trainer

Background Laparoscopic surgery requires a unique set of technical skills. More experienced laparoscopic surgeons perform certain tasks more efficiently in a video trainer than less experienced laparoscopic surgeons. The presumption is that the experienced surgeon possesses more of the skill required to complete the task. This study sought to determine the degree to which previous laparoscopic operative experience influenced the performance of selected video trainer tasks of varying complexity. Methods In this study, 19 general surgery residents with varying levels of laparoscopic operative case experience (as defined by operative case logs) were timed performing five tasks in a video trainer. The tasks were rope pass, peg drop, peg exchange, needle pass, and knot tie. All the residents watched a video demonstration of each skill before testing. None of the residents had previous exposure to video trainers, and no practice was allowed before testing. A composite score for all tasks was calculated for each resident as a measure of overall performance. Results There was a strong correlation between operative experience and time required for successful completion of each task, with the exception of the rope pass. The magnitude of correlation increased with tasks of increasing complexity. Composite scores were correlated with operative experience. Significant interval improvements in performance were observed for increasing experience up to a level of approximately 100 previous laparoscopic cases. Conclusions Overall composite scores and time required for the completion of each individual video trainer task (with the exception of the rope pass) may be an accurate reflection of laparoscopic surgical skill acquired in the operating room. A resident may need as many as 100 laparoscopic cases for full development of a basic skill set in the operating room. A more efficient and safe method of training, such as a validated skills curriculum conducted in a dry lab, is a desirable alternative to developing skill exclusively in the operating room.     

A theory-based curriculum for enhancing surgical skillfulness

BACKGROUND: Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program. STUDY DESIGN: Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years. RESULTS: Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining. CONCLUSIONS: These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.     

Technical and technological skills assessment in laparoscopic surgery.

OBJECTIVES: Surgical appraisal and revalidation are key components of good surgical practice and training. Assessing technical skills in a structured manner is still not widely used. Laparoscopic surgery also requires the surgeon to be competent in technological aspects of the operation. METHODS: Checklists for generic, specific technical, and technological skills for laparoscopic cholecystectomies were constructed. Two surgeons with >12 years postgraduate surgical experience assessed each operation blindly and independently on DVD. The technological skills were assessed in the operating room. RESULTS: One hundred operations were analyzed. Eight trainees and 10 consultant surgeons were recruited. No adverse events occurred due to technical or technological skills. Mean interrater reliability was kappa=0.88, P=<0.05. Construct validity for both technical and technological skills between trainee and consultant surgeons were significant, Mann-Whitney P=<0.05. CONCLUSIONS: Our study demonstrates that technical and technological skills can be measured to assess performance of laparoscopic surgeons. This technical and technological assessment tool for laparoscopic surgery seems to have face, content, concurrent, and construct validities and could be modified and applied to any laparoscopic operation. The tool has the possibility of being used in surgical training and appraisal. We aim to modify and apply this tool to advanced laparoscopic operations.     

Validity of the MISTELS simulator for laparoscopy training in urology

BACKGROUND AND PURPOSE: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) consists of a series of five laparoscopic exercises performed in an endotrainer box. MISTELS has been validated for use in both training and evaluation of general surgery residents in fundamental laparoscopic skills. The purpose of this study was to demonstrate the construct validity of MISTELS for urology residents. SUBJECTS AND METHODS: Seventeen participants were evaluated during performance of the five MISTELS tasks (peg transfer, pattern cutting, ligating loop, and suturing with extracorporeal and intracorporeal knots) using the standardized scoring system, which rewards both speed and precision. Participants included 13 urology residents (PGY 1-5), 1 fellow, and 3 urologists experienced in laparoscopy. Results are expressed as median (range). The Mann-Whitney U-test was used to compare MISTELS scores for 9 novice (PGY 1-4) and 8 experienced urologists (PGY 5-attending). P < 0.05 was considered statistically significant. RESULTS: The median MISTELS total normalized score for novices was 52.3 (range 15-68.9) compared with 71.7 (range 56.3-82.9) for experienced urologists (P = 0.007). Although the experienced group achieved higher scores in all five individual tasks, statistically significant differences were demonstrated for the peg transfer and intracorporeal suture tasks only. CONCLUSION: These data provide evidence for construct validity of the MISTELS system for urology residents.