Assessment of technical surgical skills.

OBJECTIVE: To review methods of assessment of technical surgical competence, and to point out the potential benefit of such assessment to training and safe practice. DESIGN: Review of publications about methods of evaluation of technical surgical competence. Studies were identified through MEDLINE using the keywords surgery, assessment, operative skills, training, and competence. The references cited in these studies were reviewed to find out whether any other trials fitted the selection criteria. SETTING: Teaching hospital, Denmark. RESULTS: There are several validated methods that allow objective assessment of operative skills for both open and laparoscopic surgery. CONCLUSIONS: Objective assessment of technical surgical skills is possible and reliable and valid methods are available. There are good reasons to implement evaluation programmes as a part of surgical education.     

What do master surgeons think of surgical competence and revalidation?

BACKGROUND: There has been on-going debate and public interest in surgical competence in recent years. METHODS: A Delphi reiterative opinion survey was conducted among master surgeons on selection of surgical trainees, methods of assessment of progress of surgical trainees, and revalidation of established consultant surgeons. RESULTS: Selection-the current methods of trainee selection were considered inadequate and in need of revision. The important attributes recognized by group are cognitive factors, innate dexterity, and personality. Important aspects of personality include decision-making ability, insight, team spirit, and emotional stability. Assessment during training-the majority view was that this should be based on clinical judgement/skills, operative skills, and cognitive ability. Assessment of technical ability should be based on standardized checklists. Research within training programs was encouraged but academic achievement does not reflect surgical competence. There was a majority verdict for an exit clinical examination. Revalidation-the group agreed on the need for competence checks during the professional career of surgeons. These should cover knowledge, clinical, operative, and humanistic skills; but expressed concern on the feasibility of a revalidation system that can reliably assess the range of skills needed for surgical competence. There was a majority vote against an internal appraisal system. External assessment by nationally appointed 'assessors' was considered preferable. CONCLUSIONS: Both selection and assessment of surgical trainees require changes and standardization. Although revalidation is necessary, concern was expressed on the reliability and validity of existing and proposed systems.     

Towards an appropriate training for vascular surgeons

The increasing demand for fully trained vascular surgeons in the UK has highlighted the need for urgent review of that training. Such a review must examine the number of trainees required and the number and role of teaching centres. Vascular surgical training should only take place where an integrated approach, including experience in interventional radiology and intensive care, is available. The relationship of vascular to general surgery must now be defined with training needs in mind if we are to provide an appropriate service within the next decade.     

Open vascular surgery as a starting point for endovascular surgery

What distinguishes vascular surgeons from other specialists who treat patients with vascular disease is their ability to combine skills in both open and endovascular treatments. Open vascular surgery should be considered the "starting point" for endovascular surgery, since training and practice in vascular surgery require extensive knowledge of the basic science and a thorough education in general surgical techniques. In addition, surgeons must possess detailed specialized knowledge of the anatomy and physiology of arteries, veins and lymphatics and of the pathological processes which may affect them. This scientific and technical background is also imperative for endovascular surgery. Open vascular surgery can also be considered as a potential finishing point of endovascular surgery. In fact, open surgery is still often the only solution for complex cases considered unsuitable for an endovascular approach, or for different types of complications following endovascular treatments. As endovascular surgery is increasingly considered as the initial treatment option for many patients with vascular disease, it is crucial that vascular surgery training programs develop methods to maintain the open surgical skills of their trainees. The only way for vascular surgeons to remain the premier specialists to care for patients with vascular disease is for them to combine skills in both open and endovascular treatments.     

Open Surgical Simulation—A Review

BackgroundSurgical simulation has benefited from a surge in interest over the last decade as a result of the increasing need for a change in the traditional apprentice model of teaching surgery. However, despite the recent interest in surgical simulation as an adjunct to surgical training, most of the literature focuses on laparoscopic, endovascular, and endoscopic surgical simulation with very few studies scrutinizing open surgical simulation and its benefit to surgical trainees. The aim of this review is to summarize the current standard of available open surgical simulators and to review the literature on the benefits of open surgical simulation.Current State of Open Surgical SimulationOpen surgical simulators currently used include live animals, cadavers, bench models, virtual reality, and software-based computer simulators. In the current literature, there are 18 different studies (including 6 randomized controlled trials and 12 cohort studies) investigating the efficacy of open surgical simulation using live animal, bench, and cadaveric models in many surgical specialties including general, cardiac, trauma, vascular, urologic, and gynecologic surgery. The current open surgical simulation studies show, in general, a significant benefit of open surgical simulation in developing the surgical skills of surgical trainees. However, these studies have their limitations including a low number of participants, variable assessment standards, and a focus on short-term results often with no follow-up assessment.Future of Open Surgical SimulationThe skills needed for open surgical procedures are the essential basis that a surgical trainee needs to grasp before attempting more technical procedures such as laparoscopic procedures. In this current climate of medical practice with reduced hours of surgical exposure for trainees and where the patient’s safety and outcome is key, open surgical simulation is a promising adjunct to modern surgical training, filling the void between surgeons being trained in a technique and a surgeon achieving fluency in that open surgical procedure. Better quality research is needed into the benefits of open surgical simulation, and this would hopefully stimulate further development of simulators with more accurate and objective assessment tools.     

Technical and technological skills assessment in laparoscopic surgery.

OBJECTIVES: Surgical appraisal and revalidation are key components of good surgical practice and training. Assessing technical skills in a structured manner is still not widely used. Laparoscopic surgery also requires the surgeon to be competent in technological aspects of the operation. METHODS: Checklists for generic, specific technical, and technological skills for laparoscopic cholecystectomies were constructed. Two surgeons with >12 years postgraduate surgical experience assessed each operation blindly and independently on DVD. The technological skills were assessed in the operating room. RESULTS: One hundred operations were analyzed. Eight trainees and 10 consultant surgeons were recruited. No adverse events occurred due to technical or technological skills. Mean interrater reliability was kappa=0.88, P=<0.05. Construct validity for both technical and technological skills between trainee and consultant surgeons were significant, Mann-Whitney P=<0.05. CONCLUSIONS: Our study demonstrates that technical and technological skills can be measured to assess performance of laparoscopic surgeons. This technical and technological assessment tool for laparoscopic surgery seems to have face, content, concurrent, and construct validities and could be modified and applied to any laparoscopic operation. The tool has the possibility of being used in surgical training and appraisal. We aim to modify and apply this tool to advanced laparoscopic operations.     

Technical competence in surgeons

In this paper, we review the literature to date on technical competence in surgeons; how it can be defined, taught to trainees and assessed. We also examine how we can predict which candidates for surgical training will most likely develop technical competence. While technical competency is just one aspect of what makes a good surgeon, we have recognized a need to review the literature in this area and to combine this with broader definitions of competency. Our review found that several methods are available to objectively measure, assess and predict technical competence and should be used in surgical training.     

SE01THE EMERGING ROLE OF SIMULATION IN SURGICAL TRAINING

Traditionally, surgical trainees have acquired their technical skills whilst working in the operating theatre alongside more senior surgeons in an apprenticeship model. Recently, changes in surgical practice have challenged this traditional approach, including reduced working hours, decreased availability of operating theatre time, increased cost of operating theatre time and increasing complexity of surgical procedures. Most importantly, public opinion is increasingly resistant to having patients used as teaching material. Moving forward in the 21st century, we need to develop a new model of technical skills acquisition. Surgical simulation offers a means of practicing surgical skills in a laboratory environment with no risk to patients. Simulators range from simple bench models, which are relatively inexpensive, to high tech virtual reality simulators. The Royal College of Surgeons in Ireland has developed a syllabus for operative surgery which is based on simulation and all trainees attend the National Surgical Training Centre on a regular recurrent basis, six times each year, for concentrated skills training. Simulation allows the standardisation of teaching technical skills and, most importantly, standardisation of assessment of technical skills. Furthermore, simulation offers trainees “permission to make mistakes”– a valuable learning exercise. Our trainees are assessed in the laboratory setting and must demonstrate proficiency before moving to the next stage of training (proficiency based progression). Simulation has played a key role in this process.     

What does leadership in surgery entail?

Background: Leadership is not formally taught at any level in surgical training; there are no mandatory leadership courses or qualifications for trainees or specialists, and leadership performance is rarely evaluated within surgical appraisal or assessment programmes. Methods: Literature obtained from a MEDLINE search was reviewed to determine the characteristics of surgical leaders; outline an analytical framework through which these characteristics can be developed both in surgeons and surgical departments; and reflect on future challenges and recommendations for the central role of leadership in the field of surgery. Results: Leadership in surgery entails professionalism, technical competence, motivation, innovation, teamwork, communication skills, decision‐making, business acumen, emotional competence, resilience and effective teaching. Leadership skills can be developed through experience, observation, and education using a framework including mentoring, coaching, networking, stretch assignments, action learning and feedback. Conclusion: Modern surgery will need leaders with superior leadership skills that are well defined. It is vital that leadership programmes to develop leadership skills are put into practice in medical education curriculum and postgraduate surgical training. This will ensure maintenance and improvement in the quality of patient care.     

A Review of the Available Urology Skills Training Curricula and Their Validation

BackgroundThe transforming field of urological surgery continues to demand development of novel training devices and curricula for its trainees. Contemporary trainees have to balance workplace demands while overcoming the cognitive barriers of acquiring skills in rapidly multiplying and advancing surgical techniques. This article provides a brief review of the process involved in developing a surgical curriculum and the current status of real and simulation-based curricula in the 4 subgroups of urological surgical practice: open, laparoscopic, endoscopic, and robotic.MethodsAn informal literature review was conducted to provide a snapshot into the variety of simulation training tools available for technical and nontechnical urological surgical skills within all subgroups of urological surgery using the following keywords: “urology, surgery, training, curriculum, validation, non-technical skills, technical skills, LESS, robotic, laparoscopy, animal models.” Validated training tools explored in research were tabulated and summarized.Results and ConclusionsA total of 20 studies exploring validated training tools were identified. Huge variation was noticed in the types of validity sought by researchers and suboptimal incorporation of these tools into curricula was noted across the subgroups of urological surgery. The following key recommendations emerge from the review: adoption of simulation-based curricula in training; better integration of dedicated training time in simulated environments within a trainee's working hours; better incentivization for educators and assessors to improvise, research, and deliver teaching using the technologies available; and continued emphasis on developing nontechnical skills in tandem with technical operative skills.     

Technical skills examination for general surgical trainees

BACKGROUND: The technical skills of surgical trainees are difficult to assess and compare objectively. This study involved a structured, multistation, technical skills examination that enables the stratification of surgical trainees. METHODS: Twenty-two surgeons (five basic surgical trainees, eight junior specialist trainees, four senior specialist trainees and five consultants) participated in the study. All undertook a five-station technical skills examination consisting of three synthetic simulations (bowel anastomosis, vascular anastomosis, saphenofemoral dissection) and two virtual reality-based (flexible sigmoidoscopy and laparoscopy) assessment stations. Video-based analyses and in-built computer scoring were used to measure each surgeon's performance. The mean rank was determined for each variable, and the sum of the mean ranks produced a total score. RESULTS: There was a significant improvement in overall performance with increasing seniority (P<0.001). Significant differences were observed between basic surgical trainees and junior specialist trainees (P=0.019), and between junior and senior specialist trainees (P=0.048), but not between senior trainees and consultants. CONCLUSION: This examination successfully differentiated surgical skill, both between surgeons with different grades of experience and within the target study group of specialist trainees. The examination is feasible in terms of the timeframe needed to complete tasks, cost, and efficiency in performing video-based assessments.     

A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre

BACKGROUND: High-risk organizations such as aviation rely on simulations for the training and assessment of technical and team performance. The aim of this study was to develop a simulated environment for surgical trainees using similar principles. METHODS: A total of 27 surgical trainees carried out a simulated procedure in a Simulated Operating Theatre with a standardized OR team. Observation of OR events was carried out by an unobtrusive data collection system: clinical data recorder. Assessment of performance consisted of blinded rating of technical skills, a checklist of technical events, an assessment of communication, and a global rating of team skills by a human factors expert and trained surgical research fellows. The participants underwent a debriefing session, and the face validity of the simulated environment was evaluated. RESULTS: While technical skills rating discriminated between surgeons according to experience (P = 0.002), there were no differences in terms of the checklist and team skills (P = 0.70). While all trainees were observed to gown/glove and handle sharps correctly, low scores were observed for some key features of communication with other team members. Low scores were obtained by the entire cohort for vigilance. Interobserver reliability was 0.90 and 0.89 for technical and team skills ratings. CONCLUSIONS: The simulated operating theatre could serve as an environment for the development of surgical competence among surgical trainees. Objective, structured, and multimodal assessment of performance during simulated procedures could serve as a basis for focused feedback during training of technical and team skills.     

Development of a valid, cost-effective laparoscopic training program

BACKGROUND: Practical programs for training and evaluating surgeons in laparoscopy are needed to keep pace with demand for minimally invasive surgery. METHODS: At the University of Kentucky five inexpensive simulations have been developed to train and assess surgical residents. Residents are videotaped performing laparoscopic procedures on models. Five surgeons assess the taped performances on 4 global skills. RESULTS: Creating mechanical models reduces training costs. Trainees agreed procedures were well represented by the simulations. Blinded assessment of performances showed high interrater agreement and correlated with the trainees' level of experience. Nonclinician evaluations on checklists correlated with evaluations by surgeons. CONCLUSIONS: Inexpensive simulations of laparoscopic appendectomy, cholecystectomy, inguinal herniorrhaphy, bowel enterotomy, and splenectomy enable surgical residents to practice laparoscopic skills safely. Obtaining masked, objective, and independent evaluations of basic skills in laparoscopic surgery can assist in reliable assessment of surgical trainees. The simulations described can anchor an innovative educational program during residency for training and assessment.     

Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater

BACKGROUND: The ability of surgeons to assess their own performance is essential for training and self-regulation. The latter is based on the premise that they recognize their weaknesses and seek remedial action accordingly. METHODS: Twenty-seven surgical trainees performed a simulated saphenofemoral high-tie on a synthetic model in a simulated operating theater. The performance assessment consisted of blinded rating of technical skills and a global rating of team skills by a human factors expert and a trained surgical research fellow. Subjects also were asked to assess their own performance using the same methods. Spearman's rho was used for data analysis. RESULTS: There was a strong correlation between the experts rating of technical skills and self-assessment (rho = .64). However, the correlation improved with increasing experience. It was .24 for junior trainees, .43 for those with intermediate experience, and .52 for senior trainees. There was a low correlation between the self-assessment and the expert scores for human factors skills (rho = .31). The correlation was higher for the 2 junior groups compared with the senior trainees. CONCLUSIONS: Unlike other studies on self-assessment, this study found that senior surgical trainees are accurate in their self-assessment of technical skills. However, this was not true in the case of human factors skills.     

Consultant surgeons' opinion of the skills required of basic surgical trainees.

BACKGROUND: Accurate and appropriate assessment of surgical trainees requires clear determination of the skills needed for surgical competence. This study was designed to identify those skills, rank them in order of importance and translate them into behavioural terms. METHODS: A Delphi technique, using anonymous postal questionnaires, was used. All consultant surgeons in South-East Scotland were asked to identify the skills they expected of surgical trainees. Skills identified were then returned to all consultants for weighting. Differences among specialties in the importance of each item were identified using analysis of variance. RESULTS: The qualities identified fell into five domains: technical skills, clinical skills, interaction with patients and relatives, teamwork, and application of knowledge. Consultants from all specialties gave high weightings to the generic domains of clinical skills, teamwork, and interaction with patients and relatives. CONCLUSION: This study has identified the skills considered necessary by consultant surgeons in Scotland for a successful surgical career. Contrary to expectation, consultant surgeons value many generic skills more highly than technical skills, indicating that they value well rounded doctors, not just those with technical ability. The characteristics identified are being used to develop an assessment tool for use on basic surgical trainees.     

Visuospatial and technical ability in the selection and assessment of higher surgical trainees in the London deanery

INTRODUCTION

Despite awareness of the limitations of current selection and competency assessments, there is little consensus and alternatives have not been readily accepted. Essential surgical skills include visuospatial and technical ability. The aim of this study was to survey current methods of higher surgical trainee selection and assessment. We suggest ways to improve the process.

MATERIALS AND METHODS

Nine surgical training programmes in the London deanery were surveyed through questionnaires to programme directors, existing trainees and examination of deanery publications.

RESULTS

Testing of visuospatial and technical ability was piloted at selection only in a single general surgical department. Practical skills were assessed in 3/9 (33%) specialties (ENT, plastic and general surgery). Once selected, no specialty tested visuospatial and technical ability. Practical skills were tested in only 1/9 (11%) specialties (plastic surgery). The remaining 8/9 (89%) were ‘assessed’ by interview.

CONCLUSIONS

Lack of visuospatial and technical ability assessment was identified at selection and during higher surgical training. Airlines have long recognised early identification of these qualities as critical for efficient training. There is a need for more objective methods in this area prior to selection as time to assess surgical trainees during long apprenticeships is no longer available. We advocate a suitably validated competency-based model during and at completion of training.     

Why we need open simulation to train surgeons in an era of work-hour restrictions

Surgical training in the USA and Europe has undergone radical changes with respect to working patterns, culture and limitation on working hours in recent years. Many surgeons who trained prior to the Accreditation Council for Graduate Medical Education (ACGME) duty-hour restrictions have expressed concern that surgeons currently exiting from training may not have had the same operative experience as in the pre-ACGME era. These concerns are particularly relevant in vascular surgery with the prevalence of endovascular therapies reducing the exposure of trainees to more traditional open vascular operations. Simulation has been used in many non-medical fields for technical skill acquisition prior to real-life performance and in recent years has been identified as a useful tool in surgical training. This article highlights the growing need for open vascular simulation as exposure to complex open vascular operations diminishes. The culture of, 'see one, do one, teach one' is fast becoming replaced by 'do many on a simulator, attain competency then perform under supervision in the operating room'. This will only be successfully achieved by the widespread incorporation of open vascular simulation into current vascular training programs if work hours remain limited and endovascular modalities continue to replace traditional open operations.     

A systematic review of simulation in open abdominal aortic aneurysm repair

ObjectiveConcerns about the long-term durability of endovascular aortic aneurysm repair and the requirement for explantation of stents in the case of infection demonstrate the continued need for open abdominal aortic aneurysm (AAA) repair. However, with the increased complexity and decreasing volume of open cases performed, maintenance of skills and training of younger surgeons are challenging. The aim of this review was to identify and to examine studies pertaining to open AAA simulation, with focus on methods and outcomes.MethodsWe performed a systematic review of the literature to identify primary research pertaining to open AAA repair through the use of simulators. The primary outcome was to identify predominant modes of simulator design and validated assessment tools that could demonstrate improvement in trainee skills. Secondary outcomes included identifying participant numbers needed to power studies and whether tools not validated externally contributed to the studies.ResultsThere were 309 unique papers identified, from which five papers met the inclusion criteria. The selected papers used a combination of synthetic (commercial and homemade) and cadaveric simulators. A variety of validated and nonvalidated assessment metrics were used, including Objective Structured Assessment of Technical Skills, global rating scales, and realism surveys. Three of the five papers used blinding as part of their assessments. Mean participant numbers were 30.8 ± 25.7 and with the exception of one paper consisted entirely of surgical trainees in dedicated general or vascular surgery training programs.ConclusionsSeveral options are currently available for open AAA simulation, all of which demonstrate improved scoring metrics after simulator use. Validated scoring systems, the Objective Structured Assessment of Technical Skills in particular, were most frequently used to deliver objective results. Whereas junior trainees derive the most benefit, senior trainees also showed significant improvements, demonstrating that simulation benefits all levels of surgical trainees. Low numbers of participants were sufficient to achieve statistical benefit within individual studies.     

Improving continuing medical education for surgical techniques: applying the lessons learned in the first decade of minimal access surgery

OBJECTIVE: To examine the first decade of experience with minimal access surgery, with particular attention to issues of training surgeons already in practice, and to provide a set of recommendations to improve technical training for surgeons in practice. SUMMARY BACKGROUND DATA: Concerns about the adequacy of training in new techniques for practicing surgeons began almost immediately after the introduction of laparoscopic cholecystectomy. The concern was restated throughout the following decade with seemingly little progress in addressing it. METHODS: A preliminary search of the medical literature revealed no systematic review of continuing medical education for technical skills. The search was broadened to include educational, medical, and psychological databases in four general areas: surgical training curricula, continuing medical education, learning curve, and general motor skills theory. RESULTS: The introduction and the evolution of minimal access surgery have helped to focus attention on technical skills training. The experience in the first decade has provided evidence that surgical skills training shares many characteristics with general motor skills training, thus suggesting several ways of improving continuing medical education in technical skills. CONCLUSIONS: The educational effectiveness of the short-course type of continuing medical education currently offered for training in new surgical techniques should be established, or this type of training should be abandoned. At present, short courses offer a means of introducing technical innovation, and so recommendations for improving the educational effectiveness of the short-course format are offered. These recommendations are followed by suggestions for research.     

Training in microsurgical skills: Assessing microsurgery training

Practical skill training courses are an increasingly popular method of teaching surgical skills. Few data are available from instructional courses indicating how successful they are at imparting practical skills to those individuals who attend them. We aimed to identify the skill benefits gained by trainee surgeons attending a 5-day microsurgical skills course. A global scoring system was devised to objectively assess the level of skill employed by trainees to complete an arterial microvascular anastamosis. Vessel patency, anastamotic construct, and care of tissue in the surrounding operative field were taken into account. Postoperative tissue viability and physiological vessel function following anastamosis were also investigated. The majority of surgeons (60%) exhibited an increase in their level of microsurgical skill during the course. The remaining candidates remained static or deteriorated. Attendance at this microsurgical training workshop resulted in an improvement in microsurgical skills in most trainees. In-course assessment by training courses would allow identification of individuals requiring further training or skill refinement.